Coordinate-wise Armijo's condition
share
Let z=(x,y) be coordinates for the product space R^m_1×R^m_2. Let f:R^m_1×R^m_2→R be a C^1 function, and ∇ f=(∂ _xf,∂ _yf) its gradient. Fix 0<α <1. For a point (x,y) ∈R^m_1×R^m_2, a number δ >0 satisfies Armijo's condition at (x,y) if the following inequality holds: f(x-δ∂ _xf,y-δ∂ _yf)-f(x,y)≤ -αδ (||∂ _xf||^2+||∂ _yf||^2). When f(x,y)=f_1(x)+f_2(y) is a coordinate-wise sum map, we propose the following coordinate-wise Armijo's condition. Fix again 0<α <1. A pair of positive numbers δ _1,δ _2>0 satisfies the coordinate-wise variant of Armijo's condition at (x,y) if the following inequality holds: [f_1(x-δ _1∇ f_1(x))+f_2(y-δ _2∇ f_2(y))]-[f_1(x)+f_2(y)]≤ -α (δ _1||∇ f_1(x)||^2+δ _2||∇ f_2(y)||^2). We then extend results in our recent previous results, on Backtracking Gradient Descent and some variants, to this setting. We show by an example the advantage of using coordinate-wise Armijo's condition over the usual Armijo's condition.
READ FULL TEXT