Coordinate-wise Armijo's condition: General case
share
Let z=(x,y) be coordinates for the product space R^m_1×R^m_2. Let f:R^m_1×R^m_2→R be a C^1 function, and ∇ f=(∂ _xf,∂ _yf) its gradient. Fix 0<α <1. For a point (x,y) ∈R^m_1×R^m_2, a number δ >0 satisfies Armijo's condition at (x,y) if the following inequality holds: f(x-δ∂ _xf,y-δ∂ _yf)-f(x,y)≤ -αδ (||∂ _xf||^2+||∂ _yf||^2). In one previous paper, we proposed the following coordinate-wise Armijo's condition. Fix again 0<α <1. A pair of positive numbers δ _1,δ _2>0 satisfies the coordinate-wise variant of Armijo's condition at (x,y) if the following inequality holds: [f(x-δ _1∂ _xf(x,y), y-δ _2∂ _y f(x,y))]-[f(x,y)]≤ -α (δ _1||∂ _xf(x,y)||^2+δ _2||∂ _yf(x,y)||^2). Previously we applied this condition for functions of the form f(x,y)=f(x)+g(y), and proved various convergent results for them. For a general function, it is crucial - for being able to do real computations - to have a systematic algorithm for obtaining δ _1 and δ _2 satisfying the coordinate-wise version of Armijo's condition, much like Backtracking for the usual Armijo's condition. In this paper we propose such an algorithm, and prove according convergent results. We then analyse and present experimental results for some functions such as f(x,y)=a|x|+y (given by Asl and Overton in connection to Wolfe's method), f(x,y)=x^3 sin (1/x) + y^3 sin(1/y) and Rosenbrock's function.
READ FULL TEXT