Riemannian stochastic recursive momentum method for non-convex optimization

08/11/2020

∙

by   Andi Han, et al.

∙

33

∙

share

We propose a stochastic recursive momentum method for Riemannian non-convex optimization that achieves a near-optimal complexity of 𝒪̃(ϵ^-3) to find ϵ-approximate solution with one sample. That is, our method requires 𝒪(1) gradient evaluations per iteration and does not require restarting with a large batch gradient, which is commonly used to obtain the faster rate. Extensive experiment results demonstrate the superiority of our proposed algorithm.