A Unified Analysis Method for Online Optimization in Normed Vector Space
share
We present a unified analysis method that relies on the generalized cosine rule and ϕ-convex for online optimization in normed vector space using dynamic regret as the performance metric. In combing the update rules, we start with strategy S (a two-parameter variant strategy covering Optimistic-FTRL with surrogate linearized losses), and obtain S-I (type-I relaxation variant form of S) and S-II (type-II relaxation variant form of S, which is Optimistic-MD) by relaxation. Regret bounds for S-I and S-II are the tightest possible. As instantiations, regret bounds of normalized exponentiated subgradient and greedy/lazy projection are better than the currently known optimal results. We extend online convex optimization to online monotone optimization, by replacing losses of online game with monotone operators and extending the definition of regret, namely regret^n, and expand the application scope of S-I and S-II.
READ FULL TEXT