Lifted Primal-Dual Method for Bilinearly Coupled Smooth Minimax Optimization
share
We study the bilinearly coupled minimax problem: min_xmax_y f(x) + y^⊤ A x - h(y), where f and h are both strongly convex smooth functions and admit first-order gradient oracles. Surprisingly, no known first-order algorithms have hitherto achieved the lower complexity bound of Ω((√(L_x/μ_x) + A/√(μ_x μ_y) + √(L_y/μ_y)) log(1/ε)) for solving this problem up to an ε primal-dual gap in the general parameter regime, where L_x, L_y,μ_x,μ_y are the corresponding smoothness and strongly convexity constants. We close this gap by devising the first optimal algorithm, the Lifted Primal-Dual (LPD) method. Our method lifts the objective into an extended form that allows both the smooth terms and the bilinear term to be handled optimally and seamlessly with the same primal-dual framework. Besides optimality, our method yields a desirably simple single-loop algorithm that uses only one gradient oracle call per iteration. Moreover, when f is just convex, the same algorithm applied to a smoothed objective achieves the nearly optimal iteration complexity. We also provide a direct single-loop algorithm, using the LPD method, that achieves the iteration complexity of O(√(L_x/ε) + A/√(μ_y ε) + √(L_y/ε)). Numerical experiments on quadratic minimax problems and policy evaluation problems further demonstrate the fast convergence of our algorithm in practice.
READ FULL TEXT