math.OCSep 16, 2026

Matching Multi-Loop Complexities with a Single Loop: Optimal Optimization Stationarity and Best-Known Game Stationarity in Nonconvex--Concave Minimax Optimization

Authors: Minghao ZhangZi Xu

Abstract

We introduce a new single-loop algorithmic framework for smooth nonconvex--concave minimax optimization. The resulting projected damped extragradient method combines projected extragradient updates, dual momentum, and a moving proximal center. Under both the optimization-stationarity and game-stationarity criteria, our method achieves the best-known complexity among single-loop first-order methods. For optimization stationarity, our method achieves a gradient complexity of O(L2DYΔˉ0ε3)O(L^2D_Y\barΔ_0\varepsilon^{-3}), where LL is the gradient Lipschitz constant, DYD_Y bounds the diameter of the dual feasible set, and Δˉ0\barΔ_0 is an initialization quantity involving the value-function gap and the initial gradients. Moreover, by incorporating a fixed-center warm-up phase, the complexity can be improved to O(L2DYΔφε3)O(L^2D_YΔ_φ\varepsilon^{-3}), up to an additive lower-order cost, where Δφ:=φ(x0)infxφ(x)Δ_φ:=φ(x_0)-\inf_xφ(x). We further establish a lower bound of Ω(L2DYΔφε3)Ω(L^2D_YΔ_φ\varepsilon^{-3}) for optimization stationarity over projected zero-respecting first-order methods. This lower bound proves that the warm-started version of our algorithm is optimal up to a constant factor for optimization stationarity within this oracle class. For game stationarity, our method achieves O ⁣(L3/2DY1/2Δφε5/2)\mathcal{O}\!(L^{3/2}D_Y^{1/2}Δ_φ\varepsilon^{-5/2}) gradient complexity. This matches the best-known complexity of multi-loop first-order methods, thereby establishing the same complexity with a single-loop algorithmic structure. Under dual strong concavity, the proposed framework achieves O ⁣(κLΔφε2)O\!(\sqrtκ\,LΔ_φ\varepsilon^{-2}) leading complexity for both stationarity criteria, where κ=L/μκ=L/μ is the dual condition number, up to an additive initialization cost. The ε2\varepsilon^{-2} accuracy dependence is optimal under fixed regularity and initialization bounds.

Explore similar work

Sep 17, 2026math.OC

Near-Optimal Pure Single-Loop Extragradient Method for Strongly Convex--Strongly Concave Minimax Optimization

We study smooth strongly convex--strongly concave minimax optimization with general nonlinear coupling in the deterministic unconstrained setting. We propose a pure single-loop damped extragradient method with fixed parameters and two new full-gradient evaluations per iteration after one initialization query. The method uses an auxiliary feedback recursion and requires no inner solves, accuracy schedules, or staged restarts. We establish last-iterate linear convergence and show that reducing the squared Euclidean distance to the saddle point to an ε\varepsilon fraction of its initial value requires O(κxκylog(2κxκy/ε))O(\sqrt{κ_xκ_y}\log(2κ_xκ_y/\varepsilon)) full-gradient queries, where κx=L/μxκ_x=L/μ_x and κy=L/μyκ_y=L/μ_y. This bound attains the optimal condition-number order up to logarithmic factors through fixed explicit updates. Numerical experiments demonstrate the effectiveness of the method.
Minhao Zhang, Zi Xu
Jun 18, 2024cs.LG

Accelerated Stochastic Min-Max Optimization Based on Bias-corrected Momentum

Lower-bound analyses for nonconvex strongly-concave minimax optimization problems have shown that stochastic first-order algorithms require at least O(ε4)\mathcal{O}(\varepsilon^{-4}) sample complexity to find an ε\varepsilon-stationary point. Some works indicate that this complexity can be improved to O(ε3)\mathcal{O}(\varepsilon^{-3}) when the stochastic loss gradient is Lipschitz continuous. The question of achieving enhanced convergence rates under distinct conditions, remains open. In this work, we address this question for optimization problems that are nonconvex in the minimization variable and strongly concave or Polyak-Lojasiewicz (PL) in the maximization variable. We introduce novel bias-corrected momentum algorithms utilizing efficient Hessian-vector products. We establish convergence conditions and demonstrate a lower iteration complexity of O(ε3)\mathcal{O}(\varepsilon^{-3}) for the proposed algorithms. The effectiveness of the proposed method is validated through applications to robust logistic regression and robust adaptive cruise control.
Haoyuan Cai, Sulaiman A. Alghunaim, Ali H. Sayed
Sep 8, 2026math.OC

How to Make the Gradient Mapping Small for Constrained Stochastic Min-Max Problems and Beyond

We study the stochastic first-order oracle complexity for constrained or regularized convex-concave min-max optimization and stochastic monotone variational inequalities. We focus on the case when suboptimality is measured in terms of the gradient mapping, also known as, forward-backward or natural residual, an optimality notion that generalizes the gradient norm for unconstrained problems. In this setting, under standard unbiased oracle access with now-standard variance assumptions, the best-known complexity for making the norm of the gradient mapping less than ε\varepsilon is O~(ε4)\widetilde{O}(\varepsilon^{-4}), compared to the near-optimal O~(ε2)\widetilde{O}(\varepsilon^{-2}) that is established in the unconstrained case. We bridge this gap to improve the gradient mapping complexity for constrained convex-concave min-max problems to O~(ε2)\widetilde{O}(\varepsilon^{-2}). We then extend to prove the same complexity for problems without the bounded variance, by using the Blum-Gladyshev assumption.
Ahmet Alacaoglu