math.OCOct 1, 2026

Lower Bounds for Stochastic First-Order Algorithms with Variance Reduction in Nonconvex--Concave Minimax Optimization

Authors: Jiayi Song, Zi Xu

Organizations: Department of Mathematics, College of Sciences, Shanghai University, Shanghai 200444, China

Abstract

We establish complexity lower bounds for stochastic first-order algorithms in nonconvex--concave minimax optimization, allowing algorithms to use variance reduction. Our main contribution is a lower bound for a zero-respecting algorithm class that permits variance reduction, extending beyond the algorithmic restrictions imposed by some existing lower bounds. We consider objectives with an LL-Lipschitz continuous joint gradient, a compact convex dual domain of Euclidean radius at most DYD_Y, and a primal value function, defined by maximizing the objective over the dual variable, with initial suboptimality at most ΔΔ. The target accuracy ε\varepsilon is measured by the gradient norm of the Moreau envelope of the constrained primal value function with parameter 1/(2L)1/(2L). Under an unbiased stochastic first-order oracle with variance at most σ2σ^2 and mean-square smoothness, we prove the lower bound Ω ⁣(L2DYΔε−3+L3DY2Δσ2ε−6)Ω\!\left(L^2D_YΔ\varepsilon^{-3}+L^3D_Y^2Δσ^2\varepsilon^{-6}\right). This result quantifies the dependence on accuracy, dual-domain radius, and oracle noise even when variance reduction is allowed. We also establish complementary lower bounds for nonconvex--strongly-concave minimax optimization. With dual strong-concavity parameter μ>0μ>0 and condition number κ:=L/μκ:=L/μ, we obtain Ω ⁣(LΔκ ε−2+LΔκσ2ε−4)Ω\!\left(LΔ\sqrtκ\,\varepsilon^{-2}+LΔκσ^2\varepsilon^{-4}\right) under the bounded-variance oracle model. Under the additional mean-square smoothness condition with constant Lˉ\bar L, we obtain Ω ⁣(LΔκ ε−2+ΔLˉσκ3/2ε−3)Ω\!\left(LΔ\sqrtκ\,\varepsilon^{-2}+Δ\bar Lσκ^{3/2}\varepsilon^{-3}\right). Together, these results identify complexity barriers across the concave and strongly concave regimes, with the main nonconvex--concave bound remaining valid for algorithms that use variance reduction.

Figures & tables

Explore similar work

CardsList
  1. Accelerated Stochastic Min-Max Optimization Based on Bias-corrected Momentum

    Jun 18, 2024Haoyuan Cai, Sulaiman A. Alghunaim, Ali H. SayedStochastic Convex OptimizationConvex Optimization