Lower Bounds for Stochastic First-Order Algorithms with Variance Reduction in Nonconvex--Concave Minimax Optimization
Organizations: Department of Mathematics, College of Sciences, Shanghai University, Shanghai 200444, China
Abstract
We establish complexity lower bounds for stochastic first-order algorithms in nonconvex--concave minimax optimization, allowing algorithms to use variance reduction. Our main contribution is a lower bound for a zero-respecting algorithm class that permits variance reduction, extending beyond the algorithmic restrictions imposed by some existing lower bounds. We consider objectives with an -Lipschitz continuous joint gradient, a compact convex dual domain of Euclidean radius at most , and a primal value function, defined by maximizing the objective over the dual variable, with initial suboptimality at most . The target accuracy is measured by the gradient norm of the Moreau envelope of the constrained primal value function with parameter . Under an unbiased stochastic first-order oracle with variance at most and mean-square smoothness, we prove the lower bound . This result quantifies the dependence on accuracy, dual-domain radius, and oracle noise even when variance reduction is allowed. We also establish complementary lower bounds for nonconvex--strongly-concave minimax optimization. With dual strong-concavity parameter and condition number , we obtain under the bounded-variance oracle model. Under the additional mean-square smoothness condition with constant , we obtain . Together, these results identify complexity barriers across the concave and strongly concave regimes, with the main nonconvex--concave bound remaining valid for algorithms that use variance reduction.
Figures & tables
| Problem class | MSS | Algorithm class | Result | Reference | Oracle complexity |
| NC–SC | No | General | Upper bound | SPDE [ 16 ] | |
| No | ZR | Lower bound | Li et al. [ 7 ] | ||
| No | ZR | Lower bound | Zhang et al. [ 18 ] | ||
| No | ZR | Lower bound | Our result | ||
| Yes | General | Upper bound | VR-SPDE [ 16 ] | ||
| Yes | ZR | Lower bound | Zhang et al. [ 18 ] |