math.OCSep 2, 2026
SaveImproved Gradient Descent Lower Bounds Beyond Nesterov
Organizations: MIT
Abstract
We study how far gradient descent (GD) can be accelerated by predetermined stepsizes in smooth convex optimization. Going beyond the classical first-order oracle lower bound of Nemirovsky and Yudin (1983), we prove an non-anytime lower bound and an anytime lower bound. These improve the recent non-anytime lower bound of Ma and Chen (2026) and the anytime lower bound of Tsai et al. (2026), respectively. Both results continue to hold when the stepsizes may be negative. Our anytime lower bound also shows that the rate of non-anytime silver schedules (Altschuler and Parrilo, 2025; Grimmer et al., 2025) is unattainable in the anytime setting. This establishes a strict separation between the two settings.