stat.MLNov 10, 2026
SaveImproved last-iterate bounds for SGD with and without replacement
Organizations: Department of Mathematics and Statistics, Auburn University, Auburn, AL 36830
Abstract
In this work, we study last-iterate convergence bounds for stochastic gradient descent (SGD) on convex and -smooth loss functions and improve the bound from to in the interpolation regime. The improvement comes from applying a horizon-dependent multiscale estimate of gradients to the comparison of trajectories. The bound applies to both sampling with replacement and sampling without replacement within one pass.
More in this topic
Last-Iterate Convergence
Last-iterate convergence studies whether an algorithm’s final update approaches an optimizer or equilibrium, rather than only its average.
See all 47 papers