Last-Iterate Convergence

Last-iterate convergence asks whether the actual sequence of updates ends near a solution, which matters when averaged outputs are impractical or do not reflect the algorithm’s ongoing behavior. The field studies stochastic optimization, variational inequalities, and learning in zero-sum games, using methods such as Halpern anchoring, extragradient and optimistic updates, and adaptive step sizes. Researchers analyze both rates and the assumptions needed for guarantees, including noise, smoothness, feedback, and whether the time horizon is known. A central tension is that final-iterate guarantees can be slower or more fragile than average-iterate ones, and lower bounds show that some costs cannot be avoided in particular settings.