Last-Iterate Convergence

Follow this topic Ask about this topic

Momentum

7 papers in the last four weeks, against 1 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 26

All topics

Last-Iterate Convergence Rate of Normalized Gradient Descent under Hölder Smoothness
Oct 5, 2026Yuki Takezawa, Eduard GorbunovLearning Rate Scheduling Last-Iterate Convergence
Toyota Motor Corporation · MBZUAI
Fast Last-Iterate Convergence in Zero-Sum Markov Games with Bandit Feedback
Oct 5, 2026Yuheng ZhangLast-Iterate Convergence Reinforcement Learning
University of Illinois Urbana-Champaign
Convergence Analysis of STORM Under Different Geometries
Oct 1, 2026Wei Jiang, Yibo Wang, Wenhao Yang +3Stochastic Optimization Last-Iterate Convergence
School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China · School of Artificial Intelligence, Nanjing University, Nanjing, China
Average-and Last-Iterate Lower Bounds for Optimistic Matrix Mirror-Prox in Quantum Zero-Sum Games
Sep 30, 2026Yiheng Su, Emmanouil-Vasileios Vlatakis-Gkaragkounis, Pucheng XiongLast-Iterate Convergence Oracle Complexity
Department of Computer Sciences, University of Wisconsin-Madison
Finite-Time Concentration and Convergence Rates for Projected Two-Time-Scale Stochastic Approximation with Markov Noise
Sep 28, 2026Rahul Singh, Vivek S. Borkar, Eric MoulinesLast-Iterate Convergence Stochastic Approximation
Laboratoire de Recherche de l’EPITA, Paris, France · Indian Institute of Technology Bombay · Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, United Arab Emirates
Minimax Last-Iterate Convergence in Matrix Games with Observed Actions
Sep 28, 2026Yuheng ZhangLast-Iterate Convergence Zero-Sum Games
University of Illinois Urbana-Champaign
Near-Optimal Single-Loop Predictor--Corrector Extragradient Method for Strongly Convex--Strongly Concave Minimax Optimization
Sep 17, 2026Minhao Zhang, Zi XuLast-Iterate Convergence Min-Max Optimization
Department of Mathematics, Shanghai University, Shanghai 200444, People’s Republic of China
Improving the Last-Iterate Guarantees of Anytime Algorithms for Stochastic Monotone Variational Inequalities
Sep 14, 2026Jun-Hyun Kim, Ahmet AlacaogluLast-Iterate Convergence Stochastic Approximation
University of British Columbia, Vancouver. · Department of Mathematics, University of British Columbia, Vancouver.
The Exact Time-Uniform Rate Frontier for Stochastic Gradient Descent on Smooth Convex Objectives
Sep 8, 2026Ruijie Li, Kang Chen, Tianyu WangLearning Rate Scheduling Last-Iterate Convergence
School of Data Science, Fudan University · Shanghai Center for Mathematical Sciences, Fudan University
A lower bound for stepsize-based acceleration of gradient descent
Aug 11, 2026Jianhao Ma, Yuxin ChenLearning Rate Scheduling Last-Iterate Convergence
Department of Industrial Engineering, Tsinghua University · Department of Statistics and Data Science, Wharton School, University of Pennsylvania
On Same-Sample and Independent-Sample Stochastic Extragradient for Monotone Variational Inequalities
Aug 6, 2026TaeHo Yoon, Nicolas LoizouStochastic Optimization Last-Iterate Convergence
Department of Applied Mathematics and Statistics & Mathematical Institute for Data Science, Johns Hopkins University.
Non-Expansive Two-Time-Scale Stochastic Approximation: A Fixed-Schedule One-Quarter Barrier and Bias-Corrected Acceleration
Jul 15, 2026Dhruv Sarkar, Vaneet AggarwalLast-Iterate Convergence Stochastic Approximation
Indian Institute of Technology Kharagpur · Mohamed bin Zayed University of Artificial Intelligence · Purdue University
New Bounds for the Last Iterate of the Stochastic subGradient Method
Jun 23, 2026Guglielmo Beretta, Tommaso Cesari, Roberto Colomboni +1Stochastic Optimization Last-Iterate Convergence
Department of Computer Science, Aarhus University, Aarhus, Denmark · School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa, Canada · School of Mathematics, University of Bristol, Bristol, United Kingdom
GARIP: A Running-Average Moving Reference for Last-Iterate Self-Play in Two-Player Zero-Sum Games
Jun 21, 2026Can SavcıNash Equilibrium Last-Iterate Convergence
Independent Researcher İzmir, Türkiye
Accelerated and Stable Convergence with Anchored Optimistic Method
Jun 19, 2026Motahareh Sohrabi, Jianxin You, Simon Lacoste-Julien +2Last-Iterate Convergence Gradient Descent
Universit´e de Montr´eal · Mila - Quebec AI Institute · CIFAR AI Chair +1
Last-Iterate Convergence of Optimistic Multiplicative Weight Update
Jun 10, 2026Francesco OrabonaLast-Iterate Convergence Optimization Convergence Analysis
King Abdullah University of Science and Technology (KAUST) Thuwal, 23955-6900, Kingdom of Saudi Arabia
Accelerating Min-Max Optimization via Power-Law Stepsizes
Jun 1, 2026Yue Wu, Weiqiang Zheng, Yang Cai +1Learning Rate Scheduling Last-Iterate Convergence
University of Southern California · Yale University
A Unifying View of Anchoring via Operator-Side Tikhonov Regularization
May 29, 2026Zihao ChenLast-Iterate Convergence Optimization Convergence Analysis
UC Berkeley
Achieving $ε^{-2}$ Sample Complexity for Single-Loop Actor-Critic under Minimal Assumptions
May 13, 2026Ishaq Hamza, Zaiwei ChenPolicy Gradient Last-Iterate Convergence
IISc · Edwardson School of Industrial Engineering, Purdue University · Purdue IE
When and Why is Optimistic Multiplicative Weights Slow? The Geometry of Energy Dissipation
May 13, 2026John Lazarsfeld, Anas Barakat, Georgios Piliouras +2Nash Equilibrium Last-Iterate Convergence
SUTD · Google DeepMind · NUS CQT +2
Augmented Lagrangian Method for Last-Iterate Convergence for Constrained MDPs
May 12, 2026Michael Lu, Max Qiushi Lin, Mo Chen +1Markov Decision Processes Last-Iterate Convergence
Simon Fraser University
Near-Optimal Last-Iterate Convergence for Zero-Sum Games with Bandit Feedback and Opponent Actions
May 10, 2026Soumita Hait, Ping Li, Haipeng Luo +1Last-Iterate Convergence Imperfect-Information Games
University of Southern California · Shanghai University of Finance and Economics · University of Iowa
The Harder Path: Last Iterate Convergence for Uncoupled Learning in Zero-Sum Games with Bandit Feedback
Apr 17, 2026Côme Fiegel, Pierre Ménard, Tadashi Kozuno +2Nash Equilibrium Last-Iterate Convergence
ENSAE Paris - CREST, Palaiseau, France · Inria - FairPlay · ENS Lyon, Lyon, France +3
Optimal last-iterate convergence in matrix games with bandit feedback using the log-barrier
Apr 16, 2026Come Fiegel, Pierre Menard, Tadashi Kozuno +2Last-Iterate Convergence Imperfect-Information Games
ENSAE Paris – CREST, France · ENS Lyon, France · Isara Labs +2
Towards Weaker Variance Assumptions for Stochastic Optimization
Apr 14, 2025Ahmet Alacaoglu, Yura Malitsky, Stephen J. WrightStochastic Optimization Last-Iterate Convergence
Department of Mathematics, University of British Columbia · Faculty of Mathematics, University of Vienna · Department of Computer Sciences, University of Wisconsin–Madison
Stability and Convergence of Optimistic Exponential Weights with Asymmetric Step Sizes in Bimatrix Games
Date pendingHédi Hadiji, Sarah SachsNash Equilibrium Last-Iterate Convergence
Laboratoire des Signaux et Systèmes, CentraleSupélec, Paris, France · School of Mathematics, University of Bristol, Bristol, United Kingdom

Oct 5, 2026, math.OCJ/K move · Enter open · S save

Last-Iterate Convergence Rate of Normalized Gradient Descent under Hölder Smoothness

Yuki Takezawa, Eduard Gorbunov

Toyota Motor Corporation · MBZUAI

Learning Rate Scheduling Last-Iterate Convergence Optimization Convergence Analysis Gradient Descent

Open paper AskPDF arXiv

Normalized gradient descent is a widely studied adaptive optimization method. Most existing analyses focus on the best iterate or a weighted average of the iterates, whereas practical implementations typically return the last iterate. In this paper, we study the last-iterate convergence of normalized gradient descent for convex,

(ν,M_ν)

-Hölder-smooth objectives. For a constant stepsize, we establish an upper bound of

\mathcal{O}\bigl((\log^2(T)/T)^{(1+ν)/2}\bigr)

, which contains a logarithmic overhead relative to the known

\mathcal{O}\bigl(T^{-(1+ν)/2}\bigr)

guarantees for the best and weighted-average iterates. For

ν= 0

, this overhead is known to be unavoidable. We complement this analysis with numerical results based on the performance estimation problem (PEP), investigating the finite-horizon worst-case behavior in the smooth setting and whether the logarithmic overhead reflects an intrinsic limitation of constant-step normalized gradient descent. We then show that a linearly decreasing stepsize yields a last-iterate guarantee of

\mathcal{O}\bigl(T^{-(1+ν)/2}\bigr)

, matching the order of the best-iterate/weighted-average guarantees without requiring knowledge of

ν

and

M_ν

Momentum

Related topics

Latest papers 26