Step Accuracy

Follow this topic Ask about this topic

Momentum

6 papers in the last four weeks, against 2 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 32

All topics

Last-Iterate Convergence Rate of Normalized Gradient Descent under Hölder Smoothness
Oct 5, 2026Yuki Takezawa, Eduard GorbunovStep Accuracy
Toyota Motor Corporation · MBZUAI
Tight Transition Time Bounds for Separable Logistic Regression at the Edge of Stability
Oct 1, 2026Haodong Wen, Kaiyue Wen, Jiaye TengLogistic Regression Optimal Sample Complexity
Stanford University · Shanghai University of Finance and Economics
Free Everywhere, Exact on Trees: PPO's Dropped Correction Buys Sample Efficiency Under Aggressive Reuse
Sep 30, 2026Nima H. SiboniReuse Step Accuracy
Juna.ai, Kastanienallee 32, 10435 Berlin, Germany
GUITAR: Structured Failure Diagnosis of GUI Agents via State Transitions
Sep 28, 2026Shaoqing Zhang, Kehai Chen, Xuefeng Bai +4Graphical User Interface Agents Graphical User Interface
Harbin Institute of Technology, Shenzhen, China · Pengcheng Laboratory, Shenzhen, China · Shanghai Jiao Tong University, China
Distributed Stochastic Approximation Algorithms and Heavy-Tailed Age of Information
Sep 23, 2026Adrian Redder, Arunselvan Ramaswamy, Holger KarlStochastic Approximation Long-Tailed Distribution
Dept. of Electrical Engineering and Information Technology, Paderborn University, Germany · Dept. of Industrial Engineering and Operations Research, Indian Institute of Technology Bombay, India · Hasso-Plattner Institute (HPI), Digital Engineering Faculty, University of Potsdam, Germany
AURA: Angular Update Rate Adaptation for training complex-valued neural networks
Sep 22, 2026Enrico Ballini, Allan Peter Engsig-Karup, Tito AndriolloNeural Network Optimization Step Accuracy
Aarhus University, Aarhus, Denmark · Technical University of Denmark, Kongens Lyngby, Denmark
The Bias of Nonlinear Two-Time-scale Stochastic Approximation under Constant Step-Sizes
Sep 17, 2026Djamel Rassem Lamouri, Dorian Baudry, Nicolas GastStochastic Approximation Step Accuracy
Univ. Grenoble Alpes, CNRS, Inria Grenoble INP, LIG, 38000 Grenoble France
Projected Riemannian Gradient Descent for the Bures-Wasserstein Barycenter: Dimension-Independent Linear Convergence at Unit Step Size
Sep 3, 2026A. AfhamRiemannian Optimization Wasserstein Distance
Centre for Quantum Technologies, National University of Singapore, Singapore
A lower bound for stepsize-based acceleration of gradient descent
Aug 11, 2026Jianhao Ma, Yuxin ChenStep Accuracy Gradient Descent
Department of Industrial Engineering, Tsinghua University · Department of Statistics and Data Science, Wharton School, University of Pennsylvania
Finite-Time Analysis of the Natural Policy Gradient in Finite-Horizon Markov Decision Processes
Jul 25, 2026Asha Barua, Sajad KhodadadianPolicy Gradient Markov Decision Processes
Grado Department of Industrial and Systems Engineering, Virginia Polytechnic Institute and State University, Blacksburg, Virginia 24061, USA.
Scaling Limits of Constant-Stepsize SGD at Flat Minima
Jul 17, 2026Jingyi Zhang, Cheng Mao, Debankur MukherjeeStochastic Gradient Descent Step Accuracy
Georgia Institute of Technology
Dynamics of Gradient Descent with Large Step Size Near a Manifold of Flat Minima
Jul 9, 2026Lachlan Ewen MacDonald, René VidalGradient Descent Hessian
Innovation in Data Engineering and Science (IDEAS) · University of Pennsylvania Pennsylvania, PA 19104
Variance Reduction for Stochastic Gradient Generalized Non-reversible Langevin Monte Carlo Algorithms
Jun 27, 2026Bingye Ni, Xiaoyu Wang, Yingli Wang +1Langevin Dynamics Variance Reduction
FinTech Thrust, Hong Kong University of Science and Technology (Guangzhou), Guangzhou, Guangdong, People's Republic of China · School of Mathematical Science, Fudan University, Shanghai, People's Republic of China · Department of Mathematics, Florida State University, Tallahassee, Florida, United States of America
New Bounds for the Last Iterate of the Stochastic subGradient Method
Jun 23, 2026Guglielmo Beretta, Tommaso Cesari, Roberto Colomboni +1Step Accuracy Lipschitz Continuity
Department of Computer Science, Aarhus University, Aarhus, Denmark · School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa, Canada · School of Mathematics, University of Bristol, Bristol, United Kingdom
A Single Stepsize Suffices for Unprojected Linear TD(0): Simultaneous Robust and Fast Rates via Polyak--Ruppert Averaging
Jun 23, 2026Wei-Cheng Lee, Francesco OrabonaStep Accuracy Temporal Difference
King Abdullah University of Science and Technology (KAUST) Thuwal, 23955-6900, Kingdom of Saudi Arabia
Closed-Loop Graph Algorithm Execution with Small Language Models: Step Accuracy and Rollout Reliability
Jun 23, 2026Michal PodstawskiStep Accuracy
NASK National Research Institute, Warsaw, Poland
Muown Implicitly Performs Angular Step-size Decay
Jun 22, 2026Florian Hübler, Kai Lion, Antonio Orvieto +1Muon Adaptive Optimizers
Department of Computer Science · ETH Zurich, Switzerland · ELLIS Institute Tübingen, MPI-IS +1
Flatland: The Adventures of Gradient Descent with Large Step Sizes
Jun 4, 2026Leonardo Galli, Curtis Fox, Wiebke Bartolomaeus +2Gradient Descent Step Accuracy
LMU Munich · Munich Center for Machine Learning · University of British Columbia +1
Accelerating Min-Max Optimization via Power-Law Stepsizes
Jun 1, 2026Yue Wu, Weiqiang Zheng, Yang Cai +1Step Accuracy Convergence
University of Southern California · Yale University
Finer Parameter Steps for Low-Rank PEFT: A Controlled Study with CP Tensor Adapters
May 29, 2026Xinjue Wang, Xiuheng Wang, Yejun Zhang +3Low-Rank Adaptation Adapters Step Accuracy
Department of Information and Communications Engineering, Aalto University, 02150, Espoo, Finland · Université de Lorraine, CNRS, CRAN, France · Department of Computer Science, Aalto University, 02150, Espoo, Finland +1
Gradient Descent with Large Step Size Restores Symmetry in Deep Linear Networks with Multi-Pathway
May 29, 2026Hee-Sung Kim, Sungyoon LeeDeep Network Gradient Descent
Department of Computer Science, Hanyang University, Seoul, Korea.
Concentration of General Stochastic Approximation Under Heavy-Tailed Markovian Noise
May 20, 2026Shubhada Agrawal, Siva Theja Maguluri, Martin ZubeldiaStochastic Approximation Concentration Bounds
Indian Institute of Science · Georgia Institute of Technology · University of Minnesota
Gaussian Approximation and Multiplier Bootstrap for Federated Linear Stochastic Approximation
May 19, 2026Ilya Levin, Maksim Shuklin, Eric Moulines +2Stochastic Approximation Model-Based Bootstrap
1HSE University, Russia · 2MBZUAI · 3CMAP, CNRS, École Polytechnique, Institut Polytechnique de Paris, 91120 Palaiseau, France
A Closed-Form Upper Bound for Admissible Learning-Rate Steps in Belief-Space Dynamics
May 7, 2026Zixi Li, Youzhen LiStep Accuracy Optimal Learner
Datawhale
State-Dependent Lyapunov Analysis of Rank-1 Matrix Factorization
Apr 28, 2026Jaehong MoonLyapunov Function Step Accuracy
Industrial & Enterprise Systems Engineering University of Illinois at Urbana-Champaign Urbana, IL 61801
Intentional Updates for Streaming Reinforcement Learning
Apr 21, 2026Arsalan Sharifnassab, Mohamed Elsayed, Kris De Asis +2Gradient-Based Optimization Streaming
Openmind Research Institute · Department of Computing Science, University of Alberta.
Variable Search Stepsize for Randomized Local Search in Multi-Objective Combinatorial Optimization
Feb 5, 2026Xuepeng Ren, Maocai Wang, Guangming Dai +4Large Neighborhood Search Combinatorial Optimization
School of Computer, China University of Geosciences, Engineering Research Center of Natural Resource Information Management and Digital Twin Engineering Software, Ministry of Education, and Hubei Key Laboratory of Intelligent Geo-Information Processing, Wuhan 430074, China · School of Computer, Science and Informatics, De Montfort University, Leicester LE1 9BH, U.K. · School of Computer Science, University of Birmingham, Birmingham B15 2TT, U.K. +1
Negative Stepsizes Make Gradient-Descent-Ascent Converge
May 2, 2025Henry Shugart, Jason M. AltschulerStep Accuracy Gradient Descent
UPenn
Gradient Descent on Logistic Regression with Non-Separable Data and Large Step Sizes
Jun 7, 2024Si Yi Meng, Antonio Orvieto, Daniel Yiming Cao +1Step Accuracy Gradient Descent
Department of Computer Science Cornell University Ithaca, NY, USA · ELLIS Institute Tübingen MPI for Intelligent Systems Tübingen AI Center, Germany
Silver Rate Is (Almost) Optimal for Gradient Descent
Date pendingYuhan Ye, Kaizhao LiuStep Accuracy Gradient Descent
MIT
Fast Convergence for High-Order ODE Solvers in Diffusion Probabilistic Models
Date pendingDaniel Zhengyu Huang, Jiaoyang Huang, Zhengjiang LinTime Discretization Ordinary Differential Equations
Beijing International Center for Mathematical Research, Center for Machine Learning Research, Peking University, Beijing, China · Department of Statistics and Data Science, University of Pennsylvania, Philadelphia, PA, USA · Department of Mathematics, Massachusetts Institute of Technology, 77 Massachusetts Ave, 02139 Cambridge MA, USA
Stability and Convergence of Optimistic Exponential Weights with Asymmetric Step Sizes in Bimatrix Games
Date pendingHédi Hadiji, Sarah SachsImperfect-Information Games Equilibrium
Laboratoire des Signaux et Systèmes, CentraleSupélec, Paris, France · School of Mathematics, University of Bristol, Bristol, United Kingdom

Oct 5, 2026, math.OCJ/K move · Enter open · S save

Last-Iterate Convergence Rate of Normalized Gradient Descent under Hölder Smoothness

Yuki Takezawa, Eduard Gorbunov

Toyota Motor Corporation · MBZUAI

Step Accuracy

Open paper AskPDF arXiv

Normalized gradient descent is a widely studied adaptive optimization method. Most existing analyses focus on the best iterate or a weighted average of the iterates, whereas practical implementations typically return the last iterate. In this paper, we study the last-iterate convergence of normalized gradient descent for convex,

(ν,M_ν)

-Hölder-smooth objectives. For a constant stepsize, we establish an upper bound of

\mathcal{O}\bigl((\log^2(T)/T)^{(1+ν)/2}\bigr)

, which contains a logarithmic overhead relative to the known

\mathcal{O}\bigl(T^{-(1+ν)/2}\bigr)

guarantees for the best and weighted-average iterates. For

ν= 0

, this overhead is known to be unavoidable. We complement this analysis with numerical results based on the performance estimation problem (PEP), investigating the finite-horizon worst-case behavior in the smooth setting and whether the logarithmic overhead reflects an intrinsic limitation of constant-step normalized gradient descent. We then show that a linearly decreasing stepsize yields a last-iterate guarantee of

\mathcal{O}\bigl(T^{-(1+ν)/2}\bigr)

, matching the order of the best-iterate/weighted-average guarantees without requiring knowledge of

ν

and

M_ν

Momentum

Related topics

Latest papers 32