Richardson Extrapolation

Latest papers

8 papers1 paper in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Stochastic Gradient Descent: A Nonlinear Time Series Perspective
Oct 14, 2026Jiaqi Li, Zhipeng Lou, Stefan Richter +1Richardson Extrapolation Stochastic Gradient Descent
Department of Statistics, Rice University, USA · Department of Mathematics, University of California, San Diego, USA · Institute of Applied Mathematics, Heidelberg University, Germany +1
Stationary Bias and Extrapolation in Nonlinear Two-Timescale Stochastic Approximation
Oct 7, 2026A. Ch. Madhusudanarao, Rahul SinghRichardson Extrapolation Stochastic Approximation
Department of Computer Science and Automation Indian Institute of Science, Bengaluru, India · Laboratoire de Recherche de l’EPITA, Paris, France
Self-Normalized Inference for Constant-Stepsize Temporal-Difference Learning under Markovian Sampling
Aug 11, 2026Min Zeng, Yichen Zhang, Xiaofeng ShaoConfidence Region Estimation Stochastic Approximation
Department of Biostatistics, City University of Hong Kong, Hong Kong SAR, China · Department of Quantitative Methods, Purdue University, West Lafayette, Indiana, USA · Department of Statistics and Data Science, Washington University in St. Louis, St. Louis, Missouri, USA +1
Extrapolating from Regularised Solutions for Solving Ill-Conditioned Linear Systems in Machine Learning
Jun 29, 2026Disha Hegde, Jon Cockayne, Chris. J. OatesInverse Problems Richardson Extrapolation
University of Southampton, UK · Newcastle University, UK · The Alan Turing Institute, UK
Increasing Missingness to Reduce Bias: Richardson-SGD with Missing Data
May 19, 2026Ferdinand Genans, Erwan ScornetLearning with Missing Data Richardson Extrapolation
SU, LPSM · Sorbonne Université and Université Paris Cité, CNRS, Laboratoire de Probabilités, Statistique et Modélisation, LPSM, F-75005 Paris, France
Information Processing Capacity of Stationary Physical Systems: Theory, Data-efficient Estimation Methods, and Photonic Demonstration
May 18, 2026Rahul Uma Ramachandran, Serge MassarPhotonic Computing Richardson Extrapolation
Laboratoire d’Information Quantique CP224, Universit´e libre de Bruxelles, Av. F. D. Roosevelt 50, B-1050 Bruxelles, Belgium
Shuffling the Data, Stretching the Step-size: Sharper Bias in constant step-size SGD
Apr 11, 2026Konstantinos Emmanouilidis, Emmanouil-Vasileios Vlatakis-Gkaragkounis, Rene VidalRandom Reshuffling Richardson Extrapolation
Department of Computer Science, University of Pennsylvania · University of Wisconsin-Madison, Archimedes Research Unit
High-Order Error Bounds for Markovian LSA with Richardson-Romberg Extrapolation
Aug 7, 2025Ilya Levin, Alexey Naumov, Sergey SamsonovPolyak-Ruppert Averaging Richardson Extrapolation
HSE University

Oct 14, 2026J/K move · Enter open · S save

Stochastic Gradient Descent: A Nonlinear Time Series Perspective

Jiaqi Li, Zhipeng Lou, Stefan Richter, Wei Biao Wu

Department of Statistics, Rice University, USA · Department of Mathematics, University of California, San Diego, USA · Institute of Applied Mathematics, Heidelberg University, Germany+1

Richardson Extrapolation Stochastic Gradient Descent Stochastic Optimization Convergence

Open paper AskPDF arXiv

This paper investigates the statistical behavior of Stochastic Gradient Descent (SGD) with constant step size under the framework of iterated random functions. Compared to previous studies establishing the convergence of SGD in probability measure, e.g., Wasserstein distance, our approach directly provides the convergence in Euclidean distance by showing the Geometric Moment Contraction (GMC) of SGD. This new convergence can address the non-stationarity of SGD due to fixed initial points and can provide a more refined asymptotic analysis of SGD. Specifically, we prove quenched central limit theorem, invariance principle and

α

-stable convergence for averaged SGD (ASGD) regardless of initialization. Furthermore, we provide a novel perspective to understand the impact of step sizes in SGD by studying its derivative with respect to the step size. The existence of stationary solutions for the derivative processes is shown, enabling the use of interpolation to recover the full-trajectory of SGD. Subsequently, we utilize multiple step sizes and show an enhanced Richardson-Romberg extrapolation with improved bias representation, which brings ASGD estimates closer to the global optimum. Finally, we propose a new online inference method and a bias-reduced variant for the extrapolated ASGD. Empirical confidence intervals are constructed and the coverage probabilities approach the nominal level as

n

grows.