Stochastic Approximation

Ask about this topic

Overview

Stochastic approximation studies how repeated updates based on noisy samples can find solutions to optimization, root-finding, and fixed-point problems, including those in reinforcement learning. Common approaches include Polyak–Ruppert averaging, two-time-scale updates, variance reduction, and Lyapunov or Poisson-equation analyses to control noise and dependence. Current work develops finite-time convergence and uncertainty guarantees for settings such as Markovian data, heavy-tailed noise, and asynchronous updates. A central challenge is to obtain sharp, reliable rates without restrictive assumptions on smoothness, stability, or noise.

Latest papers

CardsList

76 papers13 papers in the last four weeks, level with the four weeks before. 0.1% of all new papers.

  1. Stochastic approximation in non-markovian environments revisited

    Mar 22, 2026Vivek Shripad BorkarStochastic ApproximationContinual Learning

    Department of Electrical Engineering, Indian Institute of Technology Bombay, Powai, Mumbai 400076, INDIA.

  2. Heavy-Tailed and Long-Range Dependent Noise in Stochastic Approximation: A Finite-Time Analysis

    Mar 20, 2026Siddharth Chandak, Anuj Yadav, Ayfer Ozgur +1Stochastic ApproximationStochastic Optimization Convergence

    Department of Electrical Engineering, Stanford University, CA, USA. · School of Computer & Communication Sciences, EPFL, Lausanne, Switzerland.

  3. Convergence of Two Time-Scale Stochastic Approximation: A Martingale Approach

    Mar 15, 2026Mathukumalli VidyasagarStochastic ApproximationTwo-Timescale Stochastic Approximation

    Indian Institute of Technology Hyderabad, Kandi, Telangana 502285

  4. Convergence Rate of a Functional Learning Method for Contextual Stochastic Optimization

    Mar 13, 2026Noel Smith, Andrzej RuszczynskiStochastic OptimizationStochastic Approximation

    Department of Management Science and Information Systems, Rutgers University

  5. Generalized Random Direction Newton Algorithms for Stochastic Optimization

    Feb 23, 2026Soumen Pachal, Prashanth L. A., Shalabh Bhatnagar +1Stochastic OptimizationStochastic Optimization Convergence

    Indian Institute of Technology Madras · Indian Institute of Science, Bangalore · TCS Research, Chennai, India

  6. Regret and Sample Complexity of Online Q-Learning via Concentration of Stochastic Approximation with Time-Inhomogeneous Markov Chains

    Feb 18, 2026Rahul Singh, Siddharth Chandak, Eric Moulines +2Regret Minimization in RLQ-Learning

    MBZUAI, UAE · Stanford University, USA · EPITA, France +1

  7. Functional Central Limit Theorem for Stochastic Gradient Descent

    Feb 17, 2026Kessang Flamand, Victor-Emmanuel BrunelStochastic Gradient DescentStochastic Approximation

    CREST-ENSAE

  8. Constant-Stepsize Stochastic Approximation: Finite-Time Convergence, Gaussian Approximation, and Tail Bounds

    Feb 15, 2026Zedong Wang, Yuyang Wang, Ijay Narang +3Stochastic ApproximationStochastic Gradient Descent

    H. Milton Stewart School of Industrial & Systems Engineering, Georgia Institute of Technology, Atlanta, GA 30332, USA · Department of Mathematics, Brown University, Providence, RI 02912, USA · School of Computer Science, Georgia Institute of Technology, Atlanta, GA 30332, USA +2

  9. Finite-Sample Wasserstein Error Bounds and Concentration Inequalities for Nonlinear Stochastic Approximation

    Feb 2, 2026Seo Taek Kong, R. SrikantStochastic ApproximationStochastic Gradient Descent

    University of Illinois Urbana-Champaign

  10. Rethinking Langevin Thompson Sampling from A Stochastic Approximation Perspective

    Oct 6, 2025Weixin Wang, Haoyang Zheng, Guang Lin +2Thompson SamplingMulti-Armed Bandits

    Duke University · Purdue University · Morgan Stanley

  11. Uniform-in-time convergence bounds for Persistent Contrastive Divergence algorithms

    Oct 2, 2025Paul Felix Valsecchi Oliva, O. Deniz Akyildiz, Andrew DuncanParameter EstimationStochastic Approximation

    Department of Mathematics, Imperial College London

  12. A Kernel-based Stochastic Approximation Framework for Nonlinear Operator Learning

    Sep 14, 2025Jia-Qi Yang, Lei ShiStochastic ApproximationReproducing Kernel Hilbert Spaces

    School of Mathematical Sciences and Shanghai Key Laboratory for Contemporary Applied Mathematics, Fudan University, Shanghai 200433, China.

  13. High-Order Error Bounds for Markovian LSA with Richardson-Romberg Extrapolation

    Aug 7, 2025Ilya Levin, Alexey Naumov, Sergey SamsonovPolyak-Ruppert AveragingRichardson Extrapolation

    HSE University

  14. Towards Weaker Variance Assumptions for Stochastic Optimization

    Apr 14, 2025Ahmet Alacaoglu, Yura Malitsky, Stephen J. WrightStochastic OptimizationLast-Iterate Convergence

    Department of Mathematics, University of British Columbia · Faculty of Mathematics, University of Vienna · Department of Computer Sciences, University of Wisconsin–Madison

  15. Nonasymptotic CLT and Error Bounds for Linear Two-Time-Scale Stochastic Approximation

    Feb 14, 2025Seo Taek Kong, Sihan Zeng, Thinh T. Doan +1Stochastic ApproximationTwo-Timescale Stochastic Approximation

    Coordinated Science Laboratory, Department of Electrical and Computer Engineering, University of Illinois Urbana-Champaign, Urbana, IL 61801 USA · JP Morgan AI Research, Palo Alto, CA 94304 USA · Aerospace Engineering and Engineering Mechanics Department at University of Texas, Austin, Austin, TX 78712 USA

  16. Statistical Inference for Policy Evaluation with Temporal Difference Learning

    Oct 21, 2024Weichen Wu, Gen Li, Yuting Wei +1Parameter EstimationConfidence Region Estimation

    The Voleon Group, New York, NY, USA · Department of Statistics, The Chinese University of Hong Kong, Hong Kong SAR. · Department of Statistics and Data Science, The Wharton School, University of Pennsylvania, Philadelphia, PA, USA. +1