Overview

Stochastic approximation studies how repeated updates based on noisy samples can find solutions to optimization, root-finding, and fixed-point problems, including those in reinforcement learning. Common approaches include Polyak–Ruppert averaging, two-time-scale updates, variance reduction, and Lyapunov or Poisson-equation analyses to control noise and dependence. Current work develops finite-time convergence and uncertainty guarantees for settings such as Markovian data, heavy-tailed noise, and asynchronous updates. A central challenge is to obtain sharp, reliable rates without restrictive assumptions on smoothness, stability, or noise.

Latest papers

CardsList

83 papers7 papers in the last four weeks, down 56% on the four weeks before. 0.1% of all new papers.

  1. Sharp Gaussian Approximation for Q-Learning

    Nov 15, 2026Xingtu LiuQ-LearningReinforcement Learning

  2. Understanding the Asymptotic Transience of Q-Learning via Distributional Expansion

    Nov 11, 2026Shengbo WangQ-LearningTemporal-Difference Learning

  3. Optimal Last-Iterate Bounds for Nonexpansive Stochastic Approximation

    Nov 10, 2026Ramanan Tamizholi, Siddharth Chandak, Rahul SinghStochastic ApproximationLast-Iterate Convergence

  4. Finite Time Analysis of Risk-Sensitive RL via Noisy Power Iteration

    Nov 7, 2026Waqar Mirza, Yashaswini Murthy, Laixi Shi +2Risk-Sensitive RLStochastic Approximation

  5. Riemannian Newton Methods with Statistical Inference

    Oct 31, 2026Xiaotian Chang, Qirui Hu, Yangdi Jiang +1Covariance EstimationRiemannian Optimization

  6. Bootstrap Inference for Riemannian Optimization Problems

    Oct 31, 2026Xiaotian Chang, Qirui Hu, Yangdi Jiang +1Confidence Region EstimationRiemannian Optimization

  7. Stochastic Tracking of Drifting Affine Fixed Points

    Oct 19, 2026Sajad KhodadadianStochastic ApproximationNon-Stationary RL

  8. A Nonlinear Time-Series Theory for Constant-step SGD with Momentum

    Oct 12, 2026Younghun Lee, Jake Zummo, Wei Biao WuSGD with MomentumStochastic Gradient Descent

  9. Limit Theory for Online Expectile Estimation under Short and Long Range Dependence

    Oct 12, 2026Younghun Lee, Jake Zummo, Wei Biao WuLong-Range Dependency ModelingStochastic Approximation

  10. Best of Both Worlds in Federated LSA: Speedup When Possible, Personalization Always

    Oct 8, 2026Safwan Labbi, Paul Mangold, Eric MoulinesFederated LearningHeterogeneous Federated Learning

  11. Stationary Bias and Extrapolation in Nonlinear Two-Timescale Stochastic Approximation

    Oct 7, 2026A. Ch. Madhusudanarao, Rahul SinghRichardson ExtrapolationStochastic Approximation

  12. Near-Minimax-Optimal Rates for Asynchronous TD Learning with Markovian Data

    Sep 30, 2026Yang PengAsynchronous RLStochastic Approximation

  13. Probe-Space Preconditioning for Fast and Stable Zero-Order Training

    Sep 29, 2026Francois Chaubard, Mykel J. Kochenderfer, Chris RéZeroth-Order OptimizationMemory-Efficient Fine-Tuning

  14. Second-Moment Stochastic Approximation Methods

    Sep 29, 2026Tao Jiang, Lin XiaoStochastic ApproximationMuon Optimizer

  15. Dual-Anchor Acceleration Is Near-Optimal for Stochastic Monotone Root-Finding

    Sep 28, 2026TaeHo Yoon, Nicolas LoizouStochastic Approximation

  16. Finite-Time Concentration and Convergence Rates for Projected Two-Time-Scale Stochastic Approximation with Markov Noise

    Sep 28, 2026Rahul Singh, Vivek S. Borkar, Eric MoulinesLast-Iterate ConvergenceStochastic Approximation

  17. A Contraction Framework for Stochastic Operators with Bootstrapping: Application to TD Learning

    Sep 24, 2026Ids van der Werf, Sergio Rozada, Antonio G. MarquesStochastic ApproximationTemporal-Difference Learning

  18. Distributed Stochastic Approximation Algorithms and Heavy-Tailed Age of Information

    Sep 23, 2026Adrian Redder, Arunselvan Ramaswamy, Holger KarlStochastic ApproximationAge of Information

  19. Stochastic Inertial Krasnosel'skii-Mann Iteration Achieves Near-Optimal Sample Complexity

    Sep 22, 2026Tong Yang, Tao Jiang, Yuejie Chi +2Last-Iterate ConvergenceStochastic Approximation

  20. Polyak-Type Extragradient Methods for Monotone Root-Finding Problems

    Sep 22, 2026TaeHo Yoon, Sayantan Choudhury, Ezra Greenberg +1Stochastic ApproximationExtragradient Methods

  21. Locally Private Inference for Riemannian Stochastic Optimization

    Sep 18, 2026Xiaotian Chang, Yangdi Jiang, Qirui HuRiemannian OptimizationConfidence Region Estimation

  22. The Bias of Nonlinear Two-Time-scale Stochastic Approximation under Constant Step-Sizes

    Sep 17, 2026Djamel Rassem Lamouri, Dorian Baudry, Nicolas GastStochastic ApproximationTwo-Timescale Stochastic Approximation

  23. Improving the Last-Iterate Guarantees of Anytime Algorithms for Stochastic Monotone Variational Inequalities

    Sep 14, 2026Jun-Hyun Kim, Ahmet AlacaogluLast-Iterate ConvergenceStochastic Approximation

  24. Steady-State Convergence of Stochastic Approximation

    Sep 14, 2026Yixuan Zhang, Qiaomin XieStochastic ApproximationQ-Learning

  25. Regularized Emphatic Temporal-Difference Learning: Stability under Constant Stepsizes

    Sep 14, 2026Xingguo Chen, Zhaohui Wu, Jinguo Ye +5Temporal-Difference LearningStochastic Approximation

  26. Equilibrium bias and convergence in augmented primal--dual dynamics with sampled constraints

    Sep 12, 2026Kang Liu, Mengxiao Chen, Siqi Xiong +1Primal-Dual OptimizationStochastic Optimization Convergence

  27. Inference for Newton Methods with Accelerated Sketch-and-Project via Random Scaling

    Sep 11, 2026Xinchen Du, Elizaveta Rebrova, Michał Dereziński +1Stochastic ApproximationSecond-Order Optimization