Stochastic Approximation

Stochastic approximation (SA) is an iterative method for finding the root of an operator using noisy observations, crucial for solving optimization problems where exact gradients are unavailable. Current research emphasizes improving SA's efficiency and robustness, focusing on variance reduction techniques, adaptive step sizes, and handling Markovian noise and delays in various settings, including distributed and federated learning, reinforcement learning, and temporal difference learning. These advancements are significant for tackling large-scale optimization problems in diverse fields, leading to improved algorithms with stronger convergence guarantees and enhanced applicability to real-world scenarios.

Papers

June 13, 2023

User-defined Event Sampling and Uncertainty Quantification in Diffusion Models for Physical Dynamical Systems
Marc Finzi, Anudhyan Boral, Andrew Gordon Wilson, Fei Sha, Leonardo Zepeda-Núñez
Diffusion Model Uncertainty Quantification Stochastic Approximation Physical System Probabilistic Generative Model Conditional Sampling

June 2, 2023

Federated Multi-Sequence Stochastic Approximation with Local Hypergradient Estimation
Davoud Ataee Tarzanagh, Mingchen Li, Pranay Sharma, Samet Oymak
Convergence Guarantee Stochastic Approximation Optimal Rate Approximation Theory Compositional Optimization Federated Automatic Differentiation Hypergradient Estimation

May 31, 2023

Efficient Stochastic Approximation of Minimax Excess Risk Optimization
Lijun Zhang, Haomin Bai, Wei-Wei Tu, Ping Yang, Yao Hu
Robust Optimization Stochastic Approximation Stochastic Convex Optimization Excess Risk Minimax Optimization Problem

May 12, 2023

Online Learning Under A Separable Stochastic Approximation Framework
Min Gan, Xiang-xiang Su, Guang-yong Chen, Jing Chen
Stochastic Gradient Descent Online Learning Stochastic Gradient Stochastic Approximation Online Learning Algorithm Newton Method

May 11, 2023

Stability and Convergence of Distributed Stochastic Approximations with large Unbounded Stochastic Information Delays
Adrian Redder, Arunselvan Ramaswamy, Holger Karl
Early Stage Convergence Core Stability Stochastic Approximation Stochastic Process Recursive Algorithm Arbitrary Delay Borkar Meyn Stability Theorem Cheeger Inequality

May 2, 2023

Random Function Descent
Felix Benning, Leif Döring
Bayesian Optimization Stochastic Approximation Worst Case Classical Optimization

April 7, 2023

Compressed Regression over Adaptive Networks
Marco Carpentiero, Vincenzo Matta, Ali H. Sayed
Stochastic Approximation Adaptive Network Mean Squared Error Compression Operator Dual Regression

April 3, 2023

A Tutorial Introduction to Reinforcement Learning
Mathukumalli Vidyasagar
Reinforcement Learning Markov Decision Process Tutorial Review Stochastic Approximation Q$ Learning Markov Reward Process

March 28, 2023

Concentration of Contractive Stochastic Approximation: Additive and Multiplicative Noise
Zaiwei Chen, Siva Theja Maguluri, Martin Zubeldia
Stochastic Approximation Selective Focus Concentration Bound Tail Risk Concentration Inequality Multiplicative Noise Feed Additive

February 18, 2023

Stochastic Approximation Approaches to Group Distributionally Robust Optimization and Beyond
Lijun Zhang, Haomin Bai, Peng Zhao, Tianbao Yang, Zhi-Hua Zhou
Multi Armed Bandit Robust Optimization Stochastic Approximation Distributionally Robust Optimization Stochastic Mirror Descent Stochastic Saddle Point Problem

January 11, 2023

An Analysis of Quantile Temporal-Difference Learning
Mark Rowland, Rémi Munos, Mohammad Gheshlaghi Azar, Yunhao Tang, Georg Ostrovski, Anna Harutyunyan, Karl Tuyls, Marc G. Bellemare, Will Dabney
Reinforcement Learning General Analysis Stochastic Approximation Distributional Reinforcement Learning TD Learning Quantile Temporal Difference

December 20, 2022

Generalized Simultaneous Perturbation-based Gradient Search with Reduced Estimator Bias
Soumen Pachal, Shalabh Bhatnagar, L. A. Prashanth
Stochastic Approximation Estimation Bias Noisy Measurement Simultaneous Perturbation Stochastic Approximation

December 15, 2022

Multi-Resolution Online Deterministic Annealing: A Hierarchical and Progressive Learning Architecture
Christos Mavridis, John Baras
Neural Network Multi Resolution Stochastic Approximation Hierarchical Learning Data Driven Optimization

November 8, 2022

Quantization-Based Optimization: Alternative Stochastic Approximation of Global Optimization
Jinwuk Seok, Chang Sik Cho
Optimization Purpose Global Optimization Stochastic Approximation Quantization Error NP Hard

October 23, 2022

Stochastic Mirror Descent for Large-Scale Sparse Recovery
Sasila Ilandarideva, Yannis Bekri, Anatoli Juditsky, Vianney Perchet
Stochastic Approximation Stochastic Optimization Problem Sparse Recovery Stochastic Mirror Descent Random Perturbation

October 13, 2022

Reinforcement Learning with Unbiased Policy Evaluation and Linear Function Approximation
Anna Winnicki, R. Srikant
Reinforcement Learning Markov Decision Process Stochastic Approximation Linear Function Approximation Policy Iteration

October 11, 2022

Zero-Order One-Point Estimate with Distributed Stochastic Gradient-Tracking Technique
Elissa Mhanna, Mohamad Assaad
Multi Agent Barzilai Borwein Technique Objective Function Stochastic Way Stochastic Approximation Gradient Tracking Convex Objective

October 10, 2022

Actor-Critic or Critic-Actor? A Tale of Two Time Scales
Shalabh Bhatnagar, Vivek S. Borkar, Soumyajit Guin
Stochastic Approximation Actor Critic Algorithm Policy Iteration Cautionary TALE Time Scale Nonlinear Function Approximation

October 9, 2022

A Concentration Bound for Distributed Stochastic Approximation
Harsh Dolhare, Vivek Borkar
Differential Equation Stochastic Approximation Consensus Group Decision Concentration Bound Distributed Stochastic

October 3, 2022

Bias and Extrapolation in Markovian Linear Stochastic Approximation with Constant Stepsizes
Dongyan Huo, Yudong Chen, Qiaomin Xie
Absolute Stance Bias Stochastic Approximation Linear Function Approximation Extrapolation Framework Homogeneous Markov Polyak Ruppert

Stochastic Approximation

Papers

User-defined Event Sampling and Uncertainty Quantification in Diffusion Models for Physical Dynamical Systems

Federated Multi-Sequence Stochastic Approximation with Local Hypergradient Estimation

Efficient Stochastic Approximation of Minimax Excess Risk Optimization

Online Learning Under A Separable Stochastic Approximation Framework

Stability and Convergence of Distributed Stochastic Approximations with large Unbounded Stochastic Information Delays

Random Function Descent

Compressed Regression over Adaptive Networks

A Tutorial Introduction to Reinforcement Learning

Concentration of Contractive Stochastic Approximation: Additive and Multiplicative Noise

Stochastic Approximation Approaches to Group Distributionally Robust Optimization and Beyond

An Analysis of Quantile Temporal-Difference Learning

Generalized Simultaneous Perturbation-based Gradient Search with Reduced Estimator Bias

Multi-Resolution Online Deterministic Annealing: A Hierarchical and Progressive Learning Architecture

Quantization-Based Optimization: Alternative Stochastic Approximation of Global Optimization

Stochastic Mirror Descent for Large-Scale Sparse Recovery

Reinforcement Learning with Unbiased Policy Evaluation and Linear Function Approximation

Zero-Order One-Point Estimate with Distributed Stochastic Gradient-Tracking Technique

Actor-Critic or Critic-Actor? A Tale of Two Time Scales

A Concentration Bound for Distributed Stochastic Approximation

Bias and Extrapolation in Markovian Linear Stochastic Approximation with Constant Stepsizes