Variance Reduction

Variance reduction techniques aim to improve the efficiency and stability of various machine learning and optimization algorithms by decreasing the variability in estimates of gradients or other key quantities. Current research focuses on applying these techniques to diverse areas, including meta-learning, reinforcement learning, federated learning, and Monte Carlo methods, often employing algorithms like stochastic variance reduced gradient (SVRG), control variates, and importance sampling. These advancements lead to faster convergence, improved sample efficiency, and more robust performance in various applications, ultimately impacting the scalability and reliability of machine learning models and simulations.

Papers

December 31, 2023

SteinDreamer: Variance Reduction for Text-to-3D Score Distillation via Stein Identity
Peihao Wang, Zhiwen Fan, Dejia Xu, Dilin Wang, Sreyas Mohan, Forrest Iandola, Rakesh Ranjan, Yilei Li, Qiang Liu, Zhangyang Wang, Vikas Chandra
Variance Reduction Score Distillation Text to 3D Content Creation

November 23, 2023

Byzantine Robustness and Partial Participation Can Be Achieved at Once: Just Clip Gradient Differences
Grigory Malinovsky, Peter Richtárik, Samuel Horváth, Eduard Gorbunov
Machine Learning Model Variance Reduction Byzantine Robust Client Sampling Gradient Clipping Byzantine Fault Tolerance Partial Participation Byzantine Robust Method

November 15, 2023

Efficiently Escaping Saddle Points for Non-Convex Policy Optimization
Sadegh Khorasani, Saber Salehkaleybar, Negar Kiyavash, Niao He, Matthias Grossglauser
Policy Gradient Variance Reduction Saddle Point Variance Reduced Policy Gradient

November 9, 2023

A Coefficient Makes SVRG Effective
Yida Yin, Zhiqiu Xu, Zhiyuan Li, Trevor Darrell, Zhuang Liu
Variance Reduction Coefficient Prediction Variance Reduction Technique Variance Reduced

October 21, 2023

Distributionally Robust Optimization with Bias and Variance Reduction
Ronak Mehta, Vincent Roulet, Krishna Pillutla, Zaid Harchaoui
Absolute Stance Bias Stochastic Gradient Robust Optimization Risk Sensitive Variance Reduction Distributionally Robust Optimization F$ Divergence Spectral Risk

September 19, 2023

Oracle Complexity Reduction for Model-free LQR: A Stochastic Variance-Reduced Policy Gradient Approach
Leonardo F. Toso, Han Wang, James Anderson
Policy Gradient Variance Reduction Oracle Complexity Variance Reduced Policy Gradient

September 10, 2023

Variance Reduction of Resampling for Sequential Monte Carlo
Xiongming Dai, Gerald Baumgartner
Hidden Markov Model Variance Reduction Sequential Monte Carlo Resampling Technique

September 7, 2023

Byzantine-Robust Federated Learning with Variance Reduction and Differential Privacy
Zikai Zhang, Rui Hu
Differential Privacy Privacy Guarantee Variance Reduction Privacy Analysis Byzantine Attack Byzantine Robust Federated Learning Client Level Differential Privacy

September 3, 2023

Double Clipping: Less-Biased Variance Reduction in Off-Policy Evaluation
Jan Malte Lichtenberg, Alexander Buchholz, Giuseppe Di Benedetto, Matteo Ruffini, Ben London
Policy Evaluation Variance Reduction Policy Estimation High Dimensional Clipping Variance Reduction Technique

August 11, 2023

Adaptive SGD with Polyak stepsize and Line-search: Robust Convergence and Variance Reduction
Xiaowen Jiang, Sebastian U. Stich
Variance Reduction Step Size Parameterized Model Line Search Stochastic Polyak Gradient Evaluation Stochastic Line Search Polyak Step Size

July 30, 2023

Variance Control for Distributional Reinforcement Learning
Qi Kuang, Zhoufan Zhu, Liwen Zhang, Fan Zhou
Variance Reduction Distributional Reinforcement Learning Distributional Assumption Q Function Function Estimation

July 24, 2023

Policy Gradient Optimal Correlation Search for Variance Reduction in Monte Carlo simulation and Maximum Optimal Transport
Pierre Bras, Gilles Pagès
Optimal Transport Policy Gradient Variance Reduction Monte Carlo Simulation Correlation Function Optimal Coupling

June 18, 2023

DropCompute: simple and more robust distributed synchronous training via compute variance reduction
Niv Giladi, Shahar Gottlieb, Moran Shkolnik, Asaf Karnieli, Ron Banner, Elad Hoffer, Kfir Yehuda Levy, Daniel Soudry
Deep Neural Network Variance Reduction Asynchronous Training Large Scale Training Neural Network Training

June 12, 2023

Convergence of mean-field Langevin dynamics: Time and space discretization, stochastic gradient, and variance reduction
Taiji Suzuki, Denny Wu, Atsushi Nitanda
Early Stage Convergence Stochastic Gradient Mean Field Langevin Dynamic Variance Reduction Discretization Error

June 2, 2023

Reinforcement Learning with General Utilities: Simpler Variance Reduction and Large State-Action Space
Anas Barakat, Ilyas Fatkhullin, Niao He
Reinforcement Learning Action Space Task Utility Variance Reduction Global Optimality Policy Gradient Algorithm

May 30, 2023

Blockwise Stochastic Variance-Reduced Methods with Parallel Speedup for Multi-Block Bilevel Optimization
Quanqi Hu, Zi-Hao Qiu, Zhishuai Guo, Lijun Zhang, Tianbao Yang
Bilevel Optimization Hessian Matrix Variance Reduction Linear Speedup

May 27, 2023

Knowledge Distillation Performs Partial Variance Reduction
Mher Safaryan, Alexandra Peste, Dan Alistarh
Deep Neural Network Knowledge Distillation Variance Reduction Convex Loss Deep Linear Distillation Loss

April 3, 2023

Theoretical guarantees for neural control variates in MCMC
Denis Belomestny, Artur Goldman, Alexey Naumov, Sergey Samsonov
Markov Chain Monte Carlo Markov Chain Variance Reduction Neural Network Controller Theoretical Guarantee Control Variate

March 29, 2023

Infeasible Deterministic, Stochastic, and Variance-Reduction Algorithms for Optimization under Orthogonality Constraints
Pierre Ablin, Simon Vary, Bin Gao, P.-A. Absil
Optimization Purpose Stochastic Way Variance Reduction Riemannian Optimization Autonomous Landing Orthogonality Constraint Orthogonal Basis

March 8, 2023

Meta-learning Control Variates: Variance Reduction with Limited Data
Zhuo Sun, Chris J. Oates, François-Xavier Briol
Limited Data Monte Carlo Variance Reduction Control Variate Meta Control

Variance Reduction

Papers

SteinDreamer: Variance Reduction for Text-to-3D Score Distillation via Stein Identity

Byzantine Robustness and Partial Participation Can Be Achieved at Once: Just Clip Gradient Differences

Efficiently Escaping Saddle Points for Non-Convex Policy Optimization

A Coefficient Makes SVRG Effective

Distributionally Robust Optimization with Bias and Variance Reduction

Oracle Complexity Reduction for Model-free LQR: A Stochastic Variance-Reduced Policy Gradient Approach

Variance Reduction of Resampling for Sequential Monte Carlo

Byzantine-Robust Federated Learning with Variance Reduction and Differential Privacy

Double Clipping: Less-Biased Variance Reduction in Off-Policy Evaluation

Adaptive SGD with Polyak stepsize and Line-search: Robust Convergence and Variance Reduction

Variance Control for Distributional Reinforcement Learning

Policy Gradient Optimal Correlation Search for Variance Reduction in Monte Carlo simulation and Maximum Optimal Transport

DropCompute: simple and more robust distributed synchronous training via compute variance reduction

Convergence of mean-field Langevin dynamics: Time and space discretization, stochastic gradient, and variance reduction

Reinforcement Learning with General Utilities: Simpler Variance Reduction and Large State-Action Space

Blockwise Stochastic Variance-Reduced Methods with Parallel Speedup for Multi-Block Bilevel Optimization

Knowledge Distillation Performs Partial Variance Reduction

Theoretical guarantees for neural control variates in MCMC

Infeasible Deterministic, Stochastic, and Variance-Reduction Algorithms for Optimization under Orthogonality Constraints

Meta-learning Control Variates: Variance Reduction with Limited Data