Accelerated Gradient Methods

Ask about this topic

Overview

Accelerated gradient methods seek faster optimization by improving on ordinary gradient descent, often using momentum or estimates of curvature. Classical approaches include Nesterov acceleration and heavy-ball momentum, while FISTA extends acceleration to problems with nonsmooth regularizers; newer variants combine these ideas with adaptive preconditioning, stochastic mini-batches, or distributed communication. They are used in machine learning and in structured tasks such as sparse optimization, matrix problems, and linear programming. A central challenge is that acceleration can be sensitive to noise, conditioning, and problem structure, so researchers study when fast convergence guarantees hold and when momentum or predetermined step sizes can fail.

Latest papers

CardsList

44 papers7 papers in the last four weeks, up 40% on the four weeks before. 0.0% of all new papers.

  1. Deterministic Adam-Inspired Methods with Accelerated Convergence Rate

    Apr 9, 2026Yaxin Yu, Long Chen, Zeyi XuAccelerated Gradient MethodsAdaptive Gradient Methods

    School of Mathematics, Sichuan University, Chengdu, Sichuan, 610065, CHINA · Department of Mathematics, University of California, Irvine, Irvine, CA, 92697, USA

  2. Stochastic Auto-conditioned Fast Gradient Methods with Optimal Rates

    Apr 7, 2026Yao Ji, Guanghui LanStochastic OptimizationAccelerated Gradient Methods

    H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology, 225 North Ave, Atlanta, GA 30332, USA

  3. Scalable Mean-Variance Portfolio Optimization via Subspace Embeddings and GPU-Friendly Nesterov-Accelerated Projected Gradient

    Apr 3, 2026Yi-Shuai Niu, Yajuan WangPortfolio OptimizationRandomized Sketching

    Beijing Institute of Mathematical Sciences and Applications (BIMSA)

  4. Optimal Projection-Free Adaptive SGD for Matrix Optimization

    Apr 2, 2026Dmitry KovalevAdaptive Gradient MethodsMatrix Optimization

    Yandex Research

  5. Critical Damping as a Momentum Schedule: Multi-Seed Validation, a Hybrid Recipe, and an Exhaustive Negative Result on Surgical Layer Selection

    Mar 30, 2026Ivan PasichnykGradient-Based AttributionMomentum Methods

    We Label Data Inc.

  6. Improving Infinitely Deep Bayesian Neural Networks with Nesterov's Accelerated Gradient Method

    Mar 26, 2026Chenxu Yu, Wenqi FangBayesian Neural NetworksNeural Network Optimization

    Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences Shenzhen, P.R.China

  7. SHANG++: Robust Stochastic Acceleration under Multiplicative Noise

    Mar 10, 2026Yaxin Yu, Long Chen, Minfu FengAccelerated Gradient MethodsStochastic Optimization Convergence

    School of Mathematics, Sichuan University, Chengdu, Sichuan, 610065, China · Department of Mathematics, University of California, Irvine, Irvine, CA, 92697, USA

  8. Momentum SVGD-EM for Accelerated Maximum Marginal Likelihood Estimation

    Mar 9, 2026Adam Rozzio, Rafael Athanasiades, O. Deniz AkyildizMaximum Marginal Likelihood EstimationExpectation-Maximization

    Department of Mathematics, Ecole Normale Supérieure Paris-Saclay, France · Department of Mathematics, Imperial College London, UK

  9. FedBCD:Communication-Efficient Accelerated Block Coordinate Gradient Descent for Federated Learning

    Mar 5, 2026Junkang Liu, Fanhua Shang, Yuanyuan Liu +3Federated LearningAccelerated Gradient Methods

    School of Artificial Intelligence, Xi’an, Xidian University, China · College of Intelligence and Computing, Tianjin, Tianjin University, China · Medical College, Tianjin University; Peng Cheng Laboratory, Tianjin, China +1

  10. Accelerating LLM Pre-Training through Flat-Direction Dynamics Enhancement

    Feb 26, 2026Shuchen Zhu, Rizhen Hu, Mingze Wang +4Accelerated Gradient MethodsLanguage Model Pretraining

    Peking University · Zhejiang Lab, Zhejiang · Meituan, Beijing

  11. Complexity of Classical Acceleration for ℓ1\ell_1-Regularized PageRank

    Feb 24, 2026Kimon Fountoulakis, David Martínez-RubioAccelerated Gradient Methods

    University of Waterloo, Canada · IMDEA Software Institute, Madrid, Spain

  12. Accelerated Gradient Methods for Nonconvex Optimization: Escape Trajectories From Strict Saddle Points and Convergence to Local Minima

    Jul 13, 2023Rishabh Dixit, Mert Gurbuzbalaban, Waheed U. BajwaNonconvex OptimizationAccelerated Gradient Methods

    Department of Electrical and Computer Engineering, Rutgers University–New Brunswick, NJ 08854 USA · Departments of Electrical & Computer Engineering, Management Science and Information Systems, and Statistics, Rutgers University–New Brunswick, NJ 08854 USA · Departments of Electrical & Computer Engineering and Statistics, Rutgers University–New Brunswick, NJ 08854 USA

  13. NAG-GS: A Semi-Implicit Stochastic Optimizer with Large-Step Stability

    Sep 29, 2022Valentin Leplat, Daniil Merkulov, Aleksandr Katrutsa +3Stochastic OptimizationAccelerated Gradient Methods

    Skolkovo Institute of Science and Technology, Moscow, Russia · Moscow Institute of Physics and Technology, Moscow, Russia · AIRI, Moscow, Russia +1

  14. Privacy-Preserving Logistic Regression Training with A Faster Gradient Variant

    Jan 26, 2022John ChiangLogistic RegressionGradient Descent