Accelerated Gradient Methods

Follow this topic Ask about this topic

Momentum

2 papers in the last four weeks, down 33% on the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 23

All topics

Neural Algorithmic Reasoning for Graph Saddle Point Problems
Oct 5, 2026Samantha Chen, Jesse He, Coleman Clougherty +2Primal-Dual Optimization Graph Neural Networks
Oberlin College · Department of Computer Science, Oberlin College · University of California, San Diego +2
Reinforcement Learning to Accelerate Primal-Dual Hybrid Gradient for Linear Programming
Oct 1, 2026Jinhwan Sul, Alex Oshin, Evangelos A. TheodorouPrimal-Dual Optimization Reinforcement Learning
Georgia Institute of Technology, USA
Accelerated Algorithm for Sparse Regularized Partial Optimal Transport
Sep 30, 2026Khoa Nguyen, Dung T. Nguyen, Thong Huynh +4Inverse Optimal Transport Optimal Transport
Aalto University, Konemiehentie 2, 02150 Espoo, Finland · Linköping University, 581 83 Linköping, Sweden · Hanoi University of Science and Technology, Hanoi, Vietnam +4
Activation-Keyed Momentum: An Anisotropic Momentum Update via the Delta Rule
Aug 19, 2026Euijin Hong, Guannan QuDeep Learning Optimization Momentum Methods
Electrical and Computer Engineering, Carnegie Mellon University
Momentum as Residual-Driven Multiplier Correction for Deep Learning Optimization
Aug 13, 2026Zhixin Ren, Yau Lyu, Congrong Li +2Deep Learning Optimization Momentum Methods
Department of Mathmatical Science, Tsinghua University · School of Vehicle and Mobility, Tsinghua University · College of AI, Tsinghua University
A lower bound for stepsize-based acceleration of gradient descent
Aug 11, 2026Jianhao Ma, Yuxin ChenLearning Rate Scheduling Last-Iterate Convergence
Department of Industrial Engineering, Tsinghua University · Department of Statistics and Data Science, Wharton School, University of Pennsylvania
On MUON optimization: From non-convergence to an error analysis with Polar Express and the Newton-Schulz polynomial from implementations
Aug 5, 2026Thang Do, Steffen Dereich, Arnulf JentzenStochastic Optimization Newton-Schulz Iteration
School of Data Science, The Chinese University of Hong Kong, Shenzhen (CUHK-Shenzhen), China · Department of Probability and Statistics, Institute of Mathematics, Vietnam Academy of Science and Technology, Vietnam · Institute for Mathematical Stochastics, Faculty of Mathematics and Computer Science, University of Münster, Germany +2
Compute Efficiency and Serial Runtime Tradeoffs for Stochastic Momentum Methods
Jun 17, 2026Depen Morwani, Alexandru Meterez, Pranav Nair +1Stochastic Optimization Momentum Methods
1Harvard University · 2Kempner Institute at Harvard University · †: Work done while at Harvard.
When to use what Schatten- $p$ norm in deep learning?
Jun 13, 2026Thomas PethickDeep Learning Optimization Muon Optimizer
Predictive Coding with Bayesian Priors via Proximal Gradients
Jun 6, 2026Francesco BulloPredictive Coding Markov Random Fields
Mechanical Engineering, 2325 Engineering Bldg II, UCSB, Santa Barbara, CA 93106-5070, USA
Accelerated Decentralized Stochastic Gradient Descent for Strongly Convex Optimization
Jun 5, 2026Ming Sun, Kun YuanGradient Descent Accelerated Gradient Methods
Center for Machine Learning Research Peking University
Near-Optimal Decentralized Stochastic Convex Optimization over Networks
Jun 3, 2026Nitai Kluger, Amit Attia, Tomer KorenStochastic Optimization Convergence Accelerated Gradient Methods
Blavatnik School of Computer Science, Tel Aviv University · Google Research Tel Aviv
Stochastic Gradient Descent with Momentum is Algorithmically Stable
May 27, 2026Yunwen Lei, Zimeng Wang, Xiaoming YuanMomentum Methods Algorithmic Stability
Department of Mathematics, The University of Hong Kong, Hong Kong, China · Department of Mathematics and Mathematical Statistics, Umeå University, Sweden
MONA: Muon Optimizer with Nesterov Acceleration for Scalable Language Model Training
May 26, 2026Jiacheng Li, Jianchao Tan, Hongtao Xu +5Language Model Pretraining Gradient Descent
Meituan, Beijing, China
EMA-Nesterov: Stabilizing Nesterov's Lookahead for Accelerated Deep Learning Optimization
May 25, 2026Chung-Yiu Yau, Dawei Li, Athanasios Glentis +3Deep Learning Optimization Gradient Descent
University of Minnesota · 2Amazon AGI · 3The Chinese University of Hong Kong
Perfect Parallelization in Mini-Batch SGD with Classical Momentum Acceleration
May 18, 2026Sachin Garg, Michał DerezińskiMomentum Methods Stochastic Optimization Convergence
University of Michigan
Accelerated Gradient Descent for Faster Convergence with Minimal Overhead
May 15, 2026Manuel Graca, L. Miguel Silveira, Arlindo Oliveira +1Deep Learning Optimization Gradient Descent
IST, ULisboa Lisbon, Portugal · Old Dominion University Norfolk, VA USA
A Unified Lyapunov-IQC Framework for Uniform Stability of Smooth Quadratic First-Order Accelerated Optimizers
May 8, 2026Don Li, Dacian DaescuLyapunov Stability Algorithmic Stability
Department of Mathematics & Statistics, Portland State University, Portland, OR 97201, USA
A Parameter-Free First-Order Algorithm for Non-Convex Optimization with $\tilde{\mkern1mu O}(ε^{-5/3})$ Global Rate
May 4, 2026Sichao Xiong, Sadok Jerad, Coralia CartisOptimization Convergence Analysis Gradient Descent
Department of Computer Science University of Oxford Oxford, OX1 3QG · Mathematical Institue University of Oxford Oxford, OX2 6GG
Randomized Subspace Nesterov Accelerated Gradient
May 1, 2026Gaku Omiya, Pierre-Louis Poirion, Akiko TakedaGradient Descent Randomized Sketching
Department of Mathematical Informatics, The University of Tokyo, Tokyo, Japan · Center for Advanced Intelligence Project, RIKEN, Tokyo, Japan
Critical Damping as a Momentum Schedule: Multi-Seed Validation, a Hybrid Recipe, and an Exhaustive Negative Result on Surgical Layer Selection
Mar 30, 2026Ivan PasichnykGradient-Based Attribution Momentum Methods
We Label Data Inc.
Privacy-Preserving Logistic Regression Training with A Faster Gradient Variant
Jan 26, 2022John ChiangLogistic Regression Gradient Descent
Silver Rate Is (Almost) Optimal for Gradient Descent
Date pendingYuhan Ye, Kaizhao LiuLearning Rate Scheduling Optimization Convergence Analysis
MIT

Oct 5, 2026, cs.LGJ/K move · Enter open · S save

Neural Algorithmic Reasoning for Graph Saddle Point Problems

Samantha Chen, Jesse He, Coleman Clougherty, Gal Mishne+1

Oberlin College · Department of Computer Science, Oberlin College · University of California, San Diego+2

Primal-Dual Optimization Graph Neural Networks Neural Algorithmic Reasoning Min-Max Optimization Algorithmic Reasoning Accelerated Gradient Methods

Open paper AskPDF arXiv

Neural algorithmic reasoning, or aligning a neural network with an algorithmic paradigm, has emerged as an approach to solving polynomial-time-solvable and computationally harder combinatorial optimization problems. We propose a new message-passing framework based on the Chambolle-Pock Primal--Dual Hybrid Gradient (PDHG) method called \textsc{GraphPDHG} for solving general graph saddle-point problems. Theoretically, we show that \textsc{GraphPDHG} can efficiently solve a family of graph saddle-point problems by simulating PDHG. We also show that our network can learn an accelerated PDHG algorithm. Experimentally, we support our results on accelerated PDHG by evaluating the performance of our model as a learned warm start for second-order optimization techniques (SSNAL). We also show that alignment with PDHG leads to stronger size generalization than non-aligned graph neural network (GNN) baselines. Overall, we propose a novel architecture for solving a general family of optimization problems on graphs.

Momentum

Related topics

Latest papers 23