Momentum Methods

Follow this topic Ask about this topic

Momentum

2 papers in the last four weeks, down 33% on the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 43

All topics

Muon meets Tamed Langevin: Momentum Preconditioning beyond Convex and gradient-Lipschitz Potentials
Oct 1, 2026Nikolaos Makras, Sotirios SabanisMarkov Chain Monte Carlo Momentum Methods
School of Mathematics, University of Edinburgh, UK · National Technical University of Athens, Greece · Athena/Archimedes Research Centre, Greece
Optimal Momentum Methods for Stochastic Multilevel Compositional Optimization
Oct 1, 2026Wei Jiang, Rui Yan, Sifan Yang +3Stochastic Optimization Momentum Methods
School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China · School of Artificial Intelligence, Nanjing University, Nanjing, China · School of Software Technology, Zhejiang University, Ningbo, China
Momentum in large-batch training: Polyak enlarges the critical batch size, Nesterov improves data efficiency
Sep 2, 2026Jia-Nan Wang, Zixun Huang, Kairui Li +1Momentum Methods Gradient Descent
Peking University · University of Pennsylvania · AI for Science Institute, Beijing
Activation-Keyed Momentum: An Anisotropic Momentum Update via the Delta Rule
Aug 19, 2026Euijin Hong, Guannan QuDeep Learning Optimization Momentum Methods
Electrical and Computer Engineering, Carnegie Mellon University
Momentum as Residual-Driven Multiplier Correction for Deep Learning Optimization
Aug 13, 2026Zhixin Ren, Yau Lyu, Congrong Li +2Deep Learning Optimization Momentum Methods
Department of Mathmatical Science, Tsinghua University · School of Vehicle and Mobility, Tsinghua University · College of AI, Tsinghua University
An Inertial Block Proximal Linearized Method with Adaptive Momentum for Nonconvex and Nonsmooth Optimization
Aug 6, 2026Weifeng YangMomentum Methods Nonconvex Optimization
Yunnan Earthquake Administration, Kunming 650224, China
Nesterov acceleration in optimizing over probability measures
Jul 25, 2026Jiaqi Tang, Qin Li, Wilfrid GangboMomentum Methods Gradient Descent
Department of Statistics, University of Wisconsin–Madison, Madison, WI 53706, USA · Department of Mathematics, University of Wisconsin–Madison, Madison, WI 53706, USA · Department of Mathematics, University of California, Los Angeles, Los Angeles, CA 90095, USA
Muse: Representation Geometry of Muon Beyond Normalized Momentum
Jul 16, 2026Da Chang, Qiankun Shi, Lvgang Zhang +4Representation Geometry Momentum Methods
1Pengcheng Laboratory · 2Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences +3
Heavy-Ball Q-Learning with Residual Weighting Correction
Jun 25, 2026Donghwan LeeMomentum Methods Q-Learning
Department of Electrical Engineering Korea Advanced Institute of Science and Technology (KAIST) Daejeon 34141, South Korea
Compute Efficiency and Serial Runtime Tradeoffs for Stochastic Momentum Methods
Jun 17, 2026Depen Morwani, Alexandru Meterez, Pranav Nair +1Stochastic Optimization Momentum Methods
1Harvard University · 2Kempner Institute at Harvard University · †: Work done while at Harvard.
CacheMuon: Using Temporal Preconditioning To Approximate Polar Factor
Jun 15, 2026Bishnu Dev, Sushil Bohara, Martin Takáč +1Momentum Methods Neural Network Optimization
Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE · Mohamed bin Zayed University of Artificial Intelligence
OptMuon: Closed-Loop Orthogonalized Momentum Methods for Stochastic Optimization with Zero-Noise Optimality
Jun 7, 2026Ganzhao YuanMomentum Methods Nonconvex Stochastic Optimization
Faculty of Computer Science and Artificial Intelligence Shenzhen University of Advanced Technology (SUAT), China
Denoise First, Orthogonalize Later: Understanding Momentum in Muon via Spectral Filtering
Jun 2, 2026Xianliang Li, Zihan Zhang, Weiyang Liu +1Deep Learning Optimization Momentum Methods
1The Institute of Statistical Mathematics · 2The Graduate Institute for Advanced Studies, SOKENDAI · 3National Institute of Informatics +3
Spectral Scaling Laws of Muon
Jun 2, 2026Gagik Magakyan, Pablo Parrilo, Asuman OzdaglarNewton-Schulz Iteration Momentum Methods
MIT
Fast Generalization after Interpolation via Critically Damped Momentum Optimization
Jun 1, 2026Luca Muscarnera, Silas Ruhrberg Estévez, Yuanzhang Xiao +1Neural Network Generalization Momentum Methods
Dynamics of Stochastic Momentum with Sparse Updates in High Dimensions
May 27, 2026Katie Everett, Elliot PaquetteMomentum Methods Gradient Descent Dynamics
Google DeepMind & MIT · McGill University & Mila
Outer-Momentum Restarting in High-Dimensional Two-Phase Optimization
May 27, 2026Kristi Topollai, Allan Ma, Tolga Dimlioglu +2Momentum Methods Communication-Efficient Distributed Training
New York University, New York, USA
Stochastic Gradient Descent with Momentum is Algorithmically Stable
May 27, 2026Yunwen Lei, Zimeng Wang, Xiaoming YuanMomentum Methods Algorithmic Stability
Department of Mathematics, The University of Hong Kong, Hong Kong, China · Department of Mathematics and Mathematical Statistics, Umeå University, Sweden
Momentum Streams for Optimizer-Inspired Transformers
May 23, 2026Jingchu Gai, Nai-Chieh Huang, Jiayun WuDeep Learning Optimization Transformer
Carnegie Mellon University
Move on Muon : A Hamiltonian probability gradient flow perspective of Muon optimizer
May 22, 2026Aratrika Mustafi, Soumya Mukherjee, Bharath K. SriperumbudurMomentum Methods Muon Optimizer
Department of Statistics, Pennsylvania State University
Ada2MS: A Hybrid Optimization Algorithm Based on Exponential Mixing of Elementwise and Global Second-Moment Estimates
May 19, 2026Meng Zhu, Quan Xiao, Weidong MinDeep Learning Optimization Momentum Methods
School of Information Management and Mathematics, Jiangxi University of Finance and Economics, Nanchang 330032, China · School of Mathematics and Computer Science, Nanchang University, Nanchang 330031, China · Institute of Metaverse, Nanchang University, Nanchang 330031, China +1
LionMuon: Alternating Spectral and Sign Descent for Efficient Training
May 19, 2026Arman Bolatov, Artem Riabinin, Nikita Kornilov +4Stochastic Optimization Sign-Based Optimization
Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) · Basic Research of Artificial Intelligence Laboratory (BRAIn Lab) · Applied Artificial Intelligence Institute +1
Perfect Parallelization in Mini-Batch SGD with Classical Momentum Acceleration
May 18, 2026Sachin Garg, Michał DerezińskiMomentum Methods Stochastic Optimization Convergence
University of Michigan
Ringmaster LMO: Asynchronous Linear Minimization Oracle Momentum Method
May 18, 2026Abdurakhmon Sadiev, Artavazd Maranjyan, Ivan Ilin +1Momentum Methods Distributed Optimization
KAUST
Stochastic Compositional Optimization via Hybrid Momentum Frank--Wolfe
May 14, 2026El Mahdi ChaytiStochastic Optimization Momentum Methods
Machine Learning and Optimization Laboratory (MLO) EPFL, Switzerland
Spectral Flattening Is All Muon Needs: How Orthogonalization Controls Learning Rate and Convergence
May 13, 2026Tien-Phat Nguyen, Truong Nguyen, Minh-Phuc Truong +3Momentum Methods Gradient Descent
Hanoi University of Science and Technology Hanoi, Vietnam · Indiana University 107 S. Indiana Ave, Bloomington, IN 47405, USA · Monash University Clayton, VIC 3800, Australia
Adam-SHANG: A Convergent Adam-Type Method for Stochastic Smooth Convex Optimization
May 13, 2026Yaxin Yu, Long Chen, Minfu FengStochastic Optimization Momentum Methods
School of Mathematics Sichuan University Chengdu, China · Department of Mathematics University of California, Irvine Irvine, CA, USA
Refresh-Scaling the Memory of Balanced Adam
May 11, 2026Alberto Fernández-Hernández, Cristian Pérez-Corral, Jose I. Mestre +2Deep Learning Optimization Momentum Methods
Universitat Polit`ecnica de Val`encia, Val`encia, Spain · Universitat Jaume I, Castell´o de la Plana, Spain
Muon with Nesterov Momentum: Heavy-Tailed Noise and (Randomized) Inexact Polar Decomposition
May 7, 2026Sayantan Choudhury, Xiaoran Cheng, Martin Takáč +2Stochastic Optimization Momentum Methods
MBZUAI · Penn State University · Georgia Institute of Technology +1
A Rod Flow Model for Adam at the Edge of Stability
May 7, 2026Eric Regis, Sinho ChewiEdge of Stability Momentum Methods
Yale University

Oct 1, 2026, cs.LGJ/K move · Enter open · S save

Muon meets Tamed Langevin: Momentum Preconditioning beyond Convex and gradient-Lipschitz Potentials

Nikolaos Makras, Sotirios Sabanis

School of Mathematics, University of Edinburgh, UK · National Technical University of Athens, Greece · Athena/Archimedes Research Centre, Greece

Markov Chain Monte Carlo Momentum Methods Langevin Dynamics

Open paper AskPDF arXiv

We consider the problem of sampling from Gibbs distributions on matrix spaces whose potential energies are neither convex nor globally gradient-Lipschitz. We introduce a family of non-quadratic kinetic energies that lead to a new underdamped Langevin system with momentum preconditioning, in which the gradient of the kinetic energy acts as a smooth spectral taming of the momentum. We prove that, under these relaxed assumptions on the potential, the resulting dynamics leaves the target Gibbs measure invariant, and we establish exponential convergence to equilibrium in a weighted total variation distance. Finally, we show that the corresponding Euler-Maruyama discretization admits moment bounds that are uniform in time, without any modification of the potential gradient, which ensures the stability of the resulting sampling algorithm.

Momentum

Related topics

Latest papers 43