Momentum Stochastic Gradient Descent

Ask about this topic

Momentum

3 papers in the last four weeks, level with the four weeks before. 0.0% of all new papers.

Jul 6Week of Sep 21
View on map

Latest papers 38

All topics
CardsList

  1. Convergence Analysis of STORM Under Different Geometries

    Oct 1, 2026Wei Jiang, Yibo Wang, Wenhao Yang +3Stochastic Convex OptimizationConvex Optimization

    School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China · School of Artificial Intelligence, Nanjing University, Nanjing, China

  2. Convergence of Practical Muon with Finite Newton-Schulz Iterations and Nesterov Momentum

    Sep 30, 2026Hanyng Peng, Hui Wang, Yue YuMomentum Stochastic Gradient DescentNewton

    Pengcheng Laboratory, Shenzhen, China

  3. The Hidden Ratio in Adam: Stable Structure, Compression, and Sign Dynamics

    Sep 28, 2026Yihe Zhou, Tongtian Zhu, Yingxiao Huo +4AdamMomentum Stochastic Gradient Descent

    Department of Computer Science, The University of Manchester, Manchester, UK · College of Computer Science and Technology, Zhejiang University, Hangzhou, China

  4. MomWorld: Momentum-Aware Latent World Model for Long-Horizon Autonomous Driving

    Sep 27, 2026Ziying Song, Shengkai Zhang, Lei Yang +6Long-Horizon PlanningLatent World Models

    Nanyang Technological University · Beijing Jiaotong University · North University of China +2

  5. ZO-COSMO: Index-Free One-Hop Mixing for Decentralized Zeroth-Order Optimization

    Sep 23, 2026Shengjun Zhang, Tingyi Liu, Heng Zhang +1Momentum Stochastic Gradient Descent

    School of Artificial Intelligence, Hubei University, Wuhan, China · School of Economics and Management, Wuhan University, Wuhan, China · School of Electrical Engineering, Shanghai Jiao Tong University, Shanghai, China +1

  6. ReMoMask-2: Latent Retrieval-Augmented Masked Motion Generation

    Sep 8, 2026Yiran Wang, Zeyu Zhang, Ling Shao +1Text-To-Motion GenerationNeural Mask Estimation

    University of Sydney, Camperdown 2006, Australia · School of Computer Science, Peking University, Beijing 100871, China · UCAS-Terminus AI Lab, University of Chinese Academy of Sciences, Beijing 101408, China

  7. Activation-Keyed Momentum: An Anisotropic Momentum Update via the Delta Rule

    Aug 19, 2026Euijin Hong, Guannan QuMomentum Stochastic Gradient DescentMomentum

    Electrical and Computer Engineering, Carnegie Mellon University

  8. Second Order Drifting Models

    Aug 8, 2026Drake Brown, Yuhao Huang, Shih-Hsin Wang +1Generative ModelsMomentum Stochastic Gradient Descent

    Department of Mathematics Scientific Computing and Imaging (SCI) Institute University of Utah

  9. An Inertial Block Proximal Linearized Method with Adaptive Momentum for Nonconvex and Nonsmooth Optimization

    Aug 6, 2026Weifeng YangProximal OperatorsStochastic Convex Optimization

    Yunnan Earthquake Administration, Kunming 650224, China

  10. Sign compression for Muon: SignMuon, MuonSign, and the Limits of Error Feedback

    Jul 31, 2026Maria Smirnova, Alexey KravatskiyMuonMomentum Stochastic Gradient Descent

    MIRIAI Moscow, Russia

  11. Nesterov acceleration in optimizing over probability measures

    Jul 25, 2026Jiaqi Tang, Qin Li, Wilfrid GangboMomentum Stochastic Gradient DescentProbability Measures

    Department of Statistics, University of Wisconsin–Madison, Madison, WI 53706, USA · Department of Mathematics, University of Wisconsin–Madison, Madison, WI 53706, USA · Department of Mathematics, University of California, Los Angeles, Los Angeles, CA 90095, USA

  12. Compute Efficiency and Serial Runtime Tradeoffs for Stochastic Momentum Methods

    Jun 17, 2026Depen Morwani, Alexandru Meterez, Pranav Nair +1Stochastic Gradient DescentMomentum Stochastic Gradient Descent

    1Harvard University · 2Kempner Institute at Harvard University · †: Work done while at Harvard.

  13. Scaling Adaptive Depth with Norm-Agnostic Residual Networks

    Jun 15, 2026Tomás Figliolia, Beren MillidgeResidual NetworksScaling

    Zyphra San Francisco, CA

  14. A Three-Layer Framework for AI in Scientific Discovery

    Jun 11, 2026Guojun LiaoScientific DiscoveryDiscovery

    Department of Mathematics, University of Texas at Arlington

  15. OptMuon: Closed-Loop Orthogonalized Momentum Methods for Stochastic Optimization with Zero-Noise Optimality

    Jun 7, 2026Ganzhao YuanMuonMomentum Stochastic Gradient Descent

    Faculty of Computer Science and Artificial Intelligence Shenzhen University of Advanced Technology (SUAT), China

  16. ReSGA: A Large Tail Risk Model for Learning Value-at-Risk and Expected Shortfall

    Jun 3, 2026Yichi Zhang, Ke Zhu, Zhoufan ZhuConditional-Value-At-RiskLong-Tailed Distribution

    Department of Statistics and Actuarial Science, The University of Hong Kong, Hong Kong · Wang Yanan Institute for Studies in Economics, Xiamen University, China · Department of Finance, School of Economics, Xiamen University, China

  17. Dynamics of Stochastic Momentum with Sparse Updates in High Dimensions

    May 27, 2026Katie Everett, Elliot PaquetteMomentum Stochastic Gradient DescentMomentum

    Google DeepMind & MIT · McGill University & Mila

  18. MONA: Muon Optimizer with Nesterov Acceleration for Scalable Language Model Training

    May 26, 2026Jiacheng Li, Jianchao Tan, Hongtao Xu +5MuonMomentum Stochastic Gradient Descent

    Meituan, Beijing, China

  19. EMA-Nesterov: Stabilizing Nesterov's Lookahead for Accelerated Deep Learning Optimization

    May 25, 2026Chung-Yiu Yau, Dawei Li, Athanasios Glentis +3Momentum Stochastic Gradient DescentNeural Network Optimization

    University of Minnesota · 2Amazon AGI · 3The Chinese University of Hong Kong

  20. Ada2MS: A Hybrid Optimization Algorithm Based on Exponential Mixing of Elementwise and Global Second-Moment Estimates

    May 19, 2026Meng Zhu, Quan Xiao, Weidong MinAdamMomentum Stochastic Gradient Descent

    School of Information Management and Mathematics, Jiangxi University of Finance and Economics, Nanchang 330032, China · School of Mathematics and Computer Science, Nanchang University, Nanchang 330031, China · Institute of Metaverse, Nanchang University, Nanchang 330031, China +1

  21. MiMuon: Mixed Muon Optimizer with Improved Generalization for Large Models

    May 19, 2026Feihu Huang, Yuning Luo, Songcan ChenMuonMomentum Stochastic Gradient Descent

    College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing, China · MIIT Key Laboratory of Pattern Analysis and Machine Intelligence, Nanjing, China · College of Design and Engineering, National University of Singapore, Singapore

  22. Perfect Parallelization in Mini-Batch SGD with Classical Momentum Acceleration

    May 18, 2026Sachin Garg, Michał DerezińskiMomentum Stochastic Gradient DescentStochastic Gradient Descent

    University of Michigan

  23. Ringmaster LMO: Asynchronous Linear Minimization Oracle Momentum Method

    May 18, 2026Abdurakhmon Sadiev, Artavazd Maranjyan, Ivan Ilin +1Momentum Stochastic Gradient DescentMuon

    KAUST

  24. Stochastic Compositional Optimization via Hybrid Momentum Frank--Wolfe

    May 14, 2026El Mahdi ChaytiStochastic Convex OptimizationStochastic Optimization

    Machine Learning and Optimization Laboratory (MLO) EPFL, Switzerland

  25. Cosine-Gated Adam-Decay: Drop-In Staleness-Aware Outer Optimization for Decoupled DiLoCo

    May 9, 2026Vatsal Shah, Jiahao SunAdamMomentum Stochastic Gradient Descent

  26. A Unified Lyapunov-IQC Framework for Uniform Stability of Smooth Quadratic First-Order Accelerated Optimizers

    May 8, 2026Don Li, Dacian DaescuLyapunov FunctionMomentum Stochastic Gradient Descent

    Department of Mathematics & Statistics, Portland State University, Portland, OR 97201, USA

  27. Muon with Nesterov Momentum: Heavy-Tailed Noise and (Randomized) Inexact Polar Decomposition

    May 7, 2026Sayantan Choudhury, Xiaoran Cheng, Martin Takáč +2Momentum Stochastic Gradient DescentMuon

    MBZUAI · Penn State University · Georgia Institute of Technology +1

  28. MDN: Parallelizing Stepwise Momentum for Delta Linear Attention

    May 7, 2026Yulong Huang, Xiang Liu, Hongxiang Huang +5Kimi Delta AttentionLinear Attention

    The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China

  29. SignMuon: Communication-Efficient Distributed Muon Optimization

    May 4, 2026Neel Mishra, Kushagara Trivedi, Pawan KumarMuonDecentralized Optimization

    1Microsoft · 2IIIT Hyderabad

  30. Randomized Subspace Nesterov Accelerated Gradient

    May 1, 2026Gaku Omiya, Pierre-Louis Poirion, Akiko TakedaMomentum Stochastic Gradient DescentFirst Order Oracle Complexity

    Department of Mathematical Informatics, The University of Tokyo, Tokyo, Japan · Center for Advanced Intelligence Project, RIKEN, Tokyo, Japan

  31. Inference of Online Newton Methods with Nesterov's Accelerated Sketching

    Apr 25, 2026Haoxuan Wang, Xinchen Du, Sen NaNewtonMomentum Stochastic Gradient Descent

    School of Industrial and Systems Engineering, Georgia Institute of Technology.

  32. Adaptive Momentum and Nonlinear Damping for Neural Network Training

    Jan 30, 2026Aikaterini Karoni, Rajit Rajpal, Benedict Leimkuhler +1Momentum Stochastic Gradient DescentNeural Network Training

    School of Mathematics University of Bristol · School of Mathematics University of Edinburgh · CERMICS, CNRS, École des Ponts, Institut Polytechnique de Paris; Inria Paris, France

  33. A Nesterov-Accelerated Byzantine-Robust Federated Learning

    Nov 4, 2025Lihan Xu, Xiaoyi Fan, Gang Wang +3Byzantine AttacksFederated Learning

    School of Cyber Science and Engineering, Xi’an Jiaotong University, Xi’an 710049, China · Artificial Intelligence Research Institute and Guangdong-Hong Kong-Macao Joint Laboratory for Emotional Intelligence and Pervasive Computing, Shenzhen MSU-BIT University, Shenzhen 518172, China · State Key Laboratory of Autonomous Intelligent Unmanned Systems, Beijing Institute of Technology, Beijing 100081, China

  34. Better Convergence Guarantees for Sign-Based Momentum Methods

    Jul 16, 2025Wei Jiang, Dingzhi Yu, Sifan Yang +3Momentum Stochastic Gradient DescentConvergence

    National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China · School of Artificial Intelligence, Nanjing University, Nanjing, China

  35. On the Effectiveness of the z-Transform Method in Quadratic Optimization

    Jul 4, 2025Francis BachMomentum Stochastic Gradient DescentConvergence

    SIERRA · Inria - Ecole Normale Sup´erieure · PSL Research University

  36. Scaling Textual Gradients via Sampling-Based Momentum

    May 31, 2025Zixin Ding, Junyuan Hong, Zhan Shi +6Large Language Model TrainingAutomatic Prompt Optimization

    University of Chicago · University of Texas at Austin · Santa Clara University +3

  37. How Learning Dynamics Drive Adversarially Robust Generalization?

    Oct 10, 2024Yuelin Xu, Xiao ZhangAdversarial TrainingGeneralization Bounds

    CISPA Helmholtz Center for Information Security, Saarbrücken, Germany

  38. Accelerated Stochastic Min-Max Optimization Based on Bias-corrected Momentum

    Jun 18, 2024Haoyuan Cai, Sulaiman A. Alghunaim, Ali H. SayedStochastic Convex OptimizationConvex Optimization

    Ecole Polytechnique Fédérale de Lausanne, Switzerland · Kuwait University, Kuwait