Overview

Muon is a matrix-aware optimizer that accumulates momentum for weight matrices and approximately orthogonalizes each update, often with Newton–Schulz iterations, rather than scaling each parameter independently as Adam does. Research explores variants with curvature or variance preconditioning, different spectral weighting and normalization, and cheaper or distributed orthogonalization. Studies test Muon in language-model training, vision, recommendation, diffusion, and scientific machine learning, while theory examines convergence, noise, implicit bias, and generalization. A central open question is when its spectral updates improve learning enough to justify their cost: results depend on architecture, batch size, and training phase, and practical approximations can change both behavior and guarantees.

Latest papers

CardsList

159 papers26 papers in the last four weeks, up 13% on the four weeks before. 0.2% of all new papers.

  1. Does Muon Need Fine-Grained Spectral Shaping?

    Oct 5, 2026Meher Chaitanya, Tianyi Zhou, Aristides GionisDeep Learning OptimizationMuon Optimizer

  2. Muon Is Theoretically Wrong For Convolutions, But Empirically Effective

    Oct 5, 2026Thibaut Boissin, Thomas Massena, Mathieu Serrurier +1Newton-Schulz IterationNeural Network Optimization

  3. Training-Loss Guarantees for Muon with Finite-Step Newton--Schulz Orthogonalization

    Oct 2, 2026Amartya Roy, Souvik ChakrabortyMuon OptimizerNeural Network Training Dynamics

  4. Muon Learns Facts Better: Understanding the Role of Spectral Orthogonalization

    Oct 2, 2026Xuheng Li, Qiwei Di, Yuan Cao +1Neural Network Training DynamicsMuon Optimizer

  5. Inner Momentum for Differentially Private Muon

    Oct 2, 2026Bishnu Bhusal, Minh Vu, Ben Southworth +3Muon OptimizerPrivacy-Preserving Language Models

  6. MuonIO: Principled Norm-Aware Descent for Embedding Tables and Language Model Heads

    Oct 2, 2026Linkai Ma, Xinyu Luo, Mengbo Wang +3Muon OptimizerLanguage Model Pretraining

  7. AF-Muon: An AdamW-Free Muon Optimizer for Tied-Embedding Models

    Oct 1, 2026Arash Lagzian, Paniz Halvachi, Junming Zhang +2Memory-Efficient OptimizationMuon Optimizer

  8. The Row Normalization Puzzle in Muon

    Sep 30, 2026Jiayu Zhang, Tianyi LinStochastic OptimizationOptimization Convergence Analysis

  9. Second-Moment Stochastic Approximation Methods

    Sep 29, 2026Tao Jiang, Lin XiaoStochastic ApproximationMuon Optimizer

  10. Can Representation Learning Decouple from Loss Minimization? Polar Updates Have an Answer

    Sep 28, 2026Akash KumarRepresentation LearningNeural Network Training Dynamics

  11. Muon Sublates the Edge of Stability in LLM Pretraining

    Sep 28, 2026Yanzhe Chen, Qifang Zhao, Xiaoxiao Xu +1Stochastic OptimizationLanguage Model Pretraining

  12. Orthogonal Witness Control for Muon Optimization via Sigmoid Spectral Reshaping

    Sep 27, 2026Dat Phi Van, Ngo Vu Minh, Tuc Nguyen +3Neural Network OptimizationSpectral Optimization

  13. Convergence of Practical Muon

    Sep 27, 2026Haonan Wang, Yu Wu, Minghui Liwang +2Muon OptimizerStochastic Optimization Convergence

  14. Muon Under Gradient Noise and the Limits of Orthogonalization Near Optima

    Sep 26, 2026Xiaohui XieMuon OptimizerGradient Orthogonalization

  15. Convergence guarantees for Muon: New parameter regimes and generalizations

    Sep 24, 2026Arthur C. B. de Oliveira, Dhruv D. Jatkar, Guilherme S. Vicinansa +1Muon OptimizerOptimization Convergence Analysis

  16. Muon Can Outperform Dedicated Continual Learning Methods

    Sep 21, 2026Sebastian George Sincari, Bogdan Alexandru Gheorghe, Antonio BarbalauContinual LearningMuon Optimizer

  17. Derivative-Free Structured Updates for Muon

    Sep 15, 2026Pengcheng XieBlack-Box OptimizationZeroth-Order Optimization

  18. Nonsmooth Optimization via Orthogonalized Momentum

    Sep 12, 2026Lexiao Lai, Tianyi Lin, Jiayu ZhangMuon OptimizerNonconvex Optimization

  19. Musec: MomentUm SpEctral Clipping for Stable Muon-type Training

    Sep 10, 2026Zhuanghua Liu, Menglian Wang, Luo LuoMuon OptimizerLLM Training

  20. Muon-C: Operator-Aligned Muon for Convolutional Kernels

    Sep 9, 2026Jiaxin Qing, Lexin LiNeural Network OptimizationMuon Optimizer

  21. Beyond the Matrix Sign: Quadratic Spectral Descent

    Sep 7, 2026Qiaozhe Zhang, Jun Sun, Yingzhuang LiuSign-Based OptimizationMuon Optimizer

  22. FedSubMuon: Communication-Efficient Federated LLM Fine-Tuning via Structured Subspace Muon

    Sep 5, 2026Shaolong Chen, Youming Tao, Shuzhen Chen +3Subspace OptimizationFederated Fine-Tuning

  23. When Muon Meets Task Interference: A Spectral Perspective on Continual Learning and Model Merging

    Aug 27, 2026Shangge Liu, Yuehan Yin, Yinghuan Shi +2Task InterferenceContinual Learning

  24. Muon with Finite Newton-Schulz: The Smoothing Benefit in Nonsmooth Nonconvex Optimization

    Aug 26, 2026Mingyi Li, Taira TsuchiyaMuon OptimizerNewton-Schulz Iteration

  25. Spectral Allocation: Why Muon Outperforms Adam, and How to Improve Muon

    Aug 26, 2026Xiaodong Wu, Wenyi Yu, Chao Zhang +1Deep Learning OptimizationMuon Optimizer