Overview

Muon is a matrix-aware optimizer that accumulates momentum for weight matrices and approximately orthogonalizes each update, often with Newton–Schulz iterations, rather than scaling each parameter independently as Adam does. Research explores variants with curvature or variance preconditioning, different spectral weighting and normalization, and cheaper or distributed orthogonalization. Studies test Muon in language-model training, vision, recommendation, diffusion, and scientific machine learning, while theory examines convergence, noise, implicit bias, and generalization. A central open question is when its spectral updates improve learning enough to justify their cost: results depend on architecture, batch size, and training phase, and practical approximations can change both behavior and guarantees.

Latest papers

CardsList

159 papers26 papers in the last four weeks, up 13% on the four weeks before. 0.2% of all new papers.

  1. MuonRec: Shifting the Optimizer Paradigm Beyond Adam in Scalable Generative Recommendation

    Feb 28, 2026Rong Shan, Aofan Yu, Bo Chen +7Recommender SystemsMuon Optimizer

  2. MUON+: Towards More Effective Muon via One Additional Normalization Step for LLM Pre-training

    Feb 25, 2026Ruijie Zhang, Yequan Zhao, Ziyue Liu +4Muon OptimizerDeep Learning Optimization

  3. Powering Up Zeroth-Order Training via Subspace Gradient Orthogonalization

    Feb 19, 2026Yicheng Lang, Changsheng Wang, Yihua Zhang +4Zeroth-Order OptimizationMemory-Efficient Fine-Tuning

  4. Adam Improves Muon: Adaptive Moment Estimation with Orthogonalized Momentum

    Feb 19, 2026Minxin Zhang, Yuxuan Liu, Hayden SchaefferDeep Learning OptimizationAdaptive Gradient Methods

  5. Muon with Spectral Guidance: Efficient Optimization for Scientific Machine Learning

    Feb 18, 2026Binghang Lu, Jiahao Zhang, Guang LinMuon OptimizerPhysics-Informed ML

  6. Ky Fan Norms and Beyond: Dual Norms and Combinations for Matrix Optimization

    Dec 10, 2025Alexey Kravatskiy, Ivan Kozyrev, Nikolai Kozlov +3LMO MethodsDeep Learning Optimization

  7. Turbo-Muon: Almost-Orthogonal Pre-Conditioning for Fast Muon Updates

    Dec 4, 2025Thibaut Boissin, Thomas Massena, Franck Mamalet +1Newton-Schulz IterationOrthogonal Optimization