Muon

Follow this topic Ask about this topic

Recent momentum

-55%

5 papers in the last 28 days · 0.1% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-21

2 new papers

A weekly snapshot of new work published in Muon.

Period ending 2026-09-14

4 new papers

A weekly snapshot of new work published in Muon.

80 papers

Latest in Muon

Open your feed →

Derivative-Free Structured Updates for Muon
Sep 15, 2026Pengcheng XieMuon Optimizer Gradient-Based Optimization
Fast BIB simulation at a future Muon Collider with generative machine learning
Sep 14, 2026Radha Mastandrea, Shiyu Peng, Benjamin Rosser +1Muon Generative Models
Muon-C: Operator-Aligned Muon for Convolutional Kernels
Sep 9, 2026Jiaxin Qing, Lexin LiConvolutional Muon
Equivariance Breaks the Learning Rate
Sep 8, 2026Andrei Manolache, Mathias NiepertAdam Equivariant Neural Networks
Beyond the Matrix Sign: Quadratic Spectral Descent
Sep 7, 2026Qiaozhe Zhang, Jun Sun, Yingzhuang LiuMuon Optimizer Muon
Toward a First-Principles Update Geometry for the Language-Model Head
Aug 23, 2026Aditya Somasundaram, Charles Guille-Escuret, Alexander Moreno +2Spectral Norm Multi-Head Self-Attention
Federated Compositional Muon Optimizer for Matrix-Wise Models
Aug 13, 2026Wang Yan, Feihu HuangMuon Optimizer Muon
Second-Order Muon Done Right: A Principled Marriage of Spectral Geometry and Curvature
Aug 10, 2026Tong CheMuon Newton
Post-Grokking Collapse at the Representation-Readout Interface in Muon-Trained Transformers
Aug 7, 2026Ali Janati, Kaoutar El Maghraoui, Andrei Kanavalau +1Muon Grokking
Muon on the Stiefel Manifold Admits an Exact Closed-Form Update
Aug 6, 2026Mikhail Solonko, Molozhavenko Alexander, Maxim RakhubaMuon Optimizer Riemannian Optimization
MALT: Lightweight Curvature-Aware Muon via Diagonal Preconditioning
Aug 5, 2026Tongle Wu, Huanyu Dong, Ying Sun +1Curvature-Aware Quadratic Banks Spectral Preconditioning
Muon Meets Mamba: Spectral Optimization for State Space Models
Aug 4, 2026Arslan Battalov, Karim Kramin, Alexander Markotenko +1Muon Optimizer Muon
Joint Affine Spectral Shaping: Coupling Weight and Bias Updates Beyond Weight-Only Muon
Aug 4, 2026Gongyue Zhang, Honghai LiuMuon Optimizer Affine
CMuon: Accelerating and Stabilizing Diffusion Transformer Training via Chunked Momentum Orthogonalization
Aug 3, 2026Chuyan Chen, Peng Sun, Kun YuanMultimodal Diffusion Transformers Muon
Sign compression for Muon: SignMuon, MuonSign, and the Limits of Error Feedback
Jul 31, 2026Maria Smirnova, Alexey KravatskiyMuon Optimizer Muon
Sharpness-Aware Minimization and Muon: Robustness under the Spectral Norm
Jul 28, 2026Wenzhi Zhong, Edward Milsom, Michael MurraySharpness-Aware Minimization Muon Optimizer
An Isotropy-Preserving Spectral Cap for Muon: Theory and Three Case Studies
Jul 22, 2026Jiachun LiMuon Model Weights
PoLoRA: A Preconditioned Orthogonalized LoRA Optimizer
Jul 20, 2026Nikhil Ghosh, Tetiana Parshakova, Robert M. GowerLow-Rank Adaptation Optimizer Design
When Does Muon Help Agentic Reinforcement Learning?
Jul 17, 2026Kai Ruan, Jinghao Lin, Zihe Huang +4Muon Joint Reinforcement Learning
Muse: Representation Geometry of Muon Beyond Normalized Momentum
Jul 16, 2026Da Chang, Qiankun Shi, Lvgang Zhang +4Muon Optimizer Muon
Reassessing Muon for Matrix Factorization
Jul 14, 2026Ali Parviz, Gal Mishne, Alex CloningerMuon Optimizer Muon
Efficient Long-Horizon Learning for Learned Optimization
Jul 7, 2026Xiaolong Huang, Benjamin Thérien, James Harrison +1Meta-Learning Model Training
Beyond Adam: SOAP and Muon for Faster, Label-Efficient Training of Machine Learning Interatomic Potentials
Jul 2, 2026Gil Harari, Yoel Zimmermann, Ola Tangen Kulseng +4Interatomic Potentials Adam
Muon as a Residual Connection
Jul 1, 2026Hao HuangMuon Optimizer Muon
Muon learns balanced solutions in matrix factorization without slow saddle-to-saddle dynamics
Jun 29, 2026Mark Rhee, Jamie Simon, Dhruva KarkadaMuon Optimizer Matrix Factorization
Hierarchical Muon: Tiled Newton-Schulz Updates for Efficient Muon Optimization
Jun 25, 2026Ziyuan Tang, Tianshi Xu, Yousef Saad +1Muon Optimizer Muon
DMuon: Efficient Distributed Muon Training with Near-Adam Overhead
Jun 25, 2026Vincent Chen, Starrick Liu, Regis Cheng +8Muon Optimizer Muon
Tensorion: A Tensor-Aware Generalization of the Muon Optimizer
Jun 24, 2026Vladimir Bogachev, Vladimir Aletov, Alexander Molozhavenko +2Muon Optimizer Adam
Muown Implicitly Performs Angular Step-size Decay
Jun 22, 2026Florian Hübler, Kai Lion, Antonio Orvieto +1Muon Optimizer Muon
Towards Understanding the Power and Limits of the Muon Optimizer: A River-Valley Perspective
Jun 19, 2026Tianqi Shen, Jinji Yang, Runze Shi +3Muon Optimizer Muon
Fantastic Pretraining Optimizers and Where to Find Them II: Hyperball Optimization
Jun 15, 2026Kaiyue Wen, Xingyu Dang, Kaifeng Lyu +2Large Language Model Pretraining Model Weights
CacheMuon: Using Temporal Preconditioning To Approximate Polar Factor
Jun 15, 2026Bishnu Dev, Sushil Bohara, Martin Takáč +1Muon Optimizer Muon
When to use what Schatten- $p$ norm in deep learning?When to use what Schatten-p norm in deep learning?
Jun 13, 2026Thomas PethickSpectral Norm Muon
Free Heavy-Tailed Lunch for Muon: A Theoretical Justification of Empirical Success
Jun 12, 2026Florian Hübler, Thomas Pethick, Suvrit SraSpectral Norm Muon Optimizer
Muon $^p$ : Muon with Fractional Spectral PowersMuon^p: Muon with Fractional Spectral Powers
Jun 11, 2026Yihe Dong, Will SawinMuon Optimizer Muon
LoRA-Muon: Spectral Steepest Descent on the Low-Rank Manifold
Jun 11, 2026Franz Louis Cesista, Katherine Crowson, Cédric Simal +1Low-Rank Adaptation Low-Rank Factorization
Muon Learns More Robust and Transferable Features than Adam
Jun 8, 2026Tianyu Ruan, Fengzhuo Zhang, Shuche Wang +1Adam Feature Learning
The Spectral Dynamics and Noise Geometry of Muon
Jun 7, 2026Pierfrancesco Beneventano, Mahmoud Abdelmoneum, Tomaso PoggioMuon Spectrum
Why Muon Outperforms Adam: A Curvature Perspective
Jun 3, 2026Shuche Wang, Fengzhuo Zhang, Jiaxiang Li +2Adam Muon
Denoise First, Orthogonalize Later: Understanding Momentum in Muon via Spectral Filtering
Jun 2, 2026Xianliang Li, Zihan Zhang, Weiyang Liu +1Muon Optimizer Muon
Spectral Scaling Laws of Muon
Jun 2, 2026Gagik Magakyan, Pablo Parrilo, Asuman OzdaglarMuon Optimizer Muon
How Much Orthogonalization Does Muon Need?
May 29, 2026Hua HuangMuon Optimizer Muon
How the Optimizer Shapes Learned Solutions in Equivariant Neural Networks
May 26, 2026Teodor-Mihai Stupariu, Andrei ManolacheEquivariant Neural Networks Neural Network Optimization
When Muon Optimizer Meets Adversarial Training: A Theoretical and Empirical Study
May 26, 2026Jun Yan, Weiquan Huang, Jiankai Zuo +4Muon Optimizer Adversarial Training
MuCon: Clipped Muon Updates for LLM Training
May 26, 2026Albert YiMuon Optimizer Muon
Muon in Vision Transformers: Optimizer-Recipe Interactions and Gradient Spectra
May 23, 2026Ben S. Southworth, Shuai Jiang, Daniel McBride +2Vision Transformer Muon
Move on Muon : A Hamiltonian probability gradient flow perspective of Muon optimizer
May 22, 2026Aratrika Mustafi, Soumya Mukherjee, Bharath K. SriperumbudurMuon Optimizer Muon
AMUSE: Anytime Muon with Stable Gradient Evaluation
May 21, 2026Jueun Kim, Baekrok Shin, Jihun Yun +3Muon Federated Averaging
LionMuon: Alternating Spectral and Sign Descent for Efficient Training
May 19, 2026Arman Bolatov, Artem Riabinin, Nikita Kornilov +4Muon Optimizer Muon
From SGD to Muon: Adaptive Optimization via Schatten-p Norms
May 19, 2026Thomas Massena, Corentin Friedrich, Mathieu SerrurierMuon Optimizer Optimizer Design
MiMuon: Mixed Muon Optimizer with Improved Generalization for Large Models
May 19, 2026Feihu Huang, Yuning Luo, Songcan ChenMuon Optimizer Muon
Rethinking Muon Beyond Pretraining: Spectral Failures and High-Pass Remedies for VLA and RLVR
May 19, 2026Chongyu Fan, Gaowen Liu, Mingyi Hong +2Muon Reinforcement Learning With Verifiable Reward
Distance-Aware Muon: Adaptive Step Scaling for Normalized Optimization
May 18, 2026Yury Demidovich, Abhishek Chakraborty, Grigory Malinovsky +2Muon Optimizer Muon
Ringmaster LMO: Asynchronous Linear Minimization Oracle Momentum Method
May 18, 2026Abdurakhmon Sadiev, Artavazd Maranjyan, Ivan Ilin +1Adam Zeroth-Order Optimization
AMO: Adaptive Muon Orthogonalization
May 18, 2026Xinlin Zhuang, Panyi Ouyang, Yichen Li +7Muon Optimizer Adam
DynMuon: A Dynamic Spectral Shaping View of Muon
May 16, 2026Fangzhou Wu, Rikhav Shah, Sandeep Silwal +1Muon Loss Function
Spectral Flattening Is All Muon Needs: How Orthogonalization Controls Learning Rate and Convergence
May 13, 2026Tien-Phat Nguyen, Truong Nguyen, Minh-Phuc Truong +3Muon Optimizer Muon
Constrained Stochastic Spectral Preconditioning Converges for Nonconvex Objectives
May 12, 2026Konstantinos Oikonomidis, Jan Quan, Kimon Antonakopoulos +3Spectral Preconditioning Convex Optimization
MuonQ: Enhancing Low-Bit Muon Quantization via Directional Fidelity Optimization
May 12, 2026Yupeng Su, Ruijie Zhang, Ziyue Liu +2Muon Optimizer Quantization
Muon is Not That Special: Random or Inverted Spectra Work Just as Well
May 11, 2026Zakhar Shumaylov, Nathaël Da Costa, Peter Zaika +6Muon Optimizer Muon

Sep 15, 2026 · math.OCJ/K move · Enter open · S save

Derivative-Free Structured Updates for Muon

Pengcheng Xie

Muon updates matrix-valued neural-network parameters by orthogonalizing a gradient-based momentum matrix. Its reliance on derivatives limits its use when gradients are unavailable or unreliable. We develop a derivative-free framework that constructs Muon-style updates from structured finite differences. Four variants are considered: full entrywise recovery, random low-rank surrogates, basis-aligned rank-one probing, and direct structured search. Exhaustive basis-aligned probing is equivalent, up to positive scaling before ideal polar orthogonalization, to coordinate finite differences. Matrix-regression experiments show that random rank-one probing can reduce the number of function evaluations substantially, at the cost of less accurate updates. Controlled noisy-gradient experiments on regression and a neural network illustrate when accurate function values can compensate for an unreliable gradient oracle. A small CartPole study further examines orthogonal rank-one probes under a fixed episode budget. These results support structured probing as a practical option for selected black-box problems; they do not establish a general convergence guarantee or an advantage over accurate, inexpensive gradients.

Muon Optimizer Gradient-Based Optimization Muon Newton

Open paper Ask about it PDF