Muon

Follow this topic Ask about this topic

Recent momentum

-71%

4 papers in the last 28 days · 0.1% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-14

4 new papers

A weekly snapshot of new work published in Muon.

93 papers

Latest in Muon

Open your feed →

Derivative-Free Structured Updates for Muon
Sep 15, 2026Pengcheng XieMuon
Muon-C: Operator-Aligned Muon for Convolutional Kernels
Sep 9, 2026Jiaxin Qing, Lexin LiMuon Deep Unfolding Networks
Equivariance Breaks the Learning Rate
Sep 8, 2026Andrei Manolache, Mathias NiepertAdam Equivariance
Beyond the Matrix Sign: Quadratic Spectral Descent
Sep 7, 2026Qiaozhe Zhang, Jun Sun, Yingzhuang LiuSpectral Descent Muon
Toward a First-Principles Update Geometry for the Language-Model Head
Aug 23, 2026Aditya Somasundaram, Charles Guille-Escuret, Alexander Moreno +2Spectral Norm Multi-Head Attention
Federated Compositional Muon Optimizer for Matrix-Wise Models
Aug 13, 2026Wang Yan, Feihu HuangMuon Distributional Learning
Dion3: Full-Stack Orthogonal Updates
Aug 12, 2026Noah Amsel, Jack Zhang, Kwangjun Ahn +5Muon Newton
Second-Order Muon Done Right: A Principled Marriage of Spectral Geometry and Curvature
Aug 10, 2026Tong CheMuon Backward
Post-Grokking Collapse at the Representation-Readout Interface in Muon-Trained Transformers
Aug 7, 2026Ali Janati, Kaoutar El Maghraoui, Andrei Kanavalau +1Muon Grokking
Muon on the Stiefel Manifold Admits an Exact Closed-Form Update
Aug 6, 2026Mikhail Solonko, Molozhavenko Alexander, Maxim RakhubaMuon Manifolds
MALT: Lightweight Curvature-Aware Muon via Diagonal Preconditioning
Aug 5, 2026Tongle Wu, Huanyu Dong, Ying Sun +1Spectral Preconditioning Muon
Muon Meets Mamba: Spectral Optimization for State Space Models
Aug 4, 2026Arslan Battalov, Karim Kramin, Alexander Markotenko +1Muon State Space Models
Joint Affine Spectral Shaping: Coupling Weight and Bias Updates Beyond Weight-Only Muon
Aug 4, 2026Gongyue Zhang, Honghai LiuAffine Muon
CMuon: Accelerating and Stabilizing Diffusion Transformer Training via Chunked Momentum Orthogonalization
Aug 3, 2026Chuyan Chen, Peng Sun, Kun YuanDiffusion Transformers Muon
Sign compression for Muon: SignMuon, MuonSign, and the Limits of Error Feedback
Jul 31, 2026Maria Smirnova, Alexey KravatskiyMuon Sign
Sharpness-Aware Minimization and Muon: Robustness under the Spectral Norm
Jul 28, 2026Wenzhi Zhong, Edward Milsom, Michael MurraySharpness-Aware Minimization Spectral Norm
Scale Weight Decay and Train Better
Jul 26, 2026Anuj ApteWeight Decay Learning Rates
An Isotropy-Preserving Spectral Cap for Muon: Theory and Three Case Studies
Jul 22, 2026Jiachun LiMuon Spectral Norm
PoLoRA: A Preconditioned Orthogonalized LoRA Optimizer
Jul 20, 2026Nikhil Ghosh, Tetiana Parshakova, Robert M. GowerMuon Preconditioner
When Does Muon Help Agentic Reinforcement Learning?
Jul 17, 2026Kai Ruan, Jinghao Lin, Zihe Huang +4Muon Adam
Muse: Representation Geometry of Muon Beyond Normalized Momentum
Jul 16, 2026Da Chang, Qiankun Shi, Lvgang Zhang +4Muon Representation Space
Mono-Z Dark Matter Search with Neural Spline Flows Using CMS Run 2015D Open Data
Jul 15, 2026Hitesh Rasineni, Bhavishya ChebroluCosmology Muon
Reassessing Muon for Matrix Factorization
Jul 14, 2026Ali Parviz, Gal Mishne, Alex CloningerMuon Optimizer Design
Efficient Long-Horizon Learning for Learned Optimization
Jul 7, 2026Xiaolong Huang, Benjamin Thérien, James Harrison +1Meta-Learning Muon
Beyond Adam: SOAP and Muon for Faster, Label-Efficient Training of Machine Learning Interatomic Potentials
Jul 2, 2026Gil Harari, Yoel Zimmermann, Ola Tangen Kulseng +4Interatomic Potentials Adam
Muon as a Residual Connection
Jul 1, 2026Hao HuangMuon Residual Connections
MNAR- $k$ -means: A $k$ -means Clustering for Data Missing Not at Random with Magnitude-Decaying ProbabilityMNAR-k-means: A k-means Clustering for Data Missing Not at Random with Magnitude-Decaying Probability
Jun 30, 2026Xin GuanK$-Means Imputation
Muon learns balanced solutions in matrix factorization without slow saddle-to-saddle dynamics
Jun 29, 2026Mark Rhee, Jamie Simon, Dhruva KarkadaMuon Compact Matrix Factorization
Aurora: A Leverage-Aware Spectral Optimizer
Jun 26, 2026Alec Dewulf, Dhruv Pai, Li Yang +2Muon Multilayer Perceptrons
Hierarchical Muon: Tiled Newton-Schulz Updates for Efficient Muon Optimization
Jun 25, 2026Ziyuan Tang, Tianshi Xu, Yousef Saad +1Muon Newton
DMuon: Efficient Distributed Muon Training with Near-Adam Overhead
Jun 25, 2026Vincent Chen, Starrick Liu, Regis Cheng +8Muon Single-Inference Workloads
Tensorion: A Tensor-Aware Generalization of the Muon Optimizer
Jun 24, 2026Vladimir Bogachev, Vladimir Aletov, Alexander Molozhavenko +2Tensor Adam
Muown Implicitly Performs Angular Step-size Decay
Jun 22, 2026Florian Hübler, Kai Lion, Antonio Orvieto +1Muon Reparameterization
Towards Understanding the Power and Limits of the Muon Optimizer: A River-Valley Perspective
Jun 19, 2026Tianqi Shen, Jinji Yang, Runze Shi +3Muon Optimizer Design
Fantastic Pretraining Optimizers and Where to Find Them II: Hyperball Optimization
Jun 15, 2026Kaiyue Wen, Xingyu Dang, Kaifeng Lyu +2Adaptive Optimizers Weight Decay
CacheMuon: Using Temporal Preconditioning To Approximate Polar Factor
Jun 15, 2026Bishnu Dev, Sushil Bohara, Martin Takáč +1Muon Spectral Preconditioning
Schattor: Schatten-family methods for deep learning optimization
Jun 14, 2026Bohao Ma, Junyu Zhang, Chuan HeStochastic Optimization Neural Network Optimization
When to use what Schatten- $p$ norm in deep learning?When to use what Schatten-p norm in deep learning?
Jun 13, 2026Thomas PethickSpectral Norm Muon
Free Heavy-Tailed Lunch for Muon: A Theoretical Justification of Empirical Success
Jun 12, 2026Florian Hübler, Thomas Pethick, Suvrit SraSpectral Norm Heavy-Tailed Noise
Muon $^p$ : Muon with Fractional Spectral PowersMuon^p: Muon with Fractional Spectral Powers
Jun 11, 2026Yihe Dong, Will SawinMuon Spectral Descent
LoRA-Muon: Spectral Steepest Descent on the Low-Rank Manifold
Jun 11, 2026Franz Louis Cesista, Katherine Crowson, Cédric Simal +1Low-Rank Factorization Muon
Agentic Hybrid RAG for Evidence-Grounded Muon Collider Analysis
Jun 9, 2026Ruobing Jiang, Dawei Fu, Cheng Jiang +6Retrieval-Augmented Generation Multi-Hop Question Answering Benchmarks
Muon Learns More Robust and Transferable Features than Adam
Jun 8, 2026Tianyu Ruan, Fengzhuo Zhang, Shuche Wang +1Adam Muon
OptMuon: Closed-Loop Orthogonalized Momentum Methods for Stochastic Optimization with Zero-Noise Optimality
Jun 7, 2026Ganzhao YuanMuon Stochastic Optimization
The Spectral Dynamics and Noise Geometry of Muon
Jun 7, 2026Pierfrancesco Beneventano, Mahmoud Abdelmoneum, Tomaso PoggioMuon Spectral Descent
PC Layer: Polynomial Weight Preconditioning for Improving LLM Pre-Training
Jun 4, 2026Senmiao Wang, Tiantian Fang, Haoran Zhang +4Large Language Model Post-Training Spectral Preconditioning
Why Muon Outperforms Adam: A Curvature Perspective
Jun 3, 2026Shuche Wang, Fengzhuo Zhang, Jiaxiang Li +2Adam Muon
Denoise First, Orthogonalize Later: Understanding Momentum in Muon via Spectral Filtering
Jun 2, 2026Xianliang Li, Zihan Zhang, Weiyang Liu +1Muon Momentum
Spectral Scaling Laws of Muon
Jun 2, 2026Gagik Magakyan, Pablo Parrilo, Asuman OzdaglarMuon Scaling Laws
How Much Orthogonalization Does Muon Need?
May 29, 2026Hua HuangMuon Orthogonality
How the Optimizer Shapes Learned Solutions in Equivariant Neural Networks
May 26, 2026Teodor-Mihai Stupariu, Andrei ManolacheConvex Loss Muon
Convergence of Spectral Descent for Non-smooth Optimization
May 26, 2026Yixuan Yang, Yuqing He, Song LiSpectral Descent Convex Optimization
When Muon Optimizer Meets Adversarial Training: A Theoretical and Empirical Study
May 26, 2026Jun Yan, Weiquan Huang, Jiankai Zuo +4Muon Stochastic Gradient Descent
MONA: Muon Optimizer with Nesterov Acceleration for Scalable Language Model Training
May 26, 2026Jiacheng Li, Jianchao Tan, Hongtao Xu +5Muon Nesterov
MuCon: Clipped Muon Updates for LLM Training
May 26, 2026Albert YiMuon Gradient Clipping
Muon in Vision Transformers: Optimizer-Recipe Interactions and Gradient Spectra
May 23, 2026Ben S. Southworth, Shuai Jiang, Daniel McBride +2Vision Transformer Muon
Complete-muE: Optimal Hyperparameter Transfer and Scaling for MoE Models
May 22, 2026Hongwu Peng, Ohiremen Dibua, Yuanjun Xiong +3Transformer Variant Parameter-Efficient Fine-Tuning Methods
Move on Muon : A Hamiltonian probability gradient flow perspective of Muon optimizer
May 22, 2026Aratrika Mustafi, Soumya Mukherjee, Bharath K. SriperumbudurMuon Wasserstein Gradient Flows
AMUSE: Anytime Muon with Stable Gradient Evaluation
May 21, 2026Jueun Kim, Baekrok Shin, Jihun Yun +3Muon Federated Averaging
LionMuon: Alternating Spectral and Sign Descent for Efficient Training
May 19, 2026Arman Bolatov, Artem Riabinin, Nikita Kornilov +4Muon Descent

Sep 15, 2026 · math.OCJ/K move · Enter open · S save

Derivative-Free Structured Updates for Muon

Pengcheng Xie

Muon updates matrix-valued neural-network parameters by orthogonalizing a gradient-based momentum matrix. Its reliance on derivatives limits its use when gradients are unavailable or unreliable. We develop a derivative-free framework that constructs Muon-style updates from structured finite differences. Four variants are considered: full entrywise recovery, random low-rank surrogates, basis-aligned rank-one probing, and direct structured search. Exhaustive basis-aligned probing is equivalent, up to positive scaling before ideal polar orthogonalization, to coordinate finite differences. Matrix-regression experiments show that random rank-one probing can reduce the number of function evaluations substantially, at the cost of less accurate updates. Controlled noisy-gradient experiments on regression and a neural network illustrate when accurate function values can compensate for an unreliable gradient oracle. A small CartPole study further examines orthogonal rank-one probes under a fixed episode budget. These results support structured probing as a practical option for selected black-box problems; they do not establish a general convergence guarantee or an advantage over accurate, inexpensive gradients.

Muon

Open paper Ask about it PDF