Muon Optimizer

Follow this topic Ask about this topic

Recent momentum

-67%

3 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-21

2 new papers

A weekly snapshot of new work published in Muon Optimizer.

62 papers

Latest in Muon Optimizer

Browse all topics →

Design of the IBM Granite 5.0 TurboCTC ASR Model
Sep 17, 2026Brian Kingsbury, George Saon, Masayuki Suzuki +5Muon Optimizer
Derivative-Free Structured Updates for Muon
Sep 15, 2026Pengcheng XieMuon Optimizer Gradient-Based Optimization
Beyond the Matrix Sign: Quadratic Spectral Descent
Sep 7, 2026Qiaozhe Zhang, Jun Sun, Yingzhuang LiuMuon Optimizer Muon
Federated Compositional Muon Optimizer for Matrix-Wise Models
Aug 13, 2026Wang Yan, Feihu HuangMuon Optimizer Muon
MOON: Multi-Objective OrthoNormalized Updates for Multitask Learning
Aug 12, 2026Shiji Zhou, Kunlin Lyu, Lei Zhang +2Muon Optimizer Multi--Task Learning
Dion3: Full-Stack Orthogonal Updates
Aug 12, 2026Noah Amsel, Jack Zhang, Kwangjun Ahn +5Muon Optimizer Newton
Muon on the Stiefel Manifold Admits an Exact Closed-Form Update
Aug 6, 2026Mikhail Solonko, Molozhavenko Alexander, Maxim RakhubaMuon Optimizer Riemannian Optimization
On MUON optimization: From non-convergence to an error analysis with Polar Express and the Newton-Schulz polynomial from implementations
Aug 5, 2026Thang Do, Steffen Dereich, Arnulf JentzenMuon Optimizer Stochastic Gradient Descent
Muon Meets Mamba: Spectral Optimization for State Space Models
Aug 4, 2026Arslan Battalov, Karim Kramin, Alexander Markotenko +1Muon Optimizer Muon
Joint Affine Spectral Shaping: Coupling Weight and Bias Updates Beyond Weight-Only Muon
Aug 4, 2026Gongyue Zhang, Honghai LiuMuon Optimizer Affine
Sign compression for Muon: SignMuon, MuonSign, and the Limits of Error Feedback
Jul 31, 2026Maria Smirnova, Alexey KravatskiyMuon Optimizer Muon
Sharpness-Aware Minimization and Muon: Robustness under the Spectral Norm
Jul 28, 2026Wenzhi Zhong, Edward Milsom, Michael MurraySharpness-Aware Minimization Muon Optimizer
Scale Weight Decay and Train Better
Jul 26, 2026Anuj ApteLearning Rates Model Weights
Muse: Representation Geometry of Muon Beyond Normalized Momentum
Jul 16, 2026Da Chang, Qiankun Shi, Lvgang Zhang +4Muon Optimizer Muon
Reassessing Muon for Matrix Factorization
Jul 14, 2026Ali Parviz, Gal Mishne, Alex CloningerMuon Optimizer Muon
Muon as a Residual Connection
Jul 1, 2026Hao HuangMuon Optimizer Muon
Muon learns balanced solutions in matrix factorization without slow saddle-to-saddle dynamics
Jun 29, 2026Mark Rhee, Jamie Simon, Dhruva KarkadaMuon Optimizer Matrix Factorization
Aurora: A Leverage-Aware Spectral Optimizer
Jun 26, 2026Alec Dewulf, Dhruv Pai, Li Yang +2Muon Optimizer Multilayer Perceptrons
Hierarchical Muon: Tiled Newton-Schulz Updates for Efficient Muon Optimization
Jun 25, 2026Ziyuan Tang, Tianshi Xu, Yousef Saad +1Muon Optimizer Muon
DMuon: Efficient Distributed Muon Training with Near-Adam Overhead
Jun 25, 2026Vincent Chen, Starrick Liu, Regis Cheng +8Muon Optimizer Muon
Tensorion: A Tensor-Aware Generalization of the Muon Optimizer
Jun 24, 2026Vladimir Bogachev, Vladimir Aletov, Alexander Molozhavenko +2Muon Optimizer Adam
Muown Implicitly Performs Angular Step-size Decay
Jun 22, 2026Florian Hübler, Kai Lion, Antonio Orvieto +1Muon Optimizer Muon
Towards Understanding the Power and Limits of the Muon Optimizer: A River-Valley Perspective
Jun 19, 2026Tianqi Shen, Jinji Yang, Runze Shi +3Muon Optimizer Muon
CacheMuon: Using Temporal Preconditioning To Approximate Polar Factor
Jun 15, 2026Bishnu Dev, Sushil Bohara, Martin Takáč +1Muon Optimizer Muon
Schattor: Schatten-family methods for deep learning optimization
Jun 14, 2026Bohao Ma, Junyu Zhang, Chuan HeStochastic Optimization Neural Network Optimization
Free Heavy-Tailed Lunch for Muon: A Theoretical Justification of Empirical Success
Jun 12, 2026Florian Hübler, Thomas Pethick, Suvrit SraSpectral Norm Muon Optimizer
Muon $^p$ : Muon with Fractional Spectral Powers
Jun 11, 2026Yihe Dong, Will SawinMuon Optimizer Muon
LoRA-Muon: Spectral Steepest Descent on the Low-Rank Manifold
Jun 11, 2026Franz Louis Cesista, Katherine Crowson, Cédric Simal +1Low-Rank Adaptation Low-Rank Factorization
OptMuon: Closed-Loop Orthogonalized Momentum Methods for Stochastic Optimization with Zero-Noise Optimality
Jun 7, 2026Ganzhao YuanMuon Optimizer Stochastic Smooth Convex Optimization
PC Layer: Polynomial Weight Preconditioning for Improving LLM Pre-Training
Jun 4, 2026Senmiao Wang, Tiantian Fang, Haoran Zhang +4Large Language Model Pretraining Spectral Preconditioning
Denoise First, Orthogonalize Later: Understanding Momentum in Muon via Spectral Filtering
Jun 2, 2026Xianliang Li, Zihan Zhang, Weiyang Liu +1Muon Optimizer Muon
Spectral Scaling Laws of Muon
Jun 2, 2026Gagik Magakyan, Pablo Parrilo, Asuman OzdaglarMuon Optimizer Muon
How Much Orthogonalization Does Muon Need?
May 29, 2026Hua HuangMuon Optimizer Muon
Convergence of Spectral Descent for Non-smooth Optimization
May 26, 2026Yixuan Yang, Yuqing He, Song LiMuon Optimizer Convex Optimization
When Muon Optimizer Meets Adversarial Training: A Theoretical and Empirical Study
May 26, 2026Jun Yan, Weiquan Huang, Jiankai Zuo +4Muon Optimizer Adversarial Training
MONA: Muon Optimizer with Nesterov Acceleration for Scalable Language Model Training
May 26, 2026Jiacheng Li, Jianchao Tan, Hongtao Xu +5Muon Optimizer Curvature-Aware Quadratic Banks
MuCon: Clipped Muon Updates for LLM Training
May 26, 2026Albert YiMuon Optimizer Muon
Move on Muon : A Hamiltonian probability gradient flow perspective of Muon optimizer
May 22, 2026Aratrika Mustafi, Soumya Mukherjee, Bharath K. SriperumbudurMuon Optimizer Muon
LionMuon: Alternating Spectral and Sign Descent for Efficient Training
May 19, 2026Arman Bolatov, Artem Riabinin, Nikita Kornilov +4Muon Optimizer Muon
From SGD to Muon: Adaptive Optimization via Schatten-p Norms
May 19, 2026Thomas Massena, Corentin Friedrich, Mathieu SerrurierMuon Optimizer Optimizer Design
MiMuon: Mixed Muon Optimizer with Improved Generalization for Large Models
May 19, 2026Feihu Huang, Yuning Luo, Songcan ChenMuon Optimizer Muon
Distance-Aware Muon: Adaptive Step Scaling for Normalized Optimization
May 18, 2026Yury Demidovich, Abhishek Chakraborty, Grigory Malinovsky +2Muon Optimizer Muon
AMO: Adaptive Muon Orthogonalization
May 18, 2026Xinlin Zhuang, Panyi Ouyang, Yichen Li +7Muon Optimizer Adam
Spectral Flattening Is All Muon Needs: How Orthogonalization Controls Learning Rate and Convergence
May 13, 2026Tien-Phat Nguyen, Truong Nguyen, Minh-Phuc Truong +3Muon Optimizer Muon
MuonQ: Enhancing Low-Bit Muon Quantization via Directional Fidelity Optimization
May 12, 2026Yupeng Su, Ruijie Zhang, Ziyue Liu +2Muon Optimizer Quantization
Muon is Not That Special: Random or Inverted Spectra Work Just as Well
May 11, 2026Zakhar Shumaylov, Nathaël Da Costa, Peter Zaika +6Muon Optimizer Muon
Muown: Row-Norm Control for Muon Optimization
May 11, 2026Kai Lion, Florian Hübler, Bingcong Li +2Muon Optimizer Muon
Phases of Muon: When Muon Eclipses SignSGD
May 10, 2026Elliot Paquette, Noah Marshall, Lucas Benigni +3Muon Optimizer Muon
Dimension-Free Saddle-Point Escape in Muon
May 10, 2026Yanlin Long, Yufei Gu, Zeke XieMuon Optimizer Muon
Intrinsic Muon: Spectral Optimization on Riemannian Matrix Manifolds
May 10, 2026Yibang Li, Bihari Lal Pandey, Ravi Sah +4Muon Optimizer Muon
Muon-OGD: Muon-based Spectral Orthogonal Gradient Projection for LLM Continual Learning
May 9, 2026Binghang Lu, Zheyuan Deng, Runyu Zhang +6Muon Optimizer Efficient Continual Learning
OrScale: Orthogonalised Optimization with Layer-Wise Trust-Ratio Scaling
May 8, 2026Yuxuan Lou, Yang YouMuon Optimizer Model Weights
Muon with Nesterov Momentum: Heavy-Tailed Noise and (Randomized) Inexact Polar Decomposition
May 7, 2026Sayantan Choudhury, Xiaoran Cheng, Martin Takáč +2Muon Optimizer Neural Network Optimization
SignMuon: Communication-Efficient Distributed Muon Optimization
May 4, 2026Neel Mishra, Kushagara Trivedi, Pawan KumarMuon Optimizer Muon
Benchmarking Optimizers for MLPs in Tabular Deep Learning
Apr 16, 2026Yury Gorishniy, Ivan Rubachev, Dmitrii Feoktistov +1Tabular Learning Optimizer Design
To Use or not to Use Muon: How Simplicity Bias in Optimizers Matters
Feb 28, 2026Sara Dragutinović, Yedi Zhang, Rajesh RanganathMuon Optimizer Muon
Variance-Adaptive Muon: Pre-Orthogonalization Variance Modulation for Efficient Language Model Pretraining
Jan 21, 2026Jingru Li, Yibo Fan, Huan LiMuon Optimizer Muon
Ky Fan Norms and Beyond: Dual Norms and Combinations for Matrix Optimization
Dec 10, 2025Alexey Kravatskiy, Ivan Kozyrev, Nikolai Kozlov +3Muon Optimizer Spectral Norm
Turbo-Muon: Almost-Orthogonal Pre-Conditioning for Fast Muon Updates
Dec 4, 2025Thibaut Boissin, Thomas Massena, Franck Mamalet +1Muon Optimizer Muon
Low-rank Orthogonalization for Large-scale Matrix Optimization with Applications to Foundation Model Training
Sep 15, 2025Chuan He, Zhanwang Deng, Zhaosong LuLow-Rank Factorization Orthogonality

Sep 17, 2026 · cs.CLJ/K move · Enter open · S save

Design of the IBM Granite 5.0 TurboCTC ASR Model

Brian Kingsbury, George Saon, Masayuki Suzuki, Hong-Kwang J. Kuo+4

IBM Research

We describe the architecture, training methodology and inference speedups of Granite 5.0 Turbo CTC, a 470 million parameter encoder-only model with an excellent speed-accuracy tradeoff. The architecture uses pyramidal temporal subsampling within Conformer blocks using strided depthwise convolutions, block-diagonal (chunk-wise) self-attention, and conditioning on intermediate predictions from the middle layer. Training highlights are the use of only publicly available data, the novel use of a Muon optimizer, and balanced data sampling. Inference speedups include replacing 1 x 1 convolutions with linear layers and optimizing the attention computation in the Conformer blocks. Collectively, these result in a model that is on the speed-accuracy Pareto frontier of the Open ASR leaderboard for English short-form ASR while being twice as fast as the fastest competitor. The model can be used under a permissive license and downloaded from https://huggingface.co/ibm-granite/granite-speech-5.0-470m-turboctc.

Muon Optimizer

Open paper Ask about it PDF