Overview

Matrix optimization studies how to efficiently solve problems whose variables, constraints, or data have matrix structure, including low-rank factorization and completion, clustering, covariance estimation, and neural-network training. Common approaches include projected and nonconvex gradient methods, convex relaxations, Riemannian optimization on matrix manifolds, and adaptive preconditioners such as Shampoo and spectral or polar-based optimizers such as Muon. Evaluation ranges from matrix sensing, completion, and sparse-matrix benchmarks to language-model pretraining, where runtime, memory, and validation loss matter alongside convergence guarantees. A central challenge is matching the optimizer’s geometry to the problem: methods with strong theoretical guarantees or gains in one setting may be costly or perform differently across architectures, noise regimes, and matrix structures.

Latest papers

CardsList

60 papers2 papers in the last four weeks, down 33% on the four weeks before. 0.0% of all new papers.

  1. BiMuon: A Geometry-Aware Stochastic Bilevel Optimization Approach

    Nov 6, 2026Zhen Qin, Zhiyao Zhang, Yingbin Liang +1Stochastic Bilevel OptimizationMatrix Optimization

  2. Normalize-Then-Precondition: A Hierarchical Approach to Marginal Scale and Interaction Geometry for LLM Training

    Sep 29, 2026Zixuan Gong, Zeyu Gan, Jiaye Teng +1Language Model PretrainingMatrix Optimization

  3. Matrix AdaGrad: Row-wise and Column-wise Adaptive Subgradient Methods

    Sep 18, 2026Wenpeng Zhang, Runsheng Yu, Peilin ZhaoAdaptive Gradient MethodsNeural Network Optimization

  4. Optimization over covariance matrices with a parameterized metric

    Sep 15, 2026Yibang Li, Bamdev Mishra, Pratik Jawanpuria +1Riemannian OptimizationSecond-Order Optimization

  5. A Nuclear-Norm Lower Bound for Dithered Scalar Quantization of Matrix Products

    Sep 4, 2026Piyush Sao, Narasinga Miniskar, Pedro Valero-Lara +2Matrix Optimization

  6. Sharp Restricted Isometry Thresholds for Global Minima of Rank-Restricted Matrix LASSO

    Aug 29, 2026Richard Y. ZhangMatrix OptimizationSparse Recovery

  7. Improving the matrix multiplication exponent with modern optimization and AlphaEvolve

    Aug 17, 2026Emilien Dupont, Marvin Eisenberger, Borislav Kozlovskii +7Matrix Optimization

  8. MOON: Multi-Objective OrthoNormalized Updates for Multitask Learning

    Aug 12, 2026Shiji Zhou, Kunlin Lyu, Lei Zhang +2Neural Network OptimizationMulti-Task Learning

  9. Muon on the Stiefel Manifold Admits an Exact Closed-Form Update

    Aug 6, 2026Mikhail Solonko, Molozhavenko Alexander, Maxim RakhubaStiefel Manifold OptimizationRiemannian Optimization

  10. The Loss Does Not See the Basis, but Adam Does

    Aug 5, 2026Devender SinghLow-Rank Matrix DecompositionMatrix Optimization

  11. Joint Affine Spectral Shaping: Coupling Weight and Bias Updates Beyond Weight-Only Muon

    Aug 4, 2026Gongyue Zhang, Honghai LiuNeural Network OptimizationMuon Optimizer

  12. Graph-Based Correlation Matrix Generation: A Convex Optimization Approach

    Jul 24, 2026Ali Fakhar, K{é}vin Polisano, Ir{è}ne Gannaz +1Matrix OptimizationMatrix Completion

  13. Muse: Representation Geometry of Muon Beyond Normalized Momentum

    Jul 16, 2026Da Chang, Qiankun Shi, Lvgang Zhang +4Representation GeometryMomentum Methods

  14. Graph-Regularized Low-Rank Matrix Completion by Variable Projection

    Jul 10, 2026Benoît Loucheur, P. -A. Absil, Michel JournéeLow-Rank ApproximationMatrix Optimization

  15. Beyond Adam: SOAP and Muon for Faster, Label-Efficient Training of Machine Learning Interatomic Potentials

    Jul 2, 2026Gil Harari, Yoel Zimmermann, Ola Tangen Kulseng +4Machine Learning Interatomic PotentialsNeural Network Optimization

  16. Direction-Magnitude Decomposition for Low-Rank Matrix Optimization: Faster Convergence and Saddle-to-saddle Dynamics

    Jun 30, 2026Yudong Wei, Liang Zhang, Bingcong Li +1Low-Rank Matrix DecompositionNonconvex Optimization

  17. ITSPACE: Monotone Gaussian Optimal Transport Updates

    Jun 29, 2026Woojoo Na, Jennifer DyMatrix OptimizationOptimal Transport

  18. Muon learns balanced solutions in matrix factorization without slow saddle-to-saddle dynamics

    Jun 29, 2026Mark Rhee, Jamie Simon, Dhruva KarkadaMuon OptimizerMatrix Optimization

  19. Aurora: A Leverage-Aware Spectral Optimizer

    Jun 26, 2026Alec Dewulf, Dhruv Pai, Li Yang +2Neural Network OptimizationMuon Optimizer

  20. Schattor: Schatten-family methods for deep learning optimization

    Jun 14, 2026Bohao Ma, Junyu Zhang, Chuan HeDeep Learning OptimizationStochastic Optimization Convergence

  21. Free Heavy-Tailed Lunch for Muon: A Theoretical Justification of Empirical Success

    Jun 12, 2026Florian Hübler, Thomas Pethick, Suvrit SraStochastic OptimizationNonconvex Stochastic Optimization

  22. Zeta: Dual Whitening for Matrix Optimization via Coordinate-Adaptive Preconditioning

    Jun 12, 2026Kaiwen Chen, Shuhai Zhang, Zimo Liu +7Neural Network OptimizationFeature Whitening

  23. Scalable Deep Unfolding of Conic Optimizers

    Jun 11, 2026Alex Oshin, Rahul Vodeb Ghosh, Evangelos A. TheodorouSemidefinite ProgrammingImplicit Differentiation