Matrix Optimization

Ask about this topic

Overview

Matrix optimization studies how to efficiently solve problems whose variables, constraints, or data have matrix structure, including low-rank factorization and completion, clustering, covariance estimation, and neural-network training. Common approaches include projected and nonconvex gradient methods, convex relaxations, Riemannian optimization on matrix manifolds, and adaptive preconditioners such as Shampoo and spectral or polar-based optimizers such as Muon. Evaluation ranges from matrix sensing, completion, and sparse-matrix benchmarks to language-model pretraining, where runtime, memory, and validation loss matter alongside convergence guarantees. A central challenge is matching the optimizer’s geometry to the problem: methods with strong theoretical guarantees or gains in one setting may be costly or perform differently across architectures, noise regimes, and matrix structures.

Latest papers

CardsList

58 papers1 paper in the last four weeks, down 80% on the four weeks before. 0.0% of all new papers.

  1. Normalize-Then-Precondition: A Hierarchical Approach to Marginal Scale and Interaction Geometry for LLM Training

    Sep 29, 2026Zixuan Gong, Zeyu Gan, Jiaye Teng +1Language Model PretrainingMatrix Optimization

    Gaoling School of Artificial Intelligence Renmin University of China · School of Statistics and Management Shanghai University of Finance and Economics

  2. Cost-free Spectral Estimation for Adaptive Newton--Schulz in Matrix Optimizers

    Sep 27, 2026Kristi Topollai, Anna ChoromanskaMatrix OptimizationNewton-Schulz Iteration

    New York University

  3. Riemannian Structure and Optimization for a Class of Low-Parametric Orthogonal Matrices

    Sep 23, 2026Ali Aliev, Maxim RakhubaRiemannian OptimizationMatrix Optimization

    HSE University

  4. Matrix AdaGrad: Row-wise and Column-wise Adaptive Subgradient Methods

    Sep 18, 2026Wenpeng Zhang, Runsheng Yu, Peilin ZhaoAdaptive Gradient MethodsNeural Network Optimization

    Independent Reasearcher · School of Artificial Intelligence Shanghai Jiao Tong University

  5. Optimization over covariance matrices with a parameterized metric

    Sep 15, 2026Yibang Li, Bamdev Mishra, Pratik Jawanpuria +1Riemannian OptimizationSecond-Order Optimization

    Nanyang Technological University, Singapore · Microsoft India · Indian Institute of Technology Bombay, India

  6. A Nuclear-Norm Lower Bound for Dithered Scalar Quantization of Matrix Products

    Sep 4, 2026Piyush Sao, Narasinga Miniskar, Pedro Valero-Lara +2Matrix Optimization

    Oak Ridge National Laboratory, Oak Ridge, Tennessee 37831, USA

  7. Sharp Restricted Isometry Thresholds for Global Minima of Rank-Restricted Matrix LASSO

    Aug 29, 2026Richard Y. ZhangMatrix OptimizationSparse Recovery

    Department of Electrical and Computer Engineering, University of Illinois Urbana–Champaign, Urbana, IL 61801 USA

  8. Improving the matrix multiplication exponent with modern optimization and AlphaEvolve

    Aug 17, 2026Emilien Dupont, Marvin Eisenberger, Borislav Kozlovskii +7Matrix Optimization

    Google DeepMind · Carnegie Mellon University · Columbia University +1

  9. MOON: Multi-Objective OrthoNormalized Updates for Multitask Learning

    Aug 12, 2026Shiji Zhou, Kunlin Lyu, Lei Zhang +2Neural Network OptimizationMulti-Task Learning

    Institute of Artificial Intelligence, Beihang University · Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, Beihang University · Beijing Academy of Artificial Intelligence (BAAI) +1

  10. Exact Rank-Space KL Projection for Shared-Marginal Low-Rank Factors: Application to Doubly Stochastic Clustering

    Aug 9, 2026Enliang HuGraph Structure LearningLow-Rank Matrix Decomposition

  11. Muon on the Stiefel Manifold Admits an Exact Closed-Form Update

    Aug 6, 2026Mikhail Solonko, Molozhavenko Alexander, Maxim RakhubaStiefel Manifold OptimizationRiemannian Optimization

    HSE University, Moscow, Russia.

  12. The Loss Does Not See the Basis, but Adam Does

    Aug 5, 2026Devender SinghLow-Rank Matrix DecompositionMatrix Optimization

    Department of Computer Science Memorial University of Newfoundland

  13. MESH: Memory-Efficient Sinkhorn Optimization for Mixture-of-Experts Training

    Aug 5, 2026Masato FujitakeNeural Network OptimizationMemory-Efficient Optimization

    FA Research, Fast Accounting Co., Ltd.

  14. Joint Affine Spectral Shaping: Coupling Weight and Bias Updates Beyond Weight-Only Muon

    Aug 4, 2026Gongyue Zhang, Honghai LiuNeural Network OptimizationMuon Optimizer

    State Key Laboratory of Robotics and Systems, Harbin Institute of Technology, Shenzhen, Shenzhen, 518055, China.

  15. Low-Rank Dependence Decomposition via Accelerated Symmetric Non-negative Matrix Factorization

    Jul 27, 2026Lavinia Ghita, Dhruv Desai, Jake Goldberg +1Non-Negative Matrix FactorizationLow-Rank Matrix Decomposition

    NVIDIA

  16. Graph-Based Correlation Matrix Generation: A Convex Optimization Approach

    Jul 24, 2026Ali Fakhar, K{é}vin Polisano, Ir{è}ne Gannaz +1Matrix OptimizationMatrix Completion

    UGA · Univ. Grenoble Alpes, CNRS, Grenoble INP, Inria, LJK, F-38000 Grenoble, France · G-SCOP\_GROG, G-SCOP, Grenoble INP, UGA +2

  17. Muse: Representation Geometry of Muon Beyond Normalized Momentum

    Jul 16, 2026Da Chang, Qiankun Shi, Lvgang Zhang +4Representation GeometryMomentum Methods

    1Pengcheng Laboratory · 2Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences +3

  18. Graph-Regularized Low-Rank Matrix Completion by Variable Projection

    Jul 10, 2026Benoît Loucheur, P. -A. Absil, Michel JournéeLow-Rank ApproximationMatrix Optimization

    ICTEAM Institute, UCLouvain, 1348 Louvain-la-Neuve, Belgium. · Department of Climatology, Royal Meteorological Institute of Belgium, 1180 Uccle, Belgium.

  19. Beyond Adam: SOAP and Muon for Faster, Label-Efficient Training of Machine Learning Interatomic Potentials

    Jul 2, 2026Gil Harari, Yoel Zimmermann, Ola Tangen Kulseng +4Machine Learning Interatomic PotentialsNeural Network Optimization

    John A. Paulson School of Engineering and Applied Sciences, Harvard University, Cambridge, MA, USA · Robert Bosch LLC Research and Technology Center, Watertown, MA, USA

  20. Direction-Magnitude Decomposition for Low-Rank Matrix Optimization: Faster Convergence and Saddle-to-saddle Dynamics

    Jun 30, 2026Yudong Wei, Liang Zhang, Bingcong Li +1Low-Rank Matrix DecompositionNonconvex Optimization

    Department of Mathematics, ETH Zurich, Zurich, Switzerland · Department of Computer Science, ETH Zurich, Zurich, Switzerland

  21. ITSPACE: Monotone Gaussian Optimal Transport Updates

    Jun 29, 2026Woojoo Na, Jennifer DyMatrix OptimizationOptimal Transport

    Department of Computer Engineering, Northeastern University, Boston, USA.

  22. Muon learns balanced solutions in matrix factorization without slow saddle-to-saddle dynamics

    Jun 29, 2026Mark Rhee, Jamie Simon, Dhruva KarkadaMuon OptimizerMatrix Optimization

    UC Berkeley · Imbue

  23. Aurora: A Leverage-Aware Spectral Optimizer

    Jun 26, 2026Alec Dewulf, Dhruv Pai, Li Yang +2Neural Network OptimizationMuon Optimizer

    Tilde Research

  24. Spectral DPPs via NEPv: A Scalable Continuous Relaxation of Determinantal MAP for Diversity-Aware Data Selection

    Jun 17, 2026Richard Yi Da XuDeterminantal Point ProcessTraining Data Selection

    Hong Kong Baptist University · TadReamk Limited

  25. Schattor: Schatten-family methods for deep learning optimization

    Jun 14, 2026Bohao Ma, Junyu Zhang, Chuan HeDeep Learning OptimizationStochastic Optimization Convergence

    School of Data Science, The Chinese University of Hong Kong (Shenzhen), Shenzhen, Guangdong, China · Department of Industrial Systems Engineering and Management, National University of Singapore, Singapore · Department of Mathematics, Linköping University, Sweden

  26. Free Heavy-Tailed Lunch for Muon: A Theoretical Justification of Empirical Success

    Jun 12, 2026Florian Hübler, Thomas Pethick, Suvrit SraStochastic OptimizationNonconvex Stochastic Optimization

    Department of Computer Science, ETH Zurich, Switzerland · Department of Mathematics, Technical University of Munich, Germany · Munich Center for Machine Learning (MCML)

  27. Zeta: Dual Whitening for Matrix Optimization via Coordinate-Adaptive Preconditioning

    Jun 12, 2026Kaiwen Chen, Shuhai Zhang, Zimo Liu +7Neural Network OptimizationFeature Whitening

    South China University of Technology · AIGCode · Hong Kong Baptist University

  28. Scalable Deep Unfolding of Conic Optimizers

    Jun 11, 2026Alex Oshin, Rahul Vodeb Ghosh, Evangelos A. TheodorouSemidefinite ProgrammingImplicit Differentiation

    Georgia Institute of Technology

  29. The Spectral Dynamics and Noise Geometry of Muon

    Jun 7, 2026Pierfrancesco Beneventano, Mahmoud Abdelmoneum, Tomaso PoggioMuon OptimizerMatrix Optimization

    Massachusetts Institute of Technology

  30. On solving symmetric multi-type orthogonal non-negative matrix tri-factorization problem

    Jun 6, 2026Rok Hribar, Gregor Papa, Janez Povh +1Non-Negative Matrix FactorizationGraph Clustering

    Computer Systems Department, Jožef Stefan Institute, Jamova cesta 39, Ljubljana, 1000, Slovenia. · Laboratory for Engineering Design, Faculty of Mechanical Engineering, University of Ljubljana, Aškerčeva cesta 6, Ljubljana, 1000, Slovenia. · Rudolfovo – Science and Technology Centre Novo mesto, Podbreznik 15, Novo mesto, 8000, Slovenia. +1