Matrix Optimization

Follow this topic Ask about this topic

Momentum

3 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 48

All topics

An Exterior Method for Nonnegative Matrix Factorization
May 19, 2026Qiujing Lu, Tonmoy Monsoor, Ehsan Ebrahimzadeh +2Non-Negative Matrix Factorization Low-Rank Matrix Decomposition
1ECE, UCLA · 2eBay Search Science Team
Learning Fill-in Reduction Ordering via Graph Policy Optimization for Sparse Matrices
May 17, 2026Ziwei Li, Shuzi Niu, Huiyuan Li +2Policy Optimization Matrix Optimization
Institute of Software, Chinese Academy of Sciences, Beijing, China
Sample-efficient inductive matrix completion with noise and inexact side-information
May 16, 2026Yuepeng Yang, Cong MaMatrix Optimization Matrix Completion
Department of Statistics and Data Sciences, Yale University · Department of Statistics, University of Chicago
Intrinsic Muon: Spectral Optimization on Riemannian Matrix Manifolds
May 10, 2026Yibang Li, Bihari Lal Pandey, Ravi Sah +4Stiefel Manifold Optimization Riemannian Optimization
Nanyang Technological University, Singapore. · Indian Institute of Technology Bombay, India. · University of Sydney, Australia. +1
Navigating LLM Valley: From AdamW to Memory-Efficient and Matrix-Based Optimizers
May 9, 2026Aditya RanganathDeep Learning Optimization Memory-Efficient Optimization
Center for Applied Scientific Computing Lawrence Livermore National Laboratory Livermore, CA, USA
OrScale: Orthogonalised Optimization with Layer-Wise Trust-Ratio Scaling
May 8, 2026Yuxuan Lou, Yang YouNeural Network Optimization Muon Optimizer
National University of Singapore
PolarAdamW: Disentangling Spectral Control and Schur Gauge-Equivariance in Matrix Optimisation
May 8, 2026Haozhou ZhangMuon Optimizer Matrix Optimization
Department of Mathematics and Statistics University of Idaho
Muon with Nesterov Momentum: Heavy-Tailed Noise and (Randomized) Inexact Polar Decomposition
May 7, 2026Sayantan Choudhury, Xiaoran Cheng, Martin Takáč +2Stochastic Optimization Momentum Methods
MBZUAI · Penn State University · Georgia Institute of Technology +1
Pro-KLShampoo: Projected KL-Shampoo with Whitening Recovered by Orthogonalization
May 7, 2026Ruotong Sun, Ermin WeiLanguage Model Pretraining Deep Learning Optimization
†Department of Industrial Engineering & Management Sciences, Northwestern University · ∗Department of Electrical & Computer Engineering, Northwestern University
DiBA: Diagonal and Binary Matrix Approximation for Neural Network Weight Compression
May 7, 2026Nobutaka OnoEfficient Neural Network Inference Matrix Optimization
Tokyo Metropolitan University, 6–6 Asahigaoka, Hino-shi, Tokyo 191–0065, Japan.
Nora: Normalized Orthogonal Row Alignment for Scalable Matrix Optimizer
May 5, 2026Jinghui Yuan, Jiaxuan Zou, Shuo Wang +2Neural Network Optimization Matrix Optimization
1. School of Artificial Intelligence, Optics and Electronics (iOPEN), Northwestern Polytechnical University. · 2. School of Mathematics and Statistics, Xi’an Jiaotong University. · 4. Gaoling School of Artificial Intelligence, Renmin University of China. +1
Hankel and Toeplitz Rank-1 Decomposition of Arbitrary Matrices with Applications to Signal Direction-of-Arrival Estimation
Apr 29, 2026Georgios I. OrfanidisLow-Rank Matrix Decomposition Direction-of-Arrival Estimation
Center for Connected Autonomy and AI and the Dept. of Electrical Engineering and Computer Science, Florida Atlantic University, Boca Raton, FL 33431 USA
State-Dependent Lyapunov Analysis of Rank-1 Matrix Factorization
Apr 28, 2026Jaehong MoonEdge of Stability Lyapunov Stability
Industrial & Enterprise Systems Engineering University of Illinois at Urbana-Champaign Urbana, IL 61801
Provably Finding a Hidden Dense Submatrix among Many Planted Dense Submatrices via Convex Programming
Jan 7, 2026Valentine Olanubi, Phineas Agar, Brendan AmesStochastic Block Model Matrix Optimization
University of Alabama, Department of Mathematics · Department of Mathematics, University of Alabama, Tuscaloosa, AL, USA · University of Southampton, School of Mathematical Sciences +1
Ky Fan Norms and Beyond: Dual Norms and Combinations for Matrix Optimization
Dec 10, 2025Alexey Kravatskiy, Ivan Kozyrev, Nikolai Kozlov +3Deep Learning Optimization Muon Optimizer
MIRIAI · Russia · MIPT, INM RAS +3
The Optimization Landscape of Carathéodory Decomposition of Toeplitz Covariances
Nov 3, 2025Daniel Busbib, Ami WieselNonconvex Optimization Matrix Optimization
School of Computer Science and Engineering, The Hebrew University of Jerusalem, Jerusalem 91904, Israel
Low-rank Orthogonalization for Large-scale Matrix Optimization with Applications to Foundation Model Training
Sep 15, 2025Chuan He, Zhanwang Deng, Zhaosong LuNeural Network Optimization Low-Rank Approximation
Department of Mathematics Linköping University, Sweden · Academy for Advanced Interdisciplinary Studies Peking University, Beijing, People’s Republic of China · Department of Industrial and Systems Engineering University of Minnesota, USA
Doubly Stochastic Adaptive Neighbors Clustering via the Marcus Mapping
Aug 6, 2024Jinghui Yuan, Chusheng Zeng, Fangyuan Xie +5Clustering Graph Clustering
School of Artificial Intelligence, OPtics and ElectroNics (iOPEN), Northwestern Polytechnical University, Xi’an 710072, Shaanxi, China · Institute of Artificial Intelligence (TeleAI), China Telecom, P. R. China

May 19, 2026, cs.LGJ/K move · Enter open · S save

An Exterior Method for Nonnegative Matrix Factorization

Qiujing Lu, Tonmoy Monsoor, Ehsan Ebrahimzadeh, Kartik Sharma+1

1ECE, UCLA · 2eBay Search Science Team

Non-Negative Matrix Factorization Low-Rank Matrix Decomposition Nonconvex Optimization Low-Rank Approximation Constrained Optimization Matrix Optimization

Open paper AskPDF arXiv

Nonnegative matrix factorization (NMF) seeks a low-rank approximation

X \approx UV^T

with nonnegative factors and is commonly solved using interior methods that enforce feasibility throughout optimization. We show that such constraint-driven approaches can impede progress in the nonconvex landscape, leading to slow convergence or convergence to suboptimal stationary points. We propose an exterior framework for NMF (eNMF) that separates low-rank approximation from nonnegativity enforcement. Our method initializes from the optimal unconstrained factorization and introduces a rotation procedure that maps unconstrained factors to an exterior point closest to the nonnegative orthant. This viewpoint yields an algorithmic framework in which simple iterative updates converge to KKT-satisfying stationary points on the boundary of the positive orthant. The exterior formulation also enables a geometric interpretation of NMF solutions, clarifying equivalence classes of factorizations under permutation and orthogonal transformations. An intriguing numerical result, involving 400 NMF experiments across both real and synthetic datasets, show that in 99% of the cases, different algorithms tend to converge towards equivalent factor matrices. We benchmark eNMF against 9 state-of-the-art NMF algorithms with 9 initialization schemes across 3 real-world and 2 synthetic datasets. eNMF consistently outperforms all 81 competitors, achieving up to 30% lower reconstruction error under equal-time settings and up to 150% speedup under equal-error settings. The downstream experiments further demonstrate substantial performance gains in audio processing and recommendation tasks, corroborating the practical benefits of the proposed exterior optimization framework. Code is available at https://github.com/roychowdhuryresearch/eNMF

Momentum

Related topics

Latest papers 48