Nuclear Norm Minimization

Follow this topic Ask about this topic

Nuclear norm minimization recovers or learns matrices and tensors with low-dimensional structure by penalizing the sum of their singular values, a tractable stand-in for rank. Common approaches include proximal singular-value thresholding, ADMM, and iteratively reweighted least squares, with structured variants for images, tensors, and groups that share latent structure. The methods are used in matrix completion, recommendation, multitask learning, and neural-network compression and adaptation. Recovery and convergence guarantees are known under suitable assumptions, but repeated spectral decompositions, mismatched data structure, and the gap between theoretical guarantees and practical scale remain important challenges.

Latest papers

15 papers2 papers in the last four weeks, down 33% on the four weeks before. 0.0% of all new papers.

Spectral Weight Decay: Inducing Low-Rank Structure in Neural Network Weights
Oct 8, 2026Dmitrii Andriianov, Andrey Veprikov, Aleksandr BeznosikovWeight Decay Nuclear Norm Minimization
Basic Research of Artificial Intelligence Laboratory (BRAIn Lab) · SB AI Lab · Innopolis University
Robust Tensor Completion via Reflective Convolution Nuclear Norm Minimization
Oct 4, 2026Weiguo Zhou, Feng Zhang, Wenjin Qin +1Tensor Completion Nuclear Norm Minimization
School of Mathematics and Statistics, Southwest University, Chongqing, 400715, China · College of Artificial Intelligence and Advanced Interdisciplinary Sciences, Chongqing Normal University, Chongqing, 401331, China
Nuclear Norm-Regularized Bayesian Matrix Completion
Sep 24, 2026Calvin TolbertNuclear Norm Minimization Posterior Sampling
School of Operations Research and Information Engineering, Cornell University
Regularized Least Squares Training of Quadratic Neural Networks with Applications to System Identification
Sep 15, 2026Luis Rodrigues, Zachary Yetman Van Egmond, Mohammad R. Amiri FardNeural Network Optimization Nuclear Norm Minimization
Department of Electrical and Computer Engineering Concordia University Montr´eal, QC, Canada
On the Regularization Landscape for the Linear Recommendation Models
Sep 10, 2026Dong Li, Zhenming Liu, Ruoming Jin +4Nuclear Norm Minimization Low-Rank Approximation
Kent State University, USA · College of William and Mary, USA · iLambda, USA
Trace-Norm Geometry of Composed Multitask Predictors
Aug 25, 2026Mahdi MohammadigohariMulti-Task Learning Statistical Learning Theory
Faculty of Engineering, Free University of Bozen–Bolzano, Via Bruno Buozzi 1, 39100 Bolzano, Italy
Tight Majorizations and Convergence Rates of Nuclear Norm Minimization IRLS
Aug 24, 2026Christian Kümmerle, Tomas Masak, Dominik StögerNuclear Norm Minimization Optimization Convergence Analysis
School of Data, Mathematical, and Statistical Sciences, Department of Computer Science & Institute of Artificial Intelligence, University of Central Florida, Oviedo, FL 32816, USA · Institute for Statistics and Mathematics, Wirtschaftsuniversität Wien, 1020 Vienna, Austria · Department of Mathematics, KU Eichstätt-Ingolstadt, 85049 Ingolstadt, Germany
Three-Pronged Spectral Control for Federated Parameter Efficient Fine Tuning
Jul 23, 2026Shiva Raj Pokhrel, Dipsan Bhattarai, Anwar WalidFederated Learning Aggregation Nuclear Norm Minimization
School of IT, Deakin University, Australia · Columbia University, USA
SLORR: Simple and Efficient In-Training Low-Rank Regularization
Jul 9, 2026David González-Martínez, Shiwei LiuLLM Compression Low-Rank Matrix Decomposition
Max Planck Institute for Intelligent Systems · University of Tübingen · ELLIS Institute Tübingen +1
Low-Rank Decay for Grokking in Scale-Invariant Transformers: A Spectral-Geometric View
Jun 3, 2026Mingyu LiSpectral Regularization Weight Decay
Beijing Normal University
Group-Aware Matrix Estimation and Latent Subspace Recovery
May 19, 2026Hamza Golubovic, Matthew Shen, Genevera I. Allen +1Low-Rank Matrix Decomposition Nuclear Norm Minimization
Department of Statistics, Columbia University New York, NY 10027 · Department of Statistics, Irving Institute for Cellular Dynamics, Zuckerman Institute Columbia University New York, NY 10027 · School of Data and Information Sciences, University of North Carolina at Chapel Hill Chapel Hill, NC 27599
Low Rank Tensor Completion via Adaptive ADMM
May 5, 2026Niclas Führling, Getuar Rexhepi, Giuseppe Thadeu Freitas de AbreuTensor Completion Nuclear Norm Minimization
School of Computer Science and Engineering, Constructor University, Bremen, Germany
Inductive Convolution Nuclear Norm Minimization for Tensor Completion with Arbitrary Sampling
Apr 18, 2026Wei Li, Yuyang Li, Kaile Du +2Tensor Completion Tensor Decomposition
The authors are with the School of Automation, Southeast University, Nanjing, China 210018.
Towards The Implicit Bias on Multiclass Separable Data Under Norm Constraints
Mar 24, 2026Shengping Xie, Zekun Wu, Quan Chen +1Implicit Bias Gradient Descent
School of Mathematical Sciences, Peking University
Global Convergence of Multiplicative Updates for the Matrix Mechanism: A Collaborative Proof with Gemini 3
Mar 19, 2026Keith RushMatrix Optimization Nuclear Norm Minimization
Google DeepMind

Oct 8, 2026J/K move · Enter open · S save

Spectral Weight Decay: Inducing Low-Rank Structure in Neural Network Weights

Dmitrii Andriianov, Andrey Veprikov, Aleksandr Beznosikov

Basic Research of Artificial Intelligence Laboratory (BRAIn Lab) · SB AI Lab · Innopolis University

Weight Decay Nuclear Norm Minimization Low-Rank Compression LLM Compression

Open paper AskPDF arXiv

Standard weight decay treats each weight matrix as a vector and ignores its spectral structure. We introduce spectral weight decay, a post-step decoupled nuclear-norm update that applies additive rather than multiplicative spectral shrinkage. We connect the update to approximate proximal descent and show that its sensitivity to update order can exceed that of conventional

\ell_2

weight decay near rank deficiency. Across LLaMA models with

124

M to

500

M parameters, spectral weight decay lowers effective rank and improves SVD-LLM compression at matched validation loss. At

500

M and a

4\%

distortion budget, it reaches

1.89\times

compression and

1.18\times

GPU inference speedup, compared with

1.14\times

and

1.01\times

after standard weight decay. Under fixed-horizon training with

60\%

label noise, it also improves final mean clean-test accuracy over matched

\ell_2

regularization by up to

17.8

points on MNIST and

4.6

points across four BERT-base tasks. Code is available at https://github.com/brain-lab-research/SpectralWD.