Singular Value Decomposition

Also known as SVD

Follow this topic Ask about this topic

Momentum

6 papers in the last four weeks, against 1 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 26

All topics

D-SLR: The Disjoint Row-Sparse plus Low-Rank Decomposition
Oct 7, 2026Vincent SzolnokyLow-Rank Matrix Decomposition Low-Rank Compression
Department of Mathematical Sciences Chalmers University of Technology
Singular Value Decomposition: A Geometric Rediscovery, Where Proofs Become Algorithms
Oct 6, 2026Paul AgronSingular Value Decomposition
Prescriptive SVD-Inspired Attention via Spectral Energy Retention
Sep 21, 2026Vasileios Arampatzakis, Vasileios Sevetlidis, George PavlidisTransformer Interpretability Low-Rank Attention
Athena Research Center, Greece
Randomized SVD Approximations for Spectral Co-Clustering of Word-Document Matrices
Sep 16, 2026Fateme Mazdarani, Carlos ToxtliClustering Randomized Sketching
School of Computing Clemson University Clemson, SC, USA
Per-Matrix Optimality Is Not Enough: Three-Level Optimization for Low-Rank LLM Compression
Sep 14, 2026Huicheng Zhang, Xiyao Feng, Ze-Tong Li +6Model Compression LLM Compression
Hong Kong University of Science and Technology (Guangzhou), Guangdong 511453, China · QudeLeap Research, Shanghai 200030, China
A Functional SVD Framework for Regularized Multivariate Functional PCA with Dual Penalization
Sep 13, 2026Yue Zhao, Hossein Haghbin, Rebecca Sanders +1Principal Component Analysis Singular Value Decomposition
Division of Biostatistics and Health Data Science, University of Minnesota, USA · Faculty of Intelligent Systems Engineering and Data Science, Persian Gulf University, Iran · Department of Mathematical and Statistical Sciences, Marquette University, USA
Semi-Tensor Product-Based Multi-Term Randomized T-SVD and Its Visual Applications
Sep 11, 2026Xingchen Xiao (School of Mathematics and Statistics, Southwest University, Chongqing +13Tensor Completion Tensor Decomposition
School of Mathematics and Statistics, Southwest University, Chongqing, 400715, China
Mind the Approximation: Fisher-Weighted SVD Compression for ViTs
Sep 7, 2026Moritz Thoma, Maximilian Groezinger, Maximilian Forstenhäusler +7Efficient ViTs Low-Rank Compression
Technical University of Munich, DE · BMW Group, DE · University of Glasgow, GB +1
Compact and Infinite-Order Error Analysis for Null-Space SVD Estimation
Aug 31, 2026Xin Li, Jonathan Cohen, Rami PuzisRandom Matrix Theory Singular Value Decomposition
Quasi-SVD: Learning a Lie-constrained matrix factorisation for real-time imaging
Jul 28, 2026Christopher HahneLie Group Methods Computational Imaging
University of Bern 3008 Bern, Switzerland
LACE-SVD: Loss-Aware SVD with Cumulative Error Correction for LLM Compression
Jul 3, 2026Zhuowen Liu, Longkun Hao, Shiyu Feng +3Model Compression LLM Compression
Zhuowen Liu1 · Longkun Hao2 · Shiyu Feng3 +3
CORA: Per-Slice Coherent Orthogonal Rotation for SVD-based Low-Rank Adaptation
Jun 30, 2026Pengcheng Wang, Ziran Liu, Wei Wang +1LLM Fine-Tuning Low-Rank Matrix Decomposition
Purdue University · Shanghai Institute for Mathematics and Interdisciplinary Sciences / Fudan University · Futurewei Technologies, Inc.
SVD-Surgeon: Optimal Singular-Value Surgery for Large Language Model Compression
Jun 22, 2026Mahmoud Safari, Frank HutterLLM Pruning LLM Compression
University of Freiburg · Prior Labs · ELLIS Institute Tübingen
Sparse Functional Singular Value Decomposition for Biclustering and Triclustering Longitudinal Data
Jun 3, 2026Yue Zhao, Thierry Chekouo, Sandra SafoClustering Singular Value Decomposition
Division of Biostatistics and Health Data Science University of Minnesota 2221 University Avenue SE2026 Minneapolis, MN 55414, United States
LASER: Loss-Aware Singular-value Decomposition and Rank Allocation for Efficient Low-Precision Vision-Language Models
May 30, 2026Haiyu Wang, Yutong Wang, Leshu Li +2Vision-Language Models Efficient VLM Inference
Tandon School of Engineering, New York University · Courant Institute of Mathematical Sciences, New York University · Brookhaven National Laboratory
History-aware adaptive reduced-order models via incremental singular value decomposition
May 27, 2026Amirpasha Hedayat, Ali Mohaghegh, Laura Balzano +2Reduced-Order Modeling Singular Value Decomposition
Department of Aerospace Engineering, University of Michigan, Ann Arbor, MI, USA · Department of Aerospace Engineering, University of Kansas, Lawrence, KS, USA · Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI, USA
Efficient Pre-Training of LLMs through Truncated SVD Representations
May 27, 2026Kaivan Kamali, Kajetan Schweighofer, Hormoz Shahrzad +3Language Model Pretraining Low-Rank Matrix Decomposition
Cognizant AI Lab
Group-Algebraic Tensors: Provably-optimal Equivariant Learning and Physical Symmetry Discovery
May 19, 2026Paulina Hoyos, Shashanka Ubaru, Dongsung Huh +5Scientific Discovery Equivariant Neural Networks
UT Austin · IBM Research · Independent +2
Accelerating Power Method with Fast Sketching for Stronger Low-Rank Approximation
May 10, 2026Shabarish Chenakkod, Michał DerezińskiLow-Rank Matrix Decomposition Randomized Sketching
University of Michigan, Ann Arbor, MI 48105.
Phases of Muon: When Muon Eclipses SignSGD
May 10, 2026Elliot Paquette, Noah Marshall, Lucas Benigni +3Sign-Based Optimization Muon Optimizer
Mathematics and Statistics Department, McGill University · Mathematics and Statistics Department, Université de Montréal · Google DeepMind
FlashSVD v1.5: Making Low-Rank Transformers Inference Actually Fast
May 8, 2026Wenhao Wu, Zishan Shao, Kangning Cui +5Efficient Transformer Inference LLM Compression
Department of Electrical & Computer Engineering, Duke University · Department of Statistical Science, Duke University · Department of Computer Science, Wake Forest University +1
Sparse Random-Feature Neural Networks with Krylov-Based SVD for Singularly Perturbed ODE
May 8, 2026Kevin Kurian Thomas Vaidyan, Siddharth RoutNeural PDE Solvers Singular Value Decomposition
Institute of Applied Mathematics University of British Columbia Vancouver, BC V6T 1Z4
A Generalized Singular Value Theory for Neural Networks
May 7, 2026Brian Charles Brown, Robert Bridges, David Grimsman +2Neural Representation Geometry Geometric Representation Learning
AI Sweden Gothenburg, Sweden
Structured 3D-SVD: A Practical Framework for the Compression and Reconstruction of Biological Volumetric Images
Apr 18, 2026Mario Aragonés Lozano, Oscar Romero, Antonio LeónImage Reconstruction Tensor Decomposition
Communications Department, Universitat Politècnica de València, 46022 Valencia, Spain
FairLRF: Achieving Fairness through Sparse Low Rank Factorization
Nov 20, 2025Yuanbo Guo, Jun Xia, Yiyu ShiAlgorithmic Fairness Group Fairness
University of Notre Dame Notre Dame, Indiana, USA
Stacked SVD or SVD stacked? A Random Matrix Theory perspective on data integration
Jul 29, 2025Tavor Z. Baharav, Phillip B. Nicol, Rafael A. Irizarry +1Low-Rank Approximation Singular Value Decomposition
Eric and Wendy Schmidt Center, Broad Institute, Cambridge, MA, 02142, USA · Department of Data Science, Dana Farber Cancer Institute, Boston, MA, 02115, USA · Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, MA, 02115, USA

Oct 7, 2026, cs.LGJ/K move · Enter open · S save

D-SLR: The Disjoint Row-Sparse plus Low-Rank Decomposition

Vincent Szolnoky

Department of Mathematical Sciences Chalmers University of Technology

Low-Rank Matrix Decomposition Low-Rank Compression Low-Rank Approximation Singular Value Decomposition

Open paper AskPDF arXiv

Compressing a matrix for reconstruction still defaults to the truncated SVD, approximating the data with a single low-rank structure. It is common to reduce the residual further by adding an overlapping row-sparse component, but methods that solve this joint problem often require iterative solvers and tuning of regularization parameters. We propose the Disjoint Row-Sparse plus Low-Rank (D-SLR) decomposition, a closed-form drop-in for the truncated SVD that improves or exactly matches it. D-SLR restricts rows to either being stored verbatim or approximated by the low-rank fit, never both. Under squared error this restriction costs nothing: the joint optimum is attainable disjointly with fewer parameters at every non-trivial rank and stored row count (shape). With zero stored rows D-SLR reduces to the truncated SVD, so it never does worse at equal cost. The algorithm scores the entire error-versus-parameters tradeoff, and the solution is chosen afterwards by a supplied error target or parameter count, or by a selection rule. The grid and solution together cost three SVDs, with no tuning or regularization. We derive an assumption-free, a-posteriori lower bound on the error at every shape, giving each solution a computable certificate on the potential gain of any other choice of rank and stored rows. Experiments on synthetic and real data (LLM embedding tables, network traffic, hyperspectral images) confirm the gains and quantify the certificate.

Momentum

Related topics

Latest papers 26