Low-Rank Matrix Decomposition

Follow this topic Ask about this topic

Momentum

8 papers in the last four weeks, against 1 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 48

All topics

LASER: Loss-Aware Singular-value Decomposition and Rank Allocation for Efficient Low-Precision Vision-Language Models
May 30, 2026Haiyu Wang, Yutong Wang, Leshu Li +2Vision-Language Models Efficient VLM Inference
Tandon School of Engineering, New York University · Courant Institute of Mathematical Sciences, New York University · Brookhaven National Laboratory
Manifold-based Algorithms for the Hadamard Decomposition
May 27, 2026Nicolas Gillis, Subhayan Saha, Stefano Sicilia +1Low-Rank Matrix Decomposition Low-Rank Approximation
Department of Mathematics and Operational Research, University of Mons, Mons, Belgium. · SS is a member of the Gruppo Nazionale Calcolo Scientifico-Istituto Nazionale di Alta Matematica (GNCS-INdAM).
Implicit Regularization in Perturbed Deep Matrix Factorization: Spectral Conditions and Stability
May 27, 2026Jingzhe Wang, Hung-Hsu ChouAlgorithmic Stability Low-Rank Matrix Decomposition
Department of Informatics and Networked Systems, University of Pittsburgh · Department of Mathematics, University of Pittsburgh
Efficient Pre-Training of LLMs through Truncated SVD Representations
May 27, 2026Kaivan Kamali, Kajetan Schweighofer, Hormoz Shahrzad +3Language Model Pretraining Low-Rank Matrix Decomposition
Cognizant AI Lab
Group-Aware Matrix Estimation and Latent Subspace Recovery
May 19, 2026Hamza Golubovic, Matthew Shen, Genevera I. Allen +1Low-Rank Matrix Decomposition Nuclear Norm Minimization
Department of Statistics, Columbia University New York, NY 10027 · Department of Statistics, Irving Institute for Cellular Dynamics, Zuckerman Institute Columbia University New York, NY 10027 · School of Data and Information Sciences, University of North Carolina at Chapel Hill Chapel Hill, NC 27599
An Exterior Method for Nonnegative Matrix Factorization
May 19, 2026Qiujing Lu, Tonmoy Monsoor, Ehsan Ebrahimzadeh +2Non-Negative Matrix Factorization Low-Rank Matrix Decomposition
1ECE, UCLA · 2eBay Search Science Team
IO-SVD: Input-Output Whitened SVD for Adaptive-Rank LLM Compression
May 15, 2026Ali Abbasi, Chayne Thrash, Haoran Qin +2LLM Compression Low-Rank Matrix Decomposition
Vanderbilt University · University of California, Davis
Accelerating Power Method with Fast Sketching for Stronger Low-Rank Approximation
May 10, 2026Shabarish Chenakkod, Michał DerezińskiLow-Rank Matrix Decomposition Randomized Sketching
University of Michigan, Ann Arbor, MI 48105.
Different Prompts, Different Ranks: Prompt-aware Dynamic Rank Selection for SVD-based LLM Compression
May 9, 2026Hengyi Zhu, Zhendong Mi, Grace Li Zhang +1LLM Compression Low-Rank Matrix Decomposition
1Stevens Institute of Technology, Hoboken, USA · 2Technical University of Darmstadt, Darmstadt, Germany
Don't Learn the Shape: Forecasting Periodic Time Series by Rank-1 Decomposition
May 8, 2026Takato HondaLow-Rank Matrix Decomposition Time Series Forecasting
Mellon Inc.
Structural Correspondence and Universal Approximation in Diagonal plus Low-Rank Neural Networks
May 7, 2026Ying Chen, Aoxi Li, Jihun Kim +1Low-Rank Matrix Decomposition Low-Rank Approximation
Department of Industrial Engineering & Operations Research University of California, Berkeley, CA 94720
Importance-Guided Basis Selection for Low-Rank Decomposition of Large Language Models
May 2, 2026Daniel Agyei Asante, Ernie Chang, Yang LiLLM Pruning LLM Compression
Iowa State University · Meta
Hankel and Toeplitz Rank-1 Decomposition of Arbitrary Matrices with Applications to Signal Direction-of-Arrival Estimation
Apr 29, 2026Georgios I. OrfanidisLow-Rank Matrix Decomposition Direction-of-Arrival Estimation
Center for Connected Autonomy and AI and the Dept. of Electrical Engineering and Computer Science, Florida Atlantic University, Boca Raton, FL 33431 USA
State-Dependent Lyapunov Analysis of Rank-1 Matrix Factorization
Apr 28, 2026Jaehong MoonEdge of Stability Lyapunov Stability
Industrial & Enterprise Systems Engineering University of Illinois at Urbana-Champaign Urbana, IL 61801
Predicting LLM Compression Degradation from Spectral Statistics
Apr 20, 2026Mingxue XuLLM Compression Low-Rank Matrix Decomposition
Department of Electrical and Electronic Engineering, Imperial College London, London, United Kingdom
LASER: Low-Rank Activation SVD for Efficient Recursion
Apr 19, 2026Ege Çakar, Ketan Ali Raghu, Lia ZhengLow-Rank Matrix Decomposition Recurrent Neural Networks
Harvard University Cambridge, MA, USA
Stabilizing Native Low-Rank LLM Pretraining
Feb 12, 2026Paul Janson, Edouard Oyallon, Eugene BelilovskyLanguage Model Pretraining Low-Rank Matrix Decomposition
Concordia University, Montreal, Canada · Mila Quebec AI Institute, Montreal, Canada · Sorbonne University, CNRS, Paris, France
BALF: Budgeted Activation-Aware Low-Rank Factorization for Fine-Tuning-Free Model Compression
Sep 29, 2025David González-MartínezLow-Rank Matrix Decomposition Low-Rank Compression
Department of Computer Science and Artificial Intelligence, Universidad de Sevilla · Tübingen AI Center

May 30, 2026, cs.LGJ/K move · Enter open · S save

LASER: Loss-Aware Singular-value Decomposition and Rank Allocation for Efficient Low-Precision Vision-Language Models

Haiyu Wang, Yutong Wang, Leshu Li, Yihui Ren+1

Tandon School of Engineering, New York University · Courant Institute of Mathematical Sciences, New York University · Brookhaven National Laboratory

Vision-Language Models Efficient VLM Inference Low-Rank Matrix Decomposition Low-Rank Compression Singular Value Decomposition

Open paper AskPDF arXiv

Vision-language models (VLMs) deliver strong multimodal reasoning capabilities, but their large computational cost and high parameter counts make deployment challenging on resource-constrained devices. Low-rank decomposition has emerged as a promising compression technique, yet existing methods often optimize local matrix reconstruction error, rely on uniform or heuristic rank allocation, and focus mainly on attention projections while leaving feed-forward networks underexplored. In this paper, we propose~\textit{LASER} (\textbf{L}oss-\textbf{A}ware \textbf{S}ingular-value d\textbf{E}composition and \textbf{R}ank allocation), a low-rank compression framework for efficient low-precision VLM inference. LASER derives a curvature-weighted SVD objective from a second-order approximation of the model loss and uses Kronecker-factored Fisher information to guide decomposition toward downstream performance rather than reconstruction alone. We further introduce a loss-aware cross-layer rank allocation strategy based on calibration gradients, enabling more effective parameter budgeting across layers. Finally, we extend low-rank compression to FFN layers through a hybrid scheme that combines SVD with quantization. The evaluation results show that LASER achieves more than

2.3\times

decoding speedup over previous work while preserving strong accuracy under low-precision inference.

Momentum

Related topics

Latest papers 48