Low-Rank Matrix Decomposition

Ask about this topic

Overview

Low-rank matrix decomposition approximates a matrix using a small set of latent factors, revealing shared structure and reducing storage or computation. Common approaches include singular value decomposition, matrix and tensor factorization, robust principal component analysis, and matrix completion, with rank chosen to balance simplicity and fidelity. Applications range from recovering missing or corrupted data to compressing language and vision models, where results are assessed with measures such as reconstruction error, perplexity, and downstream accuracy. A central challenge is that the best rank and factorization depend on the data and task: compact models can be faster and smaller, but approximation errors may compound and harm performance.

Latest papers

CardsList

63 papers11 papers in the last four weeks, up 120% on the four weeks before. 0.1% of all new papers.

  1. Advancing SVD-based LLM Compression via Layer-Wise Error Model Search

    Oct 23, 2026Moritz Thoma, Maximilian Groezinger, Maximilian Forstenhäusler +8LLM CompressionLow-Rank Compression

    Technical University of Munich, Germany · BMW Group, Germany · University of Glasgow, United Kingdom +1

  2. Singular Geometry of Low-Rank Adaptation

    Oct 23, 2026Kamillo Ferry, Anthea Monod, Keisuke YanoLow-Rank AdaptationSingular Learning Theory

    Department of Mathematics, Imperial College London, UK · Institute of Statistical Mathematics, Japan

  3. Robust Latent Variable Graphical Models: Exact Recovery and Fundamental Limits

    Oct 19, 2026Chuansen Peng, Xiaojing ShenRobust EstimationGraph Structure Learning

    School of Mathematics, Sichuan University, Chengdu 610064, China

  4. Over-parameterization Is Neither Robust nor Fragile in Matrix Sensing: A Geometric Theory of Solution Robustness

    Oct 13, 2026Runze Shi, Ziye MaOverparameterizationMatrix Factorization

    City University of Hong Kong

  5. GA-SVD: Generation-aware Singular Value Decomposition for LLM Compression

    Oct 11, 2026Shengkun Tang, Cong Zeng, Zongfang Liu +2LLM CompressionLow-Rank Compression

    MBZUAI · Technology Innovation Institute UAE · Zhejiang Univeristy +2

  6. D-SLR: The Disjoint Row-Sparse plus Low-Rank Decomposition

    Oct 7, 2026Vincent SzolnokyLow-Rank Matrix DecompositionLow-Rank Compression

    Department of Mathematical Sciences Chalmers University of Technology

  7. Shared Low-rank Basis Factorization for Data-free Mixture-of-Experts Compression

    Oct 7, 2026Tianxiao Cao, Jiahe Shao, Yuning Qiu +3LLM CompressionMixture-of-Experts Language Models

    Kyoto University · The University of Tokyo · RIKEN AIP

  8. Learning Functional Subspaces for Neural Network Compression

    Sep 30, 2026Massimo Bini, Anders Christensen, Stephan Alaniz +5LLM CompressionLow-Rank Matrix Decomposition

    Helmholtz Munich · Technical University of Munich · MCML +6

  9. MoRE: Scaling mixture of experts with hardware-aware low-rank routing

    Sep 28, 2026Honam Wong, Surbhi Goel, Enric Boix-AdseràParameter-Free Mixture-of-Experts RoutingLow-Rank Matrix Decomposition

    University of Pennsylvania · The Wharton School, University of Pennsylvania

  10. GEM-KMeans: Memory-Efficient and Accurate Clustering on Massive Scale with GPU Optimization

    Sep 28, 2026Peng Xu, Nihar Koganti, Volodymyr Kindratenko +1ClusteringNon-Negative Matrix Factorization

    Department of Statistics, University of Illinois Urbana-Champaign · Department of Electrical and Computer Engineering, University of Illinois Urbana-Champaign · National Center for Supercomputing Applications, University of Illinois Urbana-Champaign +3

  11. Singularities of Non-negative Matrix Factorization and their application to Bayesian inference

    Sep 28, 2026Naoki Hayashi, Yota Maeda, Yasushi EsakiNon-Negative Matrix FactorizationLow-Rank Matrix Decomposition

    Toyota Central R&D Labs., Inc.

  12. Conditional Tensor Diffusion: Distributional Counterfactual Learning and Inference

    Sep 22, 2026Xinbing Kong, Zeyu Li, Junfan Mao +1Causal Counterfactual GenerationLow-Rank Matrix Decomposition

    Southeast University · University of Science and Technology of China

  13. Provable Guarantees and Efficient Learning of Structural Equation Models with Latent Confounders

    Sep 16, 2026Weijian Yu, Jean HonorioStructural Causal ModelsLow-Rank Matrix Decomposition

    CIS, The University of Melbourne

  14. Per-Matrix Optimality Is Not Enough: Three-Level Optimization for Low-Rank LLM Compression

    Sep 14, 2026Huicheng Zhang, Xiyao Feng, Ze-Tong Li +6Model CompressionLLM Compression

    Hong Kong University of Science and Technology (Guangzhou), Guangdong 511453, China · QudeLeap Research, Shanghai 200030, China

  15. Pre-Trained Low-Rank Tensor Decomposition for Multi-Dimensional Image Recovery

    Sep 11, 2026Bing-Zhang Fu, Zhi-Long Han, Ting-Zhu Huang +2Low-Rank Matrix DecompositionTensor Decomposition

    School of Mathematical Sciences, University of Electronic Science and Technology of China, Chengdu 611731, China · School of Mathematics and Statistics and Ministry of Education Key Lab of Intelligent Networks and Network Security, Xi’an Jiaotong University, Xi’an, Shaanxi 710049, China · Pazhou Laboratory (Huangpu), Guangzhou, Guangdong 510335, China

  16. Hyperspectral Anomaly Detection via Group Sparse Low-Rank Tensor Factorization With Automatic Anomaly Grouping

    Sep 8, 2026Quan Yu, Yu-Hong Dai, Xiongjun ZhangLow-Rank Matrix DecompositionTensor Decomposition

    School of Mathematics and Statistics, Central China Normal University, Wuhan 430079, China, and also with the Key Laboratory of Nonlinear Analysis and Applications (Ministry of Education), Central China Normal University · State Key Laboratory of Mathematical Sciences, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing 100190, China, and also with the School of Mathematical Sciences, University of Chinese Academy of Sciences, Beijing 100049, China

  17. Subzero matrix completion for sparse data analysis: large-scale learning of latent low-rank structure

    Aug 21, 2026Lawrence K. Saul, Ningyuan Huang, Dennis Bollweg +2Low-Rank Matrix DecompositionMatrix Completion

    Center for Computational Mathematics, Flatiron Institute, New York, NY 10010 · Scientific Computing Core, Flatiron Institute, New York, NY 10010 · Courant Institute, NYU, New York, NY 10012

  18. PathFinder: Joint Decompositions of Linked Multimodal Datasets

    Aug 15, 2026Ying-Qiu Zheng, Alex Fung, Stephen M Smith +2Low-Rank Matrix DecompositionMatrix Factorization

    Oxford Centre for Integrative Neuroimaging University of Oxford Oxford, UK

  19. Hybrid Gated Attention

    Aug 12, 2026Zekun Zhou, Ruobing Xie, Lanrui Wang +1Low-Rank Matrix DecompositionGated Attention

    Tencent Hunyuan · Peking University

  20. Exact Rank-Space KL Projection for Shared-Marginal Low-Rank Factors: Application to Doubly Stochastic Clustering

    Aug 9, 2026Enliang HuGraph Structure LearningLow-Rank Matrix Decomposition

  21. Shape Mutating Expert Compression:LorExperts and BTExperts

    Aug 7, 2026Inesh Chakrabarti, Sourjya Roy, Bowen Bao +3Mixture-of-Experts Language ModelsLow-Rank Matrix Decomposition

    Advanced Micro Devices

  22. The Loss Does Not See the Basis, but Adam Does

    Aug 5, 2026Devender SinghLow-Rank Matrix DecompositionMatrix Optimization

    Department of Computer Science Memorial University of Newfoundland

  23. FraQ: Efficient Coordinate-Space Recompression for Federated Low-Rank Adaptation

    Aug 4, 2026Shenghui Li, Thiemo VoigtFederated Learning AggregationCommunication-Efficient Distributed Training

    Uppsala University, Uppsala, Sweden · Research Institutes of Sweden, Stockholm, Sweden

  24. SAKI: Score-Aware Low-Rank Key Indexing with Random-Matrix Noise Correction for KV Retrieval

    Aug 4, 2026Lin ZhangLow-Rank AttentionKV Caching

    Data Science and Platform, Cotality

  25. Low-Rank Dependence Decomposition via Accelerated Symmetric Non-negative Matrix Factorization

    Jul 27, 2026Lavinia Ghita, Dhruv Desai, Jake Goldberg +1Non-Negative Matrix FactorizationLow-Rank Matrix Decomposition

    NVIDIA

  26. Variational Low-rank Tensor Decomposition for Multisubject Spatiotemporal Data Analysis

    Jul 24, 2026Laura M. Montaldo, Ricardo A. Borsoi, Sebastian Miron +1Latent Dynamics ModelingFunctional Magnetic Resonance Imaging

    CRAN, Université de Lorraine, CNRS, Vandoeuvre-l`es-Nancy, France. · Department of CSEE, University of Maryland Baltimore County, Baltimore, MD 21250, USA.

  27. An Efficient Newton Algorithm for Nonnegative Matrix Factorization with the Kullback-Leibler Divergence

    Jul 15, 2026Damien Lesens, Jérémy E. Cohen, Bora UçarNon-Negative Matrix FactorizationLow-Rank Matrix Decomposition

    ENS de Lyon · Univ. Lyon, INSA-Lyon, UCBL, UJM, CNRS, Inserm, CREATIS UMR 5220, U1294, F-69621, Lyon, France. · CNRS and LIP (UMR5668, Université de Lyon - ENS de Lyon - UCBL - CNRS - Inria), 46 allée d'Italie, Lyon, France.

  28. A JoLT for the KV cache: Near-Lossless KV Cache Compression via Joint Rank-bit Allocation

    Jul 14, 2026Rahul Krishnan, Volker SchulzTucker DecompositionKV Caching

    Universität Trier · Fachbereich IV, Mathematik, Universität Trier

  29. SLORR: Simple and Efficient In-Training Low-Rank Regularization

    Jul 9, 2026David González-Martínez, Shiwei LiuLLM CompressionLow-Rank Matrix Decomposition

    Max Planck Institute for Intelligent Systems · University of Tübingen · ELLIS Institute Tübingen +1

  30. LACE-SVD: Loss-Aware SVD with Cumulative Error Correction for LLM Compression

    Jul 3, 2026Zhuowen Liu, Longkun Hao, Shiyu Feng +3Model CompressionLLM Compression

    Zhuowen Liu1 · Longkun Hao2 · Shiyu Feng3 +3