Low-Rank Matrix Decomposition

Ask about this topic

Momentum

8 papers in the last four weeks, against 1 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 47

All topics
CardsList

  1. Shared Low-rank Basis Factorization for Data-free Mixture-of-Experts Compression

    Oct 7, 2026Tianxiao Cao, Jiahe Shao, Yuning Qiu +3LLM CompressionMixture-of-Experts Language Models

    Kyoto University · The University of Tokyo · RIKEN AIP

  2. Learning Functional Subspaces for Neural Network Compression

    Sep 30, 2026Massimo Bini, Anders Christensen, Stephan Alaniz +5LLM CompressionLow-Rank Matrix Decomposition

    Helmholtz Munich · Technical University of Munich · MCML +6

  3. MoRE: Scaling mixture of experts with hardware-aware low-rank routing

    Sep 28, 2026Honam Wong, Surbhi Goel, Enric Boix-AdseràParameter-Free Mixture-of-Experts RoutingLow-Rank Matrix Decomposition

    University of Pennsylvania · The Wharton School, University of Pennsylvania

  4. Singularities of Non-negative Matrix Factorization and their application to Bayesian inference

    Sep 28, 2026Naoki Hayashi, Yota Maeda, Yasushi EsakiNon-Negative Matrix FactorizationLow-Rank Matrix Decomposition

    Toyota Central R&D Labs., Inc.

  5. Conditional Tensor Diffusion: Distributional Counterfactual Learning and Inference

    Sep 22, 2026Xinbing Kong, Zeyu Li, Junfan Mao +1Causal Counterfactual GenerationLow-Rank Matrix Decomposition

    Southeast University · University of Science and Technology of China

  6. Provable Guarantees and Efficient Learning of Structural Equation Models with Latent Confounders

    Sep 16, 2026Weijian Yu, Jean HonorioStructural Causal ModelsLow-Rank Matrix Decomposition

    CIS, The University of Melbourne

  7. Per-Matrix Optimality Is Not Enough: Three-Level Optimization for Low-Rank LLM Compression

    Sep 14, 2026Huicheng Zhang, Xiyao Feng, Ze-Tong Li +6Model CompressionLLM Compression

    Hong Kong University of Science and Technology (Guangzhou), Guangdong 511453, China · QudeLeap Research, Shanghai 200030, China

  8. Pre-Trained Low-Rank Tensor Decomposition for Multi-Dimensional Image Recovery

    Sep 14, 2026Bing-Zhang Fu, Zhi-Long Han, Ting-Zhu Huang +2Low-Rank Matrix DecompositionTensor Decomposition

    School of Mathematical Sciences, University of Electronic Science and Technology of China, Chengdu 611731, China · School of Mathematics and Statistics and Ministry of Education Key Lab of Intelligent Networks and Network Security, Xi’an Jiaotong University, Xi’an, Shaanxi 710049, China · Pazhou Laboratory (Huangpu), Guangzhou, Guangdong 510335, China

  9. Hyperspectral Anomaly Detection via Group Sparse Low-Rank Tensor Factorization With Automatic Anomaly Grouping

    Sep 8, 2026Quan Yu, Yu-Hong Dai, Xiongjun ZhangLow-Rank Matrix DecompositionTensor Decomposition

    School of Mathematics and Statistics, Central China Normal University, Wuhan 430079, China, and also with the Key Laboratory of Nonlinear Analysis and Applications (Ministry of Education), Central China Normal University · State Key Laboratory of Mathematical Sciences, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing 100190, China, and also with the School of Mathematical Sciences, University of Chinese Academy of Sciences, Beijing 100049, China

  10. Hybrid Gated Attention

    Aug 12, 2026Zekun Zhou, Ruobing Xie, Lanrui Wang +1Low-Rank Matrix DecompositionGated Attention

    Tencent Hunyuan · Peking University

  11. Exact Rank-Space KL Projection for Shared-Marginal Low-Rank Factors: Application to Doubly Stochastic Clustering

    Aug 9, 2026Enliang HuGraph Structure LearningLow-Rank Matrix Decomposition

  12. Shape Mutating Expert Compression:LorExperts and BTExperts

    Aug 7, 2026Inesh Chakrabarti, Sourjya Roy, Bowen Bao +3Mixture-of-Experts Language ModelsLow-Rank Matrix Decomposition

    Advanced Micro Devices

  13. The Loss Does Not See the Basis, but Adam Does

    Aug 5, 2026Devender SinghLow-Rank Matrix DecompositionMatrix Optimization

    Department of Computer Science Memorial University of Newfoundland

  14. FraQ: Efficient Coordinate-Space Recompression for Federated Low-Rank Adaptation

    Aug 4, 2026Shenghui Li, Thiemo VoigtFederated Learning AggregationCommunication-Efficient Distributed Training

    Uppsala University, Uppsala, Sweden · Research Institutes of Sweden, Stockholm, Sweden

  15. SAKI: Score-Aware Low-Rank Key Indexing with Random-Matrix Noise Correction for KV Retrieval

    Aug 4, 2026Lin ZhangLow-Rank AttentionKV Caching

    Data Science and Platform, Cotality

  16. Low-Rank Dependence Decomposition via Accelerated Symmetric Non-negative Matrix Factorization

    Jul 27, 2026Lavinia Ghita, Dhruv Desai, Jake Goldberg +1Non-Negative Matrix FactorizationLow-Rank Matrix Decomposition

    NVIDIA

  17. Variational Low-rank Tensor Decomposition for Multisubject Spatiotemporal Data Analysis

    Jul 24, 2026Laura M. Montaldo, Ricardo A. Borsoi, Sebastian Miron +1Latent Dynamics ModelingLow-Rank Matrix Decomposition

    CRAN, Université de Lorraine, CNRS, Vandoeuvre-l`es-Nancy, France. · Department of CSEE, University of Maryland Baltimore County, Baltimore, MD 21250, USA.

  18. An Efficient Newton Algorithm for Nonnegative Matrix Factorization with the Kullback-Leibler Divergence

    Jul 15, 2026Damien Lesens, Jérémy E. Cohen, Bora UçarNon-Negative Matrix FactorizationLow-Rank Matrix Decomposition

    ENS de Lyon · Univ. Lyon, INSA-Lyon, UCBL, UJM, CNRS, Inserm, CREATIS UMR 5220, U1294, F-69621, Lyon, France. · CNRS and LIP (UMR5668, Université de Lyon - ENS de Lyon - UCBL - CNRS - Inria), 46 allée d'Italie, Lyon, France.

  19. A JoLT for the KV cache: Near-Lossless KV Cache Compression via Joint Rank-bit Allocation

    Jul 14, 2026Rahul Krishnan, Volker SchulzKV CachingLow-Rank Matrix Decomposition

    Universität Trier · Fachbereich IV, Mathematik, Universität Trier

  20. SLORR: Simple and Efficient In-Training Low-Rank Regularization

    Jul 9, 2026David González-Martínez, Shiwei LiuLLM CompressionLow-Rank Matrix Decomposition

    Max Planck Institute for Intelligent Systems · University of Tübingen · ELLIS Institute Tübingen +1

  21. LACE-SVD: Loss-Aware SVD with Cumulative Error Correction for LLM Compression

    Jul 3, 2026Zhuowen Liu, Longkun Hao, Shiyu Feng +3Model CompressionLLM Compression

    Zhuowen Liu1 · Longkun Hao2 · Shiyu Feng3 +3

  22. CORA: Per-Slice Coherent Orthogonal Rotation for SVD-based Low-Rank Adaptation

    Jun 30, 2026Pengcheng Wang, Ziran Liu, Wei Wang +1LLM Fine-TuningLow-Rank Matrix Decomposition

    Purdue University · Shanghai Institute for Mathematics and Interdisciplinary Sciences / Fudan University · Futurewei Technologies, Inc.

  23. Direction-Magnitude Decomposition for Low-Rank Matrix Optimization: Faster Convergence and Saddle-to-saddle Dynamics

    Jun 30, 2026Yudong Wei, Liang Zhang, Bingcong Li +1Low-Rank Matrix DecompositionNonconvex Optimization

    Department of Mathematics, ETH Zurich, Zurich, Switzerland · Department of Computer Science, ETH Zurich, Zurich, Switzerland

  24. DLR: Zero-Inference-Cost Latent Residuals for Low-Rank Pre-Training

    Jun 27, 2026Dong Wang, Wenwu Tang, Yun Cheng +1Language Model PretrainingLLM Compression

    Graz University of Technology, Austria · Swiss Data Science Center, Switzerland

  25. CLEAR-MoE: Shared-Basis Expert Extraction from Frozen Vision Transformers via Calibration-Driven Layer Selection

    Jun 26, 2026Md Irtiza Hossain, Humaira Ayesha, Junaid Ahmed SifatVision TransformerLow-Rank Matrix Decomposition

    Brac University

  26. SVD-Surgeon: Optimal Singular-Value Surgery for Large Language Model Compression

    Jun 22, 2026Mahmoud Safari, Frank HutterLLM PruningLLM Compression

    University of Freiburg · Prior Labs · ELLIS Institute Tübingen

  27. UniRank: Unified Rank Allocation for Low-Rank LLM Compression

    Jun 20, 2026Chao Han, Yongjie Du, Junjie Tan +1LLM CompressionLow-Rank Matrix Decomposition

    Ningbo Institute of Digital Twin, Eastern Institute of Technology, Ningbo

  28. Activation- and Influence-Aware Ranks (AIR): Function-Preserving SVD Compression for LLMs

    Jun 18, 2026Nico Harder, Daniel Becking, Karsten Mueller +1LLM CompressionLow-Rank Matrix Decomposition

    Fraunhofer HHI, Berlin, Germany

  29. STAR-KV: Low-Rank KV Cache Compression via Soft Thresholding for Adaptive Rank Control

    Jun 7, 2026Priyansh Bhatnagar, Ashkan Moradifirouzabadi, Se-Hyun Yang +3Mixed-Precision QuantizationKV Caching

    University of California San Diego, San Diego, CA, USA · Dnotitia Inc., Seoul, Republic of Korea · Hanyang University, Seoul, Republic of Korea

  30. SigmaScale: LLM Compression with SVD-based Low-Rank Decomposition and Learned Scaling Matrices

    Jun 5, 2026Ernests Lavrinovics, Marco Letizia, Roy Janco +3Model CompressionLLM Compression

    Department of Computer Science, Aalborg University Copenhagen, Denmark · MaLGa-DIBRIS, University of Genoa, Genoa, Italy · INFN, Sezione di Genova, Genoa, Italy +2

  31. LASER: Loss-Aware Singular-value Decomposition and Rank Allocation for Efficient Low-Precision Vision-Language Models

    May 30, 2026Haiyu Wang, Yutong Wang, Leshu Li +2Vision-Language ModelsEfficient VLM Inference

    Tandon School of Engineering, New York University · Courant Institute of Mathematical Sciences, New York University · Brookhaven National Laboratory

  32. Manifold-based Algorithms for the Hadamard Decomposition

    May 27, 2026Nicolas Gillis, Subhayan Saha, Stefano Sicilia +1Low-Rank Matrix DecompositionLow-Rank Approximation

    Department of Mathematics and Operational Research, University of Mons, Mons, Belgium. · SS is a member of the Gruppo Nazionale Calcolo Scientifico-Istituto Nazionale di Alta Matematica (GNCS-INdAM).

  33. Implicit Regularization in Perturbed Deep Matrix Factorization: Spectral Conditions and Stability

    May 27, 2026Jingzhe Wang, Hung-Hsu ChouAlgorithmic StabilityLow-Rank Matrix Decomposition

    Department of Informatics and Networked Systems, University of Pittsburgh · Department of Mathematics, University of Pittsburgh

  34. Efficient Pre-Training of LLMs through Truncated SVD Representations

    May 27, 2026Kaivan Kamali, Kajetan Schweighofer, Hormoz Shahrzad +3Language Model PretrainingLow-Rank Matrix Decomposition

    Cognizant AI Lab

  35. Group-Aware Matrix Estimation and Latent Subspace Recovery

    May 19, 2026Hamza Golubovic, Matthew Shen, Genevera I. Allen +1Low-Rank Matrix DecompositionNuclear Norm Minimization

    Department of Statistics, Columbia University New York, NY 10027 · Department of Statistics, Irving Institute for Cellular Dynamics, Zuckerman Institute Columbia University New York, NY 10027 · School of Data and Information Sciences, University of North Carolina at Chapel Hill Chapel Hill, NC 27599

  36. An Exterior Method for Nonnegative Matrix Factorization

    May 19, 2026Qiujing Lu, Tonmoy Monsoor, Ehsan Ebrahimzadeh +2Non-Negative Matrix FactorizationLow-Rank Matrix Decomposition

    1ECE, UCLA · 2eBay Search Science Team

  37. IO-SVD: Input-Output Whitened SVD for Adaptive-Rank LLM Compression

    May 15, 2026Ali Abbasi, Chayne Thrash, Haoran Qin +2LLM CompressionLow-Rank Matrix Decomposition

    Vanderbilt University · University of California, Davis

  38. Accelerating Power Method with Fast Sketching for Stronger Low-Rank Approximation

    May 10, 2026Shabarish Chenakkod, Michał DerezińskiLow-Rank Matrix DecompositionRandomized Sketching

    University of Michigan, Ann Arbor, MI 48105.

  39. Different Prompts, Different Ranks: Prompt-aware Dynamic Rank Selection for SVD-based LLM Compression

    May 9, 2026Hengyi Zhu, Zhendong Mi, Grace Li Zhang +1LLM CompressionLow-Rank Matrix Decomposition

    1Stevens Institute of Technology, Hoboken, USA · 2Technical University of Darmstadt, Darmstadt, Germany

  40. Don't Learn the Shape: Forecasting Periodic Time Series by Rank-1 Decomposition

    May 8, 2026Takato HondaLow-Rank Matrix DecompositionTime Series Forecasting

    Mellon Inc.

  41. Structural Correspondence and Universal Approximation in Diagonal plus Low-Rank Neural Networks

    May 7, 2026Ying Chen, Aoxi Li, Jihun Kim +1Low-Rank Matrix DecompositionLow-Rank Approximation

    Department of Industrial Engineering & Operations Research University of California, Berkeley, CA 94720

  42. Importance-Guided Basis Selection for Low-Rank Decomposition of Large Language Models

    May 2, 2026Daniel Agyei Asante, Ernie Chang, Yang LiLLM PruningLLM Compression

    Iowa State University · Meta

  43. Hankel and Toeplitz Rank-1 Decomposition of Arbitrary Matrices with Applications to Signal Direction-of-Arrival Estimation

    Apr 29, 2026Georgios I. OrfanidisLow-Rank Matrix DecompositionDirection-of-Arrival Estimation

    Center for Connected Autonomy and AI and the Dept. of Electrical Engineering and Computer Science, Florida Atlantic University, Boca Raton, FL 33431 USA

  44. State-Dependent Lyapunov Analysis of Rank-1 Matrix Factorization

    Apr 28, 2026Jaehong MoonEdge of StabilityLyapunov Stability

    Industrial & Enterprise Systems Engineering University of Illinois at Urbana-Champaign Urbana, IL 61801

  45. Predicting LLM Compression Degradation from Spectral Statistics

    Apr 20, 2026Mingxue XuLLM CompressionLow-Rank Matrix Decomposition

    Department of Electrical and Electronic Engineering, Imperial College London, London, United Kingdom

  46. LASER: Low-Rank Activation SVD for Efficient Recursion

    Apr 19, 2026Ege Çakar, Ketan Ali Raghu, Lia ZhengLow-Rank Matrix DecompositionRecurrent Neural Networks

    Harvard University Cambridge, MA, USA

  47. Stabilizing Native Low-Rank LLM Pretraining

    Feb 12, 2026Paul Janson, Edouard Oyallon, Eugene BelilovskyLanguage Model PretrainingLow-Rank Matrix Decomposition

    Concordia University, Montreal, Canada · Mila Quebec AI Institute, Montreal, Canada · Sorbonne University, CNRS, Paris, France

  48. BALF: Budgeted Activation-Aware Low-Rank Factorization for Fine-Tuning-Free Model Compression

    Sep 29, 2025David González-MartínezLow-Rank Matrix DecompositionLow-Rank Compression

    Department of Computer Science and Artificial Intelligence, Universidad de Sevilla · Tübingen AI Center