Low-Rank Approximation

Ask about this topic

Overview

Low-rank approximation studies how to represent data or computations using a small number of latent components, making tasks such as compression, matrix completion, recommendation, and scientific modeling more efficient. Common approaches include truncated or randomized singular value decomposition, tensor factorization, nuclear-norm regularization, and adaptive rank selection. In neural networks and language models, researchers use activation- or loss-aware decompositions to compress weights and attention caches, and low-rank parameterizations to reduce training costs. A central tension is that reconstruction error may not reflect downstream task quality: choosing ranks that preserve useful behavior while reducing memory and computation remains difficult, especially when structure varies across layers, inputs, or training steps.

Latest papers

CardsList

93 papers6 papers in the last four weeks, down 40% on the four weeks before. 0.0% of all new papers.

  1. Learnable Low-Rank Polynomial Sketch for Effective Linear Attention

    Oct 9, 2026Zongyue Qin, Haoran Deng, Jason Cong +1Transformer AttentionLinear Attention

    Department of Computer Science, University of California Los Angeles, CA, USA

  2. Low-rank tensor structure of precipitation and its application to satellite-reference merging

    Oct 7, 2026Ryan Solgi, Rohan Shankar, Hugo A. LoaicigaLow-Rank ApproximationRemote Sensing

    Department of Geography, University of California-Santa Barbara, CA, USA · Department of Electrical and Computer Engineering, University of California-Santa Barbara, CA, USA · Department of Computer Science, Cornell University, NY, USA

  3. D-SLR: The Disjoint Row-Sparse plus Low-Rank Decomposition

    Oct 7, 2026Vincent SzolnokyLow-Rank Matrix DecompositionLow-Rank Compression

    Department of Mathematical Sciences Chalmers University of Technology

  4. Information Limits of Low-Rank Approximation Certification

    Oct 2, 2026Kang Liu, Bohao QuLow-Rank ApproximationInformation-Theoretic Lower Bounds

    School of Future Technology, Xi’an Jiaotong University, Xi’an, China. · Centre for Frontier AI Research (CFAR), A*STAR, Singapore.

  5. DyRA: Dynamic Residual Approximation for Efficient Matrix Multiplication in DNNs

    Oct 2, 2026Daewon Chae, Hyunwon Chung, Changwoo Lee +1Efficient Neural Network InferenceLow-Rank Approximation

    Department of EECS, University of Michigan · Qualcomm AI Research

  6. Learning Conditional Expectation Operators via Functional Newton Updates

    Sep 28, 2026Thiago Ramos, Alek Fröhlich, Daniel Perazzo +1Density Ratio EstimationLow-Rank Approximation

    Federal University of São Carlos · CSML, Istituto Italiano di Tecnologia and University of Genoa · CSML, Istituto Italiano di Tecnologia and University College London

  7. Stabilizing the Dynamic Low-Rank Training

    Sep 26, 2026Zhonghan Xu, Ling Wang, Junhao Chen +2Low-Rank ApproximationNeural Network Training Dynamics

    School of Communications and Information Engineering University of Electronic Science and Technology of China

  8. When Does Dense Retrieval Need Asymmetric Geometry? A Bias-Variance Theory of Shared and Dual Projections

    Sep 26, 2026Maojun Sun, Yancheng Yuan, Jian Huang +1Dense RetrievalLow-Rank Approximation

    The Hong Kong Polytechnic University

  9. Dynamical low-rank equilibrium computation for stochastic games between advanced persistent threats and moving target defense

    Sep 20, 2026Tian Zijian, Zhang He, Chen Xinjie +2Nash Equilibrium ComputationZero-Sum Games

    College of Control Science and Engineering, Zhejiang University, PC 310027, China · Alibaba Group, Hangzhou, China

  10. Principled Koopman Representations with Kalman Inference for Efficient Time-Series Prediction

    Sep 15, 2026Ruiquan Li, Yuheng BuKoopman Operator LearningTime Series Forecasting

    Department of Computer Science, UC Santa Barbara

  11. MoARa: Module-Aware Rank Allocation and Structure-Preserving Decomposition for Low-Rank LLM Pre-training

    Sep 14, 2026Keunyoung Kim, Nojun KwakLanguage Model PretrainingLow-Rank Approximation

    Seoul National University

  12. On the Regularization Landscape for the Linear Recommendation Models

    Sep 10, 2026Dong Li, Zhenming Liu, Ruoming Jin +4Nuclear Norm MinimizationLow-Rank Approximation

    Kent State University, USA · College of William and Mary, USA · iLambda, USA

  13. Semi-Tensor Product-Based Multi-Term Randomized T-SVD and Its Visual Applications

    Sep 10, 2026Xingchen Xiao, Feng Zhang, Wenjin Qin +1Tensor CompletionTensor Decomposition

    School of Mathematics and Statistics, Southwest University, Chongqing, China

  14. A Block Tensor Train Burer-Monteiro Framework for Low-Rank Quantum State Tomography

    Sep 8, 2026Shakir Showkat Sofi, Charlotte Vermeylen, Fatemeh Mohammadi +1Quantum State TomographyTT Decomposition

    Leuven.AI - KU Leuven institute for AI, B-3000, Leuven, Belgium · Department of Electrical Engineering (ESAT), KU Leuven, Leuven, Belgium; Group Science, Engineering and Technology, KU Leuven Kulak, Kortrijk, Belgium · Departments of Computer Science and Mathematics, KU Leuven, Leuven, Belgium

  15. Two-Scale Localized PCA-Net: Coarse-Global and Local-Residual Representations for Artifact-Reduced PDE Operator Learning

    Sep 7, 2026Mrigank Dhingra, Jordan Stout, Omer SanPrincipal Component AnalysisReduced-Order Modeling

    Department of Mechanical & Aerospace Engineering University of Tennessee Knoxville, Tennessee 37917, USA · Department of Mathematics and Statistics Boston University Boston, Massachusetts 02215, USA

  16. The Gradient Does Not See Rank: Rank-Indifference in Matrix-CODI on ProsQA

    Sep 2, 2026Samuel LarsonCoT ReasoningLow-Rank Approximation

    Pebble ML

  17. Intrinsic Interaction Geometry Controls the Low-Rank Complexity of Softmax Attention

    Aug 28, 2026Yuhe Sui, Jianing Zhang, Yingzhi TangSoftmax AttentionLow-Rank Attention

    Nanyang Technological University, Singapore · Carnegie Mellon University

  18. Group-Shared Low-Rank Approximation for Mobile-Efficient Pointwise Convolutions in Large-Kernel CNNs

    Aug 26, 2026Hao Luo, Yiting Yang, Wenyi Zhao +10Convolutional Neural NetworksNeural Network Compression

    College of Computer and Information Engineering, Xi’an University of Architecture and Technology · Northwestern Polytechnical University · Nanchang University +4

  19. MaLViL: Multi-axis Low-rank Vision-LSTM for Medical Image Segmentation

    Aug 18, 2026Afshin Bozorgpour, Sina Ghorbani Kolahi, Moein Heidari +2Medical Image SegmentationUltrasound Image Segmentation

    Faculty of Informatics and Data Science, University of Regensburg, Germany · Independent Computer Science Researcher, Tehran, Iran · University of British Columbia, Vancouver, BC, Canada +1

  20. SAKI-RoPE: Score-Aware Low-Rank Key Indexing with RoPE-Aware Covariance and Random-Matrix Noise Analysis

    Aug 4, 2026Lin ZhangLow-Rank AttentionKV Caching

    Data Science and Platform, Cotality

  21. LOCKS: Page-Local Compact Key Summaries for Efficient Long-Context Decoding

    Jul 27, 2026Junsung HwangLow-Rank AttentionKV Caching

  22. DynaCalKV: Key-Value Cache Compression via Head Grouping and Adaptive Rank Allocation

    Jul 27, 2026Tan T. Nguyen, Quan V. DangKV CachingLow-Rank Approximation

    ∗Full Stack Data Science · †Department of Computer Science, University College London

  23. Leakage and Multiway Interaction Loss: What Matricization and Single-Mode Truncation Discard from Multiway Signals

    Jul 27, 2026Hiroki HasegawaTensor DecompositionLow-Rank Approximation

    Graduate School of Science and Technology, University of Tsukuba, Tsukuba, 305-8573, Japan

  24. The Entropic Bound for Transformers: Why Static Rank Fails and Attention-Native Rank Recovers

    Jul 25, 2026Byeong Hoon YoonSoftmax AttentionTransformer Expressivity

    Independent Researcher

  25. A JoLT for the KV cache: Near-Lossless KV Cache Compression via Joint Rank-bit Allocation

    Jul 14, 2026Rahul Krishnan, Volker SchulzTucker DecompositionKV Caching

    Universität Trier · Fachbereich IV, Mathematik, Universität Trier

  26. Graph-Regularized Low-Rank Matrix Completion by Variable Projection

    Jul 10, 2026Benoît Loucheur, P. -A. Absil, Michel JournéeLow-Rank ApproximationMatrix Optimization

    ICTEAM Institute, UCLouvain, 1348 Louvain-la-Neuve, Belgium. · Department of Climatology, Royal Meteorological Institute of Belgium, 1180 Uccle, Belgium.

  27. SLORR: Simple and Efficient In-Training Low-Rank Regularization

    Jul 9, 2026David González-Martínez, Shiwei LiuLLM CompressionLow-Rank Matrix Decomposition

    Max Planck Institute for Intelligent Systems · University of Tübingen · ELLIS Institute Tübingen +1

  28. No Subspace to Track: Non-Identifiability and Optimizer State in Low-Rank Training

    Jul 7, 2026Noel ThomasDeep Learning OptimizationParameter Identifiability

    Mohamed bin Zayed University of Artificial Intelligence

  29. LACE-SVD: Loss-Aware SVD with Cumulative Error Correction for LLM Compression

    Jul 3, 2026Zhuowen Liu, Longkun Hao, Shiyu Feng +3Model CompressionLLM Compression

    Zhuowen Liu1 · Longkun Hao2 · Shiyu Feng3 +3

  30. SVD-Surgeon: Optimal Singular-Value Surgery for Large Language Model Compression

    Jun 22, 2026Mahmoud Safari, Frank HutterLLM PruningLLM Compression

    University of Freiburg · Prior Labs · ELLIS Institute Tübingen