Overview

Low-rank compression represents large matrices and tensors with smaller factors, reducing storage and sometimes the computation needed for training or inference. Common approaches include truncated or randomized SVD, activation- and loss-aware factorization, adaptive rank allocation, and combinations with quantization or sparsity; some methods instead train models or optimizer states directly in low-rank form. Work spans vision and language models, with growing attention to long-context key-value caches, where benchmarks such as LongBench and RULER test whether compressed representations preserve retrieval and reasoning. A central tension is that low reconstruction error does not always preserve task behavior, and fewer parameters do not guarantee faster execution without suitable kernels and workload-aware evaluation.

Latest papers

CardsList

118 papers19 papers in the last four weeks, down 10% on the four weeks before. 0.1% of all new papers.

  1. TraceKV: Policy-Preserving Low-Rank KV Cache Compression for Reasoning Vision-Language Models

    Nov 11, 2026Ke Li, Zishan Shao, Yiran ChenKV-Cache CompressionEfficient VLM Inference

  2. Efficient LLM Adaptation with Forward-Only Passes

    Nov 7, 2026Baichuan Huang, Ananth Balashankar, Amir AminifarInference-Time Adaptation of Language ModelsLLM Fine-Tuning

  3. DictLLM: Post-training Compression of Large Language Model with Dictionary Kernels

    Nov 5, 2026Jinho Park, Se Young Chun, Mingoo SeokModel CompressionLLM Compression

  4. LowRAM: Low-Rank Keys and Amortised Values for Composable KV Cache Compression at Test-Time

    Nov 4, 2026Teresa Delgado, Hugo Hazard, Martin Benfeghoul +2KV-Cache CompressionMemory-Efficient Inference

  5. BasisKV: Value Latents for Communication and Key Routing

    Nov 2, 2026Lixun Zhang, Zishan Shao, Kangning Cui +3KV Cache SelectionKV-Cache Management

  6. QFacePlacement: From Arbitrary Tensorization to Semantic Spatial Correspondence in Modern Face Verification

    Nov 2, 2026Krishna Bhatia, Kavita Choudhary, Pruthwik Mishra +1Face RecognitionTensor Networks

  7. Nonreciprocal Spherical Attention and Retrieval: Sharp Coarsening Clocks and Certified Score Correction

    Nov 1, 2026Faruk Alpay, Taylan AlpayApproximate Nearest Neighbor SearchAttention Mechanisms

  8. LoRAForge: Co-Designing Post-Hoc Compression and Execution for Efficient Multi-LoRA Serving

    Oct 31, 2026Shenghui Li, Thiemo Voigtmulti-lora inferenceLLM Serving

  9. Hierarchical Low-Rank Reparameterization for Parameter-Efficient Fine-Tuning

    Oct 27, 2026Rong Huang, Yi Xiong, Yuan-Yuan Cheng +1LLM Fine-TuningParameter-Efficient Fine-Tuning

  10. LoRA on a Budget: Function-Aware Post-Hoc Compression for Adapter Fleets

    Oct 26, 2026Shenghui Li, Thiemo voigtLow-Rank CompressionAdaptive Rank Allocation

  11. Low-rank MERA Decomposition of 3D OCT Image Volume for Compression, Denoising, Feature Extraction and Diagnosis of Retinal Diseases

    Oct 26, 2026Ivica Kopriva, Fei Shi, Marija Štanfel +4Optical Coherence TomographyMedical Image Representation Learning

  12. Advancing SVD-based LLM Compression via Layer-Wise Error Model Search

    Oct 23, 2026Moritz Thoma, Maximilian Groezinger, Maximilian Forstenhäusler +8LLM CompressionLow-Rank Compression

  13. BiLoCo: Binary Low-Rank Corrections for Post-Training Quantization

    Oct 23, 2026David Jin, Beshr IslamBouli, Tarushii Goel +2LLM QuantizationPost-Training Quantization

  14. GS-MoE: Training-Free MoE Compression via Groupwise Subspace Sharing and Pruning

    Oct 23, 2026Carson Cary, Yukai Jin, Yihao Huang +5Mixture-of-Experts ModelsMixture-of-Experts Pruning

  15. Training-free Acceleration for Blind Compressed Image Restoration via Activation-Calibrated Singular Value Decomposition

    Oct 21, 2026Suhang Yao, Xin Li, Xijun Wang +3Inference AccelerationImage Restoration

  16. A KAN-based Non-linear Factorization for Embedding Layer Compression

    Oct 21, 2026Oliver Giudice, Anna Russo RussoEmbedding CompressionLow-Rank Compression

  17. LoRASpace: A Pool-Wide Shared Substrate for Static and Dynamic Multi-LoRA Composition

    Oct 20, 2026Thamirawaran Sathiyalogeswaran, Navindu De Silva, Chamikara Siriwardane +4Low-Rank AdaptationLLM Inference Acceleration

  18. BELT : BEhavior-Aligned Low-Rank KV Cache Compression with Learned Top-k Rank Allocation

    Oct 13, 2026Yaoyang Liu, Shuwei Shen, Jincheng Tang +5KV-Cache CompressionLow-Rank Compression

  19. GA-SVD: Generation-aware Singular Value Decomposition for LLM Compression

    Oct 11, 2026Shengkun Tang, Cong Zeng, Zongfang Liu +2LLM CompressionLow-Rank Compression

  20. LightVAE: Towards Compact and Efficient Video Autoencoders

    Oct 10, 2026Shiqiao Gu, Yang Yong, Fengjun Zhong +5Variational AutoencodersVideo Compression

  21. Spectral Weight Decay: Inducing Low-Rank Structure in Neural Network Weights

    Oct 8, 2026Dmitrii Andriianov, Andrey Veprikov, Aleksandr BeznosikovWeight DecayNuclear Norm Minimization

  22. D-SLR: The Disjoint Row-Sparse plus Low-Rank Decomposition

    Oct 7, 2026Vincent SzolnokyLow-Rank Matrix DecompositionLow-Rank Compression

  23. LRCC: Generalizing Low-Rank Compression with Conditional Computation

    Oct 5, 2026Thomas Vaitses Fontanari, Maximo Eduardo Rulli, Federico Alvetreti +2LLM CompressionLow-Rank Compression

  24. The Functional Structure of Post-Compression Recovery in Low-Rank LLMs

    Oct 4, 2026Zishan Shao, Liang Tian, Georgiy Zemlevskiy +10LLM CompressionLow-Rank Compression

  25. AttSVD:Prompt-Adaptive Low-Rank KV Cache Compression via Attention-Guided SVD

    Oct 3, 2026Sara Abdali, Jongwoo Ko, Pashmina CameronKV-Cache CompressionLow-Rank Compression

  26. Preserving Mathematical Reasoning in Compressed Diffusion Language Models via Trajectory-Aware Low-Rank Approximation

    Oct 2, 2026Tian Liang, Zishan Shao, Yiran ChenDiffusion Model CompressionLow-Rank Compression

  27. SlimKV: Joint Token-Feature KV Cache Compression with Reconstruction-Free Beacon Attention

    Oct 2, 2026Zihan Teng, Jiayu Zhao, Wentao Ren +4KV-Cache CompressionLLM Inference Acceleration

  28. iS-KV: Online Low-Rank KV Cache Compression via Block-Incremental SVD

    Oct 2, 2026Yiren Zhao, Guanghui Song, Tianrui Qin +3KV-Cache CompressionLow-Rank Compression