Low-Rank Compression

Ask about this topic

Momentum

12 papers in the last four weeks, with none the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 58

All topics
CardsList

  1. The Functional Structure of Post-Compression Recovery in Low-Rank LLMs

    Oct 4, 2026Zishan Shao, Liang Tian, Georgiy Zemlevskiy +10LLM CompressionLow-Rank Compression

    Duke University · Wake Forest University · Carnegie Mellon University +2

  2. Sequential Functional Structured Tucker Compression for Large Language Model Attentions

    Sep 30, 2026Jiangfeng Chen, Xinyu Wang, Tianshuo Yan +4LLM CompressionDecoder-Only Language Models

    University of Manitoba · McGill University · Simpleway +2

  3. Learning Functional Subspaces for Neural Network Compression

    Sep 30, 2026Massimo Bini, Anders Christensen, Stephan Alaniz +5LLM CompressionLow-Rank Matrix Decomposition

    Helmholtz Munich · Technical University of Munich · MCML +6

  4. Beyond Uniform Compression: Budgeted Transmission Allocation for Extreme Federated Learning

    Sep 30, 2026Pengfei Li, Mohammad KhalilCommunication-Efficient Distributed TrainingLow-Rank Compression

    Centre for the Science of Learning & Technology (SLATE), University of Bergen, Bergen, Norway

  5. KV-Kaizen: Learning Context-Adaptive Cache Compression Choices

    Sep 29, 2026Joao Monteiro, Louis Béthune, Anastasiia Filippova +3LLM Inference AccelerationLow-Rank Compression

    Apple

  6. Pre-registered tests of solid-state-physics-inspired LLM compression: a cluster-level negative result at small-language-model scale

    Sep 28, 2026Jun-qiang LuTensor NetworksLLM Compression

    Department of Physics, University of Puerto Rico, Mayagüez, PR 00680, USA

  7. MILO: Efficient Many-shot In-Context Learning with Block-wise Low-rank Compression

    Sep 24, 2026Youpeng Zhao, Tian Tan, Liqian Peng +2Memory-Efficient InferenceLLM Inference Acceleration

    Google · University of Central Florida

  8. Low-Rank Friction for Memory-Efficient Transformer Pretraining

    Sep 24, 2026Rajit Rajpal, Benedict LeimkuhlerDeep Learning OptimizationLow-Rank Compression

    School of Mathematics University of Edinburgh Edinburgh, UK EH9 3FD

  9. GeoPair: Geometry-Preserving Cross-Layer Factorization for Training-Free Transformer Compression

    Sep 22, 2026Baher Mohammad, Ammar Ali, Stamatios LefkimmiatisModel CompressionLow-Rank Compression

    MWS AI, ITMO University · MWS AI

  10. KV-COBRA: KV Cache Compression via Co-Optimized Bit-Rank Allocation

    Sep 21, 2026Sihyeon Ha, Jaeho Lee, Yo-Seb JeonLow-Rank CompressionKV-Cache Compression

    Pohang University of Science and Technology (POSTECH)

  11. Per-Matrix Optimality Is Not Enough: Three-Level Optimization for Low-Rank LLM Compression

    Sep 14, 2026Huicheng Zhang, Xiyao Feng, Ze-Tong Li +6Model CompressionLLM Compression

    Hong Kong University of Science and Technology (Guangzhou), Guangdong 511453, China · QudeLeap Research, Shanghai 200030, China

  12. Mind the Approximation: Fisher-Weighted SVD Compression for ViTs

    Sep 7, 2026Moritz Thoma, Maximilian Groezinger, Maximilian Forstenhäusler +7Efficient ViTsLow-Rank Compression

    Technical University of Munich, DE · BMW Group, DE · University of Glasgow, GB +1

  13. Understanding Calibration and Truncation Error Propagation in Training-Free Low-Rank Compression for LLMs

    Aug 9, 2026Mohanad Odema, Gabrielle De Micheli, Dayin Gou +3Model CompressionLLM Compression

    LG Electronics, North America

  14. Shape Mutating Expert Compression:LorExperts and BTExperts

    Aug 7, 2026Inesh Chakrabarti, Sourjya Roy, Bowen Bao +3Mixture-of-Experts Language ModelsLow-Rank Matrix Decomposition

    Advanced Micro Devices

  15. FraQ: Efficient Coordinate-Space Recompression for Federated Low-Rank Adaptation

    Aug 4, 2026Shenghui Li, Thiemo VoigtFederated Learning AggregationCommunication-Efficient Distributed Training

    Uppsala University, Uppsala, Sweden · Research Institutes of Sweden, Stockholm, Sweden

  16. Pin Once, Swap Light: Subspace-Aligned Centroid-Residual Training for Efficient Ultra-LoRA Serving

    Aug 4, 2026Xiang Li, Pengcheng Wang, Huazheng Wang +1LLM Inference EfficiencyLLM Serving

    1Purdue University · 2Oregon State University

  17. SAKI: Score-Aware Low-Rank Key Indexing with Random-Matrix Noise Correction for KV Retrieval

    Aug 4, 2026Lin ZhangLow-Rank AttentionKV Caching

    Data Science and Platform, Cotality

  18. ARCHead: Activation-Metric Residual Correction for Large Language Model Output Heads

    Aug 3, 2026Şuayp Talha Kocabay, Talha Rüzgar Akkuş, Kamer Ali YukselLLM QuantizationLLM Compression

    Independent Researcher · aiXplain, Inc.

  19. S4^4R: Selective Sampling, Subspaces, and Sparse Reconstruction for Compressed Long-Context KV Caching

    Aug 1, 2026Jialong Han, You Wu, Kewei TuKV CachingLong-Context Language Model Inference

    School of Information Science and Technology, ShanghaiTech University · Shanghai Engineering Research Center of Intelligent Vision and Imaging

  20. Fidelity Is Not Safety: Gently-Compressed LLMs Pass Every Data-Free Quality Guard Yet Invent Procedure Steps in Agentic Execution

    Jul 30, 2026I. Kennedy, T. KennedyModel CompressionAI Agent Reliability

  21. A JoLT for the KV cache: Near-Lossless KV Cache Compression via Joint Rank-bit Allocation

    Jul 14, 2026Rahul Krishnan, Volker SchulzKV CachingLow-Rank Matrix Decomposition

    Universität Trier · Fachbereich IV, Mathematik, Universität Trier

  22. CARE-LoRA: Compressed Activation REconstruction for Memory-Efficient LoRA

    Jul 11, 2026Gengyu Zhang, Haiyin Ran, Zhengbao He +4Fine-TuningMemory-Efficient Fine-Tuning

    Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University, Shanghai 200240, China

  23. SLORR: Simple and Efficient In-Training Low-Rank Regularization

    Jul 9, 2026David González-Martínez, Shiwei LiuLLM CompressionLow-Rank Matrix Decomposition

    Max Planck Institute for Intelligent Systems · University of Tübingen · ELLIS Institute Tübingen +1

  24. DepthWeave-KV: Token-Adaptive Cross-Layer Residual Factorization for Long-Context KV Cache Compression

    Jul 7, 2026Anna Cordoba, Adam Puente Tercero, Nerea Angulo Hijo +4KV CachingLong-Context Language Model Inference

    Instituto de Investigación en Visión Artificial

  25. SAD-LoRA: Spectral Alignment for Low-Rank Knowledge Distillation

    Jul 5, 2026Omer Tariq, Syed Muhammad Raza, Jeongbae SonLow-Rank AdaptationLow-Rank Compression

    Neubility Inc, Seoul, South Korea

  26. LACE-SVD: Loss-Aware SVD with Cumulative Error Correction for LLM Compression

    Jul 3, 2026Zhuowen Liu, Longkun Hao, Shiyu Feng +3Model CompressionLLM Compression

    Zhuowen Liu1 · Longkun Hao2 · Shiyu Feng3 +3

  27. DLR: Zero-Inference-Cost Latent Residuals for Low-Rank Pre-Training

    Jun 27, 2026Dong Wang, Wenwu Tang, Yun Cheng +1Language Model PretrainingLLM Compression

    Graz University of Technology, Austria · Swiss Data Science Center, Switzerland

  28. SVD-Surgeon: Optimal Singular-Value Surgery for Large Language Model Compression

    Jun 22, 2026Mahmoud Safari, Frank HutterLLM PruningLLM Compression

    University of Freiburg · Prior Labs · ELLIS Institute Tübingen

  29. UniRank: Unified Rank Allocation for Low-Rank LLM Compression

    Jun 20, 2026Chao Han, Yongjie Du, Junjie Tan +1LLM CompressionLow-Rank Matrix Decomposition

    Ningbo Institute of Digital Twin, Eastern Institute of Technology, Ningbo

  30. Activation- and Influence-Aware Ranks (AIR): Function-Preserving SVD Compression for LLMs

    Jun 18, 2026Nico Harder, Daniel Becking, Karsten Mueller +1LLM CompressionLow-Rank Matrix Decomposition

    Fraunhofer HHI, Berlin, Germany