Low-Rank Compression

Ask about this topic

Momentum

12 papers in the last four weeks, with none the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 58

All topics
CardsList

  1. The Functional Structure of Post-Compression Recovery in Low-Rank LLMs

    Oct 4, 2026Zishan Shao, Liang Tian, Georgiy Zemlevskiy +10LLM CompressionLow-Rank Compression

    Duke University · Wake Forest University · Carnegie Mellon University +2

  2. Sequential Functional Structured Tucker Compression for Large Language Model Attentions

    Sep 30, 2026Jiangfeng Chen, Xinyu Wang, Tianshuo Yan +4LLM CompressionDecoder-Only Language Models

    University of Manitoba · McGill University · Simpleway +2

  3. Learning Functional Subspaces for Neural Network Compression

    Sep 30, 2026Massimo Bini, Anders Christensen, Stephan Alaniz +5LLM CompressionLow-Rank Matrix Decomposition

    Helmholtz Munich · Technical University of Munich · MCML +6

  4. Beyond Uniform Compression: Budgeted Transmission Allocation for Extreme Federated Learning

    Sep 30, 2026Pengfei Li, Mohammad KhalilCommunication-Efficient Distributed TrainingLow-Rank Compression

    Centre for the Science of Learning & Technology (SLATE), University of Bergen, Bergen, Norway

  5. KV-Kaizen: Learning Context-Adaptive Cache Compression Choices

    Sep 29, 2026Joao Monteiro, Louis Béthune, Anastasiia Filippova +3LLM Inference AccelerationLow-Rank Compression

    Apple

  6. Pre-registered tests of solid-state-physics-inspired LLM compression: a cluster-level negative result at small-language-model scale

    Sep 28, 2026Jun-qiang LuTensor NetworksLLM Compression

    Department of Physics, University of Puerto Rico, Mayagüez, PR 00680, USA

  7. MILO: Efficient Many-shot In-Context Learning with Block-wise Low-rank Compression

    Sep 24, 2026Youpeng Zhao, Tian Tan, Liqian Peng +2Memory-Efficient InferenceLLM Inference Acceleration

    Google · University of Central Florida

  8. Low-Rank Friction for Memory-Efficient Transformer Pretraining

    Sep 24, 2026Rajit Rajpal, Benedict LeimkuhlerDeep Learning OptimizationLow-Rank Compression

    School of Mathematics University of Edinburgh Edinburgh, UK EH9 3FD

  9. GeoPair: Geometry-Preserving Cross-Layer Factorization for Training-Free Transformer Compression

    Sep 22, 2026Baher Mohammad, Ammar Ali, Stamatios LefkimmiatisModel CompressionLow-Rank Compression

    MWS AI, ITMO University · MWS AI

  10. KV-COBRA: KV Cache Compression via Co-Optimized Bit-Rank Allocation

    Sep 21, 2026Sihyeon Ha, Jaeho Lee, Yo-Seb JeonLow-Rank CompressionKV-Cache Compression

    Pohang University of Science and Technology (POSTECH)

  11. Per-Matrix Optimality Is Not Enough: Three-Level Optimization for Low-Rank LLM Compression

    Sep 14, 2026Huicheng Zhang, Xiyao Feng, Ze-Tong Li +6Model CompressionLLM Compression

    Hong Kong University of Science and Technology (Guangzhou), Guangdong 511453, China · QudeLeap Research, Shanghai 200030, China

  12. Mind the Approximation: Fisher-Weighted SVD Compression for ViTs

    Sep 7, 2026Moritz Thoma, Maximilian Groezinger, Maximilian Forstenhäusler +7Efficient ViTsLow-Rank Compression

    Technical University of Munich, DE · BMW Group, DE · University of Glasgow, GB +1

  13. Understanding Calibration and Truncation Error Propagation in Training-Free Low-Rank Compression for LLMs

    Aug 9, 2026Mohanad Odema, Gabrielle De Micheli, Dayin Gou +3Model CompressionLLM Compression

    LG Electronics, North America

  14. Shape Mutating Expert Compression:LorExperts and BTExperts

    Aug 7, 2026Inesh Chakrabarti, Sourjya Roy, Bowen Bao +3Mixture-of-Experts Language ModelsLow-Rank Matrix Decomposition

    Advanced Micro Devices

  15. FraQ: Efficient Coordinate-Space Recompression for Federated Low-Rank Adaptation

    Aug 4, 2026Shenghui Li, Thiemo VoigtFederated Learning AggregationCommunication-Efficient Distributed Training

    Uppsala University, Uppsala, Sweden · Research Institutes of Sweden, Stockholm, Sweden

  16. Pin Once, Swap Light: Subspace-Aligned Centroid-Residual Training for Efficient Ultra-LoRA Serving

    Aug 4, 2026Xiang Li, Pengcheng Wang, Huazheng Wang +1LLM Inference EfficiencyLLM Serving

    1Purdue University · 2Oregon State University

  17. SAKI: Score-Aware Low-Rank Key Indexing with Random-Matrix Noise Correction for KV Retrieval

    Aug 4, 2026Lin ZhangLow-Rank AttentionKV Caching

    Data Science and Platform, Cotality

  18. ARCHead: Activation-Metric Residual Correction for Large Language Model Output Heads

    Aug 3, 2026Şuayp Talha Kocabay, Talha Rüzgar Akkuş, Kamer Ali YukselLLM QuantizationLLM Compression

    Independent Researcher · aiXplain, Inc.

  19. S4^4R: Selective Sampling, Subspaces, and Sparse Reconstruction for Compressed Long-Context KV Caching

    Aug 1, 2026Jialong Han, You Wu, Kewei TuKV CachingLong-Context Language Model Inference

    School of Information Science and Technology, ShanghaiTech University · Shanghai Engineering Research Center of Intelligent Vision and Imaging

  20. Fidelity Is Not Safety: Gently-Compressed LLMs Pass Every Data-Free Quality Guard Yet Invent Procedure Steps in Agentic Execution

    Jul 30, 2026I. Kennedy, T. KennedyModel CompressionAI Agent Reliability

  21. A JoLT for the KV cache: Near-Lossless KV Cache Compression via Joint Rank-bit Allocation

    Jul 14, 2026Rahul Krishnan, Volker SchulzKV CachingLow-Rank Matrix Decomposition

    Universität Trier · Fachbereich IV, Mathematik, Universität Trier

  22. CARE-LoRA: Compressed Activation REconstruction for Memory-Efficient LoRA

    Jul 11, 2026Gengyu Zhang, Haiyin Ran, Zhengbao He +4Fine-TuningMemory-Efficient Fine-Tuning

    Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University, Shanghai 200240, China

  23. SLORR: Simple and Efficient In-Training Low-Rank Regularization

    Jul 9, 2026David González-Martínez, Shiwei LiuLLM CompressionLow-Rank Matrix Decomposition

    Max Planck Institute for Intelligent Systems · University of Tübingen · ELLIS Institute Tübingen +1

  24. DepthWeave-KV: Token-Adaptive Cross-Layer Residual Factorization for Long-Context KV Cache Compression

    Jul 7, 2026Anna Cordoba, Adam Puente Tercero, Nerea Angulo Hijo +4KV CachingLong-Context Language Model Inference

    Instituto de Investigación en Visión Artificial

  25. SAD-LoRA: Spectral Alignment for Low-Rank Knowledge Distillation

    Jul 5, 2026Omer Tariq, Syed Muhammad Raza, Jeongbae SonLow-Rank AdaptationLow-Rank Compression

    Neubility Inc, Seoul, South Korea

  26. LACE-SVD: Loss-Aware SVD with Cumulative Error Correction for LLM Compression

    Jul 3, 2026Zhuowen Liu, Longkun Hao, Shiyu Feng +3Model CompressionLLM Compression

    Zhuowen Liu1 · Longkun Hao2 · Shiyu Feng3 +3

  27. DLR: Zero-Inference-Cost Latent Residuals for Low-Rank Pre-Training

    Jun 27, 2026Dong Wang, Wenwu Tang, Yun Cheng +1Language Model PretrainingLLM Compression

    Graz University of Technology, Austria · Swiss Data Science Center, Switzerland

  28. SVD-Surgeon: Optimal Singular-Value Surgery for Large Language Model Compression

    Jun 22, 2026Mahmoud Safari, Frank HutterLLM PruningLLM Compression

    University of Freiburg · Prior Labs · ELLIS Institute Tübingen

  29. UniRank: Unified Rank Allocation for Low-Rank LLM Compression

    Jun 20, 2026Chao Han, Yongjie Du, Junjie Tan +1LLM CompressionLow-Rank Matrix Decomposition

    Ningbo Institute of Digital Twin, Eastern Institute of Technology, Ningbo

  30. Activation- and Influence-Aware Ranks (AIR): Function-Preserving SVD Compression for LLMs

    Jun 18, 2026Nico Harder, Daniel Becking, Karsten Mueller +1LLM CompressionLow-Rank Matrix Decomposition

    Fraunhofer HHI, Berlin, Germany

  31. EinSort: Sorting is All We Need for Tensorizing LLM

    Jun 7, 2026Toshiaki Koike-Akino, Jing Liu, Ye WangTensor NetworksLLM Compression

    Mitsubishi Electric Research Laboratories (MERL), 20 Broadway, Cambridge, MA 02139, USA.

  32. STAR-KV: Low-Rank KV Cache Compression via Soft Thresholding for Adaptive Rank Control

    Jun 7, 2026Priyansh Bhatnagar, Ashkan Moradifirouzabadi, Se-Hyun Yang +3Mixed-Precision QuantizationKV Caching

    University of California San Diego, San Diego, CA, USA · Dnotitia Inc., Seoul, Republic of Korea · Hanyang University, Seoul, Republic of Korea

  33. SigmaScale: LLM Compression with SVD-based Low-Rank Decomposition and Learned Scaling Matrices

    Jun 5, 2026Ernests Lavrinovics, Marco Letizia, Roy Janco +3Model CompressionLLM Compression

    Department of Computer Science, Aalborg University Copenhagen, Denmark · MaLGa-DIBRIS, University of Genoa, Genoa, Italy · INFN, Sezione di Genova, Genoa, Italy +2

  34. Learned Subspace Compression for Communication-Efficient Pipeline Parallelism

    Jun 3, 2026Paul Janson, Edouard Oyallon, Eugene BelilovskyLLM CompressionCommunication-Efficient Distributed Training

    Concordia University · Mila Quebec AI Institute · CNRS, Sorbonne University

  35. Compress then Merge: From Multiple LoRAs into One Low-Rank Adapter

    Jun 2, 2026Zhengbao He, Ruiqi Ding, Zhehao Huang +3Adapter TuningLow-Rank Adaptation

    Institute of Image Processing and Pattern Recognition, School of Automation and Intelligent Sensing, Shanghai Jiao Tong University, Shanghai, China · Shanghai Key Laboratory of Flexible Medical Robotics, Tongren Hospital, Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai, China

  36. GPTQ-intrinsic LoRA: A Near-optimal Algorithm for Low-precision Quantization with Low-rank Adaptation

    May 31, 2026Shihao Zhang, Rayan SaabLLM QuantizationLow-Rank Adaptation

    Department of Mathematics, University of California San Diego

  37. LASER: Loss-Aware Singular-value Decomposition and Rank Allocation for Efficient Low-Precision Vision-Language Models

    May 30, 2026Haiyu Wang, Yutong Wang, Leshu Li +2Vision-Language ModelsEfficient VLM Inference

    Tandon School of Engineering, New York University · Courant Institute of Mathematical Sciences, New York University · Brookhaven National Laboratory

  38. Cross-Layer Subspace Coupling for LLM Compression: A Unifying Framework and Its Empirical Limits

    May 29, 2026Snigdha Chandan KhilarModel CompressionLLM Compression

    Independent Researcher

  39. Efficient Pre-Training of LLMs through Truncated SVD Representations

    May 27, 2026Kaivan Kamali, Kajetan Schweighofer, Hormoz Shahrzad +3Language Model PretrainingLow-Rank Matrix Decomposition

    Cognizant AI Lab

  40. SAFE-SVD: Sensitivity-Aware Fidelity-Enforcing SVD for Physics Foundation Models

    May 18, 2026Chengjie Hong, Feixiang He, Yiheng Zeng +2Model CompressionLow-Rank Compression

    University College London, UK · Central South University, China · University of Massachusetts at Amherst, US +1

  41. IO-SVD: Input-Output Whitened SVD for Adaptive-Rank LLM Compression

    May 15, 2026Ali Abbasi, Chayne Thrash, Haoran Qin +2LLM CompressionLow-Rank Matrix Decomposition

    Vanderbilt University · University of California, Davis

  42. LRCP: Low-Rank Compressibility Guided Visual Token Pruning for Efficient LVLMs

    May 15, 2026Hongyu Lu, Feng Zhang, Wenwei Jin +5Efficient VLM InferenceLarge Vision-Language Models

    1Xiaohongshu · 2Harbin Institute of Technology · 3Fudan University

  43. Different Prompts, Different Ranks: Prompt-aware Dynamic Rank Selection for SVD-based LLM Compression

    May 9, 2026Hengyi Zhu, Zhendong Mi, Grace Li Zhang +1LLM CompressionLow-Rank Matrix Decomposition

    1Stevens Institute of Technology, Hoboken, USA · 2Technical University of Darmstadt, Darmstadt, Germany

  44. FlashSVD v1.5: Making Low-Rank Transformers Inference Actually Fast

    May 8, 2026Wenhao Wu, Zishan Shao, Kangning Cui +5Efficient Transformer InferenceLLM Compression

    Department of Electrical & Computer Engineering, Duke University · Department of Statistical Science, Duke University · Department of Computer Science, Wake Forest University +1

  45. Orth-Dion: Eliminating Geometric Mismatch in Distributed Low-Rank Spectral Optimization

    May 7, 2026Tatsuhiro Nakamori, Laura Gomezjurado Gonzalez, Ganesh Talluri +5Communication-Efficient Distributed TrainingDistributed Optimization

    1Keio University · 2Stanford University · 3Midwestern University +3

  46. Budgeted LoRA: Distillation as Structured Compute Allocation for Efficient Inference

    May 5, 2026Mohammed Sabry, Anya BelzLLM CompressionLow-Rank Adaptation

    ADAPT Centre, Dublin City University, Ireland

  47. ELAS: Efficient Pre-Training of Low-Rank Large Language Models via 2:4 Activation Sparsity

    May 5, 2026Jiaxi Li, Lu Yin, Li Shen +5Language Model PretrainingActivation Sparsity

    University of Surrey · Sun Yat-sen University · Bytedance +2

  48. Compress Then Adapt? No, Do It Together via Task-aware Union of Subspaces

    May 4, 2026Jingze Ge, Yun Liu, Xue Geng +4Model CompressionAdapter Tuning

    National University of Singapore · Nankai University · Institute for Infocomm Research (I2R), A*STAR

  49. Importance-Guided Basis Selection for Low-Rank Decomposition of Large Language Models

    May 2, 2026Daniel Agyei Asante, Ernie Chang, Yang LiLLM PruningLLM Compression

    Iowa State University · Meta

  50. Activation Compression in LLMs: Theoretical Analysis and Efficient Algorithm

    May 2, 2026Wen-Da Wei, Han-Bin Fang, Yang-Di Liu +3LLM CompressionGradient Compression

    Nanjing University, Nanjing, China · Tsinghua University, Beijing, China · Huazhong University of Science and Technology, Wuhan, China +1

  51. Post-Optimization Adaptive Rank Allocation for LoRA

    Apr 30, 2026Vishnuprasadh Kumaravelu, Sunil Gupta, P. K. SrijithLow-Rank AdaptationLow-Rank Compression

    1Indian Institute of Technology Hyderabad · 2Deakin University

  52. Hierarchical Spatio-Channel Clustering for Efficient Model Compression in Medical Image Analysis

    Apr 25, 2026Sisipho Hamlomo, Marcellin Atemkeng, Habte Tadesse Likassa +5Low-Rank CompressionLow-Rank Approximation

    Department of Mathematics, Rhodes University, Makhanda 6140, South Africa · Department of Statistics, Rhodes University, Makhanda 6140, South Africa · National Institute for Theoretical and Computational Sciences, Stellenbosch 7600, South Africa +8

  53. Predicting LLM Compression Degradation from Spectral Statistics

    Apr 20, 2026Mingxue XuLLM CompressionLow-Rank Matrix Decomposition

    Department of Electrical and Electronic Engineering, Imperial College London, London, United Kingdom

  54. LASER: Low-Rank Activation SVD for Efficient Recursion

    Apr 19, 2026Ege Çakar, Ketan Ali Raghu, Lia ZhengLow-Rank Matrix DecompositionRecurrent Neural Networks

    Harvard University Cambridge, MA, USA

  55. Attention-Weighted Value Projection for KV-Cache Compression

    Apr 13, 2026Samuel SalfatiGrouped-Query AttentionKV Caching

    fraQtl AI Research

  56. Stabilizing Native Low-Rank LLM Pretraining

    Feb 12, 2026Paul Janson, Edouard Oyallon, Eugene BelilovskyLanguage Model PretrainingLow-Rank Matrix Decomposition

    Concordia University, Montreal, Canada · Mila Quebec AI Institute, Montreal, Canada · Sorbonne University, CNRS, Paris, France

  57. Activation-Informed Pareto-Guided Low-Rank Compression for Efficient LLM/VLM

    Oct 7, 2025Ryan Solgi, Parsa Madinei, Jiayi Tian +4Model CompressionLLM Compression

    University of California-Santa Barbara, USA · Amazon, USA

  58. BALF: Budgeted Activation-Aware Low-Rank Factorization for Fine-Tuning-Free Model Compression

    Sep 29, 2025David González-MartínezLow-Rank Matrix DecompositionLow-Rank Compression

    Department of Computer Science and Artificial Intelligence, Universidad de Sevilla · Tübingen AI Center