LLM Compression

LLM: Large Language Model

Ask about this topic

Overview

LLM compression studies how to make language models smaller or cheaper to run while retaining useful abilities. Common approaches include low-bit quantization, pruning weights, layers, or experts, low-rank factorization, distillation into smaller models, and compressing prompts or KV caches. Methods increasingly use calibration data, sensitivity estimates, or adaptive budget allocation, often paired with specialized inference kernels to turn compact representations into practical speedups. A central challenge is that quality, reasoning, reliability, and actual runtime can change in different ways: fewer parameters or tokens do not always mean faster execution or preserved behavior.

Latest papers

CardsList

283 papers44 papers in the last four weeks, up 52% on the four weeks before. 0.3% of all new papers.

  1. Optimizing Korean-Centric LLMs via Token Pruning

    Apr 17, 2026Hoyeol Kim, Hyeonwoo KimMultilingual Language Model EvaluationLLM Pruning

    College of Computing · Independent Researcher

  2. Self-Distillation as a Performance Recovery Mechanism for LLMs: Counteracting Compression and Catastrophic Forgetting

    Apr 17, 2026Chi Liu, Xin Chen, Xu Zhou +2LLM CompressionLanguage Model Distillation

    PayPal AI

  3. Compressing Sequences in the Latent Embedding Space: KK-Token Merging for Large Language Models

    Apr 16, 2026Zihao Xu, John Harvill, Ziwei Fan +3Token MergingLLM Compression

    ♢Rutgers University · ♡Mistral AI · ♠AWS AI Labs +3

  4. When Less Latent Leads to Better Relay: Information-Preserving Compression for Latent Multi-Agent LLM Collaboration

    Apr 14, 2026Yiping Li, Zhiyu An, Wan DuLLM CompressionKV Caching

    Department of Computer Science and Engineering, University of California, Merced, Merced, CA 95343

  5. MoEITS: A Green AI approach for simplifying MoE-LLMs

    Apr 12, 2026Luis Balderas, Miguel Lastra, José M. BenítezLLM CompressionMixture-of-Experts Pruning

    Department of Computer Science and Artificial Intelligence, DiCITS, iMUDS, DaSCI, University of Granada, Granada, Spain 18071 · Department of Software Engineering, DiCITS, iMUDS, DaSCI, University of Granada, Granada, Spain 18071

  6. MEMENTO: Teaching LLMs to Manage Their Own Context

    Apr 10, 2026Vasilis Kontonis, Yuchen Zeng, Shivam Garg +7LLM Context ManagementEfficient Language Model Reasoning

    Microsoft Research

  7. EdgeRazor: A Lightweight Framework for Large Language Models via Mixed-Precision Quantization-Aware Distillation

    Apr 10, 2026Shu-Hao Zhang, Le-Tong Huang, Xiang-Sheng Deng +5Quantization-Aware DistillationLLM Quantization

    National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210063, China · School of Intelligent Science and Technology, Nanjing University, Suzhou 215163, China · Microsoft AI, Beijing 100080, China +1

  8. Learning is Forgetting: LLM Training As Lossy Compression

    Apr 8, 2026Henry C. Conklin, Tom Hosking, Tan Yi-Chern +5Language Model PretrainingLLM Compression

    Princeton University · Cohere

  9. Does a Global Perspective Help Prune Sparse MoEs Elegantly?

    Apr 8, 2026Zeliang Zhang, Nikhil Ghosh, Jiani Liu +2Mixture-of-Experts PruningNeural Network Pruning

    University of Rochester · Flatiron Institute · University of California, Berkeley +1

  10. SLaB: Sparse-Lowrank-Binary Decomposition for Efficient Large Language Models

    Apr 6, 2026Ziwei Li, Yuang Ma, Yi KangBinary QuantizationLLM Compression

    School of Microelectronics, University of Science and Technology of China, Hefei, Anhui, China

  11. DP-OPD: Differentially Private On-Policy Distillation for Language Models

    Apr 6, 2026Fatemeh Khadem, Sajad Mousavi, Yi Fang +1Privacy-Preserving Language ModelsPrivacy-Preserving ML

    Santa Clara University, Santa Clara, California, USA · Independent Researcher

  12. REAM: Merging Improves Pruning of Experts in LLMs

    Apr 6, 2026Saurav Jha, Maryam Hashemzadeh, Ali Saheb Pasand +3LLM CompressionMixture-of-Experts Pruning

    Mila – Quebec AI Institute · Polytechnique Montréal · Université de Montréal +3

  13. Shorter, but Still Trustworthy? An Empirical Study of Chain-of-Thought Compression

    Apr 5, 2026Lingjie Zeng, Xiaofan Chen, Yanbo Wang +1CoT CompressionLLM Compression

    Mohamed bin Zayed University of Artificial Intelligence

  14. AutoCompress: Critical Layer Isolation for Efficient Transformer Compression

    Apr 4, 2026Archit ThoratNeural Network CompressionLLM Compression

    Independent Researcher

  15. Multi-Aspect Knowledge Distillation for Language Model with Low-rank Factorization

    Apr 3, 2026Zihe Liu, Yulong Mao, Jinan Xu +2Language Model DistillationKnowledge Distillation

    Key Laboratory of Big Data & Artificial Intelligence in Transportation · School of Computer Science and Technology, Beijing Jiaotong University

  16. NativeTernary: A Self-Delimiting Binary Encoding with Unary Run-Length Hierarchy Markers for Ternary Neural Network Weights, Structured Data, and General Computing Infrastructure

    Apr 3, 2026Maharshi SavdhariyaLLM Compression

    Indian Institute of Technology Bombay · IIM Udaipur

  17. AA-SVD : Anchored and Adaptive SVD for Large Language Model Compression

    Apr 2, 2026Atul Kumar Sinha, François FleuretLLM CompressionLow-Rank Compression

    University of Geneva, Geneva, Switzerland · FAIR, Meta

  18. Swift-SVD: Theoretical Optimality Meets Practical Efficiency in Low-Rank LLM Compression

    Apr 2, 2026Ruoling Qi, Yirui Liu, Xuaner Wu +6LLM CompressionLow-Rank Compression

    Institute of Artificial Intelligence (TeleAI), China Telecom · Shanghai Jiao Tong University · University of Maryland +2

  19. Optimal Brain Decomposition for Accurate LLM Low-Rank Approximation

    Apr 1, 2026Yuhang Li, Donghyun Lee, Ruokai Yin +1Low-Rank ApproximationLLM Compression

    Yale University · University of Sounthern California

  20. Spectral Compact Training: Pre-Training Large Language Models via Permanent Truncated SVD and Stiefel QR Retraction

    Apr 1, 2026Björn Roman KohlbergerLLM TrainingMemory-Efficient Training

    EctoSpace, Dublin, Ireland

  21. Probabilistic Language Tries: A Unified Framework for Compression, Decision Policies, and Execution Reuse

    Mar 29, 2026Gregory MagarshakLossless Data CompressionLLM Inference Acceleration

  22. Density-aware Soft Context Compression with Semi-Dynamic Compression Ratio

    Mar 26, 2026Yijiong Yu, Shuai Yuan, Jie Zheng +2LLM Context CompressionContext Compression

    Oregon State University · DeepSolution

  23. Large Language Model as Token Compressor and Decompressor

    Mar 26, 2026Wenbing Li, Yiran Wang, Zikai Song +4LLM Context CompressionLLM Compression

  24. How Pruning Reshapes Features: Sparse Autoencoder Analysis of Weight-Pruned Language Models

    Mar 26, 2026Hector Borobia, Elies Seguí-Mas, Guillermina Tormo-CarbóLLM PruningRepresentation Geometry in Language Models

    VRAIN – Valencian Research Institute for Artificial Intelligence, Universitat Politècnica de València, Valencia, Spain · Department of Economics and Social Sciences, Universitat Politècnica de València, Valencia, Spain · Department of Business Organisation, Universitat Politècnica de València, Valencia, Spain

  25. Diet Your LLM: Dimension-wise Global Pruning of LLMs via Merging Task-specific Importance Score

    Mar 25, 2026Jimyung Hong, Jaehyung KimLLM PruningStructured Pruning

    Yonsei University

  26. Sparser, Faster, Lighter Transformer Language Models

    Mar 24, 2026Edoardo Cetin, Stefano Peluchetti, Emilio Castillo +3LLM Inference AccelerationSparse Neural Networks

    Sakana AI · NVIDIA

  27. Structural Sensitivity in Compressed Transformers: Relative Error Propagation and Layer Removal

    Mar 22, 2026Abhinaba Basu, Kumkum Basu, Koushik DebTransformer PruningLLM Compression

    Indian Institute of Information Technology Allahabad, India · National Institute of Electronics and Information Technology (NIELIT), India · Indian Institute of Technology Patna, India +2

  28. BEAVER: A Training-Free Hierarchical Prompt Compression Method via Structure-Aware Page Selection

    Mar 20, 2026Zhengpei Hu, Kai Li, Dapeng Fu +4LLM Context CompressionPrompt Compression

    School of Computer Technology and Application, Qinghai University · Tsinghua University · Ant Group Security and Intelligence Laboratory (SIL)

  29. Lossless Prompt Compression via Dictionary-Encoding and In-Context Learning: Enabling Cost-Effective LLM Analysis of Repetitive Data

    Mar 19, 2026Andresa Rodrigues de Campos, David Lee, Imry Kissos +1Lossless Data CompressionLLM Compression

    Amazon.com.

  30. Weight Clustering Reveals the Importance of Relative Rank in Large Language Models

    Mar 18, 2026Borja Aizpurua, Sukhbinder Singh, Román OrúsLLM CompressionLLM Quantization

    Multiverse Computing, San Sebasti´an, Spain · Department of Basic Sciences, Tecnun - University of Navarra, San Sebasti´an, Spain · Multiverse Computing, Toronto, Ontario, Canada +2