LLM Compression

LLM: Large Language Model

Overview

LLM compression studies how to make language models smaller or cheaper to run while retaining useful abilities. Common approaches include low-bit quantization, pruning weights, layers, or experts, low-rank factorization, distillation into smaller models, and compressing prompts or KV caches. Methods increasingly use calibration data, sensitivity estimates, or adaptive budget allocation, often paired with specialized inference kernels to turn compact representations into practical speedups. A central challenge is that quality, reasoning, reliability, and actual runtime can change in different ways: fewer parameters or tokens do not always mean faster execution or preserved behavior.

Latest papers

CardsList

301 papers25 papers in the last four weeks, down 42% on the four weeks before. 0.2% of all new papers.

  1. LazyWeights: Decode-on-Demand Edge LLMs

    Nov 12, 2026Arnab Sanyal, Gourav Datta, Sandeep P. ChinchaliOn-Device Language Model InferenceLLM Compression

  2. Multi-head Quantum Neural Networks: Replacing Transformer Blocks with NISQ Modules inside a 30-Billion-Parameter LLM

    Nov 11, 2026Maforikan Amoussou, Borja Aizpurua, Sukhbinder Singh +2Quantum Neural NetworksTransformer

  3. Language model capabilities fail in a shared, predictable order under compression and perturbation

    Nov 8, 2026Zhixiang Wang, Ulas BagciLanguage Model RobustnessLanguage Model Performance Prediction

  4. MINE: Entropy-coded Memory-efficient Inference for Large Language Models

    Nov 6, 2026Emil Schätzle, Markus PüschelStructured Neural Network CompressionLLM Inference Acceleration

  5. AutoCompress: Automated Hardware-Guided Model Compression

    Nov 5, 2026Sheldon Liang, Ali Taghibakhshi, Saurav Muralidharan +4LLM CompressionLLM Pruning

  6. DictLLM: Post-training Compression of Large Language Model with Dictionary Kernels

    Nov 5, 2026Jinho Park, Se Young Chun, Mingoo SeokModel CompressionLLM Compression

  7. Compression Through a Context Lens: Evaluating Adaptation, Quantization, and Distillation in Arabic Legal LLMs

    Nov 5, 2026Md. Rafiul Biswas, George Mikros, Wajdi Zaghouanilegal language model evaluationLLM Compression

  8. When Bigger Becomes Brittle: Why Compression Hurts More as Language Models Scale

    Nov 5, 2026Gautam Gupta, Dikshant Kukreja, Unique Kumar +7LLM CompressionLanguage Model Scaling

  9. CompressionBench: How can your coding agents compress LLMs?

    Nov 4, 2026Xinrui Zhong, Zhifeng Li, Zhiming Wang +7LLM CompressionLLM Agent Self-Improvement

  10. Prune Gradually, Adapt Sparsely: Efficient Pruning of Large Language Models at Extreme Sparsity

    Oct 30, 2026Jean Lucien Randrianantenaina, Hung Nguyen, Nasib Ullah +4LLM PruningNeural Network Pruning

  11. DCKV: Learned Transform Coding of the Prompt KV Cache in Hybrid Language Models

    Oct 27, 2026Egor Maximov, Aleksei Merkelov, Vasiliy Alekseev +5KV-Cache CompressionKV-Cache Quantization

  12. Balanced State Subspaces with Spectral Rank Allocation for Mamba-2 State Compression

    Oct 27, 2026Shang Tingyu, Cao Wenfei, Han ZhiLLM CompressionState Space Language Models

  13. Advancing SVD-based LLM Compression via Layer-Wise Error Model Search

    Oct 23, 2026Moritz Thoma, Maximilian Groezinger, Maximilian Forstenhäusler +8LLM CompressionLow-Rank Compression

  14. Learning Structured Sparsity for Large Models: Bridging Model Compressibility and Efficient Inference

    Oct 22, 2026Tianxiang Xia, Kai Lion, Niao He +3LLM CompressionLLM Inference Acceleration

  15. Extreme Compression of Per-Layer Embeddings via Cross-Layer Latent Vector Quantization

    Oct 22, 2026Jiacheng Liu, Yaxin Luo, Xiaohan Zhao +5Embedding CompressionLLM Compression

  16. ACE-Compression: An Ability-Centric Evaluation of Compressed Large Language Models

    Oct 20, 2026Jiahui Ning, Liang Chen, Yuchen Liang +2LLM CompressionLLM Evaluation

  17. Reconstruction-Optimized Expert Pruning for Mixture-of-Experts Language Models

    Oct 19, 2026Jelena Markovic-Voronov, Petros Prastakos, Kayhan Behdin +7Mixture-of-Experts PruningMixture-of-Experts Language Models

  18. LOOP: Recovery-Aligned Iterative Layer Pruning via On-Policy Distillation

    Oct 19, 2026Liang Xu, Shufan Shen, Zhongni Hou +4LLM PruningNeural Network Pruning

  19. N-Gram Memories at Two Bits: Quantizing Lookup Tables in Large Language Models

    Oct 15, 2026Donghoon Han, Minseop Jung, Dong Woo Kwak +2Memory CompressionLLM Compression

  20. Unlocking Structured Pruning for Reasoning Language Models via Attention-Preserving MLP Trimming

    Oct 13, 2026Woosang Lim, Haneul Yoo, Vincent-Daniel Yun +1LLM PruningLLM Compression

  21. SLEM: Spectrum-Aware Latent Expert Merging

    Oct 12, 2026Dohyeon Kim, Bedionita Soro, Bruno Andreis +1Language Model MergingMixture-of-Experts Language Models