LLM Pruning

LLM: Large Language Model

Overview

Large language model pruning studies how to remove weights, neurons, attention heads, layers, or experts while retaining useful behavior. Methods range from magnitude and activation-based scoring to gradient, curvature, causal, and representation-based criteria, often using calibration data to estimate which components matter; some also recover performance through fine-tuning or distillation. Structured pruning can reduce computation more directly, while unstructured sparsity often needs specialized hardware and kernels to deliver real speedups. A central challenge is that the best pruning decisions depend on the model, task, calibration data, and deployment setting, and preserving benchmark accuracy may not preserve generation quality, fairness, or reliability.

Latest papers

CardsList

191 papers21 papers in the last four weeks, down 19% on the four weeks before. 0.2% of all new papers.

  1. Slim Foundation Models for Latency-Critical Recommendation Serving

    Nov 7, 2026Danyal Aftab, Anas Zafar, Assadig Abakr +1CTR PredictionLarge Language Model-Based Recommendation

  2. AutoCompress: Automated Hardware-Guided Model Compression

    Nov 5, 2026Sheldon Liang, Ali Taghibakhshi, Saurav Muralidharan +4LLM CompressionLLM Pruning

  3. SKIM: Pruning Large Language Model Agents via Selective Knowledge Informed Masking

    Nov 5, 2026Moonseok Choi, Jongwon Jeong, Minki Kang +2LLM PruningLanguage Model Distillation

  4. Rethinking LLM Pruning via Test-Time Scaling

    Oct 31, 2026Mingyang Song, Mao ZhengLLM PruningEfficient Language Model Reasoning

  5. What Breaks and What Comes Back: Capability Damage and Recovery in Depth-Pruned Small Language Models

    Oct 31, 2026Piyush Dhoka, Vedant Pandhare, Varun Inamdar +2LLM PruningTransformer Pruning

  6. Compression Technique Interactions in Small Language Models: A Factorial Ablation Study

    Oct 30, 2026Bhushit Nema, Siddharth Krishnamurthy, Anjali KadiyalaSmall Language ModelsLLM Quantization

  7. Prune Gradually, Adapt Sparsely: Efficient Pruning of Large Language Models at Extreme Sparsity

    Oct 30, 2026Jean Lucien Randrianantenaina, Hung Nguyen, Nasib Ullah +4LLM PruningNeural Network Pruning

  8. Two-Stage Clipping: Linear-Time Approximate and Exact Mixed-Norm Projections for Transformers

    Oct 28, 2026Michel Barlaud, Jean-Paul MarmoratStructured SparsityTransformer Pruning

  9. Mechanistic Failure Modes of Weight Pruning in Transformer Language Models

    Oct 27, 2026Nabeelah Faumi, Deshan SumanathilakaLLM PruningAttention Head Pruning

  10. T2D: Target-to-Draft Model Pruning & Adaptation for Efficient Speculative Decoding

    Oct 25, 2026Shahriar Kabir Nahin, Xiao Ye, Muhao Chen +2Speculative DecodingLLM Inference Acceleration

  11. Building a Nested LADDER for LLM Unstructured Post-Training Pruning

    Oct 20, 2026Nicolás Mauricio Cuadrado Ávila, Ivo Gollini Navarrete, Martin Takáč +1LLM PruningNeural Network Pruning

  12. RACE: RoPE-Aligned Attention Compression Based on Query-Key Activation Energies

    Oct 20, 2026Irene Tenison, Abdulrahman Alabdulkareem, Daniel Karl I. Weidele +2KV-Cache CompressionTransformer Attention

  13. ACE-Compression: An Ability-Centric Evaluation of Compressed Large Language Models

    Oct 20, 2026Jiahui Ning, Liang Chen, Yuchen Liang +2LLM CompressionLLM Evaluation

  14. SliMOO: Interpretable Multi-Objective Evolutionary Search for LLM Depth Pruning

    Oct 20, 2026Guanchen Li, Yixing Xu, Xuanwu Yin +2LLM PruningTransformer Pruning

  15. How Fine-Tuning Objectives Shape Layer-Wise Information in LLM Hallucination Detection

    Oct 20, 2026Kaiyang Wan, Forrest Sheng Bao, Amin Ahmad +1Fine-Tuning Language ModelsLLM Hallucination Detection

  16. LOOP: Recovery-Aligned Iterative Layer Pruning via On-Policy Distillation

    Oct 19, 2026Liang Xu, Shufan Shen, Zhongni Hou +4LLM PruningNeural Network Pruning

  17. Prune the Leaky Neurons: Mitigating Private Data Leakage in Fine-Tuned LLMs via Joint-Embedding-Guided Pruning

    Oct 16, 2026Mohamed Shaaban, Mohamed ElmahallawyPrivacy Leakage in Language ModelsPrivacy-Preserving Language Models

  18. To Reason but Not to Stop: Understanding and Recovering Pruned Large Reasoning Models

    Oct 14, 2026Minh-Anh Nguyen, Nguyen T. Ha Lan, Xiyan Fu +2Repetition in Language GenerationLLM Reasoning

  19. Unlocking Structured Pruning for Reasoning Language Models via Attention-Preserving MLP Trimming

    Oct 13, 2026Woosang Lim, Haneul Yoo, Vincent-Daniel Yun +1LLM PruningLLM Compression

  20. Two For The Price Of One: Curvature-Aware Pruning Preserves Influence Functions Too

    Oct 13, 2026Mehdi Makni, Natalia Ponomareva, Rahul MazumderLLM PruningNeural Network Pruning

  21. From Spectral Saliency to Finite Deletion: Evaluating Selective Language Model Edits

    Oct 13, 2026Ibne Farabi Shihab, Joyanta Jyoti MondalLanguage Model EditingLLM Pruning

  22. Heterogeneous Structured Pruning Under Resource Constraints for Hardware-Efficient Large Language Models

    Oct 11, 2026Cameron Barker, Andreas Paraskeva, Jan N. van Rijn +2LLM PruningNeural Network Pruning

  23. The Fragility and Dynamics of LLM Pruning-Recovery Steps

    Oct 11, 2026Luis Frentzen Salim, Komang Rinartha, Chia-Chun Wu +3LLM PruningLLM Compression