LLM Pruning

LLM: Large Language Model

Large language model pruning studies how to remove weights, neurons, attention heads, layers, or experts while retaining useful behavior. Methods range from magnitude and activation-based scoring to gradient, curvature, causal, and representation-based criteria, often using calibration data to estimate which components matter; some also recover performance through fine-tuning or distillation. Structured pruning can reduce computation more directly, while unstructured sparsity often needs specialized hardware and kernels to deliver real speedups. A central challenge is that the best pruning decisions depend on the model, task, calibration data, and deployment setting, and preserving benchmark accuracy may not preserve generation quality, fairness, or reliability.

Latest papers

178 papers32 papers in the last four weeks, up 191% on the four weeks before. 0.2% of all new papers.

Diet Your LLM: Dimension-wise Global Pruning of LLMs via Merging Task-specific Importance Score
Mar 25, 2026Jimyung Hong, Jaehyung KimLLM Pruning Structured Pruning
Yonsei University
Compressed-Sensing-Guided, Inference-Aware Structured Reduction for Large Language Models
Mar 22, 2026Andrew KirulutaLLM Inference Acceleration LLM Pruning
UC Berkeley School of Information
Structural Sensitivity in Compressed Transformers: Relative Error Propagation and Layer Removal
Mar 22, 2026Abhinaba Basu, Kumkum Basu, Koushik DebTransformer Pruning LLM Compression
Indian Institute of Information Technology Allahabad, India · National Institute of Electronics and Information Technology (NIELIT), India · Indian Institute of Technology Patna, India +2
Frequency Matters: Fast Model-Agnostic Data Curation for Pruning and Quantization
Mar 17, 2026Francesco Pio Monaco, Elia Cunegatti, Flavio Vella +1LLM Compression LLM Pruning
University of Trento
GPrune-LLM: Generalization-Aware Structured Pruning for Large Language Models
Mar 12, 2026Xiaoyun Liu, Divya Saxena, Jiannong Cao +3LLM Pruning Structured Pruning
Department of Computing, The Hong Kong Polytechnic University, Hong Kong · School of Artificial Intelligence and Data Science, IIT Jodhpur, Jodhpur 342030, India
Bielik-Minitron-7B: Compressing Large Language Models via Structured Pruning and Knowledge Distillation for the Polish Language
Mar 12, 2026Remigiusz Kinas, Paweł Kiszczak, Sergio P. Perez +4LLM Compression Language Modeling
Bielik.AI, Ingenix.ai · Bielik.AI, Vstorm · NVIDIA +3
High-Fidelity Pruning for Large Language Models
Mar 9, 2026Yijun Zhu, Jianxin Wang, Chengchao ShenLLM Pruning Neural Network Pruning
Central South University
Deterministic Differentiable Structured Pruning for Large Language Models
Mar 9, 2026Weiyu Huang, Pengle Zhang, Xiaolu Zhang +3LLM Pruning Structured Pruning
Department of Computer Science and Technology, Tsinghua University, Beijing, China · Ant Group, Beijing, China
A Comparative analysis of Layer-wise Representational Capacity in AR and Diffusion LLMs
Mar 8, 2026Raghavv Goel, Risheek Garrepalli, Sudhanshu Agrawal +3Diffusion Language Models Representation Geometry in Language Models
Qualcomm AI Research, Qualcomm Technologies Inc.
EvoESAP: Non-Uniform Expert Pruning for Sparse MoE
Mar 6, 2026Zongfang Liu, Shengkun Tang, Boyang Sun +2Mixture-of-Experts Pruning LLM Pruning
Zhejiang University, Hangzhou, Zhejiang, China · Westlake University, Hangzhou, Zhejiang, China · Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, United Arab Emirates
ROSE: Reordered SparseGPT for More Accurate One-Shot Large Language Models Pruning
Mar 6, 2026Mingluo Su, Huan WangLLM Pruning Neural Network Pruning
Westlake University
Measuring the Redundancy of Decoder Layers in SpeechLLMs
Mar 5, 2026Adel Moumen, Guangzhi Sun, Philip C WoodlandLLM Pruning Transformer
Department of Engineering, University of Cambridge, UK
Curvature-Weighted Capacity Allocation: A Minimum Description Length Framework for Layer-Adaptive Large Language Model Optimization
Mar 1, 2026Theophilus Amaefuna, Hitesh Vaidya, Anshuman Chhabra +1LLM Pruning LLM Compression
Bellini College of Artificial Intelligence, Cybersecurity and Computing, University of South Florida, Tampa, Florida, USA
Sparsity Induction for Accurate Post-Training Pruning of Large Language Models
Feb 25, 2026Minhao Jiang, Zhikai Li, Xuewen Liu +3LLM Pruning LLM Compression
Institute of Automation, Chinese Academy of Sciences, Beijing, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
SPQ: An Ensemble Technique for Large Language Model Compression
Feb 20, 2026Jiamin Yao, Eren GultepeLLM Compression LLM Quantization
Dept. of Computer Science, Southern Illinois University Edwardsville, Edwardsville, USA
Sink-Aware Pruning for Diffusion Language Models
Feb 19, 2026Aidar Myrzakhan, Tianyi Li, Bowei Guo +2LLM Pruning Diffusion Model Pruning
VILA Lab, MBZUAI
GradMAP: Faster Layer Pruning with Gradient Metric and Projection Compensation
Feb 16, 2026Hao Liu, Guangyan Li, Wensheng Zhang +1LLM Pruning Neural Network Pruning
Institute of Automation, Chinese Academy of Sciences, Beijing, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China · Guangzhou University, Guangzhou, China
LLM Compression by Block Removal with Constrained Binary Optimization
Jan 29, 2026David Jansen, Roman Rausch, Ali Hashemi +2LLM Pruning LLM Compression
Multiverse Computing · Donostia International Physics Center Ikerbasque Foundation for Science
Probe and Skip: Self-Predictive Token Skipping for Efficient Long-Context LLM Inference
Jan 19, 2026Zimeng Wu, Donghao Wang, Chaozhe Jin +2LLM Inference Acceleration Long-Context LLM Inference
School of Computer Science and Engineering, Beihang University, Beijing, China · State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, China
Garbage Attention in Large Language Models: BOS Sink Heads and Sink-aware Pruning
Jan 11, 2026Jaewon Sok, Jewon Yeom, Seonghyeon Park +2LLM Pruning Attention Head Analysis
Department of Rural Systems Engineering, Seoul National University · Graduate School of Data Science, Seoul National University · Department of Aerospace Engineering, Seoul National University
OPTIMA: Optimal One-shot Pruning for LLMs via Quadratic Programming Reconstruction
Dec 15, 2025Mohammad Mozaffari, Samuel Kushnir, Maryam Mehri Dehnavi +1LLM Pruning Neural Network Pruning
University of Toronto · Google DeepMind
Data-Free Pruning of Self-Attention Layers in LLMs
Dec 3, 2025Dhananjay Saikumar, Blesson VargheseLLM Inference Acceleration LLM Compression
School of Computer Science University of St Andrews United Kingdom
PATCH: Learnable Tile-level Hybrid Sparsity for LLMs
Sep 27, 2025Younes Hourri, Mohammad Mozaffari, Maryam Mehri DehnaviLLM Pruning LLM Compression
Department of Computer Science University of Toronto
A Token is Worth over 1,000 Tokens: Efficient Knowledge Distillation through Low-Rank Clone
May 19, 2025Jitai Hao, Qiang Huang, Hao Liu +3Language Model Pretraining LLM Pruning
School of Intelligence Science and Engineering, Harbin Institute of Technology, Shenzhen · Baidu Inc. · Leiden University +1
Adaptive GoGI-Skip: Coupling Goal-Gradient Importance with Dynamic Uncertainty for Efficient Reasoning
May 13, 2025Ren ZhuangGradient-Based Attribution LLM Pruning
School of Information Science and Technology, Hangzhou Normal University
Optimizing Singular Spectrum for Large Language Model Compression
Feb 20, 2025Dengjie Li, Tiancheng Shen, Yao Zhou +3Model Compression LLM Compression
DarwinLM: Evolutionary Structured Pruning of Large Language Models
Feb 11, 2025Shengkun Tang, Oliver Sieberling, Eldar Kurtic +2LLM Pruning Structured Pruning
MBZUAI · ETH Zurich · ISTA +1
Beyond Size: How Gradients Shape Pruning Decisions in Large Language Models
Nov 8, 2023Rocktim Jyoti Das, Mukul Ranjan, Mingjie Sun +2LVLM Pruning LLM Pruning
Mohamed bin Zayed University of AI · Carnegie Mellon University

Mar 25, 2026J/K move · Enter open · S save

Diet Your LLM: Dimension-wise Global Pruning of LLMs via Merging Task-specific Importance Score

Jimyung Hong, Jaehyung Kim

Yonsei University

LLM Pruning Structured Pruning Neural Network Pruning LLM Compression

Open paper AskPDF arXiv

Large language models (LLMs) have demonstrated remarkable capabilities, but their massive scale poses significant challenges for practical deployment. Structured pruning offers a promising solution by removing entire dimensions or layers, yet existing methods face critical trade-offs: task-agnostic approaches cannot adapt to task-specific requirements, while task-aware methods require costly training to learn task adaptability. We propose DIET (Dimension-wise global pruning of LLMs via merging Task-wise importance scores), a training-free structured pruning method that combines dimension-level granularity with task-aware selection. DIET profiles activation magnitudes across tasks using only 100 samples per task, then applies majority voting to construct a single global mask. Experiments on seven zero-shot benchmarks using Gemma-2 2B and 9B models demonstrate the effectiveness of DIET. At 20% sparsity on Gemma-2 2B, DIET improves average accuracy by 2.1 percentage points over SliceGPT and by 9.2 percentage points over PuDDing. On Gemma-2 9B, DIET remains competitive with SliceGPT while consistently outperforming PuDDing at 10% and 20% sparsity.