LLM Pruning

LLM: Large Language Model

Large language model pruning studies how to remove weights, neurons, attention heads, layers, or experts while retaining useful behavior. Methods range from magnitude and activation-based scoring to gradient, curvature, causal, and representation-based criteria, often using calibration data to estimate which components matter; some also recover performance through fine-tuning or distillation. Structured pruning can reduce computation more directly, while unstructured sparsity often needs specialized hardware and kernels to deliver real speedups. A central challenge is that the best pruning decisions depend on the model, task, calibration data, and deployment setting, and preserving benchmark accuracy may not preserve generation quality, fairness, or reliability.

Latest papers

178 papers32 papers in the last four weeks, up 191% on the four weeks before. 0.2% of all new papers.

Rethinking Layer Relevance in Large Language Models Beyond Cosine Similarity
May 13, 2026Cristian Hinostroza, Rodrigo Toro Icarte, Christ Devia +4LLM Pruning LLM Interpretability
1Pontificia Universidad Cat´olica de Chile · 2National Center for Artificial Intelligence (CENIA) · 3Universidad de Chile
STOP: Structured On-Policy Pruning of Long-Form Reasoning in Low-Data Regimes
May 13, 2026Chenjun Xu, Zhennan Zhou, Zhan Su +3LLM Pruning LLM Fine-Tuning
University of Washington · University of Montreal
Layer-wise Representation Dynamics: An Empirical Investigation Across Embedders and Base LLMs
May 12, 2026Jingzhou Jiang, Yi Yang, Kar Yan TamText Embeddings LLM Pruning
The Hong Kong University of Science and Technology
Compute Where it Counts: Self Optimizing Language Models
May 11, 2026Yash Akhauri, Mohamed S. AbdelfattahLLM Inference Efficiency RL for Language Models
Cornell University
A Game Theoretic Free Energy Analysis of Higher Order Synergy in Attention Heads of Large Language Models
May 10, 2026Djamel BouchaffraLLM Pruning Attention Head Analysis
DAVID Lab, University of Paris-Saclay, UVSQ Campus, 78035 Versailles, France
Relative Kinetic Utility: Calibrating Cross-Layer Credit for Global Structured LLM Pruning
May 9, 2026Tianhao Qian, Guilin Qi, Jiayu ChenLLM Pruning Structured Sparsity
School of Mathematics, Southeast University Nanjing 210096, China
ProxyKV: Cross-Model Proxy Pruning for Efficient Long-Context LLM Inference
May 9, 2026Junjie Li, Jiong Lou, Jie LiLLM Pruning KV Caching
Shanghai Jiao Tong University
SlimQwen: Exploring the Pruning and Distillation in Large MoE Model Pre-training
May 9, 2026Shengkun Tang, Zekun Wang, Bo Zheng +7Mixture-of-Experts Pruning LLM Pruning
Qwen Team, Alibaba Inc. · MBZUAI · KAUST
Understanding Performance Collapse in Layer-Pruned Large Language Models via Decision Representation Transitions
May 8, 2026Boyu Shi, Chang Liu, ChuanBao Gao +2LLM Pruning LLM Interpretability
School of Computer Science and Engineering, Southeast University, Nanjing, China · Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China
SparseForge: Efficient Semi-Structured LLM Sparsification via Annealing of Hessian-Guided Soft-Mask
May 7, 2026Liu Hanzuo, Chaofan Lin, Weixuan Sun +4LLM Pruning Sparse Recovery
†Tsinghua University · ‡Tencent
Importance-Guided Basis Selection for Low-Rank Decomposition of Large Language Models
May 2, 2026Daniel Agyei Asante, Ernie Chang, Yang LiLLM Pruning LLM Compression
Iowa State University · Meta
Weight Pruning Amplifies Bias: A Multi-Method Study of Compressed LLMs for Edge AI
May 2, 2026Plawan Kumar Rath, Rahul MaliakkalLLM Pruning On-Device Language Model Inference
Meta
Exploring the Limits of Pruning: Task-Specific Neurons, Model Collapse, and Recovery in Task-Specific Large Language Models
Apr 29, 2026M. K. Khalidi Siam, Md. Tausif-Ul-Islam, Md. Reshad Romim Khan +5LLM Pruning Language Modeling
1BRAC University
Revisiting the Effectiveness of LLM Pruning for Test-Time Scaling
Apr 28, 2026Ocean Monjur, Shahriar Kabir Nahin, Anshuman ChhabraLLM Pruning Test-Time Scaling
Bellini College of AI, Cybersecurity, and Computing University of South Florida Tampa, FL, USA
Rethinking Layer Redundancy: Calibration Matters More Than Search in LLM Depth Pruning
Apr 27, 2026Minkyu Kim, Vincent-Daniel Yun, Youngrae Kim +3LLM Pruning LLM Inference Acceleration
Neural Superintelligence Lab, MODULABS, Republic of Korea · University of Southern California, United States · Boston University, United States +2
DepthKV: Layer-Dependent KV Cache Pruning for Long-Context LLM Inference
Apr 27, 2026Zahra Dehghanighobadi, Asja FischerLLM Pruning KV Caching
1Ruhr University Bochum · 2UAR Research Center for Trustworthy Data Science and Security
Pruning via Causal Attribution Preserves Reasoning Performance in Large Language Models
Apr 27, 2026Amogh Sheth, Andrew Lin, Biruk Assefa +2LLM Pruning Self-Attention
Edison Academy Magnet School · Massachusetts Institute of Technology · State University of New York College at Plattsburgh +2
Supernodes and Halos: Loss-Critical Hubs in LLM Feed-Forward Layers
Apr 26, 2026Audrey Cherilyn, Houman SafaaiTransformer Interpretability LLM Pruning
Kempner Institute at Harvard University
SimDiff: Depth Pruning via Similarity and Difference
Apr 21, 2026Yuli Chen, Shuhao Zhang, Fanshen Meng +4LLM Pruning LLM Inference Acceleration
GRASPrune: Global Gating for Budgeted Structured Pruning of Large Language Models
Apr 21, 2026Ziyang Wang, Jiangfeng Xiao, Chuan Xiao +3LLM Pruning LLM Compression
Beijing Institute of Technology, Zhuhai · Shenzhen University · Osaka University
CRISP: Compressing Redundancy in Chain-of-Thought via Intrinsic Saliency Pruning
Apr 19, 2026Yangsong Lan, Hongliang Dai, Piji LiLLM Pruning LLM Compression
College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics, Nanjing, China · The Key Laboratory of Brain-Machine Intelligence Technology, Ministry of Education, Nanjing, China.
Prune, Interpret, Evaluate: A Cross-Layer Transcoder-Native Framework for Efficient Circuit Discovery via Feature Attribution
Apr 18, 2026Qinhao Chen, Linyang He, Nima MesgaraniLLM Pruning Feature Attribution
Columbia University
Optimizing Korean-Centric LLMs via Token Pruning
Apr 17, 2026Hoyeol Kim, Hyeonwoo KimMultilingual Language Model Evaluation LLM Pruning
College of Computing · Independent Researcher
Pruning Unsafe Tickets: A Resource-Efficient Framework for Safer and More Robust LLMs
Apr 17, 2026Wai Man Si, Mingjie Li, Michael Backes +1LLM Pruning LLM Safety Alignment
CISPA Helmholtz Center for Information Security
MOONSHOT : A Framework for Multi-Objective Pruning of Vision and Large Language Models
Apr 14, 2026Gabriel Afriat, Xiang Meng, Shibal Ibrahim +2LLM Pruning Neural Network Pruning
Operations Research Center Massachusetts Institute of Technology · Google · OpenAI +1
MoEITS: A Green AI approach for simplifying MoE-LLMs
Apr 12, 2026Luis Balderas, Miguel Lastra, José M. BenítezLLM Compression Mixture-of-Experts Pruning
Department of Computer Science and Artificial Intelligence, DiCITS, iMUDS, DaSCI, University of Granada, Granada, Spain 18071 · Department of Software Engineering, DiCITS, iMUDS, DaSCI, University of Granada, Granada, Spain 18071
NanoSpec: Accelerating Speculative Decoding using Minimalist In-Context Vocabularies
Apr 8, 2026Zhiyang Chen, Daliang Xu, Yinyuan Zhang +3Speculative Decoding LLM Inference Acceleration
Institute for Artificial Intelligence, Peking University, Beijing, China · State Key Laboratory of Networking and Switching Technology; School of Computer Science, Beijing University of Posts and Telecommunications, Beijing, China · Key Laboratory of High Confidence Software Technologies (Peking University), Ministry of Education; School of Computer Science, Peking University, Beijing, China
SLaB: Sparse-Lowrank-Binary Decomposition for Efficient Large Language Models
Apr 6, 2026Ziwei Li, Yuang Ma, Yi KangBinary Quantization LLM Compression
School of Microelectronics, University of Science and Technology of China, Hefei, Anhui, China
How Pruning Reshapes Features: Sparse Autoencoder Analysis of Weight-Pruned Language Models
Mar 26, 2026Hector Borobia, Elies Seguí-Mas, Guillermina Tormo-CarbóLLM Pruning Representation Geometry in Language Models
VRAIN – Valencian Research Institute for Artificial Intelligence, Universitat Politècnica de València, Valencia, Spain · Department of Economics and Social Sciences, Universitat Politècnica de València, Valencia, Spain · Department of Business Organisation, Universitat Politècnica de València, Valencia, Spain
Demystifying When Pruning Works via Representation Hierarchies
Mar 25, 2026Shwai He, Guoheng Sun, Haichao Zhang +2LLM Pruning Language Modeling
University of Maryland, College Park, USA · Northeastern University, USA

May 13, 2026J/K move · Enter open · S save

Rethinking Layer Relevance in Large Language Models Beyond Cosine Similarity

Cristian Hinostroza, Rodrigo Toro Icarte, Christ Devia, Andres Carvallo De Ferari+3

1Pontificia Universidad Cat´olica de Chile · 2National Center for Artificial Intelligence (CENIA) · 3Universidad de Chile

LLM Pruning LLM Interpretability Neural Network Interpretability

Open paper AskPDF arXiv

Large language models (LLMs) have revolutionized natural language processing. Understanding their internal mechanisms is crucial for developing more interpretable and optimized architectures. Mechanistic interpretability has led to the development of various methods for assessing layer relevance, with cosine similarity being a widely used tool in the field. On this work, we demonstrate that cosine similarity is a poor proxy for the actual performance degradation caused by layer removal. Our theoretical analysis shows that a layer can exhibit an arbitrarily low cosine similarity score while still being crucial to the model's performance. On the other hand, empirical evidence from a range of LLMs confirms that the correlation between cosine similarity and actual performance degradation is often weak or moderate, leading to misleading interpretations of a transformer's internal mechanisms. We propose a more robust metric for assessing layer relevance: the actual drop in model accuracy resulting from the removal of a layer. Even though it is a computationally costly metric, this approach offers a more accurate picture of layer importance, allowing for more informed pruning strategies and lightweight models. Our findings have significant implications for the development of interpretable LLMs and highlight the need to move beyond cosine similarity in assessing layer relevance.