Neural Network Pruning

Follow this topic Ask about this topic

Neural network pruning studies how to remove parts of a model that contribute little to its predictions, reducing memory use, computation, energy, or latency. Methods range from magnitude-based removal to scores using gradients, Fisher information, curvature, activations, redundancy, or task-specific behavior; they may prune individual weights or structured units such as channels, attention heads, and layers. Pruning can happen during training or after training in one shot or iteratively, often with fine-tuning, distillation, or compensation to recover lost performance. A central challenge is that the best pruning decisions depend on the model, task, calibration data, and hardware, so lower parameter counts do not always translate into faster inference or preserve reliability and reasoning behavior.

Latest papers

292 papers37 papers in the last four weeks, up 3% on the four weeks before. 0.2% of all new papers.

Prune, Update and Trim: Robust Structured Pruning for Large Language Models
May 18, 2026Diego Coello de Portugal Mecke, Tom Hanika, Lars Schmidt-ThiemeLLM Pruning LLM Inference Acceleration
ISMLL & DARC VWFS · University of Hildesheim · Hildesheim, Germany +1
LEAP: Learnable End-to-End Adaptive Pruning of Large Language Models
May 17, 2026Mohammad Mozaffari, Younes Hourri, Mohammad Rastegari +1LLM Pruning Neural Network Pruning
Elastix AI · Department of Computer Science, University of Toronto, Toronto, Ontario, Canada.
Expandable, Compressible, Mineable: Open-World Thermal Image Restoration
May 16, 2026Pu Li, Huafeng Li, Yafei Zhang +3Continual Learning All-in-One Image Restoration
Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Yunnan, China · School of Mathematics and Statistics, Yunnan University, Yunnan, China · Shenzhen Key Laboratory of Visual Object Detection and Recognition, Harbin Institute of Technology, Shenzhen, China
No Free Swap: Protocol-Dependent Layer Redundancy in Transformers
May 15, 2026Gabriel GarciaTransformer Transformer Pruning
Independent Researcher
Ghosted Layers: Unconstrained Activation Alignment for Recovering Layer-Pruned LLMs
May 15, 2026Vincent-Daniel Yun, Junhyuk Jo, Sai Praneeth Karimireddy +1LLM Pruning LLM Compression
University of Southern California, USA · Inha University, Republic of Korea
On the Stability of Growth in Structural Plasticity
May 14, 2026Lute Lillo, Nick CheneyDynamic Neural Networks Neural Network Growth
University of Vermont
TAPIOCA: Why Task- Aware Pruning Improves OOD model Capability
May 14, 2026Krish Sharma, Omar Naim, Soumadeep Saha +3Representation Geometry LLM Pruning
ANITI, France · Meta · Apple, USA
Spectral structural distortion reveals redundant neurons in neural networks
May 14, 2026Yongyu WangLLM Pruning Neural Network Compression
Winning Lottery Tickets in Neural Networks via a Quantum-Inspired Classical Algorithm
May 13, 2026Natsuto Isogai, Hayata Yamasaki, Sho Sonoda +1Lottery Ticket Hypothesis Quantum-Inspired ML
The University of Tokyo · RIKEN AIP
MedCore: Boundary-Preserving Medical Core Pruning for MedSAM
May 13, 2026Cenwei Zhang, Suncheng Xiang, Lei YouModel Compression Image Segmentation
Shanghai Jiao Tong University, Shanghai, China · Technical University of Denmark, Ballerup, Denmark
Selection Plateau and a Sparsity-Dependent Hierarchy of Pruning Features
May 10, 2026Guangqi Li, Yongxin LiSparse Neural Networks Neural Network Compression
Zaozhuang University, Zaozhuang, Shandong, China
Relative Kinetic Utility: Calibrating Cross-Layer Credit for Global Structured LLM Pruning
May 9, 2026Tianhao Qian, Guilin Qi, Jiayu ChenLLM Pruning Structured Sparsity
School of Mathematics, Southeast University Nanjing 210096, China
Compact SO(3) Equivariant Atomistic Foundation Models via Structural Pruning
May 9, 2026Chen Wang, Siyu Hu, Guangming Tan +1Equivariant GNNs Structured Pruning
State Key Lab of Processors, Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences
Understanding Performance Collapse in Layer-Pruned Large Language Models via Decision Representation Transitions
May 8, 2026Boyu Shi, Chang Liu, ChuanBao Gao +2LLM Pruning LLM Interpretability
School of Computer Science and Engineering, Southeast University, Nanjing, China · Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China
Task Relevance Is Not Local Replaceability: A Two-Axis View of Channel Information
May 8, 2026Houman Safaai, Andrew T. Landau, Celia C. Beron +2Channel Pruning Structured Pruning
Kempner Institute for the Study of Natural and Artificial Intelligence Harvard University, Cambridge, MA 02138, USA · Department of Neurobiology, Howard Hughes Medical Institute Harvard Medical School, Boston, MA 02115, USA
SparseForge: Efficient Semi-Structured LLM Sparsification via Annealing of Hessian-Guided Soft-Mask
May 7, 2026Liu Hanzuo, Chaofan Lin, Weixuan Sun +4LLM Pruning Sparse Recovery
†Tsinghua University · ‡Tencent
Importance-Guided Basis Selection for Low-Rank Decomposition of Large Language Models
May 2, 2026Daniel Agyei Asante, Ernie Chang, Yang LiLLM Pruning LLM Compression
Iowa State University · Meta
Post-Optimization Adaptive Rank Allocation for LoRA
Apr 30, 2026Vishnuprasadh Kumaravelu, Sunil Gupta, P. K. SrijithLow-Rank Adaptation Low-Rank Compression
1Indian Institute of Technology Hyderabad · 2Deakin University
Exploring the Limits of Pruning: Task-Specific Neurons, Model Collapse, and Recovery in Task-Specific Large Language Models
Apr 29, 2026M. K. Khalidi Siam, Md. Tausif-Ul-Islam, Md. Reshad Romim Khan +5LLM Pruning Language Modeling
1BRAC University
Revisiting the Effectiveness of LLM Pruning for Test-Time Scaling
Apr 28, 2026Ocean Monjur, Shahriar Kabir Nahin, Anshuman ChhabraLLM Pruning Test-Time Scaling
Bellini College of AI, Cybersecurity, and Computing University of South Florida Tampa, FL, USA
Rethinking Layer Redundancy: Calibration Matters More Than Search in LLM Depth Pruning
Apr 27, 2026Minkyu Kim, Vincent-Daniel Yun, Youngrae Kim +3LLM Pruning LLM Inference Acceleration
Neural Superintelligence Lab, MODULABS, Republic of Korea · University of Southern California, United States · Boston University, United States +2
Structural Pruning of Large Vision Language Models: A Comprehensive Study on Pruning Dynamics, Recovery, and Data Efficiency
Apr 27, 2026Yiran Huang, Lukas Thede, Massimiliano Mancini +2Large Vision-Language Models Structured Pruning
Technical University of Munich, Germany · Munich Center for Machine Learning, Germany. · Helmholtz Munich, Germany. +3
Pruning via Causal Attribution Preserves Reasoning Performance in Large Language Models
Apr 27, 2026Amogh Sheth, Andrew Lin, Biruk Assefa +2LLM Pruning Self-Attention
Edison Academy Magnet School · Massachusetts Institute of Technology · State University of New York College at Plattsburgh +2
Supernodes and Halos: Loss-Critical Hubs in LLM Feed-Forward Layers
Apr 26, 2026Audrey Cherilyn, Houman SafaaiTransformer Interpretability LLM Pruning
Kempner Institute at Harvard University
Learn&Drop: Fast Learning of CNNs based on Layer Dropping
Apr 25, 2026Giorgio Cruciata, Luca Cruciata, Liliana Lo Presti +2Deep Learning Optimization Convolutional Neural Networks
Engineering Department, University of Palermo, V.le delle Scienze, Ed. 6, Palermo, 90128, Italy. · Computer Vision Lab, Delft University of Technology, Van Mourik Broekmanweg 6,2026 Delft, 2628 XE, The Netherlands.
SimDiff: Depth Pruning via Similarity and Difference
Apr 21, 2026Yuli Chen, Shuhao Zhang, Fanshen Meng +4LLM Pruning LLM Inference Acceleration
Pruning Unsafe Tickets: A Resource-Efficient Framework for Safer and More Robust LLMs
Apr 17, 2026Wai Man Si, Mingjie Li, Michael Backes +1LLM Pruning LLM Safety Alignment
CISPA Helmholtz Center for Information Security
A Comparative Study of CNN Optimization Methods for Edge AI: Exploring the Role of Early Exits
Apr 16, 2026Nekane Fernandez, Ivan Valdes, Steven Van Vaerenbergh +2Efficient Inference Edge Inference
Nekane Fernandez, Ivan Valdes, Idoia de la Iglesia and Julen Arratibel are with the Dept. of Connected and Distributed Intelligence, Ikerlan Technology Research Centre, Arrasate, Spain · Nekane Fernandez and Steven Van Vaerenbergh are with the Dept. of Mathematics, Statistics and Computation, Universidad de Cantabria, Santander, Spain
MOONSHOT : A Framework for Multi-Objective Pruning of Vision and Large Language Models
Apr 14, 2026Gabriel Afriat, Xiang Meng, Shibal Ibrahim +2LLM Pruning Neural Network Pruning
Operations Research Center Massachusetts Institute of Technology · Google · OpenAI +1
Topology-Aware Layer Pruning for Large Vision-Language Models
Apr 14, 2026Pengcheng Zheng, Chaoning Zhang, Ya Wen +10LVLM Pruning Neural Network Pruning
University of Electronic Science and Technology of China · Kyung Hee University · Nota Inc. +2

May 18, 2026J/K move · Enter open · S save

Prune, Update and Trim: Robust Structured Pruning for Large Language Models

Diego Coello de Portugal Mecke, Tom Hanika, Lars Schmidt-Thieme

ISMLL & DARC VWFS · University of Hildesheim · Hildesheim, Germany+1

LLM Pruning LLM Inference Acceleration Structured Pruning Attention Head Pruning Neural Network Pruning Transformer Pruning

Open paper AskPDF arXiv

Large Language Models (LLMs) have experienced significant growth and development in recent years. However, performing inference on LLMs remains costly, especially for long-context inference or in resource-constrained devices. This motivates the development of new post-training pruning (PTP) methods. These methods reduce LLMs' requirements by removing a substantial part of the model's parameters. The discarded weights are selected depending on their impact on the models performance. Current PTP methods prune the models by removing the less informative hidden nodes from the FFN layers, and the least important attention layers. We propose Putri, a PTP method that introduces three changes to the State-of-the-art. First, we update the un-pruned weights of the FFN to compensate for the introduced pruning error. Second, the FFN layers are pruned sequentially, taking into account the updates done to the previous layers. Third, instead of removing full attention layers, we remove individual attention-heads. We extend this method such that it can also address Grouped-Query Attention. In summary, Putri is a structure pruning method which remains simple while showing SOTA performance. Pruning experiments on multiple models with a wide variety of sparsity ranges and on different datasets, validate the generality of Putri. Notably, we demonstrate that, unlike previous methods, Putri can prune LLMs on extreme sparsity ratios. The code is available at: https://github.com/Coello-dev/Putri.