Neural Network Pruning

Follow this topic Ask about this topic

Neural network pruning studies how to remove parts of a model that contribute little to its predictions, reducing memory use, computation, energy, or latency. Methods range from magnitude-based removal to scores using gradients, Fisher information, curvature, activations, redundancy, or task-specific behavior; they may prune individual weights or structured units such as channels, attention heads, and layers. Pruning can happen during training or after training in one shot or iteratively, often with fine-tuning, distillation, or compensation to recover lost performance. A central challenge is that the best pruning decisions depend on the model, task, calibration data, and hardware, so lower parameter counts do not always translate into faster inference or preserve reliability and reasoning behavior.

Latest papers

292 papers37 papers in the last four weeks, up 3% on the four weeks before. 0.2% of all new papers.

DAC-Loop: Distillation-Aware Compression for Remote Sensing CNN
Oct 26, 2026Rico Staedeli, Lorena Raichle, Konstantinos Tzevelekakis +2Neural Network Compression Neural Network Pruning
AIML Lab, School of Computer Science, University of St. Gallen
FlowGenesis: Representation-Driven Self-Organization of Neural Architectures
Oct 26, 2026Maksim A KazanskiiDynamic Neural Networks Neural Network Growth
Independent Researcher
Improving the Chain-of-Thought Compressibility of Pruned Reasoning Models via Long--Short Contrastive Pruning
Oct 25, 2026Tao Jing, Chen Kang, Feifei Zhai +2Efficient Language Model Reasoning LLM Pruning
Decoupled Prune-and-Reconstruct: Training on a Compact Core with Double-Layer Sparsification
Oct 25, 2026Siqi Gu, Gaozhen Ma, Liheng Huang +4Graph Neural Networks Memory-Efficient Training
School of Physics and Electronic Engineering, Nanyang Normal University, Wolong District, Nanyang, 473061, Henan, China · School of Integrated Circuits, Huazhong University of Science and Technology, 1037 Luoyu Road, Hongshan District, Wuhan, 430074, Hubei, China · School of Architecture and Urban Planning, Huazhong University of Science and Technology, 1037 Luoyu Road, Hongshan District, Wuhan, 430074, Hubei, China +1
Sparse Bayesian Neural Networks via Sticky Piecewise Deterministic Markov Processes
Oct 23, 2026August Arnstad, Geir Olve Storvik, Filippo Pagani +1Bayesian Neural Networks Sparse Neural Networks
Department of Mathematics, University of Oslo, NO · Department of Statistics, University of Warwick, UK
Building a Nested LADDER for LLM Unstructured Post-Training Pruning
Oct 20, 2026Nicolás Mauricio Cuadrado Ávila, Ivo Gollini Navarrete, Martin Takáč +1LLM Pruning Neural Network Pruning
Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, United Arab Emirates
AC/DC on a Budget - Alternating Sparse Phases
Oct 20, 2026Rahul Nittala, Advait Gadhikar, Tom Jacobs +1Neural Network Compression Neural Network Pruning
CISPA Helmholtz Center for Information Security, Germany
Reconstruction-Optimized Expert Pruning for Mixture-of-Experts Language Models
Oct 19, 2026Jelena Markovic-Voronov, Petros Prastakos, Kayhan Behdin +7Mixture-of-Experts Pruning Mixture-of-Experts Language Models
LinkedIn · MIT · Google
PDNet: Exploring Pruning and Distillation for Real-Time Worker Activity Recognition in Industrial Assembly
Oct 19, 2026Malek Baba, Thinesh Thiyakesan Ponbagavathi, Alina RoitbergHuman Activity Recognition Neural Network Pruning
Stuttgart University, Stuttgart, Germany · Institute for Artificial Intelligence (KI), Stuttgart University, Stuttgart, Germany · Intelligent Assistive Systems (IAS) group, University of Hildesheim, Hildesheim, Germany
SPAR: Structural Pruning and Action Recovery for Vision-Language-Action Models
Oct 19, 2026Shenli Zheng, Zhou Fang, Yuan Du +2Efficient VLA Model Inference LVLM Pruning
Nanjing University, Nanjing, China · Nanjing University of Information Science and Technology, Nanjing, China
LOOP: Recovery-Aligned Iterative Layer Pruning via On-Policy Distillation
Oct 19, 2026Liang Xu, Shufan Shen, Zhongni Hou +4LLM Pruning Neural Network Pruning
State Key Lab. of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Meituan
Visual Sufficiency Landscapes: Understanding Query-Conditioned Reliability in Visual Token Pruning
Oct 18, 2026Zhenghan Li, Kexin Sun, Wei Zheng +3Visual Token Pruning LVLM Pruning
School of Software, Northwestern Polytechnical University, Xi’an 710129, China
Replay Pruning and Few-Bit Integer Inference for Hardware-Efficient Speed-Sign Recognition
Oct 16, 2026Se Un Park, Kyungkeon Chung, Huigeol Kim +1Channel Pruning Efficient Inference
UX Factory, Inc. · BTree Co., Ltd.
Prune the Leaky Neurons: Mitigating Private Data Leakage in Fine-Tuned LLMs via Joint-Embedding-Guided Pruning
Oct 16, 2026Mohamed Shaaban, Mohamed ElmahallawyPrivacy Leakage in Language Models Privacy-Preserving Language Models
School of Engineering and Applied Sciences, Washington State University, Richland, WA 99354, USA
The Dormant Spiking Neuron: A State-Driven Mechanism for Efficient Spiking Neural Networks
Oct 15, 2026Ruichen Ma, Yicai Chen, Pujun Zhou +5Spiking Neural Networks Energy-Efficient ML
University of Electronic Science and Technology of China (UESTC) · Shenzhen Institute for Advanced Study, UESTC
Calibration Language Shapes Block-Influence Pruning in Multilingual Language Models
Oct 15, 2026Vedant Pandhare, Piyush DhokaLLM Pruning Neural Network Pruning
Independent Researcher
Decision-FLIP: Budget-Free Structured Pruning for Gaussian Occupancy Models
Oct 14, 2026Jingxing Li, Anurag Paul, Subhasis Das +1Model Compression Neural Network Pruning
Zoox Inc.
Your Transformer Is Secretly Elastic
Oct 14, 2026Walter Simoncini, Dheeraj Varghese, Gertjan J. Burghouts +1Elastic Neural Networks Transformer Pruning
University of Amsterdam · TNO, Intelligent Imaging
Two For The Price Of One: Curvature-Aware Pruning Preserves Influence Functions Too
Oct 13, 2026Mehdi Makni, Natalia Ponomareva, Rahul MazumderLLM Pruning Neural Network Pruning
Operations Research Center, Massachusetts Institute of Technology · Google Research
When Sparsity Breaks Connectivity: Functional Reachability and Guided Topology Dynamics in Extreme Sparse Training
Oct 12, 2026Mohammad Pasande, Mateusz Jaworski, Jeroen Rook +1Dynamic Sparse Training Sparse Neural Networks
University of Twente, Enschede, Netherlands · Paderborn University, Paderborn, Germany
Carve VFM: Carving Specialists out of Generalists, Automatic Task-Specific Compression of Vision Foundation Models
Oct 12, 2026Lotfi Abdelkrim Mecharbat, Abdelrahman Elsayed, Ibrahim Almakky +2Model Compression Neural Network Pruning
Mohamed bin Zayed University of Artificial Intelligence
BARTER: Compact Dynamic Sparse Training with Standard Dense Kernels
Oct 12, 2026Yueyuan Sui, Payal Mohapatra, Qi Zhu +1Dynamic Sparse Training Neural Network Pruning
Northwestern University, USA · Arm Inc., USA
Heterogeneous Structured Pruning Under Resource Constraints for Hardware-Efficient Large Language Models
Oct 11, 2026Cameron Barker, Andreas Paraskeva, Jan N. van Rijn +2LLM Pruning Neural Network Pruning
School of Informatics, University of Edinburgh, UK · Leiden Institute of Advanced Computer Science, Leiden University, the Netherlands · School of Engineering, University of Edinburgh, UK
Towards Robust Calibration Dataset for Domain-General Mixture of Experts Pruning
Oct 11, 2026Luis Frentzen Salim, Chia-Chun Wu, Tran Van Nhiem +2Mixture-of-Experts Pruning Neural Network Pruning
Institute of Information Science, Academia Sinica · National Taiwan University of Science and Technology · AI Research Center, Hon Hai Research Institute
Retraining-Free Depth Pruning of Vision Transformers: A Cross Domain Study of Layer Redundancy
Oct 10, 2026Alessandro Viespoli, Loris NanniTransformer Pruning Efficient ViTs
University of Padua
Tabular Foundation Models Are Effectively Shallow
Oct 9, 2026Irene Cannistraci, Julia E. VogtTransformer Pruning Tabular Foundation Models
Department of Computer Science ETH Zurich Switzerland
DADP: Dynamic Activity-Dependent Pruning, A Reverse Hebbian-Inspired Structural Pruning Method
Oct 8, 2026Bhushan DeshpandeDynamic Sparse Training Neural Network Pruning
Independent Researcher, Pune, India
When Can You Prune Your Network? A Study of Intermediate Neurons in Multilingual Speech Parsing
Oct 8, 2026Minnie Kabra, Benjamin Lecouteux, Maximin CoavouxNeural Network Pruning
Univ. Grenoble Alpes, CNRS, Grenoble INP, LIG, 38000 Grenoble, France
CHASE: Channel-Aligned Structure Exploitation for Geometry-Aware Model Engineering
Oct 7, 2026Wei Wang, Wei Jiang, Ziran LiuNeural Network Compression KV-Cache Compression
Futurewei Technologies San Jose, CA 95131, USA · Shanghai Institute for Mathematics and Interdisciplinary Sciences (SIMIS) Shanghai, 200433, China
Towards Efficient Robotic Manipulation Models with Self-Recursive Pruning
Oct 6, 2026Zijia Chen, Yuenan Hou, Yu Li +2Robot Policy Learning Robotic Manipulation
College of Electronic Science and Technology, National University of Defense Technology Changsha, 410073, China · Shanghai AI Laboratory Shanghai, 200000, China

Oct 26, 2026J/K move · Enter open · S save

DAC-Loop: Distillation-Aware Compression for Remote Sensing CNN

Rico Staedeli, Lorena Raichle, Konstantinos Tzevelekakis, Aron Asefaw+1

AIML Lab, School of Computer Science, University of St. Gallen

Neural Network Compression Neural Network Pruning Remote Sensing Scene Classification Knowledge Distillation Structured Pruning

Open paper AskPDF arXiv

While large-scale models achieve strong results in remote sensing scene classification, deploying them on edge devices remains difficult due to limited compute, memory, and energy budgets. Compact CNN backbones are therefore a practical choice in many real-world settings, offering efficient inference with well-optimized implementations on embedded hardware. To push this efficiency further, we investigate how far a knowledge-distilled student model can be compressed while preserving high accuracy. We propose a Distillation-Aware Compression Loop (DAC-Loop) that integrates (i) knowledge distillation applied both before and after structured channel pruning, and (ii) an iterative compression schedule that alternates structured pruning with unstructured sparsification. Across EuroSAT and NWPU-RESISC45, the DAC-Loop consistently improves the accuracy-compression trade-off relative to standard one-shot pruning and single-stage KD recovery, maintaining strong accuracy even beyond 90% parameter reduction. These results demonstrate that combining distillation and pruning in a unified iterative loop offers an effective path toward compact remote sensing models.