Neural Network Pruning

Follow this topic Ask about this topic

Momentum

22 papers in the last four weeks, up 83% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 161

All topics

DADP: Dynamic Activity-Dependent Pruning, A Reverse Hebbian-Inspired Structural Pruning Method
Oct 8, 2026Bhushan DeshpandeNeural Network Pruning Neural Network Compression
Independent Researcher, Pune, India
When Can You Prune Your Network? A Study of Intermediate Neurons in Multilingual Speech Parsing
Oct 8, 2026Minnie Kabra, Benjamin Lecouteux, Maximin CoavouxNeural Network Pruning
Univ. Grenoble Alpes, CNRS, Grenoble INP, LIG, 38000 Grenoble, France
CHASE: Channel-Aligned Structure Exploitation for Geometry-Aware Model Engineering
Oct 7, 2026Wei Wang, Wei Jiang, Ziran LiuNeural Network Compression KV-Cache Compression
Futurewei Technologies San Jose, CA 95131, USA · Shanghai Institute for Mathematics and Interdisciplinary Sciences (SIMIS) Shanghai, 200433, China
Towards Efficient Robotic Manipulation Models with Self-Recursive Pruning
Oct 6, 2026Zijia Chen, Yuenan Hou, Yu Li +2Robot Policy Learning Robotic Manipulation
College of Electronic Science and Technology, National University of Defense Technology Changsha, 410073, China · Shanghai AI Laboratory Shanghai, 200000, China
Task-Aware Joint Pruning and Distillation for Efficient Audio Deepfake Detection
Oct 4, 2026Miao He, Peng Cheng, Zhongjie Ba +4Audio Deepfake Detection Knowledge Distillation
The State Key Laboratory of Blockchain and Data Security, Zhejiang University, China · Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security, China · Qingdao Institute of Software College of Computer Science and Technology, China University of Petroleum (East China), China +1
QK-Wanda: Coupling Queries and Keys for Unstructured Pruning
Oct 1, 2026Ivan Ilin, Peter RichtárikLLM Pruning Neural Network Pruning
KAUST
MWOP: Modality-aware Width-wise Operation Pruning for Efficient MLLMs
Oct 1, 2026Xudong Wang, Hao Wu, Haozhe Hu +5LLM Pruning Efficient VLM Inference
EIT-NLP Lab, Eastern Institute of Technology, Ningbo · Shanghai Jiao Tong University · The Hong Kong Polytechnic University +1
DIET: Deletion-response Expert Trimming for Video Diffusion Transformers
Sep 29, 2026Jiachang Zhang, Teng Hu, Bohao Feng +4Mixture-of-Experts Pruning Video Diffusion Models
Xi’an Jiao Tong University, China · Shanghai Jiao Tong University, China · Alibaba Token Hub, Alibaba Group, China +1
Does the VGGT Family Need All Its Layers?
Sep 29, 2026Fengyi Zhang, Holger Caesar, Xiangyu Sun +3Multi-View 3D Reconstruction Feed-Forward 3D Reconstruction
The University of Queensland · Delft University of Technology · Harbin Institute of Technology
Output-aware Residual Stream Pruning for Large Language Models
Sep 28, 2026Chayne Thrash, Kevin Chen, Soheil KolouriLLM Pruning LLM Compression
Department of Computer Science Vanderbilt University Nashville, TN 37235, USA
GroupMask: Layer-Adaptive Group-wise Sparsity for Semi-Structured LLM Pruning
Sep 27, 2026Zhengao Li, Shuoqiu Li, Xiaofang Zhang +7LLM Pruning LLM Compression
Florida State University · University of Connecticut · College of William & Mary +1
Fisher Simplicity in Kolmogorov-Arnold Networks and Multilayer Perceptrons
Sep 26, 2026Ami Tavory, Meir FederNeural Network Interpretability Kolmogorov-Arnold Networks
Meta Platforms Tel-Aviv, Israel · School of Electrical and Computer Engineering Tel-Aviv University Tel-Aviv, Israel
AERIAL: Adversarial Evaluation of Robustness in Accuracy-Preserving Low-Precision EEG Decoders
Sep 24, 2026Saim Rehman, Muhammad ShafiqueQuantization-Aware Training EEG Decoding
eBRAIN Lab, Division of Engineering, New York University Abu Dhabi (NYUAD), Abu Dhabi, UAE
Six Layers Less: Encoder Pruning for Whisper with Label-Free Recovery
Sep 23, 2026Rasmus Aagaard, Nicki Skafte DetlefsenSpeech Foundation Models Automatic Speech Recognition
Technical University of Denmark · Laerdal Medical
DTKDP: A Dual Teacher Knowledge Distillation and Pruning Framework for Lightweight Oriented SAR Ship Detection
Sep 21, 2026Yuming Li, Fan Zhang, Alin M. AchimMulti-Teacher Knowledge Distillation Object Detection
Visual Information Labs, University of Bristol, Bristol BS1 6AZ, UK
Artificial Structure Function Search: Preserving Artificial Functional Connectivity for Structured Pruning
Sep 21, 2026Mindula Illeperuma, Rafael Pina, Charuka Herath +2Model Compression Functional Connectivity
Intitute for Digital Technologies, Loughborough University London · Center for Neuromorphic Intelligence, Aston University
Prescriptive SVD-Inspired Attention via Spectral Energy Retention
Sep 21, 2026Vasileios Arampatzakis, Vasileios Sevetlidis, George PavlidisTransformer Interpretability Low-Rank Attention
Athena Research Center, Greece
Higher-order pruning of experts in mixture-of-experts language models
Sep 16, 2026Alex M. Tseng, Prannay Kaul, Luca Zancato +2Mixture-of-Experts Pruning Mixture-of-Experts Language Models
AI Fundamental Research AWS Agentic AI
Theoretical Guarantees for One-Shot Magnitude Pruning and Compute-Adaptive Early Exit
Sep 14, 2026Erdem KoyuncuNeural Network Generalization Early-Exit Neural Networks
Department of Electrical and Computer Engineering, University of Illinois Chicago, IL, USA.
X-RACE: XAI-assisted Recurrent neural network Attribution for Channel Estimation
Sep 12, 2026Abdul Karim Gizzini, Yahia MedjahdiEfficient Neural Network Inference Explainable Artificial Intelligence
University of Paris-Est Cr´eteil (UPEC), LISSI/TincNET, F-94400, Vitry-sur-Seine, France. · IMT Nord Europe, Institut Mines T´el´ecom, Centre for Digital Systems, F-59653 Villeneuve d’Ascq, France.
X-AuT: Progressive Audio-Encoder Compression for Speech LLMs with Cross-Scale Distillation
Sep 12, 2026Haojun Zhang, Yi Zou, Min Chen +7LLM Compression Language Model Distillation
XPeng Inc.
LILA: Calibration-Free Structured Pruning of Large Language Models via Latent Spectral Geometry
Sep 11, 2026Sankar Behera, Dhruv Singh, Anshika Agnihotri +3LLM Pruning LLM Compression
CSE, IIT Jammu · Mathematics, IIT Jammu · IT, EC Bikaner +1
One Loop, Two Gains: Can Active Learning win the Lottery for Free?
Sep 9, 2026Benedikt Tscheschner, Eduardo Veas, Marc MasanaActive Learning Neural Network Pruning
University of Technology Graz Know-Center Research GmbH Austria · Institute of Visual Computing University of Technology Graz Austria
LinearMask-GS: Stable-Mask Importance Pruning for Compact 3D Gaussian Splatting
Sep 9, 2026Donghun Ryu, Minhyeok Lee3DGS Compression 3D Gaussian Splatting
Department of Intelligent Semiconductor Engineering Chung-Ang University Seoul, Republic of Korea · School of Electrical and Electronics Engineering Chung-Ang University Seoul, Republic of Korea
Forward-Free LLM Depth Pruning via Weight Redundancy
Sep 9, 2026Vincent-Daniel Yun, Woosang LimLLM Pruning LLM Inference Acceleration
University of Southern California, United States · 2Seoul National University, Republic of Korea
Understanding the Impact of Model Pruning on Long-Tail Forgetting and Explanation Reliability in Medical Imaging
Sep 7, 2026Nazish Khalid, Tausifa Jan Saleem, Amal Saqib +2Model Compression Explainable Medical Image Analysis
Department of Electrical and Computer Engineering, Missouri University of Science and Technology, Rolla, MO 65409 USA · Computing and Mathematical Sciences Division, Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, United Arab Emirates
TAP-Path: Task-Adaptive Structural and Token Pruning for Efficient and Trustworthy Pathology Foundation Models
Sep 3, 2026Mehedi Hasan, Ashfak Yeafi, Md Khairul IslamComputational Pathology Pathology Foundation Models
Department of Computer Science and Engineering, Brac University, Dhaka, Bangladesh · EEE, Khulna University of Engineering & Technology, Khulna, Bangladesh · Mathematics and Computer Science, Hobart and William Smith Colleges, Geneva, NY, USA
Measurement-Driven Sub-Network Selection for On-Premise Retrieval-Augmented Factory Agents
Sep 2, 2026Vasileios Rizeakos, Georgios Paisios, Alexandros Machairas +2Hardware-Aware NAS Edge Computing
AI lab of enakronIC PC · Aspasias 72, 15561, Cholargos, Greece · Electrical & Computer Engineering +2
Debias-SparseGPT: Bias-Aware Pruning for Large Language Models
Sep 2, 2026Irina Proskurina, Guillaume Metzler, Antoine Gourru +1LLM Pruning Neural Network Pruning
1Laboratoire Hubert Curien, UMR CNRS 5516, Saint-Étienne, France · 2Université Claude Bernard Lyon 1, Université Lumière Lyon 2, ERIC · 3École Centrale de Lyon, LIRIS, CNRS UMR 5205
Scalable Kronecker-Fisher Approximation: Efficient Hessian Analysis for Billion-Parameter Language Models Compression
Sep 2, 2026Viacheslav Yusupov, Daria Cherniuk, Evgeny FrolovLLM Compression Fisher Information
1HSE University · 2AXXX

Oct 8, 2026, cs.LGJ/K move · Enter open · S save

DADP: Dynamic Activity-Dependent Pruning, A Reverse Hebbian-Inspired Structural Pruning Method

Bhushan Deshpande

Independent Researcher, Pune, India

Neural Network Pruning Neural Network Compression Structured Sparsity

Open paper AskPDF arXiv

Modern neural networks are heavily over-parameterized. This redundancy incurs substantial compute and memory overhead during training and inference. Existing pruning methods rely on post-hoc magnitude thresholds or static initialization heuristics. Consequently, they often require manual per-layer sparsity targets or expensive retraining cycles. We propose Dynamic Activity-Dependent Pruning (DADP), a biologically inspired structural plasticity mechanism. During training, DADP measures connection importance via the accumulated product of pre-synaptic activations and post-synaptic error gradients. Using a single global threshold instead of fixed layer budgets, DADP dynamically allocates sparsity across network depth while naturally inducing neuron- and channel-level pruning. Across MLP, VGG-16, ResNet-18, BiLSTM-CRF, and MiniBERT architectures, DADP matches or outperforms Magnitude, SNIP and RigL, retaining 73.67% accuracy (dense baseline: 76.06%) at 99% sparsity on ResNet-18. Finally, matrix-based Shannon entropy and effective rank measurements confirm that DADP preserves latent feature diversity at extreme sparsities without representation collapse.

Momentum

Related topics

Latest papers 161