Neural Network Pruning

Follow this topic Ask about this topic

Momentum

22 papers in the last four weeks, up 83% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 161

All topics

Attribution-Guided and Coverage-Maximized Pruning for Structural MoE Compression
Jun 16, 2026Yifu Ding, Jiacheng Wang, Ge Yang +4Mixture-of-Experts Pruning LLM Pruning
State Key Laboratory of Complex & Critical Software Environment, Beihang University · School of Computer Science and Engineering, Beihang University · Nanyang Technological University +1
DYNA-PRUNER: Input-Adaptive Data-Model Co-Pruning for Efficient and Scalable Spatio-Temporal Media Prediction
Jun 13, 2026Fuyan Zhang, Yuqi Li, Qing Xu +2Efficient Neural Network Inference Spatiotemporal Forecasting
University of Edinburgh · The City College of New York, City University of New York · Institute of Computing Technology, Chinese Academy of Sciences +1
Beyond Layer Importance in Layer-wise Sparsity: An Inter-Layer Perturbation-Absorption Perspective
Jun 13, 2026Tao Jing, Ningxin Wu, Chen Kang +3LLM Pruning LLM Compression
Squeeze-Release: Iterative Pruning with Exact Structural Minimization
Jun 12, 2026Roman Denkin, Ida Akerholm, Prashant Singh +1Structured Pruning Neural Network Compression
Department of Information Technology, Uppsala University, Uppsala, Sweden · Science for Life Laboratory, Uppsala University, Uppsala, Sweden
Efficiency-Performance Trade-offs in Neural Speaker Diarization via Structured Pruning and Low-Bit Quantization
Jun 12, 2026Rishit Chatterjee, Tahiya ChowdhuryModel Compression Speaker Diarization
Department of Computer Science, Colby College, Waterville, Maine, United States
Finding Sparse Subnetworks in One Training Cycle via Progressive Magnitude-Based Pruning
Jun 10, 2026Romana Qureshi, Hafida Benhidour, Said Kerrache +1Sparse Neural Networks Neural Network Pruning
Department of Computer Science, King Saud University, Riyadh, Saudi Arabia
REACH: Interpretability-Driven Feature Identification and Architecture Compression for Multi-Channel Vehicular Channel Estimation
Jun 10, 2026Simbarashe Aldrin Ngorima, Albert Helberg, Marelie H. DavelFeature Attribution OOD Generalization
Faculty of Engineering, North-West University, South Africa · Centre for Artificial Intelligence Research, South Africa · National Institute for Theoretical and Computational Sciences, South Africa
Towards Data-free and Training-free Compression for Speech Foundation Models Using Parameter Clustering
Jun 10, 2026Haoning Xu, Zhaoqing Li, Huimeng Wang +4Speech Foundation Models Clustering
The Chinese University of Hong Kong, Hong Kong SAR, China · National Research Council Canada, Canada
SpenseGPT: Practical One-shot Pruning Enabling Sparse and Dense GEMMs for LLM Inference
Jun 9, 2026Jaeseong Lee, Seung-won Hwang, Samyam RajbhandariLLM Pruning High-Performance Computing
Snowflake AI Research* · Seoul National University†
Less is MoE: Trimming Experts in Domain-Specialist Language Models
Jun 4, 2026Haoze He, Xinkai Zou, Xuan Jiang +4Mixture-of-Experts Pruning LLM Pruning
Carnegie Mellon University · UCSD · MIT
Knockoffs-based False Discovery Rate Control and Simplification for Deep Neural Networks
Jun 3, 2026Wenyu Liao, Yiqing Shi, Fang XieFDR Control Feature Selection
Guangdong Provincial Key Laboratory of IRADS, Beijing Normal-Hong Kong Baptist University, Zhuhai 519087, China
PrimeSVT: An Automated Memory-aware Pruning Framework with Prioritized Compression Policy for Spiking Vision Transformers
Jun 2, 2026Rachmad Vidya Wicaksana Putra, Achyuta Muthuvelan, Alberto Marchisio +1Memory-Efficient Inference Spiking Neural Networks
eBRAIN Lab, Division of Engineering, New York University (NYU) Abu Dhabi, United Arab Emirates (UAE)
PSViT: A Methodology for Structurally Pruning Spiking Vision Transformers
Jun 2, 2026Rachmad Vidya Wicaksana Putra, Achyuta Muthuvelan, Alberto Marchisio +1Efficient ViTs Spiking Neural Networks
eBRAIN Lab, Division of Engineering, New York University (NYU) Abu Dhabi, United Arab Emirates (UAE)
VEDAL: Variational Error-Driven Asynchronous Learning for 3D Gaussian Splatting Pruning
Jun 1, 2026Aoduo Li, Jiancheng Li, Huan Ye +53DGS Compression 3D Gaussian Splatting
Guangdong University of Technology · Huizhou Boluo Power Supply Bureau, Guangdong Power Grid Co., Ltd. · Shenzhen Polytechnic University +1
ProbeScale: Probing Analysis to Optimize Neural Scaling Laws for Efficient Small Language Model Inference
Jun 1, 2026Sourav DasSmall Language Models Efficient Language Model Inference
Department of Computer Science and Engineering Indian Institution of Information Technology Kalyani Kalyani, West Bengal, India
CRePE: Convolution-aware Relative Importance in Post-training Pruning with Efficient Search
Jun 1, 2026Cheonjun ParkLLM Pruning LLM Compression
Hankuk University of Foreign Studies
STARFISH: faST Accuracy Recovery in pruned networks From Internal State Healing
May 31, 2026Shir Maon, Odelia Melamed, Adi ShamirNeural Network Compression Neural Network Pruning
Weizmann Institute of Science
Structured Neuron Pruning in Deep Neural Networks Using Multi-Armed Bandits
May 29, 2026Salem Ameen, Sunil VaderaMulti-Armed Bandits Neural Network Compression
School of Science, Engineering and Environment, University of Salford, Salford, United Kingdom
PrunePath: Towards Highly Structured Sparse Language Models
May 27, 2026Zhexuan Gu, Zixun Fu, Yancheng YuanMixture-of-Experts Pruning LLM Pruning
Department of Applied Mathematics, The Hong Kong Polytechnic University
PARE: Pruning and Adaptive Routing for Efficient Video Generation
May 26, 2026Yutong Wang, Yunke Wang, Tianfan Xue +4Video Diffusion Models Diffusion Transformer
The University of Sydney · The Chinese University of Hong Kong · Shanghai AI Laboratory
Resource-Constrained Affect Modelling via Variance Regularisation Pruning
May 26, 2026Kosmas Pinitas, Konstantinos KatsifisCost-Aware Inference Affective Computing
Mediterranean College, Athens, Greece · University of Derby, Derby, UK
MuCRASP: Multimodal Chain-of-thought Reasoning aware Structured Pruning
May 25, 2026Aritra Dutta, Somak AdityaVision-Language Models Efficient VLM Inference
Indian Institute of Technology, Kharagpur
Towards the Connection between Activation Sparsity and Flat Minima
May 25, 2026Ze Peng, Jian Zhang, Lei Qi +2Efficient Neural Network Inference Activation Sparsity
State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210023, China · Institute of Brain-Machine Interface, Nanjing University, Nanjing 210023, China · School of Computer Science and Engineering, Southeast University, Nanjing 211189, China
Relative Repairability: A Calibration-Based Diagnostic for High-Sparsity Post-Pruning Allocation
May 25, 2026Qishi Zhan, Liang He, Minxuan Hu +1Neural Network Compression Neural Network Pruning
Marquette University · Tongji University · Cornell University +1
Grow-Prune-Freeze Networks: Adaptive & Continual Learning Technique for Olfactory Navigation
May 24, 2026Kordel K. France, Ovidiu DaescuNon-Stationary RL Partially Observable RL
Department of Computer Science, University of Texas at Dallas, USA
Pruning Deep Neural Networks via the Marchenko--Pastur Distribution
May 23, 2026Leonid Berlyand, Theo Bourdais, Houman Owhadi +1Structured Pruning Sparse Neural Networks
Department of Mathematics Pennsylvania State University University Park, PA 16802, USA · Department of Computing and Mathematical Sciences California Institute of Technology Pasadena, CA 91125, USA
Posterior Collapse as Automatic Spectral Pruning
May 21, 2026Johannes HirnVariational Autoencoders Posterior Collapse
1Image Processing Laboratory (IPL), Universitat de València, Paterna, València 46980, Spain∗
Partial Fusion of Neural Networks: Efficient Tradeoffs Between Ensembles and Weight Aggregation
May 21, 2026Fabian Morelli, Stephan EcksteinInverse Optimal Transport Ensemble Learning
Department of Computer Science, University of Tübingen, Germany · Department of Mathematics, University of Tübingen, Germany.
How Sparsity Allocation Shapes Label-Free Post-Pruning Recoverability
May 21, 2026Qishi Zhan, Minxuan Hu, Liang HeSparse Recovery Neural Network Pruning
Marquette University · Cornell University · Tongji University
Adaptive Signal Resuscitation: Channel-wise Post-Pruning Repair for Sparse Vision Networks
May 20, 2026Qishi Zhan, Ziheng Chen, Minxuan HuNeural Network Compression Neural Network Pruning
Department of Mathematical and Statistical Sciences, Marquette University, USA · The University of Texas at Austin, USA · Cornell Ann S. Bowers College of Computing and Information Science, Cornell University, USA

Jun 16, 2026, cs.LGJ/K move · Enter open · S save

Attribution-Guided and Coverage-Maximized Pruning for Structural MoE Compression

Yifu Ding, Jiacheng Wang, Ge Yang, Yongcheng Jing+3

State Key Laboratory of Complex & Critical Software Environment, Beihang University · School of Computer Science and Engineering, Beihang University · Nanyang Technological University+1

Mixture-of-Experts Pruning LLM Pruning Structured Pruning Mixture-of-Experts Models Neural Network Pruning

Open paper AskPDF arXiv

Mixture-of-Experts (MoE) models scale compute efficiently, yet remain expensive to deploy due to their substantial memory footprint and inference overhead. Prior compression methods mainly operate at the expert level, either removing entire experts or ranking experts by coarse-grained importance scores. However, such expert-wise decisions are often too coarse to capture fine-grained redundancy, leading to misallocated pruning budgets and limited compression. To address this problem, we observe that information within MoE experts is highly concentrated in a small subset of channels, leaving substantial redundancy even in experts deemed important. Based on this observation, we propose a structural pruning framework tailored for MoE models. Our method reformulates prune-ratio allocation as a channel-score coverage maximization problem and solves it efficiently using an attribution-based approximation. Experiments on DeepSeek and Qwen MoE models show that our method preserves model accuracy under 50% or 25% structured pruning when combined with 4-bit quantization. On Qwen3-30B-A3B, our approach reduces memory footprint by 5.27

\times

and consistently outperforms state-of-the-art baselines across diverse benchmarks.

Momentum

Related topics

Latest papers 161