Neural Network Pruning

Ask about this topic

Overview

Neural network pruning studies how to remove parts of a model that contribute little to its predictions, reducing memory use, computation, energy, or latency. Methods range from magnitude-based removal to scores using gradients, Fisher information, curvature, activations, redundancy, or task-specific behavior; they may prune individual weights or structured units such as channels, attention heads, and layers. Pruning can happen during training or after training in one shot or iteratively, often with fine-tuning, distillation, or compensation to recover lost performance. A central challenge is that the best pruning decisions depend on the model, task, calibration data, and hardware, so lower parameter counts do not always translate into faster inference or preserve reliability and reasoning behavior.

Latest papers

CardsList

292 papers37 papers in the last four weeks, up 3% on the four weeks before. 0.2% of all new papers.

  1. S3^3: Structured Sparsity Specification

    Apr 13, 2026Ayoub GhrissStructured SparsityStructured Pruning

    Department of Computer Science, Colorado University, Boulder

  2. Engineering Resource-constrained Software Systems with DNN Components: a Concept-based Pruning Approach

    Apr 11, 2026Federico Formica, Andrea Rota, Aurora Francesca Zanenga +4Neural Network PruningNeural Network Compression

    University of Bergamo, Bergamo, Italy

  3. Modality-Aware Zero-Shot Pruning and Sparse Attention for Efficient Multimodal Edge Inference

    Apr 10, 2026Yueyuan Sui, Payal Mohapatra, Doğaç Eldenk +5Efficient Multimodal InferenceNeural Network Pruning

    Northwestern University, Evanston, Illinois, USA

  4. Pruning Extensions and Efficiency Trade-Offs for Sustainable Time Series Classification

    Apr 9, 2026Raphael Fischer, Angus Dempster, Sebastian Buschjäger +3Time Series ClassificationEnergy-Efficient ML

    Lamarr Institute for ML and AI, TU Dortmund University, Dortmund, Germany · Department of Data Science & AI, Monash University, Melbourne, Australia · Department of Pulmonary Medicine, University Medicine Essen, Essen, Germany +1

  5. SubFLOT: Submodel Extraction for Efficient and Personalized Federated Learning via Optimal Transport

    Apr 8, 2026Zheng Jiang, Nan He, Yiming Chen +1Federated LearningHeterogeneous Federated Learning

    Tsinghua University · Beijing University of Technology · Key Laboratory of Pervasive Computing, Ministry of Education

  6. Does a Global Perspective Help Prune Sparse MoEs Elegantly?

    Apr 8, 2026Zeliang Zhang, Nikhil Ghosh, Jiani Liu +2Mixture-of-Experts PruningNeural Network Pruning

    University of Rochester · Flatiron Institute · University of California, Berkeley +1

  7. Neural Network Pruning via QUBO Optimization

    Apr 7, 2026Osama Orabi, Artur Zagitov, Hadi Salloum +3Neural Network PruningQuadratic Unconstrained Binary Optimization

    Department of Computer Science and Engineering, Innopolis University, 420500 Innopolis, Russia · Q Deep, Laboratory of Quantum Computing, 420502 Innopolis, Russia · Research Center of the Artificial Intelligence Institute, Innopolis University, 420500 Innopolis, Russia +2

  8. Prune-Quantize-Distill: An Ordered Pipeline for Efficient Neural Network Compression

    Apr 5, 2026Longsheng Zhou, Yu ShenNeural Network CompressionEfficient Neural Network Inference

    Institute of Advanced Technology University of Science and Technology of China Hefei, China · Supercomputing Center University of Science and Technology of China Hefei, China

  9. Collaborative Multi-Mode Pruning for Vision-Language Models

    Apr 3, 2026Zimeng Wu, Yunhong Wang, Donghao Wang +1LVLM PruningEfficient VLM Inference

    State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, China · School of Computer Science and Engineering, Beihang University, Beijing, China

  10. DDCL-INCRT: A Self-Organising Transformer with Hierarchical Prototype Structure (Theoretical Foundations)

    Apr 2, 2026Giansalvo CirrincioneTransformerHierarchical Representation Learning

    Laboratory LTI, Université de Picardie Jules Verne, Chemin du Thil, Amiens, 80025, France

  11. A Hierarchical Importance-Guided Multi-objective Evolutionary Framework for Deep Neural Network Pruning

    Apr 1, 2026Zak Khan, Azam Asilian BidgoliNeural Network PruningNeural Network Compression

    Department of Computer Science and Physics, Wilfrid Laurier University, Waterloo, N2L 3C5, Ontario, Canada

  12. On the Role of Encoder Depth: Pruning Whisper and LoRA Fine-Tuning in SLAM-ASR

    Mar 30, 2026Ganesh Pavan Kartikeya Bharadwaj Kolluri, Michael Kampouridis, Ravi ShekharNeural Network PruningSpeech Language Models

    School of Computer Science and Electronic Engineering, University of Essex

  13. PQuantML: A Tool for End-to-End Hardware-aware Model Compression

    Mar 27, 2026Roope Niemi, Anastasiia Petrovych, Arghya Ranjan Das +9Model CompressionNeural Network Compression

    European Center for Nuclear Research (CERN), CH-1211 Geneva, Switzerland · Purdue University, West Lafayette, IN 47907, USA · California Institute of Technology, Pasadena, CA 91125, United States +4

  14. Mechanistically Interpreting Compression in Vision-Language Models

    Mar 26, 2026Veeraraju Elluru, Arth Singh, Roberto Aguero +3Cultural Safety Evaluation of VLMsLVLM Pruning

    Indian Institute of Technology Jodhpur · AIM Intelligence · National Institute of Technology Agartala +3

  15. Sparse-by-Design Cross-Modality Prediction: L0-Gated Representations for Reliable and Efficient Learning

    Mar 26, 2026Filippo CenacchiActivation SparsityNeural Network Pruning

    School of Computing, Macquarie University, Sydney, Australia

  16. Diet Your LLM: Dimension-wise Global Pruning of LLMs via Merging Task-specific Importance Score

    Mar 25, 2026Jimyung Hong, Jaehyung KimLLM PruningStructured Pruning

    Yonsei University

  17. A Theoretical Framework for Energy-Aware Gradient Pruning in Federated Learning

    Mar 23, 2026Emmanouil M. AthanasakosFederated LearningEnergy-Efficient ML

    National and Kapodistrian University of Athens, Department of Informatics and Telecommunications, Athens, Greece

  18. Compressed-Sensing-Guided, Inference-Aware Structured Reduction for Large Language Models

    Mar 22, 2026Andrew KirulutaLLM Inference AccelerationLLM Pruning

    UC Berkeley School of Information

  19. Structural Sensitivity in Compressed Transformers: Relative Error Propagation and Layer Removal

    Mar 22, 2026Abhinaba Basu, Kumkum Basu, Koushik DebTransformer PruningLLM Compression

    Indian Institute of Information Technology Allahabad, India · National Institute of Electronics and Information Technology (NIELIT), India · Indian Institute of Technology Patna, India +2

  20. Centrality-Based Pruning for Efficient Echo State Networks

    Mar 21, 2026Sudip LaudariReservoir ComputingNeural Network Pruning

    Independent Researcher.

  21. SIMPLER: Efficient Foundation Model Adaptation via Similarity-Guided Layer Pruning for Earth Observation

    Mar 20, 2026Víctor Barreiro, Johannes Jakubik, Francisco Argüello +1Geospatial Foundation ModelsVision Foundation Model Adaptation

    Centro Singular de Investigación en Tecnoloxías Intelixentes (CiTIUS), Universidade de Santiago de Compostela, Spain · Departmento de Electrónica e Computación, Universidade de Santiago de Compostela, Spain · IBM Research Europe, Zurich, Switzerland

  22. Growing Networks with Autonomous Pruning

    Mar 20, 2026Charles De Lambilly, Stefan DuffnerDynamic Sparse TrainingNeural Network Growth

    INSA Lyon, CNRS, Ecole Centrale de Lyon, Universite Claude Bernard Lyon 1, Université Lumière Lyon 2, LIRIS, UMR5205, , Villeurbanne, 69621, France

  23. Prune-then-Quantize or Quantize-then-Prune? Understanding the Impact of Compression Order in Joint Model Compression

    Mar 19, 2026Minjun Kim, Jaehyeon Choi, Hyunwoo Yang +3Model CompressionNeural Network Quantization

    Seoul National University, Seoul, South Korea

  24. Dynamic sparsity in tree-structured feed-forward layers at scale

    Mar 18, 2026Reza Sedghi, Robin Schiewer, Anand Subramoney +1Transformer FFNsStructured Sparsity

    CITEC, Bielefeld University, Germany · Department of Computer Science, Royal Holloway, University of London, UK

  25. DANCE: Dynamic 3D CNN Pruning: Joint Frame, Channel, and Feature Adaptation for Energy Efficiency on the Edge

    Mar 18, 2026Mohamed Mejri, Ashiqur Rasul, Abhijit ChatterjeeEnergy-Efficient MLNeural Network Pruning

    School of Electrical and Computer Engineering, Georgia Tech, Atlanta, GA, USA

  26. Fundamental Limits of Neural Network Sparsification: Evidence from Catastrophic Interpretability Collapse

    Mar 18, 2026Dip Roy, Rajiv Misra, Sanjay Kumar SinghActivation SparsityNeural Network Interpretability

    Department of Computer Science and Engineering, Indian Institute of Technology, Patna, India · Department of Computer Science, Rajarshi School of Management Technology, Varanasi, India

  27. Mix-and-Match Pruning: Globally Guided Layer-Wise Sparsification of DNNs

    Mar 17, 2026Danial Monachan, Samira Nazari, Mahdi Taheri +4Neural Network CompressionNeural Network Pruning

    Brandenburg Technical University, Cottbus, Germany · University of Zanjan, Iran · Tallinn University of Technology, Estonia +1

  28. Understanding Pruning Regimes in Vision-Language Models Through Domain-Aware Layer Selection

    Mar 17, 2026Saeed Khaki, Nima Safaei, Kamal GinotraLVLM PruningNeural Network Pruning

    Ohio State University

  29. Mostly Text, Smart Visuals: Asymmetric Text-Visual Pruning for Large Vision-Language Models

    Mar 16, 2026Sijie Li, Biao Qian, Jungong HanLVLM PruningNeural Network Pruning

    University of Sheffield, UK · Tsinghua University, China

  30. Flash-Unified: A Training-Free and Task-Aware Acceleration Framework for Native Unified Models

    Mar 16, 2026Junlong Ke, Zichen Wen, Boxue Yang +6Efficient Multimodal InferenceUnified Multimodal Models

    Tsinghua University · Shanghai Jiao Tong University · Shanghai AI Laboratory +3