Overview

Neural network pruning studies how to remove parts of a model that contribute little to its predictions, reducing memory use, computation, energy, or latency. Methods range from magnitude-based removal to scores using gradients, Fisher information, curvature, activations, redundancy, or task-specific behavior; they may prune individual weights or structured units such as channels, attention heads, and layers. Pruning can happen during training or after training in one shot or iteratively, often with fine-tuning, distillation, or compensation to recover lost performance. A central challenge is that the best pruning decisions depend on the model, task, calibration data, and hardware, so lower parameter counts do not always translate into faster inference or preserve reliability and reasoning behavior.

Latest papers

CardsList

305 papers16 papers in the last four weeks, down 59% on the four weeks before. 0.1% of all new papers.

  1. Is Oracle Pruning the True Oracle?

    Nov 28, 2024Sicheng Feng, Keda Tao, Huan WangNeural Network Pruning

  2. TropNNC: Structured Neural Network Compression Using Tropical Geometry

    Sep 5, 2024Konstantinos Fotopoulos, Petros Maragos, Panagiotis MisiakosNeural Network CompressionNeural Network Pruning

  3. Subspace Node Pruning

    May 26, 2024Joshua Offergeld, John Chidiac, Marcel van Gerven +1Neural Network PruningLLM Pruning

  4. Beyond Size: How Gradients Shape Pruning Decisions in Large Language Models

    Nov 8, 2023Rocktim Jyoti Das, Mukul Ranjan, Mingjie Sun +2LVLM PruningLLM Pruning