DNN Pruning

DNN pruning aims to reduce the size and computational cost of deep neural networks without significantly sacrificing accuracy, improving efficiency for resource-constrained applications like mobile and embedded devices. Current research focuses on developing automated, efficient pruning algorithms, exploring various pruning strategies (structured, unstructured, channel pruning), and adapting these techniques to different network architectures (including convolutional neural networks, transformers, and even spiking neural networks). This work is significant because it enables the deployment of powerful deep learning models on devices with limited resources, impacting fields ranging from computer vision and natural language processing to edge computing and resource-constrained IoT applications.

Papers

September 11, 2024

HESSO: Towards Automatic Efficient and User Friendly Any Neural Network Training and Pruning
Tianyi Chen, Xiaoyi Qu, David Aponte, Colby Banbury, Jongwoo Ko, Tianyu Ding, Yong Ma, Vladimir Lyapunov, Ilya Zharkov, Luming Liang
Edge Pruning Structured Pruning Sparsity Constraint Sparse Optimization Sparse Identification DNN Pruning

May 21, 2024

Combining Relevance and Magnitude for Resource-Aware DNN Pruning
Carla Fabiana Chiasserini, Francesco Malandrino, Nuria Molner, Zhiqiang Zhao
Edge Pruning Machine Learning Pipeline Relevance Score Earthquake Magnitude DNN Pruning Pruning Ratio

November 18, 2023

Pursing the Sparse Limitation of Spiking Deep Learning Structures
Hao Cheng, Jiahang Cao, Erjia Xiao, Mengshu Sun, Le Yang, Jize Zhang, Xue Lin, Bhavya Kailkhura, Kaidi Xu, Renjing Xu
Spiking Neural Network Lottery Ticket Hypothesis Sparse Structure Convolutional Spiking Neural Network DNN Pruning

November 17, 2023

Archtree: on-the-fly tree-structured exploration for latency-aware pruning of deep neural networks
Rémi Ouazan Reboul, Edouard Yvinec, Arnaud Dapogny, Kevin Bailly
Deep Neural Network Structured Pruning DNN Architecture Pruning Framework Rapidly Exploring Random Tree DNN Pruning

September 13, 2023

DNNShifter: An Efficient DNN Pruning System for Edge Computing
Bailey J. Eccles, Philip Rodgers, Peter Kilpatrick, Ivor Spence, Blesson Varghese
Edge Computing DNN Model DNN Training Deep Neural Network Parameter Pruned Model DNN Pruning

August 9, 2023

SAfER: Layer-Level Sensitivity Assessment for Efficient and Robust Neural Network Inference
Edouard Yvinec, Arnaud Dapogny, Kevin Bailly, Xavier Fischer
Deep Neural Network High Efficiency Neural Network Inference DNN Pruning SAfER Instruct

May 18, 2023

PDP: Parameter-free Differentiable Pruning is All You Need
Minsik Cho, Saurabh Adya, Devang Naik
Structured Pruning Channel Pruning DNN Pruning Differentiable Pruning

September 18, 2022

Pruning Neural Networks via Coresets and Convex Geometry: Towards No Assumptions
Murad Tukan, Loay Mualem, Alaa Maalouf
Neural Network Deep Neural Network Supervised ImageNet Convex Set Modeling Assumption Neural Network Pruning Coreset Size DNN Pruning

August 7, 2022

N2NSkip: Learning Highly Sparse Networks using Neuron-to-Neuron Skip Connections
Arvind Subramaniam, Avinash Sharma
Pruning Method Sparse Network Sparse Connectivity Iterative Pruning Skip Connection DNN Pruning NN2Poly Package

July 8, 2022

SInGE: Sparsity via Integrated Gradients Estimation of Neuron Relevance
Edouard Yvinec, Arnaud Dapogny, Matthieu Cord, Kevin Bailly
Deep Neural Network Sparsity Increase Integrated Gradient DNN Pruning Attribution Robustness Gradient Pruning

July 4, 2022

Exploring Lottery Ticket Hypothesis in Spiking Neural Networks
Youngeun Kim, Yuhang Li, Hyoungseob Park, Yeshwanth Venkatesha, Ruokai Yin, Priyadarshini Panda
Spiking Neural Network Deeper Network Lottery Ticket Hypothesis DNN Pruning Energy Efficient Deep Learning

May 20, 2022

Energy-efficient Deployment of Deep Learning Applications on Cortex-M based Microcontrollers using Deep Compression
Mark Deutel, Philipp Woller, Christopher Mutschler, Jürgen Teich
Neural Network Deep Learning Model Deep Learning Application DNN Architecture 32 Bit Microcontrollers DNN Pruning Deep Compression Efficient Deployment

December 14, 2021

Pruning Coherent Integrated Photonic Neural Networks Using the Lottery Ticket Hypothesis
Sanmitra Banerjee, Mahdi Nikdast, Sudeep Pasricha, Krishnendu Chakrabarty
Gene Level GNN Lottery Ticket Hypothesis Photonic Neural Network DNN Pruning

November 22, 2021

Automatic Mapping of the Best-Suited DNN Pruning Schemes for Real-Time Mobile Acceleration
Yifan Gong, Geng Yuan, Zheng Zhan, Wei Niu, Zhengang Li, Pu Zhao, Yuxuan Cai, Sijia Liu, Bin Ren, Xue Lin, Xulong Tang, Yanzhi Wang
Fine Grained Weight Pruning DNN Pruning

DNN Pruning

Papers

HESSO: Towards Automatic Efficient and User Friendly Any Neural Network Training and Pruning

Combining Relevance and Magnitude for Resource-Aware DNN Pruning

Pursing the Sparse Limitation of Spiking Deep Learning Structures

Archtree: on-the-fly tree-structured exploration for latency-aware pruning of deep neural networks

DNNShifter: An Efficient DNN Pruning System for Edge Computing

SAfER: Layer-Level Sensitivity Assessment for Efficient and Robust Neural Network Inference

PDP: Parameter-free Differentiable Pruning is All You Need

Pruning Neural Networks via Coresets and Convex Geometry: Towards No Assumptions

N2NSkip: Learning Highly Sparse Networks using Neuron-to-Neuron Skip Connections

SInGE: Sparsity via Integrated Gradients Estimation of Neuron Relevance

Exploring Lottery Ticket Hypothesis in Spiking Neural Networks

Energy-efficient Deployment of Deep Learning Applications on Cortex-M based Microcontrollers using Deep Compression

Pruning Coherent Integrated Photonic Neural Networks Using the Lottery Ticket Hypothesis

Automatic Mapping of the Best-Suited DNN Pruning Schemes for Real-Time Mobile Acceleration