Sparse Neural Networks

Follow this topic Ask about this topic

Momentum

2 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 32

All topics

Hyperflux: Pruning Reveals Importance
Apr 6, 2025Eugen Barbulescu, Antonio Alexoaie, Lucian BusoniuSparse Neural Networks Neural Network Training Dynamics
Department of Computer Science Technical University of Cluj-Napoca · Department of Automation Technical University of Cluj-Napoca
H-FedSN: Personalized Sparse Networks for Efficient and Accurate Hierarchical Federated Learning for IoT Applications
Dec 9, 2024Jiechao Gao, Yuangang Li, Jie Wang +3Hierarchical Federated Learning Internet of Things
University of Virginia Charlottesville, VA, USA

Apr 6, 2025, stat.MLJ/K move · Enter open · S save

Hyperflux: Pruning Reveals Importance

Eugen Barbulescu, Antonio Alexoaie, Lucian Busoniu

Department of Computer Science Technical University of Cluj-Napoca · Department of Automation Technical University of Cluj-Napoca

Sparse Neural Networks Neural Network Training Dynamics Neural Network Pruning

Open paper AskPDF arXiv

Network pruning is used to reduce inference latency and power consumption in large neural networks. However, most methods focus on empirical results at the expense of understanding the pruning process. We introduce Hyperflux, a novel

L_0

method which models pruning as a continuously evolving system determined by flux, the gradient response to a weight's removal, and pressure, a global regularization driving weights toward pruning. By exploiting this model, Hyperflux's pruning behavior becomes understandable at both microscopic (weight regrowth/pruning) and macroscopic (sparsity convergence, etc.) levels. We also introduce a novel pressure scheduler that reliably targets desired sparsities. Hyperflux achieves competitive results with ResNet-50, VGG-19 and DeiT-T/S on CIFAR-10, CIFAR-100 and ImageNet datasets.

Momentum

Related topics

Latest papers 32

Hyperflux: Pruning Reveals Importance

H-FedSN: Personalized Sparse Networks for Efficient and Accurate Hierarchical Federated Learning for IoT Applications