Sparse Neural Networks

Ask about this topic

Overview

Sparse neural networks study how to train, prune, or route models so that only selected weights, connections, features, or computations are active. Approaches include magnitude- and saliency-based pruning, dynamic sparse training that regrows connections, structured sparsity and feature-selection penalties, and sparsity-inducing Bayesian priors. The goal is to reduce memory and training or inference costs while retaining accuracy, across settings such as edge devices, federated learning, and large language models. Fewer parameters do not automatically mean faster execution or more interpretable models: hardware-aware layouts and kernels matter, while accuracy, robustness, and efficiency at extreme sparsity remain in tension.

Latest papers

CardsList

66 papers10 papers in the last four weeks, up 233% on the four weeks before. 0.1% of all new papers.

  1. Bayesian Lottery Ticket Hypothesis

    Feb 21, 2026Nicholas Kuhn, Arvid Weyrauch, Lars Heyen +3Lottery Ticket HypothesisSparse Neural Networks

    Scientific Computing Center (SCC), Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany · Helmholtz AI, Eggenstein-Leopoldshafen, Germany

  2. Hebbian-Oscillatory Co-Learning

    Feb 21, 2026Hasi HaysSparse Neural NetworksOscillatory Neural Networks

    Department of Chemical Engineering, University of Arkansas, Fayetteville, AR 72701, USA

  3. Fine-Pruning: A Biologically Inspired Algorithm for Personalization of Machine Learning Models

    Feb 18, 2026Joseph Bingham, Saman Zonouz, Dvir AranNeural Network PruningSparse Neural Networks

    Faculty of Biology, Technion - Israel Institute of Technology, Haifa, IL · College of Computing, The Georgia Institute of Technology, Atlanta, GA, USA · Taub Faculty of Computer Science, Technion - Israel Institute of Technology, Haifa, IL

  4. Hyperflux: Pruning Reveals Importance

    Apr 6, 2025Eugen Barbulescu, Antonio Alexoaie, Lucian BusoniuSparse Neural NetworksNeural Network Training Dynamics

    Department of Computer Science Technical University of Cluj-Napoca · Department of Automation Technical University of Cluj-Napoca

  5. H-FedSN: Personalized Sparse Networks for Efficient and Accurate Hierarchical Federated Learning for IoT Applications

    Dec 9, 2024Jiechao Gao, Yuangang Li, Jie Wang +3Hierarchical Federated LearningInternet of Things

    University of Virginia Charlottesville, VA, USA

  6. Evolutionary Deep Nets for Non-Intrusive Load Monitoring

    Mar 6, 2023Jinsong Wang, Kenneth A. LoparoNon-Intrusive Load MonitoringSparse Neural Networks