cs.PFNov 12, 2026

Theoretical and Empirical Analysis of Sparse Convolution

Authors: Keshvi Tuteja, Yu-Hsiang Tsai, Gregor Olenik, Markus Götz, Achim Streit, Hartwig Anzt, Charlotte Debus

Organizations: Scientific Computing Center, Karlsruhe Institute of Technology, Germany · Technical University of Munich, Germany · Helmholtz AI and Scientific Computing Center, Karlsruhe Institute of Technology, Germany · Chair of Computational Mathematics, Technical University of Munich, University of Tennessee, Germany, USA

Abstract

Deep neural networks for image processing are increasingly deployed on resource-constrained edge devices such as drones, medical systems, and smartphones, where limited computational resources and memory bandwidth pose significant challenges for efficient inference. Pruning during training often produces highly sparse convolutional kernels, offering the potential to significantly reduce computation and memory requirements. However, exploiting this sparsity efficiently at runtime remains challenging due to irregular memory access patterns and workload imbalance across processing elements. In this work, we investigate the standard convolution with sparse weights and analyze its performance characteristics through roofline analysis, showing that a 3 x 3 sparse convolution achieves an arithmetic intensity of approximately 10 FLOP/byte compared to roughly 96 FLOP/byte for its dense counterpart. We propose techniques for an efficient sparse convolution at high sparsities. Preliminary results demonstrate that our approach achieves upto 1.65x speedup as compared to dense convolution on NVIDIA A100 and upto 1.52x on NVIDIA Jetson Nano (Tegra X1) , motivating further exploration of direct convolution as a practical path to efficient sparse inference on modern accelerators/edge devices.

Figures & tables

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. SparsePixels: Efficient Convolution for Sparse Data on FPGAs

    Dec 5, 2025Ho Fung Tsoi, Dylan Rankin, Vladimir Loncar +1Efficient Neural Network InferenceConvolutional Neural Networks

  2. Hardware Efficient Approximate Convolution with Tunable Error Tolerance for CNNs

    Mar 10, 2026Vishal Shashidhar, Anupam Kumari, Roy P PailyEfficient Neural Network InferenceNeural Network Hardware Acceleration

  3. At-the-Roofline Sparse Tensor Contractions on Vector Processors for Transformer Inference

    Jul 28, 2026Bowen Wang, Chi Zhang, Diyou Shen +3Efficient Neural Network InferenceTransformer Inference