Data-Rate-Aware High-Speed CNN Inference on FPGAs

Tobias Habermann; Martin Kumm

cs.ARFeb 18, 2026

Data-Rate-Aware High-Speed CNN Inference on FPGAs

Authors: Tobias Habermann, Martin Kumm

Organizations: Department of Applied Computer Science, Fulda University of Applied Sciences, Fulda, 36039, Germany

Abstract

Dataflow-based CNN accelerators on FPGAs achieve low latency and high throughput by mapping computations of each layer directly to corresponding hardware units. However, layers such as pooling and strided convolutions reduce the data at their output with respect to their input, strongly effecting the data rate of the following layers. This leads to underutilization in fully unrolled designs. While prior work introduced data-rate-aware layer-wise adaptation, determining the most efficient implementation remains challenging. This paper presents a data-rate-aware CNN accelerator architecture for multi-pixel processing. Building on existing analytical models, the proposed method performs design-space exploration to identify configurations that improve hardware utilization and resource efficiency while preserving continuous flow of data, keeping all hardware units busy. Experimental results show substantial reductions in arithmetic resources compared to previous designs, enabling efficient implementation of complex CNNs on a single FPGA across a wide range of data rates.

Figures & tables

Fig. 1: The KPU base component presented in [ 11 ] .

Fig. 2: The FCU base component presented in [ 11 ] .

Panel of Figure 3 — Fig. 3: The structure of convolutional and fully connected layers.

Fig. 4: An example of a convolutional layer implementation that can process two pixels per clock cycle.

Fig. 5: A non-transposed KPU that can process two pixels per clock cycle.

name	LUT	FF	BRAM	URAM	DSP
[ 11 ]	204,931	563,255	1,702.5	0	5,691
Ours	158,540	603,372	1,449.5	10	5,664

TABLE I: The resources used to implement the MobileNetV1 model with the same data rate.

	$\text{F}_{\text{max}}$	FPS	Latency	LUT	BRAM	URAM	DSP	Power	P. Eff.	Precision	FPGA
	MHz	Inf/s	ms					W	mJ/Inf.
RTX 3080 GPU**	1860	4105.4	14.93	-	-	-	-	328	79.89	32-bit
LUTMUL [ 14 ]	333	1627	-	529k	1119	-	106	42.12	25.89	4-bit	Alveo U280
[ 12 ]	241	4803.1	0.62	136k	1225.5	-	2709	18.04	3.75	8-bit	XCVU37P
FINN [ 15 ] *	333	925	-	501k	898	-	106	41.69	45.07	4-bit	Alveo U280
Ours (6/1)	403.71	16,020.40	0.21	186k	1,410	12	6,302	92.34	5.76	8-bit	XCVU37P

TABLE II: The same MobileNetV2 CNN model implemented for different data rates compared to the SOTA.

Panel of Figure 8 — Fig. 6: The pixel order of the input image when processing two pixels per clock cycle for a $5\times 5$ input image where the first (a) and second (b) sliding windows indicate how to delay and connect the KPUs to the input signals.

Efficient Neural Network Inference

Efficient neural network inference reduces the time, memory, and energy needed to run trained models across hardware and deployment settings.

See all 468 papers

SparsePixels: Efficient Convolution for Sparse Data on FPGAs
Dec 5, 2025Ho Fung Tsoi, Dylan Rankin, Vladimir Loncar +1Efficient Neural Network Inference Convolutional Neural Networks
University of Pennsylvania, USA · Institute of Physics Belgrade, Serbia · Massachusetts Institute of Technology, USA +1
Energy-Efficient CNN Acceleration with MSDF Digit-Serial Arithmetic on FPGA
Jun 24, 2026Muhammad Usman, Yousef Sadegheih, Dorit MerhofEnergy-Efficient ML FPGA-Based Neural Network Acceleration
Faculty of Informatics and Data Science, University of Regensburg, 93053 Regensburg, Germany
CascadeLUT: Information-Ordered Streaming Inference for Bandwidth-Constrained FPGAs
Aug 1, 2026Oliver Cassidy, Marta Andronic, George A. ConstantinidesEfficient Neural Network Inference Streaming Inference
Department of Electrical and Electronic Engineering, Imperial College London, UK

Dec 5, 2025J/K move · Enter open · S save

SparsePixels: Efficient Convolution for Sparse Data on FPGAs

Ho Fung Tsoi, Dylan Rankin, Vladimir Loncar, Philip Harris

University of Pennsylvania, USA · Institute of Physics Belgrade, Serbia · Massachusetts Institute of Technology, USA+1

Efficient Neural Network Inference Convolutional Neural Networks FPGA-Based Neural Network Acceleration

Open paper AskPDF arXiv

Inference of standard convolutional neural networks (CNNs) on FPGAs often incurs high latency and a long initiation interval due to the deep nested loops required to densely convolve every input pixel regardless of its feature value. However, input features can be spatially sparse in some image data, where semantic information may occupy only a small fraction of the pixels and most computation would be wasted on empty regions. In this work, we introduce SparsePixels, a framework that implements sparse convolution on FPGAs by selectively retaining and computing on a small subset of active input pixels while ignoring the rest, reducing the per-layer compute from

O(HWK^2C_{\text{in}}C_{\text{out}})

O(n^2C_{\text{in}}C_{\text{out}})

and the storage from

O(HWC)

O(nC)

when

n^2\ll HW

. Because computation always runs over a single pre-specified pixel budget that is frozen at synthesis, the inference latency is data-independent and constant at runtime. We show that, for identifying neutrino interactions in naturally sparse LArTPC images with 4k pixels, a standard CNN with a compact size of 4k parameters incurs an inference latency of 48.665

μ

s on an FPGA, whereas a sparse CNN of the same base architecture, computing on less than 1% of the input pixels, achieves a

\times 73

speedup to 0.665

μ

s with resource utilization well within on-chip budgets, trading only a small percent-level performance loss. This work aims to benefit future algorithm development for efficient data readout in modern experiments with strict latency requirements of microseconds or below.

Abstract

Efficient Neural Network Inference

Explore similar work

SparsePixels: Efficient Convolution for Sparse Data on FPGAs

Energy-Efficient CNN Acceleration with MSDF Digit-Serial Arithmetic on FPGA

CascadeLUT: Information-Ordered Streaming Inference for Bandwidth-Constrained FPGAs