Quantization Effects of Artificial Neural Networks for Embedded Edge-Computing Applications

Alperen Aksoy; Ilja Bekman; Vesselin Dimitrov; Qader Dorosti; Chimezie Eguzo; Sarah Fleitmann; Fabian Hader; Andre Zambanini

cs.NENov 7, 2025

Quantization Effects of Artificial Neural Networks for Embedded Edge-Computing Applications

Authors: Alperen Aksoy, Ilja Bekman, Vesselin Dimitrov, Qader Dorosti, Chimezie Eguzo, Sarah Fleitmann, Fabian Hader, Andre Zambanini,

Organizations: Peter Grünberg Institute (PGI), Integrated Computing Architectures (ICA | PGI-4), Forschungszentrum Jülich GmbH, Germany · Center for Particle Physics Siegen, Department für Physik, Universität Siegen, Germany · Faculty of Engineering, JARA-FIT Institute for Quantum Information, Forschungszentrum Jülich GmbH and RWTH Aachen University, Germany · University Duisburg-Essen, Germany

Abstract

This paper examines the use of Quantized Neural Networks (QNNs) for two resource-constrained scientific applications: automated calibration of semiconductor quantum bits (qubits) and scientific particle detectors. We evaluate the trade-offs between Post-Training Quantization (PTQ), Quantization-Aware Training (QAT), and ultra-low-bit Binary Neural Networks (BNNs) with respect to latency and resource usage. Our results demonstrate that PTQ and QAT are easily implementable solutions achieving a four-fold reduction in memory usage for Ushaped CNN (U-Net) architectures, whereas BNNs are better suitable for use cases with challenging latency requirements. For the training of non-differentiable custom BNNs, we propose a novel, hardware-constrained learning approach using Genetic Algorithms (GAs). We showcase a Look-Up Table (LUT)-based BNN architecture suitable for direct conversion to Very High-Speed Integrated Circuit Hardware Description Language (VHDL) via the HCL4BNN framework. This method achieves nanosecond-scale inference latencies at 10 ns to 15 ns without requiring specialized Digital Signal Processor (DSP) or Block RAM (BRAM) resources.

Figures & tables

Panel of Figure 1 — Figure 1: Conventional workflow to deploy a QNN on an FPGA based on gradient training, quantization, and HLS conversion.

Panel of Figure 2 — Figure 2: Proposed hardware-constrained learning approach with direct HDL generation.

Figure 3: Left: isolated simulated SiPM pulses (”Good” class). Right: two overlapping or distorted simulated SiPM pulses (”Ugly” class).

Figure 4: Left: Best training accuracy progression across generations for several GA runs, see Table 3 for validation results. Right: Progression of best accuracy and network sparsity (measured by the number of zero-weights) across generations for a longer GA run.

Panel of Figure 5 — Figure 5: Simulated charge stability diagram with and without compensation of the sensor response to the voltage changes for gates $P_{1}$ and $P_{2}$ [ 8 ] . Further data examples can be found in [ 37 ] .

Model	Unquantized	PTQ	QAT
	in %	in %	in %
UNet-447	$85\pm 28$	$83\pm 29$	$85\pm 28$
UNet-38k	$96\pm 9$	$96\pm 9$	$96\pm 9$

Table 1: Mean dice similarity on the test set with standard deviation on the test set.

# Parameters	9,112	4,268	3,147	2,062	1,013
Accuracy in %	$88.7\pm 3.7$	$89.1\pm 1.2$	$87.5\pm 1.7$	$85.3\pm 1$	$78.3\pm 1.2$
Non-convergent	18 %	29 %	20 %	15 %	18 %

Table 2: Results of compressed CNN on MNIST dataset with small amount of parameters, binary weights and activations. Accuracy mean with standard deviation over normal fraction of runs is given in percent for converged runs. A fraction of training runs do not reach accuracy of 70 % after 10 epochs and are defined as stuck/non-convergent.

Figure 6: Illustration of the accuracy distributions of 5 compressed CNN (next to each other per accuracy bin) for training reruns on MNIST dataset after 10 epochs. Largest CNN (orange color, circle) distribution has the mean at

89\%

, Smallest CNN (purple color, down-triangle) peaks at

78\%

. Means and standard deviation points are calculated over the normal part of the distributions.

Model	Accuracy	Latency	LUT	FF	DSP	BRAM	Training Time
	in %	in ns	$\times 10^{3}$	$\times 10^{3}$		18 k	min $\times$ cores
FINN	$74\pm 4$	24850	30	20	106	5	$\sim 2\times 16$
hls4ml	$93\pm 2$	3050	186	112	556	120	$\sim 2\times 16$
BNN a	$63\pm 3$	15	58	1.5	0	0	$\sim 300\times 90$
BNN b	$74\pm 5$	10	23	1.5	0	0	$\sim 105\times 90$
BNN b (static)	$72\pm 3$

Table 3: Results and comparison of FINN 2DCNN (

(128\text{-}4\text{-}6\text{-}8\text{-}2)

, Kernel [5,1], padding 2.0, int8) implementation, hls4ml CNN, and BNN a

(128\text{-}64\text{-}128\text{-}2)

, BNN b

(128\text{-}32\text{-}32\text{-}2)

, BNN c

(128\text{-}16\text{-}64\text{-}2)

with INT7 input quantization. BNN b is also run with a static training dataset.

		Predicted
		Good	Ugly
True	Good	947	53
True	Ugly	410	590

Table 4: Examples of confusion matrices for FINN (left) and hls4ml (right) for small validation sample. For the accuracies averaged over 30 runs see 3 .

		Predicted
		Good	Ugly	Either
True	Good	3052	1948	0
True	Ugly	1595	3400	5
	Noise	0	0	5000

Table 5: Example confusion matrices for one BNN a

128\text{-}64\text{-}128\text{-}2

(left) and one BNN b

128\text{-}32\text{-}32\text{-}2

(right); Either (or undecided) encodes

(1,1)

(0,0)

tuples for prediction and random input value test for truth.

Figure 7: Illustration of the difference for static (left) and dynamic (right) SiPM training datasets of large (21k 4bit parameters) compressed CNN . The training dataset here is 32+32 waveforms and 640+640 for validation. Static training data leads to exaggerated over-fitting in this illustration.

Parameter	Value	Description
pop_size	300	Number of individuals per generation
ngen	50	Number of generations the GA is evolved for = stopping criterion
nmutbit	50	Expected number of mutated bits per genome (sets per-bit mutation probability, see Sec. 3.2.2 )
tourn_size	5	Tournament size used for parent selection
cxpb	0.5	Crossover probability per individual
cxpb_bit	1	Crossover probability per bit, given that crossover occurs

Table 6: Genetic algorithm hyperparameters and dataset generation settings used for BNN training.

Neural Network Quantization

Neural network quantization represents weights, activations, and states with fewer bits to reduce memory use, computation, and deployment costs.

See all 403 papers

Latency-Constrained DNN Architecture Learning for Edge Systems using Zerorized Batch Normalization
Jul 8, 2026Shuo Huai, Di Liu, Hao Kong +4Efficient Neural Network Inference Edge Inference
aSchool of Computer Science and Engineering, Nanyang Technological University, 639798, Singapore · cDepartment of Computer Science, Norwegian University of Science and Technology, 7491, Trondheim, Norway · bHP-NTU Digital Manufacturing Corporate Lab, Nanyang Technological University, 639798, Singapore +1
HGQ-LUT: Fast LUT-Aware Training and Efficient Architectures for DNN Inference
Apr 24, 2026Chang Sun, Zhiqiang Que, Bakhtiar Zadeh +4Efficient Neural Network Inference FPGA-Based Neural Network Acceleration
California Institute of Technology, Pasadena, CA, USA · University of Bristol, UK · Imperial College London, UK +1
BiKA: Kolmogorov-Arnold-Network-inspired Ultra Lightweight Neural Network Hardware Accelerator
Feb 26, 2026Yuhao Liu, Salim Ullah, Akash KumarNeural Network Hardware Acceleration Efficient Neural Network Inference
Ruhr University Bochum, Germany · Dresden University of Technology, Germany · Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI Dresden/Leipzig), Germany

Jul 8, 2026J/K move · Enter open · S save

Latency-Constrained DNN Architecture Learning for Edge Systems using Zerorized Batch Normalization

Shuo Huai, Di Liu, Hao Kong, Weichen Liu+3

aSchool of Computer Science and Engineering, Nanyang Technological University, 639798, Singapore · cDepartment of Computer Science, Norwegian University of Science and Technology, 7491, Trondheim, Norway · bHP-NTU Digital Manufacturing Corporate Lab, Nanyang Technological University, 639798, Singapore+1

Efficient Neural Network Inference Edge Inference Hardware-Aware NAS

Open paper AskPDF arXiv

Deep learning applications have been widely adopted on edge devices, to mitigate the privacy and latency issues of accessing cloud servers. Deciding the number of neurons during the design of a deep neural network to maximize performance is not intuitive. Particularly, many application scenarios are real-time and have a strict latency constraint, while conventional neural network optimization methods do not directly change the temporal cost of model inference for latency-critical edge systems. In this work, we propose a latency-oriented neural network learning method to optimize models for high accuracy while fulfilling the latency constraint. For efficiency, we also introduce a universal hardware-customized latency predictor to optimize this procedure to learn a model that satisfies the latency constraint by only a one-shot training process. The experiment results reveal that, compared to state-of-the-art methods, our approach can well-fit the 'hard' latency constraint and achieve high accuracy. Under the same training settings as the original model and satisfying a 34 ms latency constraint on the ImageNet-100 dataset, we reduce GoogLeNet's latency from 40.32 ms to 34 ms with a 0.14% accuracy reduction on the NVIDIA Jetson Nano. When coupled with quantization, our method can be further improved to only 0.04% drop for GoogLeNet. On the NVIDIA Jetson TX2, we compress VGG-19 from 119.98 ms to 34 ms and even improve its accuracy by 0.5%, and we scale GoogLeNet up from 20.27 ms to 34 ms and achieve higher accuracy by 0.78%. We also open source this framework at https://github.com/ntuliuteam/ZeroBN

Abstract

Neural Network Quantization

Explore similar work

Latency-Constrained DNN Architecture Learning for Edge Systems using Zerorized Batch Normalization

HGQ-LUT: Fast LUT-Aware Training and Efficient Architectures for DNN Inference

BiKA: Kolmogorov-Arnold-Network-inspired Ultra Lightweight Neural Network Hardware Accelerator