GRAU: A Generic Runtime-Reconfigurable Activation-Requantization Unit for Quantized Neural Network Accelerators

Yuhao Liu; Salim Ullah; Akash Kumar

cs.ARFeb 25, 2026

GRAU: A Generic Runtime-Reconfigurable Activation-Requantization Unit for Quantized Neural Network Accelerators

Authors: Yuhao Liu, Salim Ullah, Akash Kumar

Organizations: Ruhr University Bochum, Germany · Dresden University of Technology, Germany · Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI Dresden/Leipzig), Germany

Abstract

As neural networks continue to scale, low-precision quantization is widely adopted in edge accelerators, but conventional multi-threshold activation hardware requires 2^n-1 thresholds for n-bit integer outputs, leading to rapidly increasing hardware costs. We propose GRAU, a Generic Runtime-Reconfigurable Activation-Requantization Unit that fuses batch normalization, nonlinear activation, and requantization through piecewise-linear fitting with Power-of-Two slopes. GRAU requires only basic comparators and 1-bit right shifters, while supporting mixed-precision quantization and runtime switching among post-accumulation functions, including modern nonlinear activations such as SiLU and GELU. Compared with multi-threshold activation-requantization units, GRAU reduces LUT consumption by approximately 90% or more while improving hardware efficiency, flexibility, and scalability. We further explore the trade-off between accuracy and hardware cost across different approximation settings and provide an open-source toolchain for GRAU-aware network training.

Figures & tables

Fig. 1 : Non-Integer Computation in Quantized Neural Network

Panel of Figure 2 — Fig. 2 : Correct 2-bit quantization of Multi-Threshold unit (left) in Sigmoid and the mistake of Multi-Threshold unit in non-monotonically increasing function (right)

Model	Precision	Accu.	$\Delta$ Accu.	Memory	Baseline
Model	Precision	(%)	(pp)	(Bytes)	Ratio
MLP	Full 1-bit	92.29	-3.62	7376	0.74
	Mixed (baseline)	95.91	0.00	9984	1.00
	Full 8-bit	97.36	1.45	59008	5.91
CNN	Full 1-bit	96.26	-2.53	29848	0.54
	Mixed (baseline)	98.79	0.00	55712	1.00

TABLE I : Comparison between Unified-Precision and Mixed-Precision QNN on MNIST [ bitsys2 ]

Method	SOPF	PWLF	MT	LUT	GRAU
Hardware	Low	Low	High	High	High
Friendly	Low	Low	High	High	High
Runtime	No	No	Yes	Limited	Yes
Reconfigurable	No	No	Yes	Limited	Yes
Non-monotonically	Yes	Yes	No	Yes	Yes
Increasing Function	Yes	Yes	No	Yes	Yes

TABLE II : Comparison between SOPF, PWLF, MT, LUT, and GRAU for Quantized Activation Unit

Fig. 3 : Comparing the original nonlinear function, PWLF approximated function, PoT approximated PWLF function, and APoT approximated PWLF function

	Experiment Result
Precision	4-bit			8-bit			Mixed-Precision
Activation	ReLU	Sigmoid	SiLU	ReLU	Sigmoid	SiLU	ReLU	Sigmoid	SiLU
Original	91.18%	87.81%	89.55%	92.10%	88.71%	92.35%	92.11%	88.39%	91.10%
Max. Segment	PWLF
6	91.00%	86.72%	84.88%	92.08%	76.79%	91.92%	91.78%	80.75%	88.20%
8	91.08%	85.99%	85.58%	92.10%	85.87%	92.32%	91.86%	74.87%	88.89%

TABLE III : Design Space Exploration for PoT-PWLF/APoT-PWLF Approximation based on Original QNN and PWLF Models with CIFAR-10 [ Krizhevsky2009 ] and VGG16 [ Simonyan2014 ] Architecture

Experiment Result
Precision	8-bit						Mixed-Precision
Activation	ReLU		ReLU+SiLU		ReLU+GELU		ReLU		ReLU+SiLU		ReLU+GELU
	Top-1	Top-5	Top-1	Top-5	Top-1	Top-5	Top-1	Top-5	Top-1	Top-5	Top-1	Top-5
Original	67.07%	87.23%	67.44%	87.23%	66.79%	87.02%	65.41%	86.29%	65.26%	86.10%	65.18%	86.01%
Max. Segment	PWLF
6	66.71%	86.96%	66.50%	86.66%	65.88%	86.64%	62.63%	84.13%	56.78%	79.82%	51.52%	76.48%

TABLE IV : Design Space Exploration for APoT-PWLF Approximation based on Original QNN and PWLF Models with ImageNet [ imagenet-dataset ] and ResNet-18 [ resnet ] Architecture

Fig. 4 : Encoding of segment slopes for PoT-PWLF (down) and APoT-PWLF (up) approximation

Fig. 6 : Serialized hardware implementation of Generic Activation Unit PoT-PWLF and APoT-PWLF shifter unit

Activation Unit	Design	Segment	Exponent	LUT	FF	Frequency	Total	Dynamic	Power-Delay	Area-Delay	Pipeline Depth
Activation Unit	Design	Number	Number	LUT	FF	Frequency	Delay (ns)	Power (W)	Products	Products	1-bit	2-bit	4-bit	8-bit
Multi-Threshold	Pipelined	-	-	10206	18568	200MHz	2.848	0.129	0.3674	29066.688	1	3	15	255
Multi-Threshold	Serialization	-	-	2796	8264	100MHz	5.777	0.032	0.1849	16152.492	-	-	-	-
PoT-PWLF	Pipelined	6	8	408	675	250MHz	1.658	0.013	0.0216	676.464	1	3	16	16
		6	16	647	1007		1.570	0.015	0.0236	1015.790	1	3	24	24
		8	8	507	854		1.811	0.013	0.0235	918.177	1	3	18	18

TABLE V : Hardware Results of Multi-Threshold, PoT-PWLF, and APoT-PWLF Activation Units

Neural Network Hardware Acceleration

See all 38 papers

Compiling Multi-Threshold Operators for Quantized Non-Monotonic and Fine-Granular Activations in Custom-Tailored DNN Accelerators
Oct 15, 2026Christoph Berganski, Felix Jentzsch, Marco PlatznerActivation Quantization Neural Network Activation Functions
Computer Engineering Group, Paderborn University, Paderborn, Germany
BiKA: Kolmogorov-Arnold-Network-inspired Ultra Lightweight Neural Network Hardware Accelerator
Feb 26, 2026Yuhao Liu, Salim Ullah, Akash KumarNeural Network Hardware Acceleration Efficient Neural Network Inference
Ruhr University Bochum, Germany · Dresden University of Technology, Germany · Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI Dresden/Leipzig), Germany
Bitwise Systolic Array Architecture for Runtime-Reconfigurable Multi-precision Quantized Multiplication on Hardware Accelerators
Feb 26, 2026Yuhao Liu, Salim Ullah, Akash KumarMixed-Precision Quantization FPGA-Based Neural Network Acceleration
Dresden University of Technology, Germany · Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI Dresden/Leipzig), Germany · Ruhr University Bochum, Germany

Oct 15, 2026J/K move · Enter open · S save

Compiling Multi-Threshold Operators for Quantized Non-Monotonic and Fine-Granular Activations in Custom-Tailored DNN Accelerators

Christoph Berganski, Felix Jentzsch, Marco Platzner

Computer Engineering Group, Paderborn University, Paderborn, Germany

Activation Quantization Neural Network Activation Functions Neural Network Quantization FPGA-Based Neural Network Acceleration

Open paper AskPDF arXiv

Aggressive quantization of neural networks can expose activation functions as bottlenecks for efficient hardware acceleration. Implementing quantized activations as multi-threshold operations has been shown to be a promising optimization, allowing for efficient hardware implementation of few-bit activations. However, current implementations are limited to monotonically increasing functions, quantization schemes with one-dimensional (per-channel) granularity and a small set of predefined activation-quantizer combinations. We formalize the multi-threshold representation to extend it to non-monotonic functions and fine-granular parameters, i.e., multi-dimensional thresholds. For automated conversion of quantized function compositions into threshold representation, we propose a systematic approach via algebraic rules integrated into the ONNX-based compiler infrastructure. We experimentally validate our solution on a selection of monotonic and non-monotonic functions by extending the binary search based multi-threshold operator that is integrated into the FINN framework for end-to-end synthesis of custom-tailored FPGA accelerators. The cost of hardware support for non-monotonicity and fine-granularity is characterized in comparison to an alternative approach utilizing existing operators via graph-level decomposition. We compare our method to prior work as well as the approach taken by the hls4ml framework. As an outlook, we demonstrate the prospects of using the multi-threshold representation during the training of a neural network as a form of quantization aware training.

Abstract

Neural Network Hardware Acceleration

Explore similar work

Compiling Multi-Threshold Operators for Quantized Non-Monotonic and Fine-Granular Activations in Custom-Tailored DNN Accelerators

BiKA: Kolmogorov-Arnold-Network-inspired Ultra Lightweight Neural Network Hardware Accelerator

Bitwise Systolic Array Architecture for Runtime-Reconfigurable Multi-precision Quantized Multiplication on Hardware Accelerators