Compiling Multi-Threshold Operators for Quantized Non-Monotonic and Fine-Granular Activations in Custom-Tailored DNN Accelerators

Christoph Berganski; Felix Jentzsch; Marco Platzner

cs.AROct 15, 2026

Compiling Multi-Threshold Operators for Quantized Non-Monotonic and Fine-Granular Activations in Custom-Tailored DNN Accelerators

Authors: Christoph Berganski, Felix Jentzsch, Marco Platzner

Organizations: Computer Engineering Group, Paderborn University, Paderborn, Germany

Abstract

Aggressive quantization of neural networks can expose activation functions as bottlenecks for efficient hardware acceleration. Implementing quantized activations as multi-threshold operations has been shown to be a promising optimization, allowing for efficient hardware implementation of few-bit activations. However, current implementations are limited to monotonically increasing functions, quantization schemes with one-dimensional (per-channel) granularity and a small set of predefined activation-quantizer combinations. We formalize the multi-threshold representation to extend it to non-monotonic functions and fine-granular parameters, i.e., multi-dimensional thresholds. For automated conversion of quantized function compositions into threshold representation, we propose a systematic approach via algebraic rules integrated into the ONNX-based compiler infrastructure. We experimentally validate our solution on a selection of monotonic and non-monotonic functions by extending the binary search based multi-threshold operator that is integrated into the FINN framework for end-to-end synthesis of custom-tailored FPGA accelerators. The cost of hardware support for non-monotonicity and fine-granularity is characterized in comparison to an alternative approach utilizing existing operators via graph-level decomposition. We compare our method to prior work as well as the approach taken by the hls4ml framework. As an outlook, we demonstrate the prospects of using the multi-threshold representation during the training of a neural network as a form of quantization aware training.

Figures & tables

Fig. 1 : Hardware implementation of binary search based thresholding adapted from [ 15 ] . A remapping stage is added to allow for non-monotonic output levels

L_{i}

according to Eq. ( III ). To locate the input within the list of seven sorted thresholds, three comparators are required, corresponding to the three levels of the search tree shown on the left.

Fig. 3 : Address logic and layout of the threshold parameter memory. The address logic and the thresholding units are replicated as

\mathtt{PE}

parallel processing elements resulting in a two-level loop nest

(i,j)

implemented in Vitis HLS.

Fig. 5 : Tree of branching threshold segments: Interpreting this as a parity function, this is equivalent to applying alternating weights

\pm 1

before summing over the sorted segment contributions. Sorting is always allowed due to commutativity.

Panel of Figure 4 — Fig. 6 : Inverse of non-monotonic functions. Branch points are at the extrema, separating increasing from decreasing segments. Colored dashed lines represent the result of applying the corresponding branch to the output of the function.

Panel of Figure 5 — Fig. 7 : Non-monotonic test function composition at quantization bitwidths $\beta^{*}=3$ and $\beta^{*}=6$ . The function has two increasing and one decreasing segment with branch points at approximately $(-0.126,0.078)$ and exactly $(2.4,0.0)$ .

	Original	SIRA	Ours
	[ 5 , 6 ]	[ 15 ]	Ours
Compilation
Approach	Manual Fixed Patterns	Automatic Search-Based	Semi-Automatic Fusing Primitives
Effort	Low	High 1	Low
Support
Monotonicity	Monotonic	Monotonic	Monotonic Non-Monotonic

TABLE I : Layer tail compilation approaches in FINN

Panel of Figure 7 — Fig. 10 : Non-monotonic per-tensor multi-threshold functions learned by the four activation layers of LeNet for $N=15$ steps.

Panel of Figure 8 — Fig. 11 : Comparison of learned multi-threshold functions to hand-selected quantized activation functions for MNIST classification with LeNet. Standard deviation in shaded.

Activation Quantization

Activation quantization reduces the precision of neural-network activations to lower inference and training memory use and accelerate computation.

See all 116 papers

GRAU: A Generic Runtime-Reconfigurable Activation-Requantization Unit for Quantized Neural Network Accelerators
Feb 25, 2026Yuhao Liu, Salim Ullah, Akash KumarNeural Network Hardware Acceleration Neural Network Activation Functions
Ruhr University Bochum, Germany · Dresden University of Technology, Germany · Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI Dresden/Leipzig), Germany
Quantization Robustness of Monotone Operator Equilibrium Networks
Mar 11, 2026James Li, Philip H. W. Leong, Thomas ChaffeyQuantization Robustness Neural Network Quantization
School of Electrical and Computer Engineering, The University of Sydney, NSW, Australia
Rethinking Neural Nonlinearity as Gating
Jul 3, 2026Muhammad Sabih, Frank Hannig, Jürgen TeichNeural Network Activation Functions Neural Network Approximation Theory
Department of Computer Science, Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Germany

Feb 25, 2026J/K move · Enter open · S save

GRAU: A Generic Runtime-Reconfigurable Activation-Requantization Unit for Quantized Neural Network Accelerators

Yuhao Liu, Salim Ullah, Akash Kumar

Ruhr University Bochum, Germany · Dresden University of Technology, Germany · Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI Dresden/Leipzig), Germany

Neural Network Hardware Acceleration Neural Network Activation Functions

Open paper AskPDF arXiv

As neural networks continue to scale, low-precision quantization is widely adopted in edge accelerators, but conventional multi-threshold activation hardware requires 2^n-1 thresholds for n-bit integer outputs, leading to rapidly increasing hardware costs. We propose GRAU, a Generic Runtime-Reconfigurable Activation-Requantization Unit that fuses batch normalization, nonlinear activation, and requantization through piecewise-linear fitting with Power-of-Two slopes. GRAU requires only basic comparators and 1-bit right shifters, while supporting mixed-precision quantization and runtime switching among post-accumulation functions, including modern nonlinear activations such as SiLU and GELU. Compared with multi-threshold activation-requantization units, GRAU reduces LUT consumption by approximately 90% or more while improving hardware efficiency, flexibility, and scalability. We further explore the trade-off between accuracy and hardware cost across different approximation settings and provide an open-source toolchain for GRAU-aware network training.

Abstract

Activation Quantization

Explore similar work

GRAU: A Generic Runtime-Reconfigurable Activation-Requantization Unit for Quantized Neural Network Accelerators

Quantization Robustness of Monotone Operator Equilibrium Networks

Rethinking Neural Nonlinearity as Gating