cs.AROct 15, 2026

Compiling Multi-Threshold Operators for Quantized Non-Monotonic and Fine-Granular Activations in Custom-Tailored DNN Accelerators

Authors: Christoph Berganski, Felix Jentzsch, Marco Platzner

Organizations: Computer Engineering Group, Paderborn University, Paderborn, Germany

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Aggressive quantization of neural networks can expose activation functions as bottlenecks for efficient hardware acceleration. Implementing quantized activations as multi-threshold operations has been shown to be a promising optimization, allowing for efficient hardware implementation of few-bit activations. However, current implementations are limited to monotonically increasing functions, quantization schemes with one-dimensional (per-channel) granularity and a small set of predefined activation-quantizer combinations. We formalize the multi-threshold representation to extend it to non-monotonic functions and fine-granular parameters, i.e., multi-dimensional thresholds. For automated conversion of quantized function compositions into threshold representation, we propose a systematic approach via algebraic rules integrated into the ONNX-based compiler infrastructure. We experimentally validate our solution on a selection of monotonic and non-monotonic functions by extending the binary search based multi-threshold operator that is integrated into the FINN framework for end-to-end synthesis of custom-tailored FPGA accelerators. The cost of hardware support for non-monotonicity and fine-granularity is characterized in comparison to an alternative approach utilizing existing operators via graph-level decomposition. We compare our method to prior work as well as the approach taken by the hls4ml framework. As an outlook, we demonstrate the prospects of using the multi-threshold representation during the training of a neural network as a form of quantization aware training.

Figures & tables

Fig. 1 : Hardware implementation of binary search based thresholding adapted from [ 15 ] . A remapping stage is added to allow for non-monotonic output levels LiL_{i} according to Eq. ( III ). To locate the input within the list of seven sorted thresholds, three comparators are required, corresponding to the three levels of the search tree shown on the left.
Fig. 3 : Address logic and layout of the threshold parameter memory. The address logic and the thresholding units are replicated as PE\mathtt{PE} parallel processing elements resulting in a two-level loop nest (i,j)(i,j) implemented in Vitis HLS.
Fig. 5 : Tree of branching threshold segments: Interpreting this as a parity function, this is equivalent to applying alternating weights ±1\pm 1 before summing over the sorted segment contributions. Sorting is always allowed due to commutativity.
Fig. 6 : Inverse of non-monotonic functions. Branch points are at the extrema, separating increasing from decreasing segments. Colored dashed lines represent the result of applying the corresponding branch to the output of the function.
Fig. 7 : Non-monotonic test function composition at quantization bitwidths β∗=3\beta^{*}=3 and β∗=6\beta^{*}=6 . The function has two increasing and one decreasing segment with branch points at approximately (−0.126,0.078)(-0.126,0.078) and exactly (2.4,0.0)(2.4,0.0) .
 OriginalSIRAOurs
 [ 5 , 6 ][ 15 ]
Compilation   
ApproachManual Fixed PatternsAutomatic Search-BasedSemi-Automatic Fusing Primitives
EffortLowHigh 1Low
Support   
MonotonicityMonotonicMonotonicMonotonic Non-Monotonic
TABLE I : Layer tail compilation approaches in FINN
Fig. 10 : Non-monotonic per-tensor multi-threshold functions learned by the four activation layers of LeNet for N=15N=15 steps.
Fig. 11 : Comparison of learned multi-threshold functions to hand-selected quantized activation functions for MNIST classification with LeNet. Standard deviation in shaded.
More in this topic

Activation Quantization

Activation quantization reduces the precision of neural-network activations to lower inference and training memory use and accelerate computation.

See all 116 papers

Explore similar work

CardsList
Feb 25, 2026

GRAU: A Generic Runtime-Reconfigurable Activation-Requantization Unit for Quantized Neural Network Accelerators

As neural networks continue to scale, low-precision quantization is widely adopted in edge accelerators, but conventional multi-threshold activation hardware requires 2^n-1 thresholds for n-bit integer outputs, leading to rapidly increasing hardware costs. We propose GRAU, a Generic Runtime-Reconfigurable Activation-Requantization Unit that fuses batch normalization, nonlinear activation, and requantization through piecewise-linear fitting with Power-of-Two slopes. GRAU requires only basic comparators and 1-bit right shifters, while supporting mixed-precision quantization and runtime switching among post-accumulation functions, including modern nonlinear activations such as SiLU and GELU. Compared with multi-threshold activation-requantization units, GRAU reduces LUT consumption by approximately 90% or more while improving hardware efficiency, flexibility, and scalability. We further explore the trade-off between accuracy and hardware cost across different approximation settings and provide an open-source toolchain for GRAU-aware network training.

Yuhao Liu, Salim Ullah, Akash Kumar

Ruhr University Bochum, Germany · Dresden University of Technology, Germany · Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI Dresden/Leipzig), Germany

Neural Network Hardware AccelerationNeural Network Activation Functions
AskPDF
Mar 11, 2026

Quantization Robustness of Monotone Operator Equilibrium Networks

Monotone operator equilibrium networks are implicit-layer models whose output is the unique equilibrium of a monotone operator, guaranteeing existence, uniqueness, and convergence. When deployed on low-precision hardware, weights are quantized, potentially destroying these guarantees. We analyze weight quantization as a spectral perturbation of the underlying monotone inclusion. Convergence of the quantized solver is guaranteed whenever the spectral-norm weight perturbation is smaller than the monotonicity margin; the displacement between quantized and full-precision equilibria is bounded in terms of the perturbation size and margin; and a condition number characterizing the ratio of the operator norm to the margin links quantization precision to forward error. MNIST experiments confirm a phase transition at the predicted threshold: three- and four-bit post-training quantization diverge, while five-bit and above converge. The backward-pass guarantee enables quantization-aware training, which recovers provable convergence at four bits.

James Li, Philip H. W. Leong, Thomas Chaffey

School of Electrical and Computer Engineering, The University of Sydney, NSW, Australia

Quantization RobustnessNeural Network QuantizationQuantization Sensitivity AnalysisLow-Bit QuantizationPost-Training Quantization
AskPDF
Jul 3, 2026

Rethinking Neural Nonlinearity as Gating

Activation functions are considered an essential primitive for neural nonlinearity, i.e., they enable neural networks to serve as universal approximators. In this paper, we show that this nonlinearity can also be achieved by input-conditioned threshold gating through branches as a universal primitive. We demonstrate that standard activations -- whether piecewise-linear (ReLU, PReLU, Hardtanh) or smooth (SiLU, Sigmoid, Tanh, GELU) -- are in fact instances of a single Threshold Gating (TG) primitive. For softmax, we show that it admits an exact TG conversion via its equivalent per-element Sigmoid form. We then validate these equivalences by converting pretrained networks across CNNs, transformer-based models, and recurrent architectures, preserving model performance without requiring retraining. Threshold Gating also enables training from scratch that goes beyond replacing existing activations, enabling gains in model compression, performance, and shorter training. We also propose a 'Minimal Branch Theorem' which relates the minimum number of required branches in our primitive to the trainability of general deep neural networks. In terms of hardware implementation, TG maps to a unified implementation in the case of analog in-memory systems, addressing the bottleneck of analog-to-digital and digital-to-analog converters (ADC/DAC) that is known to significantly impact power consumption and on-chip area.

Muhammad Sabih, Frank Hannig, Jürgen Teich

Department of Computer Science, Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Germany

Neural Network Activation FunctionsNeural Network Approximation Theory
AskPDF