MulClip: A table-free activation function with monotonic and folding regimes for reservoir computing on microcontrollers

Guren Matsumura; Masato Takemoto; Shinzo Koyama

cs.LGOct 29, 2026

MulClip: A table-free activation function with monotonic and folding regimes for reservoir computing on microcontrollers

Authors: Guren Matsumura, Masato Takemoto, Shinzo Koyama

Organizations: Panasonic Industry Co., Ltd., Osaka, Japan

Abstract

This letter proposes MulClip, a lightweight activation function for reservoir computing (RC) on microcontrollers (MCUs). In a ring reservoir, a representative sparse RC topology, multiply-accumulate operations are few, and evaluating tanh accounted for 75% of the state-update time on a Cortex-M4. MulClip introduces curvature into the response using only comparison, absolute value, multiplication, and subtraction, and places its clipping threshold not at the extremum but at the point where the response returns to zero after passing the extremum. As a result, a single function has both a monotonic regime and a nonmonotonic folding regime that increases the expressiveness of a single unit. Which regime is used is determined by the usual hyperparameter search, e.g., over the input scale, so no task-specific threshold tuning is required. The state-update time measured on hardware was 0.34 times that of tanh and shorter than that of any of the 11 evaluated tanh approximations. In PC-based evaluation, third-order MulClip achieved a 22% lower NMSE than tanh on NARMA-20, where the pre-activations remained in the monotonic regime, and on 8-bit parity it raised the accuracy from 0.76-0.78 for the monotonic-clip variant without folding to 0.97-0.98. For these two tasks, on-device inference yielded nearly the same values as on the PC.

Figures & tables

Fig. 1: Response of MulClip (blue:

d=2

; red:

d=3

). Small dots mark the extremum at

d/(d-1)

, and open circles mark the clipping threshold

c_{d}

. Gray dashed lines show the monotonic-clip variant, whose clipping threshold is set at

d/(d-1)

. The dotted line is

\tanh

Fig. 2: NMSE versus state-update time (500 units) on NARMA-20. Lower left is better. The horizontal axis is broken. The horizontal dashed line indicates the NMSE of

\tanh

(FPU). The connected black squares are, from left to right, hardtanh, MulClip with

d=2

, and MulClip with

d=3

. Numbers beside the LUT points denote the number of entries. Hexagons denote the functions of Abe et al. [ 15 ] ; F2 (NMSE 0.60) lies outside the vertical-axis range and is plotted at the top edge. The times for the functions of Abe et al. were measured in a separate build (Appendix B ).

	NARMA-20	Parity8	MG	KWS	State-update	Flash
Activation	NMSE $\downarrow$	Acc. $\uparrow$	VPT $\uparrow$	Acc. $\uparrow$	time ratio $\downarrow$	ratio $\downarrow$
MulClip $d=2$	0.087	0.981	473	0.771	0.34	0.94
MulClip $d=3$	0.063	0.968	599	0.781	0.34	0.94
Monotonic clip $d=2$	0.087	0.764	472	0.772	—	—
Monotonic clip $d=3$	0.063	0.781	582	0.785	—	—
$\tanh$ (FPU)	0.081	0.761	628	0.779	1.00	1.00

Table I: Accuracy (Mean Over 50 Seeds), State-Update Time, and Flash Usage (Relative to

\tanh

(FPU)) of Each Activation

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Activation	Cycles/eval.	State-update time [ms]	Flash [B]	Increment from smallest [B]
hardtanh	25.0	0.513	13,440	16
MulClip $d=2$	30.0	0.543	13,464	40
MulClip $d=3$	31.0	0.549	13,480	56
$\tanh$ (FPU)	203.2	1.615	14,304	880
Rational 3/2	38.0	0.638	13,480	56
Rational 5/4	49.0	0.704	13,520	96

Appendix

Table B1: Measured values for all activations (500 units, per step)

Activation	NARMA-20 NMSE	Parity8 accuracy	MG VPT	KWS accuracy
LUT nearest 10	0.2516	0.697	54	0.679
LUT nearest 100	0.2092	0.911	100	0.746
LUT nearest 1000	0.1558	0.980	208	0.771
LUT nearest 4096	0.1310	0.979	301	0.772
LUT nearest 16384	0.1104	0.990	391	0.776
LUT interp. 16	0.1439	0.763	435	0.757

Appendix

Table C1: Accuracy for each LUT size (mean over 50 seeds)

		Beyond the extremum	Folding regime	Zero regime
Task	$d$	( $\|x\|>d/(d-1)$ )	( $d/(d-1)<\|x\|<c_{d}$ )	( $\|x\|\geq c_{d}$ )
NARMA-20	2	0%	0%	0%
NARMA-20	3	0%	0%	0%
Parity8	2	30.1%	19.3%	10.8%
Parity8	3	26.8%	16.6%	10.2%
MG	2	0%	0%	0%

Appendix

Table D1: Distribution of the pre-activations of MulClip (50 seeds, all steps and all units in training and evaluation)

$c$	Response shape	Accuracy
2.0	Monotonic (monotonic-clip variant)	0.764
3.0	Folding stops partway	0.877
4.0	$c_{d}$ (MulClip in the main text)	0.981
5.0	Beyond $c_{d}$ (with a sign-inverted region)	0.999
4.0	Zero regime only (constant 1 from the extremum to $c_{d}$ , 0 outside)	0.810

Appendix

Table D2: Accuracy on Parity8 (

d=2

) for different clipping thresholds

c

and for the zero-regime-only variant (mean over 50 seeds)

Activation	Count8 NMSE
MulClip $d=2$	$3.31\times 10^{-8}$
MulClip $d=3$	$1.12\times 10^{-8}$
Monotonic clip $d=2$	$3.31\times 10^{-8}$
Monotonic clip $d=3$	$1.12\times 10^{-8}$
$\tanh$ (FPU)	$1.06\times 10^{-8}$

Appendix

Table E1: NMSE on Count8 (mean over 50 seeds)

Activation	$k=7$	$k=8$	$k=9$	$k=10$
MulClip $d=2$	1.000	0.981	0.802	0.637
MulClip $d=3$	1.000	0.968	0.754	0.625
Monotonic clip $d=2$	0.972	0.764	0.625	0.559
Monotonic clip $d=3$	0.960	0.781	0.637	0.568
$\tanh$ (FPU)	0.937	0.761	0.622	0.561
Rational 3/2	0.949	0.767	0.624	0.564

Appendix

Table E2: Accuracy on Parity-

k

(

k\geq 7

) (mean over 50 seeds)

Task	Activation	Device	PC
NARMA-20 (NMSE)	$\tanh$ (FPU)	0.08340	0.08340
NARMA-20 (NMSE)	MulClip $d=2$	0.08999	0.08999
NARMA-20 (NMSE)	MulClip $d=3$	0.06472	0.06472
Parity8 (accuracy)	$\tanh$ (FPU)	0.7483	0.7487
Parity8 (accuracy)	Monotonic clip $d=2$	0.7535	0.7518
Parity8 (accuracy)	Monotonic clip $d=3$	0.7588	0.7589

Appendix

Table F1: Comparison of on-device inference and PC computation (mean over five seeds)

Neural Network Activation Functions

Neural network activation functions shape how models represent nonlinear patterns, learn, remain stable, and map efficiently to hardware.

See all 79 papers

Memristive-Friendly Hadamard Reservoir Computing: Structured, Multiplier-Free Recurrences at Scale
Aug 28, 2026Andrea Ceni, Gianluca Milano, Carlo Ricciardi +1Reservoir Computing Neuromorphic Computing
Department of Computer Science, University of Pisa, Italy · Istituto Nazionale di Ricerca Metrologica (INRiM), Italy · Department of Applied Science and Technology, Politecnico di Torino, Italy
On the Role of Preprocessing and Memristor Dynamics in Reservoir Computing for Image Classification
Apr 23, 2026Rishona Daniels, Duna Wattad, Ronny Ronen +2Reservoir Computing Neuromorphic Computing
Viterbi Faculty of Electrical and Computing Engineering, Technion - Israel Institute of Technology, Haifa, Israel · College of Engineering and Physical Sciences, Aston University, Birmingham B4 7ET, United Kingdom
A Constant-Time Implementation Methodology for Activation Functions on Microcontrollers
May 21, 2026Andrii Tyvodar, Andreas Rechberger, Dirmanto Jap +4Edge Inference Neural Network Activation Functions
Faculty of Informatics and Information Technologies, Slovak University of Technology in Bratislava, Bratislava, Slovakia · Temasek Laboratories, Nanyang Technological University, Singapore · Faculty of Computer Science, Albstadt-Sigmaringen University, Albstadt, Germany +2

Aug 28, 2026J/K move · Enter open · S save

Memristive-Friendly Hadamard Reservoir Computing: Structured, Multiplier-Free Recurrences at Scale

Andrea Ceni, Gianluca Milano, Carlo Ricciardi, Claudio Gallicchio

Department of Computer Science, University of Pisa, Italy · Istituto Nazionale di Ricerca Metrologica (INRiM), Italy · Department of Applied Science and Technology, Politecnico di Torino, Italy

Reservoir Computing Neuromorphic Computing Dynamic Neural Networks

Open paper AskPDF arXiv

Reservoir Computing (RC) designs Recurrent Neural Networks around a fixed, i.e., untrained, recurrent layer, and is a natural candidate for neuromorphic hardware. Memristive-friendly reservoirs derive the neuron dynamics from memristive-device kinetics, but still rely on dense recurrent matrices, which are expensive to realize physically. In this paper, we replace the dense matrix with a structured orthogonal operator, built from sign diagonals, a permutation, and a fast Walsh-Hadamard transform. The operator is multiplier-free, requires

O(N)

parameters and

O(N\log N)

operations per step, and is never materialized as a matrix. We instantiate it in a standard and in a memristive-friendly Echo State Network, with one binary input connection per unit. Our mathematical analysis shows that exact orthogonality yields an echo state condition that is tight in the recurrent scaling, and a noise response that is predictable at design time. Moreover, the operator mixes the whole state in a single application. Experiments on twenty classification and seven regression benchmarks, at reservoir sizes up to

N = 8192

, show that the structured models match dense orthogonal reservoirs, and achieve better mean performance than the cycle reservoir by a margin that widens with size. Furthermore, we time the recurrent step on three hardware platforms, where it is up to

50\times

faster than a dense product and

10^4\times

smaller in memory. Finally, we ablate the operator and measure the response to noise, quantization, device mismatch and discrete faults.

Abstract

Neural Network Activation Functions

Explore similar work

Memristive-Friendly Hadamard Reservoir Computing: Structured, Multiplier-Free Recurrences at Scale

On the Role of Preprocessing and Memristor Dynamics in Reservoir Computing for Image Classification

A Constant-Time Implementation Methodology for Activation Functions on Microcontrollers