Edge-Centric Adaptive Neural Activation via Parametric Tanh Functions

Andreas Falkenberg

cs.LGOct 25, 2026

Edge-Centric Adaptive Neural Activation via Parametric Tanh Functions

Authors: Andreas Falkenberg

Organizations: QPI Laboratories, Inc., USA

Abstract

Within the broader landscape of artificial intelligence architectures, this paper introduces a compact, edge-centric neural framework that resolves the computational bottlenecks of modern edge-activated paradigms like Kolmogorov-Arnold Networks (KANs) while retaining their expressiveness over traditional Multi-Layer Perceptrons (MLPs). While KANs demonstrate remarkable representational capacity by placing learnable univariate functions on connection edges rather than hidden nodes, their reliance on non-local B-splines introduces severe optimization hurdles, high memory overhead, and poor hardware scaling. To bridge this gap, we propose the Parametric Tanh Connection Function, a smooth, bounded architectural primitive defined as $a_{\tanh}(x;δ,α,β,γ) = δ+ α\tanh(βx + γ)$ . This formulation unifies offset, amplitude, local slope, and horizontal displacement into a lightweight, 4-parameter differentiable mapping optimized edge-by-edge via backpropagation. Evaluated on dense architectures using MNIST and CIFAR-10, our parametric tanh connection model achieves significant absolute accuracy gains over traditional affine baselines-reaching 95.63% $\pm$ 0.08% on MNIST and 44.74% $\pm$ 0.18% on CIFAR-10. Crucially, it matches or exceeds the expressiveness of unconstrained 7-parameter exponential connection baselines while requiring up to 85% less training runtime. By replacing complex splines or hyper-flexible exponents with a bounded, analytically smooth hyperbolic tangent kernel, this work delivers a highly scalable, hardware-friendly compromise for heterogeneous connection-level deep learning.

Figures & tables

Dataset	Configuration	Test Accuracy (%)	Training Time (s)
MNIST	Affine only	91.65 $\pm$ 0.12	150.37 $\pm$ 3.11
MNIST	Parametric tanh	95.63 $\pm$ 0.08	415.16 $\pm$ 5.42
MNIST	Full seven-parameter	94.91 $\pm$ 0.15	2017.15 $\pm$ 24.80
CIFAR-10	Affine only	39.94 $\pm$ 0.25	133.01 $\pm$ 2.95
CIFAR-10	Parametric tanh	44.74 $\pm$ 0.18	1092.58 $\pm$ 12.14
CIFAR-10	Full seven-parameter	36.13 $\pm$ 0.31	7393.19 $\pm$ 86.50

TABLE I: Final Benchmark Results for the Fully Connected Topology (Mean

\pm

SD over 5 Seeds)

Fig. 1: Cross-entropy training loss as a function of epoch for the affine-only, parametric tanh, and full seven-parameter connection configurations on MNIST. The legend reports the corresponding final test accuracies.

Fig. 2: Cross-entropy training loss as a function of epoch for the affine-only, parametric tanh, and full seven-parameter connection configurations on CIFAR-10. The legend reports the corresponding final test accuracies.

Dataset	Configuration	Test Accuracy (%)	Training Time (s)
MNIST	Affine only	98.99 $\pm$ 0.03	129.26 $\pm$ 1.84
MNIST	Parametric tanh	98.77 $\pm$ 0.05	550.32 $\pm$ 7.20
MNIST	Full seven-parameter	98.76 $\pm$ 0.06	3073.40 $\pm$ 35.15
CIFAR-10	Affine only	69.30 $\pm$ 0.14	117.46 $\pm$ 1.50
CIFAR-10	Parametric tanh	65.24 $\pm$ 0.21	579.81 $\pm$ 6.94
CIFAR-10	Full seven-parameter	57.72 $\pm$ 0.42	3192.85 $\pm$ 41.10

TABLE II: Final Benchmark Results for the Convolutional Topology (Mean

\pm

SD over 5 Seeds)

Fig. 3: Cross-entropy training loss as a function of epoch for the affine-only, parametric tanh, and full seven-parameter connection configurations on MNIST using the convolutional network. The legend reports the corresponding final test accuracies.

Fig. 4: Cross-entropy training loss as a function of epoch for the affine-only, parametric tanh, and full seven-parameter connection configurations on CIFAR-10 using the convolutional network. The legend reports the corresponding final test accuracies.

Neural Network Activation Functions

Neural network activation functions shape how models represent nonlinear patterns, learn, remain stable, and map efficiently to hardware.

See all 78 papers

DualFlexKAN: Dual-stage Kolmogorov-Arnold Networks with Independent Function Control
Mar 9, 2026Andrés Ortiz, Nicolás J. Gallego-Molina, Carmen Jiménez-Mesa +2Function Approximation Kolmogorov-Arnold Networks
Department of Communications Engineering University of Malaga Malaga, Spain · DaSCI Andalusian Institute of Data Science and Computational Intelligence University of Granada Granada, Spain · Department of signal Theory, Communications and Telematics University of Granada Granada, Spain
ANCRe: Adaptive Neural Connection Reassignment for Efficient Depth Scaling
Feb 9, 2026Yilang Zhang, Bingcong Li, Niao He +1Residual Learning Neural Network Optimization
ML Research, Morgan Stanley, New York, NY 10019 · Department of CS, ETH Zürich, 8092 Zürich, Switzerland · Department of ECE, University of Minnesota, Minneapolis, MN 55455
Sigma-Branch: Hierarchical Single-Path Network Reconstruction for Dynamic Inference with Reduced Active Parameters
Jun 7, 2026Kohga Tanaka, Hiroaki NishiEfficient Neural Network Inference Dynamic Neural Networks
Graduate School of Science and Technology, Keio University, Kohoku-ku, Yokohama 223-8522, Japan · Department of System Design, Faculty of Science and Technology, Keio University, Kohoku-ku, Yokohama 223-8522, Japan

Mar 9, 2026J/K move · Enter open · S save

DualFlexKAN: Dual-stage Kolmogorov-Arnold Networks with Independent Function Control

Andrés Ortiz, Nicolás J. Gallego-Molina, Carmen Jiménez-Mesa, Juan M. Górriz+1

Department of Communications Engineering University of Malaga Malaga, Spain · DaSCI Andalusian Institute of Data Science and Computational Intelligence University of Granada Granada, Spain · Department of signal Theory, Communications and Telematics University of Granada Granada, Spain

Function Approximation Kolmogorov-Arnold Networks Neural Network Approximation Theory

Open paper AskPDF arXiv

Multi-Layer Perceptrons (MLPs) rely on pre-defined, fixed activation functions, imposing a static inductive bias that forces the network to approximate complex topologies solely through increased depth and width. Kolmogorov-Arnold Networks (KANs) address this limitation through edge-centric learnable functions, yet their formulation suffers from quadratic parameter scaling and architectural rigidity that hinders the effective integration of standard regularization techniques. This paper introduces the DualFlexKAN (DFKAN), a flexible architecture featuring a dual-stage mechanism that independently controls pre-linear input transformations and post-linear output activations. This decoupling enables hybrid networks that optimize the trade-off between expressiveness and computational cost. Unlike standard formulations, DFKAN supports diverse basis function families, including orthogonal polynomials, B-splines, and radial basis functions, integrated with configurable regularization strategies that stabilize training dynamics. Comprehensive evaluations across regression benchmarks, physics-informed tasks, and function approximation demonstrate that DFKAN outperforms both MLPs and conventional KANs in accuracy, convergence speed, and gradient fidelity. The proposed hybrid configurations achieve superior performance with one to two orders of magnitude fewer parameters than standard KANs, effectively mitigating the parameter explosion problem while preserving KAN-style expressiveness. DFKAN provides a principled, scalable framework for incorporating adaptive non-linearities, proving particularly advantageous for data-efficient learning and interpretable function discovery in scientific applications.

Abstract

Neural Network Activation Functions

Explore similar work

DualFlexKAN: Dual-stage Kolmogorov-Arnold Networks with Independent Function Control

ANCRe: Adaptive Neural Connection Reassignment for Efficient Depth Scaling

Sigma-Branch: Hierarchical Single-Path Network Reconstruction for Dynamic Inference with Reduced Active Parameters