Surrogates, Spikes, and Sparsity: Performance Analysis and Characterization of SNN Hyperparameters on Hardware

Ilkin Aliyev; Jesus Lopez; Tosiron Adegbija

cs.ARMar 26, 2026

Surrogates, Spikes, and Sparsity: Performance Analysis and Characterization of SNN Hyperparameters on Hardware

Authors: Ilkin Aliyev, Jesus Lopez, Tosiron Adegbija

Organizations: Department of Electrical and Computer Engineering, The University of Arizona, Tucson, AZ, USA

Abstract

Spiking Neural Networks (SNNs) offer inherent advantages for low-power inference through sparse, event-driven computation. However, the theoretical energy benefits of SNNs are often decoupled from real hardware performance due to the opaque relationship between training-time choices and inference-time sparsity. While prior work has focused on weight pruning and compression, the role of training hyperparameters -- specifically surrogate gradient functions and neuron model configurations -- in shaping hardware-level activation sparsity remains underexplored. This paper presents a workload characterization study quantifying the sensitivity of hardware latency to SNN hyperparameters. We decouple the impact of surrogate gradient functions (e.g., Fast Sigmoid, Spike Rate Escape) and neuron models (LIF, Lapicque) on classification accuracy and inference efficiency across three event-based vision datasets: DVS128-Gesture, N-MNIST, and DVS-CIFAR10. Our analysis reveals that standard accuracy metrics are poor predictors of hardware efficiency. While Fast Sigmoid achieves the highest accuracy on DVS-CIFAR10, Spike Rate Escape reduces inference latency by up to 12.2% on DVS128-Gesture with minimal accuracy trade-offs. We also demonstrate that neuron model selection is as critical as parameter tuning; transitioning from LIF to Lapicque neurons yields up to 28% latency reduction. We validate on a custom cycle-accurate FPGA-based SNN instrumentation platform, showing that sparsity-aware hyperparameter selection can improve accuracy by 9.1% and latency by over 2x compared to baselines. These findings establish a methodology for predicting hardware behavior from training parameters. The RTL and reproducibility artifacts are at https://zenodo.org/records/18893738.

Figures & tables

Panel of Figure 1 — Fig. 1 : Hardware dataflow comparison between the LIF and Lapicque neuron models. LIF uses bit-shifts for decay (3 ops/update); Lapicque requires explicit multiplications for RC constants (4 ops + 2 $\times$ MUL). Despite higher per-update cost, Lapicque’s temporal dynamics suppress total spike events, yielding lower system-level latency.

Panel of Figure 2 — Fig. 2 : Visualization of the derivative profiles ( $\frac{\partial S}{\partial U}$ ) for different surrogate functions plotted against the membrane potential’s proximity to the firing threshold ( $U-U_{thr}$ ). Fast Sigmoid (a) maintains meaningful gradient support in the tails as $k$ increases. Arctangent (b) exhibits similar behavior with Cauchy-like tails that preserve gradient flow farther from the threshold. Spike Rate Escape (c) exhibits exponential decay, leading to vanishing gradients (gradient $\approx$ 0) for neurons far from the threshold (assuming $\beta=k$ ). The Stochastic Spike Operator (d) uses a constant gradient of 1 above threshold and injects uniform noise below threshold, enabling gradient flow through sub-threshold neurons. These mathematical properties explain the sharp accuracy degradation (“cliffs”) observed in our experimental characterization (Section IV ).

Surrogate Function	Backward Pass Formula	Key Characteristics
Fast Sigmoid (FS)	$\displaystyle\frac{\partial S}{\partial U}\approx\frac{1}{(1+k\|x\|)^{2}}$	Default snnTorch surrogate; computationally cheap with a sharp peak at threshold and quadratic heavy-tail decay. Parameter $k$ (“slope”) controls sharpness (larger $k$ $\rightarrow$ narrower peak).
Arctangent (ATAN)	$\displaystyle\frac{\partial S}{\partial U}\approx\frac{1}{\pi}\cdot\frac{1}{1+\left(\pi x\frac{\alpha}{2}\right)^{2}}$	Smooth, symmetric gradient with Cauchy-like tails; tends to preserve gradient flow farther from threshold than FS. Parameter $\alpha$ controls sharpness.
Spike Rate Escape (SRE)	$\displaystyle\frac{\partial S}{\partial U}\approx k\,\exp\!\big(-\beta\,\|x+(U_{\text{thr}}-1)\|\big)$	Escape-rate (Boltzmann-like) exponential surrogate as implemented in snnTorch: centered at $U\!=\!1$ . If $U_{\text{thr}}=1$ (common default), this simplifies to $\frac{\partial S}{\partial U}\approx ke^{-\beta\|x\|}$ . Parameters: $k$ sets peak scale; $\beta$ controls decay with distance from threshold.
Stochastic Spike Operator (SSO)	$\displaystyle\frac{\partial S}{\partial U}\approx\begin{cases}1&x\geq 0\\ \left(\mathcal{U}(-0.5,\,0.5)+\mu\right)\sigma^{2}&x<0\end{cases}$	Stochastic sub-threshold gradients: above threshold uses straight-through gradient 1; below threshold samples a uniform random value (shifted by mean $\mu$ ) scaled by variance term $\sigma^{2}$ . Intended to mitigate “dead” neurons by injecting stochastic gradient signal when $U<U_{\text{thr}}$ .

TABLE I : Overview of surrogate approximation functions as implemented in snnTorch [ 15 ] . All functions are written in threshold-centered form with

x=U-U_{\text{thr}}

Panel of Figure 4 — Fig. 3 : Overview of our design space exploration workflow.

Panel of Figure 5 — Fig. 4 : Dataflow of the hardware instrumentation platform (Parametric Sparsity-Aware Convolution Engine). The design features an Event Control Unit (ECU) that dynamically schedules only active neurons, and Neural Cores (NCs) with configurable accumulation logic. This event-driven architecture allows for precise isolation of latency reductions caused by algorithmic sparsity tuning.

Panel of Figure 6 — Fig. 5 : Accuracy trends for surrogate gradient functions across three datasets. Each plot shows classification accuracy as a function of the derivative scaling factor (slope) $\alpha$ for four surrogate functions: Fast Sigmoid , Arctangent , SRE , and SSO . For each surrogate, representative top-performing $\alpha$ values were selected from 40 Optuna trials per candidate configuration (each trained for up to 200 epochs with early stopping).

Panel of Figure 7 — Fig. 6 : Hardware inference latency across surrogate functions for three datasets. Each box represents the latency range for the top two slope configurations (best accuracy) of a surrogate. The red line connects the latency of the best-accuracy slope per surrogate. Accuracy values (blue) are annotated for the best configuration. Lower latency combined with higher accuracy indicates better efficiency.

Panel of Figure 8 — Fig. 7 : Pareto analysis of neuron configurations across datasets. Each plot shows accuracy versus latency for different $(\beta,\theta)$ parameter pairs using either LIF (blue markers) or LAP (yellow markers). Optimal configurations appear toward the upper-left (high accuracy, low latency). LAP consistently yields lower latency and often competitive or superior accuracy compared to LIF .

Dataset	Study	Net	Acc.	Latency	Power	EDP	$\textbf{F}_{Max}$
Dataset	Study	Net	[%]	[ms]	[W]	[mJ $\cdot$ ms]	[MHz]
NMNIST	ASIC [ 16 ]	net 1	97.8	3.8-12.5	0.011	0.16–1.72	400
NMNIST	TW	net 3	96.4	5.9	0.31	10.8	100
DVS	ASIC [ 16 ]	net 1	92.4	7.1-23.2	0.011	0.55–5.92	400
Gesture	FPGA [ 25 ]	net 2	86.3	0.46	0.42	0.09	100
	TW	net 3	95.4	11.2	1.35	169.3	100

TABLE II: Benchmarking tuned configurations against previous work. Note: Our VGG9 network is significantly deeper (9 layers) than the 2–4 layer baselines, yet achieves competitive latency. EDP = Energy-Delay Product.

Spiking Neural Networks

Spiking neural networks process information through discrete, time-dependent neural events, aiming for efficient computation and biologically inspired learning.

See all 351 papers

SupraSNN: Exploiting Synapse-Level Parallelism in Spiking Neural Network Accelerators through Co-Optimized Mapping and Scheduling
Jun 11, 2026Seyed Sadra Ghavami, Mohammad Hossein Nikkhah, Mohammad Rasoul Roshanshah +1Spiking Neural Networks Hardware-Software Co-Design
High-Performance Embedded Architecture Laboratory (HiPEAL) School of Electrical and Computer Engineering College of Engineering, University of Tehran Tehran, Iran
Sharpness-Aware Surrogate Training for On-Sensor Spiking Neural Networks
Apr 6, 2026Maximilian NicholsonSpiking Neural Networks Event-Based Vision
University of Bath, United Kingdom
Spiking Neural Network inference on FPGAs with hls4ml
Jun 8, 2026Barry M. DillonEfficient Neural Network Inference Spiking Neural Networks
ISRC, Ulster University, Derry, BT48 7JL, Northern Ireland

Jun 11, 2026J/K move · Enter open · S save

SupraSNN: Exploiting Synapse-Level Parallelism in Spiking Neural Network Accelerators through Co-Optimized Mapping and Scheduling

Seyed Sadra Ghavami, Mohammad Hossein Nikkhah, Mohammad Rasoul Roshanshah, Saeed Safari

High-Performance Embedded Architecture Laboratory (HiPEAL) School of Electrical and Computer Engineering College of Engineering, University of Tehran Tehran, Iran

Spiking Neural Networks Hardware-Software Co-Design Neuromorphic Computing FPGA-Based Neural Network Acceleration

Open paper AskPDF arXiv

Spiking Neural Networks (SNNs) offer a brain-inspired path toward highly efficient computation, but their practical deployment is constrained by the challenge of managing and executing their massive parallelism on physical hardware. This problem mirrors the historical challenge in processor design of moving beyond serial execution, a barrier broken by superscalar architectures that dispatch multiple instructions to parallel functional units. Drawing inspiration from this paradigm, we introduce a hardware-software co-design framework that treats synaptic events as parallelizable micro-operations. We present SupraSNN, a superscalar-inspired architecture that achieves high synapse-level parallelism by physically decoupling synaptic and neuronal computations. Within this architecture, a Multi-Cast Tree routes spike data to multiple parallel Synapse Processing Units serve as the computational pipelines, while a Merge Tree consolidates distributed results for processing by a unified Neuron Unit--deliberately centralizing complex neuron state dynamics to mitigate hardware overhead and resource duplication. The efficacy of this architecture is enabled by a sophisticated partitioning and scheduling framework that first maps the SNN onto hardware respecting memory constraints, then heuristic scheduling determines the synaptic execution order, maximizing throughput and resource utilization. Implementing a feedforward SNN trained on MNIST (93.44% accuracy), SupraSNN achieves 149

μs

inference latency and 0.025 mJ per image (0.276 nJ per synapse) on the Xilinx Zynq XC7Z020 FPGA--delivering 47.6% lower latency and 5.6

\times

better energy efficiency than prior FPGA-based SNN accelerators. Beyond vision tasks, a recurrent SNN on the Spiking Heidelberg Dataset (71.82% accuracy) achieves 1.41 ms latency and 0.77 mJ per sample on XC7Z030.

Abstract

Spiking Neural Networks

Explore similar work

SupraSNN: Exploiting Synapse-Level Parallelism in Spiking Neural Network Accelerators through Co-Optimized Mapping and Scheduling

Sharpness-Aware Surrogate Training for On-Sensor Spiking Neural Networks

Spiking Neural Network inference on FPGAs with hls4ml