Spiking Neural Networks (SNNs) offer inherent advantages for low-power inference through sparse, event-driven computation. However, the theoretical energy benefits of SNNs are often decoupled from real hardware performance due to the opaque relationship between training-time choices and inference-time sparsity. While prior work has focused on weight pruning and compression, the role of training hyperparameters -- specifically surrogate gradient functions and neuron model configurations -- in shaping hardware-level activation sparsity remains underexplored. This paper presents a workload characterization study quantifying the sensitivity of hardware latency to SNN hyperparameters. We decouple the impact of surrogate gradient functions (e.g., Fast Sigmoid, Spike Rate Escape) and neuron models (LIF, Lapicque) on classification accuracy and inference efficiency across three event-based vision datasets: DVS128-Gesture, N-MNIST, and DVS-CIFAR10. Our analysis reveals that standard accuracy metrics are poor predictors of hardware efficiency. While Fast Sigmoid achieves the highest accuracy on DVS-CIFAR10, Spike Rate Escape reduces inference latency by up to 12.2% on DVS128-Gesture with minimal accuracy trade-offs. We also demonstrate that neuron model selection is as critical as parameter tuning; transitioning from LIF to Lapicque neurons yields up to 28% latency reduction. We validate on a custom cycle-accurate FPGA-based SNN instrumentation platform, showing that sparsity-aware hyperparameter selection can improve accuracy by 9.1% and latency by over 2x compared to baselines. These findings establish a methodology for predicting hardware behavior from training parameters. The RTL and reproducibility artifacts are at https://zenodo.org/records/18893738.
Figures & tables
Fig. 1 : Hardware dataflow comparison between the LIF and Lapicque neuron models. LIF uses bit-shifts for decay (3 ops/update); Lapicque requires explicit multiplications for RC constants (4 ops + 2 × MUL). Despite higher per-update cost, Lapicque’s temporal dynamics suppress total spike events, yielding lower system-level latency.
Fig. 2 : Visualization of the derivative profiles ( ∂U∂S ) for different surrogate functions plotted against the membrane potential’s proximity to the firing threshold ( U−Uthr ). Fast Sigmoid (a) maintains meaningful gradient support in the tails as k increases. Arctangent (b) exhibits similar behavior with Cauchy-like tails that preserve gradient flow farther from the threshold. Spike Rate Escape (c) exhibits exponential decay, leading to vanishing gradients (gradient ≈ 0) for neurons far from the threshold (assuming β=k ). The Stochastic Spike Operator (d) uses a constant gradient of 1 above threshold and injects uniform noise below threshold, enabling gradient flow through sub-threshold neurons. These mathematical properties explain the sharp accuracy degradation (“cliffs”) observed in our experimental characterization (Section IV ).
Surrogate Function
Backward Pass Formula
Key Characteristics
Fast Sigmoid (FS)
∂U∂S≈(1+k∣x∣)21
Default snnTorch surrogate; computationally cheap with a sharp peak at threshold and quadratic heavy-tail decay. Parameter k (“slope”) controls sharpness (larger k→ narrower peak).
Arctangent (ATAN)
∂U∂S≈π1⋅1+(πx2α)21
Smooth, symmetric gradient with Cauchy-like tails; tends to preserve gradient flow farther from threshold than FS. Parameter α controls sharpness.
Spike Rate Escape (SRE)
∂U∂S≈kexp(−β∣x+(Uthr−1)∣)
Escape-rate (Boltzmann-like) exponential surrogate as implemented in snnTorch: centered at U=1 . If Uthr=1 (common default), this simplifies to ∂U∂S≈ke−β∣x∣ . Parameters: k sets peak scale; β controls decay with distance from threshold.
Stochastic Spike Operator (SSO)
∂U∂S≈{1(U(−0.5,0.5)+μ)σ2x≥0x<0
Stochastic sub-threshold gradients: above threshold uses straight-through gradient 1; below threshold samples a uniform random value (shifted by mean μ ) scaled by variance term σ2 . Intended to mitigate “dead” neurons by injecting stochastic gradient signal when U<Uthr .
TABLE I : Overview of surrogate approximation functions as implemented in snnTorch [ 15 ] . All functions are written in threshold-centered form with x=U−Uthr .
Fig. 3 : Overview of our design space exploration workflow.
Fig. 4 : Dataflow of the hardware instrumentation platform (Parametric Sparsity-Aware Convolution Engine). The design features an Event Control Unit (ECU) that dynamically schedules only active neurons, and Neural Cores (NCs) with configurable accumulation logic. This event-driven architecture allows for precise isolation of latency reductions caused by algorithmic sparsity tuning.
Fig. 5 : Accuracy trends for surrogate gradient functions across three datasets. Each plot shows classification accuracy as a function of the derivative scaling factor (slope) α for four surrogate functions: Fast Sigmoid , Arctangent , SRE , and SSO . For each surrogate, representative top-performing α values were selected from 40 Optuna trials per candidate configuration (each trained for up to 200 epochs with early stopping).
Fig. 6 : Hardware inference latency across surrogate functions for three datasets. Each box represents the latency range for the top two slope configurations (best accuracy) of a surrogate. The red line connects the latency of the best-accuracy slope per surrogate. Accuracy values (blue) are annotated for the best configuration. Lower latency combined with higher accuracy indicates better efficiency.
Fig. 7 : Pareto analysis of neuron configurations across datasets. Each plot shows accuracy versus latency for different (β,θ) parameter pairs using either LIF (blue markers) or LAP (yellow markers). Optimal configurations appear toward the upper-left (high accuracy, low latency). LAP consistently yields lower latency and often competitive or superior accuracy compared to LIF .
Dataset
Study
Net
Acc.
Latency
Power
EDP
FMax
[%]
[ms]
[W]
[mJ ⋅ ms]
[MHz]
NMNIST
ASIC [ 16 ]
net 1
97.8
3.8-12.5
0.011
0.16–1.72
400
TW
net 3
96.4
5.9
0.31
10.8
100
DVS
ASIC [ 16 ]
net 1
92.4
7.1-23.2
0.011
0.55–5.92
400
Gesture
FPGA [ 25 ]
net 2
86.3
0.46
0.42
0.09
100
TW
net 3
95.4
11.2
1.35
169.3
100
TABLE II: Benchmarking tuned configurations against previous work. Note: Our VGG9 network is significantly deeper (9 layers) than the 2–4 layer baselines, yet achieves competitive latency. EDP = Energy-Delay Product.
Spiking neural networks process information through discrete, time-dependent neural events, aiming for efficient computation and biologically inspired learning.
Spiking Neural Networks (SNNs) offer a brain-inspired path toward highly efficient computation, but their practical deployment is constrained by the challenge of managing and executing their massive parallelism on physical hardware. This problem mirrors the historical challenge in processor design of moving beyond serial execution, a barrier broken by superscalar architectures that dispatch multiple instructions to parallel functional units. Drawing inspiration from this paradigm, we introduce a hardware-software co-design framework that treats synaptic events as parallelizable micro-operations. We present SupraSNN, a superscalar-inspired architecture that achieves high synapse-level parallelism by physically decoupling synaptic and neuronal computations. Within this architecture, a Multi-Cast Tree routes spike data to multiple parallel Synapse Processing Units serve as the computational pipelines, while a Merge Tree consolidates distributed results for processing by a unified Neuron Unit--deliberately centralizing complex neuron state dynamics to mitigate hardware overhead and resource duplication. The efficacy of this architecture is enabled by a sophisticated partitioning and scheduling framework that first maps the SNN onto hardware respecting memory constraints, then heuristic scheduling determines the synaptic execution order, maximizing throughput and resource utilization. Implementing a feedforward SNN trained on MNIST (93.44% accuracy), SupraSNN achieves 149 μs inference latency and 0.025 mJ per image (0.276 nJ per synapse) on the Xilinx Zynq XC7Z020 FPGA--delivering 47.6% lower latency and 5.6× better energy efficiency than prior FPGA-based SNN accelerators. Beyond vision tasks, a recurrent SNN on the Spiking Heidelberg Dataset (71.82% accuracy) achieves 1.41 ms latency and 0.77 mJ per sample on XC7Z030.
Seyed Sadra Ghavami, Mohammad Hossein Nikkhah, Mohammad Rasoul Roshanshah +1
High-Performance Embedded Architecture Laboratory (HiPEAL) School of Electrical and Computer Engineering College of Engineering, University of Tehran Tehran, Iran
Spiking neural networks (SNNs) are a natural computational model for on-sensor and near-sensor vision, where event driven processors must operate under strict power budgets with hard binary spikes. However, models trained with surrogate gradients often degrade sharply when the smooth surrogate nonlinearity is replaced by a hard threshold at deployment; a surrogate-to-hard transfer gap that directly limits on-sensor accuracy. We study Sharpness-Aware Surrogate Training (SAST), which applies Sharpness-Aware Minimization (SAM) to a surrogate-forward SNN so that the training objective is smooth and the gradient is exact, and position it as one gap-reduction strategy under the tested settings rather than the only viable mechanism. Under explicit contraction assumptions we provide state-stability, input-Lipschitz, and smoothness bounds, together with a corresponding nonconvex convergence result. On two event-camera benchmarks, swap-only hard-spike accuracy improves from 65.7% to 94.7% on N-MNIST and from 31.8% to 63.3% on DVS Gesture. Under a hardware-aware inference simulation (INT8/INT4 weight quantization, fixed-point membrane potentials, discrete leak factors), SAST remains strong: on N-MNIST, hard-spike accuracy improves from 47.6% to 96.9% (INT8) and from 43.2% to 81.0% (INT4), while on DVS Gesture it improves from 25.3% to 47.6% (INT8) and from 26.0% to 43.8% (INT4). SynOps also decrease under the same hardware-aware setting, including 1734k→1315k (N-MNIST, INT8) and 86221k→4323k (DVS Gesture, INT8). These results suggest that SAST is a promising component in a broader toolbox for on-sensor spiking inference under the tested settings.
Spiking Neural Networks (SNNs) provide a naturally temporal machine-learning framework. Their neurons maintain an internal state and propagate information through discrete spikes, enabling low-latency temporal inference. Although SNNs are often associated with asynchronous neuromorphic processors, many scientific real-time inference systems rely on conventional synchronous field-programmable gate arrays (FPGAs) and high-level synthesis (HLS) workflows. In this paper we present an extension of hls4ml that enables clock-driven deployment of SNNs trained in pytorch onto FPGA firmware. We demonstrate the workflow using a dense quantised SNN trained on the Heidelberg Spiking Digits dataset where it achieves inference latencies of approximately 34μs. We validate the generated design through software reference comparisons, HLS C simulation, HLS synthesis, export, and Vivado synthesis reports. This work opens up the hls4ml toolkit to neuromorphic computing, allowing streamlined optimisation, synthesis, and deployment of SNN models for real-time inference.