Sharpness-Aware Surrogate Training for On-Sensor Spiking Neural Networks

Maximilian Nicholson

cs.NEApr 6, 2026

Sharpness-Aware Surrogate Training for On-Sensor Spiking Neural Networks

Authors: Maximilian Nicholson

Organizations: University of Bath, United Kingdom

Abstract

Spiking neural networks (SNNs) are a natural computational model for on-sensor and near-sensor vision, where event driven processors must operate under strict power budgets with hard binary spikes. However, models trained with surrogate gradients often degrade sharply when the smooth surrogate nonlinearity is replaced by a hard threshold at deployment; a surrogate-to-hard transfer gap that directly limits on-sensor accuracy. We study Sharpness-Aware Surrogate Training (SAST), which applies Sharpness-Aware Minimization (SAM) to a surrogate-forward SNN so that the training objective is smooth and the gradient is exact, and position it as one gap-reduction strategy under the tested settings rather than the only viable mechanism. Under explicit contraction assumptions we provide state-stability, input-Lipschitz, and smoothness bounds, together with a corresponding nonconvex convergence result. On two event-camera benchmarks, swap-only hard-spike accuracy improves from 65.7% to 94.7% on N-MNIST and from 31.8% to 63.3% on DVS Gesture. Under a hardware-aware inference simulation (INT8/INT4 weight quantization, fixed-point membrane potentials, discrete leak factors), SAST remains strong: on N-MNIST, hard-spike accuracy improves from 47.6% to 96.9% (INT8) and from 43.2% to 81.0% (INT4), while on DVS Gesture it improves from 25.3% to 47.6% (INT8) and from 26.0% to 43.8% (INT4). SynOps also decrease under the same hardware-aware setting, including 1734k $\rightarrow$ 1315k (N-MNIST, INT8) and 86221k $\rightarrow$ 4323k (DVS Gesture, INT8). These results suggest that SAST is a promising component in a broader toolbox for on-sensor spiking inference under the tested settings.

Figures & tables

Panel of Figure 1 — Figure 1: SAST pipeline: two-pass SAM with state reset, then hard-spike deployment without post-hoc calibration.

Dataset	Method	Surrogate forward	Swap-only hard-spike	$\Delta_{\text{transfer}}$
N-MNIST	Baseline surrogate training	.9606 $\pm$ .0033	.6572 $\pm$ .0974	.3034
N-MNIST	SAST $\rho\!=\!0.10$	.9786 $\pm$ .0009	.8335 $\pm$ .0894	.1451
N-MNIST	SAST $\rho\!=\!0.30$	.9721 $\pm$ .0012	.9473 $\pm$ .0462	.0248
DVS Gest.	Baseline surrogate training	.7502 $\pm$ .0142	.3182 $\pm$ .0732	.4320
DVS Gest.	SAST $\rho\!=\!0.20$	.8087 $\pm$ .0043	.5957 $\pm$ .0116	.2130
DVS Gest.	SAST $\rho\!=\!0.40$	.7685 $\pm$ .0151	.6327 $\pm$ .0116	.1358

Table 1: Main results: surrogate-forward and swap-only hard-spike accuracy, plus transfer gap. Best values are in bold .

Dataset	Arch./control	Method	Surrogate forward	Swap-only hard-spike	$\Delta_{\text{transfer}}$
N-MNIST	Conv-SNN ( ${\sim}0.42$ M)	Baseline surrogate training	.9868 $\pm$ .0009	.9091 $\pm$ .0153	.0777
N-MNIST	Conv-SNN ( ${\sim}0.42$ M)	SAST (best $\rho$ )	.9877 $\pm$ .0003	.9733 $\pm$ .0032	.0144
DVS Gest.	Conv-SNN ( ${\sim}0.42$ M)	Baseline surrogate training	.7438 $\pm$ .0044	.4630 $\pm$ .0075	.2808
DVS Gest.	Conv-SNN ( ${\sim}0.42$ M)	SAST (best $\rho$ )	.7284 $\pm$ .0115	.6204 $\pm$ .0151	.1080

Table 2: Added controls under swap-only hard-spike evaluation: Conv-SNN architecture ablation. Best values are in bold .

Figure 2: (a) Swap-only hard-spike membrane margins

(u_{t}\!-\!\theta)

on N-MNIST. Baseline mass within

\pm 0.2

: 4.8%; SAST: 2.3%. (b) Hardware-aware hard-spike accuracy under hardware constraints; on N-MNIST at INT8/Q8.8, SAST is about 97% vs. about 48% for baseline.

Dataset	Method	Epochs	Swap-only hard-spike	$\Delta_{\text{transfer}}$
N-MNIST	Baseline (compute-matched)	100	.657 $\pm$ .097	.303
N-MNIST	SAST (best $\rho$ )	48	.939 $\pm$ .022	.036
DVS Gest.	Baseline (compute-matched)	400	.280 $\pm$ .070	.504
DVS Gest.	SAST (best $\rho$ )	226	.578 $\pm$ .038	.125

Table 3: Compute-matched comparison under swap-only hard-spike evaluation (wall-clock matched); SAST uses per-dataset best

\rho

Dataset	Profile	Baseline acc.	SAST acc.	Baseline kSynOps	SAST kSynOps	$r_{\text{ops}}$
N-MNIST	Loihi-like (INT8, Q8.8)	0.476	0.969	1734.0	1315.0	0.758
N-MNIST	Aggressive (INT4, Q4.4)	0.432	0.810	1666.0	1346.0	0.808
DVS Gest.	Loihi-like (INT8, Q8.8)	0.253	0.476	86221.3	4323.5	0.050
DVS Gest.	Aggressive (INT4, Q4.4)	0.260	0.438	82317.0	4145.6	0.050

Table 4: Hardware-aware hard-spike summary (independent runs). kSynOps: synaptic accumulations per sample-sequence (

\times 10^{3}

T{=}10

), and

r_{\text{ops}}\mathrel{\mathop{:}}=\mathrm{kSynOps}_{\text{SAST}}/\mathrm{kSynOps}_{\text{Baseline}}

Spiking Neural Networks

Spiking neural networks process information through discrete, time-dependent neural events, aiming for efficient computation and biologically inspired learning.

See all 345 papers

Mar 14, 2026

Sharpness Aware Surrogate Training for Spiking Neural Networks

Surrogate gradients are a standard tool for training spiking neural networks (SNNs), but conventional hard forward or surrogate backward training couples a nonsmooth forward model with a biased gradient estimator. We study sharpness aware Surrogate Training (SAST), which applies sharpness aware Minimization (SAM) to a surrogate forward SNN trained by backpropagation. In this formulation, the optimization target is an ordinary smooth empirical risk, so the training gradient is exact for the auxiliary model being optimized. Under explicit boundedness and contraction assumptions, we derive compact state stability and input Lipschitz bounds, establish smoothness of the surrogate objective, provide a first order SAM approximation bound, and prove a nonconvex convergence guarantee for stochastic SAST with an independent second minibatch. We also isolate a local mechanism proposition, stated separately from the unconditional guarantees, that links per sample parameter gradient control to smaller input gradient norms under local Jacobian conditioning. Empirically, we evaluate clean accuracy, hard spike transfer, corruption robustness, and training overhead on N-MNIST and DVS Gesture. The clearest practical effect is transfer gap reduction: on N-MNIST, hard spike accuracy rises from 65.7% to 94.7% (best at

ρ=0.30

) while surrogate forward accuracy remains high; on DVS Gesture, hard spike accuracy improves from 31.8% to 63.3% (best at

ρ=0.40

). We additionally specify the compute matched, calibration, and theory alignment controls required for a final practical assessment.

Maximilian Nicholson

University of Bath, United Kingdom

Spiking Neural Networks Sharpness-Aware Minimization

Ask PDF

Aug 13, 2026

SAGE: Surrogate-gradient Adaptation via Attention-Guided Entropy for Spiking Transformers

Spiking neural networks (SNNs) offer an energy-efficient alternative to conventional deep neural networks by exploiting sparse event-driven computation, but their training remains challenging because the non-differentiable spike function requires surrogate gradients whose fixed shape may be suboptimal across layers and training stages. In this work, we introduce SAGE, an uncertainty-modulated surrogate-gradient mechanism for Transformer-based SNNs. SAGE estimates block-level uncertainty from normalized self-attention entropy and uses this signal to adapt the surrogate-gradient slope during training while leaving the inference model unchanged. By modulating only the training-time surrogate parameter, the proposed method preserves the original architecture and deployment cost while improving optimization flexibility. Experiments on CIFAR-10/100 demonstrate that SAGE achieves improved accuracy over fixed-surrogate baselines, with results up to 1-2% consistent gains across multiple simulation time steps. These results highlight the potential of attention-derived uncertainty as a lightweight training signal for adaptive surrogate-gradient learning in transformer-based SNNs.

Kiran Nair, Rodrigue Rizk, KC Santosh

USD Artificial Intelligence Research Lab Department of Computer Science University of South Dakota Vermillion, SD 57069, USA

Spiking Neural Networks Neural Network Optimization Spiking Transformers

Ask PDF

Oct 21, 2026

Spike-Aware Gradient-Based Mixed-Precision Quantization of Spiking Neural Networks for Vision Applications

In recent years, Spiking Neural Networks (SNNs) have gained massive attention as an energy-efficient neural computing paradigm due to their event-driven and sparse computing capability. However, maintaining accuracy under low-precision constraints, necessary for practical deployment, has been largely unexplored for SNN models. In addition, existing Hessian-based approaches for quantization, such as HAWQ-V2, incur significant memory and computational overhead due to temporal unrolling across multiple SNN timesteps. We propose a spike-aware gradient-based mixed-precision quantization framework for SNNs that adaptively assigns bit-widths using first-order gradient information, avoiding costly second-order Hessian computations. We evaluate our method across various SNN-based architectures, including VGG, ResNet, and Vision Transformer models obtained from both ANN-to-SNN conversion and direct training frameworks, on CIFAR10/100, ImageNet, and DVS-CIFAR-10 datasets. Notably, on ResNet-34 evaluated on ImageNet, our method achieves 73.30% accuracy at 4.78

\times

compression, which is comparable to that provided by HAWQ-V2, while reducing memory consumption by 2.31

\times

and bit-allocation runtime by 100

\times

. We also present an energy analysis for mixed-precision SNNs showing that the proposed quantization strategy enables energy-efficient deployment without compromising performance.

Rajshekhar Rakshit, Devgokul B., Rakshitha Kalkura +1

Indian Institute Of Technology Madras, Chennai, India

Mixed-Precision Quantization Spiking Neural Networks Neural Network Compression

Ask PDF

Abstract

Spiking Neural Networks

Explore similar work

Sharpness Aware Surrogate Training for Spiking Neural Networks

SAGE: Surrogate-gradient Adaptation via Attention-Guided Entropy for Spiking Transformers

Spike-Aware Gradient-Based Mixed-Precision Quantization of Spiking Neural Networks for Vision Applications