Quantifying How Training Gradient Sparsity Affect Spiking Neural Network Accuracy And Robustness

Nhan Trong Luu; Duong Trung Luu

cs.NESep 28, 2025

Quantifying How Training Gradient Sparsity Affect Spiking Neural Network Accuracy And Robustness

Authors: Nhan Trong Luu, Duong Trung Luu

Organizations: College of Information and Communication Technology, Can Tho University, Vietnam · College of Engineering and Computer Science, VinUniversity, Hanoi, Vietnam · Center of Digital Transformation and Communication, Can Tho University, Vietnam

Abstract

Spiking Neural Networks (SNNs) have recently received increasing attention in both computational neuroscience and artificial intelligence owing to their potential for energy-efficient computation and reduced memory requirements. Despite these advantages, improving adversarial robustness in SNNs (particularly for vision-based applications) remains an emerging and relatively underexplored research problem. Recent work has suggested that encouraging sparse gradients can act as a regularization mechanism to improve resistance against adversarial perturbations. In this study, we report an unexpected observation: under certain architectural configurations, SNNs inherently exhibit sparse gradients and can attain state-of-the-art adversarial defense performance without requiring any explicit regularization strategy. Further investigation reveals an inherent trade-off between robustness and generalization. Specifically, increased gradient sparsity enhances resistance to adversarial attacks but may reduce the model's generalization capability, whereas denser gradients tend to improve generalization while simultaneously increasing susceptibility to adversarial perturbations. These findings provide new perspectives on the role of gradient sparsity in the training dynamics of SNNs.

Figures & tables

Table 1 : Comparison of validation accuracy of undefended SNN variants with various SNN adversarial defense methods. Best performance per attack are denoted in bold. Notably, undefended SEW-ResNet achieve SOTA performance against PGD attacks and reasonable performance against FGSM attacks.

Panel of Figure 2 — Fig. 1 : Visualization of mean input gradient sparsity and gradient norm across the different operations within SEW-ResNet18.

Attack	CIFAR-10			CIFAR-100			CIFAR10-DVS
Attack	SEW-ResNet18	SEW-ResNet34	SEW-ResNet50	SEW-ResNet18	SEW-ResNet34	SEW-ResNet50	SEW-ResNet18	SEW-ResNet34	SEW-ResNet50
FGSM	45.71 $\pm$ 0.257 \textcolor red( $\downarrow$ 1.28)	48.72 $\pm$ 0.581 \textcolor red( $\downarrow$ 0.48)	49.88 $\pm$ 0.374 \textcolor red( $\downarrow$ 1.21)	17.93 $\pm$ 0.364 \textcolor red( $\downarrow$ 2.88)	20.07 $\pm$ 0.425 \textcolor red( $\downarrow$ 0.16)	22.02 $\pm$ 0.476 \textcolor red( $\downarrow$ 0.74)	46.27 $\pm$ 0.377 \textcolor red( $\downarrow$ 0.92)	47.66 $\pm$ 0.317 \textcolor red( $\downarrow$ 1.37)	47.01 $\pm$ 0.258 \textcolor red( $\downarrow$ 1.51)
PGD	25.72 $\pm$ 0.395 \textcolor red( $\downarrow$ 8.61)	29.82 $\pm$ 0.624 \textcolor red( $\downarrow$ 7.16)	33.29 $\pm$ 0.273 \textcolor red( $\downarrow$ 7.26)	8.65 $\pm$ 0.219 \textcolor red( $\downarrow$ 4.82)	10.37 $\pm$ 0.388 \textcolor red( $\downarrow$ 2.88)	13.37 $\pm$ 0.451 \textcolor red( $\downarrow$ 2.66)	13.94 $\pm$ 0.276 \textcolor red( $\downarrow$ 7.53)	12.54 $\pm$ 0.402 \textcolor red( $\downarrow$ 6.11)	14.30 $\pm$ 0.442 \textcolor red( $\downarrow$ 8.04)
Clean	75.76 $\pm$ 0.286 \textcolor green( $\uparrow$ 1.98)	76.17 $\pm$ 0.398 \textcolor green( $\uparrow$ 2.01)	75.28 $\pm$ 0.397 \textcolor green( $\uparrow$ 1.94)	43.32 $\pm$ 0.377 \textcolor green( $\uparrow$ 1.13)	44.58 $\pm$ 0.634 \textcolor green( $\uparrow$ 2.77)	43.40 $\pm$ 0.321 \textcolor green( $\uparrow$ 4.49)	77.77 $\pm$ 0.308 \textcolor green( $\uparrow$ 2.59)	76.43 $\pm$ 0.358 \textcolor green( $\uparrow$ 1.14)	78.35 $\pm$ 0.508 \textcolor green( $\uparrow$ 1.82)

Table 2 : Validation accuracy comparison on CIFAR-10 and CIFAR-100 for undefended SEW-ResNet variants [ 6 ] when average pooling is employed. The highest accuracy for each attack scenario on a given dataset is highlighted in bold. The adversarial attack configurations follow the same settings used in Table 1(a) . Accuracy decreases relative to Table 1(b) are indicated in red, while improvements are shown in green. The results reveal a clear trade-off between clean accuracy and adversarial robustness.

Fig. 2 : Backpropagation structure corresponding to Equation 4 . Red arrows represent gradient components shared across different derivative paths, whereas blue arrows indicate gradients that are specific to individual variables. Sparsity arising along shared gradient paths propagates to both the input and trainable parameters.

Explore similar work

Burst Spiking Neural Networks
Jul 5, 2026Jiahong Zhang, Sijun Shen, Man Yao +5Spiking Neural Networks Neural Network
Institute of Automation, Chinese Academy of Sciences, Beijing 100045, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China · State Key Laboratory of Media Convergence and Communication, Communication University of China, Beijing 100024, China +3
Controllable Stochastic Quantization Encoding for Adversarially Robust Spiking Neural Networks
Oct 1, 2026Yujia Liu, Peiyu Liu, Yajing Zheng +1Spiking Neural Networks Neural Network
NERCVT, School of Computer Science, Peking University, China · State Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University, China · Institute for Artificial Intelligence, Peking University, China
A2SG:Adaptive and Asymmetric Surrogate Gradients for Training Deep Spiking Neural Networks
May 29, 2026Yechan Kang, Yongjin Kweon, Mingyeong Seo +8Spiking Neural Networks Surrogate Gradient
Center for Semiconductor Technology, Korea Institute of Science and Technology (KIST), Seoul, Republic of Korea · Department of Computer Science and Engineering, Korea University, Seoul, Republic of Korea · LIG Defense&Aerospace, Seongnam, Republic of Korea +1

Jul 5, 2026, cs.NEJ/K move · Enter open · S save

Burst Spiking Neural Networks

Jiahong Zhang, Sijun Shen, Man Yao, Han Xu+4

Institute of Automation, Chinese Academy of Sciences, Beijing 100045, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China · State Key Laboratory of Media Convergence and Communication, Communication University of China, Beijing 100024, China+3

Spiking Neural Networks Neural Network

Open paper AskPDF arXiv

A central goal of current Spiking Neural Network (SNN) research is to improve their accuracy toward becoming low-power alternatives to Artificial Neural Networks (ANNs). This work further argues that realizing this ambition requires improving not only accuracy but also robustness, defined as the ability to maintain correct predictions under input perturbations. We identify two key issues in existing SNN methods that undermine robustness. First, binary spiking activations can produce large activation-state changes under small perturbations. Second, the lack of effective weight constraints makes network outputs more sensitive to input variations. To this end, we propose Burst Spiking Neural Networks (BuSNNs), built upon Burst-enhanced Spiking Neurons (BSNs) and a Dynamic Weight Constraint (DWC) mechanism. BSNs incorporate burst firing to provide a graded spiking pattern. This spiking mechanism mitigates perturbation-induced transitions in activation states and thereby enhances robustness. DWC penalizes connection weights based on activation states, effectively reducing weight magnitudes and improving robustness while preserving accuracy. We provide theoretical analyses to support these robustness effects. Experimental results further show that, on smaller-scale benchmarks such as CIFAR-10, BuSNNs outperform both SNN and ANN counterparts in accuracy and robustness. On large-scale ImageNet, BuSNN with the MS ResNet-34 backbone further improves top-1 accuracy and corruption robustness over the corresponding SNN baseline by 3.18% and 2.66%, respectively. Despite using spike-based activations, BuSNNs surpass 4-bit activation-quantized ANN baselines and approach 8-bit ANN baselines on ImageNet. They also preserve SNNs' low-power advantage. This work studies the accuracy-robustness problem in SNNs, advancing their practical viability in robust and energy-efficient applications.

Abstract

Explore similar work

Burst Spiking Neural Networks

Controllable Stochastic Quantization Encoding for Adversarially Robust Spiking Neural Networks

A2SG:Adaptive and Asymmetric Surrogate Gradients for Training Deep Spiking Neural Networks