The Dormant Spiking Neuron: A State-Driven Mechanism for Efficient Spiking Neural Networks

Ruichen Ma; Yicai Chen; Pujun Zhou; Yue Zuo; Chongyang Li; Shuang Liu; Shaogang Hu; Guanchao Qiao

cs.CVOct 15, 2026

The Dormant Spiking Neuron: A State-Driven Mechanism for Efficient Spiking Neural Networks

Authors: Ruichen Ma, Yicai Chen, Pujun Zhou, Yue Zuo, Chongyang Li, Shuang Liu, Shaogang Hu, Guanchao Qiao

Organizations: University of Electronic Science and Technology of China (UESTC) · Shenzhen Institute for Advanced Study, UESTC

Abstract

Spiking neural networks (SNNs) offer a compelling pathway toward energy-efficient artificial intelligence, fundamentally anchored in presynaptic input-driven sparsity. However, a substantial yet chronically overlooked source of computational overhead persists: the continuous subthreshold dynamics of postsynaptic neurons. Quantitative analysis reveals that up to 44% of neurons in an ImageNet-trained ResNet-18 model reside in a deeply inhibited state, imposing a persistent computational burden despite having a negligible probability of firing. To address this inefficiency, the Dormant Spiking Neuron (DSN) model is introduced. By incorporating a state-driven dynamic gate, the DSN allows a neuron to enter a dormant state when its membrane potential falls below a predefined negative threshold, dynamically pruning redundant computations at the instance level. Crucially, this dormancy mechanism is directly integrated into the neuronal dynamics and trained end-to-end via backpropagation through time using surrogate gradients, enabling the network to actively learn a robust dormancy policy that adaptively preserves critical information flow. Extensive experiments across static and neuromorphic datasets demonstrate that the DSN framework consistently reduces computational energy by 40% to over 70%. Notably, on the large-scale ImageNet dataset, the DSN outperforms state-of-the-art static pruning techniques at comparable reported cost reductions and simultaneously improves the corresponding baseline SNN accuracy by 1.34%. By establishing a trainable, state-driven sparsity paradigm orthogonal to conventional input-driven methods, the DSN unlocks synergistic efficiency gains, presenting an effective approach for advancing neuromorphic computing.

Figures & tables

Figure 1: The four-step dynamics of the Dormant Spiking Neuron (DSN). At each timestep $t$ , a DSN executes a sequence of operations. (1) Dormancy Gate Update: The neuron first checks whether its previous membrane potential $V_{n}^{t-1}$ has fallen below the dormancy threshold $D_{th}$ . If so, it enters a permanent dormant state, and all subsequent stages are bypassed for the current and all future timesteps. (2) Potential Integration: If active ( $D_{n}^{t}=1$ ), the neuron integrates presynaptic inputs with its decayed membrane potential. (3) Gated Spike Firing: A spike $S_{n}^{t}$ is generated only if the neuron is active and its integrated potential $H_{n}^{t}$ surpasses the firing threshold $V_{th}$ . (4) Potential Update & Reset: The membrane potential $V_{n}^{t}$ is updated for the next timestep, incorporating a reset mechanism.

Panel of Figure 2 — Figure 2: Orthogonal sparsity mechanisms in the DSN model. (a) In dense ANNs, all operations are active. (b) Traditional SNNs exploit input-driven sparsity, wherein computations involving inactive input rows are skipped. (c) The dormancy mechanism introduces state-driven sparsity, wherein computations involving inactive output columns are skipped. (d) The DSN combines both mechanisms, yielding a multiplicative reduction in the number of active synaptic operations.

Dataset	Model	Architecture	T	GSOPs/ GFLOPs	Energy (mJ)	Acc. (%)
CIFAR10	ANN	ResNet-18	-	0.56	2.55	95.93
	Transformer	2-512	-	1.24	5.72	96.82
	MS-ResNet	ResNet-18	4	0.30	0.28	95.64
	SD-Transformer	2-512	4	0.51	0.46	95.60
	DSN (ITO)	ResNet-18	4	0.15	0.14	94.90
		2-512	4	0.31	0.30	94.29

Table 1: Main results on static and neuromorphic classification benchmarks. The DSN is evaluated under two paradigms: inference-time optimization (ITO) applied to pre-trained SNNs, and direct training with BPTT from scratch. Results show reductions in estimated energy ranging from 40% to over 70%, while maintaining accuracy comparable to or better than that of the corresponding baseline SNNs. Notably, the DSN outperforms state-of-the-art static pruning methods on ImageNet, delivering comparable reported cost reductions and improved accuracy.

Panel of Figure 3 — Figure 3: Ablation studies on DSN dynamics. (a) Impact of the dormancy threshold $D_{th}$ on accuracy and energy savings evaluated across three random seeds. (b) Energy savings of the DSN compared with the baseline SNN as a function of the number of simulation timesteps $T$ . (c) ITO on network representations. Layer-wise comparison of the SSIM and spike train accuracy (STA) metrics between a pre-trained baseline ResNet-18 SNN and the same network modified using the DSN ITO paradigm.

	CIFAR10	CIFAR100	ImageNet	DVS-Gesture	DVS-CIFAR10
Accuracy (%)	$95.30\pm 0.10$	$76.61\pm 0.13$	$64.36\pm 0.19$	$96.65\pm 0.40$	$77.83\pm 0.12$

Table 2: Three-seed accuracy results for the BPTT-trained DSN. Values are reported as mean

\pm

standard deviation over three runs.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

		Vanilla Spike-Driven Transformer	Spike-Driven Transformer with DSN
SPS	First Conv	$E_{mac}\cdot\text{MACs}_{Conv_{1}}$	$E_{mac}\cdot\text{MACs}_{Conv_{1}}$
SPS	Other Conv	$E_{ac}\cdot\text{MACs}_{Conv_{n}}\cdot\sum_{t=1}^{T}\overline{R_{n-1}^{t}}$	$E_{ac}\cdot\text{MACs}_{Conv_{n}}\cdot\sum_{t=1}^{T}(\overline{R_{n-1}^{t}}\cdot\overline{D_{n}^{t}})$
Self-Attention	$Q$	$E_{ac}\cdot ND^{2}\cdot\sum_{t=1}^{T}\overline{R_{in}^{t}}$	$E_{ac}\cdot ND^{2}\cdot\sum_{t=1}^{T}(\overline{R_{in}^{t}}\cdot\overline{D_{Q}^{t}})$
	$K$	$E_{ac}\cdot ND^{2}\cdot\sum_{t=1}^{T}\overline{R_{in}^{t}}$	$E_{ac}\cdot ND^{2}\cdot\sum_{t=1}^{T}(\overline{R_{in}^{t}}\cdot\overline{D_{K}^{t}})$
	$V$	$E_{ac}\cdot ND^{2}\cdot\sum_{t=1}^{T}\overline{R_{in}^{t}}$	$E_{ac}\cdot ND^{2}\cdot\sum_{t=1}^{T}(\overline{R_{in}^{t}}\cdot\overline{D_{V}^{t}})$
	$f(Q,K,V)$	$E_{ac}\cdot ND\cdot\sum_{t=1}^{T}\overline{R_{KV}^{t}}$	$E_{ac}\cdot ND\cdot\sum_{t=1}^{T}\overline{R_{KV}^{t}}$

Appendix

Table 3: Energy calculation example for a Spike-Driven Transformer architecture.

Panel of Figure 4 — Figure 4: (a) The cumulative distribution function (CDF) of membrane potentials in a ResNet-18 SNN trained on ImageNet reveals a tripartite split. A substantial fraction (44%) of neurons reside in the deeply inhibited region ( $V_{m}\leq 0$ ). (b) The Heaviside step function, whose derivative is zero almost everywhere, preventing gradient flow during backpropagation. (c) To enable gradient-based training, the true derivative is replaced by a smooth surrogate function $\sigma^{\prime}(\cdot)$ during the backward pass.

$D_{th}$	DVS-CIFAR10 (SD-Transformer)	DVS-Gesture (SD-Transformer)	ImageNet (ResNet-18)	CIFAR100 (SD-Transformer)
$-0.01$	$75.9\%$ ( $\downarrow 71\%$ )	$94.44\%$ ( $\downarrow 65\%$ )	$64.20\%$ ( $\downarrow 73\%$ )	$79.22\%$ ( $\downarrow 44\%$ )
$-0.05$	$76.6\%$ ( $\downarrow 70\%$ )	$95.48\%$ ( $\downarrow 63\%$ )	$64.44\%$ ( $\downarrow 73\%$ )	$79.36\%$ ( $\downarrow 43\%$ )
$-0.1$	$77.2\%$ ( $\downarrow 68\%$ )	$96.18\%$ ( $\downarrow 60\%$ )	$64.34\%$ ( $\downarrow 72\%$ )	$79.40\%$ ( $\downarrow 42\%$ )
$-0.5$	$77.9\%$ ( $\downarrow 52\%$ )	$96.88\%$ ( $\downarrow 42\%$ )	$64.50\%$ ( $\downarrow 54\%$ )	$79.50\%$ ( $\downarrow 32\%$ )
$-1$	$78.0\%$ ( $\downarrow 38\%$ )	$96.88\%$ ( $\downarrow 30\%$ )	$64.66\%$ ( $\downarrow 37\%$ )	$79.55\%$ ( $\downarrow 21\%$ )

Appendix

Table 4: Threshold-sweep results across datasets and architectures for BPTT-trained DSN models. Each entry reports accuracy (%) and estimated energy reduction.

Panel of Figure 5 — Figure 5: (a) Layer-wise firing rates, non-dormant rates, and the Neuronal Computational Utilization (NCU) on a ResNet-18 architecture. (b) Layer-wise Jaccard similarity of dormancy masks for a ResNet-34 model, showing their heterogeneity across inputs.

Dataset	Architecture	Learning Rate	Weight Decay	Batch Size	Epochs	Optimizer
CIFAR	MS-ResNet	2e-2	5e-4	64	300+10	SGD
CIFAR	SD-Transformer-2-512	3e-4	6e-2	64	300+10	AdamW
ImageNet	MS-ResNet	5e-2	2e-5	64	150+5	SGD
ImageNet	SD-Transformer-8-384	7.5e-4	1e-2	28	300+10	LAMB
DVS-Gesture	SD-Transformer-2-256	1e-4	6e-2	16	200+10	LAMB
DVS-CIFAR10	SD-Transformer-2-256	2e-3	1e-4	32	200+10	AdamW

Appendix

Table 5: Training hyperparameters. A summary of the key hyperparameters utilized across all datasets and network architectures.

Spiking Neural Networks

Spiking neural networks process information through discrete, time-dependent neural events, aiming for efficient computation and biologically inspired learning.

See all 351 papers

Criticality-Constrained Iterative Pruning for Energy-Efficient Spiking Neural Networks via Combined Importance Scoring
Jun 26, 2026Muhammad HamzaEfficient Neural Network Inference Spiking Neural Networks
Indian Institute of Technology Kharagpur, West Bengal 721302, India
Intrinsically Stable Spiking Neural Networks: Overcoming the Performance Barrier in the Absence of Batch Normalization
Jun 30, 2026Ruichen Ma, Xiaoyang Zhang, Jian Bai +5Spiking Neural Networks
University of Electronic Science and Technology of China (UESTC) · Beijing Institute of Remote-Sensing Equipment · Shenzhen Institute for Advanced Study, UESTC
SPARQ: Spiking Early-Exit Neural Networks for Energy-Efficient Edge AI
Mar 15, 2026Parth Patne, Mahdi Taheri, Ali Mahani +3Spiking Neural Networks Energy-Efficient ML
Brandenburg Technical University, Cottbus, Germany · Tallinn University of Technology, Tallinn, Estonia · Shahid Bahonar University of Kerman, Iran +2

Jun 26, 2026J/K move · Enter open · S save

Criticality-Constrained Iterative Pruning for Energy-Efficient Spiking Neural Networks via Combined Importance Scoring

Muhammad Hamza

Indian Institute of Technology Kharagpur, West Bengal 721302, India

Efficient Neural Network Inference Spiking Neural Networks Neuromorphic Computing Neural Network Pruning

Open paper AskPDF arXiv

Deploying spiking neural networks (SNNs) on neuromorphic hardware demands aggressive synaptic pruning while preserving temporal computation integrity. Existing strategies either neglect neuronal criticality or rely on convex relaxations of the inherently combinatorial pruning problem whose fractional masks, upon binarisation, destroy accuracy at moderate-to-high sparsity. We present Criticality-Constrained Quadratic Pruning (CQP), a native PyTorch pipeline that fuses weight magnitude with surrogate-gradient criticality into an analytically exact importance metric, eliminating the rounding artefacts endemic to solver-based approaches. We formally characterise a continuous-relaxation trap wherein OSQP-solver fractional masks overshoot the intended sparsity by up to 12 percentage points (pp), precipitating a 44 pp accuracy collapse. We identify and remediate a zombie-weight failure mode in which Adam's first-moment tensors resurrect pruned synapses, violating the binary sparsity guarantee. An iterative schedule - prune, fine-tune with gradient masking, recompute criticality, and repeat - eliminates gradient staleness at high sparsity. A KL-divergence temporal analysis identifies a redundant simulation timestep, enabling a free 10% theoretical energy reduction without weight modification. On MNIST (60,000 training examples), CQP yields 95.6% accuracy at 90% sparsity versus 93.4% for magnitude pruning (+2.2 pp). A criticality-threshold sweep reveals an empirical criticality cliff: accuracy falls from 87.0% to 14.4% as the threshold reaches tau = 0.9, constituting a quantitative SNN-level analogue of the Critical Brain Hypothesis. Combined weight sparsification and temporal truncation yield a compound 73% reduction in per-inference energy at 70% sparsity, confirming the practical value of the proposed pipeline for neuromorphic deployment.

Abstract

Spiking Neural Networks

Explore similar work

Criticality-Constrained Iterative Pruning for Energy-Efficient Spiking Neural Networks via Combined Importance Scoring

Intrinsically Stable Spiking Neural Networks: Overcoming the Performance Barrier in the Absence of Batch Normalization

SPARQ: Spiking Early-Exit Neural Networks for Energy-Efficient Edge AI