Activation-Energy Pruning for Spiking Neural Networks: Unsupervised Personalization via Spike-Count Saliency

Joseph Bingham

cs.LGAug 11, 2026

Activation-Energy Pruning for Spiking Neural Networks: Unsupervised Personalization via Spike-Count Saliency

Authors: Joseph Bingham

Organizations: Technion – Israel Institute of Technology, Haifa, Israel

Abstract

Activation-energy pruning -- removing weights whose product of magnitude and cumulative pre-synaptic spike count falls below a threshold -- was established as an effective unsupervised personalization strategy for conventional deep neural networks~\citep{BINGHAM2025101242}. This paper asks what happens when the same criterion is applied to spiking neural networks (SNNs), where activation energy is not merely a useful heuristic but a literal physical quantity proportional to the metabolic cost of each synapse. The answer is surprising on three counts. First, gradient-based pruning methods that perform competitively on conventional networks (SNIP, GraSP, magnitude pruning) consistently underperform on SNNs, collapsing to near-chance accuracy by $σ= 0.2$ sparsity across all tested architectures and datasets. We trace this to a systematic incompatibility between surrogate-gradient saliency estimation and the binary spike-train representation, though we cannot rule out that alternative surrogate choices or hyperparameter settings might partially mitigate the effect. Second, activation-energy pruning applied to a neuromorphic benchmark improves over the source model at high sparsity ( $98.4 \pm 0.4\%$ vs.\ $97.2 \pm 0.7\%$ at $σ= 0.8$ on N-MNIST), a phenomenon with no counterpart in the conventional network setting. We interpret this result as consistent with experience-dependent cortical specialisation: removing connections active only for non-target classes may reduce cross-class interference and produce a cleaner target representation, though we note this is an interpretive analogy rather than a mechanistic demonstration.

Figures & tables

Figure 1: The FP-SNN pipeline. A pre-trained source SNN and an unlabeled target stream are the only inputs. A single forward pass accumulates activation energy $E[w_{ij}]=|w_{ij}|\cdot\sum_{t,x}s_{i}(x,t)$ per weight. Global or per-layer thresholding then removes the lowest-energy fraction $\sigma$ of weights. An optional BatchNorm recalibration step (recommended for $\sigma\geq 0.5$ ) resets running statistics using the same unlabeled stream. No labels are consumed at any stage.

Figure 2: Accuracy vs. sparsity for FP-SNN Global, FP-SNN Per-layer+BN, and Random+BN (best baseline) across all three datasets. Dashed horizontal lines mark unpruned source accuracy. Error bars show $\pm$ 1 standard deviation over 5 seeds (CIFAR-100 and N-MNIST) or 5 seeds with undertrained sources (Tiny-ImageNet; see Appendix B ). FP-SNN Global matches source accuracy up to $\sigma=0.5$ on CIFAR-100 and maintains near-source accuracy up to $\sigma=0.8$ on N-MNIST, while all baselines collapse well before $\sigma=0.5$ on both primary benchmarks.

	Sparsity $\sigma$
Method	0.1	0.2	0.3	0.4	0.5	0.6
FP-SNN global	65.0 $\pm$ 1.0	64.8 $\pm$ 0.9	64.9 $\pm$ 1.1	64.7 $\pm$ 1.3	63.6 $\pm$ 1.6	51.8 $\pm$ 2.5
FP-SNN per-layer+BN	51.8 $\pm$ 2.3	51.5 $\pm$ 3.1	49.6 $\pm$ 3.0	47.8 $\pm$ 3.2	43.6 $\pm$ 3.8	38.6 $\pm$ 2.4
Random+BN	43.9 $\pm$ 3.5	37.3 $\pm$ 3.1	28.5 $\pm$ 1.2	19.4 $\pm$ 2.0	9.4 $\pm$ 1.6	4.2 $\pm$ 1.3
Magnitude+BN	4.7 $\pm$ 2.1	5.0 $\pm$ 0.0	5.0 $\pm$ 0.0	5.0 $\pm$ 0.0	5.0 $\pm$ 0.0	0.0 $\pm$ 0.0
SNIP+BN	49.9 $\pm$ 1.4	0.0 $\pm$ 0.0	5.0 $\pm$ 0.0	0.0 $\pm$ 0.0	0.0 $\pm$ 0.0	0.0 $\pm$ 0.0

Table 1: Top-1 accuracy (%, mean

\pm

std over 5 seeds) on the CIFAR-100 target split (20 classes) as a function of pruning sparsity

\sigma

. FP-SNN (global) and FP-SNN (per-layer + BN recalib.) are our methods. All baselines include BN recalibration (+BN). Source accuracy on the same 20-class split is

65.0\pm 0.9\%

. Best result at each sparsity in bold .

Figure 3: Accuracy vs. sparsity on CIFAR-100 (Wide VGG-SNN, 37.7M parameters, 20-class target split, 5 seeds). FP-SNN Global maintains source-level accuracy from $\sigma=0.1$ to $\sigma=0.5$ , while all gradient-based baselines (Magnitude, SNIP, GraSP) collapse to near chance by $\sigma=0.2$ . The per-layer+BN variant becomes superior only above $\sigma=0.7$ .

	$\sigma=0.3$	$\sigma=0.4$	$\sigma=0.5$	$\sigma=0.6$	$\sigma=0.7$
FP-SNN global	64.9	64.7	63.6	51.8	12.7
FP-SNN global +BN	49.7	50.7	50.3	43.6	27.1
$\Delta$	$-15.2$	$-14.0$	$-13.3$	$-8.2$	$+14.4$

Table 2: Effect of BatchNorm recalibration (+BN) on FP-SNN (global scope) at selected sparsities on CIFAR-100. Mean top-1 accuracy over 5 seeds.

Figure 4: Effect of BatchNorm recalibration on CIFAR-100 (FP-SNN Global scope). Left : accuracy curves with and without BN recalibration. Right : per-sparsity accuracy delta ( $+$ BN minus no BN). Green bars indicate recalibration helps; red bars indicate it hurts. The crossover from harmful to beneficial occurs at $\sigma\approx 0.4$ , reflecting the point at which pruning-induced activation-distribution shift outweighs estimation noise from the small unlabeled target stream.

	Sparsity $\sigma$
Method	0.1	0.2	0.3	0.4	0.5	0.6
FP-SNN global	97.2 $\pm$ 0.7	97.2 $\pm$ 0.7	97.2 $\pm$ 0.7	97.3 $\pm$ 0.6	97.4 $\pm$ 0.6	97.6 $\pm$ 0.5
FP-SNN per-layer+BN	96.1 $\pm$ 1.0	96.3 $\pm$ 0.8	96.0 $\pm$ 1.1	95.7 $\pm$ 1.4	94.2 $\pm$ 3.1	87.6 $\pm$ 7.3
Random+BN	89.7 $\pm$ 7.1	91.0 $\pm$ 3.6	77.2 $\pm$ 10.2	67.0 $\pm$ 9.0	53.8 $\pm$ 16.0	26.0 $\pm$ 7.0
Magnitude+BN	5.0 $\pm$ 0.0	5.0 $\pm$ 0.0	5.0 $\pm$ 0.0	5.0 $\pm$ 0.0	5.0 $\pm$ 0.0	5.0 $\pm$ 0.0

Table 3: Top-1 accuracy (%, mean

\pm

std over 5 seeds) on the N-MNIST target split (3 classes). Source accuracy is

97.2\pm 0.7\%

. Best result at each sparsity in bold .

Panel of Figure 5 — Figure 5: N-MNIST (SCNN, 3-class target, 5 seeds). FP-SNN Global maintains near-source accuracy across all sparsities up to $\sigma=0.8$ , and uniquely improves over source accuracy ( $98.4\pm 0.4\%$ vs. $97.2\pm 0.7\%$ source) by recovering a target-specialized winning subnetwork.

	Sparsity $\sigma$
Criterion	0.1	0.2	0.3	0.4	0.5	0.6
$\|w_{ij}\|$ (magnitude)	6.5 $\pm$ 3.4	3.0 $\pm$ 2.7	3.0 $\pm$ 2.7	3.0 $\pm$ 2.7	2.0 $\pm$ 2.7	2.0 $\pm$ 2.7
$\sum_{t}s_{i}$ (spike-count)	65.1 $\pm$ 0.9	65.0 $\pm$ 1.3	64.2 $\pm$ 1.9	52.8 $\pm$ 2.2	1.4 $\pm$ 1.2	0.0 $\pm$ 0.0
$\|w_{ij}\|\cdot\sum_{t}s_{i}$ (FP-SNN)	65.0 $\pm$ 1.1	64.8 $\pm$ 1.0	64.9 $\pm$ 1.3	64.7 $\pm$ 1.5	63.6 $\pm$ 1.8	51.8 $\pm$ 2.8

Table 4: Criterion ablation on CIFAR-100 (5 seeds, global scope, no BN recalib.). Source:

65.0\pm 0.9\%

. Best result at each sparsity in bold .

	Sparsity $\sigma$
Criterion	0.1	0.2	0.3	0.4	0.5	0.6
$\|w_{ij}\|$ (magnitude)	96.2 $\pm$ 1.1	96.1 $\pm$ 1.0	95.6 $\pm$ 1.5	96.4 $\pm$ 1.0	96.0 $\pm$ 1.1	96.1 $\pm$ 1.5
$\sum_{t}s_{i}$ (spike-count)	97.2 $\pm$ 0.7	97.2 $\pm$ 0.7	97.3 $\pm$ 0.7	97.5 $\pm$ 0.5	97.8 $\pm$ 0.5	98.0 $\pm$ 0.5
$\|w_{ij}\|\cdot\sum_{t}s_{i}$ (FP-SNN)	97.2 $\pm$ 0.7	97.2 $\pm$ 0.7	97.2 $\pm$ 0.7	97.3 $\pm$ 0.7	97.4 $\pm$ 0.6	97.6 $\pm$ 0.6

Table 5: Criterion ablation on N-MNIST (5 seeds, global scope, no BN recalib.). Source:

97.2\pm 0.7\%

. Best result at each sparsity in bold .

Panel of Figure 6 — Figure 6: Criterion ablation: magnitude-only ( $|w_{ij}|$ ), spike-count-only ( $\sum_{t}s_{i}$ ), and the full activation-energy product ( $|w_{ij}|\cdot\sum_{t}s_{i}$ , FP-SNN). All use global scope and no BN recalibration. Error bars: $\pm$ 1 std over 5 seeds.

Panel of Figure 7 — Figure 7: Per-layer weight retention fraction for spike-count-only pruning. Dashed line: uniform target $(1-\sigma)$ . Non-uniform distributions indicate that global spike-count thresholding implicitly concentrates pruning in particular layers.

Figure 8: Method comparison at $\sigma=0.5$ sparsity across all three datasets (5 seeds each). FP-SNN Global (blue) and FP-SNN Per-layer+BN (purple) dominate on the two primary benchmarks (CIFAR-100 and N-MNIST) by large margins. All gradient-based baselines (Magnitude, SNIP, GraSP) remain near chance on both primary datasets. Tiny-ImageNet results use undertrained source models and are included for indicative purposes only; full results are in Appendix B . Error bars show $\pm$ 1 std over 5 seeds.

Sparsity $\sigma$	0.0	0.1	0.3	0.5	0.7	0.9
Energy (nJ)	77.4	76.5	75.1	72.8	64.3	43.4
Reduction (%)	—	1.2	3.0	5.9	16.9	43.9

Table 6: Inference energy (nJ/sample) for FP-SNN (global scope) on CIFAR-100 at selected sparsities, estimated using the 45 nm CMOS AC model (0.9 pJ/operation). Source model: 77.4 nJ/sample.

Figure 9: Accuracy–energy Pareto frontier on N-MNIST. FP-SNN Global (solid blue) achieves high accuracy ( $\geq 97\%$ ) only at high energy (dense model), then dramatically reduces energy at $\sigma=0.8$ while increasing accuracy to $98.4\%$ . FP-SNN Per-layer+BN (dashed purple) traces a smoother frontier, trading accuracy for energy more gradually. All baselines occupy the lower-left region (low accuracy, low-to-moderate energy), confirming that they do not achieve a favourable accuracy–energy tradeoff at any operating point.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 10: Per-layer weight retention for magnitude-only (red), spike-count-only (orange), and activation-energy / FP-SNN (blue) pruning at $\sigma=0.1$ , $0.3$ , and $0.6$ on CIFAR-100 (seed 0; VGG-SNN, 10 prunable layers L1–L10). Dashed line: uniform target retention $(1-\sigma)$ . At $\sigma=0.1$ , magnitude-only retains nearly 100% of weights in L1–L3 but near 0% in L7–L10, decapitating the deep classifier layers while leaving early feature-detectors intact. This layer-decapitation pattern explains the immediate accuracy collapse seen in Table 4 : the network loses its task-discriminative deep representation before losing any early-layer feature capacity. Spike-count-only and activation-energy pruning distribute retention uniformly near the target at $\sigma=0.1$ ; their distributions diverge from each other only at $\sigma=0.6$ , where the weight-magnitude factor in activation-energy begins to further differentiate among active neurons.

	Sparsity $\sigma$
Method	0.1	0.2	0.3	0.4	0.5	0.6
FP-SNN global	11.7 $\pm$ 3.1	11.7 $\pm$ 3.1	11.6 $\pm$ 2.9	11.5 $\pm$ 3.1	10.3 $\pm$ 3.4	5.8 $\pm$ 2.7
FP-SNN per-layer+BN	10.7 $\pm$ 2.8	10.4 $\pm$ 2.7	10.2 $\pm$ 2.4	10.6 $\pm$ 2.4	9.9 $\pm$ 2.4	10.1 $\pm$ 2.4
SNIP+BN	12.4 $\pm$ 2.5	12.4 $\pm$ 2.5	11.7 $\pm$ 2.2	1.4 $\pm$ 0.6	0.8 $\pm$ 1.4	0.8 $\pm$ 1.4
Magnitude+BN	10.0 $\pm$ 2.1	3.9 $\pm$ 1.8	2.9 $\pm$ 1.5	2.4 $\pm$ 0.5	1.5 $\pm$ 1.1	1.2 $\pm$ 1.4
Random+BN	8.9 $\pm$ 1.8	8.2 $\pm$ 2.5	6.3 $\pm$ 2.1	3.9 $\pm$ 2.7	3.0 $\pm$ 1.2	1.8 $\pm$ 1.8

Appendix

Table 7: Tiny-ImageNet preliminary results (ResNet-19 SNN, 40-class target, 5 seeds, undertrained source models at

11.7\pm 3.1\%

; chance: 2.5%). All comparisons are relative to the

11.7\%

source, not to literature benchmarks.

Figure 11: Accuracy vs. sparsity on Tiny-ImageNet (ResNet-19 SNN, 40-class target, 5 seeds, undertrained source). All comparisons relative to source. The qualitative pattern – FP-SNN Global flat to $\sigma=0.5$ , cliff at $\sigma=0.6$ , Per-layer+BN dominant above $\sigma=0.6$ , all baselines below chance by $\sigma=0.5$ – is consistent with the two primary benchmarks.

Neural Network Pruning

Neural network pruning removes weights, neurons, channels, heads, or layers to reduce model size and computation while retaining useful behavior.

See all 282 papers

Criticality-Constrained Iterative Pruning for Energy-Efficient Spiking Neural Networks via Combined Importance Scoring
Jun 26, 2026Muhammad HamzaEfficient Neural Network Inference Spiking Neural Networks
Indian Institute of Technology Kharagpur, West Bengal 721302, India
Spiking Layer-Adaptive Magnitude-based Pruning
Mar 16, 2026Junqiao Wang, Zhehang Ye, Yuqi OuyangSpiking Neural Networks Neural Network Pruning
Sichuan University-Pittsburgh Institute, Sichuan University, Chengdu, 610207, China · College of Computer Science, Sichuan University, Chengdu, 610065, China
The Sparsity Ceiling: Where Spiking Networks Can and Cannot Trade Activity for Energy
Jul 29, 2026Zeyu WangEfficient Neural Network Inference Spiking Transformers
Georgia Institute of Technology

Jun 26, 2026J/K move · Enter open · S save

Criticality-Constrained Iterative Pruning for Energy-Efficient Spiking Neural Networks via Combined Importance Scoring

Muhammad Hamza

Indian Institute of Technology Kharagpur, West Bengal 721302, India

Efficient Neural Network Inference Spiking Neural Networks Neuromorphic Computing Neural Network Pruning

Open paper AskPDF arXiv

Deploying spiking neural networks (SNNs) on neuromorphic hardware demands aggressive synaptic pruning while preserving temporal computation integrity. Existing strategies either neglect neuronal criticality or rely on convex relaxations of the inherently combinatorial pruning problem whose fractional masks, upon binarisation, destroy accuracy at moderate-to-high sparsity. We present Criticality-Constrained Quadratic Pruning (CQP), a native PyTorch pipeline that fuses weight magnitude with surrogate-gradient criticality into an analytically exact importance metric, eliminating the rounding artefacts endemic to solver-based approaches. We formally characterise a continuous-relaxation trap wherein OSQP-solver fractional masks overshoot the intended sparsity by up to 12 percentage points (pp), precipitating a 44 pp accuracy collapse. We identify and remediate a zombie-weight failure mode in which Adam's first-moment tensors resurrect pruned synapses, violating the binary sparsity guarantee. An iterative schedule - prune, fine-tune with gradient masking, recompute criticality, and repeat - eliminates gradient staleness at high sparsity. A KL-divergence temporal analysis identifies a redundant simulation timestep, enabling a free 10% theoretical energy reduction without weight modification. On MNIST (60,000 training examples), CQP yields 95.6% accuracy at 90% sparsity versus 93.4% for magnitude pruning (+2.2 pp). A criticality-threshold sweep reveals an empirical criticality cliff: accuracy falls from 87.0% to 14.4% as the threshold reaches tau = 0.9, constituting a quantitative SNN-level analogue of the Critical Brain Hypothesis. Combined weight sparsification and temporal truncation yield a compound 73% reduction in per-inference energy at 70% sparsity, confirming the practical value of the proposed pipeline for neuromorphic deployment.

Abstract

Neural Network Pruning

Explore similar work

Criticality-Constrained Iterative Pruning for Energy-Efficient Spiking Neural Networks via Combined Importance Scoring

Spiking Layer-Adaptive Magnitude-based Pruning

The Sparsity Ceiling: Where Spiking Networks Can and Cannot Trade Activity for Energy