Spiking Layer-Adaptive Magnitude-based Pruning

Junqiao Wang; Zhehang Ye; Yuqi Ouyang

cs.LGMar 16, 2026

Spiking Layer-Adaptive Magnitude-based Pruning

Authors: Junqiao Wang, Zhehang Ye, Yuqi Ouyang

Organizations: Sichuan University-Pittsburgh Institute, Sichuan University, Chengdu, 610207, China · College of Computer Science, Sichuan University, Chengdu, 610065, China

Abstract

Spiking Neural Networks (SNNs) provide energy-efficient computation but their deployment is constrained by dense connectivity and high spiking operation costs. Existing magnitude-based pruning strategies, when naively applied to SNNs, fail to account for temporal accumulation, non-uniform timestep contributions, and membrane stability, often leading to severe performance degradation. This paper proposes Spiking Layer-Adaptive Magnitude-based Pruning (SLAMP), a theory-guided pruning framework that generalizes layer-adaptive magnitude pruning to temporal SNNs by explicitly controlling worst-case output distortion across layers and timesteps. SLAMP formulates sparsity allocation as a temporal distortion-constrained optimization problem, yielding time-aware layer importance scores that reduce to conventional layer-adaptive pruning in single-timestep limit. An efficient two-stage procedure is derived, combining temporal score estimation, global sparsity allocation, and magnitude pruning with retraining for stability recovery. Experiments on CIFAR10, CIFAR100, and the event-based CIFAR10-DVS datasets demonstrate that SLAMP achieves substantial connectivity and spiking operation reductions while preserving accuracy, enabling efficient and deployable SNN inference.

Figures & tables

Panel of Figure 1 — Fig. 1: Structures of the two SNNs studied in this work.

Fig. 2: A complete illustration of the pruning performance across the three datasets, including pruned model density (histograms) and resulted Top-1 accuracy (red curve) and Top-5 accuracy (green curve).

Dataset	Pruning Method	Architecture	T	Acc. Diff(%)	Top-1 Acc.(%)	Conn.(%)	Param. (M)	SOPs (M)
CIFAR10	ADMM [ 14 ]	7 Conv, 2 FC	8	-0.13	90.19	25.03	15.54	–
	Grad R [ 15 ]	6 Conv, 2 FC	8	-0.30	92.54	36.72	10.43	–
	ESLSNN [ 16 ]	ResNet19	2	-1.70	91.09	50.00	6.30	180.56
	STDS [ 17 ]	6 Conv, 2 FC	8	-0.35	92.49	11.33	1.71	147.22
	UPR [ 18 ]	6 Conv, 2 FC	8	-0.79	92.05	1.16	9.56	16.47
	SLAMP (Ours)	ResNet19	2	+1.23	92.42	40.00	4.47	112.35

TABLE I: Performance comparisons with previous baseline methods on the three datasets, where Acc. Diff denotes the TOP-1 accuracy difference between pruned and original SNN.

Dataset	Connectivity	Accuracy (%)		Membrane Variance
Dataset	Connectivity	With	Without	With	Without
CIFAR10	70%	92.10	88.32	0.081	0.153
	40%	92.42	84.17	0.092	0.221
	25%	91.20	76.45	0.105	0.298
	10%	88.40	63.28	0.134	0.412
CIFAR100	70%	69.82	65.14	0.095	0.182

TABLE II: Accuracy and membrane potential variance with and without fine-tuning, across different datasets and connectivity.

Hyperparameter	Value	CIFAR10		CIFAR10-DVS		Sparsity
Hyperparameter	Value	Accuracy	$\Delta$ Acc	Accuracy	$\Delta$ Acc	Deviation
Pruning Freq $f$	10	92.15%	-0.27	77.92%	-0.29	1.8%
	15	92.42%	0.00	78.21%	0.00	0.9%
	25	92.38%	-0.04	78.15%	-0.06	1.2%
Learning Rate $\eta_{0}$	0.005	91.25%	-1.17	76.84%	-1.37	2.1%
	0.01	92.03%	-0.39	78.21%	0.00	1.3%

TABLE III: Hyperparameter analysis of pruning frequency and learning rate across datasets.

Spiking Neural Networks

Spiking neural networks process information through discrete, time-dependent neural events, aiming for efficient computation and biologically inspired learning.

See all 336 papers

Criticality-Constrained Iterative Pruning for Energy-Efficient Spiking Neural Networks via Combined Importance Scoring
Jun 26, 2026Muhammad HamzaEfficient Neural Network Inference Spiking Neural Networks
Indian Institute of Technology Kharagpur, West Bengal 721302, India
Activation-Energy Pruning for Spiking Neural Networks: Unsupervised Personalization via Spike-Count Saliency
Aug 11, 2026Joseph BinghamNeural Network Pruning Spiking Neural Networks
Technion – Israel Institute of Technology, Haifa, Israel
BASC : Behavior-Aligned Quantization and Pruning for Low-Bit Spiking Neural Networks
Aug 12, 2026Linliang Chen, Yan Zhong, Xin Liu +2Spiking Neural Networks Neural Network Quantization
Hangzhou International Innovation Institute, Beihang University, Hangzhou 311115, China · School of Mathematical Sciences, Peking University, Beijing 100871, China · School of Electronic and Information Engineering; Beihang University, Beijing 100191, China +1

Jun 26, 2026J/K move · Enter open · S save

Criticality-Constrained Iterative Pruning for Energy-Efficient Spiking Neural Networks via Combined Importance Scoring

Muhammad Hamza

Indian Institute of Technology Kharagpur, West Bengal 721302, India

Efficient Neural Network Inference Spiking Neural Networks Neuromorphic Computing Neural Network Pruning

Open paper AskPDF arXiv

Deploying spiking neural networks (SNNs) on neuromorphic hardware demands aggressive synaptic pruning while preserving temporal computation integrity. Existing strategies either neglect neuronal criticality or rely on convex relaxations of the inherently combinatorial pruning problem whose fractional masks, upon binarisation, destroy accuracy at moderate-to-high sparsity. We present Criticality-Constrained Quadratic Pruning (CQP), a native PyTorch pipeline that fuses weight magnitude with surrogate-gradient criticality into an analytically exact importance metric, eliminating the rounding artefacts endemic to solver-based approaches. We formally characterise a continuous-relaxation trap wherein OSQP-solver fractional masks overshoot the intended sparsity by up to 12 percentage points (pp), precipitating a 44 pp accuracy collapse. We identify and remediate a zombie-weight failure mode in which Adam's first-moment tensors resurrect pruned synapses, violating the binary sparsity guarantee. An iterative schedule - prune, fine-tune with gradient masking, recompute criticality, and repeat - eliminates gradient staleness at high sparsity. A KL-divergence temporal analysis identifies a redundant simulation timestep, enabling a free 10% theoretical energy reduction without weight modification. On MNIST (60,000 training examples), CQP yields 95.6% accuracy at 90% sparsity versus 93.4% for magnitude pruning (+2.2 pp). A criticality-threshold sweep reveals an empirical criticality cliff: accuracy falls from 87.0% to 14.4% as the threshold reaches tau = 0.9, constituting a quantitative SNN-level analogue of the Critical Brain Hypothesis. Combined weight sparsification and temporal truncation yield a compound 73% reduction in per-inference energy at 70% sparsity, confirming the practical value of the proposed pipeline for neuromorphic deployment.

Abstract

Spiking Neural Networks

Explore similar work

Criticality-Constrained Iterative Pruning for Energy-Efficient Spiking Neural Networks via Combined Importance Scoring

Activation-Energy Pruning for Spiking Neural Networks: Unsupervised Personalization via Spike-Count Saliency

BASC : Behavior-Aligned Quantization and Pruning for Low-Bit Spiking Neural Networks