Robust Representation Learning for Spiking Neural Networks based on Conditional Information Bottleneck

Yuli Zha; Wang Fan; Yixiao Dai; Lingjie Wang; Bang Hu; Wei Cao; Fengzhe Zhang; Fan Zhang

cs.AIOct 20, 2026

Robust Representation Learning for Spiking Neural Networks based on Conditional Information Bottleneck

Authors: Yuli Zha, Wang Fan, Yixiao Dai, Lingjie Wang, Bang Hu, Wei Cao, Fengzhe Zhang, Fan Zhang

Organizations: College of Computer Science and Artificial Intelligence, Fudan University, Shanghai, 200433, China · Institute of Big Data, Fudan University, Shanghai, 200433, China

Abstract

Spiking Neural Networks (SNNs) have shown great potential for low-power neuromorphic computing due to their event-driven processing and spatiotemporal sparsity, but they remain vulnerable to input perturbations, hardware-intrinsic noise, and adversarial attacks in real-world deployment. We attribute this vulnerability partly to input-specific hidden representations learned by standard SNNs trained with conventional cross-entropy loss, as our information-plane analysis empirically shows that substantial input-related information can still be retained after label-predictive information becomes stable. To address this issue, we propose SCIB, a simple yet effective robust learning objective for SNNs based on the Conditional Information Bottleneck. Different from conventional variational IB methods, SCIB replaces the mutual information compression term $I(X;S)$ with the conditional mutual information $I(X;S|Y)$ and derives a tractable variational upper bound. Moreover, it introduces a learnable class-conditional prior instead of a label-agnostic isotropic Gaussian prior, improving the expressiveness of variational compression and encouraging class-structured representations when combined with the supervised classification objective. Systematic experiments on Fashion-MNIST, CIFAR-10, and CIFAR-100 under external environmental noise, neuromorphic hardware noise, and adversarial attacks show that SCIB improves robustness over standard SNNs and IB-based baselines under most evaluated noise conditions, improves adversarial accuracy under the evaluated attacks without using adversarial training, and reduces the SynOps-based single-sample inference energy consumption by 24.9% relative to an unconstrained SNN baseline.

Figures & tables

Panel of Figure 1 — Figure 1: Information-plane trajectories of standard SNNs and conventional IB-regularized SNNs on Fashion-MNIST and CIFAR-10. The first row shows the trajectories of standard SNNs, whereas the second row shows those obtained with conventional IB regularization.

Figure 2: Information diagram for the SCIB framework.

Figure 3: The overall architecture of the SCIB framework.

Noise Type	Noise Intensity	Dataset	Accuracy (%)
Noise Type	Noise Intensity	Dataset	w.o. IB	IB	NBP-SNN	SIB	QIB	SCIB
Salt-and-Pepper	NNI=0.1	Fashion-MNIST	92.96	93.09	92.85	92.94	93.08	93.28
		CIFAR-10	88.38	89.07	88.84	88.75	89.33	89.99
		CIFAR-100	61.52	62.32	62.84	63.27	62.92	62.91
	NNI=0.3	Fashion-MNIST	92.39	92.50	92.65	92.47	92.48	92.73
		CIFAR-10	85.59	86.61	86.01	86.88	86.31	87.13

Table 1: Classification accuracy (%) of all methods under 18 external environmental noise configurations. Boldface and underlining denote the best and second-best results in each row, respectively.

Panel of Figure 4 — Figure 4: Training convergence curves of all methods under membrane potential noise (MPN). Solid and dashed lines correspond to NNI $=1$ and NNI $=5$ , respectively.

Panel of Figure 5 — Figure 5: Training convergence curves of all methods under synaptic weight noise (SN). Solid and dashed lines correspond to NNI $=1$ and NNI $=5$ , respectively.

Timestep	Dataset	Method	Clean	$\epsilon=4/255$			$\epsilon=8/255$
Timestep	Dataset	Method	Clean	FGSM	PGD	BIM	FGSM	PGD	BIM
$T=4$	CIFAR-10	Vanilla	92.94	19.63	0.78	0.67	8.29	0.02	0.01
		FEEL-SNN	88.07	39.17	21.33	21.10	16.19	1.73	1.38
		SCIB	89.69	52.08	47.58	47.73	37.02	26.65	26.04
	CIFAR-100	Vanilla	73.80	6.99	0.38	0.36	3.64	0.01	0.00
		FEEL-SNN	63.08	19.51	9.08	8.74	8.27	0.94	0.76

Table 2: Classification accuracy (%) under adversarial attacks with perturbation budgets

\epsilon\in\{4/255,8/255\}

. SCIB does not employ adversarial training. Bold and underline denote the best and second-best results, respectively.

Layer	Neurons	Spikes	Rate	SynOps
Conv#1	65536	131859.7	2.0120	$1.8\times 10^{6}$ (M)
Conv#2	65536	142810.9	2.1791	$7.6\times 10^{7}$ (A)
Conv#3	32768	74278.7	2.2668	$4.1\times 10^{7}$ (A)
Conv#4	32768	69054.9	2.1074	$8.6\times 10^{7}$ (A)
Conv#5	16384	37739.2	2.3034	$4.0\times 10^{7}$ (A)
Conv#6	16384	40166.4	2.4516	$8.7\times 10^{7}$ (A)

Table 3: Per-layer spike firing statistics and synaptic operations of SCIB on CIFAR-10. M denotes MAC operations; A denotes AC operations.

Figure 6: Comparison of average spike firing rate and per-layer energy consumption during single-sample inference.

Panel of Figure 7 — Figure 7: Training evolution of the class-conditional prior distribution $r(s|y)$ . Row (a): PCA projections of prior mean vectors; colored dots mark mean positions and $1\sigma$ confidence ellipses indicate the local coverage of each class prior. Row (b): KL divergence heatmaps between class-conditional priors.

Panel of Figure 8 — Figure 8: Sensitivity analysis of the compression strength hyperparameter $\beta$ .

Spiking Neural Networks

Spiking neural networks process information through discrete, time-dependent neural events, aiming for efficient computation and biologically inspired learning.

See all 345 papers

Burst Spiking Neural Networks
Jul 5, 2026Jiahong Zhang, Sijun Shen, Man Yao +5Distribution Shift Robustness Neural Network Robustness
Institute of Automation, Chinese Academy of Sciences, Beijing 100045, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China · State Key Laboratory of Media Convergence and Communication, Communication University of China, Beijing 100024, China +3
Controllable Stochastic Quantization Encoding for Adversarially Robust Spiking Neural Networks
Oct 1, 2026Yujia Liu, Peiyu Liu, Yajing Zheng +1Adversarial Robustness Spiking Neural Networks
NERCVT, School of Computer Science, Peking University, China · State Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University, China · Institute for Artificial Intelligence, Peking University, China
Driving up Inference Energy on SNNs: Per-Sample and Universal Sponge Attacks
Jul 30, 2026Spyridon Raptis, Haralampos-G. StratigopoulosAdversarial Attacks Spiking Neural Networks
Sorbonne Universit´e, CNRS, LIP6, Paris, France

Jul 5, 2026J/K move · Enter open · S save

Burst Spiking Neural Networks

Jiahong Zhang, Sijun Shen, Man Yao, Han Xu+4

Institute of Automation, Chinese Academy of Sciences, Beijing 100045, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China · State Key Laboratory of Media Convergence and Communication, Communication University of China, Beijing 100024, China+3

Distribution Shift Robustness Neural Network Robustness Spiking Neural Networks Corruption Robustness

Open paper AskPDF arXiv

A central goal of current Spiking Neural Network (SNN) research is to improve their accuracy toward becoming low-power alternatives to Artificial Neural Networks (ANNs). This work further argues that realizing this ambition requires improving not only accuracy but also robustness, defined as the ability to maintain correct predictions under input perturbations. We identify two key issues in existing SNN methods that undermine robustness. First, binary spiking activations can produce large activation-state changes under small perturbations. Second, the lack of effective weight constraints makes network outputs more sensitive to input variations. To this end, we propose Burst Spiking Neural Networks (BuSNNs), built upon Burst-enhanced Spiking Neurons (BSNs) and a Dynamic Weight Constraint (DWC) mechanism. BSNs incorporate burst firing to provide a graded spiking pattern. This spiking mechanism mitigates perturbation-induced transitions in activation states and thereby enhances robustness. DWC penalizes connection weights based on activation states, effectively reducing weight magnitudes and improving robustness while preserving accuracy. We provide theoretical analyses to support these robustness effects. Experimental results further show that, on smaller-scale benchmarks such as CIFAR-10, BuSNNs outperform both SNN and ANN counterparts in accuracy and robustness. On large-scale ImageNet, BuSNN with the MS ResNet-34 backbone further improves top-1 accuracy and corruption robustness over the corresponding SNN baseline by 3.18% and 2.66%, respectively. Despite using spike-based activations, BuSNNs surpass 4-bit activation-quantized ANN baselines and approach 8-bit ANN baselines on ImageNet. They also preserve SNNs' low-power advantage. This work studies the accuracy-robustness problem in SNNs, advancing their practical viability in robust and energy-efficient applications.

Abstract

Spiking Neural Networks

Explore similar work

Burst Spiking Neural Networks

Controllable Stochastic Quantization Encoding for Adversarially Robust Spiking Neural Networks

Driving up Inference Energy on SNNs: Per-Sample and Universal Sponge Attacks