Predictive Suppression Layers for Communication-Efficient Spiking Neural Networks

Aidin Attar; Michele Rossi

cs.NESep 18, 2026

Predictive Suppression Layers for Communication-Efficient Spiking Neural Networks

Authors: Aidin Attar, Michele Rossi

Organizations: University of Padova Department of Information Engineering Padova, Italy · University of Padova Dept. of Inf. Eng. and Dept. of Mathematics Padova, Italy

Abstract

Feedforward Spiking Neural Networks (SNNs) typically propagate every generated spike indiscriminately, disregarding whether the information is redundant from an information-theoretic perspective. This lack of selectivity induces high redundancy in inter-layer communication, creating an expensive overhead, e.g., in scenarios involving many-core neuromorphic hardware or communication-dominated Internet-of-Things (IoT) where features are transmitted wirelessly. To address this challenge, we trade localized processing for leaner network channels by introducing a minimal predictive coding framework for SNNs. We propose two layer variants sharing a predictor block: error units, which transmit signed spiking residuals, and predictive suppression, which uses residual magnitude to dynamically gate and forward only unpredictable, "surprising" activity. Evaluated on the N-MNIST and Spiking Heidelberg Digits (SHD) datasets using diagnostic metrics that decouple local processing from cross-layer communication, our new predictive coding layers achieve significant communication savings. Numerical results reveal a three-fold reduction in communicated activity, while increasing the task accuracy for both datasets. The latter finding is notable, and suggests that predictive coding layers not only minimize communication overhead, but also produce output feature vectors with a higher representation power.

Figures & tables

Figure 1. Predictive coding layer. (A) Plain feedforward SNN forwards the entire representation to the next layer. (B) Predictive suppression keeps the layers spiking but adds a local error-driven gate that suppresses predictable spikes and forwards the surprising ones, giving less communicated activity. Schematic comparison between a plain feedforward spiking network, where every spike a layer produces is forwarded to the next layer, and predictive suppression, where local error-driven gates suppress predictable spikes and let surprising ones through.

Figure 2. What each layer communicates. The two schemes share the same encoder/predictor skeleton; only the forwarded message differs. (A) The error-units variant forwards the prediction error

e_{\ell}=x_{\ell}-\hat{x}_{\ell}

. (B) Predictive suppression encodes

z_{\ell}

, predicts its own input

\hat{x}_{\ell}

, and forwards

y_{\ell}=g_{\ell}z_{\ell}

, where the gate

g_{\ell}=1-e^{-\alpha m_{\ell}}

is driven by the prediction-error magnitude. Two-panel block diagram of the per-layer mechanisms. Both panels show a shared skeleton with an encoder, a predictor, the prediction estimate, and the layer input routed into a bottom box. In the left panel (error units) the bottom box computes the prediction error and forwards it to the next layer. In the right panel (predictive suppression) the bottom box computes the prediction-error magnitude that drives a multiplicative gate on the encoder output, and the gated representation is forwarded.

Dataset	Model	Channel	Accuracy	$E_{\text{comm}}$	Compr.	SSI / RCF
N-MNIST	SNN baseline	events	$0.941{\scriptstyle\,\pm\,0.002}$	3006	1.00	—
	Error units	events	$0.946{\scriptstyle\,\pm\,0.004}$	855	0.33	—
	Predictive suppression	weighted	$0.976{\scriptstyle\,\pm\,0.001}$	821	0.52	0.770 / 0.050
	Respike	events	$0.959{\scriptstyle\,\pm\,0.002}$	943	0.81	0.995 / 0.001
	Hard-STE	events	$0.950{\scriptstyle\,\pm\,0.012}$	736	0.73	0.985 / 0.003
SHD	SNN baseline	events	$0.621{\scriptstyle\,\pm\,0.005}$	3094	1.00	—

Table 1. Summary of main results. Accuracy is mean

\pm

standard deviation over five seeds. SSI and RCF are only reported for suppression-based communication.

E_{\mathrm{comm}}

is computed on the actual transmitted signal

y_{\ell}

; it is a summed magnitude for graded channels and an event count for binary channels.

Figure 3. (A) Accuracy versus communicated activity proxy (summed over layers) on N-MNIST. Predictive suppression (filled circle) keeps high accuracy while communicating less than the matched baseline. (B) Communication-weighted cost sensitivity

C(\rho)=E_{\mathrm{local}}+\rho\,E_{\mathrm{comm}}

. As predictive suppression spends extra local processing to reduce communication, it becomes cheaper than the baseline once communication is weighted about

\rho^{\star}\!\approx\!2.1

times more than local computation. Two-panel figure on N-MNIST. The left panel is a scatter plot of model accuracy versus weighted communicated activity, with predictive suppression sitting at the top-left (high accuracy, low communication) and the matched baseline at the bottom-right. The right panel is a log-log plot of the weighted cost C(rho) as a function of the relative communication cost rho between 1 and 1000, with two curves: a solid line for the baseline and a dashed line for predictive suppression; the dashed line is above the solid one at low rho and crosses below it at rho about 2.1, marked by a vertical dotted line.

Figure 4. Linear-probe decodability for predictive suppression. At every layer the communicated message (filled) is essentially as decodable as the full representation (open); the paired dots nearly coincide. The first layer message carries about

7\times

less weighted activity than the baseline. Inset: the communicated activity that this message costs, per layer. Dumbbell plot of linear-probe accuracy at three layers on N-MNIST. Each layer is a horizontal row with two markers, a filled dot for the probe trained on the communicated message and an open dot for the probe trained on the full local representation; the two markers nearly overlap at every layer. A small inset shows the communicated activity used per layer.

Figure 5. Communication is selective, not merely reduced: at the first layer of predictive suppression, communicated activity rises monotonically with the pred.-error magnitude

m_{\ell}

. Inset: SSI (filled) is high and the RCF (open) is low. Plot of mean communicated activity at the first predictive-suppression layer on N-MNIST as a function of the per-layer prediction-error magnitude, with error bars; the curve rises monotonically. A small inset shows two short horizontal lollipops: a filled marker for the surprise-selectivity index near 0.75 and an open marker for the redundant-communication fraction near 0.07.

Spiking Neural Networks

Spiking neural networks process information through discrete, time-dependent neural events, aiming for efficient computation and biologically inspired learning.

See all 345 papers

Mar 16, 2026

Spiking Layer-Adaptive Magnitude-based Pruning

Spiking Neural Networks (SNNs) provide energy-efficient computation but their deployment is constrained by dense connectivity and high spiking operation costs. Existing magnitude-based pruning strategies, when naively applied to SNNs, fail to account for temporal accumulation, non-uniform timestep contributions, and membrane stability, often leading to severe performance degradation. This paper proposes Spiking Layer-Adaptive Magnitude-based Pruning (SLAMP), a theory-guided pruning framework that generalizes layer-adaptive magnitude pruning to temporal SNNs by explicitly controlling worst-case output distortion across layers and timesteps. SLAMP formulates sparsity allocation as a temporal distortion-constrained optimization problem, yielding time-aware layer importance scores that reduce to conventional layer-adaptive pruning in single-timestep limit. An efficient two-stage procedure is derived, combining temporal score estimation, global sparsity allocation, and magnitude pruning with retraining for stability recovery. Experiments on CIFAR10, CIFAR100, and the event-based CIFAR10-DVS datasets demonstrate that SLAMP achieves substantial connectivity and spiking operation reductions while preserving accuracy, enabling efficient and deployable SNN inference.

Junqiao Wang, Zhehang Ye, Yuqi Ouyang

Sichuan University-Pittsburgh Institute, Sichuan University, Chengdu, 610207, China · College of Computer Science, Sichuan University, Chengdu, 610065, China

Spiking Neural Networks Neural Network Pruning Efficient Neural Network Inference

Ask PDF

Jul 29, 2026

The Sparsity Ceiling: Where Spiking Networks Can and Cannot Trade Activity for Energy

Spiking neural networks (SNNs) are promoted as an energy-efficient substrate because sparse, event-driven activity replaces dense multiply-accumulates with cheap accumulates. We argue the energy dividend of sparsity is not a property of SNNs but of the task. Holding architecture fixed and swapping only the hidden unit (continuous vs. leaky-integrate-and-fire), plus a two-sided target-firing-rate probe, we measure how far activity can be pushed down before quality breaks. Low-load feed-forward perception sparsifies to 5% firing at no accuracy cost; a recurrent language model cannot go below ~50% -- the recurrent state must stay active to carry information. A spiking Transformer, by contrast, sparsifies freely to 2% (3 seeds) -- so the ceiling is a property of recurrent compression, not sequence modeling. Attention escapes the floor only by storing the full key-value cache, trading a firing floor for a memory wall: on neuromorphic hardware, recurrence and attention pay on different axes, neither escapes. We formalize the ceiling with an information-theoretic bound rho >= H_b^{-1}(log2 M / H) and confirm its predictions: the floor rises with memory load, falls with state width, and (refuting a naive memory-only reading) rises with task difficulty. A layer-wise input floor further caps op reduction under dense input, isolating event-driven perception as where neuromorphic hardware wins.

Zeyu Wang

Georgia Institute of Technology

Efficient Neural Network Inference Spiking Transformers Activation Sparsity Recurrent Neural Networks Spiking Neural Networks Neuromorphic Computing

Ask PDF

Aug 11, 2026

Activation-Energy Pruning for Spiking Neural Networks: Unsupervised Personalization via Spike-Count Saliency

Activation-energy pruning -- removing weights whose product of magnitude and cumulative pre-synaptic spike count falls below a threshold -- was established as an effective unsupervised personalization strategy for conventional deep neural networks~\citep{BINGHAM2025101242}. This paper asks what happens when the same criterion is applied to spiking neural networks (SNNs), where activation energy is not merely a useful heuristic but a literal physical quantity proportional to the metabolic cost of each synapse. The answer is surprising on three counts. First, gradient-based pruning methods that perform competitively on conventional networks (SNIP, GraSP, magnitude pruning) consistently underperform on SNNs, collapsing to near-chance accuracy by

σ= 0.2

sparsity across all tested architectures and datasets. We trace this to a systematic incompatibility between surrogate-gradient saliency estimation and the binary spike-train representation, though we cannot rule out that alternative surrogate choices or hyperparameter settings might partially mitigate the effect. Second, activation-energy pruning applied to a neuromorphic benchmark improves over the source model at high sparsity (

98.4 \pm 0.4\%

vs.\

97.2 \pm 0.7\%

σ= 0.8

on N-MNIST), a phenomenon with no counterpart in the conventional network setting. We interpret this result as consistent with experience-dependent cortical specialisation: removing connections active only for non-target classes may reduce cross-class interference and produce a cleaner target representation, though we note this is an interpretive analogy rather than a mechanistic demonstration.

Joseph Bingham

Technion – Israel Institute of Technology, Haifa, Israel

Neural Network Pruning Spiking Neural Networks

Ask PDF

Abstract

Spiking Neural Networks

Explore similar work

Spiking Layer-Adaptive Magnitude-based Pruning

The Sparsity Ceiling: Where Spiking Networks Can and Cannot Trade Activity for Energy

Activation-Energy Pruning for Spiking Neural Networks: Unsupervised Personalization via Spike-Count Saliency