Causal pieces: analysing and improving spiking neural networks piece by piece

Dominik Dold; Philipp Christian Petersen

cs.NEApr 18, 2025

Causal pieces: analysing and improving spiking neural networks piece by piece

Authors: Dominik Dold, Philipp Christian Petersen

Abstract

We introduce "causal pieces", a novel concept for analysing spiking neural networks (SNNs), inspired by "linear pieces" used to study expressivity and trainability in artificial neural networks (ANNs). Causal pieces partition the input and parameter space of a feedforward SNN with single-spike coding into distinct regions where the same subnetwork causes the output spikes. For networks of current-based leaky integrate-and-fire (LIF) neurons with large membrane time constants, we show that within each causal piece, output spike times are locally Lipschitz continuous with respect to inputs and network parameters. We further prove a lower bound on the approximation error that depends on the number of causal pieces. Thus, the number of causal pieces is a measure of the approximation capabilities of SNNs, which is valid despite spike-time discontinuities and applies to networks with both excitatory and inhibitory synapses. Empirically, we find that parameter initialisations yielding more causal pieces on the training set strongly correlate with SNN training success across multiple benchmarks, including Yin-Yang, Fashion-MNIST, and EuroSAT. Moreover, simulations with standard single-spike LIF neurons indicate that our findings extend beyond the theoretically analysed regime. These results establish causal pieces as a powerful and principled tool for analysing and improving the computational capabilities of SNNs.

Figures & tables

Figure 1: Causal pieces. (A) Every coloured region is a causal piece, fully partitioning the input space (here: 2D inputs). Within each piece, the same subnetwork causes the SNN output (left, coloured like the corresponding piece). Here we scan a 2D hyperplane $I$ in the $40$ -dimensional input space, parametrised by $(\alpha_{0},\alpha_{1})\in[0,1]^{2}$ . (B) On each piece (coloured), the output spike time of the SNN is a continuous function. (C) Causal pieces for an SNN with no (left) and one (right) hidden layer.

Figure 2: Estimating the number of causal pieces. (A) The probabilities $p^{q}_{k}$ are obtained by counting how many trajectories (cumulative sum of weights) are above the threshold at step $k$ . The top panel shows the corresponding values of $p^{q}_{k}$ , where $k$ is the number of weights. (B) Estimated number of pieces for weights sampled from normal distribution with different mean (y-axis) and standard deviation (x-axis). Colours are shown in log-scale. (C) $p^{q}_{k}$ for two points in (B), denoted by markers.

Figure 3: Network initialisation strongly affects training success. (A) The logarithm of the number of pieces (here: of the output layer) at network initialisation strongly correlates with performance after training ( $r=0.70$ , log-linear). Shown for $\tau_{\mathrm{s}}=0.5$ . (B) Same as (A), but with the number of pieces after training. (C) Number of pieces before and after training. The diagonal indicates no change in pieces. (D) Same as (A), but for nLIF neurons with time constant $\tau_{\mathrm{s}}=0.1$ . (E) Illustration of the Yin-Yang dataset with three classes: the two halves and the dots. (F) Causal pieces (each piece is indicated by a different colour) of the hidden layer for a poor initialisation, evaluated using only training samples. (G) Same as (F), but for one of the best initialisations.

Figure 4: Experimental validation using more complex neuron models and datasets. (A-C) Same as Fig. 3 A-C, but for LIF neurons with

\tau_{\mathrm{m}}=0.5

and

\tau_{\mathrm{s}}=0.25

. (D) Same as Fig. 3 D, but for LIF neurons with

\tau_{\mathrm{m}}=0.1

and

\tau_{\mathrm{s}}=0.05

. (E-F) The number of causal pieces at initialisation correlates with final test accuracy for nLIF SNNs trained on Fashion MNIST and EuroSAT. (G-H) Same as E-F, but for LIF SNNs with

\tau_{\mathrm{m}}=0.5

and

\tau_{\mathrm{s}}=0.25

. E-H are plotted on linear axes for easier readability. Reported correlations are also calculated using the untransformed number of causal pieces. Log-linear correlations are (from E to H): 0.74, 0.69, 0.62, and 0.62.

Figure 5: Width-and depth dependence of causal pieces. (A)

p^{q}_{k}

of the optimised (top, dot) normal, and (bottom, square) uniform initialisation. (B) Number of pieces for shallow and deep networks. The maximum number, which is the number of input samples used to evaluate the number of causal pieces, is shown as a dash-dotted line. (C) Number of pieces per layer in a single network, before and after training. (D) Increase in the total number of pieces for deep and shallow networks. Markers denote results that belong together. We show medians (lines) and quartiles (shaded areas).

Figure 6: SNNs with only positive weights. (A)

p^{q}_{k}

for (top, star) lognormal, and (bottom, diamond) uniform initialisation. (B) Number of pieces for shallow and deep networks. Labels as in Fig. 5 B. (C) Used network architecture. (D) Performance on benchmarks. Each training run was repeated 5 times for different random seeds. Markers denote results that belong together.

Defined	Causal sets, subnetworks, and pieces for feedforward, single-spike SNNs, calculated from connectivity and observed spike times. The definitions are not specific to nLIF dynamics and apply unchanged to standard LIF neurons.
Proved	Theorem 1 , local Lipschitz continuity within pieces ( Theorem 3 , and counting results for feedforward nLIF networks. The limited multi-spike extension follows through the cited transference principle.
Empirically supported	Correlation results for single-spike nLIF and finite-time-constant LIF networks trained with exact spike-time gradients on the three reported static classification benchmarks, including signed- and positive-weight neural networks.
Not established here	Arbitrary multi-spike dynamics, surrogate-gradient training, and recurrent neural networks.

Table 1: Summary of our work’s scope.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Left: A single output neuron (orange) receiving input from three neurons. Only those input neurons (blue) that spike before the output neuron (i.e., before the dotted line) are part of the causal set. Right: In deep networks, this corresponds to subnetworks (blue), here shown for a single output neuron (left), or the whole output layer (right).

Figure 8: Wall-clock time required to calculate the causal piece for a single data sample, using MNIST. Network architectures range from a single to three hidden layers. For each network architecture, the average of 40 different runs is shown. Error bars show the standard deviation, and the dashed line is a linear fit.

Figure 9: Fraction of data samples that switched causal piece compared to five epochs before. For

10

nLIF SNNs trained on Yin-Yang with low/intermediate initial causal piece count, and

10

nLIF SNNs with high initial count. The neurons use

\tau_{\mathrm{s}}=0.5

Model / Dataset	Feature set	SSR
nLIF on Yin-Yang	Active fraction only	5.17
nLIF on Yin-Yang	Log initial piece count only	2.95
nLIF on Yin-Yang	Active fraction + log initial piece count	2.63
LIF on Yin-Yang	Active fraction only	3.90
LIF on Yin-Yang	Log initial piece count only	1.17
LIF on Yin-Yang	Active fraction + log initial piece count	1.10

Appendix

Table 2: Predictive power of feature sets for explaining final test accuracy. Lower Sum of Squared Residuals (SSR) is better.

Figure 10: Causal pieces (coloured regions) of one of the output neurons for an nLIF neural network with $[4,30,3]$ neurons, using the initialisations obtained through evolutionary optimisation ( Fig. 5 and Fig. 6 ). Causal pieces are evaluated using a 400 $\times$ 400 grid on the data domain.

Explore similar work

Sep 11, 2026cs.LG

Polyhedral Geometry of Time-to-First-Spike Neural Networks

We study the expressivity of spiking neural networks, which provide a natural framework for asynchronous, event-driven computation complementary to conventional feedforward neural networks. We consider the time-to-first-spike model in a setting for which the input-output map is continuous and piecewise linear, with affine pieces governed by causal feasibility constraints that determine which presynaptic spikes occur before a neuron fires. We first show that each neuron's firing time admits a maxout-like representation with exponentially many, highly constrained affine pieces. We then formalize causal regions as polyhedral regions with fixed causal sets and derive upper and lower bounds on the maximal number of causal regions in both shallow and multilayer feedforward spiking networks. Our theoretical and experimental results show that spiking networks can generate richer partitions of the input space than conventional feedforward ReLU networks.

Manjot Singh, Guido Montúfar, Gitta Kutyniok

Department of Mathematics, Ludwig-Maximilians-Universität München · Munich Center for Machine Learning · Departments of Mathematics and Statistics & Data Science, University of California, Los Angeles +2

Spiking Neural Networks Time-To-First-Spike Neural Network Polytopes

Ask PDF

Apr 26, 2026cs.NE

Generalization Bounds of Spiking Neural Networks via Rademacher Complexity

Spiking Neural Networks (SNNs) have garnered increasing attention as one of bio-inspired models due to their great potential in neuromorphic computing and sparse computation. Many practical algorithms and techniques have been developed; however, theoretical understandings of the generalization, that is, the extent to which SNNs perform well on unseen data, are far from clear. Recent advances disclosed an excitation-dependent and architecture-related generalization bound such that the Rademacher complexity of SNNs with stochastic firing can be upper bounded by an exponential function relative to the excitation probability and the architecture depth. In this paper, we theoretically investigate the generalization bounds of SNNs with several integration-and-fire schemes via Rademacher complexity. We recognize that the empirical Rademacher complexity of SNNs is close to the SNN configurations, which is exponential to the network depth and the maximum time duration of received spike sequences, superlinear and subquadratic to the network width, polynomial to the parameter norm, inverse-linear to the number of training samples, and independent of the computations within spiking neurons, achieving a more precise rate than conventional studies. Our theoretical results may support the scope of SNN theories and shed some insight into the development of SNNs.

Shao-Qun Zhang, Zhi-Hua Zhou

National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210063, China. · School of Intelligent Science and Technology, Nanjing University, Suzhou 215163, China. · School of Artificial Intelligence, Nanjing University, Nanjing 210063, China.

Spiking Neural Networks Neural Network Generalization Bounds Single Excitation Time History Stochastic

Ask PDF

Apr 29, 2026cs.AI

Binary Spiking Neural Networks as Causal Models

We provide a causal analysis of Binary Spiking Neural Networks (BSNNs) to explain their behavior. We formally define a BSNN and represent its spiking activity as a binary causal model. Thanks to this causal representation, we are able to explain the output of the network by leveraging logic-based methods. In particular, we show that we can successfully use a SAT as well as a SMT solver to compute abductive explanations from this binary causal model. To illustrate our approach, we trained the BSNN on the standard MNIST dataset and applied our SAT-based and SMT-based methods to finding abductive explanations of the network's classifications based on pixel-level features. We also compared the found explanations against SHAP, a popular method used in the area of explainable AI. We show that, unlike SHAP, our approach guarantees that a found explanation does not contain completely irrelevant features.

Aditya Kar, Emiliano Lorini, Timothée Masquelier

CNRS, IRIT · Institut de Recherche en Informatique de Toulouse (IRIT), Centre de Recherche Cerveau et Cognition (CerCo), CNRS, France · Institut de Recherche en Informatique de Toulouse (IRIT), CNRS, France +2

Spiking Neural Networks Explainability Explainable Artificial Intelligence Neural Network

Ask PDF