FidLIF: Parallel Training of High-Fidelity Spiking Neurons with Exact BPTT

Kaixiang He; Yujie Wu

cs.NEOct 12, 2026

FidLIF: Parallel Training of High-Fidelity Spiking Neurons with Exact BPTT

Authors: Kaixiang He, Yujie Wu

Organizations: Department of Computing, The Hong Kong Polytechnic University

Abstract

Although spiking neural networks (SNNs) show promise in energy-efficient computing through event-driven computation and stateful temporal processing, their recurrent membrane dynamics render their training inherently sequential and inefficient for long sequences. Existing methods achieve full-sequence parallelism in SNNs through two main strategies: redesigning or approximating spiking dynamics to avoid reset-coupled temporal dependencies, or relying on iterative relaxed fixed-point solvers that suffer from excessive memory costs and iteration-dependent backward overhead. To address these limitations, we present FidLIF, a novel framework that reconciles the incremental update and hard-reset behavior of standard spiking LIF neurons with parallel forward propagation and BPTT-equivalent backward propagation. FidLIF iteratively recovers the full-sequence membrane-potential trajectory in parallel. By exploiting the lower bidiagonal structure induced by the local recurrence of LIF neurons, FidLIF formulates each forward trajectory correction and backward adjoint propagation as associative scans with an $O(\log T)$ temporal depth. This formulation also supports input-dependent neuronal parameters and allows the same neuron to perform full-sequence parallel training and incremental step-by-step inference at deployment. In our numerical convergence evaluation, FidLIF achieves a mean absolute error below $10^{-7}$ relative to the matched serial relaxed LIF dynamics, while yielding gradients numerically equivalent to exact BPTT. Our Triton implementation delivers favorable long-sequence latency and memory scaling, while achieving competitive performance on four sequential classification benchmarks. By reconciling parallel training with step-by-step execution, FidLIF enables high-fidelity spiking dynamics to benefit from parallel hardware without compromising incremental inference at deployment.

Figures & tables

Figure 1: Overall workflow of FidLIF. (a) In forward propagation, FidLIF uses Newton’s method to recover the membrane potential

\mathbf{v}

across all time steps. (b) Illustration of one forward iteration. Given the result of

m

-th iteration

\mathbf{v}^{(m)}

and its corresponding Jacobian

\mathbf{J}^{(m)}

, the result of

(m+1)

-th iteration is obtained by a parallel scan. (c) In backpropagation, only one scan is needed. The transposed Jacobian matrix at the final trajectory is used to compute the adjoint vector

\mathbf{y}

from upstream gradient

\mathbf{g}

Figure 2: Fidelity to Heaviside-gated hard-reset LIF dynamics. (a) Representative membrane-potential (top) and spike trajectories (bottom) for

T=32

, comparing serial LIF with FidLIF using

K=1

and

3

iterations. (b) Mean absolute error (MAE) of membrane potential. (c) Spike mismatch rate relative to LIF. Curves in (b) and (c) show the mean over five independent trials, and shaded regions denote one standard deviation.

Figure 3: Convergence to matched sigmoid-relaxed LIF dynamics. Each method is compared against its own matched serial baseline.

Figure 4: Gradient recovery and computational efficiency. (a) Gradient error vs. maximum iteration budgets

K

(mean

\pm

std, five seeds). (b) Backward and (c) combined forward–backward latency at fixed iterations

K=3

. (d) Peak memory at

T=256

vs. fixed iterations

K

Dataset	Method	Arch.	Efficient Step-by- step inference	Parallelization Strategy	Accuracy (%)
Seq. CIFAR10	PSN / Sliding PSN	ConvNet	$\times$ / $\surd$ ∗	Reset Removal	$88.45$ / $86.70$
	PSU / IPSU	ConvNet	$\surd$ / $\times$ †	Reset Estimation	$79.52$ / $87.28$
	MF-PSN	ConvNet	$\surd$ ∗	Reset Removal	91.17
	FPT / FPT with $\mathbf{A}$	ConvNet	$\surd$ / $\times$ †	Repeated Substitution	$81.54{\pm}0.34$ / $89.53{\pm}0.13$
	FidLIF (Ours)	ConvNet	$\surd$	Newton’s method + Scan	$90.99{\pm}0.25$ ( 91.25 )
Seq. CIFAR100	PSN / Sliding PSN	ConvNet	$\times$ / $\surd$ ∗	Reset Removal	$62.21$ / $62.11$

Table 1: Comparison with other full-sequence parallel SNNs. Error bars denote mean

\pm

standard deviation, with the best run shown in parentheses. ∗ : Fixed-length temporal window. † : Learnable temporal parameters break the recurrent structure.

Dataset	Parallel Acc. (%)	Serial Acc. (%)
Seq. CIFAR10	$90.99\pm 0.25$	$90.69\pm 0.19$
Seq. CIFAR100	$66.42\pm 0.17$	$65.98\pm 0.24$
CIFAR10-DVS	$84.13\pm 0.65$	$83.50\pm 0.96$
SHD	$94.38\pm 0.31$	$94.30\pm 0.39$

Table 2: Accuracy comparison between parallel full-sequence execution and serial step-by-step execution.

$K$	Seq. CIFAR10 (%)	Seq. CIFAR100 (%)
3	$90.99\pm 0.25$	$66.42\pm 0.17$
5	$90.79\pm 0.20$	$66.57\pm 0.42$

Table 3: Effect of the maximum iterations

K

Method	Seq. CIFAR10 (%)	Seq. CIFAR100 (%)
FidLIF	$\mathbf{91.25}$	$\mathbf{66.59}$
w/o input-dependent	$80.66$	$54.52$

Table 4: Ablation of input-dependent neuronal dynamics.

LIF Neurons

See all 12 papers

Feb 27, 2026

Bullet Trains: Parallelizing Training of Temporally Precise Spiking Neural Networks

Continuous-time, event-native spiking neural networks (SNNs) operate strictly on spike events, treating spike timing and ordering as the representation rather than an artifact of time discretization. This viewpoint aligns with biological computation and with the native resolution of event sensors and neuromorphic processors, while enabling compute and memory that scale with the number of events. However, two challenges hinder practical, end-to-end trainable event-based SNN systems: 1) exact charge--fire--reset dynamics impose inherently sequential processing of input spikes, and 2) precise spike times must be solved without time bins. We address both. First, we use parallel associative scans to consume multiple input spikes at once, yielding up to 44x speedups over sequential simulation while retaining exact hard-reset dynamics. Second, we implement differentiable spike time solvers that compute spike times to machine-precision without discrete-time approximations or restrictive analytic assumptions. We demonstrate the viability of training SNNs using our solutions on four event-based datasets on GPUs.

Todd Morrill, Christian Pehle, Anthony Zador

Department of Computer Science, Columbia University, New York, NY, USA · Department of Neuroscience, Cold Spring Harbor Laboratory, Cold Spring Harbor, NY, USA

Spiking Neural Networks Neuromorphic Computing

Ask PDF

Apr 28, 2026

QB-LIF: Learnable-Scale Quantized Burst Neurons for Efficient SNNs

Binary spike coding enables sparse and event-driven computation in spiking neural networks (SNNs), yet its 1-bit-per-timestep representation fundamentally limits information throughput. This bottleneck becomes increasingly restrictive in deep architectures under short simulation horizons. We propose the Quantized Burst-LIF (QB-LIF) neuron, which reformulates burst spiking as a saturated uniform quantization of membrane potentials with a learnable scale. Instead of relying on predefined multi-threshold structures, QB-LIF treats the quantization scale as a trainable parameter, allowing each layer to autonomously adapt its spiking resolution to the underlying membrane-potential statistics. To preserve hardware efficiency, we introduce an absorbable scale strategy that folds the learned quantized scale into synaptic weights during inference, maintaining a strict accumulate-only (AC) execution paradigm. To enable stable optimization in the discrete multi-level space, we further design ReLSG-ET, a rectified-linear surrogate gradient with exponential tails that sustains gradient flow across burst intervals. Extensive experiments on static (CIFAR-10/100, ImageNet) and event-driven (CIFAR10-DVS, DVS128-Gesture) benchmarks demonstrate that QB-LIF consistently outperforms binary and fixed-burst SNNs, achieving higher accuracy under ultra-low latency while preserving neuromorphic compatibility.

Dewei Bai, Hongxiang Peng, Jiajun Mei +4

Efficient Neural Network Inference Spiking Neural Networks Activation Quantization

Ask PDF

Jun 11, 2026

LongSpike: Fractional Order Spiking State Space Models for Efficient Long Sequence Learning

Spiking Neural Networks (SNNs) are well-regarded for their biological plausibility and energy efficiency in processing sequential data. However, dominant SNN architectures typically rely on first-order Ordinary Differential Equations (ODEs) to govern neuronal state transitions. This first-order assumption imposes a "memoryless" bottleneck, limiting the model's capacity to capture the complex, long-range dependencies inherent in long-sequence tasks. In this work, we propose LongSpike, a novel SNN framework that integrates fractional-order State-Space Modeling, or f-SSM, from control theory into the spiking domain. By extending traditional integer-order SSMs to the fractional-calculus regime, LongSpike enables the hierarchical integration of neuronal dynamics with long-memory kernels. To mitigate the computational overhead and parallelization challenges typically associated with fractional operators, we leverage a state-space formulation that supports efficient, parallel training. Empirical evaluations on challenging benchmarks, including Long Range Arena (LRA), large-scale WikiText-103, and Speech Commands, demonstrate that LongSpike outperforms state-of-the-art SNNs in accuracy while preserving sparse synaptic computation. The code is available at https://github.com/xinruihe389-commits/LongSpike.

Xinrui He, Qiyu Kang, Xuhao Li +1

Wuhan University · University of Science and Technology of China · Anhui University

Long-Context Modeling Spiking Neural Networks Sequence Modeling State Space Models

Ask PDF

Abstract

LIF Neurons

Explore similar work

Bullet Trains: Parallelizing Training of Temporally Precise Spiking Neural Networks

QB-LIF: Learnable-Scale Quantized Burst Neurons for Efficient SNNs

LongSpike: Fractional Order Spiking State Space Models for Efficient Long Sequence Learning