Local Timescale Gates for Timescale-Robust Continual Spiking Neural Networks

Devansh Deep; Ansh Tiwari; Ayush Chauhan

cs.LGOct 13, 2025

Local Timescale Gates for Timescale-Robust Continual Spiking Neural Networks

Authors: Devansh Deep, Ansh Tiwari, Ayush Chauhan

Abstract

Spiking neural networks (SNNs) promise energy-efficient artificial intelligence on neuromorphic hardware but struggle with tasks requiring both fast adaptation and long-term memory, especially in continual learning. We propose Local Timescale Gating (LT-Gate), a neuron model that combines dual time-constant dynamics with an adaptive gating mechanism. Each spiking neuron tracks information on a fast and a slow timescale in parallel, and a learned gate locally adjusts their influence. This design enables individual neurons to preserve slow contextual information while responding to fast signals, addressing the stability-plasticity dilemma. We further introduce a variance-tracking regularization that stabilizes firing activity, inspired by biological homeostasis. Empirically, LT-Gate yields significantly improved accuracy and retention in sequential learning tasks: on a challenging temporal classification benchmark it achieves about 51 percent final accuracy, compared to about 46 percent for a recent Hebbian continual-learning baseline and lower for prior SNN methods. Unlike approaches that require external replay or expensive orthogonalizations, LT-Gate operates with local updates and is fully compatible with neuromorphic hardware. In particular, it leverages features of Intel's Loihi chip (multiple synaptic traces with different decay rates) for on-chip learning. Our results demonstrate that multi-timescale gating can substantially enhance continual learning in SNNs, narrowing the gap between spiking and conventional deep networks on lifelong-learning tasks.

Figures & tables

Panel of Figure 1 — Figure 1 : Architecture of the LT-Gate neuron model. Each neuron contains two parallel LIF compartments with different time constants ( $\tau_{f}\ll\tau_{s}$ ). A learnable gate $\gamma_{i}$ combines their outputs. When the combined membrane potential exceeds threshold, both compartments are reset, preserving the dual-timescale dynamics.

Panel of Figure 2 — Figure 2 : Continual learning under temporal shift and cross-timescale generalization. (a) Accuracy over 200 epochs with a fast $\to$ slow task switch at epoch 100 (dashed line). LT-Gate consistently achieves higher peak accuracy pre-switch and retains more performance post-switch than HLOP and DSD-SNN, indicating reduced catastrophic forgetting. (b) After training only at 1000 Hz, accuracy is evaluated from $10$ – $1000$ Hz (log scale). LT-Gate remains strongest at slow inputs and degrades most gracefully as frequency departs from the training regime, evidencing a robust multi-timescale bias.

Method	Final Acc.	Task A Forget.	Task B Acc.	Conv. Speed
	(%)	(%)	(%)	(epochs)
Standard SNN	38.2	12.5	41.8	45
HLOP	45.2	5.8	47.1	40
DSD-SNN	42.8	7.1	44.5	35
LT-Gate	51.0	3.2	52.8	25
LT-Gate (no gate)	46.5	8.9	48.2	42

Table 1 : Quantitative Comparison of Continual Learning Methods

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Network Architecture
Parameter	Value
Architecture type	3-layer CNN
Layer 1 (Conv)	32 filters, 3 $\times$ 3 kernel, stride 1
Layer 2 (Conv)	64 filters, 3 $\times$ 3 kernel, stride 2
Layer 3 (FC)	128 units $\rightarrow$ 10 classes
Neuron Model Parameters

Appendix

Table 2 : Network Architecture and Hyperparameters

Panel of Figure 3 — Figure 3 : Example membrane potential dynamics for fast and slow compartments responding to two input spikes at $t=2$ ms and $t=10$ ms (marked by vertical dotted lines). The fast compartment (blue) responds quickly but decays rapidly. The slow compartment (red) integrates over longer periods. The gated output (green dashed) combines both pathways according to the learned gate value $\gamma=0.7$ . This demonstrates how LT-Gate neurons simultaneously capture transient events and maintain longer-term context.

Panel of Figure 4 — Figure 4 : Learned specialization and hardware efficiency. (a) Learned gate statistics after training: early layers cluster near $\gamma\!\approx\!0.5$ (balanced integration), the hidden layer becomes bimodal, and the output layer polarizes toward $\gamma\!\approx\!0$ (fast) or $\gamma\!\approx\!1$ (slow), revealing a division of labor across timescales. (b) On Loihi 2, power scales roughly linearly with network size; LT-Gate’s overhead versus a standard SNN remains small even at 1024 neurons, supporting hardware-compatible continual learning.

Method	Spikes/Inference	Memory Usage	Power (mW)	Loihi Compatible
	( $\times$ Baseline)	( $\times$ Baseline)
Standard SNN	1.00	1.00	12.5	✓
HLOP	1.35	2.10	18.2	✗ (requires SVD)
DSD-SNN	0.85	1.34	15.7	✓
LT-Gate	1.02	1.15	13.1	✓

Appendix

Table 3 : Comprehensive Hardware Efficiency Comparison

Explore similar work

Scalable Training of Continuous-Time Spiking Neural Networks with Differentiable Spike-Time Discretization
Jul 16, 2026Yusuke Sakemi, Tomoya Takeuchi, Takeo Hosomi +1Spiking Neural Networks
Research Center for Mathematical Engineering, Chiba Institute of Technology, Narashino, Japan · International Research Center for Neurointelligence (WPI-IRCN), The University of Tokyo, Tokyo, Japan · NEC Corporation, Kawasaki, Japan
Multi-Timescale Conductance Spiking Networks: A Sparse, Gradient-Trainable Framework with Rich Firing Dynamics for Enhanced Temporal Processing
May 12, 2026Alex Fulleda-Garcia, Saray Soldado-Magraner, Josep Maria Margarit-TauléSpiking Neural Networks Neuromorphic Computing
Instituto de Microelectrónica de Barcelona (IMB-CNM), Consejo Superior de Investigaciones Cientíﬁcas (CSIC) Cerdanyola del Vallès, Spain · Department of Neurobiology, University of California Los Angeles (UCLA) Los Angeles, USA · Joint Senior authors
QB-LIF: Learnable-Scale Quantized Burst Neurons for Efficient SNNs
Apr 28, 2026Dewei Bai, Hongxiang Peng, Jiajun Mei +4Efficient Neural Network Inference Spiking Neural Networks

Jul 16, 2026, cs.LGJ/K move · Enter open · S save

Scalable Training of Continuous-Time Spiking Neural Networks with Differentiable Spike-Time Discretization

Yusuke Sakemi, Tomoya Takeuchi, Takeo Hosomi, Kazuyuki Aihara

Research Center for Mathematical Engineering, Chiba Institute of Technology, Narashino, Japan · International Research Center for Neurointelligence (WPI-IRCN), The University of Tokyo, Tokyo, Japan · NEC Corporation, Kawasaki, Japan

Spiking Neural Networks

Open paper AskPDF arXiv

Continuous-time spiking neural networks (SNNs) provide an event-driven framework for temporal computation, computational neuroscience, and neuromorphic hardware. However, training deep continuous-time SNNs is severely constrained by the memory required for exact spike-time computation, which evaluates and retains candidate firing times over intervals determined by presynaptic spike ordering. Here we introduce a memory-efficient training framework based on differentiable spike-time discretization (DSTD) for leaky integrate-and-fire neurons with general membrane and synaptic time constants. DSTD maps irregular presynaptic spikes onto differentiable weighted events at fixed time points, replacing the input-dependent candidate dimension with

M

fixed time intervals while accurately approximating continuous-time membrane-potential dynamics. This reduces candidate-related activation memory from

O(N_{\mathrm{out}}N_{\mathrm{in}})

O(N_{\mathrm{out}}M)

in the case of time-to-first-spike (TTFS) coding, where

N_{\mathrm{in}}

and

N_{\mathrm{out}}

denote the numbers of presynaptic and postsynaptic neurons, respectively. We further introduce synfire-chain-inspired temporal regularization that organizes layer-wise firing windows, mitigates dead-neuron failures, and enables pipeline-like processing. In dense LIF layers, DSTD reduced peak memory consumption by up to approximately 100-fold and training time by up to approximately 20-fold compared with exact spike-time computation. Together, these methods allowed us to train 9-layer convolutional SNNs on CIFAR-10 and 20-layer convolutional SNNs on Fashion-MNIST on a single GPU.

Abstract

Explore similar work

Scalable Training of Continuous-Time Spiking Neural Networks with Differentiable Spike-Time Discretization

Multi-Timescale Conductance Spiking Networks: A Sparse, Gradient-Trainable Framework with Rich Firing Dynamics for Enhanced Temporal Processing

QB-LIF: Learnable-Scale Quantized Burst Neurons for Efficient SNNs