Organizations: University of Illinois Urbana-Champaign, Department of Nuclear, Plasma & Radiological Engineering, Urbana, IL, USA · Indian Institute of Technology Delhi, Department of Applied Mechanics, New Delhi, India · University of Texas at El Paso, Department of Computer Science, El Paso, TX, USA · National Center for Supercomputing Applications, Urbana, IL, USA
Continual learning, the ability to acquire new tasks sequentially without forgetting prior knowledge, is essential for deploying neural networks in dynamic real-world environments, from nuclear digital twin monitoring to grid-edge fault detection. Existing synaptic importance methods, such as Elastic Weight Consolidation (EWC) and Synaptic Intelligence (SI), rely on gradient computation, making them incompatible with neuromorphic hardware that lacks backpropagation support. We propose ISI-CV, the first gradient-free synaptic importance metric for SNN continual learning, derived from the Coefficient of Variation (CV) of Inter-Spike Intervals (ISIs). Neurons that fire regularly (low CV) encode stable, task-relevant features and are protected from overwriting; neurons with irregular firing are permitted to adapt freely. ISI-CV requires only spike time counters and integer arithmetic, all of which are native to every neuromorphic chip. We evaluate on four benchmarks of increasing difficulty: Split-MNIST, Permuted-MNIST, Split-FashionMNIST, and Split-N-MNIST using real Dynamic Vision Sensor (DVS) event data. Across three seeds, ISI-CV achieves zero forgetting (AF = 0.000 +/- 0.000) on Split-MNIST and Split-FashionMNIST, near-zero forgetting on Permuted-MNIST (AF = 0.001 +/- 0.000), and the highest accuracy with the lowest forgetting on real neuromorphic DVS data (AA = 0.820 +/- 0.012, AF = 0.221 +/- 0.014). On N-MNIST, gradient-based methods produce unreliable importance estimates and perform worse than no regularization; ISI-CV avoids this failure by design.
Figures & tables
Figure 1. ISI-CV regularization as a four-step continual learning cycle. (1) The SNN is trained on task k via surrogate-gradient descent. The shared trunk ( W1 +LIF) feeds K per-task heads. (2) After training, a gradient-free inference pass collects hidden spike trains. Neuron i (blue) fires regularly (equal ISIs, low CV); Neuron j (orange) fires in burst–gap bursts (variable ISIs, high CV). (3) ISI-CV importance scores are assigned: Ωi=(CVi+ε)−1 . High- Ω neurons protect W1 (blue bar); low- Ω neurons update freely (orange bar). (4) Task k+1 trains with the regularized loss (Eq. 5 ). The new head (teal) adapts freely while W1 is penalized by Ω(k) . The orange loop arrow marks the increment to task k+2 . Four-step diagram showing ISI-CV continual learning cycle: train task k, collect spike trains, assign importance scores via CV, train task k+1 with regularization.
Method
AA ↑
BWT ↑
AF ↓
No Reg
.982±.004
−.018±.006
.018±.006
EWC
.967±.011
−.037±.014
.037±.014
SI
.993±.001
+.001±.000
.000±.000
ISI-CV
.983±.001
−.000±.000
.000±.000
Table 1. Split-MNIST (5 tasks, multi-head SNN, 3 seeds). Fixed λ : ISI-CV = 500; EWC, SI = 1000. Bold = best per column.
Method
AA ↑
BWT ↑
AF ↓
No Reg
.929±.005
−.052±.005
.052±.005
EWC
.954±.004
−.021±.004
.021±.004
SI
.917±.001
−.003±.001
.003±.001
ISI-CV
.900±.002
−.001±.000
.001±.000
Table 2. Permuted-MNIST (5 tasks, multi-head SNN, 3 seeds). Fixed λ : ISI-CV = 500; EWC, SI = 1000. Bold = best per column.
Figure 2. Average Accuracy (top row) and Average Forgetting (bottom row) for all methods across four benchmarks (mean ± std, 3 seeds). ISI-CV achieves the lowest forgetting on every benchmark and the highest accuracy on FashionMNIST and N-MNIST. Bar charts comparing Average Accuracy and Average Forgetting for No Reg, EWC, SI, and ISI-CV across four benchmarks.
Method
AA ↑
BWT ↑
AF ↓
No Reg
.964±.009
−.034±.010
.034±.010
EWC
.970±.012
−.025±.014
.025±.014
SI
.977±.012
−.017±.016
.017±.016
ISI-CV
.985±.003
−.000±.000
.000±.000
Table 3. Split-FashionMNIST (5 tasks, multi-head SNN, 3 seeds). Fixed λ : ISI-CV = 500; EWC, SI = 1000. Bold = best per column.
Method
AA ↑
BWT ↑
AF ↓
No Reg
.812±.044
−.231±.055
.231±.055
EWC
.748±.042
−.312±.051
.312±.051
SI
.752±.029
−.305±.036
.305±.036
ISI-CV
.820±.012
−.221±.014
.221±.014
Table 4. Split-N-MNIST (5 tasks, convolutional SNN, real DVS data, 3 seeds). Fixed λ : ISI-CV = 100; EWC = 100; SI = 1000. Bold = best per column.
Figure 3. Accuracy–forgetting Pareto analysis on Split-N-MNIST (7 λ values per method, 21 configurations total). (a) Full sweep: large markers show each method’s best λ∗ ; small faded markers show all other tested values. (b) Competitive region: ISI-CV ( λ∗=5000 , AF =0.160 ) dominates; EWC ( λ∗=1000 , AF =0.187 ) is the closest competitor. No EWC or SI configuration achieves both higher accuracy and lower forgetting than ISI-CV simultaneously. Scatter plot showing accuracy versus forgetting for ISI-CV, EWC, and SI across seven lambda values on N-MNIST, with ISI-CV dominating the Pareto frontier.
Figure 4. ISI-CV stability–plasticity tradeoff on Split-MNIST (AF scaled ×10 for visibility). Forgetting stays near zero across λ∈[10,5000] ; average accuracy peaks at λ=500 (AA =0.983 ). The tradeoff is smooth and controlled by a single hyperparameter with no architectural changes. Line plot showing ISI-CV average accuracy and forgetting as a function of lambda on Split-MNIST, with forgetting near zero across three orders of magnitude.
Figure 5. Backward Transfer (BWT) across four benchmarks (mean ± std, 3 seeds). ISI-CV achieves BWT ≥0 on three of four benchmarks. EWC performs worse than No Reg on Split-MNIST and N-MNIST. Horizontal bar chart showing Backward Transfer for all methods across four benchmarks, with ISI-CV reaching zero or positive BWT on three of four benchmarks.
Spiking neural networks process information through discrete, time-dependent neural events, aiming for efficient computation and biologically inspired learning.
Spiking neural networks (SNNs) promise energy-efficient artificial intelligence on neuromorphic hardware but struggle with tasks requiring both fast adaptation and long-term memory, especially in continual learning. We propose Local Timescale Gating (LT-Gate), a neuron model that combines dual time-constant dynamics with an adaptive gating mechanism. Each spiking neuron tracks information on a fast and a slow timescale in parallel, and a learned gate locally adjusts their influence. This design enables individual neurons to preserve slow contextual information while responding to fast signals, addressing the stability-plasticity dilemma. We further introduce a variance-tracking regularization that stabilizes firing activity, inspired by biological homeostasis. Empirically, LT-Gate yields significantly improved accuracy and retention in sequential learning tasks: on a challenging temporal classification benchmark it achieves about 51 percent final accuracy, compared to about 46 percent for a recent Hebbian continual-learning baseline and lower for prior SNN methods. Unlike approaches that require external replay or expensive orthogonalizations, LT-Gate operates with local updates and is fully compatible with neuromorphic hardware. In particular, it leverages features of Intel's Loihi chip (multiple synaptic traces with different decay rates) for on-chip learning. Our results demonstrate that multi-timescale gating can substantially enhance continual learning in SNNs, narrowing the gap between spiking and conventional deep networks on lifelong-learning tasks.
Continual learning enables neural networks to learn tasks sequentially without forgetting previously acquired knowledge. However, neural networks suffer from catastrophic forgetting, where learning new tasks degrades performance on earlier ones. We address this problem with Shapley Neuron Valuation (SNV), a principled framework that quantifies Neuron importance in continual learning, grounded in cooperative game theory. SNV selectively freezes important Neurons while keeping others plastic, enabling buffer-free continual learning without expanding architecture. Experiments on ImageNet-1k show that SNV consistently outperforms existing buffer-free methods. In particular, SNV improves accuracy by +2.88% in the class incremental learning and +6.46% in the task incremental learning scenarios compared to the second baseline.
Mohammad Ali Vahedifar, Abhisek Ray, Qi Zhang
DIGIT and Department of Electrical and Computer Engineering, Aarhus University, Denmark
Although deep neural networks perform extremely well in controlled environments, they fail in real-world scenarios where data isn't available all at once, and the model must adapt to a new data distribution that may or may not follow the initial distribution. Previously acquired knowledge is lost during subsequent updates based on new data. a phenomenon commonly known as catastrophic forgetting. In contrast, the brain can learn without such catastrophic forgetting, irrespective of the number of tasks it encounters. Existing spiking neural networks (SNNs) for class-incremental learning (CIL) suffer a sharp performance drop as tasks accumulate. We here introduce CATFormer (Context Adaptive Threshold Transformer), a scalable framework that overcomes this limitation. We observe that the key to preventing forgetting in SNNs lies not only in synaptic plasticity but also in modulating neuronal excitability. At the core of CATFormer is the Dynamic Threshold Leaky Integrate-and-Fire (DTLIF) neuron model, which leverages context-adaptive thresholds as the primary mechanism for knowledge retention. This is paired with a Gated Dynamic Head Selection (G-DHS) mechanism for task-agnostic inference. Extensive evaluation on both static (CIFAR-10/100/Tiny-ImageNet) and neuromorphic (CIFAR10-DVS/SHD) datasets reveals that CATFormer outperforms existing rehearsal-free CIL algorithms across various task splits, establishing it as an ideal architecture for energy-efficient, true-class incremental learning.
Vaishnavi Nagabhushana, Kartikay Agrawal, Ayon Borthakur