Organizations: University of Illinois Urbana-Champaign, Department of Nuclear, Plasma & Radiological Engineering, Urbana, IL, USA · Indian Institute of Technology Delhi, Department of Applied Mechanics, New Delhi, India · University of Texas at El Paso, Department of Computer Science, El Paso, TX, USA · National Center for Supercomputing Applications, Urbana, IL, USA
Continual learning, the ability to acquire new tasks sequentially without forgetting prior knowledge, is essential for deploying neural networks in dynamic real-world environments, from nuclear digital twin monitoring to grid-edge fault detection. Existing synaptic importance methods, such as Elastic Weight Consolidation (EWC) and Synaptic Intelligence (SI), rely on gradient computation, making them incompatible with neuromorphic hardware that lacks backpropagation support. We propose ISI-CV, the first gradient-free synaptic importance metric for SNN continual learning, derived from the Coefficient of Variation (CV) of Inter-Spike Intervals (ISIs). Neurons that fire regularly (low CV) encode stable, task-relevant features and are protected from overwriting; neurons with irregular firing are permitted to adapt freely. ISI-CV requires only spike time counters and integer arithmetic, all of which are native to every neuromorphic chip. We evaluate on four benchmarks of increasing difficulty: Split-MNIST, Permuted-MNIST, Split-FashionMNIST, and Split-N-MNIST using real Dynamic Vision Sensor (DVS) event data. Across three seeds, ISI-CV achieves zero forgetting (AF = 0.000 +/- 0.000) on Split-MNIST and Split-FashionMNIST, near-zero forgetting on Permuted-MNIST (AF = 0.001 +/- 0.000), and the highest accuracy with the lowest forgetting on real neuromorphic DVS data (AA = 0.820 +/- 0.012, AF = 0.221 +/- 0.014). On N-MNIST, gradient-based methods produce unreliable importance estimates and perform worse than no regularization; ISI-CV avoids this failure by design.
Figures & tables
Figure 1. ISI-CV regularization as a four-step continual learning cycle. (1) The SNN is trained on task k via surrogate-gradient descent. The shared trunk ( W1 +LIF) feeds K per-task heads. (2) After training, a gradient-free inference pass collects hidden spike trains. Neuron i (blue) fires regularly (equal ISIs, low CV); Neuron j (orange) fires in burst–gap bursts (variable ISIs, high CV). (3) ISI-CV importance scores are assigned: Ωi=(CVi+ε)−1 . High- Ω neurons protect W1 (blue bar); low- Ω neurons update freely (orange bar). (4) Task k+1 trains with the regularized loss (Eq. 5 ). The new head (teal) adapts freely while W1 is penalized by Ω(k) . The orange loop arrow marks the increment to task k+2 . Four-step diagram showing ISI-CV continual learning cycle: train task k, collect spike trains, assign importance scores via CV, train task k+1 with regularization.
Method
AA ↑
BWT ↑
AF ↓
No Reg
.982±.004
−.018±.006
.018±.006
EWC
.967±.011
−.037±.014
.037±.014
SI
.993±.001
+.001±.000
.000±.000
ISI-CV
.983±.001
−.000±.000
.000±.000
Table 1. Split-MNIST (5 tasks, multi-head SNN, 3 seeds). Fixed λ : ISI-CV = 500; EWC, SI = 1000. Bold = best per column.
Method
AA ↑
BWT ↑
AF ↓
No Reg
.929±.005
−.052±.005
.052±.005
EWC
.954±.004
−.021±.004
.021±.004
SI
.917±.001
−.003±.001
.003±.001
ISI-CV
.900±.002
−.001±.000
.001±.000
Table 2. Permuted-MNIST (5 tasks, multi-head SNN, 3 seeds). Fixed λ : ISI-CV = 500; EWC, SI = 1000. Bold = best per column.
Figure 2. Average Accuracy (top row) and Average Forgetting (bottom row) for all methods across four benchmarks (mean ± std, 3 seeds). ISI-CV achieves the lowest forgetting on every benchmark and the highest accuracy on FashionMNIST and N-MNIST. Bar charts comparing Average Accuracy and Average Forgetting for No Reg, EWC, SI, and ISI-CV across four benchmarks.
Method
AA ↑
BWT ↑
AF ↓
No Reg
.964±.009
−.034±.010
.034±.010
EWC
.970±.012
−.025±.014
.025±.014
SI
.977±.012
−.017±.016
.017±.016
ISI-CV
.985±.003
−.000±.000
.000±.000
Table 3. Split-FashionMNIST (5 tasks, multi-head SNN, 3 seeds). Fixed λ : ISI-CV = 500; EWC, SI = 1000. Bold = best per column.
Method
AA ↑
BWT ↑
AF ↓
No Reg
.812±.044
−.231±.055
.231±.055
EWC
.748±.042
−.312±.051
.312±.051
SI
.752±.029
−.305±.036
.305±.036
ISI-CV
.820±.012
−.221±.014
.221±.014
Table 4. Split-N-MNIST (5 tasks, convolutional SNN, real DVS data, 3 seeds). Fixed λ : ISI-CV = 100; EWC = 100; SI = 1000. Bold = best per column.
Figure 3. Accuracy–forgetting Pareto analysis on Split-N-MNIST (7 λ values per method, 21 configurations total). (a) Full sweep: large markers show each method’s best λ∗ ; small faded markers show all other tested values. (b) Competitive region: ISI-CV ( λ∗=5000 , AF =0.160 ) dominates; EWC ( λ∗=1000 , AF =0.187 ) is the closest competitor. No EWC or SI configuration achieves both higher accuracy and lower forgetting than ISI-CV simultaneously. Scatter plot showing accuracy versus forgetting for ISI-CV, EWC, and SI across seven lambda values on N-MNIST, with ISI-CV dominating the Pareto frontier.
Figure 4. ISI-CV stability–plasticity tradeoff on Split-MNIST (AF scaled ×10 for visibility). Forgetting stays near zero across λ∈[10,5000] ; average accuracy peaks at λ=500 (AA =0.983 ). The tradeoff is smooth and controlled by a single hyperparameter with no architectural changes. Line plot showing ISI-CV average accuracy and forgetting as a function of lambda on Split-MNIST, with forgetting near zero across three orders of magnitude.
Figure 5. Backward Transfer (BWT) across four benchmarks (mean ± std, 3 seeds). ISI-CV achieves BWT ≥0 on three of four benchmarks. EWC performs worse than No Reg on Split-MNIST and N-MNIST. Horizontal bar chart showing Backward Transfer for all methods across four benchmarks, with ISI-CV reaching zero or positive BWT on three of four benchmarks.
Spiking neural networks process information through discrete, time-dependent neural events, aiming for efficient computation and biologically inspired learning.