cs.LGNov 3, 2025
SaveOnline Continual Learning on Intel Loihi 2 via a Co-designed Spiking Neural Network
Organizations: Institute of Informatics, University of Munich (LMU), Munich, Germany · Intel Labs, Intel Corporation, Santa Clara, CA, USA · Southwest Research Institute, San Antonio, TX, USA · Munich Center for Machine Learning (MCML) ·
Abstract
Edge AI must adapt to new classes and shifting data within a tight power budget, but online continual learning (learning from each sample before the next arrives) is where conventional processors are least efficient: each small update pays a full dispatch at full power. We present CLP-SNN, a spiking architecture that recasts the Continually Learning Prototypes (CLP) algorithm as event-driven neural dynamics: a self-normalizing local rule replaces CLP's global renormalization, prototype allocation becomes novelty-triggered neurogenesis, and correctness-modulated plasticity becomes reward-gated per-neuron metaplasticity. Deployed on Intel Loihi 2 as the first online continual learner on that chip, rehearsal-free CLP-SNN matches Replay and SLDA on OpenLORIS at 1-shot; at 25-shot, it stays within 0.8 percentage points of Replay and 6.2 points below covariance-based SLDA. Each prediction-and-update step on extracted features completes on-chip in 0.33 ms at 0.05 mJ, 7.3x lower latency and 341x lower energy than Replay on a Jetson Orin Nano.
Figures & tables
Figure 1: From dense global updates to event-driven local learning on the neuromorphic chip Loihi 2, as an efficient solution to online continual learning. a Catastrophic forgetting happens when learning tasks (e.g., objects) are presented sequentially in non-i.i.d. data streams (adapted from [ 23 ] ). Online continual learning (OCL) is such a setting, where inference and learning occur per sample. b (Left) Global learning with backpropagation (BP) relies on non-local error signals (orange dashed arrows), which must be routed back across layers and lock each layer’s update until they arrive, and it updates all weights at every step (orange flashes). As tracked here for two weights of interest (blue arrows), the resulting learning is spatially and temporally dense, which is especially inefficient in OCL: at batch size 1, every full update serves a single sample. (Middle) Conventional rate-based local learning (Hebbian) replaces the non-local error signal with local activity trace signals confined to a local information zone. While this eliminates cross-layer routing and backward update locking of BP, the updates are still enacted continuously at each time step, keeping the learning spatially and temporally dense. (Right) In event-driven local learning, spike-timing-dependent plasticity (STDP) rules, including the rule of the proposed architecture, CLP-SNN, restrict weight updates to occur only when and where discrete spikes are emitted (e.g., at for the first weight and and for the second). CLP-SNN’s three-factor rule uses only variables local to each synapse, so it can exploit the locality of neuromorphic hardware ( e ), and a selective modulatory signal triggers it, making updates both spatially and temporally sparse. c CLP-SNN realizes the on-demand prototype allocation of the Continually Learning Prototypes (CLP) algorithm as neurogenesis, recruiting new neurons as new concepts are learned, and d CLP’s correctness-modulated learning rate as metaplasticity, which modulates plasticity over time (from very plastic to consolidated) to address catastrophic forgetting. e The Intel Loihi 2 neuromorphic chip, whose neurocores support on-chip, event-driven communication across axons, neurons and synapses, enables local, sparse learning.
Figure 2: The proposed spiking neural network (SNN) architecture for the Continually Learning Prototypes (CLP) algorithm . a The SNN consists of multiple neural populations, where the prototype neurons are central. Each prototype neuron has its own plasticity level, which controls the scale of the update applied to its input synapses. The prototypes compete by time-to-first-spike coding, in which the most similar prototype fires first, and lateral inhibition then silences the others (winner-take-all). For each sample, only one (winner) prototype neuron updates its weights. If no neuron spikes, a novelty-detector neuron flags the input as novel and triggers a modulator neuron, which in turn triggers learning in one of the unallocated prototype neurons. If a prototype neuron wins, its label is the prediction, and the supervisor sends a feedback signal based on the correctness of this inference. The modulator neuron relays this signal to the prototype population to update the winning prototype neuron. b A pictographic view of the feature space and representation of the prototypes in this space. Each class may have a variable number of prototype neurons. c Temporal dynamics of inference, novelty detection, feedback and learning at SNN time-step resolution. The first sample of “laptop” triggers the allocation of a new (transparent) purple prototype neuron, while the second sample updates this neuron’s synapses further. The dynamics are depicted for one of this neuron’s input synapses over the processing of two samples.
Figure 3: Few-shot online continual learning (OCL) experiments on the OpenLORIS dataset. Acronyms: CLP, Continually Learning Prototypes; CLP-SNN, the proposed CLP spiking neural network; NCM, Nearest Class Mean; SLDA, Streaming Linear Discriminant Analysis. In a and b , lines show the mean over three class orders; s.d. of final accuracy is given in Table 1 . a Average accuracy on the classes seen so far during the 1-shot experiment. While the non-continual baselines (grays) fail, OCL algorithms retain accuracy, and CLP outperforms the baselines. The gradual decay of accuracy is a result of the increase in task difficulty (more classes to differentiate among). b 25-shot OCL experiment. Each incremental learning step corresponds to one shot. CLP outperforms all methods except SLDA, which, thanks to its use of the covariance matrix, can leverage the data better, albeit with higher latency and energy costs (see Table 1 ). In both experiments, CLP-SNN slightly underperforms CLP ( Supplementary Note 1 ). c Learning energy efficiency (samples per joule) versus accuracy at the end of the 25-shot experiment, for each algorithm on each platform (Table 1 ); the staircase marks the Pareto frontier. d Learning throughput (samples per second) versus accuracy for the same configurations. e Log–log relationship between learning throughput and energy efficiency. The dashed blue line is a least-squares fit through the conventional hardware configurations (slope = 1.16, = 0.99). The near-unit slope reflects a shared power envelope rather than a constant cost per operation: the operation counts behind these points span two orders of magnitude (Supplementary Table S2 ), but every conventional configuration draws similar power during the step, so energy per sample tracks latency. CLP-SNN on Loihi 2 sits above the line because its per-sample cycle runs at far lower power, not because it performs fewer operations; at the benchmark working point it issues more multiply–accumulate-equivalents per step than CLP on the Orin.
Figure 4: Per-class accuracy retention across all 40 classes and 40 learning steps. Acronyms: OCL, online continual learning; CLP, Continually Learning Prototypes; CLP-SNN, the proposed CLP spiking neural network; FM, forgetting metric. Each heatmap shows , the test accuracy on class after training through step , for one method (seed-averaged), with the black diagonal marking each class’s introduction step. Green indicates high accuracy and red low accuracy. Non-continual baselines (Perceptron FM=0.55, Fine-tuning FM=0.65) lose entire rows of early classes to near-zero accuracy within a few steps. OCL methods retain green cells across most of the triangle (CLP FM=0.11, CLP-SNN FM=0.13). The red diagonals of Replay and Fine-tuning mark classes not yet learned rather than forgotten: both learn each class only partly at its introduction and reach their peak accuracy on it in later steps.
| Method | System | Quantization | Acc (%) (1-shot) | Acc (%) (25-shot) | Latency (ms) | Dynamic energy (mJ) | Total energy (mJ) |
| CLP-SNN | Loihi 2 1 | INT8 | 55.4 2.5 | 90.0 0.4 | 0.33 | 0.01 | 0.05 |
| CLP | GPU 2 | FP32 | 57.0 2.4 | 93.0 0.2 | 1.39 | 1.15 | 9.15 |
| CPU 3 | FP32 | 57.0 2.4 | 93.0 0.2 | 0.54 | 0.41 | 3.52 | |
| NCM | GPU | FP32 | 53.1 2.9 | 84.5 0.8 | 1.60 | 1.79 | 11.02 |
| CPU | FP32 | 53.1 2.9 | 84.5 0.8 | 0.73 | 0.61 | 4.84 | |
| Replay | GPU | FP32 | 55.1 2.8 | 90.8 0.6 | 2.41 | 3.11 | 17.04 |
Table 1: Accuracy and per-sample cost of one online continual learning step. The step comprises producing the prediction for the incoming sample and then updating the model on its label, the test-then-train order in which an online learner meets every sample. It runs at batch size 1, with each method in an efficient implementation of the same step; feature extraction and host–device I/O are excluded on all platforms. For CLP and Replay the update pass already computes the prediction (winner selection and the training forward pass, respectively), so the step is the update alone; for NCM and SLDA the update does not score the sample, so the step is the sum of the two primitives. On Loihi 2 the step is a single on-chip pass. The per-primitive breakdown, the half-precision variants and an explicit-inversion SLDA are reported in Supplementary Table S1 . Accuracies are final values, mean s.d. over three class orders. Best value per column in bold, second best underlined.
| Latency | Dyn. energy | |||
| FP32 | INT8 | FP32 | INT8 | |
| Extractor | 4.8 ms | 2.4 ms | 24.0 mJ | 6.7 mJ |
| Learning-step share | ||||
| NCM | 25% | 40% | 7% | 21% |
| CLP | 22% | 37% | 5% | 15% |
| Replay | 33% | 50% | 11% | 32% |
Table 2: Learning-step share of the per-sample pipeline cost (feature extractor + learning step) on the Jetson Orin Nano GPU, for the FP32 and INT8 EfficientNet-B0 extractors (TensorRT, batch size 1), whose own cost per sample is given in the first row. Step costs are the GPU rows of Table 1 . Dynamic rather than total energy is shown because static power belongs to the device and is shared with any concurrent workload.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Figure S1: Catastrophic forgetting analysis for all OCL methods under the 1-shot class-incremental protocol. a Forgetting Metric (FM) for each method, averaged over all classes and three random seeds (mean std). Lower is better. FM [ 52 ] is defined as the drop from a class’s peak accuracy to its final accuracy, FM (where is the test accuracy on class after training through step , and is the number of steps), and is averaged across all classes . This forgetting measure confounds actual loss of knowledge with an accuracy drop due to the classification task becoming harder as new classes are added over the course of learning. Nevertheless, it is the practically relevant measure in any realistic class-incremental continual learning setting. Non-continual baselines (Perceptron, Fine-tuning) show severe forgetting (FM ); OCL methods cluster between and . b Per-class forgetting heatmap: each cell shows FM j for one method (row) and one class (column, ordered by introduction sequence). Dark cells indicate classes that have been forgotten more. All methods’ forgetting concentrates on early-introduced classes, as expected. However, non-continual baselines exhibit very large forgetting across almost two-thirds of the classes, while retaining knowledge of the last few classes. The prototype-based OCL methods (NCM, CLP and CLP-SNN) show less forgetting and more uniformly light cells, indicating stable retention across all class positions. Replay shows more forgetting than the prototype-based methods. c Temporal evolution of the forgetting metric as the number of seen classes grows. Shaded bands show standard deviation across seeds. Perceptron and Fine-tuning accumulate forgetting rapidly; CLP and CLP-SNN stabilize at low forgetting and track each other closely.
Figure S2: Learning rule convergence and weight-norm stability across input dimensionalities. The first three columns show synthetic clusters at dimensionality , and the rightmost column shows a real OpenLORIS class (class 13). Synthetic clusters are generated with a fixed effective spread: the minimum cosine similarity of any cluster point to its center is held at , matching CLP’s allocation threshold; the same criterion selects 119 of the 180 training frames of the OpenLORIS class. Three rows track three complementary diagnostics over 230 update steps (118 for the OpenLORIS class): (Row 1) cosine similarity of the prototype weight vector to the true cluster center; (Row 2) L2 norm of (ideal ); (Row 3) raw (unnormalized) dot product , which conflates alignment and norm growth. Three variants are compared: CLP (explicit L2 renormalization after each update, black), CLP-SNN Float (Taylor self-normalizing rule, 64-bit, blue) and CLP-SNN INT8 (integer arithmetic of the Loihi 2 learning engine, red dashed). CLP and CLP-SNN Float converge nearly identically across all dimensionalities. The initially higher norm of CLP-SNN Float results from the high initial learning rates of CLP’s learning-rate adaptation, which violate the small-learning-rate assumption of the Taylor expansion. The norm therefore overshoots at first and converges to one as the learning rates decay. CLP-SNN INT8 tracks both variants closely at low dimensionality. At high dimensionality the feature values per dimension are smaller, so once the learning rate has decayed many weight updates fall below one step of the 8-bit weight, and the INT8 prototype settles at a higher norm and a lower cosine similarity, with 5–10% approximation errors. The two errors offset each other in the dot product, the similarity measure the network uses, which converges to one. The OpenLORIS column shows the same behavior on real features.
Figure S3: Sensitivity of the self-normalizing learning rule to intra-class cluster spread . Synthetic clusters are generated at two representative dimensionalities ( , left column; , right column) with isotropic Gaussian spread at four levels . Three rows show the same diagnostics as Supplementary Fig. S2 : (Row 1) cosine similarity to the cluster center; (Row 2) L2 norm evolution; (Row 3) raw dot product . CLP (explicit normalization, black) and CLP-SNN Float (Taylor rule, 64-bit, blue) are robust to all tested values at both dimensionalities: norms remain near unity and cosine similarity converges to even at . CLP-SNN INT8 (Loihi 2 integer arithmetic, red) shows increasing norm deviation and slower convergence as grows, especially at , due to amplified quantization noise in wide clusters. Nevertheless, the raw dot product converges to for all variants and values, indicating that the integer rule retains usable prototype alignment even under high intra-class variance. The OpenLORIS features ( , empirically measured effective – ) lie within the tested range.
Figure S4: Hardware deployment of the CLP-SNN online-learning network on Intel Loihi 2. Core-map schematic rendered from the Lava partitioner output for the network size used for benchmarking. Cores are color-coded by population (input, prototype, novelty detector and modulator), with arrows, labeled with connection names and shapes, showing spike routing between populations and the third-factor feedback path from the supervisor. The 1280-feature 300-prototype CLP-SNN occupies 41 of the 128 neurocores of one Loihi 2 chip; the remaining cores (hatched, right) are idle and available for larger models. Lava’s default partitioner places each of the four neuron populations on its own cores, with the number of cores per population set by the tightest on-core memory bank. Input Neurons (1280 neurons, light green) fit on a single core: the upstream input-injection connection is diagonal (1280 1280, non-plastic), so synapse and learning-trace memory remain almost empty. Prototype Neurons (300 neurons, purple) are sharded across 38 cores at prototypes per core. The dominant constraint is the dense plastic 1280 300 connection: each prototype-hosting core stores the plastic synapses of its prototypes (weight, delay and learning tag) in synapse memory, which shares one memory bank with the table of the 1,581 input axons that every such core receives. The partitioner reduces the number of prototypes per core by an integer factor whenever this bank overflows, which leaves 8 prototypes per core and the prototype cores about half full (49% of synapse memory). The novelty detector and the modulator neuron each occupy one core. The supervisor and the data sources run on the host computer. The supervisor maps the winning prototype to a class label, compares it with the ground-truth label and sends the result back to the modulator, exchanging spikes with the chip through the embedded CPU (24-bit graded spike messages).
Figure S5: The rank-one streaming LDA implementation is accuracy-equivalent to the reference parameterization. a Final accuracy on a held-out class order (seed-10 1-shot stream, class blocks permuted with an independent random generator) as a function of the fixed ridge ; the variation spans 0.6 percentage points across two decades, and was selected here and used everywhere else. b 1-shot and c 25-shot accuracy trajectories of the reference implementation (shrinkage , inverse recomputed at every checkpoint) and of the rank-one implementation ( ), mean over three class orders under the evaluation protocols of Table 1 . Final accuracies: 54.4 2.3% versus 54.8 2.5% (1-shot) and 95.7 0.2% versus 96.2 0.1% (25-shot).
| Latency (ms) | p99 latency (ms) | Dyn. energy (mJ) | |||||||
| Method | Device | Quant. | upd. | qry. | upd. | qry. | upd. | qry. | Step |
| CLP | GPU | FP32 | 1.39 | — | 2.23 | — | 1.15 | — | update |
| CLP | CPU | FP32 | 0.54 | — | 0.83 | — | 0.41 | — | update |
| NCM | GPU | FP32 | 0.94 | 0.66 | 1.57 | 1.10 | 1.06 | 0.73 | update + query |
| NCM | GPU | FP16 | 0.92 | 0.71 | 1.57 | 1.27 | 1.04 | 0.78 | update + query |
| NCM | CPU | FP32 | 0.45 | 0.28 | 0.50 | 0.46 | 0.41 | 0.20 | update + query |
Table S1: Cost primitives of the online continual learning step on the NVIDIA Jetson Orin Nano (batch size 1, feature dimensionality , 40 classes, unified harness). The update primitive is the model update on a labeled sample; the query primitive is scoring one incoming sample. For CLP and Replay the update pass already computes the prediction, so no separate query enters the step (Table 1 ). Latency is the mean over all stream samples; the 99th percentile is given per primitive and is not additive across primitives. The explicit-inversion SLDA rows invert the regularized covariance at every sample; they are a cost anchor that quantifies the gain of the rank-one update, not the paper’s SLDA baseline. Substituting the rank-one update removes 21.4 ms of the 22.9 ms per-sample update on the GPU (94%). In the FP16 rows of the explicit-inversion SLDA the statistics are maintained in FP32 and half precision is applied to the matrix products only ( Supplementary Note 3 ).
| CLP-SNN on Loihi 2 (per sample) | ||||
| Term | Symbolic | Count | ||
| Feedforward spike fan-out (spiking phase) | 169,350 | |||
| Plasticity sweep, prototype synapses (learning phase) | 384,000 | |||
| Control plane (WTA reset, novelty, allocation, feedback) | 2,032 | |||
| Step, algorithm | 555,382 | |||
| Plasticity sweep, lateral-inhibition and third-factor connections | 90,300 | |||
Table S2: Per-sample operation counts at the benchmark working point ( input dimensions, classes, batch size 1, 1-shot stream). One multiply–accumulate (MAC) is charged per scalar product term. Top: inventory for CLP-SNN on Loihi 2 as the Lava-Loihi compiler instantiates the network ( prototype slots, time steps per sample, one learning epoch per sample). A synaptic operation (one synaptic-memory read for an incoming graded spike, with its multiply–accumulate when the weight is non-zero) and one evaluation of the plasticity rule at one synapse each count as one MAC-equivalent. The input layer emits one graded spike per non-zero feature, so the spiking phase costs , where is the number of non-zero input features, measured over the three benchmark streams. Loihi 2 reads every stored weight of a dense connection, zeros included, and skips only the arithmetic for zero weights [ 53 ] , so this term covers every stored synapse of each spiking input. The compiler instantiates every synapse of the dense plastic connection, zeros included, and configures the learning engine to traverse all of them at every epoch, so the learning phase costs whatever the input sparsity; the engine offers pre-synaptic traversal conditions only, so the post-synaptic factor of the rule cannot reduce the sweep either. The lateral-inhibition ( ) and third-factor ( ) connections are also deployed as plastic connections with a rule that changes no weight, because in Lava one neuron cannot have both plastic and non-plastic synapses. The engine sweeps them as well, so they are listed separately as deployment overhead. Dividing the measured dynamic energy per step (0.013 mJ, rounded to 0.01 mJ in Table 1 ) by the deployed count gives about 20 pJ per MAC-equivalent. Bottom: per-step MACs of the conventional learners on the Orin under the composition rule of Table 1 , with their measured step latencies. CLP computes only against its allocated prototypes, so its count is evaluated at the stream-mean allocation of the benchmark stream ( ) and grows with the number of allocated prototypes, as does the chip’s count with the number of instantiated slots; the counts of NCM, Replay and SLDA are fixed. Replay uses samples per update (the incoming sample plus 50 replayed). The explicit-inversion SLDA row is the cost anchor of Supplementary Table S1 . Across the four reported learners a range of operation counts produces a range of GPU step latency and an range on the CPU.
| Parameter | Value | Description |
| / | Effective cosine-similarity threshold (1-shot / 25-shot) | |
| # of SNN time steps for novelty-detector countdown window | ||
| # of SNN time steps per learning epoch (one epoch per input sample) | ||
| Pre-allocated prototype slots (benchmark network; the accuracy runs size the pool per protocol) | ||
| Input feature dimensionality | ||
| Number of classes | OpenLORIS category count |
Table S3: CLP-SNN hyperparameters used in this work. The hardware values of and satisfy the constraint (Eq. 10 of main text); the float and INT8 simulators parameterize this directly via the effective threshold , while the on-chip deployment selects integer and consistent with this constraint.
| Configuration | 1-shot | 25-shot |
| CLP (Table 1 ) | 57.0 (127) | 93.0 (1,769) |
| Imprint-only, 8-bit, CLP’s threshold (0.75) | 56.9 (242) | 92.8 (3,673) |
| + input clipping | 57.2 (210) | 92.5 (3,083) |
| + deployed threshold (0.71 / 0.51) | 55.9 (167) | 90.4 (1,217) |
| + spike-time winner selection | 55.4 (168) | 90.5 (1,302) |
Table S4: Final accuracy (%) and number of prototypes (in parentheses) as the features of the Loihi 2 deployment are added to CLP one at a time, computed with the deployment emulator. Mean over three class orders.
| Similarity threshold | |||||
| Setting | Method | 0.70 | 0.75 | 0.80 | 0.85 |
| 1-shot | CLP | 56.6 (99) | 57.0 (127) | 56.8 (183) | 57.2 (281) |
| Adaptive, FP32 | 55.0 (80) | 56.1 (104) | 56.6 (145) | 56.4 (217) | |
| Adaptive, 8-bit | 48.9 (87) | 51.4 (107) | 52.4 (159) | 54.5 (243) | |
| Imprint-only, 8-bit | 56.4 (191) | 56.9 (242) | 56.6 (328) | 57.2 (455) | |
| 25-shot | CLP | 92.3 (1,127) | 93.0 (1,769) | 93.1 (2,847) | 93.2 (4,831) |
Table S5: Final accuracy (%) and number of prototypes (in parentheses) of CLP and of CLP-SNN in simulation across similarity thresholds. Adaptive: the full CLP-SNN rule; imprint-only: prototypes are not updated after allocation. Mean over three class orders.
More in this topic
Online Continual Learning
Online continual learning adapts models to sequential data streams while preserving useful knowledge and limiting forgetting.
See all 106 papers