cs.NEOct 22, 2026

RobustNCL: Robustifying Neuromorphic Continual Learning under Transient Faults with Lightweight Detect-and-Cure Strategy

Authors: Rachmad Vidya Wicaksana Putra, Shuochen Zhou, Muhammad Shafique

Organizations: eBRAIN Lab, New York University (NYU) Abu Dhabi, United Arab Emirates

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

In neural networks, continual learning (CL) is a prominent approach for enabling deployed systems to adapt to dynamic operational environments by effectively learning from new unseen data at runtime. Recently, the neuromorphic continual learning (NCL) concept has emerged as an energy-efficient alternative to CL by leveraging sparse event-based operations. However, NCL performance can be undermined by transient faults in the hardware platforms, which may come from high-energy particle strikes at any time. Currently, NCL reliability under transient faults has not been studied, and hence the corresponding fault-tolerance techniques remain unexplored. To address this, we propose a novel RobustNCL framework to improve NCL reliability in the presence of transient faults in the DRAM-based off-chip memory of hardware platforms through a lightweight detect-and-cure strategy. Specifically, it first monitors the presence of faults during weight data access from off-chip memory, and then cures the accessed weights if anomalous signatures are detected. It provides three curing variants: replacement with zero, min/max value, and most-probable value from the original/clean weight distribution. To support this, the weight curing values are stored in the radiation-hardened memory. Experimental results show that, in the presence of high fault rates (e.g., 5x10^{-4} for memory replay), RobustNCL maintains high accuracy close to the accuracy of fault-free models (i.e., within -0.5% accuracy for the best RobustNCL variant) and significantly improves accuracy from the baseline without fault mitigation across different NCL methods and datasets (i.e., Split MNIST, Split Fashion-MNIST, Split CIFAR-10, and Split CIFAR-100), while incurring negligible latency and energy overheads. These results highlight that our RobustNCL framework is a promising approach for enabling reliable and energy-efficient NCL systems.

Figures & tables

Figure 1: RobustNCL effectively improves NCL reliability againts transient faults in the DRAM off-chip/main memory under a task-incremental learning (TIL) scenario across different benchmarks: (a) Split MNIST, (b) Split Fashion-MNIST, and (c) Split CIFAR-10, across different NCL methods: task-specific subnetwork (Subnet) and memory replay (MR). The fault rate is denoted as fRf_{R} considering a DDR3-1600 2Gb DRAM chip ( Micron, 2010 ) , RobustNCL variants are with v1-v3 codes.
Figure 2: The RobustNCL framework, showing the transient fault exposure stage, our detect-and-cure (D&C) strategy stage supported by weight replacement (curing) value selection, and the NCL training stage. RobustNCL variants depend on how the faulty weights are replaced by selected curing weights.
Figure 3: (a) The NCL pipeline consists of three phases: the offline training at design time, the NCL training at runtime, and the inference at runtime. (b) DRAM organization overview, including channels, ranks, chips, and banks.
Figure 4: Overview of (a) the task-specific subnetwork (Subnet) method and (b) the memory replay (MR) method that are considered in this paper.
Figure 5: Transient-fault modeling in the DRAM with several key steps: flattening multiple banks into a continuous memory space, injecting transient faults, and curing anomalous weights. Transient fault injection can happen at any time, as illustrated in Fig. 2 .
Figure 6: Experimental results from different NCL methods: (a) Subnet and (b) MR. These results show weight distributions of both the fault-free and faulty models from their layer 2 at certain model states (i.e., after training task 1 and task 2), inference tasks (i.e., task 1 and task 2), and fault rates ( fR=1×10−5f_{R}=1{\times}10^{-5} for Subnet and fR=1×10−4f_{R}=1{\times}10^{-4} for MR).
DatasetTechnique5×10−95{\times}10^{-9}1×10−81{\times}10^{-8}5×10−85{\times}10^{-8}1×10−71{\times}10^{-7}5×10−75{\times}10^{-7}1×10−61{\times}10^{-6}5×10−65{\times}10^{-6}1×10−51{\times}10^{-5}
Split MNISTBaseline99.8499.8390.0576.3851.2850.3251.2851.28
RobustNCLv199.8199.7899.7999.6699.7299.7799.7799.79
RobustNCLv299.8199.7899.8099.6699.7499.7999.8099.82
RobustNCLv399.8199.7899.7999.6699.7699.7699.7299.80
Split FMNISTBaseline99.0189.3666.7957.5450.0050.0050.0050.00
RobustNCLv199.0199.0599.0399.0098.9899.0699.0599.01
Table 1: Final average accuracy (%) obtained by the Subnet method on the 5-task datasets evaluated across all trained tasks (task 1 – task 5) and across different fault-mitigation techniques.
DatasetTechnique1×10−71{\times}10^{-7}5×10−75{\times}10^{-7}1×10−61{\times}10^{-6}5×10−65{\times}10^{-6}1×10−51{\times}10^{-5}5×10−55{\times}10^{-5}1×10−41{\times}10^{-4}5×10−45{\times}10^{-4} 
Split MNISTBaseline99.4099.3799.3899.4798.6151.5051.5051.50 
RobustNCLv199.3399.3199.4199.2299.4099.4199.3799.34 
RobustNCLv299.3999.3599.1999.4499.4999.3099.2899.51 
RobustNCLv399.2999.3299.4999.3899.4099.2399.3999.37 
Split FMNISTBaseline99.0398.9999.0898.8987.3794.1450.0050.00 
RobustNCLv199.0198.9799.0798.9699.0099.0699.0198.31 
Table 2: Final average accuracy (%) obtained by the MR method on the 5-task datasets evaluated across all trained tasks (task 1 – task 5) and across different fault-mitigation techniques.
TechniqueSubnetMR
1×10−81{\times}10^{-8}1×10−71{\times}10^{-7}1×10−61{\times}10^{-6}1×10−51{\times}10^{-5}1×10−81{\times}10^{-8}1×10−71{\times}10^{-7}1×10−61{\times}10^{-6}1×10−51{\times}10^{-5}
Baseline60.0010.0010.7110.0066.6666.6210.0010.00
RobustNCLv170.7370.6170.6170.8267.1166.9166.8466.35
RobustNCLv270.7370.6670.5070.5966.7566.3566.5965.90
RobustNCLv370.7370.5770.6671.0266.6566.6366.2566.82
Table 3: Final average accuracy (%) obtained by both the Subnet and MR methods on the 10-task dataset (Split CIFAR-100) evaluated across all trained tasks (task 1 – task 10).
More in this topic

Neuromorphic Computing

Neuromorphic computing uses brain-inspired, event-driven algorithms and hardware to process information with low power and latency.

See all 192 papers

Explore similar work

CardsList
Oct 22, 2026

ENCL: Enabling Efficient Neuromorphic Continual Learning with Local Learning Rule and Sub-Network Freezing Strategy

Continual learning (CL) in neural networks is a promising learning approach for updating the knowledge of deployed systems based on new data encountered at run time, that have not been learned during the offline training phase. To enable their power/energy-efficient CL processing, neuromorphic continual learning (NCL) concept with sparse event-based operations have recently been introduced. However, state-of-the-art NCL methods typically employ the memory-hungry data/experience replay, coupled with costly global learning rule (i.e., backpropagation-based surrogate gradient). To address these limitations, we propose ENCL, a novel methodology that enables efficient NCL by employing a lightweight local learning rule and sub-network freezing strategy. ENCL leverages the temporally- and spatially-local learning rule to perform efficient credit assignment for training new tasks, alleviating the expensive cost of global learning rule. ENCL also employs a sub-network freezing strategy based on the given tasks to avoid the needs for network architecture expansion, hence further improving the efficiency and scalability of the NCL training process. To efficiently support this, we train each task with a small number of epochs as well as adjusted learning rate and timestep. Experiment results on task-incremental learning (TIL) scenario show that, our ENCL consistently achieves strong performance across different datasets as compared to state-of-the-art works. Specifically, ENCL achieves competitive accuracy comparable to that of the state-of-the-art data/experience replay, while reducing memory cost by up to 88%, NCL training time by up to 51%, average power consumption by up to 40%, and energy consumption by up to 65% across different benchmarking datasets (i.e., Split MNIST, Split Fashion MNIST, Split CIFAR-10, and Split CIFAR-100).

Rachmad Vidya Wicaksana Putra, Shuochen Zhou, Hassan Raza +1

eBRAIN Lab, New York University (NYU) Abu Dhabi, United Arab Emirates

Energy-Efficient MLLocal LearningTask-Incremental Learning
AskPDF
Feb 28, 2026

IDER: IDempotent Experience Replay for Reliable Continual Learning

Catastrophic forgetting, the tendency of neural networks to forget previously learned knowledge when learning new tasks, has been a major challenge in continual learning (CL). To tackle this challenge, CL methods have been proposed and shown to reduce forgetting. Furthermore, CL models deployed in mission-critical settings can benefit from uncertainty awareness by calibrating their predictions to reliably assess their confidences. However, existing uncertainty-aware continual learning methods suffer from high computational overhead and incompatibility with mainstream replay methods. To address this, we propose idempotent experience replay (IDER), a novel approach based on the idempotent property where repeated function applications yield the same output. Specifically, we first adapt the training loss to make model idempotent on current data streams. In addition, we introduce an idempotence distillation loss. We feed the output of the current model back into the old checkpoint and then minimize the distance between this reprocessed output and the original output of the current model. This yields a simple and effective new baseline for building reliable continual learners, which can be seamlessly integrated with other CL approaches. Extensive experiments on different CL benchmarks demonstrate that IDER consistently improves prediction reliability while simultaneously boosting accuracy and reducing forgetting. Our results suggest the potential of idempotence as a promising principle for deploying efficient and trustworthy continual learning systems in real-world applications.Our code is available at https://github.com/YutingLi0606/Idempotent-Continual-Learning.

Zhanwang Liu, Yuting Li, Haoyuan Gao +4

School of Computer Science, Shanghai Jiao Tong University · State Key Laboratory of General Artificial Intelligence, BIGAI · Lehigh University +1

Continual LearningCatastrophic ForgettingReplay-Based Continual Learning
AskPDF
Jul 22, 2026

Leveraging ECRAM for Edge Continual Learning

Several edge computing platforms, such as autonomous vehicles and smart sensing devices, need to adapt to dynamic environments in real time by learning from new data in the field. Continual learning has emerged as a promising solution for edge training, by incorporating techniques that successfully combine a highly summarized version of previously trained data (to avoid catastrophic forgetting) with recently sensed data. However, as is the case with other ML algorithms, continual learning generates significant data movement between general-purpose CPUs/GPUs and memory, impacting the suitability of continual learning for edge platforms. In-memory computing (IMC; also known as processing-using-memory) can curtail this waste and make continual learning feasible at the edge, but it faces two unique challenges: (1) IMC architectures make use of noisy computation operations that significantly harm training accuracy; and (2) IMC architectures have poor and often incomplete support for resource-efficient training. To address these challenges, we propose CLASP (the Continual Learning Acceleration System Platform), which to our knowledge is the first end-to-end system with IMC acceleration for continual learning. The hardware and software of CLASP are co-designed to support a wide range of continual learning algorithms, through software-visible assembly-level instructions that can be incorporated without constraints into ML-based algorithms. CLASP is designed around a back-end-of-line (BEOL) compatible ECRAM device that we fabricate, which can overcome the challenges of IMC-based training using other emerging memory devices. We show that CLASP with ECRAM approaches the accuracy of in-GPU training, while delivering a speedup of 67x and energy savings of 132x for learning without forgetting and experience replay using MNIST.

Nabila Tasnim, Haoran Liu, Qing Cao +1

University of Illinois Urbana-Champaign

Compute-in-MemoryContinual LearningEdge Computing
AskPDF