Replay-Based Continual Learning

Ask about this topic

Overview

Replay-based continual learning studies how models can learn from a changing stream of tasks without losing what they learned earlier. Methods store and revisit selected examples, compressed features, prototypes, or generated data, often combining replay with distillation, class balancing, or adaptive scheduling. These approaches are used in settings such as class-incremental vision, language-model fine-tuning, robotics, reinforcement learning, and federated learning. Replay is a strong practical way to balance retention and adaptation, but its benefits depend on memory limits, replay quality and timing, and the cost or privacy risks of retaining past data.

Latest papers

CardsList

134 papers20 papers in the last four weeks, up 82% on the four weeks before. 0.1% of all new papers.

May 7, 2026

Beyond Forgetting in Continual Medical Image Segmentation: A Comprehensive Benchmark Study

Continual learning (CL) is essential for deploying medical image segmentation models in clinical environments where imaging domains, anatomical targets, and diagnostic tasks evolve over time. However, continual segmentation still faces three main challenges. First, the scenarios for this task remain insufficiently standardized for real-world clinical settings. Second, existing research has been primarily focused on mitigating forgetting, overlooking the other essential properties such as plasticity. Third, a benchmark work with comprehensive evaluation on existing methods is stll desirable. To address these gaps, we present such benchmark study of continual medical image segmentation. We first define three clinically motivated scenarios, namely Domain-CL, Class-CL, and Organ-CL, to respectively capture the cross-center domain shift, the incremental anatomical structure segmentation, and the cross-organ segmentation. We then introduce an evaluation framework that measures not only general performance and forgetting, but also plasticity, forward generalizability, parameter efficiency, and replay burden. The results, from extensive experiments with representative CL methods, showed that it was still challenging to develop a model that could satisfy all the requirements simultaneously. Nevertheless, these studies also suggested that the replay-based methods achieve the best overall balance between stability and plasticity, the parameter-isolation methods should be effective at reducing forgetting, though at the cost of increased model size, and the forward generalizability remain a significantly understudied aspect of this research field. Finally, we discuss related learning paradigms and outline future directions for continual medical image segmentation.

Bomin Wang, Hangqi Zhou, Yibo Gao +1

School of Data Science, Fudan University, Shanghai, China.

Interactive Medical Image SegmentationStability-Plasticity DilemmaReplay-Based Continual LearningMedical Image SegmentationMedical Image Benchmarks
AskPDF
May 7, 2026

CoMemNet: Contrastive Sampling with Memory Replay Network for Continual Traffic Prediction

In recent years, the integration of non-topological space modeling with temporal learning methods has emerged as an effective approach for capturing spatio-temporal information in non-Euclidean graphs. However, most existing methods rely on static underlying graph structures, which are inadequate for capturing the continuously expanding and evolving patterns in streaming traffic networks. To address this challenge, we propose a simple yet efficient dual-branch continual learning framework for traffic prediction, named CoMemNet. The fast-converging Online branch undertakes the primary prediction tasks, while the momentum-updated Target branch extracts historical information using Wasserstein Distance features to create a Dynamic Contrastive Sampler (DC Sampler). This sampler selects a node set with significant dynamic network feature changes for training, effectively mitigating the issue of catastrophic forgetting. Additionally, the backbone incorporates a lightweight Node-Adaptive Temporal Memory Buffer (TMRB-N) to consolidate old knowledge through memory replay and address the risk of memory explosion. Finally, we provide two newly curated open-source datasets. Experimental results demonstrate that CoMemNet achieves state-of-the-art (SOTA) performance across all three large-scale real-world datasets. The code is available at: https://github.com/meiwu5/CoMemNet.

Mei Wu, Wenchao Weng, Wenxin Su +2

College of Computer Science, Shanghai Jiao Tong University, Shanghai 200240, China · College of Computer Science and Technology, Zhejiang University of Technology, Hangzhou 310023, China · School of Automation, Nanjing University of Science and Technology, Nanjing 210094, China

Contrastive LearningTraffic Flow EstimationDynamic Graph LearningReplay-Based Continual LearningSpatiotemporal Forecasting
AskPDF
May 6, 2026

Replay-Based Continual Learning for Physics-Informed Neural Operators

Neural operators generally demonstrate strong predictive performance on in-distribution (ID) problems. However, a critical limitation of existing methods is their significant performance degradation when encountering out-of-distribution (OOD) data. To address this issue, this work introduces continual learning into physics-informed neural operators, with particular emphasis on neural operators built upon the Transolver architecture, and proposes a simple yet effective replay-based continual learning strategy. The proposed method is fully physics-informed and does not require labeled data, relying solely on input fields together with physical constraints for training. When new OOD data become available, a small number of past data are incorporated through a distillation-based constraint to preserve previously acquired knowledge and alleviate catastrophic forgetting. Meanwhile, a transfer learning LoRA is employed to enable rapid adaptation to the new data. The proposed framework is systematically validated on three representative physical problems, including the Darcy flow problem in fluid mechanics, a two-dimensional hyperelastic brain tumor problem in biomechanics, and a three-dimensional linear elastic Triply Periodic Minimal Surfaces problem in solid mechanics. The results demonstrate that the proposed method effectively mitigates catastrophic forgetting on previously learned data while maintaining fast adaptability to new data. Compared with conventional joint training strategies, the proposed method significantly improves training efficiency while reducing additional memory usage and computational cost.

Yizheng Wang, Mohammad Sadegh Eshaghi, Xiaoying Zhuang +2

Department of Engineering Mechanics, Tsinghua University, Beijing 100084, China · Institute of Photonics, Department of Mathematics and Physics, Leibniz University Hannover, Germany · Institute of Structural Mechanics, Bauhaus-Universität Weimar, Marienstr. 15, D-99423 Weimar, Germany

OOD GeneralizationReplay-Based Continual LearningPhysics-Informed Neural OperatorsNeural Operators
AskPDF
May 5, 2026

Memory-Efficient Continual Learning with CLIP Models

Contrastive Language-Image Pretraining (CLIP) models excel at understanding image-text relationships but struggle with adapting to new data without forgetting prior knowledge. To address this, models are typically fine-tuned using both new task data and a memory buffer of past tasks. However, CLIP's contrastive loss suffers when the memory buffer is small, leading to performance degradation on previous tasks. We propose a memory-efficient, distributionally robust method that dynamically reweights losses per class during training. Our approach, tested on class incremental settings (CIFAR-100, ImageNet1K) and a domain incremental setting (DomainNet) adapts CLIP models quickly while minimizing catastrophic forgetting, even with minimal memory usage.

Ryan King, Gang Li, Bobak Mortazavi +1

Department of Computer Science Texas A&M University College Station, TX 77843 · Texas A&M University College Station, TX 77843

Class-Incremental LearningContrastive LearningVision-Language ModelsVLM AdaptationReplay-Based Continual LearningContinual Learning for VLMs
AskPDF
May 5, 2026

A Domain Incremental Continual Learning Benchmark for ICU Time Series Model Transportability

In recent years, machine learning has made significant progress in clinical outcome prediction, demonstrating increasingly accurate results. However, the substantial resources required for hospitals to train these models, such as data collection, labeling, and computational power, limit the feasibility for smaller hospitals to develop their own models. An alternative approach involves transferring a machine learning model trained by a large hospital to smaller hospitals, allowing them to fine-tune the model on their specific patient data. However, these models are often trained and validated on data from a single hospital, raising concerns about their generalizability to new data. Our research shows that there are notable differences in measurement distributions and frequencies across various regions in the United States. To address this, we propose a benchmark that tests a machine learning model's ability to transfer from a source domain to different regions across the country. This benchmark assesses a model's capacity to learn meaningful information about each new domain while retaining key features from the original domain. Using this benchmark, we frame the transfer of a machine learning model from one region to another as a domain incremental learning problem. While the task of patient outcome prediction remains the same, the input data distribution varies, necessitating a model that can effectively manage these shifts. We evaluate two popular domain incremental learning methods: data replay, which stores examples from previous data sources for fine-tuning on the current source, and Elastic Weight Consolidation (EWC), a model parameter regularization method that maintains features important for both data sources.

Ryan King, Conrad Krueger, Ethan Veselka +2

Computer Science & Engineering, Texas A&M University, College Station, United States

Clinical Outcome PredictionDomain-Incremental LearningReplay-Based Continual LearningCross-Dataset Generalization
AskPDF
May 4, 2026

Adaptive Data Compression and Reconstruction for Memory-Bounded EEG Continual Learning

Electroencephalography (EEG) signals provide millisecond-level temporal resolution but their analysis is limited by remarkable noise and inter-subject variability, making robust personalization difficult under limited annotations. Unsupervised Individual Continual Learning (UICL) has been proposed to address this practical challenge, where a model pretrained on a labeled cohort must adapt online to unlabeled subject streams under strict memory constraints. However, existing UICL methods typically store full past samples, which undermine the continual learning goal of avoiding retraining. Observing that EEG signals exhibit well-structured morphologies to be exploited via morphology-aware selection, compression, and reconstruction, here we propose Adaptive Data Compression and Reconstruction (ADaCoRe) for UICL. This is a memory-efficient pipeline composed of saliency-driven keyframe protection, rational polyphase compression, adjoint reconstruction with verbatim overwrite on protected indices, and prototype-confidence selection for adaptive exemplar maintenance. Across three representative benchmarks, ADaCoRe consistently outperforms recent strong baselines under tight buffer regimes (eg., the performance gains are at least +2.7 and +15.3 ACC on ISRUC and FACED datasets, respectively). Ablation studies quantify compression-fidelity trade-offs and highlight the contribution of each design, while visualizations confirm the preservation of key EEG morphology during compression and reconstruction.

Chengcheng Xie

Beihang University

ElectroencephalographyContinual LearningReplay-Based Continual Learning
AskPDF
May 1, 2026

Continual Learning of Feedback-based Molecular Communication

This paper proposes and evaluates a new performance estimation method that leverages continual learning (CL) algorithms to carry out sequential simulation experiments for a feedback-based molecular communication protocol. As the protocol is sequentially examined in various experimental settings, the proposed CL-based performance estimators incrementally learn a series of unexperienced estimation tasks without compromising those that have been learned in the past. They are designed to work on a standard neural network architecture by customizing regularization and replay strategies in the loss function. Experimental results demonstrate that the proposed estimators can effectively learn on a continuous stream of simulation results and enhance the baseline neural network by improving estimation accuracy at a variety of computational costs. This paper's contribution is to establish the implications of CL in the field of molecular communication.

Siddhant Setia, Junichi Suzuki, Tadashi Nakano

Department of Computer Science, University of Massachusetts, Boston Boston, MA, 02125, USA · Graduate School of Informatics, Osaka Metropolitan University Osaka, Japan

Continual LearningReplay-Based Continual Learning
AskPDF
Apr 27, 2026

Continual Calibration: Coverage Can Collapse Before Accuracy in Lifelong LLM Fine-Tuning

Continual learning for large language models is typically evaluated through accuracy retention under sequential fine-tuning. We argue that this perspective is incomplete, because uncertainty reliability can degrade earlier and more sharply than top-1 performance. We study this empirically by measuring conformal coverage and calibration error on sequentially fine-tuned models across three model families and eight task sequences drawn primarily from classification and multiple-choice benchmarks. Across the classification-style settings we study, coverage loss exceeds accuracy loss by a factor of roughly 3.4×±0.5×3.4\times \pm 0.5\times on average across seeds; in the most pronounced case, coverage drops from 0.920.92 to 0.610.61, while accuracy remains within three points of baseline. Standard continual-learning methods that preserve accuracy do not automatically preserve coverage, and naive calibration baselines recover only part of the gap. We propose calibration replay, a lightweight post-hoc procedure that maintains a task-specific held-out buffer and refits a task-specific conformal threshold under the current model after each update. It adds no training-time gradient cost, uses less than one percent of the memory of ordinary experience replay, and typically restores coverage to within two points of nominal at buffer size m=200m = 200. We accompany the empirical study with a drift decomposition, a finite-sample recovery theorem showing exact conformal validity under exchangeability, and a mixture-validity proposition explaining why pooled thresholds do not suffice. Our guarantees are stated for classification-style tasks with task-specific buffers; extensions to open-ended generation are exploratory.

Ibne Farabi Shihab, Sanjeda Akter, Anuj Sharma

Department of Computer Science, Iowa State University · Department of Civil, Construction & Environmental Engineering, Iowa State University

Continual Learning for LLMsConformal PredictionLanguage Model CalibrationLLM ReliabilityReplay-Based Continual Learning
AskPDF
Apr 22, 2026

Lifecycle-Aware Federated Continual Learning in Mobile Autonomous Systems

Federated continual learning (FCL) allows distributed autonomous fleets to adapt collaboratively to evolving terrain types across extended mission lifecycles. However, current approaches face several key challenges: 1) they use uniform protection strategies that do not account for the varying sensitivities to forgetting on different network layers; 2) they focus primarily on preventing forgetting during training, without addressing the long-term effects of cumulative drift; and 3) they often depend on idealized simulations that fail to capture the real-world heterogeneity present in distributed fleets. In this paper, we propose a lifecycle-aware dual-timescale FCL framework that incorporates training-time (pre-forgetting) prevention and (post-forgetting) recovery. Under this framework, we design a layer-selective rehearsal strategy that mitigates immediate forgetting during local training, and a rapid knowledge recovery strategy that restores degraded models after long-term cumulative drift. We present a theoretical analysis that characterizes heterogeneous forgetting dynamics and establishes the inevitability of long-term degradation. Our experimental results show that this framework achieves up to 8.3% mIoU improvement over the strongest federated baseline and up to 31.7% over conventional fine-tuning. We also deploy the FCL framework on a real-world rover testbed to assess system-level robustness under realistic constraints; the testing results further confirm the effectiveness of our FCL design.

Beining Wu, Jun Huang

Department of Electrical Engineering and Computer Science, South Dakota State University, Brookings, 57006 SD

Mobile RoboticsFederated Continual LearningReplay-Based Continual LearningFederated LearningCatastrophic Forgetting
AskPDF
Apr 19, 2026

Recovery Guarantees for Continual Learning of Dependent Tasks: Memory, Data-Dependent Regularization, and Data-Dependent Weights

Continual learning (CL) is concerned with learning multiple tasks sequentially without forgetting previously learned tasks. Despite substantial empirical advances over recent years, the theoretical development of CL remains in its infancy. At the heart of developing CL theory lies the challenge that the data distribution varies across tasks, and we argue that properly addressing this challenge requires understanding this variation--dependency among tasks. To explicitly model task dependency, we consider nonlinear regression tasks and propose the assumption that these tasks are dependent in such a way that the data of the current task is a nonlinear transformation of previous data. With this model and under natural assumptions, we prove statistical recovery guarantees (more specifically, bounds on estimation errors) for several CL paradigms in practical use, including experience replay with data-independent regularization and data-independent weights that balance the losses of tasks, replay with data-dependent weights, and continual learning with data-dependent regularization (e.g., knowledge distillation). To the best of our knowledge, our bounds are informative in cases where prior work gives vacuous bounds.

Liangzu Peng, Uday Kiran Reddy Tadipatri, Ziqing Xu +2

University of Pennsylvania

Continual LearningReplay-Based Continual LearningStatistical Learning Theory
AskPDF
Apr 17, 2026

Continual Hand-Eye Calibration for Open-world Robotic Manipulation

Hand-eye calibration through visual localization is a critical capability for robotic manipulation in open-world environments. However, most deep learning-based calibration models suffer from catastrophic forgetting when adapting into unseen data amongst open-world scene changes, while simple rehearsal-based continual learning strategy cannot well mitigate this issue. To overcome this challenge, we propose a continual hand-eye calibration framework, enabling robots to adapt to sequentially encountered open-world manipulation scenes through spatially replay strategy and structure-preserving distillation. Specifically, a Spatial-Aware Replay Strategy (SARS) constructs a geometrically uniform replay buffer that ensures comprehensive coverage of each scene pose space, replacing redundant adjacent frames with maximally informative viewpoints. Meanwhile, a Structure-Preserving Dual Distillation (SPDD) is proposed to decompose localization knowledge into coarse scene layout and fine pose precision, and distills them separately to alleviate both types of forgetting during continual adaptation. As a new manipulation scene arrives, SARS provides geometrically representative replay samples from all prior scenes, and SPDD applies structured distillation on these samples to retain previously learned knowledge. After training on the new scene, SARS incorporates selected samples from the new scene into the replay buffer for future rehearsal, allowing the model to continuously accumulate multi-scene calibration capability. Experiments on multiple public datasets show significant anti scene forgetting performance, maintaining accuracy on past scenes while preserving adaptation to new scenes, confirming the effectiveness of the framework.

Fazeng Li, Gan Sun, Chenxi Liu +3

School of Automation Science and Engineering, South China University of Technology, Guangzhou 510641, China · State Key Laboratory of Robotics, the Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang 110169, China · University of Chinese Academy of Sciences, Beijing 100049, China

Hand-Eye CalibrationCamera Pose EstimationRobotic ManipulationReplay-Based Continual Learning
AskPDF
Apr 15, 2026

Continual Learning for fMRI-Based Brain Disorder Diagnosis via Functional Connectivity Matrices Generative Replay

Functional magnetic resonance imaging (fMRI) is widely used for studying and diagnosing brain disorders, with functional connectivity (FC) matrices providing powerful representations of large-scale neural interactions. However, existing diagnostic models are trained either on a single site or under full multi-site access, making them unsuitable for real-world scenarios where clinical data arrive sequentially from different institutions. This results in limited generalization and severe catastrophic forgetting. This paper presents the first continual learning framework specifically designed for fMRI-based diagnosis across heterogeneous clinical sites. Our framework introduces a structure-aware variational autoencoder that synthesizes realistic FC matrices for both patient and control groups. Built on this generative backbone, we develop a multi-level knowledge distillation strategy that aligns predictions and graph representations between new-site data and replayed samples. To further enhance efficiency, we incorporate a hierarchical contextual bandit scheme for adaptive replay sampling. Experiments on multi-site datasets for major depressive disorder (MDD), schizophrenia (SZ), and autism spectrum disorder (ASD) show that the proposed generative model enhances data augmentation quality, and the overall continual learning framework substantially outperforms existing methods in mitigating catastrophic forgetting. Our code is available at https://github.com/4me808/FORGE.

Qianyu Chen, Shujian Yu

†Nanyang Technological University · ‡VU Amsterdam · §UiT The Arctic University of Norway

Continual LearningReplay-Based Continual LearningNeuroimagingVariational AutoencodersFunctional Connectivity
AskPDF
Apr 14, 2026

Direct Discrepancy Replay: Distribution-Discrepancy Condensation and Manifold-Consistent Replay for Continual Face Forgery Detection

Continual face forgery detection (CFFD) requires detectors to learn emerging forgery paradigms without forgetting previously seen manipulations. Existing CFFD methods commonly rely on replaying a small amount of past data to mitigate forgetting. Such replay is typically implemented either by storing a few historical samples or by synthesizing pseudo-forgeries from detector-dependent perturbations. Under strict memory budgets, the former cannot adequately cover diverse forgery cues and may expose facial identities, while the latter remains strongly tied to past decision boundaries. We argue that the core role of replay in CFFD is to reinstate the distributions of previous forgery tasks during subsequent training. To this end, we directly condense the discrepancy between real and fake distributions and leverage real faces from the current stage to perform distribution-level replay. Specifically, we introduce Distribution-Discrepancy Condensation (DDC), which models the real-to-fake discrepancy via a surrogate factorization in characteristic-function space and condenses it into a tiny bank of distribution discrepancy maps. We further propose Manifold-Consistent Replay (MCR), which synthesizes replay samples through variance-preserving composition of these maps with current-stage real faces, yielding samples that reflect previous-task forgery cues while remaining compatible with current real-face statistics. Operating under an extremely small memory budget and without directly storing raw historical face images, our framework consistently outperforms prior CFFD baselines and significantly mitigates catastrophic forgetting. Replay-level privacy analysis further suggests reduced identity leakage risk relative to selection-based replay.

Tianshuo Zhang, Haoyuan Zhang, Siran Peng +3

SAI, UCAS · MAIS, CASIA · IIE, CAS +2

Deepfake DetectionReplay-Based Continual Learning
AskPDF
Apr 12, 2026

Preventing Latent Rehearsal Decay in Online Continual SSL with SOLAR

This paper explores Online Continual Self-Supervised Learning (OCSSL), a scenario in which models learn from continuous streams of unlabeled, non-stationary data, where methods typically employ replay and fast convergence is a central desideratum. We find that OCSSL requires particular attention to the stability-plasticity trade-off: stable methods (e.g. replay with Reservoir sampling) are able to converge faster compared to plastic ones (e.g. FIFO buffer), but incur in performance drops under certain conditions. We explain this collapse phenomenon with the Latent Rehearsal Decay hypothesis, which attributes it to latent space degradation under excessive stability of replay. We introduce two metrics (Overlap and Deviation) that diagnose latent degradation and correlate with accuracy declines. Building on these insights, we propose SOLAR, which leverages efficient online proxies of Deviation to guide buffer management and incorporates an explicit Overlap loss, allowing SOLAR to adaptively managing plasticity. Experiments demonstrate that SOLAR achieves state-of-the-art performance on OCSSL vision benchmarks, with both high convergence speed and final performance.

Giacomo Cignoni, Simone Magistri, Andrew D. Bagdanov +1

Department of Computer Science, University of Pisa, Pisa, Italy · Media Integration and Communication Center (MICC), University of Florence, Italy

Online Continual LearningSelf-Supervised LearningReplay-Based Continual LearningStability-Plasticity DilemmaSelf-Supervised Visual Representation Learning
AskPDF
Apr 10, 2026

Towards Lifelong Aerial Autonomy: Geometric Memory Management for Continual Visual Place Recognition in Dynamic Environments

Robust geo-localization under changing environmental and operational conditions is critical for long-term aerial autonomy. Aerial visual place recognition (VPR) commonly uses pre-acquired remote-sensing imagery of the intended operating area, so the geographic label space can remain fixed while successive airborne missions introduce substantial visual distribution shifts. Continual adaptation to these shifts can cause catastrophic forgetting. We therefore formulate aerial VPR as a mission-based domain-incremental learning (DIL) problem and develop a heterogeneous memory framework. Before sequential adaptation, the satellite reference dataset is used once to train the initial model and construct a static satellite exemplar memory; a bounded replay buffer then retains selected airborne observations across missions. For replay management, we compare loss- and diversity-based selection criteria and introduce DBS-Hybrid, which combines prototype-based diversity trimming with representative-first feature-space coverage. Experiments on 21 visible and infrared UAV missions evaluate generalization to held-out missions, immediate adaptation, and knowledge retention. Under the primary Forward mission order, DBS-Hybrid achieves the highest mean final average accuracy, generalization, and knowledge retention among the evaluated methods, improving over the Random baseline by 5.065.06, 5.325.32, and 6.336.33 percentage points, respectively, and improving backward transfer from -6.41% to 1.07%. Across five additional random mission orders, DBS-Hybrid ranks second in mean final average accuracy, backward transfer, generalization, and knowledge retention. Overall, heterogeneous memory and diversity-aware replay provide an effective basis for continual aerial VPR in mapped operating areas.

Xingyu Shao, Zhiqiang Yan, Liangzheng Sun +5

Cross-View Geo-LocalizationVisual Place RecognitionDomain-Incremental LearningReplay-Based Continual Learning
AskPDF
Apr 9, 2026

Leveraging Complementary Embeddings for Replay Selection in Continual Learning with Small Buffers

Catastrophic forgetting remains a key challenge in Continual Learning (CL). In replay-based CL with severe memory constraints, performance critically depends on the sample selection strategy for the replay buffer. Most existing approaches construct memory buffers using embeddings learned under supervised objectives. However, class-agnostic, self-supervised representations often encode rich, class-relevant semantics that are overlooked. We propose a new method, Multiple Embedding Replay Selection, MERS, which replaces the buffer selection module with a graph-based approach that integrates both supervised and self-supervised embeddings. Empirical results show consistent improvements over SOTA selection strategies across a range of continual learning algorithms, with particularly strong gains in low-memory regimes. On CIFAR-100 and TinyImageNet, MERS outperforms single-embedding baselines without adding model parameters or increasing replay volume, making it a practical, drop-in enhancement for replay-based continual learning.

Danit Yanowsky, Daphna Weinshall

School of Computer Science and Engineering The Hebrew University of Jerusalem

Experience ReplayReplay-Based Continual LearningCatastrophic Forgetting
AskPDF
Apr 9, 2026

From Selection to Scheduling: Federated Geometry-Aware Correction Makes Exemplar Replay Work Better under Continual Dynamic Heterogeneity

Exemplar replay has become an effective strategy for mitigating catastrophic forgetting in federated continual learning (FCL) by retaining representative samples from past tasks. Existing studies focus on designing sample-importance estimation mechanisms to identify information-rich samples. However, they typically overlook strategies for effectively utilizing the selected exemplars, which limits their performance under continual dynamic heterogeneity across clients and tasks. To address this issue, this paper proposes a Federated gEometry-Aware correcTion method, termed FEAT, which alleviates imbalance-induced representation collapse that drags rare-class features toward frequent classes across clients. Specifically, it consists of two key modules: 1) the Geometric Structure Alignment module performs structural knowledge distillation by aligning the pairwise angular similarities between feature representations and their corresponding Equiangular Tight Frame prototypes, which are fixed and shared across clients to serve as a class-discriminative reference structure. This encourages geometric consistency across tasks and helps mitigate representation drift; 2) the Energy-based Geometric Correction module removes task-irrelevant directional components from feature embeddings, which reduces prediction bias toward majority classes. This improves sensitivity to minority classes and enhances the model's robustness under class-imbalanced distributions.

Zhuang Qi, Ying-Peng Tang, Lei Meng +4

School of Software, Shandong University, China · College of Computing and Data Science, Nanyang Technological University, Singapore · School of Computer Science and Technology, Harbin Institute of Technology, China

Federated Continual LearningReplay-Based Continual LearningHeterogeneous Federated LearningClass-Imbalanced Learning
AskPDF
Mar 31, 2026

Temporal Memory for Resource-Constrained Agents: Continual Learning via Stochastic Compress-Add-Smooth

An agent that operates sequentially must incorporate new experience without forgetting old experience, under a fixed memory budget. We propose a framework in which memory is not a parameter vector but a stochastic process: a Bridge Diffusion on a replay interval [0,1][0,1], whose terminal marginal encodes the present and whose intermediate marginals encode the past. New experience is incorporated via a three-step Compress--Add--Smooth (CAS) recursion. We test the framework on the class of models with marginal probability densities modeled via Gaussian mixtures of fixed number of components~KK in dd dimensions; temporal complexity is controlled by a fixed number~LL of piecewise-linear protocol segments whose nodes store Gaussian-mixture states. The entire recursion costs O(LKd2)O(LKd^2) flops per day -- no backpropagation, no stored data, no neural networks -- making it viable for controller-light hardware. Forgetting in this framework arises not from parameter interference but from lossy temporal compression: the re-approximation of a finer protocol by a coarser one under a fixed segment budget. We find that the retention half-life scales linearly as a1/2≈c La_{1/2}\approx c\,L with a constant c>1c>1 that depends on the dynamics but not on the mixture complexity~KK, the dimension~dd, or the geometry of the target family. The constant~cc admits an information-theoretic interpretation analogous to the Shannon channel capacity. The stochastic process underlying the bridge provides temporally coherent movie'' replay -- compressed narratives of the agent's history, demonstrated visually on an MNIST latent-space illustration. The framework provides a fully analytical Ising model'' of continual learning in which the mechanism, rate, and form of forgetting can be studied with mathematical precision.

Michael Chertkov

Graduate Inter-Disciplinary Program in Applied Mathematics, & Department of Mathematics, University Arizona, Tucson, AZ 85721

Memory CompressionContinual LearningStorage-Efficient Continual LearningCatastrophic ForgettingReplay-Based Continual Learning
AskPDF
Mar 31, 2026

APEX-EM: Non-Parametric Online Learning for Autonomous Agents via Structured Procedural-Episodic Experience Replay

LLM agents rerun full reasoning for every task, even one they solved moments earlier. We introduce APEX-EM, a non-parametric experience memory that stores complete procedural-episodic traces in a typed Procedural Knowledge Graph (PKG) and retrieves them through three channels: semantic search, structural-signature matching over abstract operation sequences, and graph traversal. A Plan-Retrieve-Generate-Iterate-Ingest (PRGII) workflow produces, quality-gates, and commits experiences, indexing both successes and failures so the agent learns what to reuse and what to avoid. No weights change during deployment. We evaluate on five benchmarks: BigCodeBench, KGQAGen-10k, HLE, Lifelong Agent Bench, and ALFWorld. Because prior work uses different backbones, we base our claims on same-backbone comparisons that hold model capability fixed. On held-out BigCodeBench transfer with a shared GPT-4o backbone, APEX-EM gains +7.6,pp over the no-memory baseline, 3.3×3.3\times MemRL's +2.3,pp under the identical setup. On Lifelong Agent Bench with a shared GPT-4o-mini backbone, it gains +1.4,pp (OS) and +1.0,pp (DB) cumulative success. On KGQAGen-10k, frozen memory transfers to a blind 1{,}079-question test split at 73.7% versus 42.0% with no memory, approaching an oracle handed the ground-truth subgraph (84.9%). Across three Opus scales the memory gain stays at +27 to +32,pp, so it adds to model capability rather than substituting for it. Component analysis shows no single mechanism dominates: teacher feedback is negligible for code but adds +10.3,pp on structured queries, structural signatures give 3.3×3.3\times the transfer of semantic-only retrieval, and within-epoch iteration recovers most of the gain when rich feedback is unavailable. These results argue for modular memory composed per domain.

Pratyay Banerjee, Masud Moshtaghi, Ankit Chadha

Amazon, AGI / Sunnyvale, USA

Replay-Based Continual LearningPersistent Memory for Language ModelsGraph-Based Agent MemoryContinual Learning for LLM AgentsMemory-Augmented Language Model AgentsAgent Memory Management+1
AskPDF
Mar 30, 2026

FeDMRA: Federated Incremental Learning with Dynamic Memory Replay Allocation

In federated healthcare systems, Federated Class-Incremental Learning (FCIL) has emerged as a key paradigm, enabling continuous adaptive model learning among distributed clients while safeguarding data privacy. However, in practical applications, data across agent nodes within the distributed framework often exhibits non-independent and identically distributed (non-IID) characteristics, rendering traditional continual learning methods inapplicable. To address these challenges, this paper covers more comprehensive incremental task scenarios and proposes a dynamic memory allocation strategy for exemplar storage based on the data replay mechanism. This strategy fully taps into the inherent potential of data heterogeneity, while taking into account the performance fairness of all participating clients, thereby establishing a balanced and adaptive solution to mitigate catastrophic forgetting. Unlike the fixed allocation of client exemplar memory, the proposed scheme emphasizes the rational allocation of limited storage resources among clients to improve model performance. Furthermore, extensive experiments are conducted on three medical image datasets, and the results demonstrate significant performance improvements compared to existing baseline models.

Tiantian Wang, Xiang Xiang, Simon S. Du

Huazhong University of Science and Technology, Wuhan, China · University of Washington, Seattle, USA

Federated Continual LearningFederated LearningReplay-Based Continual LearningNon-IID Federated Learning
AskPDF
Mar 19, 2026

A Comparative Empirical Study of Catastrophic Forgetting Mitigation in Sequential Task Adaptation for Continual Natural Language Processing Systems

Neural language models deployed in real-world applications must continually adapt to new tasks and domains without forgetting previously acquired knowledge. This work presents a comparative empirical study of catastrophic forgetting mitigation in continual intent classification. Using the CLINC150 dataset, we construct a 10-task label-disjoint scenario and evaluate three backbone architectures: a feed-forward Artificial Neural Network (ANN), a Gated Recurrent Unit (GRU), and a Transformer encoder, under a range of continual learning (CL) strategies. We consider one representative method from each major CL family: replay-based Maximally Interfered Retrieval (MIR), regularization-based Learning without Forgetting (LwF), and parameter-isolation via Hard Attention to Task (HAT), both individually and in all pairwise and triple combinations. Performance is assessed with average accuracy, macro F1, and backward transfer, capturing the stability-plasticity trade-off across the task sequence. Our results show that naive sequential fine-tuning suffers from severe forgetting for all architectures and that no single CL method fully prevents it. Replay emerges as a key ingredient: MIR is the most reliable individual strategy, and combinations that include replay (MIR+HAT, MIR+LwF, MIR+LwF+HAT) consistently achieve high final performance with near-zero or mildly positive backward transfer. The optimal configuration is architecture-dependent. MIR+HAT yields the best result for ANN and Transformer, MIR+LwF+HAT, on the other hand, works the best for GRU, and in several cases CL methods even surpass joint training, indicating a regularization effect. These findings highlight the importance of jointly selecting backbone architecture and CL mechanism when designing continual intent-classification systems.

Aram Abrahamyan, Sachin Kumar

Zaven P. and Sonia Akian College of Science and Engineering, American University of Armenia, Yerevan 0019, Armenia

Catastrophic ForgettingReplay-Based Continual LearningIntent Classification
AskPDF
Mar 17, 2026

MedCL-Bench: Benchmarking stability-efficiency trade-offs and scaling in biomedical continual learning

Medical language models must be updated as evidence and terminology evolve, yet sequential updating can trigger catastrophic forgetting. Although biomedical NLP has many static benchmarks, no unified, task-diverse benchmark exists for evaluating continual learning under standardized protocols, robustness to task order and compute-aware reporting. We introduce MedCL-Bench, which streams ten biomedical NLP datasets spanning five task families and evaluates eleven continual learning strategies across eight task orders, reporting retention, transfer, and GPU-hour cost. Across backbones and task orders, direct sequential fine-tuning on incoming tasks induces catastrophic forgetting, causing update-induced performance regressions on prior tasks. Continual learning methods occupy distinct retention-compute frontiers: parameter-isolation provides the best retention per GPU-hour, replay offers strong protection at higher cost, and regularization yields limited benefit. Forgetting is task-dependent, with multi-label topic classification most vulnerable and constrained-output tasks more robust. MedCL-Bench provides a reproducible framework for auditing model updates before deployment.

Min Zeng, Shuang Zhou, Zaifu Zhan +1

Division of Computational Health Sciences, Department of Surgery, University of Minnesota, Minneapolis, 55455, MN, USA.

Continual LearningContinual Learning for LLMsCatastrophic ForgettingReplay-Based Continual Learning
AskPDF
Mar 14, 2026

Memory-efficient Continual Learning with Prototypical Exemplar Condensation

Rehearsal-based continual learning (CL) mitigates catastrophic forgetting by maintaining a subset of samples from previous tasks for replay. Existing studies primarily focus on optimizing memory storage through coreset selection strategies. While these methods are effective, they typically require storing a substantial number of samples per class (SPC), often exceeding 20, to maintain satisfactory performance. In this work, we propose to further compress the memory footprint by synthesizing and storing prototypical exemplars, which can form representative prototypes when passed through the feature extractor. Owing to their representative nature, these exemplars enable the model to retain previous knowledge using only a small number of samples while preserving privacy. Moreover, we introduce a perturbation-based augmentation mechanism that generates synthetic variants of previous data during training, thereby enhancing CL performance. Extensive evaluations on widely used benchmark datasets and settings demonstrate that the proposed algorithm achieves superior performance compared to existing baselines, particularly in scenarios involving large-scale datasets and a high number of tasks.

Minh-Duong Nguyen, Thien-Thanh Dao, Le-Tuan Nguyen +2

College of Engineering and Computer Science, VinUniversity, Hanoi, Vietnam · Center for AI Research, VinUniversity, Hanoi, Vietnam · Faculty of Information Systems, Phenikaa University, Hanoi, Vietnam +1

Replay-Based Continual LearningMemory-Efficient TrainingDataset Condensation
AskPDF
Mar 12, 2026

ARROW: Augmented Replay for RObust World models

Continual reinforcement learning challenges agents to acquire new skills while retaining previously learned ones with the goal of improving performance in both past and future tasks. Most existing approaches rely on model-free methods with replay buffers to mitigate catastrophic forgetting; however, these solutions often face significant scalability challenges due to large memory demands. Drawing inspiration from neuroscience, where the brain replays experiences to a predictive World Model rather than directly to the policy, we present ARROW (Augmented Replay for RObust World models), a model-based continual RL algorithm that extends DreamerV3 with a memory-efficient, distribution-matching replay buffer. Unlike standard fixed-size FIFO buffers, ARROW maintains two complementary buffers: a short-term buffer for recent experiences and a long-term buffer that preserves task diversity through intelligent sampling. We evaluate ARROW on two challenging continual RL settings: Tasks without shared structure (Atari), and tasks with shared structure, where knowledge transfer is possible (Procgen CoinRun variants). Compared to model-free and model-based baselines with replay buffers of the same-size, ARROW demonstrates substantially less forgetting on tasks without shared structure, while maintaining comparable forward transfer. Our findings highlight the potential of model-based RL and bio-inspired approaches for continual reinforcement learning, warranting further research.

Abdulaziz Alyahya, Abdallah Al Siyabi, Markus R. Ernst +3

Department of Information Systems, College of Computer and Information Sciences, Imam Mohammad Ibn Saud Islamic University (IMSIU) · Department of Data Science & AI, Faculty of Information Technology, Monash University · School of Computer Science and Engineering, Faculty of Engineering, University of New South Wales, Sydney

Continual RLReplay-Based Continual LearningModel-Based RLWorld Models
AskPDF
Mar 11, 2026

Lifelong Imitation Learning with Multimodal Latent Replay and Incremental Adjustment

We introduce a lifelong imitation learning framework that enables continual policy refinement across sequential tasks under realistic memory and data constraints. Our approach departs from conventional experience replay by operating entirely in a multimodal latent space, where compact representations of visual, linguistic, and robot's state information are stored and reused to support future learning. To further stabilize adaptation, we introduce an incremental feature adjustment mechanism that regularizes the evolution of task embeddings through an angular margin constraint, preserving inter-task distinctiveness. Our method establishes a new state of the art in the LIBERO benchmarks, achieving 10-17 point gains in AUC and up to 65% less forgetting compared to previous leading methods. Ablation studies confirm the effectiveness of each component, showing consistent gains over alternative strategies. The code is available at: https://github.com/yfqi/lifelong_mlr_ifa.

Fanqi Yu, Matteo Tiezzi, Tommaso Apicella +2

AI for Good (AIGO), Istituto Italiano di Tecnologia, Genoa, Italy · DITEN, University of Genoa, Genoa, Italy · PAVIS, Istituto Italiano di Tecnologia, Genoa, Italy +1

Imitation LearningRobot Imitation LearningReplay-Based Continual Learning
AskPDF
Mar 10, 2026

MSSR: Memory-Aware Adaptive Replay for Continual LLM Fine-Tuning

Continual fine-tuning of large language models (LLMs) is becoming increasingly crucial as these models are deployed in dynamic environments where tasks and data distributions evolve over time. While strong adaptability enables rapid acquisition of new knowledge, it also exposes LLMs to catastrophic forgetting, where previously learned skills degrade during sequential training. Existing replay-based strategies, such as fixed interleaved replay, accuracy-supervised, and loss-driven scheduling, remain limited: some depend on heuristic rules and provide only partial mitigation of forgetting, while others improve performance but incur substantial computational overhead. Motivated by retention dynamics under sequential fine-tuning, we propose Memory-Inspired Sampler and Scheduler Replay (MSSR), an experience replay framework that estimates sample-level memory strength and schedules rehearsal at adaptive intervals to mitigate catastrophic forgetting while maintaining fast adaptation. Extensive experiments across three backbone models and 11 sequential tasks show that MSSR consistently outperforms state-of-the-art replay baselines, with particularly strong gains on reasoning-intensive and multiple-choice benchmarks.

Yiyang Lu, Yu He, Jianlong Chen +1

The Chinese University of Hong Kong, Shenzhen · Nanjing University

Continual Learning for LLMsReplay-Based Continual LearningCatastrophic Forgetting
AskPDF
Mar 4, 2026

Pretrained Vision-Language-Action Models are Surprisingly Resistant to Forgetting in Continual Learning

Continual learning is a long-standing challenge in robot policy learning, where a policy must acquire new skills over time without catastrophically forgetting previously learned ones. While prior work has extensively studied continual learning in relatively small behavior cloning (BC) policy models trained from scratch, its behavior in modern large-scale pretrained Vision-Language-Action (VLA) models remains underexplored. In this work, we found that pretrained VLAs are remarkably resistant to forgetting compared with smaller policy models trained from scratch. Simple Experience Replay (ER) works surprisingly well on VLAs, sometimes achieving zero forgetting even with a small replay data size. Our analysis reveals that pretraining plays a critical role in downstream continual learning performance: large pretrained models mitigate forgetting with a small replay buffer size while maintaining strong forward learning capabilities. Furthermore, we found that VLAs can retain relevant knowledge from prior tasks despite performance degradation during learning new tasks. This knowledge retention enables rapid recovery of seemingly forgotten skills through finetuning. Together, these insights imply that large-scale pretraining fundamentally changes the dynamics of continual learning, enabling models to continually acquire new skills over time with simple replay. Code and more information can be found at https://continual-vlas.github.io/forget-me-not/

Huihan Liu, Changyeon Kim, Bo Liu +2

The University of Texas at Austin · KAIST · Microsoft Superintelligence.

Continual Robot LearningContinual Learning for VLA ModelsCatastrophic ForgettingReplay-Based Continual Learning
AskPDF
Feb 28, 2026

IDER: IDempotent Experience Replay for Reliable Continual Learning

Catastrophic forgetting, the tendency of neural networks to forget previously learned knowledge when learning new tasks, has been a major challenge in continual learning (CL). To tackle this challenge, CL methods have been proposed and shown to reduce forgetting. Furthermore, CL models deployed in mission-critical settings can benefit from uncertainty awareness by calibrating their predictions to reliably assess their confidences. However, existing uncertainty-aware continual learning methods suffer from high computational overhead and incompatibility with mainstream replay methods. To address this, we propose idempotent experience replay (IDER), a novel approach based on the idempotent property where repeated function applications yield the same output. Specifically, we first adapt the training loss to make model idempotent on current data streams. In addition, we introduce an idempotence distillation loss. We feed the output of the current model back into the old checkpoint and then minimize the distance between this reprocessed output and the original output of the current model. This yields a simple and effective new baseline for building reliable continual learners, which can be seamlessly integrated with other CL approaches. Extensive experiments on different CL benchmarks demonstrate that IDER consistently improves prediction reliability while simultaneously boosting accuracy and reducing forgetting. Our results suggest the potential of idempotence as a promising principle for deploying efficient and trustworthy continual learning systems in real-world applications.Our code is available at https://github.com/YutingLi0606/Idempotent-Continual-Learning.

Zhanwang Liu, Yuting Li, Haoyuan Gao +4

School of Computer Science, Shanghai Jiao Tong University · State Key Laboratory of General Artificial Intelligence, BIGAI · Lehigh University +1

Continual LearningCatastrophic ForgettingReplay-Based Continual Learning
AskPDF
Feb 27, 2026

Quantifying Catastrophic Forgetting in IoT Intrusion Detection Systems

Distribution shifts in attack patterns within RPL-based IoT networks pose a critical threat to the reliability and security of large-scale connected systems. Intrusion Detection Systems (IDS) trained on static datasets often fail to generalize to unseen threats and suffer from catastrophic forgetting when updated with new attacks. Ensuring continual adaptability of IDS is therefore essential for maintaining robust IoT network defence. In this focused study, we formulate intrusion detection as a domain continual learning problem and propose a method-agnostic IDS framework that can integrate diverse continual learning strategies. We systematically benchmark five representative approaches across multiple domain-ordering sequences using a comprehensive multi-attack dataset comprising 48 domains. Results show that continual learning mitigates catastrophic forgetting while maintaining a balance between plasticity, stability, and efficiency, a crucial aspect for resource-constrained IoT environments. Among the methods, Replay-based approaches achieve the best overall performance, while Synaptic Intelligence (SI) delivers near-zero forgetting with high training efficiency, demonstrating strong potential for stable and sustainable IDS deployment in dynamic IoT networks.

Sourasekhar Banerjee, David Bergqvist, Salman Toor +2

Uppsala University, Department of Information Technology, Sweden · Scaleout Systems AB, Sweden · Ericsson Research, Research Area AI, Sweden

IoT Intrusion DetectionCatastrophic ForgettingDomain-Incremental LearningReplay-Based Continual Learning
AskPDF
Feb 27, 2026

CoPeP: Benchmarking Continual Pretraining for Protein Language Models

Protein language models (pLMs) have recently gained significant attention for their ability to uncover relationships between sequence, structure, and function from evolutionary statistics, thereby accelerating therapeutic drug discovery. These models learn from large protein databases that are continuously updated by the biology community and whose dynamic nature motivates the application of continual learning, not only to keep up with the ever-growing data, but also as an opportunity to take advantage of the temporal meta-information that is created during this process. As a result, we introduce the Continual Pretraining of Protein Language Models (CoPeP) benchmark, a novel benchmark for evaluating continual learning approaches on pLMs. Specifically, we curate a sequence of protein datasets derived from the UniProt Knowledgebase spanning a decade and define metrics to assess pLM performance across 31 protein understanding tasks. We evaluate several methods from the continual learning literature, including replay, unlearning, and plasticity-based methods, some of which have never been applied to models and data of this scale. Our findings reveal that incorporating temporal meta-information improves perplexity by up to 7% even when compared to training on data from all tasks jointly. Moreover, even at scale, several continual learning methods outperform naive continual pretraining. The CoPeP benchmark offers an exciting opportunity to study these methods at scale in an impactful real-world application.

Darshan Patil, Pranshu Malviya, Mathieu Reymond +2

Mila - Quebec AI Institute · Polytechnique Montréal · Chandar Research Lab +1

Protein Language ModelsReplay-Based Continual Learning
AskPDF