Catastrophic Forgetting
Overview
Catastrophic forgetting occurs when learning new tasks or adapting to shifting data degrades performance on earlier tasks, a central challenge for systems expected to learn over time. Common approaches include replaying examples, distilling prior outputs, regularizing important weights or gradients, and isolating updates with adapters or modular experts. Researchers study these methods in sequential vision and language benchmarks as well as foundation-model fine-tuning, multimodal systems, and robotics, measuring both new-task performance and retention. No single approach works best everywhere: stronger protection can limit adaptation, while memory, compute, and benchmark choices affect results; apparent forgetting may also reflect knowledge becoming inaccessible rather than erased.
Latest papers
374 papers32 papers in the last four weeks, down 45% on the four weeks before. 0.2% of all new papers.
Does Learning Order Explain Forgetting? The Lie Bracket as an Early-Warning Signal for Catastrophic Forgetting Without Old Data
Bring Back Language to Actions: Restoring VLM capabilities in VLA Models via Parameter Merging
Maximizing Capacity Utilization of LoRA Adapters for Mitigating the Learning-Forgetting Trade-off
A Margin Perspective on LoRA: Robustness to Catastrophic Forgetting and Adapter Merging (MaLoRA)
Unified Factored Representations Resist Catastrophic Forgetting
Audit Before You Merge: Provenance, Probing, and Continual LoRA Composition
Continual Instruction Tuning for LLM-based Management of Continuum Resources
ROSE: Risk-Aware Orthogonal Subspace Navigation for Lifelong Knowledge Editing in Multimodal Large Language Models
Human Motion Prediction via Continual Prior Compensation
near-future'' prediction) could be hindered by the predictions of long-term (referring to far-future'' prediction) motions. In this paper, we develop a novel temporal continual learning framework called Continual Prior Compensation (CPC) to progressively train HMP models, in which we divide the prediction task of motions corresponding to varied temporal moments into several subtasks and train the model in a multi-stage manner. To mitigate the prior information forgetting in the progressive training, we further introduce a learnable random variable Prior Compensation Factor (PCF) to explicitly measure the prior knowledge loss. We theoretically show that the PCF can be efficiently learned together with the model parameters by minimizing a reasonable upper bound of the objective function. The proposed CPC is further enhanced to estimate the prior information loss for each subtask and a new framework called Continual Prior Compensation++ (CPC++) with Fine-Grained Prior Compensation Factor (FGPCF) is finally developed. Our CPC and CPC++ frameworks are quite flexible and can be easily integrated with different HMP backbone models and adapted to various datasets and applications. Extensive experiments on three HMP benchmark datasets using multiple SOTA HMP backbones (PGBIG, siMLPe, MotionMixer, and LTD) demonstrate the effectiveness and flexibility of our frameworks.