Catastrophic Forgetting

Follow this topic Ask about this topic

Catastrophic forgetting occurs when learning new tasks or adapting to shifting data degrades performance on earlier tasks, a central challenge for systems expected to learn over time. Common approaches include replaying examples, distilling prior outputs, regularizing important weights or gradients, and isolating updates with adapters or modular experts. Researchers study these methods in sequential vision and language benchmarks as well as foundation-model fine-tuning, multimodal systems, and robotics, measuring both new-task performance and retention. No single approach works best everywhere: stronger protection can limit adaptation, while memory, compute, and benchmark choices affect results; apparent forgetting may also reflect knowledge becoming inaccessible rather than erased.

Latest papers

351 papers59 papers in the last four weeks, up 119% on the four weeks before. 0.3% of all new papers.

Memory Anchors for Continual Robot Learning
Aug 27, 2026Maximilian Du, Zhanyi Sun, Chen Xu +3Continual Robot Learning Catastrophic Forgetting
Stanford University · Toyota Research Institute
A Visual Dependence-Aware Framework for Multimodal Unsupervised Continual Post-Training
Aug 26, 2026Kaichen Li, Zhilin Zhu, Jianhao Huang +7Continual Learning for VLMs VLM Adaptation
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China · Pengcheng Laboratory, Shenzhen, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China +1
Learning New Facts with QLoRA: An Acquisition-Retention Frontier
Aug 26, 2026Estelle Zheng, Sébastien Warichet, Emmanuel Helbert +1Factual Knowledge in Language Models Low-Rank Adaptation
LORIA, CNRS, France · Alcatel-Lucent Enterprise, France
Preference Data Selection for Mitigating the Alignment Tax in Large Language Models
Aug 25, 2026Minsu Kim, Jianxun Lian, Xing Xie +1LLM Alignment Preference Alignment
KAIST · Microsoft Research Asia · Work done during an internship at Microsoft Research Asia
Functional compatibility as a determinant of persistent neural learning
Aug 23, 2026Hossein JavidniaStability-Plasticity Dilemma Continual Learning
School of Computing Dublin City University Dublin, Ireland
Federated Continual Learning as a Distributed Drift-Plus-Penalty Control Problem
Aug 21, 2026Nazreen Shah, Naveen Kumar Reddy Somireddy, Zubair Shaban +2Federated Continual Learning Catastrophic Forgetting
IIIT Delhi India · IIT Dharwad India
Socialized Division and Collaboration: Rethinking Class-Incremental Learning under Optimization Conflicts
Aug 21, 2026Xinjie Yao, Zhihe Fan, Yunqi Zhu +6Class-Incremental Learning Continual Learning
Faculty of Information Engineering and Automation, Kunming University of Science and Technology · School of Sports Training, Tianjin University of Sport · School of Computer Science and Engineering, University of New South Wales +3
TASSO: TAsk-Specific Subspace Optimization for Continual Learning of Vision-Language Models
Aug 21, 2026Chang Sun, Francesco Barbato, Matteo Caligiuri +1Continual Learning for VLMs Catastrophic Forgetting
University of Padova, Padova(PD), Italy
Forgetting, plasticity, and co-observation: a third facet of continual learning
Aug 19, 2026Timm Hess, Abhishek Jha, Gido M. van de Ven +1Continual Learning Catastrophic Forgetting
VIB.AI † KU Leuven, Belgium · Bernoulli Institute ‡ University of Groningen, Netherlands
When to Review: Spaced Repetition for Continual Pre-Training of Language Models
Aug 18, 2026Alankar Atreya, Devesh Batra, Yoages Kumar Mantri +3Continual Learning for LLMs Continued Pretraining
NatWest AI Research · University College London
Agents unlock new capabilities through Switching LoRA Adapters as a Tool (SLAaaT)
Aug 17, 2026Kenneth GeLong-Horizon LLM Agents LLM Agent Skill Learning
Independent San Francisco, CA, United States of America
Layers Matter: Why Continual Learning Regularization Should Be Layer-Adaptive
Aug 16, 2026Brian B. Moser, Ahmed Anwar, Tobias Christian Nauen +5Regularization-Based Continual Learning Continual Learning
German Research Center for Artificial Intelligence (DFKI) · RPTU University Kaiserslautern-Landau
Geometry of Forgetting: Representation Flux in Continual Learning
Aug 16, 2026Maksim A. KazanskiiCatastrophic Forgetting Representation Drift
Self-Specialized Teachers for Domain Post-Training
Aug 16, 2026Yifei Li, Rongman Xu, Lingling Zhang +5Domain Adaptation for LLMs Catastrophic Forgetting
School of Computer Science and Technology, Xi’an Jiaotong University · MOE KLNN Lab, Xi’an Jiaotong University
What to Preserve, Where to Adapt: A Depth-Wise Analysis of Forgetting in Continual Gynecological Image Segmentation
Aug 13, 2026Amal Saqib, Tausifa Jan Saleem, Numan Saeed +1Medical Image Segmentation 3D Medical Image Segmentation
Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), Abu Dhabi, UAE
Uniform Herding: Exemplar Replay with Representation Refresh
Aug 13, 2026Krishna SubediClass-Incremental Learning Replay-Based Continual Learning
Neryva
AWARe: Mitigating Catastrophic Forgetting via Activation-Weighted Adaptive REtention
Aug 12, 2026Juncheng Liao, Jinfan Lv, Guoming Wang +3Continual Learning for LLMs LLM Fine-Tuning
School of Software Technology, Zhejiang University, Hangzhou, China · College of Intelligent Robotics and Advanced Manufacturing, Fudan University, Shanghai, China · School of Artificial Intelligence, Sun Yat-Sen University, Guangzhou, China +1
Drift and Dependence: Layer-wise Information-Theoretic Bounds for Replay-Based Continual Learning
Aug 12, 2026Tieliang Gong, Zhongbo Zhang, Wen Wen +1Information-Theoretic Generalization Bounds Replay-Based Continual Learning
Tieliang Gong, Zhongbo Zhang and Wen Wen are with the School of Computer Science and Technology, Xi’an Jiaotong University, China · Yong-Jin Liu is with Department of Computer Science and Technology, Tsinghua University, China
Hyperbolic Multimodal Continual Learning
Aug 10, 2026Jiahong Liu, Ming Shen, Xiaohao Liu +4Cross-Modal Representation Learning Invariant Representation Learning
Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong, China · School of Computing, National University of Singapore, Singapore · Department of Computer Science, Yale University, New Haven, CT, United States +1
Catastrophic Forgetting in Continual Reinforcement Learning
Aug 9, 2026Emma GrahamContinual RL Catastrophic Forgetting
UCL · Machine Learning MSc · [cs.LG]
A continually expandable foundation model for brain MRI
Aug 8, 2026Michail Mamalakis, Carmen Jimenez-Mesa, Yonghao Li +8Medical Imaging Foundation Models Continual Learning
1*Cancer Research UK Cambridge Institute, University of Cambridge Li Ka Shing Centre, Robinson Way, Cambridge, CB2 0RE, Cambridgeshire, United Kingdom. · Department of Computer Science and Technology, University of Cambridge, 15 JJ Thomson Ave, Cambridge, CB3 0FD, Cambridgeshire, United Kingdom. · Department of Psychiatry, University of Cambridge, Hills Road, Cambridge, CB2 2QQ, Cambridgeshire, United Kingdom. +7
Stockmark-Nemotron-3-Nano-Omni-JapanDocReader: Structured Document Parsing via Capability Injection and Forgetting Control
Aug 7, 2026Shi Chen, Hayato Aida, Makoto Morinaga +2Document Understanding Multimodal Large Language Models
Stockmark Inc.
Omega-S: A Functional Resilience Index for LLM Fine-Tuning
Aug 4, 2026Alberto AcedoRegularization-Based Continual Learning Continual Learning for LLMs
Biome Makers Inc.
Taming the Implicit: Dual-Channel Risk-Aware Reinforcement Fine-Tuning for Continual Multimodal Post-Training
Aug 4, 2026Yibei Liu, Jiajun Chen, Qianle Zhang +4Multimodal Large Language Models Risk-Sensitive RL
University of Electronic Science and Technology of China · Zhejiang University · South China University of Technology +1
Trajectory-Guided Forget-Recover Network for Continual LLM Unlearning
Aug 4, 2026Zezheng Wu, Xinghe Cheng, Qinggang Zhang +4Machine Unlearning LLM Unlearning
Guilin University of Electronic Technology · Jinan University · Jilin University +2
AdaHAT: Adaptive Hard Attention to the Task in Task-Incremental Learning
Aug 2, 2026Pengxiang Wang, Hongbo Bo, Jun Hong +2Task-Incremental Learning Stability-Plasticity Dilemma
School of Mathematical Sciences, Peking University, Beijing, China · Population Health Sciences Institute, Newcastle University, Newcastle, UK · School of Engineering Mathematics and Technology, University of Bristol, Bristol, UK +1
PATH-Bench: Path-Dependent Evaluation of Lifelong Agents
Aug 2, 2026Xidong Yang, Xingyi Zhang, Wenhao Li +7Continual Learning for LLM Agents Lifelong Learning Agents
East China Normal University · Tongji University · Ant Group +5
The Grokked Illusion: True Equilibrium Mitigates Catastrophic Forgetting
Jul 31, 2026Xiaotian Zhang, Lai Shun Chan, Yue Shang +2Neural Network Robustness Catastrophic Forgetting
Department of Physics, City University of Hong Kong, Hong Kong, China · Department of Physics and Astronomy, University of Pennsylvania, Philadelphia, PA, USA · Innovation Campus Delaware, Air Liquide, Newark, DE, USA
CACHE-UK: A Stability-Aware Memory Editor for Sequentially Updated Quantized LLMs in Finance
Jul 30, 2026Anubhav Lakra, Yue FengFinancial Language Model Evaluation Knowledge Editing
University of Birmingham, Birmingham, United Kingdom
DECODE: Tackling Representation and Decision Degradation in Continual AI-Generated Image Detection
Jul 30, 2026Zihao Cai, Xinghan Li, Ruiyan Yang +3Continual Learning AI-Generated Image Detection
Visual Laboratory of Fudan University, College of Computer Science and Artificial Intelligence, Fudan University, Shanghai, China · CEC GienTech Technology Co., Ltd., Shanghai, China

Aug 27, 2026J/K move · Enter open · S save

Memory Anchors for Continual Robot Learning

Maximilian Du, Zhanyi Sun, Chen Xu, Paarth Shah+2

Stanford University · Toyota Research Institute

Continual Robot Learning Catastrophic Forgetting Replay-Based Continual Learning Experience Replay

Open paper AskPDF arXiv

Robot policies deployed in the wild should have the capability to continually learn new tasks without forgetting existing behaviors. A common approach to combat such catastrophic forgetting is to train on new task data with a replay buffer of previously learned task data. Although this buffer is commonly sampled randomly from all prior experiences, we show that a small set of these experiences contributes greatly in anchoring past performance. We call these experiences Memory Anchors. We identify Memory Anchors in regions where representations of new-task observations collapse onto those of old-task observations even though the tasks require conflicting actions, like when a familiar object must be manipulated in a new way. Rehearsing old data in this region plays a key role in preventing destructive overwriting of past task knowledge, serving as this critical Memory Anchor role. Excluding only 10% Memory Anchors before sampling the buffer leads to more than a 4.5x increase in catastrophic forgetting on the LIBERO benchmark suites. Conversely, enriching the replay buffer with Memory Anchors can decrease high-conflict task forgetting by 63% and enables successful continual learning of two task sequences on a real robot. Videos and additional visualizations can be found at https://robot-adaptation.github.io/MemoryAnchors