Catastrophic Forgetting

Ask about this topic

Overview

Catastrophic forgetting occurs when learning new tasks or adapting to shifting data degrades performance on earlier tasks, a central challenge for systems expected to learn over time. Common approaches include replaying examples, distilling prior outputs, regularizing important weights or gradients, and isolating updates with adapters or modular experts. Researchers study these methods in sequential vision and language benchmarks as well as foundation-model fine-tuning, multimodal systems, and robotics, measuring both new-task performance and retention. No single approach works best everywhere: stronger protection can limit adaptation, while memory, compute, and benchmark choices affect results; apparent forgetting may also reflect knowledge becoming inaccessible rather than erased.

Latest papers

CardsList

351 papers59 papers in the last four weeks, up 119% on the four weeks before. 0.3% of all new papers.

  1. Beyond Parameter Protection: Fisher-Shaped EWC for Replay-Free Continual Learning

    Oct 27, 2026Liying Wang, Simone Magistri, Joost van de WeijerElastic Weight ConsolidationRegularization-Based Continual Learning

    Department of Computer Science, Universitat Aut`onoma de Barcelona, Spain · Computer Vision Center, Barcelona, Spain · Media Integration and Communication Center, University of Florence, Italy

  2. Generative Replay for Equilibrium Propagation: Toward Edge-Friendly Continual Learning in Hopfield Networks

    Oct 27, 2026Victor Cook, Gilles Sassatelli, Thierry Gil +2Replay-Based Continual LearningGenerative Replay

    LIRMM | ADAC · LIRMM, Univ. Montpellier / CNRS, Montpellier, France · LIRMM +2

  3. Entropic Control for Homeostatic Optimization: Pre-empting Catastrophic Forgetting via Dynamic Feedback

    Oct 25, 2026Shangxi Wu, Xiaorong Yan, Xiaojun Meng +2Catastrophic ForgettingContinual Learning for LLMs

    Beijing Jiaotong University · The Hong Kong Polytechnic University · Huawei Noah’s Ark Lab

  4. DRIVE: Fine-tuning via Data Contribution- and Diversity-aware Weighting with Prior Regularization

    Oct 23, 2026Qing Liu, Xinrui Chen, Yi Du +2Fine-TuningInstance Reweighting

    Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Hangzhou, China · Computer Network Information Center, Chinese Academy of Sciences, Beijing, China · School of Mathematics, Tianjin University, Tianjin, China

  5. Beyond Data Schedule: Fast LLM Pretraining on Data Mixtures by Loss-Reweighting

    Oct 23, 2026Jiazheng Li, Huaqing Zhang, Jing Xu +2Language Model PretrainingAdaptive Loss Weighting

    Tsinghua University · ByteDance Seed

  6. Fine-Grained Analysis of Catastrophic Forgetting in Continual Learning via Algorithmic Stability

    Oct 22, 2026Guodong Zheng, Yan Sun, Peng Wang +5Catastrophic ForgettingGeneralization Bounds

    School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University, Shenzhen, China. · College of Computing and Data Science and the Generative AI Lab at Nanyang Technological University, Singapore. · Tencent Hunyuan, Shenzhen, China.

  7. Why Constrained Updates Still Forget: A Functional-Path Analysis of Continual SFT

    Oct 21, 2026Qingsen Ma, Dianyun Wang, Jiahong Liu +1Supervised Fine-TuningContinual Learning for LLMs

    The Chinese University of Hong Kong · Beijing University of Posts and Telecommunications

  8. Combating Catastrophic Forgetting in Continual Domain Adaptation via Knowledge Recasting

    Oct 20, 2026Zijie Liu, Zhen Tan, Charles Fleming +1Continual Learning for LLMsDomain Adaptation

    UNC at Chapel Hill · Stevens Institute of Technology · Cisco Research

  9. Staged Continual Adaptation of Multimodal Foundation Models for Japanese Financial Documents

    Oct 20, 2026Genshin Kakimoto, Atsushi YanagisawaMultimodal Foundation ModelsCatastrophic Forgetting

    Kyoto University, Kyoto, Japan.

  10. From Generic to Dedicated: A Novel Optimizer for Online Continual Learning

    Oct 20, 2026Yongyi Wu, Zheng Wang, Sen LinOnline Continual LearningCatastrophic Forgetting

    University of Houston Houston, USA

  11. Are Self-Evolving Agents Continual Learners?

    Oct 19, 2026Xiaoyin Chen, Yizheng Huang, Yinglun Xu +4Continual Learning for LLM AgentsSelf-Evolving Agents

    Mila – Quebec AI Institute · Université de Montréal · Work done as a student researcher at Google DeepMind +1

  12. NADS: Navigator-Guided Data Selection for Mitigating Catastrophic Forgetting in Fine-Tuning

    Oct 19, 2026Jianhao Zhang, Ou Wu, Yi DuLLM Fine-TuningData Selection for Language Model Fine-Tuning

    Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Hangzhou, China · Computer Network Information Center, Chinese Academy of Sciences, Beijing, China

  13. OPDArena: Understanding Learning and Forgetting in On-Policy Distillation

    Oct 19, 2026Yuanze Xu, Tianzhu Ye, Hui Sun +3On-Policy DistillationCatastrophic Forgetting

    Microsoft Research · Nankai University

  14. Learning without Overwriting: Structural Knowledge Fusion for Continual Learning

    Oct 18, 2026Kanishk Bakshi, Amitansu Pattanaik, Kathiravan SrinivasanClass-Incremental LearningIncremental Learning

    School of Computer Science and Engineering, Vellore Institute of Technology, Vellore, Tamil Nadu 632014, India · Defence Geoinformatics Research Establishment, Defence Research and Development Organisation, New Delhi 110054, India

  15. Towards Continual Learning: Mitigating Forgetting With Subspace Tuning

    Oct 15, 2026Arnav Chavan, Kusha Sareen, Rishabh Tiwari +6Continual Learning for LLMsLLM Fine-Tuning

    UC Berkeley · Mila · McGill University +5

  16. Sparse Autoencoders Can Induce Local Geometry for Mitigating Catastrophic Forgetting

    Oct 15, 2026Kamil Faber, Katarzyna Filus, Łukasz PawelaCatastrophic ForgettingRegularization-Based Continual Learning

    Faculty of Computer Science, AGH University of Krakow, Krakow, Poland · Institute of Theoretical and Applied Informatics, Polish Academy of Sciences, Gliwice, Poland

  17. Functional Subnetwork Localization for Efficient Continual Learning in Large Language Models

    Oct 15, 2026Shuoran Jiang, Wentao Zhu, Yang Xiang +8Continual Learning for LLMsIncremental Learning

    School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen), Shenzhen, China · China Academy of Engineering Physics, Mianyang, China · Peng Cheng Laboratory, Shenzhen, China +2

  18. Overcoming Catastrophic Forgetting: Continual Learning for Radar Waveform Classification

    Oct 13, 2026Mohamed Rabie, Chinthana J. Panagamuwa, Konstantinos G. KyriakopoulosContinual LearningCatastrophic Forgetting

    Wolfson School of Mechanical, Electrical and Manufacturing Engineering, Loughborough University, U.K.

  19. Collapse, Repair, and Drift: Dissecting Fine Tuning Dynamics in Speech Language Models

    Oct 13, 2026Yichi Wang, Wangjin Zhou, Yizhou Zhang +4Supervised Fine-TuningNeural Network Training Dynamics

    Department of Intelligence Science and Technology, Kyoto University · Xiaowei, WXG, Tencent · Communication University of China

  20. Confine to the Task, Anchor the Rest: Concept-Guided Retention Regularizer for Parameter-Efficient Fine-Tuning

    Oct 13, 2026Ibna Kowsar, Ivan Medri, Fanyu Meng +1Fine-TuningParameter-Efficient Fine-Tuning

    Department of Computer Science, University of California, Davis · University of Virginia

  21. Effect of Spherical Parameter Space on Number of Non-Forgetting Tasks and Capacity of Continual Learning

    Oct 12, 2026Nazanin MirhosseiniContinual LearningCatastrophic Forgetting

    Colorado State University, Fort Collins, Co, USA

  22. When Skills Remain But Access Fails: Factorized Skill Memory for Continual Skill Internalization

    Oct 11, 2026Haomiao Qiu, Yining Tang, Hongyi Wan +3Continual Learning for LLM AgentsAgent Skill Learning

    Harbin Institute of Technology, Shenzhen · Great Bay University · Zhejiang University

  23. Representation Drift in Continual Learning: LoRA versus N-gram Memory

    Oct 10, 2026Hao WuContinual Learning for LLMsOnline Continual Learning

    Illinois Institute of Technology

  24. Formalizing Forgetting: Theory of Forgetting for Online Continual Learning

    Oct 10, 2026Renjie Cao, Yifei Cheng, Li ShenOnline Continual LearningCatastrophic Forgetting

    Boston College · Sun Yat-sen University

  25. Compositional Concepts Erasing: Disentangling Object-Style via Domain Manifold Unlearning

    Oct 10, 2026Yang Xiao, Kaiyuan Deng, Gen Li +4Diffusion Model UnlearningMachine Unlearning

    University of Tulsa · The University of Arizona · Clemson University +1

  26. Mitigating Capability Degradation in LLM Self-Improvement via Representation Geometry Regularization

    Oct 9, 2026Jiaqi Tian, Ting LongLanguage Model Self-ImprovementOn-Policy Self-Distillation

    School of Artificial Intelligence Jilin University China

  27. Static Gradient Attribution Underperforms a Density-Matched Random Mask on Loss-Based Forgetting Within LoRA B-Matrix

    Oct 8, 2026Yu Sun, Junwei Zhou, Zuodong Xiang +5Storage-Efficient Continual LearningContinual Learning for LLMs

    University of Pennsylvania · Matter Innovation Inc. · University of California, Davis +5

  28. Self-Preservation in Language Models

    Oct 8, 2026Dongkyu Cho, Rumi Chunara, Ali BehrouzContinual Learning for LLMsReplay-Based Continual Learning

    New York University · Google Research · Cornell University

  29. Where to Adapt Matters: Layer-Selective Fine-Tuning for Capability Retention

    Oct 8, 2026Zhiqiang Pang, Zihong Sun, Qi Xie +3LLM Fine-TuningSelective Fine-Tuning

    School of Mathematics and Statistics, Xi’an Jiaotong University, Xi’an, Shaanxi, China

  30. From a Prompt to Repertoires: Evolving Functional REpertoires Enable LLM Continual Learning

    Oct 8, 2026Fengyuan Liu, Yue Wang, Hangxi Guo +4Continual Learning for LLMsPrompt Learning

    The Chinese University of Hong Kong, Shenzhen · Shanghai AI Laboratory · University of Science and Technology of China +2