Catastrophic Forgetting

Ask about this topic

Overview

Catastrophic forgetting occurs when learning new tasks or adapting to shifting data degrades performance on earlier tasks, a central challenge for systems expected to learn over time. Common approaches include replaying examples, distilling prior outputs, regularizing important weights or gradients, and isolating updates with adapters or modular experts. Researchers study these methods in sequential vision and language benchmarks as well as foundation-model fine-tuning, multimodal systems, and robotics, measuring both new-task performance and retention. No single approach works best everywhere: stronger protection can limit adaptation, while memory, compute, and benchmark choices affect results; apparent forgetting may also reflect knowledge becoming inaccessible rather than erased.

Latest papers

CardsList

351 papers59 papers in the last four weeks, up 119% on the four weeks before. 0.3% of all new papers.

  1. BidirLM: From Text to Omnimodal Bidirectional Encoders by Adapting and Composing Causal LLMs

    Apr 2, 2026Nicolas Boizard, Théo Deschamps-Berger, Hippolyte Gisserot-Boukhlef +2Language ModelingMultimodal Learning

    Diabolocom · MICS, CentraleSupélec, Université Paris-Saclay · Artefact Research Center +1

  2. Countering Catastrophic Forgetting of Large Language Models for Better Instruction Following via Weight-Space Model Merging

    Apr 2, 2026Mengxian Lyu, Cheng Peng, Ziyi Chen +3Domain Adaptation for LLMsMedical LLMs

    Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, FL, USA · Department of Engineering Education, Herbert Wertheim College of Engineering, University of Florida, Gainesville, FL, USA · Preston A. Wells, Jr. Center for Brain Tumor Therapy, Lillian S. Wells Department of Neurosurgery, University of Florida, Gainesville, Florida, USA

  3. Continual Vision-Language Learning for Remote Sensing: Benchmarking and Analysis

    Apr 1, 2026Xingxing Weng, Ruifeng Ni, Chao Pang +5Remote Sensing VLMsContinual Learning for VLMs

    School of Computer Science, Wuhan University · College of Resources and Environmental Sciences, Shanxi Agricultural University · School of Artificial Intelligence, Wuhan University +2

  4. CL-VISTA: Benchmarking Continual Learning in Video Large Language Models

    Apr 1, 2026Haiyang Guo, Yichen Shi, Fei Zhu +6Continual Learning for VLMsVideo-Language Model Evaluation

    School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences · MAIS, Institute of Automation, Chinese Academy of Sciences · Centre for Artificial Intelligence and Robotics, Hong Kong Institute of Science & Innovation, CAS +2

  5. Chameleons do not Forget: Prompt-Based Online Continual Learning for Next Activity Prediction

    Apr 1, 2026Marwan Hassani, Tamara Verbeek, Sjoerd van StratenNext-Action PredictionPredictive Process Monitoring

    Department of Mathematics and Computer Science, Eindhoven University of Technology, The Netherlands

  6. Adapting Text LLMs to Speech via Multimodal Depth Up-Scaling

    Apr 1, 2026Kazuki Yano, Jun Suzuki, Shinji WatanabeSpeech Language ModelsAutomatic Speech Recognition

    Tohoku University, Japan · Carnegie Mellon University, USA

  7. Temporal Memory for Resource-Constrained Agents: Continual Learning via Stochastic Compress-Add-Smooth

    Mar 31, 2026Michael ChertkovMemory CompressionContinual Learning

    Graduate Inter-Disciplinary Program in Applied Mathematics, & Department of Mathematics, University Arizona, Tucson, AZ 85721

  8. Low-Rank Adaptation Reduces Catastrophic Forgetting in Sequential Transformer Encoder Fine-Tuning: Controlled Empirical Evidence and Frozen-Backbone Representation Probes

    Mar 29, 2026Ashish PandeyCatastrophic ForgettingLow-Rank Adaptation

  9. The Price of Meaning: Why Every Semantic Memory System Forgets

    Mar 28, 2026Sambartha Ray Barman, Andrey Starenky, Sofia Bodnar +2Semantic SearchAssociative Memory

    Sentra, 235 2nd Street, San Francisco, CA 94105, USA · Department of Mechanical Engineering, Massachusetts Institute of Technology, 77 Massachusetts Avenue, Cambridge, MA 02139, USA

  10. Bi-CRCL: Bidirectional Conservative-Radical Complementary Learning with Pre-trained Foundation Models for Class-incremental Medical Image Analysis

    Mar 24, 2026Xinyao Wu, Zhe Xu, Cheng Chen +3Class-Incremental LearningMedical Image Analysis

    Department of Biomedical Engineering, The Chinese University of Hong Kong, Hong Kong, China. · Department of Radiation Oncology, Columbia University Irving Medical Center and Data Science Institute, Columbia University, New York, NY, USA. · Department of Electrical and Electronic Engineering and School of Biomedical Engineering, The University of Hong Kong, Hong Kong, China. +2

  11. Natural Gradient Descent for Online Continual Learning

    Mar 21, 2026Joe Khawand, David ColliauxKronecker-Factored ApproximationOnline Continual Learning

    Ecole Polytechnique & Télécom Paris Paris, France · Sony Computer Science Laboratories Paris, France

  12. A Comparative Empirical Study of Catastrophic Forgetting Mitigation in Sequential Task Adaptation for Continual Natural Language Processing Systems

    Mar 19, 2026Aram Abrahamyan, Sachin KumarCatastrophic ForgettingReplay-Based Continual Learning

    Zaven P. and Sonia Akian College of Science and Engineering, American University of Armenia, Yerevan 0019, Armenia

  13. Elastic Weight Consolidation Done Right for Continual Learning

    Mar 19, 2026Xuan Liu, Xiaobin ChangRegularization-Based Continual LearningContinual Learning

    School of Artificial Intelligence, Sun Yat-sen University, China · Key Laboratory of Intelligent Assessment Technology for Sustainable Tourism, Ministry of Culture and Tourism, Sun Yat-sen University · Key Laboratory of Machine Intelligence and Advanced Computing, Ministry of Education, China

  14. HSI Image Enhancement Classification Based on Knowledge Distillation: A Study on Forgetting

    Mar 19, 2026Songfeng ZhuClass-Incremental LearningHyperspectral Image Classification

    School of Software Engineering, Henan Polytechnic University, Henan 454000, china

  15. CLeAN: Continual Learning Adaptive Normalization in Dynamic Environments

    Mar 18, 2026Isabella Marasco, Davide Evangelista, Elena Loli Piccolomini +1Continual LearningTabular ML

    Department of Computer Science and Engineering University of Bologna Bologna, 40126, Italy

  16. Structural Abstraction as an Inductive Bias for Non-Stationary Language Model Training

    Mar 17, 2026Elnaz Rahmati, Nona Ghazizadeh, Zhivar Sourati +2LLM TrainingContinual Learning for LLMs

    University of Southern California

  17. MedCL-Bench: Benchmarking stability-efficiency trade-offs and scaling in biomedical continual learning

    Mar 17, 2026Min Zeng, Shuang Zhou, Zaifu Zhan +1Continual LearningContinual Learning for LLMs

    Division of Computational Health Sciences, Department of Surgery, University of Minnesota, Minneapolis, 55455, MN, USA.

  18. The Finetuner's Fallacy: When to Pretrain with Your Finetuning Data

    Mar 17, 2026Christina Baek, Ricardo Pio Monti, David Schwab +31Language Model PretrainingDomain-Adaptive Pretraining

    DatologyAI

  19. Detection of Autonomous Shuttles in Urban Traffic Images Using Adaptive Residual Context

    Mar 16, 2026Mohamed Aziz Younes, Nicolas Saunier, Guillaume-Alexandre BilodeauContinual Object DetectionObject Detection

    Polytechnique Montréal, Montréal, Canada

  20. MeMix: Writing Less, Remembering More for Streaming 3D Reconstruction

    Mar 16, 2026Jiacheng Dong, Huan Li, Sicheng Zhou +3Streaming 3D Reconstruction3D Reconstruction

    Zhejiang University · Institute for AI Industry Research, Tsinghua University

  21. SCAN: Sparse Circuit Anchor Interpretable Neuron for Lifelong Knowledge Editing

    Mar 16, 2026Yuhuan Liu, Haitian Zhong, Xinyuan Xia +3Knowledge EditingLanguage Model Editing

    New Laboratory of Pattern Recognition (NLPR), State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences · Cuiying Honors College, Lanzhou University · Zhongguancun Academy +1

  22. CATFormer: When Continual Learning Meets Spiking Transformers With Dynamic Thresholds

    Mar 16, 2026Vaishnavi Nagabhushana, Kartikay Agrawal, Ayon BorthakurClass-Incremental LearningSpiking Neural Networks

    SustainAI Lab, MFSDS&AI, IIT Guwahati

  23. MMKU-Bench: A Multimodal Update Benchmark for Diverse Visual Knowledge

    Mar 16, 2026Baochen Fu, Yuntao Du, Cheng Chang +6Multimodal Knowledge EditingMultimodal Large Language Models

    Shandong University, Jinan, China · Jinzhong Group, Jinan, China.

  24. Fine-tuning MLLMs Without Forgetting Is Easier Than You Think

    Mar 15, 2026He Li, Yuhui Zhang, Xiaohan Wang +2LLM Fine-TuningMultimodal Large Language Models

    Stanford University · Tsinghua University

  25. All-day Multi-scenes Lifelong Vision-and-Language Navigation with Tucker Adaptation

    Mar 15, 2026Xudong Wang, Gan Li, Zhiyu Liu +3Tucker DecompositionVision-Language Navigation

    State Key Laboratory of Robotics and Intelligent Systems, Shenyang Institute of Automation, Chinese Academy of Sciences · University of Chinese Academy of Sciences · North University of China +1

  26. MLFCIL: A Multi-Level Forgetting Mitigation Framework for Federated Class-Incremental Learning in LEO Satellites

    Mar 14, 2026Heng Zhang, Xiaohong Deng, Sijing Duan +4Remote Sensing Scene ClassificationFederated Continual Learning

    School of Information Engineering, Gannan University of Science and Technology, Ganzhou 341000, China · Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China · Department of Electrical and Computer Engineering, Queen University, Kingston, ON K7L 3N6, Canada +2

  27. Zero-Forgetting CISS via Dual-Phase Cognitive Cascades

    Mar 14, 2026Yuquan Lu, Yifu Guo, Zishan Xu +6Semantic SegmentationClass-Incremental Learning

    Sun Yat-sen University · Shanghai Jiao Tong University · Southwest University +3

  28. Beyond Static Models: An Evolving Framework for Continual Learning in Large Language Models across Training Stages

    Mar 13, 2026Hongyang Chen, Zhongwu Sun, Hongfei Ye +2Continual Learning for LLMsCatastrophic Forgetting

    Zhejiang Lab, Hangzhou 311100, China

  29. A Quantitative Characterization of Forgetting in Post-Training

    Mar 12, 2026Krishnakumar Balasubramanian, Shiva Prasad KasiviswanathanLLM Post-TrainingCatastrophic Forgetting

    Department of Statistics, University of California, Davis · Amazon

  30. Continual Learning with Vision-Language Models via Semantic-Geometry Preservation

    Mar 12, 2026Chiyuan He, Zihuan Qiu, Fanman Meng +4Continual LearningContinual Learning for VLMs

    School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China