Catastrophic Forgetting

Ask about this topic

Overview

Catastrophic forgetting occurs when learning new tasks or adapting to shifting data degrades performance on earlier tasks, a central challenge for systems expected to learn over time. Common approaches include replaying examples, distilling prior outputs, regularizing important weights or gradients, and isolating updates with adapters or modular experts. Researchers study these methods in sequential vision and language benchmarks as well as foundation-model fine-tuning, multimodal systems, and robotics, measuring both new-task performance and retention. No single approach works best everywhere: stronger protection can limit adaptation, while memory, compute, and benchmark choices affect results; apparent forgetting may also reflect knowledge becoming inaccessible rather than erased.

Latest papers

CardsList

351 papers59 papers in the last four weeks, up 119% on the four weeks before. 0.3% of all new papers.

  1. Curvature-Guided Mixing for MLLM Adaptation

    Jun 23, 2026Jinglong Yang, Jiaxuan He, Wenjian Huang +2VLM AdaptationMultimodal Large Language Models

    Research Institute of Trustworthy Autonomous Systems and Department of Computer Science and Engineering, Southern University of Science and Technology · Department of Computer Science, City University of Hong Kong

  2. Towards Continuous Power Forecasting: Practical Continual Learning for Real-World Energy Systems in Nonstationary Time Series

    Jun 23, 2026Yujiang He, Frederic Uhrweiller, Bernhard SickContinual LearningTime Series Forecasting

    Intelligent Embedded Systems, University of Kassel, 34121 Kassel, Germany

  3. RECALL: Recovery Experience Collection for Active Lifelong Learning in Vision-Language-Action Models

    Jun 22, 2026Ulas Berk Karli, Tesca FitzgeraldContinual Robot LearningRobot Skill Learning

    Department of Computer Science Yale University United States

  4. SamatNext v0.2-B: An Exploratory Study of RMS-Normalized Hybrid Decoders for Curriculum Retention in Small Code Models

    Jun 20, 2026Samat ZharassovContinual Learning for LLMsDecoder-Only Language Models

  5. Task-Differentiated Atomic Skill Expansion and Routing for Continual Learning Across Highly Heterogeneous Tasks

    Jun 19, 2026Jiacheng Wang, Xinjia He, Qi Ding +5Dynamic Neural NetworksContinual Learning

    East China Normal University · Fudan University · Ocean University of China

  6. Sparsity, Superposition, and Forgetting: A Mechanistic Study of Representation Retention in Continual Learning

    Jun 18, 2026Jan Wasilewski, Jędrzej Kozal, Michał Woźniak +1Feature SuperpositionRepresentation Learning

    Rochester Institute of Technology · Rochester NY, USA · Wrocław University of Science and Technology +1

  7. Beyond Uniform Forgetting: A Study of Sequential Direct Preference Optimization Across Preference Settings

    Jun 18, 2026Pranav Bhandari, Nicolas Fay, Amitava Datta +2Continual Learning for LLMsDirect Preference Optimization

    1Network Analysis and Social Influence Modelling (NASIM) Lab · School of Physics Maths and Computing, The University of Western Australia · School of Psychological Science, The University of Western Australia +1

  8. LOKI: Memory-Free Null-Space Constrained Lifelong Knowledge Editing

    Jun 18, 2026Masih Eskandar, Miquel Sirera Perelló, Stratis Ioannidis +1Knowledge EditingContinual Learning for LLMs

    Department of Electrical and Computer Engineering Northeastern University Boston, MA, 02115

  9. Catastrophic Forgetting is Low-Rank: A Function-Space Theory for Continual Adaptation

    Jun 16, 2026Ido Nitzan Hidekel, Dan RavivContinual LearningCatastrophic Forgetting

    Tel Aviv University, Tel Aviv, Israel.

  10. When the Same Musical Knowledge Forgets Differently: A Clean Probe of Pathway-Dependent Forgetting

    Jun 13, 2026Yu Liu, Zhiwei Yang, Wenxiao Zhang +8Music UnderstandingAudio-Language Models

    Institute of Information Engineering, CAS · School of Cyber Security, UCAS · The University of Western Australia +1

  11. The Stable Recovery Manifold: Geometric Principles Governing Recoverability in Continual Learning

    Jun 11, 2026Ayushman Trivedi, Bhavika MelwaniContinual LearningGeometric Representation Learning

  12. ECA: Efficient Continual Alignment for Open-Ended Image-to-Text Generation

    Jun 10, 2026Jiangtao Kong, Peijun Zhao, Chun-Fu Chen +4Large Vision-Language ModelsIncremental Learning

    Department of Computer Science, William & Mary, Virginia, USA · Global Technology Applied Research, JPMorgan-Chase, USA · Mohamed bin Zayed University of Artificial Intelligence, Masdar City, Abu Dhabi

  13. FOGO: Forgetting-aware Orthogonalization Optimizer

    Jun 9, 2026Toan Nguyen, Yang Liu, Trung Le +2Deep Learning OptimizationContinual Learning

    School of Computer Science and Engineering University of New South Wales Sydney, Australia · Department of Data Science & AI Monash University Melbourne, Australia · DEVCOM Army Research Laboratory USA

  14. Theoretical Foundations of Continual Learning via Drift-Plus-Penalty

    Jun 7, 2026Nazreen Shah, Govinda Arya, Bharath B. N. +1Continual LearningReplay-Based Continual Learning

    IIIT Delhi · IIT Dharwad

  15. Not Just After One: Sleep-Inspired Replay Prevents Catastrophic Forgetting After Sequential Tasks

    Jun 7, 2026Anthony Bazhenov, Jean Erik Delanois, Giri P. KrishnanReplay-Based Continual LearningMemory Consolidation

    Khoury College of Computer Sciences, Northeastern University, Boston, MA · Department of Medicine, University of California, San Diego, La Jolla, CA, USA · ARTISAN, Georgia Institute of Technology, Atlanta, GA USA

  16. Evaluating the Impact of Task Granularity on Catastrophic Forgetting in Continual Learning

    Jun 6, 2026Emre Alyamac, Himanshu Janmeda, Shashwat Krishna +1Class-Incremental LearningContinual Learning

    College of Engineering · College of Natural Science

  17. Sparse Subspace-to-Expert Sharing for Task-Agnostic Continual Learning

    Jun 5, 2026Fatema Siddika, Md Anwar Hossen, Tanwi Mallick +1Continual Learning for LLMsMixture-of-Experts Language Models

    Iowa State University, Ames, USA · Argonne National Laboratory, USA

  18. CoMetaPNS: Continually Meta-learning Personalized Neural Surrogates for Cardiac Electrophysiology Simulations

    Jun 5, 2026Ryan Missel, Xiajun Jiang, Linwei WangNeural Surrogate ModelingSurrogate Modeling

  19. Unsupervised Continual Clustering via Forward-Backward Knowledge Distillation

    Jun 5, 2026Mohammadreza Sadeghi, Sareh Soleimani, Zihan Wang +1ClusteringUnsupervised Clustering

    Department of Electrical and Computer Engineering, McGill University, Montreal, QC H3A 0E9, Canada · Mila – Quebec AI Institute, Montreal, QC H2S 3H1, Canada

  20. Catastrophic Forgetting as Accessibility Collapse: A Three-Level Framework for Knowledge Persistence in Continual Learning

    Jun 4, 2026Ayushman Trivedi, Bhavika MelwaniContinual LearningCatastrophic Forgetting

    Independent Researchers

  21. Two-Way Is Better Than One: Bidirectional Alignment with Cycle Consistency for Exemplar-Free Class-Incremental Learning

    Jun 4, 2026Hongye Xu, Bartosz KrawczykClass-Incremental LearningPrototype-Based Classification

    Chester F. Carlson Center for Imaging Science · Rochester Institute of Technology

  22. BBR-Net: Boundary-Balanced Replay for Continual Medical Image Segmentation

    Jun 2, 2026Zahid Ullah, Sieun Choi, Jihie KimInteractive Medical Image SegmentationUltrasound Image Segmentation

    Department of Computer Science and Artificial Intelligence, Dongguk University, Seoul, 04620, Republic of Korea

  23. Forgetting is Not Erasure: Recovering Latent Knowledge via Transport Keys

    Jun 1, 2026Archie ChaudhuryContinual LearningRepresentation Alignment

    Axionic Labs

  24. Continual Learning as a Multiphase Moving-Boundary Problem

    Jun 1, 2026Snigdha Chandan KhilarContinual LearningCatastrophic Forgetting

    Independent Researcher

  25. AlphaToken: Decoupling Adaptation and Stability for Path-Aware Response Token Valuation in LLM Post-Training

    Jun 1, 2026Liu Qing, Ou Wu, Yi DuLLM Fine-TuningLanguage Model Post-Training

    Hangzhou Institute for Advanced Study

  26. Turning Back Without Forgetting: Selective Backward Refinement for Parameter-Efficient Continual Learning

    May 31, 2026Anushka Tiwari, Kaiyi JiStorage-Efficient Continual LearningParameter-Efficient Fine-Tuning

    Institute for Artificial Intelligence and Data Science, University at Buffalo, Buffalo, NY, USA · Department of Computer Science and Engineering, University at Buffalo, Buffalo, NY, USA

  27. Revisiting Parameter-Based Knowledge Editing in Large Language Models: Theoretical Limits and Empirical Evidence

    May 30, 2026Wanying Ren, Xin Song, Futing Wang +2Model EditingLanguage Model Editing

    East China Normal University, Shanghai, China · Westlake University, Hangzhou, China · Nanyang Technological University, Singapore

  28. Multi-Objective Reference-Aligned Machine Unlearning

    May 29, 2026Rasa Khosrowshahli, Stephen Asobiela, Beatrice Ombuki-Berman +1Machine UnlearningCatastrophic Forgetting

    Department of Computer Science, Brock University, St. Catharines, ON, Canada · Yousef Haj-Ahmad Department of Engineering, Brock University, St. Catharines, ON, Canada

  29. CRMA: A Spectrally-Bounded Backbone for Modular Continual Fine-Tuning of LLMs

    May 29, 2026Kiran Nayudu, Aswini Nutakki, Sai Vinay Naidu +1Continual Learning for LLMsFine-Tuning

    ModelBrew AI

  30. Parameter Alignment Mitigates Catastrophic Forgetting in Multilingual Expert Language Models

    May 29, 2026Sanchit Ahuja, Terra BlevinsMultilingual Language ModelsContinual Learning for LLMs

    Northeastern University