Overview

Catastrophic forgetting occurs when learning new tasks or adapting to shifting data degrades performance on earlier tasks, a central challenge for systems expected to learn over time. Common approaches include replaying examples, distilling prior outputs, regularizing important weights or gradients, and isolating updates with adapters or modular experts. Researchers study these methods in sequential vision and language benchmarks as well as foundation-model fine-tuning, multimodal systems, and robotics, measuring both new-task performance and retention. No single approach works best everywhere: stronger protection can limit adaptation, while memory, compute, and benchmark choices affect results; apparent forgetting may also reflect knowledge becoming inaccessible rather than erased.

Latest papers

CardsList

374 papers32 papers in the last four weeks, down 45% on the four weeks before. 0.2% of all new papers.

  1. Are Self-Evolving Agents Continual Learners?

    Oct 19, 2026Xiaoyin Chen, Yizheng Huang, Yinglun Xu +4Continual Learning for LLM AgentsSelf-Evolving Agents

  2. NADS: Navigator-Guided Data Selection for Mitigating Catastrophic Forgetting in Fine-Tuning

    Oct 19, 2026Jianhao Zhang, Ou Wu, Yi DuLLM Fine-TuningData Selection for Language Model Fine-Tuning

  3. OPDArena: Understanding Learning and Forgetting in On-Policy Distillation

    Oct 19, 2026Yuanze Xu, Tianzhu Ye, Hui Sun +3On-Policy DistillationCatastrophic Forgetting

  4. Learning without Overwriting: Structural Knowledge Fusion for Continual Learning

    Oct 18, 2026Kanishk Bakshi, Amitansu Pattanaik, Kathiravan SrinivasanClass-Incremental LearningIncremental Learning

  5. Towards Continual Learning: Mitigating Forgetting With Subspace Tuning

    Oct 15, 2026Arnav Chavan, Kusha Sareen, Rishabh Tiwari +6Continual Learning for LLMsLLM Fine-Tuning

  6. Sparse Autoencoders Can Induce Local Geometry for Mitigating Catastrophic Forgetting

    Oct 15, 2026Kamil Faber, Katarzyna Filus, Łukasz PawelaCatastrophic ForgettingRegularization-Based Continual Learning

  7. Functional Subnetwork Localization for Efficient Continual Learning in Large Language Models

    Oct 15, 2026Shuoran Jiang, Wentao Zhu, Yang Xiang +8Continual Learning for LLMsIncremental Learning

  8. Overcoming Catastrophic Forgetting: Continual Learning for Radar Waveform Classification

    Oct 13, 2026Mohamed Rabie, Chinthana J. Panagamuwa, Konstantinos G. KyriakopoulosContinual LearningCatastrophic Forgetting

  9. Collapse, Repair, and Drift: Dissecting Fine Tuning Dynamics in Speech Language Models

    Oct 13, 2026Yichi Wang, Wangjin Zhou, Yizhou Zhang +4Supervised Fine-TuningNeural Network Training Dynamics

  10. Confine to the Task, Anchor the Rest: Concept-Guided Retention Regularizer for Parameter-Efficient Fine-Tuning

    Oct 13, 2026Ibna Kowsar, Ivan Medri, Fanyu Meng +1Fine-TuningParameter-Efficient Fine-Tuning

  11. When Skills Remain But Access Fails: Factorized Skill Memory for Continual Skill Internalization

    Oct 11, 2026Haomiao Qiu, Yining Tang, Hongyi Wan +3Continual Learning for LLM AgentsAgent Skill Learning

  12. Formalizing Forgetting: Theory of Forgetting for Online Continual Learning

    Oct 10, 2026Renjie Cao, Yifei Cheng, Li ShenOnline Continual LearningCatastrophic Forgetting

  13. Compositional Concepts Erasing: Disentangling Object-Style via Domain Manifold Unlearning

    Oct 10, 2026Yang Xiao, Kaiyuan Deng, Gen Li +4Diffusion Model UnlearningMachine Unlearning

  14. Static Gradient Attribution Underperforms a Density-Matched Random Mask on Loss-Based Forgetting Within LoRA B-Matrix

    Oct 8, 2026Yu Sun, Junwei Zhou, Zuodong Xiang +5Storage-Efficient Continual LearningContinual Learning for LLMs

  15. Self-Preservation in Language Models

    Oct 8, 2026Dongkyu Cho, Rumi Chunara, Ali BehrouzContinual Learning for LLMsReplay-Based Continual Learning

  16. Where to Adapt Matters: Layer-Selective Fine-Tuning for Capability Retention

    Oct 8, 2026Zhiqiang Pang, Zihong Sun, Qi Xie +3LLM Fine-TuningSelective Fine-Tuning

  17. From a Prompt to Repertoires: Evolving Functional REpertoires Enable LLM Continual Learning

    Oct 8, 2026Fengyuan Liu, Yue Wang, Hangxi Guo +4Continual Learning for LLMsPrompt Learning

  18. SFT-as-Context Mitigates Forgetting in Supervised Fine-Tuning

    Oct 8, 2026Kenan Tang, Andong Hua, Chengxuan Qian +2LLM Fine-TuningCatastrophic Forgetting

  19. Stability-Plasticity Balance via Singular-Vector Selection in LLM Continual Learning

    Oct 8, 2026Lingxiang Wang, Hainan Zhang, Liang Pang +2Continual Learning for LLMsParameter-Efficient Fine-Tuning

  20. A Deafening Silence: Catastrophic Forgetting Lives in the Output Embeddings of Tokens the Data Never Speaks

    Oct 7, 2026Jonghyun Han, Younghoon Song, Jongyoul ParkCatastrophic ForgettingContinual Learning for LLMs

  21. Are Parameter-Efficient Fine-tuning Methods Really Different?

    Oct 6, 2026Yikuan Li, Pinyan Lu, Fanghui LiuParameter-Efficient Fine-TuningLow-Rank Adaptation

  22. When Forgetting is not Catastrophic: On the Mechanics of Spurious Forgetting

    Oct 6, 2026Vedant Palit, Florent Draye, Nicolas Zucchet +2Memorization in Language ModelsLLM Fine-Tuning

  23. Catastrophic Forgetting in Sequential Thermal Anti-UAV Detection: The Role of Scale-Conditioned Gradient Imbalance

    Oct 6, 2026Khac Duc Giang Nguyen, Seyed Sahand Mohammadi Ziabari, Ali Mohammed Mansoor AlsahagContinual Object DetectionReplay-Based Continual Learning

  24. CoDe-LoRA: Mitigating the Orthogonality Dilemma in Continual Learning of LLMs via Knowledge Consolidation and Decoupling

    Oct 6, 2026Maoqi Liu, Quan Fang, Yufei HeContinual Learning for LLMsLow-Rank Adaptation

  25. Off-Policy Merging Beats On-Policy Self-Distillation for Continual Learning

    Oct 5, 2026Chen Henry Wu, Thomas Zhang, Aditi RaghunathanContinual Learning for LLMsOn-Policy Distillation

  26. Learning without Overwriting: A Theory of Self-Distillation and Supervised Fine-Tuning in Continual Reasoning

    Oct 4, 2026Shinichi Uemura, Taiji SuzukiSupervised Fine-TuningContinual Learning for LLMs

  27. CoDG-Net: Structure-Guided Style Diffusion and Collaborative Learning to Mitigate Catastrophic Forgetting in Medical Image Domain Generalization

    Oct 4, 2026Yucheng Song, Jincan Wang, Haokang Ding +3Data AugmentationCross-Domain Generalization