Catastrophic Forgetting

Follow this topic Ask about this topic

Catastrophic forgetting occurs when learning new tasks or adapting to shifting data degrades performance on earlier tasks, a central challenge for systems expected to learn over time. Common approaches include replaying examples, distilling prior outputs, regularizing important weights or gradients, and isolating updates with adapters or modular experts. Researchers study these methods in sequential vision and language benchmarks as well as foundation-model fine-tuning, multimodal systems, and robotics, measuring both new-task performance and retention. No single approach works best everywhere: stronger protection can limit adaptation, while memory, compute, and benchmark choices affect results; apparent forgetting may also reflect knowledge becoming inaccessible rather than erased.

Latest papers

351 papers59 papers in the last four weeks, up 119% on the four weeks before. 0.3% of all new papers.

Representation Finetuning for Continual Learning
Mar 11, 2026Haihua Luo, Xuming Ran, Tommi Kärkkäinen +4Continual Learning Fine-Tuning
Faculty of Information Technology, University of Jyväskylä, Jyväskylä, Finland · National University of Singapore, Singapore, Singapore · School of Computer Science and Technology, Dalian University of Technology, Dalian, China
MSSR: Memory-Aware Adaptive Replay for Continual LLM Fine-Tuning
Mar 10, 2026Yiyang Lu, Yu He, Jianlong Chen +1Continual Learning for LLMs Replay-Based Continual Learning
The Chinese University of Hong Kong, Shenzhen · Nanjing University
On Catastrophic Forgetting in Low-Rank Decomposition-Based Parameter-Efficient Fine-Tuning
Mar 10, 2026Muhammad Ahmad, Jingjing Zheng, Yankai CaoParameter-Efficient Fine-Tuning Catastrophic Forgetting
Department of Electrical and Computer Engineering, The University of British Columbia · Department of Mathematics, The University of British Columbia · Department of Chemical and Biological Engineering, The University of British Columbia
Guided Prompt Evolution for Vision-Language Models Adaptation
Mar 10, 2026Enming Zhang, Jiayang Li, Yanlong Wang +3VLM Adaptation Few-Shot Learning
Tsinghua Shenzhen International Graduate School, Tsinghua University · Sun Yat-sen University · Chinese University of Hong Kong, Shenzhen
Causally Sufficient and Necessary Feature Expansion for Class-Incremental Learning
Mar 10, 2026Zhen Zhang, Jielei Chu, Jiangtao Hu +4Class-Incremental Learning Catastrophic Forgetting
School of Computing and Artificial Intelligence, Southwest Jiaotong University, Chengdu 611756, China. · Engineering Research Center of Sustainable Urban Intelligent Transportation, Ministry of Education, Chengdu, Sichuan 611756, China · National Engineering Laboratory of Integrated Transportation Big Data Application Technology, Southwest Jiaotong University, Chengdu, Sichuan 611756, China. +2
Can You Hear, Localize, and Segment Continually? An Exemplar-Free Continual Learning Benchmark for Audio-Visual Segmentation
Mar 9, 2026Siddeshwar Raghavan, Gautham Vinod, Bruce Coburn +1Audio-Visual Understanding Continual Learning
Purdue University, West Lafayette, IN 47906, USA
Grow, Don't Overwrite: Fine-tuning Without Forgetting
Mar 9, 2026Dyah Adila, Hanna Mazzawi, Benoit Dherin +1Catastrophic Forgetting Parameter-Efficient Fine-Tuning
University of Wisconsin-Madison · Google Research
Don't Look Back in Anger: MAGIC Net for Streaming Continual Learning with Temporal Dependence
Mar 9, 2026Federico Giannini, Sandro D'Andrea, Emanuele Della ValleContinual Learning Online Continual Learning
DEIB - Politecnico di Milano
Context Channel Capacity: An Information-Theoretic Framework for Understanding Catastrophic Forgetting
Mar 8, 2026Ran ChengCatastrophic Forgetting Online Continual Learning
UnSCAR: Universal, Scalable, Controllable, and Adaptable Image Restoration
Mar 8, 2026Debabrata Mandal, Soumitri Chattopadhyay, Yujie Wang +2All-in-One Image Restoration Catastrophic Forgetting
University of North Carolina at Chapel Hill · University of California, San Diego
Continual Adaptation for Pacific Indigenous Speech Recognition
Mar 6, 2026Yang Xiao, Aso Mahmudi, Nick Thieberger +3Automatic Speech Recognition Low-Resource Speech Recognition
The University of Melbourne · UNSW Sydney
Lifelong Language-Conditioned Robotic Manipulation Learning
Mar 5, 2026Xudong Wang, Zebin Han, Zhiyu Liu +5Language-Conditioned Robot Manipulation Continual Robot Learning
State Key Laboratory of Robotics and Intelligent Systems, Shenyang Institute of Automation, Chinese Academy of Sciences · University of Chinese Academy of Sciences · North University of China +2
Why Do Neural Networks Forget: A Study of Collapse in Continual Learning
Mar 4, 2026Yunqin Zhu, Jun JinCatastrophic Forgetting Incremental Learning
Department of Electrical and Computer Engineering, University of Alberta, Edmonton, Canada
Pretrained Vision-Language-Action Models are Surprisingly Resistant to Forgetting in Continual Learning
Mar 4, 2026Huihan Liu, Changyeon Kim, Bo Liu +2Continual Robot Learning Continual Learning for VLA Models
The University of Texas at Austin · KAIST · Microsoft Superintelligence.
cPNN: Continuous Progressive Neural Networks for Evolving Streaming Time Series
Mar 3, 2026Federico Giannini, Giacomo Ziffer, Emanuele Della ValleOnline Continual Learning Concept Drift
DEIB, Politecnico di Milano, Milano, Italy
Temporal Imbalance of Positive and Negative Supervision in Class-Incremental Learning
Mar 2, 2026Jinge Ma, Fengqing ZhuClass-Incremental Learning Catastrophic Forgetting
Purdue University, West Lafayette, Indiana, U.S.A.
Principled Fast and Meta Knowledge Learners for Continual Reinforcement Learning
Mar 1, 2026Ke Sun, Hongming Zhang, Jun Jin +4Reinforcement Learning Continual RL
University of Alberta · National Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institution of Automation, Chinese Academy of Sciences · Edmonton Research Center, Huawei Canada +2
IDER: IDempotent Experience Replay for Reliable Continual Learning
Feb 28, 2026Zhanwang Liu, Yuting Li, Haoyuan Gao +4Continual Learning Catastrophic Forgetting
School of Computer Science, Shanghai Jiao Tong University · State Key Laboratory of General Artificial Intelligence, BIGAI · Lehigh University +1
Decoupling Stability and Plasticity for Multi-Modal Test-Time Adaptation
Feb 28, 2026Yongbo He, Zirun Guo, Tao JinTest-Time Adaptation Test-Time Adaptation of VLMs
Zhejiang University
Quantifying Catastrophic Forgetting in IoT Intrusion Detection Systems
Feb 27, 2026Sourasekhar Banerjee, David Bergqvist, Salman Toor +2IoT Intrusion Detection Catastrophic Forgetting
Uppsala University, Department of Information Technology, Sweden · Scaleout Systems AB, Sweden · Ericsson Research, Research Area AI, Sweden
Fine-Tuning Without Forgetting In-Context Learning: A Theoretical Analysis of Linear Attention Models
Feb 26, 2026Chungpa Lee, Jy-yong Sohn, Kangwook LeeLLM Fine-Tuning In-Context Learning
Yonsei University, Korea · University of Wisconsin–Madison, USA · KRAFTON, Korea +1
$φ$ -DPO: Fairness Direct Preference Optimization Approach to Continual Learning in Large Multimodal Models
Feb 26, 2026Thanh-Dat Truong, Huu-Thien Tran, Jackson Cothren +2Continual Learning Direct Preference Optimization
CVIU Lab, University of Arkansas, USA · Dep. of Geosciences, University of Arkansas, USA · Carnegie Mellon University, USA
Exploring the Impact of Parameter Update Magnitude on Forgetting and Generalization of Continual Learning
Feb 24, 2026JinLi He, Liang Bai, Xian YangContinual Learning Catastrophic Forgetting
Institute of Intelligent Information Processing, Shanxi University, Taiyuan, 030006, China · Alliance Manchester Business School, The University of Manchester, Manchester, M13 9PL, UK
Understanding the Role of Rehearsal Scale in Continual Learning under Varying Model Capacities
Feb 24, 2026JinLi He, Liang Bai, Xian YangContinual Learning Replay-Based Continual Learning
Institute of Intelligent Information Processing, Shanxi University, Taiyuan, 030006, China · Alliance Manchester Business School, The University of Manchester, Manchester, M13 9PL, UK
GONE: Structural Knowledge Unlearning via Neighborhood-Expanded Distribution Shaping
Feb 21, 2026Chahana Dahal, Ashutosh Balasubramaniam, Zuobin XiongMachine Unlearning LLM Unlearning
University of Nevada, Las Vegas · Indian Institute of Technology Guwahati
Non-Interfering Weight Fields: Treating Model Parameters as a Continuously Extensible Function
Feb 20, 2026Sarim ChaudhryContinual Learning for LLMs Catastrophic Forgetting
Purdue University
Audio-Visual Continual Test-Time Adaptation without Forgetting
Feb 20, 2026Sarthak Kumar Maharana, Akshay Mehra, Bhavya Ramakrishna +2Cross-Modal Learning Source-Free Domain Adaptation
The University of Texas at Dallas, Richardson, TX 75080, USA · Dolby Laboratories, Inc., San Francisco, CA 94103, USA
Catastrophic Forgetting Resilient One-Shot Incremental Federated Learning
Feb 19, 2026Obaidullah Zaland, Zulfiqar Ahmad Khan, Monowar BhuyanFederated Learning Federated Continual Learning
Department of Computing Science, Umeå University, Umeå, SE-90187, Sweden
CapTrack: Multifaceted Evaluation of Forgetting in LLM Post-Training
Feb 19, 2026Lukas Thede, Stefan Winzeck, Zeynep Akata +1LLM Post-Training Catastrophic Forgetting
Thomson Reuters Foundational Research, 5 Canada Square, London · Tübingen AI Center, University of Tübingen · Helmholtz Munich +2
Updating Parametric Knowledge with Context Distillation Retains Post-Training Capabilities
Feb 17, 2026Shankar Padmanabhan, Mustafa Omer Gul, Tanya GoyalContinual Learning for LLMs Context Distillation
Department of Computer Science, Cornell University.

Mar 11, 2026J/K move · Enter open · S save

Representation Finetuning for Continual Learning

Haihua Luo, Xuming Ran, Tommi Kärkkäinen, Huiyan Xue+3

Faculty of Information Technology, University of Jyväskylä, Jyväskylä, Finland · National University of Singapore, Singapore, Singapore · School of Computer Science and Technology, Dalian University of Technology, Dalian, China

Continual Learning Fine-Tuning Parameter-Efficient Fine-Tuning Catastrophic Forgetting

Open paper AskPDF arXiv

The world is inherently dynamic, and continual learning aims to enable models to adapt to ever-evolving data streams. While pre-trained models have shown powerful performance in continual learning, they still require finetuning to adapt effectively to downstream tasks. However, prevailing Parameter-Efficient Fine-Tuning (PEFT) methods operate through empirical, black-box optimization at the weight level. These approaches lack explicit control over representation drift, leading to sensitivity to domain shifts and catastrophic forgetting in continual learning scenarios. In this work, we introduce Continual Representation Learning (CoRe), a novel framework that for the first time shifts the finetuning paradigm from weight space to representation space. Unlike conventional methods, CoRe performs task-specific interventions within a low-rank linear subspace of hidden representations, adopting a learning process with explicit objectives, which ensures stability for past tasks while maintaining plasticity for new ones. By constraining updates to a low-rank subspace, CoRe achieves exceptional parameter efficiency. Extensive experiments across multiple continual learning benchmarks demonstrate that CoRe not only preserves parameter efficiency but also significantly outperforms existing state-of-the-art methods. Our work introduces representation finetuning as a new, more effective and interpretable paradigm for continual learning.