Catastrophic Forgetting

Catastrophic forgetting describes the phenomenon where artificial neural networks, upon learning new tasks, lose previously acquired knowledge. Current research focuses on mitigating this issue through various strategies, including parameter-efficient fine-tuning methods (like LoRA), generative model-based data replay, and novel optimization algorithms that constrain gradient updates or leverage hierarchical task structures. Addressing catastrophic forgetting is crucial for developing robust and adaptable AI systems capable of continuous learning in real-world applications, particularly in domains like medical imaging, robotics, and natural language processing where data streams are constantly evolving.

Papers

March 27, 2024

March 22, 2024

Defying Imbalanced Forgetting in Class Incremental Learning
Shixiong Xu, Gaofeng Meng, Xing Nie, Bolin Ni, Bin Fan, Shiming Xiang
Catastrophic Forgetting Class Incremental Learning Anti Forgetting Replay Based

March 21, 2024

Preventing Catastrophic Forgetting through Memory Networks in Continuous Detection
Gaurav Bhatt, James Ross, Leonid Sigal
Object Detection Catastrophic Forgetting Event Detection Detection Transformer Memory Network Continual Text Classification

March 20, 2024

A Unified and General Framework for Continual Learning
Zhenyi Wang, Yan Li, Li Shen, Heng Huang
Continual LEArning Catastrophic Forgetting Unified Alignment General Framework

March 15, 2024

March 11, 2024

March 9, 2024

Towards Efficient Replay in Federated Incremental Learning
Yichen Li, Qunwei Li, Haozhao Wang, Ruixuan Li, Wenliang Zhong, Guannan Zhang
Federated Learning Catastrophic Forgetting Data Heterogeneity Efficient Generated Object Replay

March 8, 2024

Continual Learning and Catastrophic Forgetting
Gido M. van de Ven, Nicholas Soures, Dhireesha Kudithipudi
Neural Network Deep Learning Continual LEArning Catastrophic Forgetting Non Stationary

March 6, 2024

GUIDE: Guidance-based Incremental Learning with Diffusion Models
Bartosz Cywiński, Kamil Deja, Tomasz Trzciński, Bartłomiej Twardowski, Łukasz Kuciński
Diffusion Model Continual LEArning Catastrophic Forgetting Continual Learning Incremental Learning Generative Approach Generative Replay

March 4, 2024

Gradient Correlation Subspace Learning against Catastrophic Forgetting
Tammuz Dubnov, Vishal Thengane
Catastrophic Forgetting Class Incremental Learning Efficient Continual Learning Gradient Subspace

March 2, 2024

Mitigating Catastrophic Forgetting in Large Language Models with Self-Synthesized Rehearsal
Jianheng Huang, Leyang Cui, Ante Wang, Chengyi Yang, Xinting Liao, Linfeng Song, Junfeng Yao, Jinsong Su
Large Language Model Continual LEArning Catastrophic Forgetting Rehearsal Based Generative Rehearsal

February 29, 2024

February 28, 2024

Generalizable Two-Branch Framework for Image Class-Incremental Learning
Chao Wu, Xiaobin Chang, Ruixuan Wang
Deep Neural Network Catastrophic Forgetting Continual Learning Class Incremental Learning Dual Branch Lightweight Convolutional Neural Network

February 27, 2024

MCF-VC: Mitigate Catastrophic Forgetting in Class-Incremental Learning for Multimodal Video Captioning
Huiyu Xiong, Lanxiao Wang, Heqian Qiu, Taijin Zhao, Benliu Qiu, Hongliang Li
Catastrophic Forgetting Incremental Learning Class Incremental Learning Anti Forgetting Video Captioning

February 26, 2024

Incremental Concept Formation over Visual Images Without Catastrophic Forgetting
Nicki Barari, Xin Lian, Christopher J. MacLellan
Deep Neural Network Catastrophic Forgetting Vision Task Human Cognition Visual Concept Visual Classification Concept Formation

February 23, 2024

MemoryPrompt: A Light Wrapper to Improve Context Tracking in Pre-trained Language Models
Nathanaël Carraz Rakotonirina, Marco Baroni
Pre Trained Language Model Catastrophic Forgetting Transformer Based Language Model Context Awareness Memory Augmented New Wrapper Method

Catastrophic Forgetting

Papers

Self-Expansion of Pre-trained Models with Mixture of Adapters for Continual Learning

Intelligent Learning Rate Distribution to reduce Catastrophic Forgetting in Transformers

Defying Imbalanced Forgetting in Class Incremental Learning

Preventing Catastrophic Forgetting through Memory Networks in Continuous Detection

A Unified and General Framework for Continual Learning

Regularization-Based Efficient Continual Learning in Deep State-Space Models

Don't Half-listen: Capturing Key-part Information in Continual Instruction Tuning

Semantic Residual Prompts for Continual Learning

On the Diminishing Returns of Width for Continual Learning

Towards Efficient Replay in Federated Incremental Learning

Continual Learning and Catastrophic Forgetting

GUIDE: Guidance-based Incremental Learning with Diffusion Models

Gradient Correlation Subspace Learning against Catastrophic Forgetting

Mitigating Catastrophic Forgetting in Large Language Models with Self-Synthesized Rehearsal

Leveraging AI Predicted and Expert Revised Annotations in Interactive Segmentation: Continual Tuning or Full Training?

Analyzing and Reducing Catastrophic Forgetting in Parameter Efficient Tuning

Generalizable Two-Branch Framework for Image Class-Incremental Learning

MCF-VC: Mitigate Catastrophic Forgetting in Class-Incremental Learning for Multimodal Video Captioning

Incremental Concept Formation over Visual Images Without Catastrophic Forgetting

MemoryPrompt: A Light Wrapper to Improve Context Tracking in Pre-trained Language Models