Catastrophic Forgetting

Catastrophic forgetting describes the phenomenon where artificial neural networks, upon learning new tasks, lose previously acquired knowledge. Current research focuses on mitigating this issue through various strategies, including parameter-efficient fine-tuning methods (like LoRA), generative model-based data replay, and novel optimization algorithms that constrain gradient updates or leverage hierarchical task structures. Addressing catastrophic forgetting is crucial for developing robust and adaptable AI systems capable of continuous learning in real-world applications, particularly in domains like medical imaging, robotics, and natural language processing where data streams are constantly evolving.

Papers

August 29, 2023

Incorporating Neuro-Inspired Adaptability for Continual Learning in Artificial Intelligence
Liyuan Wang, Xingxing Zhang, Qian Li, Mingtian Zhang, Hang Su, Jun Zhu, Yi Zhong
Neural Network Artificial Intelligence Continual LEArning Catastrophic Forgetting Memory Stability

August 27, 2023

Rethinking Exemplars for Continual Semantic Segmentation in Endoscopy Scenes: Entropy-based Mini-Batch Pseudo-Replay
Guankun Wang, Long Bai, Yanan Wu, Tong Chen, Hongliang Ren
Continual LEArning Catastrophic Forgetting Endoscopic Image Continual Semantic Segmentation Latent Replay

August 25, 2023

August 23, 2023

An Intentional Forgetting-Driven Self-Healing Method For Deep Reinforcement Learning Systems
Ahmed Haj Yahmed, Rached Bouchoucha, Houssem Ben Braiek, Foutse Khomh
Deep Reinforcement Learning Continual LEArning Catastrophic Forgetting Practical Method Self Healing

August 22, 2023

Variational Density Propagation Continual Learning
Christopher Angelini, Nidhal Bouaynaya, Ghulam Rasool
Continual LEArning Catastrophic Forgetting Continual Learning Variational Posterior Distribution Drift Variational Continual Learning

August 18, 2023

NAPA-VQ: Neighborhood Aware Prototype Augmentation with Vector Quantization for Continual Learning
Tamasha Malepathirana, Damith Senanayake, Saman Halgamuge
Continual LEArning Catastrophic Forgetting Class Incremental Learning Vector Quantization Prototype Based Prototype Highlighting Current Challenge

August 17, 2023

An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning
Yun Luo, Zhen Yang, Fandong Meng, Yafu Li, Jie Zhou, Yue Zhang
Fine Tuning Catastrophic Forgetting Empirical Study Encoder Decoder Continual Fine Tuning Continual Instruction Tuning

August 7, 2023

August 4, 2023

Class Incremental Learning with Self-Supervised Pre-Training and Prototype Learning
Wenzhuo Liu, Xinjian Wu, Fei Zhu, Mingming Yu, Chuang Wang, Cheng-Lin Liu
Deep Neural Network Catastrophic Forgetting Incremental Learning Class Incremental Learning Self Supervised Pre Training Representation Drift

August 3, 2023

July 31, 2023

Subspace Distillation for Continual Learning
Kaushik Roy, Christian Simon, Peyman Moghadam, Mehrtash Harandi
Continual LEArning Catastrophic Forgetting Continual Learning Tiny ImageNet Novel Knowledge Distillation

July 21, 2023

CLR: Channel-wise Lightweight Reprogramming for Continual Learning
Yunhao Ge, Yuecheng Li, Shuo Ni, Jiaping Zhao, Ming-Hsuan Yang, Laurent Itti
Continual LEArning Catastrophic Forgetting Continual Learning Task Agnostic Channel Wise Lightweight Reprogramming

July 18, 2023

HAT-CL: A Hard-Attention-to-the-Task PyTorch Library for Continual Learning
Xiaotian Duan
Continual LEArning Catastrophic Forgetting Pytorch Model Hard Attention

July 16, 2023

A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning
Zhenyi Wang, Enneng Yang, Li Shen, Heng Huang
Deep Learning Continual LEArning Catastrophic Forgetting Comprehensive Survey Anti Forgetting Task Knowledge

July 2, 2023

Don't Memorize; Mimic The Past: Federated Class Incremental Learning Without Episodic Memory
Sara Babakniya, Zalan Fabian, Chaoyang He, Mahdi Soltanolkotabi, Salman Avestimehr
Deep Learning Model Continual LEArning Catastrophic Forgetting Incremental Learning Episodic Memory Novel Mimic Algorithm

June 29, 2023

The Importance of Robust Features in Mitigating Catastrophic Forgetting
Hikmat Khan, Nidhal C. Bouaynaya, Ghulam Rasool
Continual LEArning Adversarial Robustness Catastrophic Forgetting Importance Aware Robust Dataset

Catastrophic Forgetting

Papers

Incorporating Neuro-Inspired Adaptability for Continual Learning in Artificial Intelligence

Rethinking Exemplars for Continual Semantic Segmentation in Endoscopy Scenes: Entropy-based Mini-Batch Pseudo-Replay

GRASP: A Rehearsal Policy for Efficient Online Continual Learning

Dynamic Residual Classifier for Class Incremental Learning

An Intentional Forgetting-Driven Self-Healing Method For Deep Reinforcement Learning Systems

Variational Density Propagation Continual Learning

NAPA-VQ: Neighborhood Aware Prototype Augmentation with Vector Quantization for Continual Learning

An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning

Heterogeneous Forgetting Compensation for Class-Incremental Learning

Do You Remember? Overcoming Catastrophic Forgetting for Fake Audio Detection

AdaER: An Adaptive Experience Replay Approach for Continual Lifelong Learning

Class Incremental Learning with Self-Supervised Pre-Training and Prototype Learning

Improving Replay Sample Selection and Storage for Less Forgetting in Continual Learning

Balanced Destruction-Reconstruction Dynamics for Memory-replay Class Incremental Learning

Subspace Distillation for Continual Learning

CLR: Channel-wise Lightweight Reprogramming for Continual Learning

HAT-CL: A Hard-Attention-to-the-Task PyTorch Library for Continual Learning

A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning

Don't Memorize; Mimic The Past: Federated Class Incremental Learning Without Episodic Memory

The Importance of Robust Features in Mitigating Catastrophic Forgetting