Catastrophic Forgetting

Catastrophic forgetting describes the phenomenon where artificial neural networks, upon learning new tasks, lose previously acquired knowledge. Current research focuses on mitigating this issue through various strategies, including parameter-efficient fine-tuning methods (like LoRA), generative model-based data replay, and novel optimization algorithms that constrain gradient updates or leverage hierarchical task structures. Addressing catastrophic forgetting is crucial for developing robust and adaptable AI systems capable of continuous learning in real-world applications, particularly in domains like medical imaging, robotics, and natural language processing where data streams are constantly evolving.

Papers

November 12, 2024

On the Convergence of Continual Federated Learning Using Incrementally Aggregated Gradients
Satish Kumar Keshri, Nazreen Shah, Ranjitha Prasad
Catastrophic Forgetting Early Stage Convergence End to End Learning Federated Continual Learning Aggregated Gradient

November 11, 2024

November 8, 2024

November 7, 2024

November 6, 2024

November 5, 2024

Hierarchical Orchestra of Policies
Thomas P Cannon, Özgür Simsek
Catastrophic Forgetting Part Whole Hierarchy Prior Policy Continual Reinforcement Learning Lifelong Reinforcement Learning Co Orchestration Sequential Task

November 4, 2024

November 3, 2024

Efficient Non-Exemplar Class-Incremental Learning with Retrospective Feature Synthesis
Liang Bai, Hong Song, Yucong Lin, Tianyu Fu, Deqiang Xiao, Danni Ai, Jingfan Fan, Jian Yang
Catastrophic Forgetting Class Incremental Learning Exemplar Image Incremental Learning Framework Feature Synthesis

October 31, 2024

EXACFS -- A CIL Method to mitigate Catastrophic Forgetting
S Balasubramanian, M Sai Subramaniam, Sai Sriram Talasu, P Yedu Krishna, Manepalli Pranav Phanindra Sai, Ravi Mukkamala, Darshan Gera
Deep Neural Network Continual LEArning Catastrophic Forgetting Feature Importance ImageNet LT

October 28, 2024

October 23, 2024

CLEAR: Character Unlearning in Textual and Visual Modalities
Alexey Dontsov, Dmitrii Korzh, Alexey Zhavoronkin, Boris Mikheev, Denis Bobkov, Aibek Alanov, Oleg Y. Rogov, Ivan Oseledets, Elena Tutubalina
Deep Learning Model Multimodal Large Language Model Catastrophic Forgetting Machine Unlearning Visual Modality Knowledge Unlearning Class Enhanced Attentive Response

October 22, 2024

Catastrophic Forgetting

Papers

On the Convergence of Continual Federated Learning Using Incrementally Aggregated Gradients

Slowing Down Forgetting in Continual Learning

An Efficient Memory Module for Graph Few-Shot Class-Incremental Learning

Online-LoRA: Task-free Online Continual Learning via Low Rank Adaptation

Towards Lifelong Few-Shot Customization of Text-to-Image Diffusion

Alopex: A Computational Framework for Enabling On-Device Function Calls with LLMs

Gradient Localization Improves Lifelong Pretraining of Language Models

Reducing catastrophic forgetting of incremental learning in the absence of rehearsal memory with task-specific token

Exploring the Stability Gap in Continual Learning: The Role of the Classification Head

Hierarchical Orchestra of Policies

SAFE: Slow and Fast Parameter-Efficient Tuning for Continual Learning with Pre-Trained Models

Masked Autoencoders are Parameter-Efficient Federated Continual Learners

FPPL: An Efficient and Non-IID Robust Federated Continual Learning Framework

Efficient Non-Exemplar Class-Incremental Learning with Retrospective Feature Synthesis

EXACFS -- A CIL Method to mitigate Catastrophic Forgetting

UFT: Unifying Fine-Tuning of SFT and RLHF/DPO/UNA through a Generalized Implicit Reward Function

Task Confusion and Catastrophic Forgetting in Class-Incremental Learning: A Mathematical Framework for Discriminative and Generative Modelings

CLEAR: Character Unlearning in Textual and Visual Modalities

Exploring Forgetting in Large Language Model Pre-Training

Controlled Low-Rank Adaptation with Subspace Regularization for Continued Training on Large Language Models