Catastrophic Forgetting

Catastrophic forgetting describes the phenomenon where artificial neural networks, upon learning new tasks, lose previously acquired knowledge. Current research focuses on mitigating this issue through various strategies, including parameter-efficient fine-tuning methods (like LoRA), generative model-based data replay, and novel optimization algorithms that constrain gradient updates or leverage hierarchical task structures. Addressing catastrophic forgetting is crucial for developing robust and adaptable AI systems capable of continuous learning in real-world applications, particularly in domains like medical imaging, robotics, and natural language processing where data streams are constantly evolving.

Papers

June 16, 2023

Continual Adaptation of Vision Transformers for Federated Learning
Shaunak Halbe, James Seale Smith, Junjiao Tian, Zsolt Kira
Catastrophic Forgetting Federat\textbf{T}ed Cl\textbf{A}ss Continual Lea\textbf{R}nin\textbf{G Computational Cost Federated Continual Learning Liver Tumor Three Dimensional Terzaghi Consolidation

June 7, 2023

Meta-Learning in Spiking Neural Networks with Reward-Modulated STDP
Arsham Gholamzadeh Khoee, Alireza Javaheri, Saeed Reza Kheradpisheh, Mohammad Ganjtabesh
Catastrophic Forgetting Spiking Neural Network Neuromorphic Hardware Spike Based Neuromorphic

June 5, 2023

Continual Learning with Pretrained Backbones by Tuning in the Input Space
Simone Marullo, Matteo Tiezzi, Marco Gori, Stefano Melacci, Tinne Tuytelaars
Deep Learning Model Continual LEArning Catastrophic Forgetting Continual Learning Structured Output Pre Trained Backbone

June 2, 2023

On Knowledge Editing in Federated Learning: Perspectives, Challenges, and Future Directions
Leijie Wu, Song Guo, Junxiao Wang, Zicong Hong, Jie Zhang, Jingren Zhou
Technical Challenge Catastrophic Forgetting Anti Forgetting Synthesized View Future Direction Knowledge Editing

June 1, 2023

May 30, 2023

History Repeats: Overcoming Catastrophic Forgetting For Event-Centric Temporal Knowledge Graph Completion
Mehrnoosh Mirtaheri, Mohammad Rostami, Aram Galstyan
Catastrophic Forgetting Continual Learning Temporal Knowledge Graph Experience Replay COntinual Learning Framework Temporal Knowledge Graph Completion Temporal Regularization

May 26, 2023

Mitigating Catastrophic Forgetting in Long Short-Term Memory Networks
Ketaki Joshi, Raghavendra Pradyumna Pothukuchi, Andre Wibisono, Abhishek Bhattacharjee
Continual LEArning Long Short Term Memory Catastrophic Forgetting LSTM Network

May 25, 2023

May 24, 2023

Dealing with Cross-Task Class Discrimination in Online Continual Learning
Yiduo Guo, Bing Liu, Dongyan Zhao
Continual LEArning Catastrophic Forgetting Class Incremental Learning Online Continual Learning Replay Based

May 22, 2023

Mitigating Catastrophic Forgetting for Few-Shot Spoken Word Classification Through Meta-Learning
Ruan van der Merwe, Herman Kamper
Natural Language Catastrophic Forgetting Meta Learning Shot Classification Continual Learning Scenario Semantic Class Speech Classification Learning Loop

May 20, 2023

Mitigating Catastrophic Forgetting in Task-Incremental Continual Learning with Adaptive Classification Criterion
Yun Luo, Xiaotian Lin, Zhen Yang, Fandong Meng, Jie Zhou, Yue Zhang
Continual LEArning Catastrophic Forgetting Contrastive Loss Supervised Contrastive Learning Task Learning Adaptive Classification

May 16, 2023

May 12, 2023

May 10, 2023

Accelerating Batch Active Learning Using Continual Learning Techniques
Arnav Das, Gantavya Bhatt, Megh Bhalerao, Vianne Gao, Rui Yang, Jeff Bilmes
Active Learning Catastrophic Forgetting Continual Learning Replay Based Continual Learning Batch Active Learning Continual Learning Paradigm

May 8, 2023

BiRT: Bio-inspired Replay in Vision Transformers for Continual Learning
Kishaan Jeeveswaran, Prashant Bhat, Bahram Zonooz, Elahe Arani
Vision Transformer Continual LEArning Catastrophic Forgetting Rehearsal Based Continual Learning Rehearsal Based Video

Catastrophic Forgetting

Papers

Continual Adaptation of Vision Transformers for Federated Learning

Meta-Learning in Spiking Neural Networks with Reward-Modulated STDP

Continual Learning with Pretrained Backbones by Tuning in the Input Space

On Knowledge Editing in Federated Learning: Perspectives, Challenges, and Future Directions

Out-of-distribution forgetting: vulnerability of continual learning to intra-class distribution shift

Teacher Agent: A Knowledge Distillation-Free Framework for Rehearsal-based Video Incremental Learning

History Repeats: Overcoming Catastrophic Forgetting For Event-Centric Temporal Knowledge Graph Completion

Mitigating Catastrophic Forgetting in Long Short-Term Memory Networks

SketchOGD: Memory-Efficient Continual Learning

Overcoming Catastrophic Forgetting in Massively Multilingual Continual Learning

Condensed Prototype Replay for Class Incremental Learning

Dealing with Cross-Task Class Discrimination in Online Continual Learning

Mitigating Catastrophic Forgetting for Few-Shot Spoken Word Classification Through Meta-Learning

Mitigating Catastrophic Forgetting in Task-Incremental Continual Learning with Adaptive Classification Criterion

CQural: A Novel CNN based Hybrid Architecture for Quantum Continual Machine Learning

Retentive or Forgetful? Diving into the Knowledge Memorizing Mechanism of Language Models

Continual Learning for End-to-End ASR by Averaging Domain Experts

Prompt Learning to Mitigate Catastrophic Forgetting in Cross-lingual Transfer for Open-domain Dialogue Generation

Accelerating Batch Active Learning Using Continual Learning Techniques

BiRT: Bio-inspired Replay in Vision Transformers for Continual Learning