Catastrophic Forgetting

Follow this topic Ask about this topic

Catastrophic forgetting occurs when learning new tasks or adapting to shifting data degrades performance on earlier tasks, a central challenge for systems expected to learn over time. Common approaches include replaying examples, distilling prior outputs, regularizing important weights or gradients, and isolating updates with adapters or modular experts. Researchers study these methods in sequential vision and language benchmarks as well as foundation-model fine-tuning, multimodal systems, and robotics, measuring both new-task performance and retention. No single approach works best everywhere: stronger protection can limit adaptation, while memory, compute, and benchmark choices affect results; apparent forgetting may also reflect knowledge becoming inaccessible rather than erased.

Latest papers

351 papers59 papers in the last four weeks, up 119% on the four weeks before. 0.3% of all new papers.

FedOGL: Combating Catastrophic Forgetting in Federated Open-World Multimodal Graph Learning
Jul 30, 2026Zekai Chen, Haodong Lu, Shihao Li +5Multimodal Federated Learning Federated Continual Learning
Beijing Institute of Technology, Beijing, China
The Art of Not Forgetting A Local Learning Architecture for Continual Learning
Jul 29, 2026Ashmith Atmuri, Yashaswini Rao BhogarajulaMemory-Augmented Neural Networks Continual Learning
Arkadhi Research
ForgetBench: Benchmarking Forgetting Dynamics of Long-Term Parametric Memory in Language Models
Jul 29, 2026Ruxi Gu, Zhenliang Zhang, Zilei Wang +1Knowledge Editing Continual Learning for LLMs
Department of Automation, University of Science and Technology of China, Hefei, China · State Key Laboratory of General Artificial Intelligence, BIGAI, Beijing, China · School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing, China.
The Intruder Threshold: A Spectral Law for LoRA Fine-Tuning
Jul 26, 2026Peng XieLLM Fine-Tuning Low-Rank Adaptation
Technical University of Munich, Campus Heilbronn
NeuroSynth: A Biologically Inspired Continual Reinforcement Learning Architecture for Mitigating Catastrophic Forgetting
Jul 24, 2026Yash KiniContinual Learning Continual RL
James Madison High School Vienna, Virginia, USA
Source-Prior-Driven Selective Adaptation for Efficient Diffusion Model Finetuning
Jul 23, 2026Yi Xiong, Yuan-Yuan Cheng, Xiao-Ming FuFine-Tuning Diffusion Model Fine-Tuning
University of Science and Technology of China, China
The World Model Remembers, the Actor Forgets: Dream Rehearsal for Continual Model-Based RL
Jul 22, 2026Gurp NijjerGenerative Replay Reinforcement Learning
Quantegra Research, Canada
Code Division Modulation Layers Against Forgetting and Inference in Continual Gait Identification
Jul 21, 2026Simone MilaniMembership Inference Attacks Privacy-Preserving ML
Department of Information Engineering, University of Padova, Padova, Italy
The Art of Not Forgetting
Jul 20, 2026Ashmith Atmuri, Akshay Kumar, Yashaswini Rao BhogarajulaMemory-Augmented Neural Networks Domain-Incremental Learning
Arkadhi Labs
Rethinking Quantum Continual Learning with Quantum Fisher Information
Jul 17, 2026Yu-Chao Hsu, Yu-Cheng Lin, Tai-Yue Li +2Quantum Machine Learning Variational Quantum Circuits
School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), 291 Daehak-ro, Yuseong-gu, Daejeon 34141, Republic of Korea · Department of Electrophysics, National Yang Ming Chiao Tung University, Hsinchu 300092, Taiwan. · National Center for High-Performance Computing , National Institutes of Applied Research (NIAR), Hsinchu 300092, Taiwan
Towards Human-like Physical Intelligence: LifelongVision-Language-Action Learning for Robotic Manipulation
Jul 16, 2026Yao He, Gan Sun, Wenqi Liang +2Continual Robot Learning Robot Skill Learning
South China University of Technology · University of Trento
Symbiosis-Inspired Knowledge Distillation for Incremental Object Detection
Jul 15, 2026Mingyue Zeng, De Cheng, Zhipeng Xu +3Class-Incremental Learning Object Detection
State Key Laboratory of Integrated Services Networks, School of Telecommunications Engineering, Xidian University, Xi’an, China · School of Electronic Engineering, Xidian University, Xi’an, China
Domain-Incremental Remote Sensing Change Detection via Difference-Guided Adaptation and Frequency-Decoupled Distillation
Jul 14, 2026Daifeng Peng, Yaning Li, Haiyan GuanRemote Sensing Change Detection Domain-Incremental Learning
School of Remote Sensing and Geomatics Engineering, Nanjing University of Information Science and Technology, Nanjing 210044, China
Traceback Translators Against Forgetting in Continual Fake Speech Detection
Jul 14, 2026Enrico Gottardis, Mattia Tamiazzo, Simone MilaniAudio Deepfake Detection Catastrophic Forgetting
Department of Information Engineering · University of Padova · Padova, Italy
An Empirical Analysis of Continual Learning for Heterogeneous Medical Visual Question Answering
Jul 13, 2026Mai A. Shaaban, Tausifa Jan Saleem, Alaa Mohamed +3Medical VQA Continual Learning for VLMs
Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE · Department of Mathematics and Computer Science, Faculty of Science, Alexandria University, Alexandria, Egypt
Unified Gradient Projection: Language-Balanced Continual Learning for Multilingual Low-Resource ASR
Jul 13, 2026Ziang Ren, Guodong Lin, Yuchen Ai +2Automatic Speech Recognition Gradient Projection
Department of Electronic Engineering, Tsinghua University, China
CA-DGCL: Dynamic Graph Continual Learning via Condensation and Attachment
Jul 13, 2026Tingxu Yan Ye YuanReplay-Based Continual Learning Catastrophic Forgetting
1Tingxu Yan and Ye Yuan* are with the College of Computer and Information Science, Southwest University, Chongqing 400715, China
Can a Language Model Learn Facts Continually in Its Weights?
Jul 13, 2026Charles O'Neill, Max Kirkby, Jonathon Liu +1Knowledge Editing Continual Learning for LLMs
Baseten
Interference and Retention in Continual Learning
Jul 10, 2026Julius StörkContinual Learning Online Continual Learning
VARTA Microbattery GmbH, Ellwangen, Germany
Lifelong Representations: A Survey on Continual Self-Supervised Learning for Vision Models
Jul 8, 2026Sergi Masip, Alicja Dobrzeniecka, Jonathan Swinnen +4Self-Supervised Learning Self-Supervised Visual Representation Learning
PSI, KU Leuven, Leuven, Belgium · NASK - National Research Institute, Warsaw, Poland · Computer Vision Center, Universitat Autonoma de Barcelona, Barcelona, Spain +1
Intrinsic-Noise Consolidation: A Doob-Barrier-Conditioned Diffusion Turns Analog Device Noise into a Continual-Learning Resource
Jul 8, 2026Gunner Levi HoweContinual Learning Stability-Plasticity Dilemma
REDDIT: Forgetting-Resistant Correction of Timestamp Drift in ASR via Replay-Based Distribution Editing
Jul 6, 2026Cheng-Kang Chou, Ming-To Chuang, Ke-Han Lu +2ASR Evaluation Speech Processing
National Taiwan University · Carnegie Mellon University · NTU Artificial Intelligence Center of Research Excellence (NTU AI-CoRE)
RL Forgets! Towards Continual Policy Optimization
Jul 5, 2026Mao-Lin Luo, Zhe-Xu Wang, Zi-Hao Zhou +4RL Post-Training Continual Learning for VLMs
School of Computer Science and Engineering, Southeast University, Nanjing 210096, China · Key Laboratory of Computer Network and Information Integration (Southeast University), Ministry of Education, China · Zhongguancun Academy +1
Denser $\neq$ Better: Limits of On-Policy Self-Distillation for Continual Post-Training
Jul 2, 2026Meng Wang, Haohan Zhao, Wenzhuo Liu +7Continual Learning for LLMs On-Policy Self-Distillation
Centre for Artificial Intelligence and Robotics, HKISI, CAS · Institute of Automation, CAS · University of Chinese Academy of Sciences +1
Rosetta: Composable Native Multimodal Pretraining
Jul 1, 2026Xiangyue Liu, Zijian Zhang, Miles Yang +3Unified Multimodal Models Multimodal Pretraining
HKUST · Tencent Hunyuan
Neural Subspace Reallocation: Continual Learning as Retrieval-Based Subspace Memory Management
Jun 29, 2026Byeong Hoon YoonStorage-Efficient Continual Learning Continual Learning
Independent Researcher
Continual Learning for Sequential Personalization of Small Language Models: A Stability Monitoring Analysis
Jun 26, 2026Thomas S. Paula, Lucas S. Kupssinskü, Rodrigo C. BarrosLLM Personalization Small Language Models
MALTA, Machine Learning Theory and Applications Lab, PUCRS, Porto Alegre, Brazil · Kunumi Institute, Brazil
Reproducibility Study of "AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models"
Jun 25, 2026Ananth K Suresh, Arya HariharanKnowledge Editing Model Editing
Independent
From Weights to Features: SAE-Guided Activation Regularization for LLM Continual Learning
Jun 25, 2026Evan Ning, Wei Xue, Dong Lou +1Continual Learning for LLMs Catastrophic Forgetting
1The Hong Kong University of Science and Technology
The Gentle Collapse: Distributional Metrics for Continual Learning
Jun 23, 2026Ahmed Anwar, Andreas Wagner, Federico Raue +2Continual Learning Catastrophic Forgetting
1German Research Center for Artificial Intelligence (DFKI) · 2RPTU University Kaiserslautern-Landau, Kaiserslautern, Germany · 3Hochschule Karlsruhe University of Applied Sciences, Karlsruhe, Germany

Jul 30, 2026J/K move · Enter open · S save

FedOGL: Combating Catastrophic Forgetting in Federated Open-World Multimodal Graph Learning

Zekai Chen, Haodong Lu, Shihao Li, Weiwei Ji+4

Beijing Institute of Technology, Beijing, China

Multimodal Federated Learning Federated Continual Learning Federated Graph Learning Multimodal Graph Learning Catastrophic Forgetting

Open paper AskPDF arXiv

Federated graph learning enables collaborative training over decentralized graph data without sharing raw graph information. As such risks evolve, clients must learn emerging classes from private multimodal graph streams, retain historical categories, and reject samples outside the known class space. In this setting, clients must learn emerging classes from private multimodal graph streams while preserving historical categories and rejecting samples outside the current known class space. The core challenge is catastrophic forgetting, which in federated multimodal graphs is not merely a classifier-level failure: old knowledge can be erased through modality-semantic overwriting, topology-induced structural erosion, and federated memory fragmentation. To address this challenge, we propose FedOGL, a semantic-structural memory preservation framework. On the client side, FedOGL preserves historical decision behavior through replay and task-start distillation, while protecting graph-propagation memory via projection onto a globally shared structure basis. On the server side, FedOGL maintains and transfers compact category prototypes to facilitate cross-client knowledge sharing without exposing raw graph data. Extensive experiments demonstrate that, compared with the best-performing baselines, FedOGL reduces performance degradation caused by catastrophic forgetting by 42.67%, while maintaining or improving performance on downstream tasks.