Catastrophic Forgetting

Follow this topic Ask about this topic

Catastrophic forgetting occurs when learning new tasks or adapting to shifting data degrades performance on earlier tasks, a central challenge for systems expected to learn over time. Common approaches include replaying examples, distilling prior outputs, regularizing important weights or gradients, and isolating updates with adapters or modular experts. Researchers study these methods in sequential vision and language benchmarks as well as foundation-model fine-tuning, multimodal systems, and robotics, measuring both new-task performance and retention. No single approach works best everywhere: stronger protection can limit adaptation, while memory, compute, and benchmark choices affect results; apparent forgetting may also reflect knowledge becoming inaccessible rather than erased.

Latest papers

351 papers59 papers in the last four weeks, up 119% on the four weeks before. 0.3% of all new papers.

Sequential Learning and Catastrophic Forgetting in Differentiable Resistor Networks
May 2, 2026Maniru IbrahimContinual Learning Physical Neural Networks
Mathematics Applications Consortium for Science and Industry (MACSI), Department of Mathematics and Statistics, University of Limerick, Limerick, Ireland
FedKPer: Tackling Generalization and Personalization in Medical Federated Learning via Knowledge Personalization
May 1, 2026Zoe Fowler, Ghassan AlRegibFederated Learning Aggregation Cross-Domain Generalization
Georgia Institute of Technology
Diversity in Large Language Models under Supervised Fine-Tuning
Apr 30, 2026Roman Klypa, Oleksandr CherednichenkoSupervised Fine-Tuning Focal Loss
Univ. Grenoble Alpes, CNRS, Grenoble INP, LJK, 38000 Grenoble, France · Department of Mathematics and Mathematical Statistics, Integrated Science Lab, Umeå University, Sweden.
Learning to Forget: Continual Learning with Adaptive Weight Decay
Apr 29, 2026Aditya A. Ramesh, Alex Lewandowski, Jürgen SchmidhuberDeep Learning Optimization Weight Decay
The Swiss AI Lab, IDSIA USI-SUPSI, Lugano, Switzerland · University of Alberta, Alberta Machine Intelligence Institute, Edmonton, Canada · Center of Excellence for Generative AI, KAUST, Thuwal, Saudi Arabia
When Continual Learning Moves to Memory: A Study of Experience Reuse in LLM Agents
Apr 29, 2026Qisheng Hu, Quanyu Long, Wenya WangContinual Learning for LLM Agents LLM Agent Memory
Nanyang Technological University
Cortex-Inspired Continual Learning: Unsupervised Instantiation and Recovery of Functional Task Networks
Apr 27, 2026Kevin McKee, Thomas Hazy, Yicong Zheng +2Continual Learning Incremental Learning
Astera Institute
A Parametric Memory Head for Continual Generative Retrieval
Apr 25, 2026Kidist Amde Mekonnen, Yubao Tang, Maarten de RijkeMemory-Augmented Neural Networks LLM Memory
University of Amsterdam Amsterdam, The Netherlands
Hidden Failure Modes of Gradient Modification under Adam in Continual Learning, and Adaptive Decoupled Moment Routing as a Repair
Apr 24, 2026Yuelin Hu, Zhenbo Yu, Zhengxue Cheng +2Continual Learning for LLMs Adaptive Gradient Methods
Shanghai Jiao Tong University · Shanghai Maritime University
Lifecycle-Aware Federated Continual Learning in Mobile Autonomous Systems
Apr 22, 2026Beining Wu, Jun HuangMobile Robotics Federated Continual Learning
Department of Electrical Engineering and Computer Science, South Dakota State University, Brookings, 57006 SD
Revisiting Catastrophic Forgetting in Continual Knowledge Graph Embedding
Apr 21, 2026Gerard Pons, Carlos Escolano, Besim Bilalli +1Graph Representation Learning KG Embedding
Data-intensive Technologies and Knowledge Systems (DTAK), Department of Service and Information System Engineering, Universitat Politècnica de Catalunya, Barcelona, 08034, Catalonia, Spain · Intelligent Data Science and Artificial Intelligence Research Center (IDEAI), Department of Computer Science, Universitat Politècnica de Catalunya, Barcelona, 08034, Catalonia, Spain
Neural Network Optimization Reimagined: Decoupled Techniques for Scratch and Fine-Tuning
Apr 21, 2026Xin Ning, Qiankun Li, Xiaolong Huang +5Fine-Tuning Neural Network Optimization
Institute of Semiconductors, Chinese Academy of Sciences, Beijing, 10083, China · College of Computing and Data Science (CCDS), Nanyang Technological University, 639798, Singapore · Mila-Quebec AI Institute, Montreal, QC H2S 3H1, Canada +3
Robust Continual Unlearning against Knowledge Erosion and Forgetting Reversal
Apr 21, 2026Eun-Ju Park, Youjin Shin, Simon S. WooMachine Unlearning Catastrophic Forgetting
1Sungkyunkwan University · 2The Catholic University of Korea · 3Secure Machines Lab
Task Switching Without Forgetting via Proximal Decoupling
Apr 20, 2026Pourya Shamsolmoali, Masoumeh Zareapoor, Eric Granger +2Proximal Optimization Continual Learning
Department of Computer Science, University of York, UK · SEIEE, Shanghai Jiao Tong University, China · LIVIA, Dept. of Systems Engineering, ETS Montreal, Canada +1
HiP-LoRA: Budgeted Spectral Plasticity for Robust Low-Rank Adaptation
Apr 20, 2026Lixian Chen, Jianhong TanContinual Learning for LLMs Low-Rank Adaptation
Guangdong University of Technology
AEGIS: Anchor-Enforced Gradient Isolation for Knowledge-Preserving Vision-Language-Action Fine-Tuning
Apr 17, 2026Guransh SinghVLM Adaptation Robotic Manipulation
Independent Researcher
Self-Distillation as a Performance Recovery Mechanism for LLMs: Counteracting Compression and Catastrophic Forgetting
Apr 17, 2026Chi Liu, Xin Chen, Xu Zhou +2LLM Compression Language Model Distillation
PayPal AI
Why Fine-Tuning Encourages Hallucinations and How to Fix It
Apr 16, 2026Guy Kaplan, Zorik Gekhman, Zhen Zhu +5Supervised Fine-Tuning Continual Learning for LLMs
♡Hebrew University of Jerusalem · ♣Technion – Israel Institute of Technology · ♢University of Illinois Urbana-Champaign +1
AIM: Asymmetric Information Masking for Visual Question Answering Continual Learning
Apr 16, 2026Peifeng Zhang, Zice Qiu, Donghua Yu +4Visual Question Answering Vision-Language Models
Sun Yat-Sen University · National Supercomputing Center in Shenzhen · Tsinghua University
CI-CBM: Class-Incremental Concept Bottleneck Model for Interpretable Continual Learning
Apr 16, 2026Amirhosein Javadi, Tuomas Oikarinen, Tara Javidi +1Class-Incremental Learning Concept Bottleneck Models
Department of Electrical and Computer Engineering, University of California San Diego · Department of Computer Science and Engineering, University of California San Diego · Halıcıoğlu Data Science Institute, University of California San Diego
MAny: Merge Anything for Multimodal Continual Instruction Tuning
Apr 15, 2026Zijian Gao, Wangwang Jia, Xingxing Zhang +6Continual Learning for VLMs Multimodal Instruction Tuning
College of Computer Science and Technology, National University of Defense Technology · State Key Laboratory of Complex & Critical Software Environment · National Key Laboratory of Parallel and Distributed Computing, National University of Defense Technology +1
Parameter Importance is Not Static: Evolving Parameter Isolation for Supervised Fine-Tuning
Apr 15, 2026Zekai Lin, Chao Xue, Di Liang +8Supervised Fine-Tuning Selective Fine-Tuning
Tencent Hunyuan · University of New South Wales · Tencent Yuanbao +2
(How) Learning Rates Regulate Catastrophic Overtraining
Apr 15, 2026Mark Rofin, Aditya Varre, Nicolas FlammarionLLM Training Supervised Fine-Tuning
EPFL
From Order to Distribution: An Exact Operator Framework for Forgetting in Continual Learning
Apr 15, 2026Zonghuan Xu, Xingjun MaContinual Learning Catastrophic Forgetting
Institute of Trustworthy Embodied AI, Fudan University, Shanghai, China · Shanghai Key Laboratory of Multimodal Embodied AI, Shanghai, China
Gradient-Free Continual Learning in Spiking Neural Networks via Inter-Spike Interval Regularization
Apr 14, 2026Samrendra Roy, Kazuma Kobayashi, Souvik Chakraborty +2Spiking Neural Networks Catastrophic Forgetting
University of Illinois Urbana-Champaign, Department of Nuclear, Plasma & Radiological Engineering, Urbana, IL, USA · Indian Institute of Technology Delhi, Department of Applied Mechanics, New Delhi, India · University of Texas at El Paso, Department of Computer Science, El Paso, TX, USA +1
Continual Distillation of Teachers from Different Domains
Apr 10, 2026Nicolas Michel, Maorong Wang, Jiangpeng He +1Knowledge Distillation Catastrophic Forgetting
The University of Tokyo · Japanese-French Laboratory for Informatics, CNRS · National Institute of Informatics +1
Leveraging Complementary Embeddings for Replay Selection in Continual Learning with Small Buffers
Apr 9, 2026Danit Yanowsky, Daphna WeinshallExperience Replay Replay-Based Continual Learning
School of Computer Science and Engineering The Hebrew University of Jerusalem
Information as Structural Alignment: A Dynamical Theory of Continual Learning
Apr 8, 2026Radu NegulescuContinual Learning Catastrophic Forgetting
The Informational Buildup Foundation
Improving Sparse Memory Finetuning
Apr 6, 2026Satyam Goyal, Anirudh Kanchi, Garv Shah +1Continual Learning for LLMs Sparse Fine-Tuning
University of Michigan, Ann Arbor
GAIN: Multiplicative Modulation for Domain Adaptation
Apr 6, 2026Hengshuai Yao, Xing Chen, Ahmed Murtadha +1Domain Adaptation for LLMs Continual Learning for LLMs
Sapient Intelligence · Department of Computing Science, University of Alberta
Adaptive Memory Crystallization for Autonomous AI Agent Learning in Dynamic Environments
Apr 2, 2026Rajat Khanda, Mohammad Baqar, Sambuddha Chakrabarti +1Continual Learning Memory-Augmented Agents
Supermicro San Jose, USA · Cisco Systems San Jose, USA · Princeton University New Jersey, USA +1

May 2, 2026J/K move · Enter open · S save

Sequential Learning and Catastrophic Forgetting in Differentiable Resistor Networks

Maniru Ibrahim

Mathematics Applications Consortium for Science and Industry (MACSI), Department of Mathematics and Statistics, University of Limerick, Limerick, Ireland

Continual Learning Physical Neural Networks Catastrophic Forgetting

Open paper AskPDF arXiv

Differentiable physical networks provide a simple setting in which learning can be studied through the interaction between trainable parameters and physical equilibrium constraints. We investigate sequential learning in differentiable resistor networks governed by Kirchhoff's laws. Although individual input--output mappings can be learned by gradient-based adjustment of edge conductances, sequential training on conflicting tasks produces catastrophic forgetting. We show that forgetting is controlled by task conflict and by the degree of adaptation to the new task. Uniform anchoring and normalised gradient-weighted anchoring reduce forgetting only by increasing the final loss on the new task, giving a clear forgetting--adaptation trade-off. We also show that forgetting is associated with localised conductance changes on high-current edges, giving a physical interpretation as reconfiguration of dominant transport pathways. Broader random-task ensembles show that the strongest forgetting occurs when the second task reverses the output ordering imposed by the first task. Finally, comparisons across Erdős--Rényi, small-world, scale-free, and random-geometric graph ensembles show that topology changes the forgetting--adaptation balance. These results position differentiable resistor networks as compact, physically interpretable testbeds for studying continual learning in tunable matter.