Elastic Weight Consolidation

Also known as EWC

Follow this topic Ask about this topic

Momentum

2 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 11

All topics

Towards Eliminating Catastrophic Forgetting in the Curriculum Learning of Math Reasoning Tasks
Sep 27, 2026Zengyan Yang, Yangyang Wu, Kai Huang +3Elastic Weight Consolidation Mathematical Reasoning
Zhejiang University · Ant Group
Fine-Tuning LLMs for Translation: General Forgetting Mitigation Does Not Preserve MT-Specific Instruction Following
Sep 23, 2026Niklas Scholz, David Thulke, Abdallah Nasir +3LLM Fine-Tuning Elastic Weight Consolidation
AppTek GmbH, Aachen, Germany · Machine Learning and Human Language Technology, RWTH Aachen University, Germany · Applied Science Private University, Amman, Jordan
The Sequential Price of Continual Learning
Aug 30, 2026Zonghuan Xu, Xingjun MaContinual Learning Elastic Weight Consolidation
Institute of Trustworthy Embodied AI, Fudan University, Shanghai, China · Shanghai Key Laboratory of Multimodal Embodied AI, Shanghai, China
An active-learning framework for real-time depth perception from monocular vision streams
Aug 5, 2026Xiaorong Zeng, Weiqiang Chen, Peng Shi +4Edge Inference Monocular Depth Estimation
School of Mechanical and Automotive Engineering, Xiamen University of Technology, Xiamen, Fujian 361024, China · King Long United Automotive Industry Co. Ltd., Xiamen 361023, China · School of Electrical and Electronic Engineering, The University of Adelaide, Adelaide, SA 5005, Australia +2
Continual Learning with Elastic Regularization and Synthetic Replay for Federated MLLM Fine-Tuning
Jul 13, 2026Jing Liu, Chenxuanyin Zou, Jiayang Ren +5Multimodal Federated Learning Generative Replay
Department of Electrical and Computer Engineering, The University of British Columbia, Vancouver, BC V6T 1Z4, Canada · College of Future Information Technology, Fudan University, Shanghai 200433, China · Department of Chemical and Biological Engineering, The University of British Columbia, Vancouver, BC V6T 1Z4, Canada +3
RECALL: Recovery Experience Collection for Active Lifelong Learning in Vision-Language-Action Models
Jun 22, 2026Ulas Berk Karli, Tesca FitzgeraldContinual Robot Learning Robot Skill Learning
Department of Computer Science Yale University United States
CADRE: Stable, Parameter Efficient Adaptation of Medical Vision Language Models with Bounded Forgetting and Prior Drift
Jun 22, 2026Amrita Singh, Rishabh JhaVLM Adaptation Elastic Weight Consolidation
Mindriser’s Consortium, Kathmandu, Nepal · University of Victoria, BC, Canada
Lifelong Learning in Vision-Language Models: Enhanced EWC with Cross-Modal Knowledge Retention
May 12, 2026Hamza Ahmed Durrani, Rafay Suleman DurraniVision-Language Models Cross-Modal Alignment
Sejong University, Computer Science Engineering · Technische Universität Ilmenau, Computer Engineering
Beyond the False Trade-off: Adaptive EWC for Stealthy and Generalizable T2I Backdoors
May 8, 2026Lu Bowen, Xinyu Tang, Yin Yin Low +1Backdoor Attacks Elastic Weight Consolidation
Monash University
MPCS: Neuroplastic Continual Learning via Multi-Component Plasticity and Topology-Aware EWC
May 4, 2026Joern HentschStability-Plasticity Dilemma Dynamic Neural Networks
DIRA-SS:Dynamic Domain Incremental Regularised Adaptation -- Self-Supervised
Nov 13, 2023Abanoub Ghobrial, Kerstin EderTest-Time Adaptation Unsupervised Domain Adaptation
University of Bristol, Bristol, UK

Sep 27, 2026, cs.LGJ/K move · Enter open · S save

Towards Eliminating Catastrophic Forgetting in the Curriculum Learning of Math Reasoning Tasks

Zengyan Yang, Yangyang Wu, Kai Huang, Pengfei Lyu+2

Zhejiang University · Ant Group

Elastic Weight Consolidation Mathematical Reasoning Curriculum Learning Catastrophic Forgetting

Open paper AskPDF arXiv

Curriculum learning has found broad application across numerous domains. Nevertheless, its effectiveness is intrinsically curtailed by catastrophic forgetting, driven by the shifts in model parameter distributions between curriculum tasks. In this paper, we investigate the phenomenon of catastrophic forgetting in this training paradigm, building on the established efficacy of curriculum learning. Our theoretical analyses of parameter update dynamics demonstrate that catastrophic forgetting in curriculum learning stems from the divergence of task optima, which is generally essential to the faster convergence of curriculum learning; therefore, forgetting cannot be completely eliminated. Based on this finding, we augment the training process and propose IV-EWC, which incorporates Elastic Weight Consolidation (EWC) into the curriculum learning objective to curb catastrophic forgetting in mathematical reasoning, a prototypical curriculum learning scenario. IV-EWC employs the influence function to construct a representative validation set from the curriculum's training data, which is used to drive dynamic regularization during training. We further present an extended theoretical analysis to show that EWC-based regularization methods mitigate catastrophic forgetting in curriculum learning, thereby providing theoretical support for IV-EWC. Empirical evaluations on three backbone models and three benchmarks indicate that curriculum learning exhibits catastrophic forgetting. IV-EWC alleviates this issue, reducing forgetting by 162% on average relative to vanilla curriculum learning and yielding positive backward transfer, as evidenced by improved performance on easier tasks after subsequent training on challenging tasks.

Momentum

Related topics

Latest papers 11