Replay-Based Continual Learning

Follow this topic Ask about this topic

Momentum

9 papers in the last four weeks, up 125% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 82

All topics

Towards Understanding Continual Factual Knowledge Acquisition of Language Models: From Theory to Algorithm
May 11, 2026Haoyu Wang, Yifan Shang, Zhongxiang Sun +3Continual Learning for LLMs Generative Replay
Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China · School of Artificial Intelligence and Data Science, University of International Business and Economics, Beijing, China
UFO: A Unified Flow-Oriented Framework for Robust Continual Graph Learning
May 11, 2026Danhui Zhang, Zhe Wang, Qing Qing +6Replay-Based Continual Learning Flow-Based Generative Modeling
Jilin University · Adelaide University · RMIT University +1
Beyond Forgetting in Continual Medical Image Segmentation: A Comprehensive Benchmark Study
May 7, 2026Bomin Wang, Hangqi Zhou, Yibo Gao +1Interactive Medical Image Segmentation Stability-Plasticity Dilemma
School of Data Science, Fudan University, Shanghai, China.
CoMemNet: Contrastive Sampling with Memory Replay Network for Continual Traffic Prediction
May 7, 2026Mei Wu, Wenchao Weng, Wenxin Su +2Contrastive Learning Traffic Flow Estimation
College of Computer Science, Shanghai Jiao Tong University, Shanghai 200240, China · College of Computer Science and Technology, Zhejiang University of Technology, Hangzhou 310023, China · School of Automation, Nanjing University of Science and Technology, Nanjing 210094, China
Replay-Based Continual Learning for Physics-Informed Neural Operators
May 6, 2026Yizheng Wang, Mohammad Sadegh Eshaghi, Xiaoying Zhuang +2OOD Generalization Replay-Based Continual Learning
Department of Engineering Mechanics, Tsinghua University, Beijing 100084, China · Institute of Photonics, Department of Mathematics and Physics, Leibniz University Hannover, Germany · Institute of Structural Mechanics, Bauhaus-Universität Weimar, Marienstr. 15, D-99423 Weimar, Germany
Memory-Efficient Continual Learning with CLIP Models
May 5, 2026Ryan King, Gang Li, Bobak Mortazavi +1Class-Incremental Learning Contrastive Learning
Department of Computer Science Texas A&M University College Station, TX 77843 · Texas A&M University College Station, TX 77843
A Domain Incremental Continual Learning Benchmark for ICU Time Series Model Transportability
May 5, 2026Ryan King, Conrad Krueger, Ethan Veselka +2Clinical Outcome Prediction Domain-Incremental Learning
Computer Science & Engineering, Texas A&M University, College Station, United States
Adaptive Data Compression and Reconstruction for Memory-Bounded EEG Continual Learning
May 4, 2026Chengcheng XieElectroencephalography Continual Learning
Beihang University
Continual Learning of Feedback-based Molecular Communication
May 1, 2026Siddhant Setia, Junichi Suzuki, Tadashi NakanoContinual Learning Replay-Based Continual Learning
Department of Computer Science, University of Massachusetts, Boston Boston, MA, 02125, USA · Graduate School of Informatics, Osaka Metropolitan University Osaka, Japan
Continual Calibration: Coverage Can Collapse Before Accuracy in Lifelong LLM Fine-Tuning
Apr 27, 2026Ibne Farabi Shihab, Sanjeda Akter, Anuj SharmaContinual Learning for LLMs Conformal Prediction
Department of Computer Science, Iowa State University · Department of Civil, Construction & Environmental Engineering, Iowa State University
Lifecycle-Aware Federated Continual Learning in Mobile Autonomous Systems
Apr 22, 2026Beining Wu, Jun HuangMobile Robotics Federated Continual Learning
Department of Electrical Engineering and Computer Science, South Dakota State University, Brookings, 57006 SD
Recovery Guarantees for Continual Learning of Dependent Tasks: Memory, Data-Dependent Regularization, and Data-Dependent Weights
Apr 19, 2026Liangzu Peng, Uday Kiran Reddy Tadipatri, Ziqing Xu +2Continual Learning Replay-Based Continual Learning
University of Pennsylvania
Continual Hand-Eye Calibration for Open-world Robotic Manipulation
Apr 17, 2026Fazeng Li, Gan Sun, Chenxi Liu +3Hand-Eye Calibration Camera Pose Estimation
School of Automation Science and Engineering, South China University of Technology, Guangzhou 510641, China · State Key Laboratory of Robotics, the Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang 110169, China · University of Chinese Academy of Sciences, Beijing 100049, China
Towards Lifelong Aerial Autonomy: Geometric Memory Management for Continual Visual Place Recognition in Dynamic Environments
Apr 10, 2026Xingyu Shao, Zhiqiang Yan, Liangzheng Sun +5Cross-View Geo-Localization Visual Place Recognition
Leveraging Complementary Embeddings for Replay Selection in Continual Learning with Small Buffers
Apr 9, 2026Danit Yanowsky, Daphna WeinshallExperience Replay Replay-Based Continual Learning
School of Computer Science and Engineering The Hebrew University of Jerusalem
APEX-EM: Non-Parametric Online Learning for Autonomous Agents via Structured Procedural-Episodic Experience Replay
Mar 31, 2026Pratyay Banerjee, Masud Moshtaghi, Ankit ChadhaContinual Learning for LLM Agents Persistent Memory for Language Models
Amazon, AGI / Sunnyvale, USA
Knowledge-Aware Evolution for Task-Free Streaming Federated Continual Learning with Arbitrary Class Overlap
Jan 27, 2026Sixing Tan, Xianmin LiuContinual Learning Federated Continual Learning
Faculty of Computing Harbin Institute of Technology
When Less is More: 8-bit Quantization Improves Continual Learning in Large Language Models
Dec 22, 2025Michael S. Zhang, Rishi A. Ruia, Arnav Kewalram +3LLM Quantization Continual Learning for LLMs
Algoverse
ProDER: A Continual Learning Approach for Fault Classification and Localization in Evolving Smart Grids
Nov 7, 2025Emad Efatinasab, Nahal Azadi, Davide Dalle Pezze +3Class-Incremental Learning Prototype-Based Classification
Department of Information Engineering, University of Padova. · School of Computing, Newcastle University.
Taming Modality Entanglement in Continual Audio-Visual Segmentation
Oct 20, 2025Yuyang Hong, Qi Yang, Tao Zhang +5Audio-Visual Understanding Replay-Based Continual Learning
School of Artificial Intelligence, UCAS · 2MAIS, Institute of Automation · School of Intelligent Science and Technology, University of Science and Technolog Beijing
Beyond Discrete Samples: High Information Density Replay for Efficient Lifelong Person Re-Identification
Aug 3, 2025Mingyu Wang, Haojie Liu, Zhiyong Li +3Replay-Based Continual Learning Dataset Condensation
College of Control Science and Engineering, Zhejiang University, Hangzhou 310027, China
Exploiting Stability-Plasticity Asymmetry in Pretrained Detectors for Incremental Object Detection
Apr 14, 2025Songze Li, Qixing Xu, Tonghua Su +3Class-Incremental Learning Detection Transformer
Harbin Institute of Technology, Harbin, China · Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ), Shenzhen, China · Chongqing Research Institute of HIT, Chongqing, China +2

May 11, 2026, cs.CLJ/K move · Enter open · S save

Towards Understanding Continual Factual Knowledge Acquisition of Language Models: From Theory to Algorithm

Haoyu Wang, Yifan Shang, Zhongxiang Sun, Weijie Yu+2

Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China · School of Artificial Intelligence and Data Science, University of International Business and Economics, Beijing, China

Continual Learning for LLMs Generative Replay Continual Learning Replay-Based Continual Learning Continued Pretraining Catastrophic Forgetting

Open paper AskPDF arXiv

Continual Pre-Training (CPT) is essential for enabling Language Models (LMs) to integrate new knowledge without erasing old. While classical CPT techniques like data replay have become the standard paradigm, the mechanisms underlying how LMs acquire and retain facts over time, termed as continual Factual Knowledge Acquisition (cFKA), remain unclear. In this work, we present a theoretical framework that characterizes the training dynamics of cFKA using a single-layer Transformer, offering a unified explanation for the behavior of representative CPT methods. Our analysis reveals that regularization-based methods merely adjust the convergence rate of parameters without altering the inherent forgetting tendency, whereas data replay methods succeed in shifting convergence dynamics and stabilizing pretrained knowledge. Building on these insights, we propose a novel generative data replay approach, called \textbf{S}electing \textbf{T}okens via attenti\textbf{O}n \textbf{C}ontribution~(STOC), which identifies influential factual snippets to guide replay data generation. Extensive experiments on both synthetic and real-world datasets validate our findings and demonstrate that STOC effectively enhances cFKA by mitigating catastrophic forgetting.

Momentum

Related topics

Latest papers 82