Replay-Based Continual Learning

Follow this topic Ask about this topic

Momentum

29 papers in the last four weeks, up 263% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 231

All topics

CoDe-LoRA: Mitigating the Orthogonality Dilemma in Continual Learning of LLMs via Knowledge Consolidation and Decoupling
Oct 6, 2026Maoqi Liu, Quan Fang, Yufei HeReplay-Based Continual Learning Large Language Model Adaptation
Beijing University of Posts and Telecommunications · National University of Singapore
ServeLearnBench: How Well Can Agents Self-Improve from Serving Experience?
Oct 6, 2026Haizhong Zheng, Yizhuo Di, Ranajoy Sadhukhan +2Replay-Based Continual Learning
Carnegie Mellon University · Amazon
The Optimization Landscape of Learning Compacted Context Models
Oct 5, 2026Thomas Villeneuve, Alex Sandomirsky, Charles O'Neill +2Compressed Model Replay-Based Continual Learning
Base Labs
Off-Policy Merging Beats On-Policy Self-Distillation for Continual Learning
Oct 5, 2026Chen Henry Wu, Thomas Zhang, Aditi RaghunathanReplay-Based Continual Learning Off-Policy Learning
Carnegie Mellon University
Task Vector Descent: Learning from Non-IID Batches
Oct 4, 2026Anton Baumann, Jonas Hübotter, Zeynep Akata +1Replay-Based Continual Learning
ETH Zurich · Technical University of Munich
Rethinking Tabular Foundation Models On Data Streams
Oct 4, 2026Nilesh Verma, Daniel Nowak-Assis, Afonso Lourenço +4Tabular Foundation Models Time Series Foundation Models
AI Institute, University of Waikato, New Zealand · LIP6, CNRS, Sorbonne Universit´e, France · GECAD, Polytechnic of Porto, Portugal
Continual Reinforcement Learning with Neuroevolution
Oct 1, 2026Eleni Nisioti, Andrea Cossu, Kathrin Korte +1Neuroevolution Plasticity
IT University of Copenhagen · University of Pisa · Sakana AI
CLASP: Continual Low-rank Adapters for Spatially Placed Concepts from One Hypernetwork
Oct 1, 2026Wojciech Gromski, Patryk Krukowski, Jan Miksa +2Replay-Based Continual Learning Text-To-Image Diffusion Models
Wrocław University of Science and Technology · IDEAS Research Institute · Jagiellonian University, Krak´ow +2
Continual Learning for 6-DoF Grasp Synthesis via Experience and Demonstrations
Oct 1, 2026Giulio Schiavi, Andrei Cramariuc, Michael Pantic +1Robotic Grasping Replay-Based Continual Learning
Autonomous Systems Lab ETH Zurich, Switzerland · Robotics Systems Lab ETH Zurich, Switzerland
ChainLoRA: Geometry-Preserving Task Vector Merging for Continual Learning in LLMs
Sep 30, 2026Hang Yin, Haozhe Wang, Yuhua Luo +3Replay-Based Continual Learning Parameter-Efficient Fine-Tuning Methods
Shanghai Jiao Tong University · Shanghai Innovation Institute · Beijing Zhongguancun Academy +3
Experience-Driven Continual Learning of Terrain Traversability for Quadruped Robots
Sep 30, 2026Luca Bricarello, João Carlos Virgolino Soares, Alberto Sanchez-Delgado +2Legged Robots Traversability
Dynamic Legged Systems Lab, Istituto Italiano di Tecnologia, Genoa, Italy. · Universit`a degli Studi di Genova, Genoa, Italy.
Evaluating Persistent Calibration under Evolving Model Knowledge
Sep 30, 2026Victor Wang, Thomas Hofweber, Mohit Bansal +1Confidence Estimation Replay-Based Continual Learning
University of Texas at Austin · University of North Carolina at Chapel Hill
Continual Learning of Dynamical Systems in Recurrent Neural Networks through Recyclable Unit Gating
Sep 29, 2026Sima Hashemi, Daniel Durstewitz, Georgia KoppeRecurrent Neural Networks Recurrent Model
Faculty of Mathematics and Computer Science, Interdisciplinary Center for Scientific Computing, Heidelberg University, Heidelberg, Germany · Department of Theoretical Neuroscience, Central Institute of Mental Health (CIMH), Medical Faculty Mannheim, Heidelberg University, Heidelberg, Germany · Hector Institute for AI in Psychiatry & Department of Psychiatry and Psychotherapy, CIMH, Medical Faculty Mannheim, Heidelberg University, Heidelberg, Germany +1
Generalizable Lifelong Model Editing via Preference Optimization
Sep 29, 2026Dahyun Jung, Suhyune Son, Heuiseok LimMultimodal Knowledge Editing Replay-Based Continual Learning
Department of Computer Science and Engineering, Korea University
HiTS-CL: A Continual Learning Framework for Long-Horizon Temporal Knowledge Graph Extrapolation
Sep 29, 2026Yansong Liu, Rui Liu, Yuan Zuo +6Temporal Knowledge Graphs Replay-Based Continual Learning
Beihang University Beijing, China · Beijing University of Posts and Telecommunications Beijing, China · Hubei Engineering University Xiaogan, Hubei, China
SPACE-LoRA: Allocating Activation-Subspace Protection for Continual Learning
Sep 28, 2026Seunghyun Yoo, Kiseok Kim, Hyeontae Joo +2Replay-Based Continual Learning Catastrophic Forgetting
School of Electrical Engineering Korea University Seoul, Republic of Korea
Reliable Replay through Spatial Coherence in Online Continual Learning
Sep 27, 2026Haixiang Sun, Jiefu Zhang, Yinghao He +4Prioritized Experience Replay Replay-Based Continual Learning
Purdue University West Lafayette, IN 47907, USA
SCLATE: a Substrate for Continual-Learning Agent Training and Evaluation
Sep 26, 2026Youngmok Jung, Sirajul Salekin, Henry Tran +3Replay-Based Continual Learning Long-Term Agent Memory
Apple
Hyperbolic Multimodal Continual Learning: A Closest-Admissible Solution
Sep 24, 2026Jiahong Liu, Ming Shen, Xiaohao Liu +4Multimodal Learning Replay-Based Continual Learning
The Chinese University of Hong Kong · National University of Singapore · Yale University +1
Muon Can Outperform Dedicated Continual Learning Methods
Sep 21, 2026Sebastian George Sincari, Bogdan Alexandru Gheorghe, Antonio BarbalauReplay-Based Continual Learning Continual Learning
Faculty of Mathematics and Computer Science, University of Bucharest, Bucharest, Romania · Faculty of Mathematics and Computer Science University of Bucharest Bucharest, Romania · Bitdefender, Bucharest, Romania +1
Brain-Inspired Hierarchical Modularity for General Continual Learning
Sep 21, 2026Hongwei Yan, Kanglei Zhou, Qi Cheng +7Replay-Based Continual Learning Continual Learning
School of Life Sciences, Tsinghua University, Beijing, China. · IDG/McGovern Institute for Brain Research, Tsinghua University, Beijing, China. · Tsinghua-Peking Center for Life Sciences, Beijing, China. +2
Stable Unsupervised Continual Chunking with Sheaf SyncMap
Sep 21, 2026Xueyuan Li, Danilo Vasconcellos VargasReplay-Based Continual Learning Chunk
Time-Incremental Continued Pretraining of LLMs: Knowledge Updates Without Catastrophic Forgetting
Sep 20, 2026Fırat Öncel, Salman Hussain Ali, Mirco Ravanelli +2Replay-Based Continual Learning Catastrophic Forgetting
Concordia University · Mila – Quebec AI Institute · Université de Montréal +3
CE $^4$ L: Continual Ego, Exo, and Ego-Exo Learning
Sep 20, 2026Hongwei Yan, Kanglei Zhou, Yuchen Liu +3Replay-Based Continual Learning Continual Learning
School of Life Sciences, IDG/McGovern Institute for Brain Research, Tsinghua University, Beijing, China · Tsinghua-Peking Center for Life Sciences · Department of Psychological and Cognitive Sciences, Tsinghua University, Beijing, China +2
Uncertainty-Aware Continual Learning for Open-World Intent Discovery Under an evolving Label Space
Sep 15, 2026Pisante Aida, Formentin SimoneReplay-Based Continual Learning Uncertainty-Aware Classification
Dipartimento di Elettronica, Informazione e Bioingegneria, Politecnico di Milano, Milano, Italy.
Continual Learning for Traversability Prediction with Uncertainty-Aware Adaptation
Sep 15, 2026Hojin Lee, Yunho Lee, Daniel A Duecker +1Traversability Ego-Trajectory Prediction
Ulsan National Institute of Science and Technology, Ulsan, 44919, Republic of Korea · Munich Institute of Robotics and Machine Intelligence (MIRMI), Technical University of Munich (TUM), Germany
CLARE: Scalable Class-Incremental Continual Learning via a Sparsity-Based Framework
Sep 15, 2026Yunxiang Fu, Meng Lou, Zicheng Liao +1Replay-Based Continual Learning Class-Incremental Learning
School of Computing and Data Science, The University of Hong Kong · Hong Kong Generative AI Research and Development Center
Smarter by the Moment: Environment-Driven Dynamic Policies for Continual LLM Improvement
Sep 15, 2026Ting-Wei Chang, Po-Chun Chen, Hen-Hsen Huang +1Large Language Model Policy Optimization Replay-Based Continual Learning
Department of Computer Science and Information Engineering, National Taiwan University, Taiwan · Institute of Information Science, Academia Sinica, Taiwan · AI Research Center (AINTU), National Taiwan University, Taiwan
Continuous-Time Machine Learning: A Unified Mathematical Perspective
Sep 15, 2026Waleed Razzaq, Yun-Sheng Zhao, Yun-Bo ZhaoReplay-Based Continual Learning Continuum
School of Automation University of Science and Technology of China Hefei, Anhui
Implementing a White-Box Undetectable Backdoor for Random Fourier Features
Sep 14, 2026Michael Collins, Jada Cumberland, Brianne Dunn +3Atom-Averaged Features Black Box
Laboratory for Advanced Cybersecurity Research Laurel, MD 20707 · School of Cybersecurity Old Dominion University Norfolk, VA 23259 · Old Dominion University Norfolk, VA 23259 +1
Sylvas: Synergistic Learning Value based Device Scheduling in Federated Continual Learning
Sep 14, 2026Yuxuan Sun, Yuxuan Bai, Tan Chen +2Federated Learning Replay-Based Continual Learning
School of Electronic and Information Engineering, Beijing Jiaotong University, Beijing 100044, China · Department of Electronic Engineering, Tsinghua University, Beijing 100084, China, and the Beijing National Research Center for Information Science and Technology
When Validation Stops Learning: Auditing Update Admission for Continual Embodied Agents
Sep 12, 2026Qinzhen Ma, Ruihai WuAdmissibility Replay-Based Continual Learning
Rice University · University of California, Berkeley
DODR: Deterministic Operator-Driven Reasoning in Latent Space
Sep 7, 2026Weicai Huang (Beijing MQPat Technologies, Co., Ltd.)Efficient Latent Reasoning Design Structure Matrices
Beijing MQPat Technologies, Co., Ltd.
Hypotheses-Guided Self Distillation for Continual Personalization
Aug 31, 2026EunJeong Hwang, Kushan Mitra, Dan Zhang +2Personalization Preference Alignment Learning
University of British Columbia · Work done as an intern at Megagon Labs. · Megagon Labs, USA
The Sequential Price of Continual Learning
Aug 30, 2026Zonghuan Xu, Xingjun MaReplay-Based Continual Learning Continual Learning
Institute of Trustworthy Embodied AI, Fudan University, Shanghai, China · Shanghai Key Laboratory of Multimodal Embodied AI, Shanghai, China
Harness Continual Learning: Continual Adaptation Beyond Model Parameters
Aug 19, 2026Borui Kang, Jinrui Gu, Junhan Lv +3Replay-Based Continual Learning Continual Learning
State Key Laboratory for Novel Software Technology, Nanjing University, China · University of Wollongong, Australia
Sustaining Plasticity via Learnable Wavelet Activations in Continual Learning
Aug 13, 2026Zeyang Zhang, Tieliang Gong, Junyan Lu +1Replay-Based Continual Learning Continual Learning
School of Computer Science and Technology, Xi’an Jiaotong University and Institute of Multimedia Knowledge Fusion and Engineering, Xi’an 710049, China
Drift and Dependence: Layer-wise Information-Theoretic Bounds for Replay-Based Continual Learning
Aug 12, 2026Tieliang Gong, Zhongbo Zhang, Wen Wen +1Replay-Based Continual Learning Continual Learning
Tieliang Gong, Zhongbo Zhang and Wen Wen are with the School of Computer Science and Technology, Xi’an Jiaotong University, China · Yong-Jin Liu is with Department of Computer Science and Technology, Tsinghua University, China
Macaron-V1: Towards Open Continual Learning with Self-Improvement and Mixture-of-LoRA
Aug 10, 2026Mind Lab, :, Vin Bo +74Replay-Based Continual Learning Mixture-Of-Agents
Hyperbolic Multimodal Continual Learning
Aug 10, 2026Jiahong Liu, Ming Shen, Xiaohao Liu +4Hyperbolic Learning Replay-Based Continual Learning
Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong, China · School of Computing, National University of Singapore, Singapore · Department of Computer Science, Yale University, New Haven, CT, United States +1
Agentic Router: An Execution-Grounded Continual Learning Approach With Memory
Aug 10, 2026Yuxuan Chen, Rongpeng Li, Zhifeng Zhao +3Qwen3 Data Generation
Zhejiang University, Hangzhou 310027, China · Zhejiang Lab, Hangzhou 310012, China · State Grid Hebei Electric Power Co., Ltd., Hebei, China +1
Continual-Learning Physics-Informed Neural Networks for Parameterized Partial Differential Equations
Aug 5, 2026Xujia Chen, Xinyue Hu, Letian Chen +2Parametric Physics-Informed Neural Network Partial Differential Equations
Department of Automation, Tsinghua University, Beijing, 100084, China
A Model Merging Approach for Continual MLLM Unlearning
Aug 5, 2026Yuhang Wang, Linlin Zhang, Haoxuan Ji +3Large Language Model Unlearning Exact Unlearning
1Xidian University, Xi’an, China · 2Xi’an Jiaotong University, Xi’an, China
NeuMoSync: End-to-End Neuromodulatory Control for Plasticity and Adaptability in Continual Learning
Aug 5, 2026Seyed Roozbeh Razavi Rohani, Khashayar Khajavi, Wesley Chung +2Replay-Based Continual Learning Plasticity
Department of Computer Science Simon Fraser University · Department of Computer Science McGill Univeristy, Mila - Quebec AI Institute
KC-Agent: A Dual-Process Cognitive Architecture for Efficient ML Model Improvement
Aug 3, 2026Gusseppe Bravo-Rocca, Jordi Guitart, Ajay Dholakia +2Replay-Based Continual Learning Mixture-Of-Agents
Barcelona Supercomputing Center, Barcelona, Spain · Cotiviti, South Jordan, UT, United States · Universitat Polit`ecnica de Catalunya, Barcelona, Spain +1
Learning What to Remember: Test-Time Training via Context Distillation
Aug 3, 2026Zixuan Wang, Xingyu Dang, Rui-Jie Zhu +4Efficient Long-Context Inference Test-Time Training
Princeton University · UC Santa Cruz · Carnegie Mellon University +1
UCBound-Net: Uncertainty-Guided Boundary-Aware Continual Learning for Domain-Incremental Ultrasound Segmentation
Aug 2, 2026Mohammad Amanour RahmanSemi-Supervised Medical Image Segmentation Replay-Based Continual Learning
Ahsanullah University of Science and Technology, Dhaka, Bangladesh
Plasticity of Growing and Elastic Neural Networks in Online Continual Learning
Aug 2, 2026Jeong Min Kong, Richard S. SuttonPlasticity Replay-Based Continual Learning
Department of Electrical and Computer Engineering University of California, Los Angeles (UCLA) · Department of Computing Science University of Alberta Alberta Machine Intelligence Institute (Amii)
SAFE-Merge: Data-Free Continual Model Merging with General Knowledge Preservation
Aug 2, 2026Zihuan Qiu, Zhiyang Liao, Chiyuan He +5Continual Model Merging Replay-Based Continual Learning
University of Electronic Science and Technology of China, Chengdu, China · Dalian University of Technology, Dalian, China
Relative Parameter Importance in Task-Agnostic Replay-Free Continual Learning
Aug 1, 2026Malavika Suresh, Ikechukwu Nkisi-Orji, Nirmalie WiratungaReplay-Based Continual Learning Continual Learning
Robert Gordon University
DSETA: A Dual-Stage Continual Learning Framework for Travel Time Prediction in Dynamic Traffic Environments
Aug 1, 2026Yanming Lyu, Yue Cheng, Lingkun Li +4Accurate Traffic Forecasting Traffic
Beijing Jiaotong University Beijing, China · Didichuxing Co. Ltd Beijing, China · Lancaster University Lancaster, United Kingdom
S-CEReBrO: Breaking the Memory Barrier in Continuous EEG Monitoring
Jul 30, 2026Glenn Anta Bucagu, Thorir Mar Ingolfsson, Yawei Li +1Electroencephalography Replay-Based Continual Learning
ETH Zurich, Zurich, Switzerland · Nanyang Technological University, Singapore · University of Bologna, Bologna, Italy
First-order Constrained Trilevel Optimization Over Distributed Networks for Robust Coreset Selection
Jul 30, 2026Yang Jiao, Kaixuan Jiao, Kai Yang +4Geometry-Aware Uncertainty Coresets Federated Learning
Richard · Southeast University · Anhui University +3
Training Skills Like Parameters via Self-Supervised Semantic Diffusion
Jul 30, 2026Mo Li, Zixin Yin, Ting Cao +1Contrastive Learning Replay-Based Continual Learning
Tsinghua University · Shanghai AI Laboratory · The Hong Kong University of Science and Technology
The Art of Not Forgetting A Local Learning Architecture for Continual Learning
Jul 29, 2026Ashmith Atmuri, Yashaswini Rao BhogarajulaCatastrophic Forgetting Replay-Based Continual Learning
Arkadhi Research
Multi-Scale Structural Features for Continual, Comprehensible Visual Recognition in a Developmental Learning Framework
Jul 28, 2026Zeki Doruk ErdenReplay-Based Continual Learning Multiscale
Sabanci University, Istanbul, Turkey
Calibrated Partial Resets: Preventing Policy Collapse in Continual Reinforcement Learning
Jul 27, 2026Luc McCutcheon, Evangelos Chatzaroulas, Saber FallahReplay-Based Continual Learning Plasticity
Department of Mechanical Engineering, University of Surrey, United Kingdom · Department of Computing Science, University of Surrey, United Kingdom
Latent-LoRA: Compact Latent-Space Adapters with Gradient-Free Routing for Continual Learning
Jul 26, 2026Reza Rahimi Azghan, Gautham Krishna Gudur, Giulia Pedrielli +2Replay-Based Continual Learning Catastrophic Forgetting
College of Health Solutions, Arizona State University, Phoenix, Arizona, USA · Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, Texas, USA · School of Computing and Augmented Intelligence, Arizona State University, Tempe, Arizona, USA +1
Spectral-Aware Analytic Class-Incremental Learning for Long-Tailed Distributions
Jul 24, 2026Quyen Tran, Hai Nguyen, Quan Dao +4Class-Incremental Learning Regularization
1Rutgers University · 2Tuft University · 3HUST +1
FSE: Continual Learning for Named Entity Recognition by Fast-Slow Experts
Jul 24, 2026Yunan Zhang, Yang Fan, Heng Li +2Named-Entity Recognition Replay-Based Continual Learning
School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen, Guangdong, China

Oct 6, 2026, cs.AIJ/K move · Enter open · S save

CoDe-LoRA: Mitigating the Orthogonality Dilemma in Continual Learning of LLMs via Knowledge Consolidation and Decoupling

Maoqi Liu, Quan Fang, Yufei He

Beijing University of Posts and Telecommunications · National University of Singapore

Replay-Based Continual Learning Large Language Model Adaptation

Open paper AskPDF arXiv

Continual learning (CL) is essential for Large Language Models (LLMs) to sequentially adapt to evolving tasks. To mitigate catastrophic forgetting, recent advances implement low-rank adaptation with orthogonal projections (e.g., O-LoRA) to isolate task parameters. However, we reveal that such strict geometric constraints trigger an "Orthogonality Dilemma": rigid parameter isolation impedes the transfer and accumulation of shared representations across semantically related tasks. In this work, we propose a new replay-free method, called Consolidation and Decoupling LoRA (CoDe-LoRA), for CL of LLMs. CoDe-LoRA disentangles the learning process into Consolidating Universal Knowledge and Decoupling Task-Specific Knowledge. To achieve this, CoDe-LoRA leverages an adaptive null space projection mechanism and semantic routing to balance knowledge accumulation with task-specific adaptation. Experimental results across four backbones and three CL benchmarks show that CoDe-LoRA achieves the best average accuracy. Our code is available at https://github.com/Estrellajer/CoDe-LoRA.

Momentum

Related topics

Latest papers 231