Continual Learning for LLM Agents

LLM: Large Language Model

Continual learning for large language model agents asks how they can improve from ongoing interaction, feedback, and changing environments rather than starting each task from scratch. Common approaches store and retrieve episodes, rules, or reusable skills; refine prompts and agent harnesses; or use reinforcement learning and fine-tuning to consolidate experience into model parameters. Work evaluates these methods in long-horizon settings such as web tasks, coding, games, and embodied environments, where agents must transfer what they learn to later tasks. A central tension is balancing adaptation with retention: accumulated memories and updates can cause interference, stale guidance, or forgetting, and gains in one setting do not always transfer to another.

Latest papers

202 papers45 papers in the last four weeks, up 105% on the four weeks before. 0.3% of all new papers.

SOLAR: A Self-Optimizing Open-Ended Autonomous Agent for Lifelong Learning and Continual Adaptation
Mar 23, 2026Nitin Vetcha, Dianbo LiuContinual Learning for LLM Agents Continual Learning for LLMs
Department of Ophthalmology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore · Department of Computational and Data Sciences, Indian Institute of Science, Bangalore, Karnataka, India
REVERE: Reflective Evolving Research Engineer
Mar 21, 2026Balaji Dinesh Gangireddi, Aniketh Garikaparthi, Manasi Patwardhan +1LLM Agent Self-Improvement Prompt Optimization
TCS Research · Yale University
Memento-Skills: Let Agents Design Agents
Mar 19, 2026Huichi Zhou, Siyuan Guo, Anjie Liu +14Agentic AI LLM Agent Skill Learning
AgentFactory: A Self-Evolving Framework Through Executable Subagent Accumulation and Reuse
Mar 18, 2026Zhang Zhang, Shuqi Lu, Hongjin Qian +2Self-Evolving Agents LLM Agent Skill Learning
Peking University · Beijing Academy of Artificial Intelligence · Hong Kong Polytechnic University
MetaClaw: Just Talk -- An Agent That Meta-Learns and Evolves in the Wild
Mar 17, 2026Peng Xia, Jianwen Chen, Xinyu Yang +10Continual Learning for LLM Agents LLM Agent Skill Learning
UNC-Chapel Hill · Carnegie Mellon University · UC Santa Cruz +1
VERDICT: Verifiable Evolving Reasoning with Directive-Informed Collegial Teams for Legal Judgment Prediction
Mar 13, 2026Hui Liao, Chuan Qin, Yongwen Ren +4Legal Judgment Prediction Legal Reasoning
University of Science and Technology of China · Chinese Academy of Sciences · iFLYTEK AI Research
Can RL Improve Generalization of LLM Agents? An Empirical Study
Mar 12, 2026Zhiheng Xi, Xin Guo, Jiaqi Liu +11LLM Agents RL for LLM Agents
Fudan University · Shanghai Artificial Intelligence Laboratory · Meituan
Compiled Memory: Not More Information, but More Precise Instructions for Language Agents
Mar 12, 2026James Rhodes, George KangLLM Agent Memory Memory-Augmented Language Model Agents
AlphaBitCore, Inc.
Towards Cold-Start Drafting and Continual Refining: A Value-Driven Memory Approach with Application to NPU Kernel Synthesis
Mar 11, 2026Yujie Zheng, Zhuo Li, Shengtao Zhang +8Memory-Augmented Language Model Agents RL for Code Generation
Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory · Independent Researcher +1
Reflection-Driven Self-Improving Language Model Agents for Adaptive RAN Slicing
Mar 11, 2026Yuanhao Li, Haozhe Wang, Geyong Min +2Self-Improving Agents Language Model-Based Control
Department of Computer Science, Faculty of Environment, Science and Economy, University of Exeter, Exeter, EX4 4QF, UK. · British Telecom, UK.
RetroAgent: From Solving to Evolving via Retrospective Dual Intrinsic Feedback
Mar 9, 2026Xiaoying Zhang, Zichen Liu, Yipeng Zhang +2Reinforcement Learning RL for LLM Agents
Shanghai AI Lab · National University of Singapore · Independent Researcher
Adaptive Collaboration with Humans: Metacognitive Policy Optimization for Multi-Agent LLMs with Continual Learning
Mar 9, 2026Wei Yang, Defu Cao, Jiacheng Pang +2Human-in-the-Loop AI Multi-Agent LLM Systems
University of Southern California
Lifelong Embodied Navigation Learning
Mar 6, 2026Xudong Wang, Jiahua Dong, Baichen Liu +3Embodied Navigation Continual Learning for LLM Agents
State Key Laboratory of Robotics and Intelligent Systems, Shenyang Institute of Automation, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Mohamed bin Zayed University of Artificial Intelligence
CGL: Advancing Continual GUI Learning via Reinforcement Fine-Tuning
Mar 3, 2026Zhenquan Yao, Zitong Huang, Yihan Zeng +5Continual Learning for LLM Agents RL for GUI Agents
Harbin Institute of Technology · Huawei Noah’s Ark Lab · University College Dublin +1
AOI: Turning Failed Trajectories into Training Signals for Autonomous Cloud Diagnosis
Mar 3, 2026Pei Yang, Wanyi Chen, Asuka Yuxi Zheng +13LLM Agent Training AI Agent Reliability
Gradient · Soochow University · UC Santa Cruz +4
Position: Modular Memory is the Key to Continual Learning Agents
Mar 2, 2026Vaggelis Dorovatas, Malte Schwerin, Andrew D. Bagdanov +21Continual Learning for LLM Agents Memory-Augmented Agents
Toyota Motor Europe · University of Bremen · University of Florence +17
Help Without Being Asked: A Deployed Proactive Agent System for On-Call Support with Continuous Self-Improvement
Feb 25, 2026Fengrui Liu, Xiao He, Tieying ZhangProactive Assistance Customer Support Automation
ByteDance Beijing, China · ByteDance Hangzhou, China · ByteDance San Jose, United States
Your Code Agent Can Grow Alongside You with Structured Memory
Feb 25, 2026Yi-Xuan Deng, Xiaoqin Liu, Yi Zhang +2Software Engineering Agents Memory-Augmented Agents
Tsinghua University · Intern at Proxseer Inc · Proxseer Inc.
Just-In-Time Reinforcement Learning: Continual Learning in LLM Agents Without Gradient Updates
Jan 26, 2026Yibo Li, Zijie Lin, Ailin Deng +5Continual Learning for LLM Agents Reinforcement Learning
National University of Singapore, Singapore.
A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agents
Jan 18, 2026Tianxin Wei, Ting-Wei Li, Zhining Liu +26Continual Learning for LLM Agents Multi-Agent Reasoning
University of Illinois Urbana-Champaign · Meta · UCSD +2
Evolving Programmatic Skill Networks
Jan 7, 2026Haochen Shi, Xingdi Yuan, Bang LiuContinual Learning for LLM Agents LLM-Based Program Synthesis
DIRO & Institut Courtois, Université de Montréal · Mila – Québec AI Institute · Microsoft Research +1
Memento No More: Coaching AI Agents to Master Multiple Tasks via Hints Internalization
Feb 3, 2025Minttu Alakuijala, Ya Gao, Georgy Ananov +4Continual Learning for LLM Agents LLM Agent Skill Learning
Department of Computer Science, Aalto University · Department of Computer Science, University of Manchester · System 2 AI

Mar 23, 2026J/K move · Enter open · S save

SOLAR: A Self-Optimizing Open-Ended Autonomous Agent for Lifelong Learning and Continual Adaptation

Nitin Vetcha, Dianbo Liu

Department of Ophthalmology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore · Department of Computational and Data Sciences, Indian Institute of Science, Bangalore, Karnataka, India

Continual Learning for LLM Agents Continual Learning for LLMs RL for LLM Agents Meta-Learning Test-Time Adaptation Memory-Augmented Language Model Agents

Open paper AskPDF arXiv

Despite the remarkable success of large language models (LLMs), they still face bottlenecks while deploying in dynamic, real-world settings with primary challenges being concept drift and the high cost of gradient-based adaptation. Traditional fine-tuning (FT) struggles to adapt to non-stationary data streams without resulting in catastrophic for getting or requiring extensive manual data curation. To address these limitations within the streaming and continual learning paradigm, we propose the Self-Optimizing Lifelong Autonomous Reasoner (SOLAR) which is an open-ended autonomous agent that leverages parameter-level meta-learning to self-improve, treating model weights as an environment for exploration. It initiates the process by consolidating a strong prior over common-sense knowledge making it effective for transfer-learning. By utilizing a multi-level reinforcement learning approach, SOLAR autonomously discovers adaptation strategies, enabling efficient test-time adaptation to unseen domains. Crucially, SOLAR maintains an evolving knowledge base of valid modification strategies, implicitly acting as an episodic memory buffer to balance plasticity (adaptation to new tasks) and stability (retention of meta-knowledge). Experiments demonstrate that SOLAR outperforms strong baselines on common-sense, mathematical, medical, coding, social and logical reasoning tasks, marking a significant step toward autonomous agents capable of lifelong adaptation in evolving environments.