Continual Learning for LLM Agents

LLM: Large Language Model

Overview

Continual learning for large language model agents asks how they can improve from ongoing interaction, feedback, and changing environments rather than starting each task from scratch. Common approaches store and retrieve episodes, rules, or reusable skills; refine prompts and agent harnesses; or use reinforcement learning and fine-tuning to consolidate experience into model parameters. Work evaluates these methods in long-horizon settings such as web tasks, coding, games, and embodied environments, where agents must transfer what they learn to later tasks. A central tension is balancing adaptation with retention: accumulated memories and updates can cause interference, stale guidance, or forgetting, and gains in one setting do not always transfer to another.

Latest papers

CardsList

221 papers32 papers in the last four weeks, down 27% on the four weeks before. 0.2% of all new papers.

  1. When Agent Skills Go Stale: Runtime Behavior and Selective Maintenance in Evolving Environments

    Nov 17, 2026Ömer Faruk Akgül, Shuli Jiang, Yuting Zhang +4LLM Agent Skill LearningLLM Agent Skill Evaluation

  2. When Should Agents Remember? Falsification-Gated Self-Evolution for LLM Agents

    Nov 10, 2026Runxuan Tang, Haoyu Gao, Yuyan Ding +4LLM Agent Self-ImprovementLLM Agent Memory

  3. The RSI Tax: When Recursive Self-Improvement Is Not Worth the Cost

    Nov 10, 2026Yitong Zhang, Ang Li, Yisen WangLLM Agent Self-ImprovementRecursive Self-Improvement

  4. SkillDuet: Co-Evolving Models and Skills through Response Feedback

    Nov 9, 2026Xingguang Ji, Ziqiang Dong, Yubo Zhang +4Agent Skill LearningExperience Learning for LLM Agents

  5. Towards Robust Long-Lived Agents: Benchmarking Experience Lifecycles in Evolving Environments

    Nov 3, 2026Shijue Huang, Hangyu Guo, Hengheng Zhang +4Experience Learning for LLM AgentsAI Agent Benchmarks

  6. From AutoResearch to Continual AutoResearch: Do Agents Learn Across Tasks?

    Nov 3, 2026Jeremy Qin, Ruizhi Deng, Mohamed Osama Ahmed +2Autonomous Research AgentsContinual Learning for LLM Agents

  7. From Watching to Steering: Enabling Human-Coding Agent Collaboration through Execution Traces

    Nov 3, 2026Xiaoke Yan, Jiaying Lei, Danqing Shi +2AI Coding AgentsLLM Agents

  8. WASP-Mem: Let Agent Organize Procedural Memory. Human Spend Effort on Diagnosis

    Nov 2, 2026Peizhi Niu, Ge Shi, Vaibhav Gorde +4Procedural Memory for LLM AgentsMemory Consolidation for LLM Agents

  9. CARE: Grounded Recursive Self-Improvement for Production Customer-Service Agents

    Nov 2, 2026Yanbei Jiang, Chao Lei, Rena Gao +17Recursive Self-ImprovementContinual Learning for LLM Agents

  10. Do agents need to sleep?

    Nov 1, 2026Ido Haber, Giulio TononiMemory Consolidation for LLM AgentsMemory Management for LLM Agents

  11. Skill++: Towards Lifelong Agents via Routing-Aware Skill Refinement

    Oct 31, 2026Hanrong Zhang, Minghao Liang, Jiawei Wu +10LLM Agent Skill RetrievalContinual Learning for LLM Agents

  12. Exo: An Architecture for Self-Improving Systems at Runtime

    Oct 30, 2026Alexander Krentsel, Sylvia Ratnasamy, Ion Stoica +2AI Agent ArchitectureAgent Harness Optimization

  13. TradeWar: A Benchmark for Execution-Grounded Trade Negotiation with Adaptive LLM Agents

    Oct 30, 2026Xiangyu Kong, Ruirui Yang, Xia HouAutomated NegotiationMulti-Agent Negotiation

  14. KARMA: Training-Free Knowledge-Atomic Harness Evolution for Industrial Advertising Agents

    Oct 29, 2026Xinhua Miao, Yuanzhen Rong, Weiru Zhang +1LLM AgentsSelf-Improving Agents

  15. Routed Continual Learning

    Oct 28, 2026Michael Philip, Mahesh Balakrishnan, David Geraghty +3Continual Learning for LLM AgentsMemory-Augmented Language Model Agents

  16. FilingLoop: Evaluating and Evolving LLM Agents for Corporate Disclosure Review and Revision

    Oct 27, 2026Wentao Zhang, Yilei Zhao, Guangyan Gan +2LLM Agent EvaluationContinual Learning for LLM Agents

  17. When the Critic Falls Behind: Co-Evolving Rubrics and Harnesses for Continual Agent Self-Improvement

    Oct 27, 2026Hao Chen, Yukun Yan, Ziyu Han +7Self-Improving AgentsAgent Harness Optimization

  18. Skill Dreaming: Evolving Repository Skills through Pull Request Streams

    Oct 26, 2026Xinyuan Song, Hasibul Haque, Jay Somasundaram +3Automated Software EngineeringLLM Agent Skill Learning

  19. Harness-Model Collaboration for Long-Horizon Continual Learning

    Oct 26, 2026Zhen Bi, Zhizhi Peng, Yan Wang +6Continual Learning for LLM AgentsContinual Learning

  20. CauProphet: Teaching Agents to Develop Causal Cognition like Humans

    Oct 23, 2026Kaiwen Wu, Chaochao LuCausal Reasoning in Language ModelsContinual Learning for LLM Agents

  21. Don't Forget What You Just Learned: Self-Directed Repair Turns Agent Mistakes Into Lasting Lessons

    Oct 22, 2026Yuval Kansal, Karen Ullrich, Polina Kirichenko +4Self-Improving AgentsLLM Self-Correction

  22. HERD: Harness-Evolving Recursive Distillation for Continual Learning

    Oct 22, 2026Xun Sun, Thanh Thi Nguyen, Thuy-Trang VuContinual LearningContinual Learning for LLM Agents