Self-Evolving Agents

Follow this topic Ask about this topic

Self-evolving agents are systems that use experience from task execution to change how they solve future tasks, rather than relying only on a fixed model or hand-designed workflow. Common approaches store and refine reusable skills or memories, optimize prompts and agent harnesses, train policies with reinforcement learning, or co-evolve task generators, environments, and teams of agents. These methods are tested in settings such as web and GUI use, coding, embodied robotics, scientific workflows, and reasoning benchmarks, with feedback ranging from execution outcomes to verifiers and paired validation. A central challenge is making improvements transfer to new tasks without forgetting earlier abilities or introducing unsafe behavior; reliable evaluation, update verification, and long-term monitoring remain open problems.

Latest papers

427 papers104 papers in the last four weeks, up 148% on the four weeks before. 0.6% of all new papers.

Self-evolving LLM agents with in-distribution Optimization
Jun 5, 2026Yudi Zhang, Meng Fang, Zhenfang Chen +1Reinforcement Learning Process Reward Models
Eindhoven University of Technology, Netherlands · University of Liverpool, United Kingdom · MIT-IBM Watson AI Lab.
FinEvolveBench: A Benchmark for Self-Evolving Agents on Low-Repetition Tasks with Implicit Rewards
Jun 5, 2026Yining Zhu, Zihao Deng, Leiming Wang +5Continual Learning for LLM Agents AI Agent Benchmarks
OpenSkill: Open-World Self-Evolution for LLM Agents
Jun 4, 2026Zhiling Yan, Dingjie Song, Hanrong Zhang +8LLM Agent Skill Learning LLM Agent Verification
Lehigh University · University of Illinois Chicago · University of British Columbia +3
MLEvolve: A Self-Evolving Framework for Automated Machine Learning Algorithm Discovery
Jun 4, 2026Shangheng Du, Xiangchao Yan, Jinxin Shi +13Automated Algorithm Discovery Multi-Agent LLMs
Shanghai Artificial Intelligence Laboratory · East China Normal University
Towards Healthy Evolution: Exploring the Role and Mechanisms of Human-Agent Interaction in Self-Evolving Systems
Jun 4, 2026Dianxing Shi, Bowen Wang, Junqi He +2Human-in-the-Loop Evaluation LLM Safety Alignment
The University of Osaka · Independent researcher
Agentic Software: How AI Agents Are Restructuring the Software Paradigm
Jun 4, 2026Zhenfeng CaoSoftware Engineering Agents Automated Software Engineering
Lingxi Intelligent Investment (Shenzhen) Development Co., Ltd.
EpiEvolve: Self-Evolving Agents for Streaming Pandemic Forecasting under Regime Shifts
Jun 3, 2026Yiming Lu, Sihang Zeng, Zhengxu Tang +3Online Time Series Forecasting Epidemic Forecasting
Emory University · University of Washington
VASO: Formally Verifiable Self-Evolving Skills for Physical AI Agents
Jun 3, 2026Yunhao Yang, Neel P. Bhatt, Kevin Wang +3Robot Skill Learning Large Language Model-Based Robot Planning
1The University of Texas at Austin · 2Iowa State University
Autonomous heterogeneous catalyst discovery with a self-evolving multi-agent digital twin
Jun 3, 2026Zhilong Song, Zongmin Zhang, Lixue ChengMaterials Science Digital Twins
Department of Chemistry, Hong Kong University of Science and Technology, Kowloon, Hong Kong 999077, China · IAS Center for AI for Scientific Discoveries, Hong Kong University of Science and Technology, Kowloon, Hong Kong 999077, China · Department of Computer Science and Engineering, Hong Kong University of Science and Technology, Kowloon, Hong Kong 999077, China +1
Scaling Self-Evolving Agents via Parametric Memory
Jun 3, 2026Tao Ren, Weiyao Luo, Hui Yang +8Online Language Model Adaptation RL for LLM Agents
Qwen-Character Team · Alibaba Group · Peking University
Agent libOS: A Runtime Substrate for Capability-Controlled Self-Evolving LLM Agents
Jun 2, 2026Yingqi ZhangAI Agent Security LLM Agents
Department of Computer Science and Technology, Tsinghua University Beijing, China
EvoDS: Self-Evolving Autonomous Data Science Agent with Skill Learning and Context Management
Jun 2, 2026Zherui Yang, Fan Liu, Yansong Ning +1Agentic RL LLM Agent Skill Learning
The Hong Kong University of Science and Technology (Guangzhou)
SkillPyramid: A Hierarchical Skill Consolidation Framework for Self-Evolving Agents
Jun 2, 2026Yuan Xiong, Ziqi Miao, Qian Chen +6Lifelong Learning Agents LLM Agent Skill Learning
The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China · Shanghai Artificial Intelligence Laboratory, Shanghai, China +1
EvoTrainer: Co-Evolving LLM Policies and Training Harnesses for Autonomous Agentic Reinforcement Learning
Jun 2, 2026Guhong Chen, Yingcheng Shi, Yongbin Li +6RL for Code Generation Agentic RL
1Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences · 2Tongyi Lab , Alibaba Group · 3Alibaba Group +1
Economy of Minds: Emerging Multi-Agent Intelligence with Economic Interactions
Jun 1, 2026Zhenting Qi, Huangyuan Su, Ao Qu +13Multi-Agent Coordination Multi-Agent Reasoning
Harvard · MIT
Traj-Evolve: A Self-Evolving Multi-Agent System for Patient Trajectory Modeling in Lung Cancer Early Detection
Jun 1, 2026Sihang Zeng, Matthew Thompson, Ruth Etzioni +1Retrieval-Augmented Generation Medical Diagnosis
University of Washington · Fred Hutch Cancer Center · Google
CoMAP: Co-Evolving World Models and Agent Policies for LLM Agents
Jun 1, 2026Youwei Liu, Jian Wang, Hanlin Wang +1LLM World Models LLM Agent Training
Central South University · College of Computer Science, Sichuan University · Department of Computing, The Hong Kong Polytechnic University
Unified Context Evolution for LLM Agents
Jun 1, 2026Zixuan Zhu, Yitong Hu, Yong Dai +4Continual Learning for LLM Agents LLM Agent Skill Learning
Nanyang Technological University · Beijing University of Posts and Telecommunications · Fudan University +4
MetaForge: A Self-Evolving Multimodal Agent that Retrieves, Adapts, and Forges Tools On Demand
Jun 1, 2026Shouang Wei, Houcheng Min, Xinpeng Dong +8LLM Agent Skill Learning Multimodal Agents
East China Normal University · Zhejiang University · Tsinghua University +2
Adaptive Auto-Harness: Sustained Self-Improvement for Agentic System Deployment on Open-Ended Task Streams
Jun 1, 2026Zewen Liu, Zhan Shi, Yisi Sang +7Continual Learning for LLM Agents Agent Harness Optimization
Emory University · Amazon · The Pennsylvania State University +2
MobEvolve: An Agentic Self-Evolving Heuristic System for Interpretable Human Mobility Generation
Jun 1, 2026Junlin He, Yihong Tang, Tong Nie +6Human Behavior Simulation Self-Evolving Agents
1The Hong Kong Polytechnic University · 2McGill University · 8Mila – Quebec AI Institute +4
SkillSmith: Co-Evolving Skills and Tools for Self-Improving Agent Systems
May 31, 2026Yangbo Wei, Zhen Huang, Shaoqiang Lu +4Tool-Using Agents Agent Skill Learning
Shanghai Jiao Tong University, Shanghai, China · Eastern Institute of Technology, Ningbo, China · University of Science and Technology of China, Hefei, China +2
MemPro: Agentic Memory Systems as Evolvable Programs
May 30, 2026Qingshan Liu, Guoqing Wang, Wen Wu +5Memory-Augmented Agents Agent Memory Management
East China Normal University · Xiaohongshu Inc.
MetaEvo: A Meta-Optimization Framework for Experience-Driven Agent Evolution
May 29, 2026Bowen Ren, Heyan Huang, Yinghao Li +1Continual Learning for LLM Agents LLM Agent Self-Improvement
School of Computer Science and Technology, Beijing Institute of Technology, Beijing, China · Beijing Institute of Technology Southeast Academy of Information Technology, Putian, China
Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agents
May 28, 2026Minhua Lin, Juncheng Wu, Zijun Wang +14LLM Agent Evaluation Agent Harness Optimization
1The Pennsylvania State University · 2UC Santa Cruz · 3Amazon +3
Evolve as a Team: Collaborative Self-Evolution for LLM-based Multi-Agent Systems
May 28, 2026Zhezheng Hao, Tianfu Wang, Huanshuo Dong +7Multi-Agent System Optimization Multi-Agent Coordination
Zhejiang University · Hong Kong University of Science and Technology · Tencent
PTCG-Bench: Can LLM Agents Master Pokémon Trading Card Game?
May 28, 2026Dongdong Hua, Yifei Sun, Renhong Huang +3Game-Playing Agents LLM Agent Evaluation
Zhejiang University · FinVolution Group
BenchTrace: A Benchmark for Testing Reflection Ability and Controlled Evolution in LLM Agents
May 28, 2026Jiahao Huang, Fei Cheng, Junfeng Jiang +2LLM Agent Evaluation Self-Reflection in Language Model Agents
University of Tokyo · Kyoto University · National Institute of Informatics
You Live More Than Once: Towards Hierarchical Skill Meta-Evolving
May 27, 2026Xujun Li, Kehan Zheng, Mingyuan Zhao +7Meta-Learning LLM Agent Skill Learning
Department of Computer Science and Technology, Tsinghua University · Huawei Foundation Model Department
ESC-Skills: Discovering and Self-Evolving Skills for Emotional Support Conversations
May 27, 2026Jie Zhu, Huaixia Dou, Shuo Jiang +5Emotional Support Conversation LLM Agent Skill Learning
School of Computer Science and Technology, Soochow University · 2Qwen DianJin Team, Alibaba Cloud Computing

Jun 5, 2026J/K move · Enter open · S save

Self-evolving LLM agents with in-distribution Optimization

Yudi Zhang, Meng Fang, Zhenfang Chen, Mykola Pechenizkiy

Eindhoven University of Technology, Netherlands · University of Liverpool, United Kingdom · MIT-IBM Watson AI Lab.

Reinforcement Learning Process Reward Models Credit Assignment in RL RL for LLM Agents Self-Evolving Agents

Open paper AskPDF arXiv

Large Language Models (LLMs) have recently emerged as powerful controllers for interactive agents in complex environments, yet training them to perform reliable long-horizon decision making remains a fundamental challenge. A key difficulty lies in credit assignment: agents often receive delayed rewards only at the end of episodes. In this paper, we propose Q-Evolve, a self-evolving framework for LLM agents that unifies automatic process-reward labeling and policy learning within a principled in-distribution reinforcement learning paradigm. In each evolving iteration, our method learns an in-distribution critic from a hybrid off-policy dataset that combines expert demonstrations with agent-generated trajectories, stabilizing Bellman backups in sparse-reward settings via a weighted Implicit Q-Learning objective. The learned value function is then used to derive step-wise process rewards through advantage estimation, enabling dense and reliable supervision without environment backtracking or human annotation. Leveraging these signals, we perform behavior-proximal policy optimization that evolves the agent over the data used for process reward labeling, allowing iterative self-improvement without exacerbating distribution shift. We evaluate our method on AlfWorld, WebShop, and ScienceWorld, showing Q-Evolve outperforms strong baselines in sample efficiency, robustness, and overall task performance. Our results demonstrate that stable agent self-evolution is achievable through the co-evolution of process-level supervision and policy, both grounded within a shared in-distribution learning loop.