LLM Agent Self-Improvement

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

44 papers in the last four weeks, up 267% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 189

All topics

AEL: Agent Evolving Learning for Open-Ended Environments
Apr 23, 2026Wujiang Xu, Jiaojiao Han, Minghao Guo +4LLM Agent Self-Improvement LLM Agents
Rutgers University · Independent Researcher
The Last Harness You'll Ever Build
Apr 22, 2026Haebin Seong, Li Yin, Haoran Zhang +1Meta-Learning Agent Harness Optimization
Training LLM Agents for Spontaneous, Reward-Free Self-Evolution via World Knowledge Exploration
Apr 20, 2026Qifan Zhang, Dongyang Ma, Tianqing Fang +5Intrinsic Motivation LLM Agent Self-Improvement
1Tencent · 2The Hong Kong University of Science and Technology (Guangzhou)
Autogenesis: A Self-Evolving Agent Protocol
Apr 16, 2026Wentao Zhang, Zhe Zhao, Haibin Wen +4Multi-Agent LLM Systems LLM Agent Self-Improvement
Nanyang Technological University · Stanford University · City University of Hong Kong +1
Semantic Feature Analysis: Improving Agents Without Searching Over Rollouts
Apr 12, 2026Yuval David, Fabiana Fournier, Lior Limonad +1LLM Agent Self-Improvement Tool-Using Agents
IBM Software Innovation Lab Haifa, Israel
TRACE: Capability-Targeted Agentic Training
Apr 7, 2026Hangoo Kang, Tarun Suresh, Jon Saad-Falcon +1LLM Agent Self-Improvement Agent Skill Learning
Stanford University
GAIA: A Data Flywheel System for Training GUI Test-Time Scaling Critic Models
Jan 26, 2026Shaokang Wang, Pei Fu, Ruoceng Zhang +7GUI Agents Test-Time Scaling
MiLM Plus, Xiaomi Inc, Beijing, China · School of Computer Science, Peking University, Beijing, China
Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoning
Nov 20, 2025Peng Xia, Kaide Zeng, Jiaqi Liu +5LLM Agent Self-Improvement Multi-Agent LLMs
UNC-Chapel Hill · Salesforce Research · Stanford University
SimSkill: A Self-Evolving LLM Agent for Skill and Knowledge Accumulation in Traffic Simulation
Date pendingQi Liu, Qinzheng Wang, Can Li +2Continual Learning for LLM Agents LLM Agent Self-Improvement
School of Transportation, Jilin University, 5988 Renmin St., Changchun, Jilin 130022, China · Key Laboratory of Road and Traffic Engineering of the Ministry of Education, College of Transportation, Tongji University, 4800 Cao’an Rd., Shanghai 201804, China

Apr 23, 2026, cs.CLJ/K move · Enter open · S save

AEL: Agent Evolving Learning for Open-Ended Environments

Wujiang Xu, Jiaojiao Han, Minghao Guo, Kai Mei+3

Rutgers University · Independent Researcher

LLM Agent Self-Improvement LLM Agents Memory-Augmented Language Model Agents

Open paper AskPDF arXiv

LLM agents increasingly operate in open-ended environments spanning hundreds of sequential episodes, yet they remain largely stateless: each task is solved from scratch without converting past experience into better future behavior. The central obstacle is not \emph{what} to remember but \emph{how to use} what has been remembered, including which retrieval policy to apply, how to interpret prior outcomes, and when the current strategy itself must change. We introduce \emph{Agent Evolving Learning} (\ael{}), a two-timescale framework that addresses this obstacle. At the fast timescale, a Thompson Sampling bandit learns which memory retrieval policy to apply at each episode; at the slow timescale, LLM-driven reflection diagnoses failure patterns and injects causal insights into the agent's decision prompt, giving it an interpretive frame for the evidence it retrieves. On a sequential portfolio benchmark (10 sector-diverse tickers, 208 episodes, 5 random seeds), \ael{} achieves a Sharpe ratio of 2.13

\pm

0.47, outperforming five published self-improving methods and all non-LLM baselines while maintaining the lowest variance among all LLM-based approaches. A nine-variant ablation reveals a ``less is more'' pattern: memory and reflection together produce a 58% cumulative improvement over the stateless baseline, yet every additional mechanism we test (planner evolution, per-tool selection, cold-start initialization, skill extraction, and three credit assignment methods) \emph{degrades} performance. This demonstrates that the bottleneck in agent self-improvement is \emph{self-diagnosing how to use} experience rather than adding architectural complexity. Code and data: https://github.com/WujiangXu/AEL.

Momentum

Related topics

Latest papers 189