LLM Agent Self-Improvement

LLM: Large Language Model

This field studies how language-model agents can improve across tasks by learning from their own executions, failures, human feedback, or interaction with an environment. Common approaches update prompts, harnesses, skills, memory, or tools while keeping model weights fixed; others use reinforcement learning, self-distillation, or co-evolving tasks and agents. Methods often analyze trajectories to diagnose weaknesses, then test candidate changes with benchmarks, paired comparisons, or held-out regression gates. A central challenge is making gains transfer to new settings without eroding previously reliable behavior, while keeping evaluation trustworthy and improvement costs manageable.

Latest papers

277 papers73 papers in the last four weeks, up 121% on the four weeks before. 0.4% of all new papers.

GRASP: Gated Regression-Aware Skill Proposer for Self-Improving LLM Agents
May 28, 2026Johannes Moll, Jean-Philippe Corbeil, Jiazhen Pan +4LLM Agent Self-Improvement LLM Agent Skill Learning
1Technical University of Munich and TUM University Hospital · 2Microsoft Healthcare & Life Sciences
SkillBrew: Multi-Objective Curation of Skill Banks for LLM Agents
May 28, 2026Wentao Hu, Zhendong Chu, Yiming Zhang +6LLM Agent Self-Improvement LLM Agent Skill Learning
City University of Hong Kong · Squirrel Ai Learning · University of Science and Technology of China +4
SkillGrad: Optimizing Agent Skills Like Gradient Descent
May 26, 2026Hanyu Wang, Yifan Lan, Bochuan Cao +2Large Language Model-Guided Optimization LLM Agent Self-Improvement
College of Information Sciences and Technology The Pennsylvania State University University Park, PA, USA
Libra: Training the Environment for Agentic Information Retrieval
May 26, 2026Xuan Zhao, Andy Chiu, Gengyu WangAI Coding Agents Code Localization
Salesforce
SIA: Self Improving AI with Harness & Weight Updates
May 26, 2026Prannay Hebbar, Yogendra Manawat, Samuel Verboomen +4Agent Harness Optimization LLM Agent Self-Improvement
Hexo Labs Palo Alto · Hexo Labs Brussels · University of Oxford +1
CyberEvolver: Structured Self-Evolution for Cybersecurity Agents On the Fly
May 25, 2026Yihe Fan, Changyi Li, Lichen Xu +4LLM Agent Self-Improvement Automated Penetration Testing
Fudan University, Shanghai, China · Shanghai Innovation Institute, Shanghai, China · Shanghai Pudong Research Institute of Cryptology
SetupX: Can LLM Agents Learn from Past Failures in Functionality-Correct Code Repository Setup?
May 25, 2026Zihang Zhou, Ziqian Ren, Yukai Wu +7Software Engineering Agents LLM Agent Self-Improvement
Shanghai Jiao Tong University · Beijing University of Posts and Telecommunications · Independent Researcher +2
DemoEvolve: Demonstration-Guided Harness Evolution under Sparse Feedback
May 23, 2026Lirong Che, Yuzhe yang, Peiwen lin +4Agent Harness Optimization LLM Agent Self-Improvement
Tsinghua University · AgiBot
SkillOpt: Executive Strategy for Self-Evolving Agent Skills
May 22, 2026Yifan Yang, Ziyang Gong, Weiquan Huang +12Large Language Model-Guided Optimization LLM Agent Self-Improvement
Microsoft · Shanghai Jiao Tong University · Tongji University +1
MOSS: Self-Evolution through Source-Level Rewriting in Autonomous Agent Systems
May 21, 2026Qianshu Cai, Yonggang Zhang, Xianzhang Jia +5Software Engineering Agents LLM Agent Self-Improvement
University of Science and Technology of China · Hong Kong Generative AI Research & Development Center · The Hong Kong University of Science and Technology +1
EVE-Agent: Evidence-Verifiable Self-Evolving Agents
May 21, 2026Yamato Arai, Yuma IchikawaLLM Agent Self-Improvement LLM Agent Training
Fujitsu Limited Department of Basic Science · The University of Tokyo · Fujitsu Limited +1
Search-E1: Self-Distillation Drives Self-Evolution in Search-Augmented Reasoning
May 21, 2026Zihan Liang, Yufei Ma, Ben Chen +4Tool-Augmented Language Model Agents On-Policy Self-Distillation
Ratchet: A Minimal Hygiene Recipe for Self-Evolving LLM Agents
May 21, 2026Xing Zhang, Yanwei Cui, Guanghui Wang +4Continual Learning for LLM Agents LLM Agent Self-Improvement
AWS Generative AI Innovation Center · HSBC Holdings Plc., HSBC Technology Center, China
Echo: Learning from Experience Data via User-Driven Refinement
May 21, 2026Hande Dong, Xiaoyun Liang, Jiarui Yu +15Continual Learning for LLM Agents LLM Agent Self-Improvement
Tencent, CodeBuddy
Trace2Skill: Verifier-Guided Skill Evolution for Long-Context EDA Agents
May 20, 2026Zijian Du, Nathaniel PinckneyElectronic Design Automation Code Generation
NVIDIA · Santa Clara, CA, USA
GenEvolve: Self-Evolving Image Generation Agents via Tool-Orchestrated Visual Experience Distillation
May 20, 2026Sixiang Chen, Zhaohu Xing, Tian Ye +7Agentic Image Generation LLM Agent Self-Improvement
The Hong Kong University of Science and Technology (Guangzhou) · Meituan · The Hong Kong University of Science and Technology +1
Library Drift: Diagnosing and Fixing a Silent Failure Mode in Self-Evolving LLM Skill Libraries
May 19, 2026Xing Zhang, Yanwei Cui, Guanghui Wang +4Continual Learning for LLM Agents LLM Agent Self-Improvement
1AWS Generative AI Innovation Center · 2HSBC Holdings Plc., HSBC Technology Center, China.
Harnessing LLM Agents with Skill Programs
May 18, 2026Hongjun Liu, Yifei Ming, Shafiq Joty +1LLM Agent Self-Improvement LLM Agent Skill Learning
New York University · Salesforce AI Research
FORGE: Self-Evolving Agent Memory With No Weight Updates via Population Broadcast
May 15, 2026Igor Bogdanov, Chung-Horng Lung, Thomas Kunz +3LLM Agent Memory LLM Agent Self-Improvement
Carleton University Ottawa, Ontario, Canada · Defence R&D Canada Ottawa, Ontario, Canada · Cistel Technology Ottawa, Ontario, Canada
Beyond Individual Intelligence: Surveying Collaboration, Failure Attribution, and Self-Evolution in LLM-based Multi-Agent Systems
May 14, 2026Shihao Qi, Jie Ma, Rui Xing +15Multi-Agent Collaboration LLM Agent Self-Improvement
MOE KLINNS Lab · School of Cyber Science and Engineering, Xi’an Jiaotong University · School of Software Engineering, Xi’an Jiaotong University +6
SkillFlow: Flow-Driven Recursive Skill Evolution for Agentic Orchestration
May 13, 2026Mingda Zhang, Tiesunlong Shen, Haoran Luo +4LLM Agent Orchestration LLM Agent Self-Improvement
The Chinese University of Hong Kong, Shenzhen · National University of Singapore · Nanyang Technological University +1
PIVOT: Bridging Planning and Execution in LLM Agents via Trajectory Refinement
May 11, 2026Tuo Zhang, Alin-Ionut Popa, Yan Xu +2LLM Agent Self-Improvement Long-Horizon LLM Agents
Amazon
SkillEvolver: Skill Learning as a Meta-Skill
May 11, 2026Genrui Zhang, Erle Zhu, Jinfeng Zhou +2LLM Agent Self-Improvement LLM Agent Skill Learning
2Beijing Jiaotong University · 1Tsinghua University
Workspace Optimization: How to Train Your Agent
May 10, 2026Elad Sarafian, Gal Kaplun, Ron Banner +2Multi-Agent LLM Systems LLM Agent Self-Improvement
NVIDIA · NVIDIA & Technion
Kintsugi: Learning Policies by Repairing Executable Knowledge Bases
May 10, 2026Teng Cao, Yu Deng, Hikaru Shindo +6Robot Policy Learning LLM Agent Self-Improvement
Artificial Intelligence and Machine Learning Lab, Technical University of Darmstadt, Germany · Hessian Center for Artificial Intelligence (hessian.AI), Germany · Department of Computer Science, Technical University of Darmstadt, Germany +6
Do Self-Evolving Agents Forget? Capability Degradation and Preservation in Lifelong LLM Agent Adaptation
May 10, 2026Ye Yu, Xiaopeng Yuan, Haibo Jin +3Continual Learning for LLM Agents LLM Agent Self-Improvement
University of Illinois Urbana-Champaign, IL, USA
SkillGen: Verified Inference-Time Agent Skill Synthesis
May 9, 2026Yuchen Ma, Yue Huang, Han Bao +5LLM Agent Self-Improvement LLM Agent Skill Learning
Munich Center for Machine Learning, LMU Munich · University of Notre Dame · Microsoft Research
OPT-BENCH: Evaluating the Iterative Self-Optimization of LLM Agents in Large-Scale Search Spaces
May 9, 2026Xiaozhe Li, Jixuan Chen, Xinyu Fang +4Large Language Model-Guided Optimization LLM Agent Self-Improvement
Tongji University1 · Shanghai AI Lab2 · University of California San Diego5 +2
SkillMaster: Toward Autonomous Skill Mastery in LLM Agents
May 9, 2026Min Yang, Jinghua Piao, Xu Xia +4LLM Agent Self-Improvement LLM Agent Skill Learning
Shandong University · Zhongguancun Academy · Tsinghua University +2
FlashEvolve: Accelerating Agent Self-Evolution with Asynchronous Stage Orchestration
May 8, 2026Zhengding Hu, Mingge Lu, Zhen Wang +8LLM Agent Orchestration LLM Agent Workflow Optimization
University of California, San Diego · Georgia Institute of Technology

May 28, 2026J/K move · Enter open · S save

GRASP: Gated Regression-Aware Skill Proposer for Self-Improving LLM Agents

Johannes Moll, Jean-Philippe Corbeil, Jiazhen Pan, Martin Hadamitzky+3

1Technical University of Munich and TUM University Hospital · 2Microsoft Healthcare & Life Sciences

LLM Agent Self-Improvement LLM Agent Skill Learning LLM Agents LLM Agent Reliability

Open paper AskPDF arXiv

LLM agents acting in structured environments fail in operational rather than conversational ways, and reliability depends on procedural knowledge of the environment. Prior self-improvement methods accumulate natural-language guidance without checking that each new item preserves previously correct behavior, so a note that fixes one trajectory can silently regress another. We introduce GRASP (Gated Regression-Aware Skill Proposer), which treats agent improvement as a sequence of edits to a bounded skill library, admitting each candidate only if it produces a net improvement on a balanced held-out probe under a hard regression budget. We evaluate GRASP across five base models (gpt-oss-120b, DeepSeek V4 Flash, Gemini 3.1 Flash Lite, GPT-4.1, GPT-5.4) on two FHIR-based clinical benchmarks. On MedAgentBench, GRASP lifts gpt-oss-120b from 40.6% to 88.8%, exceeds the strongest of five self-improvement baselines by 21.0 points, and improves every other base model by 17.2 to 40.3 points. Ablations attribute the gain to comparative proposal generation, the acceptance gate, and the hard regression budget rather than to skill writing itself, which without validation is no better than using no skills. The mechanism generalizes beyond the clinical domain, improving agents on three of four non-clinical environments and remaining flat only where the action space is open-ended. Frozen libraries transfer across models, where skills from a stronger model improve weaker executors beyond what they learn for themselves while the reverse does not, an asymmetry that no ungated baseline reproduces.