ReCreate: Reasoning and Creating Domain Agents Driven by Experience

Zhezheng Hao; Hong Wang; Jian Luo; Jianqing Zhang; Yuyan Zhou; Qiang Lin; Can Wang; Hande Dong

cs.AIJan 16, 2026

ReCreate: Reasoning and Creating Domain Agents Driven by Experience

Authors: Zhezheng Hao, Hong Wang, Jian Luo, Jianqing Zhang, Yuyan Zhou, Qiang Lin, Can Wang, Hande Dong,

Organizations: Zhejiang University · State Key Laboratory of Blockchain and Data Security, Zhejiang University · Tencent · Independent Researcher ·

Abstract

Large Language Model (LLM) agents are reshaping the industrial landscape. However, most practical agents still rely on human-designed scaffolds, making their creation and adaptation across diverse domains labor-intensive. This situation poses a central question: can we automatically create and evolve domain agents through interaction experience in real-world environments? While recent approaches have sought to automate agent creation, several treat scaffold optimization as a black-box procedure guided primarily by final performance metrics. Such strategies overlook critical evidence explaining why an agent succeeds or fails and often incur high computational costs. To address these limitations, we propose ReCreate, an experience-driven framework for creating and evolving specialized domain agents. ReCreate systematically leverages agent interaction histories to identify the causes of success and failure and guide targeted scaffold updates. Specifically, we introduce an agent-as-optimizer paradigm in which an optimizer agent actively inspects execution experience and iteratively refines the task agent's scaffold through three key components: (i) an experience storage and retrieval mechanism for on-demand evidence inspection; (ii) a reasoning-creating synergy pipeline that translates execution evidence into updates to prompts, workflows, tools, and memory; and (iii) hierarchical updates that consolidate instance-level refinements into reusable domain-level improvements. Experiments across thirteen evaluation settings spanning four domains show that ReCreate consistently outperforms human-designed agents and existing automated agent generation and self-evolving baselines, even when evolving from minimal seed scaffolds.

Figures & tables

Figure 1: Overall performance across four domains with gpt-5-mini . Bars report domain-level average scores of three method families and the proposed ReCreate. Red annotations indicate ReCreate’s improvement over the runner-up in each domain.

Figure 3: The pipeline of ReCreate. ReCreate-Agent iteratively reasons and acts to locate key evidence on why the agent succeeds or fails and reflect how to improve the agent scaffold.

Domain	Dataset	Human-designed			Self-Evolve			Agent Generation		Ours
Domain	Dataset	CoT	SBA	Refine	LIVE	OPRO	ExpeL	ADAS	Square	ReCreate
SWE	Django	$58.29$	$58.77$	$56.87$	$58.77$	$52.13$	$59.24$	$55.92$	$57.82$	$\mathbf{60.19}$
SWE	Sympy	$61.82$	$61.82$	$58.18$	$58.18$	$54.55$	$56.36$	$58.18$	$54.55$	$\mathbf{63.64}$
DS	DW	$42.81$	$41.66$	$42.43$	$38.55$	$18.55$	$47.50$	$37.98$	$45.98$	$\mathbf{51.94}$
	ML	$34.32$	$36.25$	$35.85$	$41.68$	$39.08$	$40.27$	$21.53$	$22.90$	$\mathbf{42.88}$
	SA	$19.33$	$20.15$	$20.39$	$21.83$	$17.16$	$22.44$	$15.66$	$11.99$	$\mathbf{24.50}$

Table 1: Pass rate or testing score on various real-world agent benchmarks.

Panel of Figure 4 — Figure 4: Action distributions of the ReCreate-Agent in various domains.

Figure 5: Ablations on experience components.

Method	SWE	DA-Code	DS
ReCreate	60.19	39.77	77.74
w/o creation router	58.00	37.13	75.39
w/o DomUpd	57.09	37.83	75.96

Table 2: Action-level ablation.

Figure 6: Reasoning-level ablation.

Figure 7: Post-update Gain Verification on SWE tasks.

Figure 8: Cost Comparison.

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 9: Scaffolds gate what a base model can do.

SWE		DA-Code			DS-1000
Django	Sympy	Data Wrangling	Machine Learning	Statistical Analysis	NumPy	Pandas	Matplotlib
231	75	100	100	78	220	291	155
Math					AppWorld
AIME24	AIME25	Algebra	Number Theory	Counting&Probability	dev	Normal	Challenge
30	30	124	62	38	57	168	417

Appendix

Table 3: The counts for datasets used in our experiments.

Domain	Dataset	$t=0.0$	$t=0.5$	$t=1.0$
SWE	Django	60.66	61.14	63.51
SWE	Sympy	61.82	58.18	60.00
DS	DW	51.94	47.79	50.55
	ML	40.88	51.16	45.78
	SA	23.00	20.33	27.33
	Numpy	77.00	78.00	81.50

Appendix

Table 4: ReCreate performance with different temperature.

Figure 10: ReCreate on various base models for ReCreate Agent.

Figure 11: ReCreate on various base models for task agent.

Comparison	exact $p$	significant?
vs SBA	3.43e-11	✓
vs ExpeL	7.57e-8	✓
vs AgentSquare	9.24e-11	✓

Appendix

Table 5: Paired significance tests (ReCreate vs baselines).

Method	SWE-bench Verified
ReCreate-20	57.4
ReCreate-50	60.4
ReCreate-100	60.8

Appendix

Table 6: Changing the size of dev set in SWE tasks.

Method	SWE-bench Verified
ReCreate-dev1	60.4
ReCreate-dev2	59.8
ReCreate-dev3	59.8

Appendix

Table 7: Changing the composition of dev set in SWE tasks.

Panel of Figure 12 — Figure 12: A case study for adding rules.

Panel of Figure 13 — Figure 13: A case study for adding tools.

Panel of Figure 14 — Figure 14: A case study for enforcing workflows.

Panel of Figure 15 — Figure 15: Main prompt for ReCreate-Agent.

Panel of Figure 16 — Figure 16: Batch synthesis prompt for aggregating instance-level scaffold edits into a unified global scaffold.

Panel of Figure 17 — Figure 17: The Minimal Seed Scaffold in Django.

Panel of Figure 18 — Figure 18: The System Template created from Django experience.

Panel of Figure 19 — Figure 19: The Instance Template created from Django experience.

Panel of Figure 20 — Figure 20: The Memory Template created from Django experience.

Panel of Figure 21 — Figure 21: A snapshot of the static memory accumulated by ReCreate.

Panel of Figure 22 — Figure 22: A tool created from Django experience.

LLM Agents

Large language model agents use language models to plan, use tools, and act across multi-step tasks and environments.

See all 705 papers

Jun 9, 2026

Role-Agent: Bootstrapping LLM Agents via Dual-Role Evolution

Although Large Language Model (LLM) agents have demonstrated strong performance on complex tasks, their learning is often limited by inefficient interaction feedback and static training environments, which hinder broader generalization. To address these limitations, this paper introduces Role-Agent, \textcolor{black}{a framework} that harnesses a single LLM to function concurrently as both the agent and the environment, enabling a bootstrapped co-evolution. Role-Agent comprises two synergistic components: World-In-Agent (WIA) and Agent-In-World (AIW). In WIA, the LLM acts as the agent and predicts future states after each action; the alignment between predicted and actual states is then used as a process reward, encouraging environment-aware reasoning. In AIW, the LLM analyzes failure modes from failed trajectories and retrieves tasks with similar failure patterns, thereby reshaping the training data distribution for targeted practice. Experiments on multiple benchmarks show that Role-Agent consistently improves performance, yielding an average gain of over 4% over strong baselines.

Xucong Wang, Ziyu Ma, Shidong Yang +4

University of Science and Technology of China · AMAP, Alibaba Group

LLM Agent Self-Improvement LLM Agent Training Self-Evolving Agents

Ask PDF

Mar 25, 2026

Experiential Reflective Learning for Self-Improving LLM Agents

Recent advances in large language models (LLMs) have enabled the development of autonomous agents capable of complex reasoning and multi-step problem solving. However, these agents struggle to adapt to specialized environments and do not leverage past interactions, approaching each new task from scratch regardless of their accumulated experience. We introduce Experiential Reflective Learning (ERL), a simple self-improvement framework that enables rapid environment adaptation through experiential learning. ERL reflects on task trajectories and outcomes to generate heuristics, capturing actionable lessons that transfer across tasks. At test time, relevant heuristics are retrieved based on the current task and injected into the agent's context to guide execution. On the Gaia2 benchmark, ERL improves success rate by 7.8% over a ReAct baseline, with large gains in task completion reliability, and outperforms prior experiential learning methods. Through systematic ablations, we find that selective retrieval is essential and that heuristics provide more transferable abstractions than few-shot trajectory prompting. These results demonstrate that reflecting on single-attempt experiences to extract transferable heuristics enables effective agent self-improvement.

Marc-Antoine Allard, Arnaud Teinturier, Victor Xing +1

Illuin Technology

Self-Improving Agents LLM Agent Training Memory-Augmented Language Model Agents

Ask PDF

Mar 18, 2026

AgentFactory: A Self-Evolving Framework Through Executable Subagent Accumulation and Reuse

Building LLM-based agents has become increasingly important. Recent works on LLM-based agent self-evolution primarily record successful experiences as textual prompts or reflections, which cannot reliably guarantee efficient task re-execution in complex scenarios. We propose AgentFactory, a new self-evolution paradigm that preserves successful task solutions as executable subagent code rather than textual experience. Crucially, these subagents are continuously refined based on execution feedback, becoming increasingly robust and efficient as more tasks are encountered. Saved subagents are pure Python code with standardized documentation, enabling portability across any Python-capable system. We demonstrate that AgentFactory enables continuous capability accumulation: its library of executable subagents grows and improves over time, progressively reducing the effort required for similar tasks without manual intervention. Our implementation is open-sourced at https://github.com/zzatpku/AgentFactory, and our demonstration video is available at https://youtu.be/iKSsuAXJHW0.

Zhang Zhang, Shuqi Lu, Hongjin Qian +2

Peking University · Beijing Academy of Artificial Intelligence · Hong Kong Polytechnic University

Self-Evolving Agents LLM Agent Skill Learning Continual Learning for LLM Agents

Ask PDF

Abstract

LLM Agents

Explore similar work

Role-Agent: Bootstrapping LLM Agents via Dual-Role Evolution

Experiential Reflective Learning for Self-Improving LLM Agents

AgentFactory: A Self-Evolving Framework Through Executable Subagent Accumulation and Reuse