ReGE: Stronger Agents through Experience Synthesis

Jiarong Liang; Yubo Wang; Zijie Liu; Yuxuan Zhang; Ping Nie; Wenhu Chen

cs.AIOct 24, 2026

ReGE: Stronger Agents through Experience Synthesis

Authors: Jiarong Liang, Yubo Wang, Zijie Liu, Yuxuan Zhang, Ping Nie, Wenhu Chen

Organizations: University of Waterloo · University of North Carolina at Chapel Hill · University of British Columbia

Abstract

Repeated attempts at interactive tasks uncover facts, useful operations, and failures that can inform subsequent execution. We introduce ReGE, a framework for turning this recorded experience into guidance for a fresh attempt. Its main implementation, ReGE-Syn, uses a single coach call to jointly synthesize bounded execution records and available artifacts into self-contained guidance, connecting task-specific findings with proposed actions and relevant pitfalls. A student starts from the task's initial environment state and adapts this guidance to current observations. Source generation, synthesis, and student execution use the same base model, without parameter updates or a stronger teacher. Across four benchmarks, ReGE improves accuracy by an average of 9.9 percentage points for GPT and 8.8 percentage points for Claude compared with unguided execution. On matched BrowseComp-Plus subsets, ReGE-Syn reduces guidance-construction tokens by 98.2% on GPT and 95.5% on Claude relative to adapted AggAgent, with observed success. Combined construction and execution tokens decrease by 15.7% and 35.4%, respectively, counting input plus output and excluding source generation. BrowseComp-Plus ablations further associate access to process information with fewer student steps and show that joint synthesis produces substantially shorter guidance than Direct history or Independent summaries, with observed success. These results demonstrate how agents can use their own recorded experience to improve subsequent task completion, and suggest a low-cost way to consolidate the execution records of parallel agent systems such as agent swarms.

Figures & tables

Figure 1: With ReGE, gpt-5.6-luna matches gpt-5.6-sol at 28% of its cost. Costs include source pools; see Table 5 .

Figure 2: The joint-synthesis implementation of ReGE. The coach reads the rendered source pool; the student receives the resulting guidance and interacts with its own environment. Source generation, synthesis, and execution share the same base model.

Criterion	SC (source)	Self-MoA	Reflexion	Agentic PDR	AggAgent	ReGE-Syn
Single-pass Synthesis	$\times$	$\checkmark$	$\times$	$\times$	$\times$	$\bm{\checkmark}$
Trajectory Info.	$\times$	$\times$	$\checkmark$	$\checkmark$	$\checkmark$	$\bm{\checkmark}$
Student Rerun	$\times$	$\times$	$\checkmark$	$\checkmark$	$\times$	$\bm{\checkmark}$
Aggregation Calls	0	1	Iterative	$K$	Iterative	1

Table 1: Comparison of aggregation methods.

	Task success (%)						Cost ( $\times$ Control)
Benchmark	Control	SC	Best-of- $N$	Reflexion	AggAgent	ReGE	Control	SC	Best-of- $N$	Reflexion	AggAgent	ReGE
gpt-5.6-luna
Terminal-Bench 2.0	41.6	—	54.0	46.1	46.1	56.2	1.00	—	4.17	2.71	5.83	5.25
Spider2-Lite	59.3	64.4	63.7	64.4	55.6	66.7	1.00	4.00	4.32	1.50	11.71	5.36
AndroidWorld	53.8	—	57.9	55.3	51.5	58.8	1.00	—	4.02	2.30	4.92	5.06
BrowseComp-Plus	64.1	74.0	74.0	73.3	76.1	76.6	1.00	4.00	4.01	2.09	4.74	4.52

Table 2: Task success and source-inclusive token costs across four benchmarks. Cost combines source-generation tokens with measured construction and execution tokens, relative to Control (

\times

). — denotes unavailable or inapplicable entries. Shaded columns show our method; bold and underline mark the best and second-best success in each row.

gpt-5.6-luna
Method	Success	Construction		Execution	Total
	(%)	Tokens (k)	Saved (%)	Tokens (k)	Tokens (k)
AggAgent	75.3	232.9	—	44.8	277.7
ReGE-Syn	75.2	4.2	98.2	229.8	234.1
claude-sonnet-5
AggAgent	87.0	211.7	—	134.8	346.6

Table 3: Joint synthesis reduces guidance-construction tokens by over 95% on both models. Results on matched BrowseComp-Plus queries. Token counts are mean input plus output per query, including failures and excluding source generation. Construction savings are relative to AggAgent. See Appendix A.3 for accounting details.

Benchmark	Student output tokens	Student steps
claude-sonnet-5 ReGE-Syn / Control
Terminal-Bench 2.0	$0.86\times$	$0.88\times$
Spider2-Lite	$0.70\times$	$0.50\times$
AndroidWorld	$1.02\times$	$1.01\times$

Table 4: ReGE-Syn reduces student steps and output tokens on Terminal-Bench 2.0 and Spider2-Lite. Ratios compare ReGE-Syn with unguided Control. Only student output tokens and steps are counted, excluding input tokens, source generation, judging, and construction.

	Task success (%)					Cost per task (USD)
	gpt-5.6-				gpt-6-	gpt-5.6-luna + ReGE		gpt-5.6-		gpt-6-
Benchmark (tasks)	luna	luna + ReGE	terra	sol	luna	incl. pool	existing pool	terra	sol	luna
Terminal-Bench 2.0 (44)	40.9	63.6	59.1	68.2	34.1	0.187	0.028	0.251	0.601	0.005
Spider2-Lite (68)	66.2	77.9	66.2	63.2	66.2	0.043	0.007	0.044	0.090	0.001
AndroidWorld (56)	50.0	48.2	55.4	57.1	44.6	0.313	0.062	0.568	1.138	0.025
BrowseComp-Plus (300)	70.0	80.0	75.0	79.3	52.7	0.314	0.038	0.615	1.237	0.004

Table 5: With ReGE, gpt-5.6-luna matches gpt-5.6-terra and gpt-5.6-sol at lower cost. Task success (%) and list-price cost per task (USD) on the task subset of § 4.2 . The other models run once per task without guidance; the cost of ReGE is shown including source-pool generation and with an existing source pool. Shaded columns show our method. Bold marks the best value in each row, and underline marks the second best.

Configuration	Control No coach	Closed-book $K^{\prime}=0$	ReGE-Syn $K^{\prime}=1$	ReGE-Syn $K^{\prime}=2$	ReGE-Syn $K^{\prime}=4$
Task success (%)	74.7	75.7	84.3	86.3	88.3

Table 6: Task success increases as the coach receives more source attempts. Control has no coach; Closed-book uses the coach with zero source attempts. Coached configurations share the same coach, synthesis request, delivery header, and requested output budget;

K^{\prime}=1,2,4

expose prefixes of the same ordered source pools.

Overall performance
Metric / source-answer group	FAO	ReGE-Syn
Task success (%) $\uparrow$	85.0	88.3
Mean student steps $\downarrow$	14.6	9.1
Task success (%), grouped by submitted source answers
0 source answers (10.0% of queries)	10.0	6.7
1 source answer (6.3% of queries)	47.4	94.7

Table 7: Adding process records yields higher task success with fewer student steps. FAO uses final artifacts only; ReGE-Syn additionally receives process records. Both conditions share the source pools, coach, plan request, and student protocol.

		gpt-5.6-luna			claude-sonnet-5
Benchmark	Source-pool condition	Tasks (%)	Control	ReGE-Syn	Tasks (%)	Control	ReGE-Syn
Terminal-Bench 2.0	All failed	36.0	0.0	9.4	52.8	12.8	14.9
	Mixed	41.6	51.4	73.0	25.8	52.2	65.2
	All succeeded	22.5	90.0	100.0	21.3	94.7	100.0
Spider2-Lite	All failed	24.4	0.0	0.0	26.7	2.8	0.0
	Mixed	32.6	52.3	72.7	25.9	48.6	62.9

Table 8: ReGE-Syn achieves its largest observed success-rate gains over Control on mixed source pools. Tasks are grouped by whether none, some, or all source attempts succeed. Entries report fresh student success rates in percent; Tasks (%) is the percentage of the benchmark’s tasks that fall in each group. Source correctness labels are used only for analysis and are unavailable to the coach and student.

Method	Success (%) $\uparrow$	Guidance length (characters) $\downarrow$	Total tokens $\downarrow$
Direct history	88.3	20,571	268.9k
Independent summaries	86.9	19,590	316.5k
ReGE-Syn	88.3	5,063	215.8k

Table 9: Joint synthesis reduces guidance length and total token usage while achieving observed success. Total tokens sum construction and student input plus output, including cache reads/writes and construction retries, and exclude source generation.

claude-sonnet-5 BrowseComp-Plus
Variant	Guidance form	Success (%) $\uparrow$	Guidance construction $\downarrow$	Student execution $\downarrow$	Total $\downarrow$
ReGE-Sel	Selected trajectory	89.3	$0.86\times$	$0.79\times$	$0.80\times$
ReGE-A	Explanation + answer	90.7	$\mathbf{0.85}\times$	$\mathbf{0.67}\times$	$\mathbf{0.68}\times$
ReGE-Syn	Execution plan	88.3	$1.00\times$	$1.00\times$	$1.00\times$

Table 10: Explanation-and-answer guidance lowers token usage with higher observed task success. Total is construction plus student execution, excluding source pool generation.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Component	Output-token request	Scope / evidence
ReGE-Syn , ReGE-A , Closed-book	10,000 per call	Shared construction implementation.
ReGE-Sel judge	2,000 / 8,000 / 20,000	Attempts 1–2 / 3–4 / 5–8, respectively.
AggAgent	10,000 per round	Default limit of 100 aggregation rounds.
Independent summaries	2,500 per source	Matched PDR adaptation; Appendix A.2 .
Claude Terminal-Bench 2.0 student	12,000 per call	30 agent turns
GPT Terminal-Bench 2.0 student	12,000 per call	30 agent turns

Appendix

Table 11: Requested generation limits and available serving evidence.

Benchmark	GPT coverage (%)	Claude coverage (%)
Terminal-Bench 2.0	64.0	47.2
Spider2-Lite	75.6	73.3
AndroidWorld	64.0	69.3
BrowseComp-Plus	76.1	85.9

Appendix

Table 12: Source-pool coverage.

gpt-5.6-luna
Benchmark	Tasks (%)	Control	AggAgent	ReGE-Sel	ReGE-Syn
Terminal-Bench 2.0	52.8	68.1	78.7	97.9	89.4
Spider2-Lite	63.7	89.5	79.1	95.3	96.5
AndroidWorld	58.8	82.1	76.6	72.6	87.6
BrowseComp-Plus	74.0	83.9	98.4	100.0	98.7
claude-sonnet-5

Appendix

Table 13: Fresh student success given a successful selected source. Tasks (%) is the percentage of the benchmark’s tasks on which the judge selects a successful source; the other columns report fresh student success rates in percent.

Metric	Control	ReGE-Syn
Mean success (%)	48.6	63.8
Tasks solved on all three executions (%)	25.7	60.0
Tasks solved on at least one execution (%)	71.4	65.7

Appendix

Table 14: Three claude-sonnet-5 student executions on 35 mixed-pool Spider2-Lite tasks.

Check	Recorded evidence
Database initial state	Initial database and log fingerprints match across the source attempts and student; source-created backups and recovered outputs are not inherited.
Browser task instance	Source and student task parameters, both random seeds, and the 22-step budget match. The collector resets and initializes each task. Saved summaries record the opening sequence, product entry, and success message.
Record linkage	Student records lack an injected-guidance hash; linkage uses saved guides, construction/execution logs, and collector configurations.

Appendix

Table 15: Record checks for the main-text execution cases.

LLM Agent Self-Improvement

Large language model agent self-improvement studies how agents use experience and feedback to improve their behavior, tools, and workflows.

See all 277 papers

May 9, 2026

SkillGen: Verified Inference-Time Agent Skill Synthesis

Skills are a promising way to improve LLM agent capabilities without retraining, while keeping the added procedure reusable and controllable. However, high-quality skills are still largely written by hand. We introduce SkillGen, a multi-agent framework that synthesizes a single auditable skill from trajectories generated by a base agent. The output is a human-readable artifact that can be inspected before use. Rather than merely summarizing trajectories, SkillGen leverages contrastive induction over both successful and failed trajectories to identify reusable success patterns, recurring failure modes, and behaviors that appear in nearby successes but are missing from failures. SkillGen then generates candidate skills and iteratively refines the skill. A key novelty in SkillGen is that we model agent skills as interventions to empirically verify the net effect of skills on the overall performance. Specifically, we compare outcomes on the same instances with and without the skill, so that we account for both repairs (cases where the skill fixes a baseline failure) and regressions (cases where the skill breaks a baseline success). Across a broad range of agents and datasets, SkillGen consistently improves held-out performance, outperforms existing skill-generation baselines, and produces skills that transfer across models.

Yuchen Ma, Yue Huang, Han Bao +5

Munich Center for Machine Learning, LMU Munich · University of Notre Dame · Microsoft Research

LLM Agent Self-Improvement LLM Agent Skill Learning LLM Agents

Ask PDF

Jul 18, 2026

AgentBrew: Lifelong Knowledge Brewing from Strong Teachers to Weak LLM Agents

Deploying LLM agents typically requires a compact test-time student, even if a stronger teacher is available during training. We study knowledge brewing: distilling a teacher's interactive experience into a persistent external memory for the student. Crucially, this requires no weight updates, expert demonstrations, ground-truth labels, or test-time teacher access. This setting poses two challenges: environments provide only sparse, binary feedback, and teacher-authored notes must be inherently tailored to be concretely executable by a substantially weaker student. To address these hurdles, we propose AgentBrew, comprising two coupled components. First, a failure-triggered teacher--Ralph Loop mitigates sparse feedback by transforming student failures into environment-validated notes. Second, student-aware synthesis calibrates teacher knowledge to the weak executor's operational granularity, yielding model-specific, actionable guidance. Extensive evaluations and comprehensive ablations across coding, math, and tool-use tasks demonstrate that this asymmetric, training-free brewing paradigm produces highly capable yet deployable LLM agents.

Yangqin Jiang, Chao Huang

The University of Hong Kong

LLM Agent Self-Improvement LLM Agent Skill Learning Knowledge Distillation Memory-Augmented Language Model Agents

Ask PDF

Sep 27, 2026

CORTEX: A Verified Experience Layer for Generalist Agents

An agent can solve a task today and face the same task under new facts, tools, or governing knowledge tomorrow. Most agent systems can retrieve relevant text or recall prior conversations, but they lack a principled way to decide when a previous solution is still valid, when it must be adapted, and when it should be discarded. We introduce CORTEX (Contextual Orchestration and Reuse of Task EXperience), a general AI systems framework that connects specialized agents through an external layer of verified experience. Each episode records its task conditions, source and tool state, decisive predicates, proof trace, verifier, and outcome. A meta-controller chooses exact replay, checked adaptation, fresh synthesis, or escalation. Accepted episodes can become task patterns and procedural strategies through a challenge-driven development loop. This gives the system an implicit competence layer that can grow without changing model weights. We formalize system contracts for exact replay and source-version separation, and derive when reuse saves computation. A controlled two-domain implementation tests the exact-replay core on 1,000 synthetic cases. Complete-family holdouts test procedural transfer on 1,000 new-family cases across eight clinical and policy splits, with complete fresh-evidence grounding and perfect invariance to irrelevant-field and insertion-order perturbations. The transfer trace exposes the work required for verified strategy execution. These results establish an initial path toward general intelligence through reusable procedures, typed experience, and developmental transfer.

Garapati Keerthana, Manik Gupta

Birla Institute of Technology and Science, Pilani, Hyderabad, India

AI Agent Reliability Lifelong Learning Agents Experience Replay Agent Skill Learning Memory-Augmented Agents

Ask PDF

Abstract

LLM Agent Self-Improvement

Explore similar work

SkillGen: Verified Inference-Time Agent Skill Synthesis

AgentBrew: Lifelong Knowledge Brewing from Strong Teachers to Weak LLM Agents

CORTEX: A Verified Experience Layer for Generalist Agents