Drafting Agents, Not Just Tokens: Speculative Recursive Self-Improvement for Agentic Systems

Xin Xu; Ke Cheng; Hao Wang; Yangru Huang; Yanghao Xiao; Zhichao Chen; Zhisheng Zhong; Xun Liu

cs.LGOct 12, 2026

Drafting Agents, Not Just Tokens: Speculative Recursive Self-Improvement for Agentic Systems

Authors: Xin Xu, Ke Cheng, Hao Wang, Yangru Huang, Yanghao Xiao, Zhichao Chen, Zhisheng Zhong, Xun Liu,

Organizations: Tencent AMS, Beijing, China · State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University, Beijing, China · CCSE lab, Beihang University. · Xiaohongshu ·

Abstract

Improving an agent through successive revisions requires balancing exploration and evaluation under a limited budget. A narrow search can miss useful sequences of edits, while fully evaluating every intermediate version can exhaust the budget before those sequences yield a verified improvement. We introduce Speculative Recursive Self-Improvement (SpecRSI), which drafts executable successor agents before their predecessors are fully evaluated, while requiring complete verification before adoption. Verification-Aware Tree Search (VATS) allocates the budget between further edits and outstanding checks using the predicted verified return of a continuation. Group Verification Optimization (GVO) learns the improver from completed searches while keeping the real evaluator fixed. In the main scientific comparison, terminal gains exceed Janus by 33.3%--40.0%. Additional evaluations show that SpecRSI outperforms Janus in all 50 scientific backbone--task pairs and exceeds MetaRSI-v1 by 1.85%--2.38% in macro-average system score across twenty backbones. Coding-validation success reaches 69.4% versus HGM's 64.3% at a 20.8% lower target-cost index.

Figures & tables

Panel of Figure 1 — Figure 1: Explore ahead, verify before adoption. Agent patterns (a–c) and their token-level counterparts (d–f). Agent checks are external and paid; only the adopted version needs complete, admissible evidence.

Panel of Figure 3 — Figure 3: Two-level Speculative RSI Loop. VATS allocates drafts and checks under a frozen improver; Accept returns the best complete admissible version. GVO updates the improver between searches; $\Gamma$ stays fixed.

Method	PyBaMM	AQM	FDTD	Reactor	IRP
Alg.-Only	0.12±0.01 [2pt]7.3±0.4	0.09±0.01 [2pt]5.1±0.3	0.14±0.01 [2pt]8.5±0.5	0.11±0.01 [2pt]6.7±0.3	0.10±0.01 [2pt]5.8±0.2
Direct-LLM	0.21±0.01 [2pt]19.2±0.6	0.17±0.01 [2pt]14.3±0.5	0.24±0.02 [2pt]22.6±0.7	0.19±0.01 [2pt]16.2±0.4	0.18±0.01 [2pt]13.7±0.4
Embed-GP	0.30±0.02 [2pt]30.5±0.8	0.25±0.01 [2pt]24.8±0.7	0.33±0.02 [2pt]36.2±0.9	0.27±0.01 [2pt]27.4±0.6	0.25±0.01 [2pt]23.1±0.5
Janus	0.36±0.02 [2pt]42.8±1.0	0.30±0.02 [2pt]34.7±0.8	0.39±0.02 [2pt]51.3±1.1	0.32±0.02 [2pt]36.9±0.8	0.30±0.01 [2pt]31.5±0.7
SpecRSI (ours)	0.48 ±0.02 [2pt] 56.7 ±1.2	0.41 ±0.02 [2pt] 45.8 ±1.0	0.52 ±0.02 [2pt] 66.9 ±1.3	0.44 ±0.02 [2pt] 49.3 ±1.0	0.42 ±0.02 [2pt] 42.1 ±0.9

Table 1: Performance across scientific discovery and agent improvement. A: terminal gain (upper entry) and call-axis AUBC (lower entry), in native units. B: task scores (%) within matched edit permissions. Entries show mean

\pm 1.0\times

SD; B uses SD. Bold denotes the highest mean within each comparable group.

Panel of Figure 4 — Figure 4: Verification scope and planning ablations. (a) Gain versus total-cost index for one check or bundles at fixed Summary or Full-state information; bars show reported SD. (b) Verified gain and (c) total-cost index across eight budget caps. In (b), means average five reported seed means; SD averages within-seed SDs. Lines join means; bands/bars show $\pm 1$ SD.

Start	Update	Gain
		(pp)
Reset	Frozen	$4.1\pm 0.18$
	SFT only	$4.6\pm 0.19$
	GVO only	$5.0\pm 0.20$
	All roles	$\mathbf{5.2\pm 0.21}$
Carry	Frozen	$4.8\pm 0.20$

Table 2: Learning the improver. Gains (pp). (a) Reported uncertainty; compare within each starting condition. (b) Updated roles: D, drafter; Sc, scorer; P, policy; Sh, shared backbone. Sci.: Science. Last two rows have unmatched training costs. (c) Incremental (Inc.)/cumulative (Cum.) gains for original (Orig.)/updated (Upd.) improvers. All carry the agent; round 1 initializes. Full controls: Appendix D .

Plans	Gain (pp)	Overhead	Missing (%)	Net
8	$5.2\pm 0.19$	$\mathbf{1.8}$	8.6	3.9
16	$5.8\pm 0.21$	2.7	5.2	4.4
32	$6.3\pm 0.25$	3.9	3.1	$\mathbf{4.7}$
64	$\mathbf{6.5}\pm 0.24$	5.6	$\mathbf{2.0}$	4.5

Table 3: Planning capacity and lifecycle cost. (a) MENU varies the offered-plan cap; gain is in pp with reported SD, and Missing is the percentage of omitted good options. Overhead and Net retain their reported definitions; Net is not recomputed as Gain minus Overhead. (b) Reported lifecycle indices; overlapping components are not independently additive.

Appendix figures & tables40 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 8 — Figure S1: Main paper and Appendix.

Janus	SpecRSI (ours)
LLM program generator	Drafter / DRAFT : propose executable edits
Target program / offspring	Immutable successor agent $c^{\prime}$
Proxy evaluator	Scorer: learned score evaluator $\widehat{Q}_{\phi}(S,\omega)$
Promotion / acquisition rule	VATS Solver + Policy: allocate draft and check work
Real evaluator $f_{\rm real}$	$\Gamma$ + $\operatorname{Acquire}_{\Gamma}$ ; invoked by VERIFY
Real-validated incumbent	ACCEPT : best complete admissible version

Appendix

Table S1: Functional correspondence between Janus and SpecRSI. The Scorer predicts continuation value; the real evaluator supplies evidence for adoption.

Category	Alg.-Only	Direct-LLM	Embed-GP	Janus	SpecRSI (ours)
Evaluation	$\mathbf{100.0}$	$\mathbf{100.0}$	$\mathbf{100.0}$	$\mathbf{100.0}$	$\mathbf{100.0}$
Proposals	$\mathbf{0}$	$12.4$	18.7	22.3	29.5
Proxy training	$\mathbf{0}$	$\mathbf{0}$	$15.2$	18.6	20.7
Planning	$\mathbf{0}$	$\mathbf{0}$	$\mathbf{0}$	$7.2$	8.5
Total	$\mathbf{100.0}$	$112.4$	133.9	148.1	158.7

Appendix

Table S2: Scientific cost breakdown. Normalized source indices, not monetary lifecycle totals.

Operators	Gain	Relative gain	Relative cost
D only	2.8	21.4%	$\mathbf{32.5}$ %
H only	6.6	50.4%	$58.7$ %
M only	3.9	29.8%	65.2%
D+H	7.7	58.8%	67.3%
D+M	5.2	39.7%	72.1%
H+M	8.9	67.9%	84.6%

Appendix

Table S3: Operator combinations. Prior deployment-profile study; absolute gains (

\uparrow

) and cost ratios (

\downarrow

Panel of Figure 12 — Figure S2: Performance across backbones. (a) Relative terminal-gain and AUBC uplift over Janus across ten backbones and five scientific tasks. (b) Macro-average scores over TB2.1, SWE-Pro, GPQA-hard100, and AIME across twenty backbones, ordered by Initial Agent mean. Within-backbone segments connect the three methods; markers and horizontal error bars show means and $\pm 1$ sample SD over five training seeds. The right-hand difference panel shows seed-paired SpecRSI $-$ MetaRSI-v1 differences in percentage points with the same uncertainty convention.

Panel of Figure 13 — Figure S3: Performance across code, prompt, and workflow tasks. Reported means: Score, Succ., and Comp. in percent; Gain in pp; Tok. in thousands. Darker cells indicate better within-metric performance; dashes mark unreported values. Code completion uses a separate profile, and costs retain task-specific definitions. Beam: Beam-Spec; FE-A/FE-O: FlashEvolve async/optional; M2/M2–D: MIPROv2 with/without demos; Tr.: Trace; TG: TextGrad; G+M: GEPA+Merge; Gen./G+S: Genesis/Genesis+SOP.

Method	Complete (%)
native FlashEvolve async-stages	91.2
native FlashEvolve optional-spec	$92.8$
SpecRSI (ours)	$\mathbf{94.7}$

Appendix

Table S4: Asynchronous completion. Completion rate supplements the coding-validation scores; it is not another estimate of coding success. The two native FlashEvolve modes remain separate.

Method	Audit score (%)	Target hits
SpecRSI (ours)	$69.4\pm 1.3$	23/25

Appendix

Table S5: Separately reported frozen-successor audit.

Method	Score (%)	Gain (%, abs.)
MIPROv2	74.8	+3.6
MIPROv2 (no demos)	73.5	+2.3
Trace	75.9	+4.7
TextGrad	76.5	+5.3
GEPA	77.6	+6.4
GEPA+Merge	$78.5$	$+7.3$

Appendix

Table S6: Prompt optimization. GPQA-hard100 means; no reported SD. Gain is an absolute change on the % score scale.

Method	Success (%)	Task tokens
Genesis	51.3	11260
Genesis+SOP	57.2	11080
AFlow	59.6	10720
ADAS	61.1	10490
EMAS	$62.7$	$10210$
SpecRSI (ours)	$\mathbf{67.8}$	$\mathbf{9290}$

Appendix

Table S7: Workflow optimization on PlanBench. Success is a task score; tokens count task inference only, excluding search and training. Both columns retain the supplied means.

Method	Math	MBPP	Game24
SpecRSI (ours)	62.3 / 10390	59.4 / 11020	54.7 / 12080

Appendix

Table S8: Additional workflow tasks. Success (%) / serving tokens; PlanBench is already in Table S7 .

Information	Work scope	Gain (pp) $\uparrow$	Cost $\downarrow$	Fail (%) $\downarrow$	Hit (%) $\uparrow$
Summary	One check	$3.2\pm 0.21$	67.4	15.2	72.0
Summary	Bundle	$4.1\pm 0.24$	65.8	12.7	78.0
Full state	One check	$4.8\pm 0.25$	63.1	9.8	84.0
Full state	Bundle	$\mathbf{6.3}\pm 0.26$	$\mathbf{59.2}$	$\mathbf{6.5}$	$\mathbf{92.0}$

Appendix

Table S9: Verification scope with information held fixed. Gain is in percentage points (pp), with reported SD. Cost is the study’s total-cost index; failure and hit rates retain the study’s definitions. The two full-state rows isolate work scope at a fixed information level.

Figure S4: Paired gain differences and planning time. (a) Method minus Full SpecRSI gain uses the separately reported paired-seed difference means and SDs, not differences recomputed from Figure 4 b. Full relative to itself is zero. Lines join means; bars/bands show

\pm 1

SD. (b) Color encodes planning-time mean; cells report mean

\pm

SD in milliseconds. Methods and eight budget caps match Figure 4 ; Full (ours) denotes Full SpecRSI (ours). Seeds only: completion seeds only; No seeds: no completion seeds; Frozen policy: refinement/frozen policy; Coupled: coupled scorer/policy; Greedy: greedy scorer selection.

Panel of Figure 21 — Figure S5: What changes when planning components are removed? These are the original reduced- and standard-budget panels, not four experiments. A/B show raw gains and each cohort’s difference from its own full method; C shows planning milliseconds and total-cost indices. In D, fin. is finalist completion (%), chk. checks per block, dep. draft depth, and cand. candidates. A larger work count is not intrinsically better. Except for B, shading is within-column scaling; darker does not always mean better. The expanded eight-budget gain/cost curves and SDs are in Figure 4 ; paired differences and planning times are in Figure S4 .

Condition	Gain (%, abs.)	Draft depth	Checks/block
Allocator blind	$3.2\pm 0.17$	1.1	1.1
Mask only	$4.7\pm 0.20$	1.6	1.6
No relations	$3.8\pm 0.18$	1.4	1.3
Restored valid fields	$5.8\pm 0.23$	2.3	2.5
Full evidence	$\mathbf{6.3}\pm 0.25$	2.6	2.8

Appendix

Table S10: Evidence and selected work. Gain

\pm

SD; depth and checks per block are observed means, not optimization targets.

Panel of Figure 23 — Figure S6: What information changes verification allocation? All panels describe the evidence-input study. C reports valid-field fraction, relation coverage, and observed blocks. D min–max rescales the same summaries to $[0,1]$ , reversing cost only; it is not another evaluation. Dep. means draft depth, chk. checks per block, and cand. candidates. Source gain SDs remain in Table S10 .

Predictor	MAE	NDCG @3	Regret @3	Fit cost
GP	$0.087$	$0.782$	$0.124$	12.3
RF	0.102	0.735	0.156	8.7
MLP	0.095	0.758	0.141	10.5
Quadratic	0.114	0.703	0.172	7.2
RBF-SVR	0.091	0.771	0.133	13.8
Direct LLM	0.136	0.654	0.208	$5.3$

Appendix

Table S11: Predictive-model diagnostics. Lower MAE, regret and fitting cost are preferred; higher NDCG is preferred. The historical row name “evolved evaluator” denotes a candidate predictor, not the protected Verification Contract.

Panel of Figure 25 — Figure S7: Starting-artifact controls and updated roles. (a) Reset, carry, and same-artifact controls; gain is in pp. Reported uncertainty, regression values, and cost/hit/failure shading are retained. Compare rows within each starting condition. (b) Filled switches mark updated roles: D, drafter; Sc, scorer; P, policy; Sh, shared backbone. Columns report Science (Sci.), Code, and Agent gains (pp). The final two rows use a separate shared-backbone extension; training costs are unmatched.

Improver	New-task gain (pp)	Cost index	Regressions
Original	5.7	54.2	0.1
Updated	$\mathbf{6.9}$	$\mathbf{50.8}$	$\mathbf{0.0}$

Appendix

Table S12: Continuation from the same released artifact. Released-fork means and costs.

Condition	Gain (pp) $\uparrow$	RMSE $\downarrow$	Training cost $\downarrow$
Same arms / means	$5.4\pm 0.23$	1.05	23.7
Same arms / means + pair	$\mathbf{6.3}\pm 0.25$	$\mathbf{0.85}$	28.4
Equal-acquisition-cost search	$5.1\pm 0.22$	1.17	25.3

Appendix

Table S13: Learning signal versus data acquisition. The first two rows use the reported same acquired arms and differ in loss. The third uses ordinary-search acquisition at the reported equal acquisition cost. Gain follows the source Gain field.

Acquisition	Paid runs	Data cost	Reset gain	State relation
Ordinary searches	24.6	1.0x	4.2	Independent
Extra ordinary equal-cost	24.6	1.0x	4.5	Independent
Same arms / means	24.6	1.0x	$4.8$	Paired
Same arms / means + pair	24.6	1.0x	$\mathbf{5.3}$	Paired
Different policy snapshot	24.6	1.0x	4.0	Mismatched

Appendix

Table S14: Continuation acquisition. Paid runs are reported averages; equal-data and equal-cost controls answer different questions.

Panel of Figure 29 — Figure S8: Which contrast is being read in the recursive-round study? A/B subtract original from updated within a method; C/D subtract Meta from Spec (ours) at the same improver status. All values are differences of Table 2 (c)’s means, not new runs. Units are percentage points (pp).

Panel of Figure 30 — Figure S9: Repetition and protocol for the recursive-round results. Each row in A is a supplied seed mean, not an individual search. B declares carried roots in all four conditions, initialization at round 1, evaluated rounds 2–5, five training seeds, and three searches per seed.

Menu	Forecast	Plan	Work	Final
Singleton only	3.8	$\mathbf{1.2}$	$54.7$	3.7
Seeds	5.2	$2.8$	55.3	5.1
Seeds + growth	6.1	3.9	54.8	$5.9$
Larger menu	$6.4$	5.6	55.1	$\mathbf{6.0}$
Exhaustive	$\mathbf{6.7}$	8.9	$\mathbf{54.6}$	5.8

Appendix

Table S15: Menu construction and planning cost. Comparison of singleton-only menus, completion seeds, seeded growth, larger menus, and exhaustive construction. This study varies the construction strategy separately from the menu-cap sweep in Table 3 (a).

Panel of Figure 32 — Figure S10: More planning is useful only if its options repay its cost. Menu-cap and menu-construction results are separate studies. In (a), over is reported overhead, miss the missing-option percentage, and net the reported net return; the outline marks the largest tested net. In (b), fore and final denote Forecast and Final; (c) shows Final minus Forecast. Comp. denotes completion, cand. candidates, chk. checks, dep. depth, and $n$ retained searches. Darker shading indicates a larger value within each column.

Regime	Batch	Serial	$\Delta$
Dependent fail	$\mathbf{10.2}$	$12.3$	-2.1
Independent fail	$\mathbf{11.5}$	$12.3$	-0.8
Cheap dispatch	$13.2$	$\mathbf{12.0}$	+1.2
Costly dispatch	$15.4$	$\mathbf{12.0}$	+3.4
Early cancellation	$\mathbf{10.8}$	$12.3$	-1.5
Reinvestment	$14.7$	$\mathbf{12.0}$	+2.7

Appendix

Table S16: Batch versus serial verification.

\Delta=C_{\mathrm{batch}}-C_{\mathrm{serial}}

; negative values favor batching. Comparisons are within each row; the six configurations do not constitute a controlled single-factor sweep.

Panel of Figure 34 — Figure S11: Cost trade-offs support adaptive verification scope. (a) Batch and serial cost components and execution conditions. (b) Mean batch-minus-serial cost. Batching lowers mean verification cost by 6.5%–17.1% in three configurations, while serial execution is cheaper in the other three, supporting VATS’s state-dependent choice between singleton checks and completion bundles. SD.b/SD.s denote the batch/serial cost SDs; stems show mean differences. Dashes denote unreported values.

$w$	Gain (%)	Time (s)	Work (s)
1	$6.3\pm 0.25$	1287	$\mathbf{1287}$
2	$6.4\pm 0.24$	672	$1312$
4	$6.4\pm 0.26$	$354$	1348
8	$\mathbf{6.5}\pm 0.25$	$\mathbf{192}$	1403

Appendix

Table S17: Worker configurations. Time is elapsed wall-clock time; Work is total worker time. Eight workers achieve a

6.7\times

speedup with 9.0% more total work.

Condition	$\Delta J$	Lower	Upper	Cost
Baseline	0.0	-	-	$1.00$
No penalty	$\mathbf{+2.4}$	+1.8	+3.0	1.03
Standard	$+2.1$	+1.6	+2.6	$1.00$
Heavy	+1.7	+1.2	+2.2	$\mathbf{0.98}$

Appendix

Table S18: Regression-penalty sensitivity. Lower and Upper retain the supplied interval endpoints. Stronger penalties reduce normalized cost while retaining positive objective improvement.

Procedure	Returns	Changes and calls
Algorithm 1 : search	Complete agent and log	Updates versions, evidence and budget; calls Algorithm 2
Algorithm 2 : VATS	Feasible selected plan	Settles preparation; fixed feasibility rules and frozen scorer/policy
Algorithm 3 : GVO	Updated improver	Calls Algorithm 1 ; updates only after collection

Appendix

Table S19: Two timescales, three procedures. Only the outer training procedure updates the improver’s parameters.

Object	Permitted change	Update boundary
Task agent $c_{k}$	Prompts, tool code, workflow and scoped persistent memory	Executed edit creates an immutable successor
Task-model snapshot	Weights only when M is allowed	D/H/M profile; absent from direct fixed-weight core
Improver $\Theta_{k}$	Drafter, continuation value, plan-selection parameters	GVO after complete searches; frozen inside each search
Verification contract $\Gamma$	None	Fixed tasks, admission and scoring rules
Outer update rule	None	GVO algorithm fixed; learned parameters may change

Appendix

Table S20: What changes, and when? A legal search need not edit every task surface in one round.

Symbol	Meaning
$\Gamma,\mathcal{I},J_{\Gamma}$	Fixed verification contract, required items, and score aggregation
$\mathcal{T}=(\mathcal{C},\mathcal{A})$	Actual immutable versions and executed parent–child edit edges
$\mathcal{E},H$	Admitted outcome records and observed execution history
$m_{ci},U_{c},K_{c}$	Outcome completeness bit, missing items, and candidate–item target keys
$\mathcal{C}_{\mathrm{full}},F$	Complete admissible versions and their best improvement over the initial root
$B,N,w$	Remaining expense, primitive-attempt allowance, and worker count

Appendix

Table S21: The state separates versions, observations, and predictions. These objects are shared by all three algorithms.

Figure S12: Observed results, planned work, and predictions have different roles. Planning may use missingness and source proposals; only contracted direct execution or sound scoped equality adds a valid result. Observed does not mean passed; a missing item blocks adoption but not further editing.

Candidate work	Required information	Role
One edit or check	Existing parent or item; expense and attempt caps	Feasible fallback
Completion bundle	All missing keys; dependency and setup closure	Preserve joint options
Refined bundle	Same proposal state and frozen value model	Compare continuations
Selected plan	Paid state, canonical menu, policy preference	Execute; then replan

Appendix

Table S22: What the solver must preserve at one decision. These are executable constraints, not learned predictions of correctness.

Figure S13: Two data paths train different quantities. Ordinary complete searches provide actor likelihoods and return labels. Paid forced continuations provide critic labels only. The next epoch freezes the updated improver; the verification contract

\Gamma

stays unchanged.

Reader question	Result to read	Essential condition
Can predictions corrupt a completed result?	Outcome consistency (Proposition 1 )	Sound records; fixed contract
Does grouping enlarge every policy class?	Singleton simulation (Proposition 4 )	Free serial decision boundaries
When does commitment save expense?	Feedback threshold (Proposition 5 )	Identical output and charged blocks
What do success summaries omit?	Scope reversal and matched control (Propositions 7 , 8 )	Equal marginals/all-pass rate; distinct joint laws
How can an approximate menu lose value?	Menu bound (Proposition 9 )	Fixed continuation; bounded value error

Appendix

Table S23: What each formal result establishes. The assumptions belong to the reference model; these results are not neural benchmark measurements.

Figure S14: Early feedback changes the cost of committing work. Exact sensitivity of Equation ( 24 ), with dispatch 1 and planning 0.2. These curves do not fit the batch-versus-serial study or assert that batching always wins.

Outcome	A	B
000	0	3/8
001	1/4	0
010	1/4	0
011	0	1/8
100	1/4	0
101	0	1/8

Appendix

Table S24: Exact laws and complete scheduling costs. Each bit is a required check outcome. The left panel lists all probability atoms; the right lists all surviving-path block compositions at

c=1

d=3/2

. Both contexts have marginal pass probability

1/2

and all-pass probability

1/4

. This is one finite construction, not two experimental datasets.

Panel of Figure 46 — Figure S15: The optimal scope depends on commitment costs. Both plots evaluate the stated analytical costs, not fitted outcomes. The one-dimensional slice fixes dispatch at $3/2$ ; the phase map varies dispatch and pair-survival probability. Boundaries and ties follow the declared formulas.

Term	Full form or published name
Recursive Self-improvement
RSI	Recursive Self-Improvement.
SpecRSI (ours)	Speculative Recursive Self-Improvement (this work; Section 3 ).
VATS	Verification-Aware Tree Search (this work; Section 3.3 ).
GVO	Group Verification Optimization (this work; Section 3.4 ).
DGM	Darwin Gödel Machine ( Zhang et al., 2026a ) .

Appendix

Table S25: Abbreviations and terminology.

LLM Agent Self-Improvement

Large language model agent self-improvement studies how agents use experience and feedback to improve their behavior, tools, and workflows.

See all 273 papers

RRSI: Regularized Recursive Self-Improvement of Agent Harnesses
Sep 21, 2026Peng Xia, Rujun Han, Zifeng Wang +11OOD Generalization Agent Harness Optimization
Google Cloud AI Research · UNC-Chapel Hill · Stanford University +1
RSIAgent: Autonomous Exploration for Recursive Self-improvement in New Environments
Sep 14, 2026Sibo Zhu, Shicheng Fan, Xinyue Wang +3Multi-Agent LLM Systems LLM Agent Self-Improvement
Aether AI · University of California San Diego · ‡Work done during internship in Aether AI +1
The Red Queen Gödel Machine: Co-Evolving Agents and Their Evaluators
Jun 24, 2026Alex Iacob, Andrej Jovanović, William F. Shen +12Evolutionary Optimization AI Agent Evaluation
University of Cambridge · Flower Labs · NVIDIA +2

Sep 21, 2026J/K move · Enter open · S save

RRSI: Regularized Recursive Self-Improvement of Agent Harnesses

Peng Xia, Rujun Han, Zifeng Wang, Yanfei Chen+10

Google Cloud AI Research · UNC-Chapel Hill · Stanford University+1

OOD Generalization Agent Harness Optimization LLM Agent Self-Improvement Recursive Self-Improvement

Open paper AskPDF arXiv

An LLM agent's capability is largely magnified by its harness, namely the prompts, control flow, tooling, memory, and context management surrounding the frozen backbone model. Recent methods increasingly automate this process by iteratively proposing and selecting component-wise edits of an agent harness, practically establishing a form of recursive self-improvement (RSI) at the agent-system level. However, such recursive evolution may overfit by memorizing the training tasks, showing large in-distribution gains that shrink or even vanish on out-of-distribution benchmarks. We introduce Regularized Recursive Self-Improvement of Agent Harnesses (RRSI), which incorporates the principles of regularizations into harness self-improvement by constraining the evolution candidate proposal and selection. The proposer operates with a temporally annealed budget, limiting how many edits a candidate can bundle, and it encourages unexplored trajectories based on evolution history. The selector is equipped with a critic and a pruner: the critic screens benchmark-specific proposals, while the pruner, removes changes that are too small, too expensive, or no longer useful. Together these constraints favor reusable agent mechanisms over benchmark-specific ones or even noises. Across eight benchmarks spanning coding, agentic workspace and engineering design tasks, RRSI gains up to 6.0 points on the split it evolves against and up to 4.3 points on the five out-of-distribution benchmarks, while producing a harness that runs on 30% fewer policy tokens than the unregularized evolution. Code is available at https://github.com/google-research/rrsi and project page is https://regularized-rsi.com/.

Abstract

LLM Agent Self-Improvement

Explore similar work

RRSI: Regularized Recursive Self-Improvement of Agent Harnesses

RSIAgent: Autonomous Exploration for Recursive Self-improvement in New Environments

The Red Queen Gödel Machine: Co-Evolving Agents and Their Evaluators