Drafting Agents, Not Just Tokens: Speculative Recursive Self-Improvement for Agentic Systems
Authors: Xin Xu, Ke Cheng, Hao Wang, Yangru Huang, Yanghao Xiao, Zhichao Chen, Zhisheng Zhong, Xun Liu, Yi Li, Qian Li, Haibin Lv, Jun Zhang, Jie Jiang, Zhouchen Lin
Organizations: Tencent AMS, Beijing, China · State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University, Beijing, China · CCSE lab, Beihang University. · Xiaohongshu · University of Chinese Academy of Sciences · Tencent Hunyuan
Improving an agent through successive revisions requires balancing exploration and evaluation under a limited budget. A narrow search can miss useful sequences of edits, while fully evaluating every intermediate version can exhaust the budget before those sequences yield a verified improvement. We introduce Speculative Recursive Self-Improvement (SpecRSI), which drafts executable successor agents before their predecessors are fully evaluated, while requiring complete verification before adoption. Verification-Aware Tree Search (VATS) allocates the budget between further edits and outstanding checks using the predicted verified return of a continuation. Group Verification Optimization (GVO) learns the improver from completed searches while keeping the real evaluator fixed. In the main scientific comparison, terminal gains exceed Janus by 33.3%--40.0%. Additional evaluations show that SpecRSI outperforms Janus in all 50 scientific backbone--task pairs and exceeds MetaRSI-v1 by 1.85%--2.38% in macro-average system score across twenty backbones. Coding-validation success reaches 69.4% versus HGM's 64.3% at a 20.8% lower target-cost index.
Figures & tables
Figure 1: Explore ahead, verify before adoption. Agent patterns (a–c) and their token-level counterparts (d–f). Agent checks are external and paid; only the adopted version needs complete, admissible evidence.
Figure 2: Performance across ten backbones. Target-cost indices and macro-average scores on Terminal-Bench 2.1 (TB2.1), SWE-Pro, GPQA-hard100, and AIME (mean ± 1.0 × sample SD across five training seeds). Panels follow Initial Agent means; lines connect model categories. On all ten backbones, SpecRSI scores higher than MetaRSI-v1 at lower target cost but 0.83–0.90% higher total-cost indices.
Figure 3: Two-level Speculative RSI Loop. VATS allocates drafts and checks under a frozen improver; Accept returns the best complete admissible version. GVO updates the improver between searches; Γ stays fixed.
Method
PyBaMM
AQM
FDTD
Reactor
IRP
Alg.-Only
0.12±0.01 [2pt]7.3±0.4
0.09±0.01 [2pt]5.1±0.3
0.14±0.01 [2pt]8.5±0.5
0.11±0.01 [2pt]6.7±0.3
0.10±0.01 [2pt]5.8±0.2
Direct-LLM
0.21±0.01 [2pt]19.2±0.6
0.17±0.01 [2pt]14.3±0.5
0.24±0.02 [2pt]22.6±0.7
0.19±0.01 [2pt]16.2±0.4
0.18±0.01 [2pt]13.7±0.4
Embed-GP
0.30±0.02 [2pt]30.5±0.8
0.25±0.01 [2pt]24.8±0.7
0.33±0.02 [2pt]36.2±0.9
0.27±0.01 [2pt]27.4±0.6
0.25±0.01 [2pt]23.1±0.5
Janus
0.36±0.02 [2pt]42.8±1.0
0.30±0.02 [2pt]34.7±0.8
0.39±0.02 [2pt]51.3±1.1
0.32±0.02 [2pt]36.9±0.8
0.30±0.01 [2pt]31.5±0.7
SpecRSI (ours)
0.48 ±0.02 [2pt] 56.7 ±1.2
0.41 ±0.02 [2pt] 45.8 ±1.0
0.52 ±0.02 [2pt] 66.9 ±1.3
0.44 ±0.02 [2pt] 49.3 ±1.0
0.42 ±0.02 [2pt] 42.1 ±0.9
Table 1: Performance across scientific discovery and agent improvement. A: terminal gain (upper entry) and call-axis AUBC (lower entry), in native units. B: task scores (%) within matched edit permissions. Entries show mean ±1.0× SD; B uses SD. Bold denotes the highest mean within each comparable group.
Figure 4: Verification scope and planning ablations. (a) Gain versus total-cost index for one check or bundles at fixed Summary or Full-state information; bars show reported SD. (b) Verified gain and (c) total-cost index across eight budget caps. In (b), means average five reported seed means; SD averages within-seed SDs. Lines join means; bands/bars show ±1 SD.
Start
Update
Gain
(pp)
Reset
Frozen
4.1±0.18
SFT only
4.6±0.19
GVO only
5.0±0.20
All roles
5.2±0.21
Carry
Frozen
4.8±0.20
Table 2: Learning the improver. Gains (pp). (a) Reported uncertainty; compare within each starting condition. (b) Updated roles: D, drafter; Sc, scorer; P, policy; Sh, shared backbone. Sci.: Science. Last two rows have unmatched training costs. (c) Incremental (Inc.)/cumulative (Cum.) gains for original (Orig.)/updated (Upd.) improvers. All carry the agent; round 1 initializes. Full controls: Appendix D .
Plans
Gain (pp)
Overhead
Missing (%)
Net
8
5.2±0.19
1.8
8.6
3.9
16
5.8±0.21
2.7
5.2
4.4
32
6.3±0.25
3.9
3.1
4.7
64
6.5±0.24
5.6
2.0
4.5
Table 3: Planning capacity and lifecycle cost. (a) MENU varies the offered-plan cap; gain is in pp with reported SD, and Missing is the percentage of omitted good options. Overhead and Net retain their reported definitions; Net is not recomputed as Gain minus Overhead. (b) Reported lifecycle indices; overlapping components are not independently additive.
Appendix figures & tables40 assets
Supplementary material from the paper’s appendix.
Appendix
Figure S1: Main paper and Appendix.
Janus
SpecRSI (ours)
LLM program generator
Drafter / DRAFT : propose executable edits
Target program / offspring
Immutable successor agent c′
Proxy evaluator
Scorer: learned score evaluator Qϕ(S,ω)
Promotion / acquisition rule
VATS Solver + Policy: allocate draft and check work
Real evaluator freal
Γ + AcquireΓ ; invoked by VERIFY
Real-validated incumbent
ACCEPT : best complete admissible version
Appendix
Table S1: Functional correspondence between Janus and SpecRSI. The Scorer predicts continuation value; the real evaluator supplies evidence for adoption.
Figure S2: Performance across backbones. (a) Relative terminal-gain and AUBC uplift over Janus across ten backbones and five scientific tasks. (b) Macro-average scores over TB2.1, SWE-Pro, GPQA-hard100, and AIME across twenty backbones, ordered by Initial Agent mean. Within-backbone segments connect the three methods; markers and horizontal error bars show means and ±1 sample SD over five training seeds. The right-hand difference panel shows seed-paired SpecRSI − MetaRSI-v1 differences in percentage points with the same uncertainty convention.
Figure S3: Performance across code, prompt, and workflow tasks. Reported means: Score, Succ., and Comp. in percent; Gain in pp; Tok. in thousands. Darker cells indicate better within-metric performance; dashes mark unreported values. Code completion uses a separate profile, and costs retain task-specific definitions. Beam: Beam-Spec; FE-A/FE-O: FlashEvolve async/optional; M2/M2–D: MIPROv2 with/without demos; Tr.: Trace; TG: TextGrad; G+M: GEPA+Merge; Gen./G+S: Genesis/Genesis+SOP.
Method
Complete (%)
native FlashEvolve async-stages
91.2
native FlashEvolve optional-spec
92.8
SpecRSI (ours)
94.7
Appendix
Table S4: Asynchronous completion. Completion rate supplements the coding-validation scores; it is not another estimate of coding success. The two native FlashEvolve modes remain separate.
Table S6: Prompt optimization. GPQA-hard100 means; no reported SD. Gain is an absolute change on the % score scale.
Method
Success (%)
Task tokens
Genesis
51.3
11260
Genesis+SOP
57.2
11080
AFlow
59.6
10720
ADAS
61.1
10490
EMAS
62.7
10210
SpecRSI (ours)
67.8
9290
Appendix
Table S7: Workflow optimization on PlanBench. Success is a task score; tokens count task inference only, excluding search and training. Both columns retain the supplied means.
Method
Math
MBPP
Game24
SpecRSI (ours)
62.3 / 10390
59.4 / 11020
54.7 / 12080
Appendix
Table S8: Additional workflow tasks. Success (%) / serving tokens; PlanBench is already in Table S7 .
Information
Work scope
Gain (pp) ↑
Cost ↓
Fail (%) ↓
Hit (%) ↑
Summary
One check
3.2±0.21
67.4
15.2
72.0
Summary
Bundle
4.1±0.24
65.8
12.7
78.0
Full state
One check
4.8±0.25
63.1
9.8
84.0
Full state
Bundle
6.3±0.26
59.2
6.5
92.0
Appendix
Table S9: Verification scope with information held fixed. Gain is in percentage points (pp), with reported SD. Cost is the study’s total-cost index; failure and hit rates retain the study’s definitions. The two full-state rows isolate work scope at a fixed information level.
Figure S4: Paired gain differences and planning time. (a) Method minus Full SpecRSI gain uses the separately reported paired-seed difference means and SDs, not differences recomputed from Figure 4 b. Full relative to itself is zero. Lines join means; bars/bands show ±1 SD. (b) Color encodes planning-time mean; cells report mean ± SD in milliseconds. Methods and eight budget caps match Figure 4 ; Full (ours) denotes Full SpecRSI (ours). Seeds only: completion seeds only; No seeds: no completion seeds; Frozen policy: refinement/frozen policy; Coupled: coupled scorer/policy; Greedy: greedy scorer selection.
Figure S5: What changes when planning components are removed? These are the original reduced- and standard-budget panels, not four experiments. A/B show raw gains and each cohort’s difference from its own full method; C shows planning milliseconds and total-cost indices. In D, fin. is finalist completion (%), chk. checks per block, dep. draft depth, and cand. candidates. A larger work count is not intrinsically better. Except for B, shading is within-column scaling; darker does not always mean better. The expanded eight-budget gain/cost curves and SDs are in Figure 4 ; paired differences and planning times are in Figure S4 .
Condition
Gain (%, abs.)
Draft depth
Checks/block
Allocator blind
3.2±0.17
1.1
1.1
Mask only
4.7±0.20
1.6
1.6
No relations
3.8±0.18
1.4
1.3
Restored valid fields
5.8±0.23
2.3
2.5
Full evidence
6.3±0.25
2.6
2.8
Appendix
Table S10: Evidence and selected work. Gain ± SD; depth and checks per block are observed means, not optimization targets.
Figure S6: What information changes verification allocation? All panels describe the evidence-input study. C reports valid-field fraction, relation coverage, and observed blocks. D min–max rescales the same summaries to [0,1] , reversing cost only; it is not another evaluation. Dep. means draft depth, chk. checks per block, and cand. candidates. Source gain SDs remain in Table S10 .
Predictor
MAE
NDCG @3
Regret @3
Fit cost
GP
0.087
0.782
0.124
12.3
RF
0.102
0.735
0.156
8.7
MLP
0.095
0.758
0.141
10.5
Quadratic
0.114
0.703
0.172
7.2
RBF-SVR
0.091
0.771
0.133
13.8
Direct LLM
0.136
0.654
0.208
5.3
Appendix
Table S11: Predictive-model diagnostics. Lower MAE, regret and fitting cost are preferred; higher NDCG is preferred. The historical row name “evolved evaluator” denotes a candidate predictor, not the protected Verification Contract.
Figure S7: Starting-artifact controls and updated roles. (a) Reset, carry, and same-artifact controls; gain is in pp. Reported uncertainty, regression values, and cost/hit/failure shading are retained. Compare rows within each starting condition. (b) Filled switches mark updated roles: D, drafter; Sc, scorer; P, policy; Sh, shared backbone. Columns report Science (Sci.), Code, and Agent gains (pp). The final two rows use a separate shared-backbone extension; training costs are unmatched.
Improver
New-task gain (pp)
Cost index
Regressions
Original
5.7
54.2
0.1
Updated
6.9
50.8
0.0
Appendix
Table S12: Continuation from the same released artifact. Released-fork means and costs.
Condition
Gain (pp) ↑
RMSE ↓
Training cost ↓
Same arms / means
5.4±0.23
1.05
23.7
Same arms / means + pair
6.3±0.25
0.85
28.4
Equal-acquisition-cost search
5.1±0.22
1.17
25.3
Appendix
Table S13: Learning signal versus data acquisition. The first two rows use the reported same acquired arms and differ in loss. The third uses ordinary-search acquisition at the reported equal acquisition cost. Gain follows the source Gain field.
Acquisition
Paid runs
Data cost
Reset gain
State relation
Ordinary searches
24.6
1.0x
4.2
Independent
Extra ordinary equal-cost
24.6
1.0x
4.5
Independent
Same arms / means
24.6
1.0x
4.8
Paired
Same arms / means + pair
24.6
1.0x
5.3
Paired
Different policy snapshot
24.6
1.0x
4.0
Mismatched
Appendix
Table S14: Continuation acquisition. Paid runs are reported averages; equal-data and equal-cost controls answer different questions.
Figure S8: Which contrast is being read in the recursive-round study? A/B subtract original from updated within a method; C/D subtract Meta from Spec (ours) at the same improver status. All values are differences of Table 2 (c)’s means, not new runs. Units are percentage points (pp).
Figure S9: Repetition and protocol for the recursive-round results. Each row in A is a supplied seed mean, not an individual search. B declares carried roots in all four conditions, initialization at round 1, evaluated rounds 2–5, five training seeds, and three searches per seed.
Menu
Forecast
Plan
Work
Final
Singleton only
3.8
1.2
54.7
3.7
Seeds
5.2
2.8
55.3
5.1
Seeds + growth
6.1
3.9
54.8
5.9
Larger menu
6.4
5.6
55.1
6.0
Exhaustive
6.7
8.9
54.6
5.8
Appendix
Table S15: Menu construction and planning cost. Comparison of singleton-only menus, completion seeds, seeded growth, larger menus, and exhaustive construction. This study varies the construction strategy separately from the menu-cap sweep in Table 3 (a).
Figure S10: More planning is useful only if its options repay its cost. Menu-cap and menu-construction results are separate studies. In (a), over is reported overhead, miss the missing-option percentage, and net the reported net return; the outline marks the largest tested net. In (b), fore and final denote Forecast and Final; (c) shows Final minus Forecast. Comp. denotes completion, cand. candidates, chk. checks, dep. depth, and n retained searches. Darker shading indicates a larger value within each column.
Regime
Batch
Serial
Δ
Dependent fail
10.2
12.3
-2.1
Independent fail
11.5
12.3
-0.8
Cheap dispatch
13.2
12.0
+1.2
Costly dispatch
15.4
12.0
+3.4
Early cancellation
10.8
12.3
-1.5
Reinvestment
14.7
12.0
+2.7
Appendix
Table S16: Batch versus serial verification. Δ=Cbatch−Cserial ; negative values favor batching. Comparisons are within each row; the six configurations do not constitute a controlled single-factor sweep.
Figure S11: Cost trade-offs support adaptive verification scope. (a) Batch and serial cost components and execution conditions. (b) Mean batch-minus-serial cost. Batching lowers mean verification cost by 6.5%–17.1% in three configurations, while serial execution is cheaper in the other three, supporting VATS’s state-dependent choice between singleton checks and completion bundles. SD.b/SD.s denote the batch/serial cost SDs; stems show mean differences. Dashes denote unreported values.
w
Gain (%)
Time (s)
Work (s)
1
6.3±0.25
1287
1287
2
6.4±0.24
672
1312
4
6.4±0.26
354
1348
8
6.5±0.25
192
1403
Appendix
Table S17: Worker configurations. Time is elapsed wall-clock time; Work is total worker time. Eight workers achieve a 6.7× speedup with 9.0% more total work.
Condition
ΔJ
Lower
Upper
Cost
Baseline
0.0
-
-
1.00
No penalty
+2.4
+1.8
+3.0
1.03
Standard
+2.1
+1.6
+2.6
1.00
Heavy
+1.7
+1.2
+2.2
0.98
Appendix
Table S18: Regression-penalty sensitivity. Lower and Upper retain the supplied interval endpoints. Stronger penalties reduce normalized cost while retaining positive objective improvement.
Procedure
Returns
Changes and calls
Algorithm 1 : search
Complete agent and log
Updates versions, evidence and budget; calls Algorithm 2
Algorithm 2 : VATS
Feasible selected plan
Settles preparation; fixed feasibility rules and frozen scorer/policy
Algorithm 3 : GVO
Updated improver
Calls Algorithm 1 ; updates only after collection
Appendix
Table S19: Two timescales, three procedures. Only the outer training procedure updates the improver’s parameters.
Object
Permitted change
Update boundary
Task agent ck
Prompts, tool code, workflow and scoped persistent memory
Executed edit creates an immutable successor
Task-model snapshot
Weights only when M is allowed
D/H/M profile; absent from direct fixed-weight core
GVO after complete searches; frozen inside each search
Verification contract Γ
None
Fixed tasks, admission and scoring rules
Outer update rule
None
GVO algorithm fixed; learned parameters may change
Appendix
Table S20: What changes, and when? A legal search need not edit every task surface in one round.
Symbol
Meaning
Γ,I,JΓ
Fixed verification contract, required items, and score aggregation
T=(C,A)
Actual immutable versions and executed parent–child edit edges
E,H
Admitted outcome records and observed execution history
mci,Uc,Kc
Outcome completeness bit, missing items, and candidate–item target keys
Cfull,F
Complete admissible versions and their best improvement over the initial root
B,N,w
Remaining expense, primitive-attempt allowance, and worker count
Appendix
Table S21: The state separates versions, observations, and predictions. These objects are shared by all three algorithms.
Figure S12: Observed results, planned work, and predictions have different roles. Planning may use missingness and source proposals; only contracted direct execution or sound scoped equality adds a valid result. Observed does not mean passed; a missing item blocks adoption but not further editing.
Candidate work
Required information
Role
One edit or check
Existing parent or item; expense and attempt caps
Feasible fallback
Completion bundle
All missing keys; dependency and setup closure
Preserve joint options
Refined bundle
Same proposal state and frozen value model
Compare continuations
Selected plan
Paid state, canonical menu, policy preference
Execute; then replan
Appendix
Table S22: What the solver must preserve at one decision. These are executable constraints, not learned predictions of correctness.
Figure S13: Two data paths train different quantities. Ordinary complete searches provide actor likelihoods and return labels. Paid forced continuations provide critic labels only. The next epoch freezes the updated improver; the verification contract Γ stays unchanged.
Reader question
Result to read
Essential condition
Can predictions corrupt a completed result?
Outcome consistency (Proposition 1 )
Sound records; fixed contract
Does grouping enlarge every policy class?
Singleton simulation (Proposition 4 )
Free serial decision boundaries
When does commitment save expense?
Feedback threshold (Proposition 5 )
Identical output and charged blocks
What do success summaries omit?
Scope reversal and matched control (Propositions 7 , 8 )
Table S23: What each formal result establishes. The assumptions belong to the reference model; these results are not neural benchmark measurements.
Figure S14: Early feedback changes the cost of committing work. Exact sensitivity of Equation ( 24 ), with dispatch 1 and planning 0.2. These curves do not fit the batch-versus-serial study or assert that batching always wins.
Outcome
A
B
000
0
3/8
001
1/4
0
010
1/4
0
011
0
1/8
100
1/4
0
101
0
1/8
Appendix
Table S24: Exact laws and complete scheduling costs. Each bit is a required check outcome. The left panel lists all probability atoms; the right lists all surviving-path block compositions at c=1 , d=3/2 . Both contexts have marginal pass probability 1/2 and all-pass probability 1/4 . This is one finite construction, not two experimental datasets.
Figure S15: The optimal scope depends on commitment costs. Both plots evaluate the stated analytical costs, not fitted outcomes. The one-dimensional slice fixes dispatch at 3/2 ; the phase map varies dispatch and pair-survival probability. Boundaries and ties follow the declared formulas.
An LLM agent's capability is largely magnified by its harness, namely the prompts, control flow, tooling, memory, and context management surrounding the frozen backbone model. Recent methods increasingly automate this process by iteratively proposing and selecting component-wise edits of an agent harness, practically establishing a form of recursive self-improvement (RSI) at the agent-system level. However, such recursive evolution may overfit by memorizing the training tasks, showing large in-distribution gains that shrink or even vanish on out-of-distribution benchmarks. We introduce Regularized Recursive Self-Improvement of Agent Harnesses (RRSI), which incorporates the principles of regularizations into harness self-improvement by constraining the evolution candidate proposal and selection. The proposer operates with a temporally annealed budget, limiting how many edits a candidate can bundle, and it encourages unexplored trajectories based on evolution history. The selector is equipped with a critic and a pruner: the critic screens benchmark-specific proposals, while the pruner, removes changes that are too small, too expensive, or no longer useful. Together these constraints favor reusable agent mechanisms over benchmark-specific ones or even noises. Across eight benchmarks spanning coding, agentic workspace and engineering design tasks, RRSI gains up to 6.0 points on the split it evolves against and up to 4.3 points on the five out-of-distribution benchmarks, while producing a harness that runs on 30% fewer policy tokens than the unregularized evolution. Code is available at https://github.com/google-research/rrsi and project page is https://regularized-rsi.com/.
Peng Xia, Rujun Han, Zifeng Wang +11
Google Cloud AI Research · UNC-Chapel Hill · Stanford University +1
Digital agents must often adapt to new environments whose interfaces, tools, and failure modes are not fully captured by pretrained models. We introduce RSIAgent, a training-free multi-agent framework for recursive self-improvement through autonomous memory construction. RSIAgent coordinates curriculum, actor, and verifier agents to continually explore the environment, validate outcomes, and retain environment-specific knowledge, including reusable causal relationships between actions, conditions, and consequences. It further adopts a broad-then-deep exploration strategy, combining parallel broad recursive self-exploration for discovering diverse environment structures with focused deep self-exploration for uncovering hard cases, hidden constraints, boundary conditions, and previously unknown causal dependencies. The resulting memory is frozen and can be directly reused for downstream tasks without updating model parameters. Experiments on OSWorld-v2 and Agent's Last Exam show that RSIAgent substantially improves strong open-source models, enabling Kimi-K3 and GLM-5.3 to outperform frontier closed-source models including GPT-6.
Sibo Zhu, Shicheng Fan, Xinyue Wang +3
Aether AI · University of California San Diego · ‡Work done during internship in Aether AI +1
Self-improving agents are state-of-the-art on agentic coding benchmarks, yet their search methods assume a stationary evaluation criterion. This ignores a central feature of evolution: species adapt as their environments change with them. We introduce the Red Queen Gödel Machine (RQGM), an evolutionary framework for recursive self-improvement under non-stationary utilities. This allows learned evaluators to improve alongside the agents they guide. On DeepSWE, the RQGM improves over its fixed-evaluator baseline by adding a complementary agent-as-a-judge code-review signal: a co-evolved reviewer grades coder patches to guide search. At low reasoning effort, the RQGM coder passes 82.1% of held-out tasks against the baseline's 75.0%, and nearly matches the GPT-6 Astra model at high effort. In scientific paper writing and reviewing, and Olympiad-level proof writing and grading, co-evolved evaluators provide an evaluation criterion. Anchored to human IMO grades, a co-evolved grader writes its own milestone rubric and exceeds static baselines at a 3x lower search cost, driving the prover to the best mean score. Since the RQGM can modify the search objective across epochs, it can regularize the search. For example, the RQGM reduces self-preference bias via an additional adversarial objective to discover reviewers equally stringent on AI and human work. Guided by these calibrated reviewers, co-evolved writers reach 1.78x-1.86x higher acceptance rates than the baseline under an agent-as-a-judge panel. The RQGM enables self-improving systems where agents and evaluators recursively bootstrap each other beyond static evaluation.