Organizations: Tencent HY LLM Frontier · University of Maryland, College Park · University of Georgia · National University of Singapore · Indiana University · Washington University in St. Louis · Nanyang Technological University
Successful trajectories on difficult tasks provide valuable supervision for model improvement, but specialized harnesses introduce interventions that may be unavailable during deployment. We propose Recursive Self-Rewrite (RSR), a framework that uses one base model, Qwen-3.8-27B, to discover successful solutions under diverse harnesses and reconstruct them as training trajectories under a general harness. A planner extracts procedures into runbooks, a critic screens for verifier and solution leakage and guides recursive revision, and an executor follows qualified runbooks in fresh sandboxes. Across approximately 3K self-curated terminal tasks, three harnesses jointly solve 759 tasks, 34.3% more than the strongest individual harness in the recorded pool. RSR expands 2,001 successful source trajectories into 11,094 rewritten trajectories for supervised finetuning. Training on these trajectories outperforms both the base model and direct trajectory SFT. Compared with the base model, pass@3 increases from 57.0% to 74.2% on Terminal-Bench 2, from 1.5% to 9.1% on Terminal-Bench 4, from 39.0% to 63.0% on our self-curated Terminal-Bench Hard, and from 3.0% to 6.0% on our Software Terminal-Bench. Process reward on Long-Horizon Terminal-Bench rises from 0.21 to 0.29. These results show how diverse harness-assisted experiences can be reconstructed into reusable capabilities for a model operating under a general harness.
Figures & tables
Figure 2: Turning multi-harness successes into learning experiences with Recursive Self-Rewrite . Different discovery harnesses enable the same model to solve more difficult tasks. Recursive Self-Rewrite rewrites these harness-assisted solutions into demonstrations under a general harness, allowing the model to learn from successful experiences that a single harness alone would not produce.
Figure 3: Three harnesses represent different logics of solving problems. Terminus 2 supports a general model–terminal interaction loop. StateM adds persistent workflow state and checked transitions. Recursive Self-Reflect Terminus uses verifier outcomes to trigger reflection and continued execution, stopping on success or timeout.
RST (420)
SWR (2,509)
Pooled (2,929)
Harness
Solved
Unique
Solved
Unique
Solved
Unique
Terminus 2
94 (22.4%)
8
471 (18.8%)
121
565 (19.3%)
129
RSRT
108 (25.7%)
14
404 (16.1%)
77
512 (17.5%)
91
StateM
95 (22.6%)
10
366 (14.6%)
58
461 (15.7%)
68
Terminus 2 ∪ RSRT
119 (28.3%)
34
572 (22.8%)
264
691 (23.6%)
298
Terminus 2 ∪ StateM
115 (27.4%)
21
553 (22.0%)
226
668 (22.8%)
247
Table 1: The union of three harnesses lead to the most unique tasks solved. Unique denotes tasks solved by only that harness. For the final row, “+” denotes the additional tasks solved relative to the strongest individual harness. A task is counted as solved when at least one recorded rollout passes. Rollout counts differ across harnesses in the full pool due to sandbox errors or task errors.
Harness
Rollouts
Passing
Success rate
Tasks solved
Terminus 2
5,405
766
14.2%
565
RSRT
3,777
599
15.9%
512
StateM
5,416
636
11.7%
461
Pooled
14,598
2,001
13.7%
759
Table 2: Discovery rollouts on the terminal task set. Passing counts successful trajectories. Tasks solved counts distinct tasks; the pooled row uses the union across harnesses. Success rate is per rollout.
Figure 4: Complementary task coverage across harnesses. (a) Task coverage of each harness and their union on our terminal tasks, and the pooled task set. (b) Exclusive and overlapping solve regions on the pooled set. Colored dots indicate the harnesses included in each region. Of the 2,929 tasks, 2,170 are not solved by any harness. Coverage uses all recorded rollouts; per-harness rollout counts differ.
Figure 5: Different harness covers differnt domains of expertise. Cell values report the percentage of tasks with at least one passing rollout, using all recorded rollouts on tasks evaluated by all three harnesses. A red marker identifies the strongest individual harness in each row. The final column reports the union, and the right-hand annotations report tasks solved exclusively by Terminus 2 (T), RSRT (R), or StateM (S).
Terminus 2
RSRT
StateM
Passing trajectories
766
599
636
Turns (mean)
19.4
23.2
24.0
Turns (median)
17.0
18.0
24.0
Commands per turn
2.10
1.96
2.37
Completion claims per trajectory
1.72
2.20
1.11
Passes after rejection (%)
0.0
4.5
0.0
Table 3: Execution styles on passing trajectories. All passing rollouts on the common task set are included. Command categories are assigned by the audit’s rule-based classifier and reported as percentages of issued commands.
Figure 6: From source experience to successful and unsuccessful rewrites. Each case compares a source trajectory with a passing and a failing rewrite guided by the same runbook. Headers report success rates and median passing lengths across all recorded rewrites (12 Markdown; eight OpenFOAM). Commands are abbreviated; displayed errors do not establish the cause of final verifier failure. Verifier success does not imply quality-audit acceptance.
Markdown (12 rewrites)
OpenFOAM (8 rewrites)
Similarity metric
Same Runbook
Different Runbook
Same Runbook
Different Runbook
Pairs compared
18
48
12
16
Exact command overlap (Jaccard)
19.1
12.1
12.6
10.4
Command order (normalized LCS)
24.6
15.7
18.2
14.7
Tool-set overlap (Jaccard)
66.9
52.9
83.3
84.4
Tool-transition overlap (Jaccard)
34.0
18.9
28.4
28.5
Table 4: Command similarity within and across runbooks. Same pairs share a runbook; different pairs use distinct runbooks for the same task. Scores are mean pairwise similarities on a 0–100 scale, including both passing and failing rewrites. Higher means more similar. The final row excludes each rewrite’s first two assistant turns.
Benchmark
Metric
Base
Direct SFT
RSR
TB 2
Pass@3
57.0% (51/89)
53.4% (48/89)
74.2% (66/89)
TB 3
Pass@3
0.0% (0/74)
5.4% (4/74)
9.5% (7/74)
TB 4
Pass@3
1.5% (1/66)
4.5% (3/66)
9.1% (6/66)
TBH (ours)
Pass@3
39.0% (39/100)
56.0% (56/100)
63% (63/100)
SWR100 (ours)
Pass@3
3.0% (3/100)
3.0% (3/100)
6.0% (6/100)
TB 2
Mean (3 runs)
51.7%
43.8%
70.1%
Table 5: RSR achieves the best results across benchmarks. We report pass@3 and the mean per-run pass rate over three runs for TB2, TB3, TB4, TBH, and SWR100, and process reward for LHTB. In addition, Direct SFT performs worse on TB 2 after training due to circular and deadend behaviors in raw trajectories from RSR .
Harness
TB2 (89)
TBH (100)
TB3 (74)
Terminus 2
51.7 (46)
33.0 (33)
0.0 (0)
Recursive Self-Reflect Terminus
55.1 (49)
66.0 (66)
4.1 (3)
StateM
58.4 (52)
46.0 (46)
1.3 (1)
Union of all three
68.5 (61)
70.0 (70)
4.3 (3)
Table 6: Different harnesses also affect Qwen-3.8-27B Base Model performance on different benchmarks. All rows use the same base model without additional training. Entries show reported success rates (%) with the supplied task counts in parentheses.