Organizations: Tencent HY LLM Frontier · University of Georgia · University of Maryland, College Park · National University of Singapore · Indiana University · University of Illinois at Chicago · Hong Kong Polytechnic University
Terminal agents are increasingly deployed beyond software engineering in science and other specialized domains. Constructing training environments requires executable reference behavior and a domain-specific verifier that distinguishes semantic correctness from superficially plausible artifacts. Authoring these components for each task requires repeated engineering and limits reuse. We introduce software-in-the-loop reconstruction, a self-supervised framework that obtains reference outputs and verification targets from existing software workflows, executable programs mapping structured inputs to outputs. For each workflow, we execute multiple input configurations and partition cases into public observations and hidden evaluations. Given the instruction, input schema, and public input--output observations, an agent constructs an editable program without access to the source workflow. The candidate is evaluated on hidden configurations against workflow outputs. A hierarchical verifier combines domain-specific semantic comparison, structural validity, and anti-shortcut checks, while public feedback supports iterative revision. The construction admits additional workflows and configurations without authoring a reference solution for each task. We instantiate SWR with 500 workflows and 46 software families across six domains. Across three attempts per task, Qwen3.8-Max solves 838 tasks and produces 1,422 verified trajectories, which we oversample to 3,000 reconstruction-only training examples. Supervised fine-tuning of Qwen3.8-27B improves mean Terminal-Bench 2 performance from 47.94% to 53.56% across three seeds and achieves the highest mean among four matched-token corpus controls on all four reported evaluations. These results indicate that existing scientific software can provide scalable, behaviorally verified supervision for terminal agents.
Figures & tables
Figure 2: Software-in-the-loop reconstruction. Public observations are the input–output examples released to the agent. They guide construction of a reusable program. Withheld inputs supply the reference outputs used for verification.
Figure 3: Comparison of verification standards. Structural checks can accept an invalid constant output, while byte-identical comparison can reject a behaviorally equivalent output. Semantic verification evaluates task-relevant behavior and is designed to reduce both failure modes.
Figure 4: Composition of SWR. Panel (a) is the task distribution across software domains. Panel (b) is the artifact types. Panel (c) is the scenario parameter types and dimensions.
Dataset
Tasks
Families
Workflows
Public scenarios
Hidden scenarios
SWR
3,000
46
500
15,600
8,400
Table 1: Composition of the evaluated SWR release. A workflow is an existing program that maps an input to an output. The columns count tasks, software families, workflows, and public and hidden input configurations.
Figure 5: Sensitivity of SWR outputs to input changes. The four metrics record whether a pair of configurations changes any output, how many semantic fields vary, how many fields change in an average pair, and how many declared parameters affect the output.
Scheduled
Records
Strict pass
Pass@1
Record rate
Pass/record
3,000
2,978
685
22.8%
99.3%
23.0%
Table 2: Single-attempt evaluation of Qwen3.8-Max on SWR with the finalized verifier. Pass@1 uses all 3,000 scheduled tasks. Missing records count as failures.
Figure 6: Eight-model comparison on SWR100. Panel (a) is overall strict success. Panel (b) is success by domain.
Figure 7: Eight-model outputs for an OpenSCAD snap-fit task with a 16 mm public example and a 21 mm hidden reference. GPT-5.6 Sol preserves separate body and lid components, whereas the other seven outputs collapse to plate-like meshes at wall-thickness height.
Result
Verified tasks/trajectories
Rate
Pass@3 / task union
838
27.9%
All verified trajectories
1,422
–
Table 3: Three-attempt result of Qwen3.8-Max on SWR. The columns give verified tasks or trajectories and the corresponding rate.
Training entries
TB2
TB4
LHTB reward
SWR100 success
Base
47.94±1.30
0.51±0.87
20.67±1.15
1.33±0.58
SFT-750
50.19±0.65
1.01±0.87
22.33±0.58
1.33±0.58
SFT-1500
50.56±1.12
2.02±0.87
24.33±1.53
2.67±0.58
SFT-3000
53.56±0.65
3.54±0.87
27.67±0.58
3.33±0.58
Table 4: Training-entry budgets for Qwen3.8-27B reconstruction SFT, including repeated samples. Entries are means and standard deviations over seeds 42, 43, and 44. Each seed is trained and tested separately. All entries are percentages.
Training corpus
Loss tokens
TB2
TB4
LHTB
SWR100
Base (no SFT)
–
47.94±1.30
0.51±0.87
20.67±1.15
1.33±0.58
OpenThoughts-Agent-v1
88.48M
49.06±1.30
0.00±0.00
22.67±1.53
1.67±0.58
TMax
88.48M
49.44±2.25
1.01±0.87
20.67±1.53
1.33±0.58
LiteCoder-Terminal
88.48M
48.69±1.72
0.51±0.87
23.00±1.00
1.00±1.00
Nemotron-Terminal
88.48M
47.19±1.12
0.51±0.87
23.67±1.15
1.33±1.53
Reconstruction (ours)
88.48M
53.56±0.65
3.54±0.87
27.67±0.58
3.33±0.58
Table 5: Matched-token comparison of Qwen3.8-27B training corpora. Entries are means and standard deviations over three seeds, reported as percentages. Every fine-tuned model receives 88.48M assistant-loss tokens. The reconstruction corpus contains 3,000 sampled entries, including repeats.
Appendix figures & tables22 assets
Supplementary material from the paper’s appendix.
Appendix
Study
Tasks
Runs/model
Turn limit
Wall time
Retained outcomes
Full-suite evaluation
3,000
1
100
1 hour
Success and failure
Three-attempt Pass@3
3,000
3
–
–
Final task union
SWR100 comparison
100
1
500
3 hours
Success and failure
SFT scaling
4 suites
3
–
–
Train-and-test seeds
Appendix
Table 6: Protocols for the reported evaluations. Columns give the task count, runs per model, turn limit, wall-clock limit, and which outcomes are retained.
Figure 8: Output types of the 46 software families. Families are grouped by domain. A filled mark means every task in the family produces that output type. An open mark means only some tasks do. In CMake builds, source code appears in 36 of 60 tasks and configuration files appear in 54 of 60. The bars give the median number of checkable output fields on a logarithmic scale.
Figure 9: Released public artifacts from the six domains. Each row is one domain and contains three examples taken from the files released with the task. The software panels redraw a released JSON schema, a released parse tree, and the RTL module names in a released file list. The other panels are released waveforms, source-program meshes, a map boundary, a molecule, biological sequences, a query result, a musical score, glyph positions, a well log, and two views of one seismic trace.
Figure 10: Domain and software-family composition of SWR. Families are grouped into six domains. No family exceeds 2.4% of the tasks.
Figure 11: Number of checkable output fields per task in SWR. Digest-like profiling fields are excluded.
Control
Hidden-field agreement
Strict success
Oracle
100.0%
100.0%
Oracle-selected best-public replay
44.3%
0.0%
Nearest public scenario
38.0%
0.0%
Constant public output
34.3%
0.0%
Appendix
Table 7: Public-output replay audit on SWR. Hidden-field agreement is the share of hidden fields matched by a replay control. Strict success requires a complete hidden pass.
Figure 12: Interaction cost in the single-attempt SWR evaluation. Panel (a) is the distribution of trajectory length. Panel (b) is family pass rate against median turns.
Model
Seed
TB2 (%)
TB4 (%)
LHTB
SWR100
Base
42
47.19
0.00
22.00
1.00
Base
43
49.44
1.52
20.00
1.00
Base
44
47.19
0.00
20.00
2.00
SFT-750
42
50.56
1.52
23.00
1.00
SFT-750
43
49.44
1.52
22.00
2.00
SFT-750
44
50.56
0.00
22.00
1.00
Appendix
Table 8: Per-seed training budgets for Qwen3.8-27B. These rows underlie Table 4 . LHTB and SWR100 entries are percentages.
Training corpus
Seed
TB2
TB4
LHTB
SWR100
OpenThoughts-Agent-v1
42
48.31
0.00
21.00
1.00
OpenThoughts-Agent-v1
43
50.56
0.00
23.00
2.00
OpenThoughts-Agent-v1
44
48.31
0.00
24.00
2.00
TMax
42
49.44
0.00
21.00
2.00
TMax
43
47.19
1.52
19.00
1.00
TMax
44
51.69
1.52
22.00
1.00
Appendix
Table 9: Per-seed scores for the four matched-token control corpora. Every evaluation entry is a percentage.
Figure 13: SWR100 trajectory profiles under a three-hour limit per task. Panel (a) is agent turns. Panel (b) is prompt and output tokens. Points are medians. Horizontal segments are interquartile ranges.
Model
Records
Scored
Pass
Turns
Tokens (M)
GPT-5.6 Sol
99
89
20
191.0
10.12
Kimi-K3
98
98
6
131.5
6.94
Grok 4.6
100
99
4
98.0
5.03
DeepSeek-V4-Pro
100
99
3
187.5
10.00
Qwen3.8-Max
100
100
3
356.5
18.59
GLM-5.3
100
100
3
69.5
4.49
Appendix
Table 10: Per-model results on SWR100. Pass@1 uses all 100 scheduled tasks for each model. Turns and tokens are medians.
Mutation class
Trials
Accepted
Rejected
Valid near-boundary controls
7,065
7,065
0
Single deep-leaf change
2,940
0
2,940
Correlated numeric shift
2,868
0
2,868
Wrong hidden scenario
2,939
0
2,939
Ignored input axis
9,901
0
9,901
Numeric perturbation >1× tolerance
9,260
0
9,260
Appendix
Table 11: Stress test of the enhanced verifier on the full SWR release. Acceptance counts as correct for a valid control and as incorrect for every other mutation class.
Figure 14: Stress tests of the enhanced verifier. Panel (a) uses 2,315 outputs with finite floating-point values at each perturbation size. Panel (b) reports detection rates with Wilson 95% lower bounds. Labels give rejected counts and totals.
Audit stratum
Trials
False accept
False reject
Correct
95% CI
Structured valid controls
376
0
0
100.00%
[98.99, 100.00]
Geometry valid controls
24
0
6
75.00%
[55.10, 88.00]
Structured invalid mutations
752
2
0
99.73%
[99.04, 99.93]
Geometry invalid mutations
48
1
0
97.92%
[89.10, 99.63]
All held-out trials
1,200
3
6
99.25%
[98.58, 99.60]
Appendix
Table 12: Verifier audit after freezing and before human adjudication. Intervals are Wilson 95% intervals for the correct-decision rate.
Audit stratum
Tasks
Bundles
Label validity
Verifier agreement
Annotator agreement
Family-stratified core
92
184
98.91% (182/184)
98.37% (181/184)
96.74% (178/184)
Risk-enriched
8
16
87.50% (14/16)
93.75% (15/16)
87.50% (14/16)
All
100
200
98.00% (196/200)
98.00% (196/200)
96.00% (192/200)
Appendix
Table 13: Blinded human audit of the held-out verifier set. Label validity and verifier agreement use adjudicated judgments. Annotator agreement is measured before adjudication.
Human-adjudicated label
Verifier accepts
Verifier rejects
Total
Valid / equivalent
98
2
100
Invalid / non-equivalent
2
98
100
Total
100
100
200
Appendix
Table 14: Verifier decisions after adjudication on 200 blinded bundles. Human-adjudicated semantic validity is the reference label.
Figure 15: Primary outcomes on SWR100, grouped by the order of evaluation checks. Every model receives the same 100 tasks. Rare geometry errors, missing records, and residual outcomes are grouped as Other.
Task mechanism
Passing trajectory
Failing trajectory
Decisive evidence and interpretation
Color-transform numerics
GPT-5.6 Sol, 213 turns, 41 candidate tests
Qwen3.8-Max, 376 turns, 25 candidate tests
The hidden mean is 0.291619 rather than 0.267706, and every reported XYZ component exceeds tolerance after 45 public-feedback calls.
Container SBOM completion
GPT-5.6 Sol, 110 turns, 11 edits, 17 tests
Grok 4.6, 149 turns, one edit, one test
No executable candidate remains, so semantic grading never begins. The final plan still defers implementation until after another search.
Robotics hidden token
GPT-5.6 Sol, 31 turns, 0.97M tokens
Qwen3.8-Max, 368 turns, 20.60M tokens
After six public passes, an unseen categorical value is cast to an integer and causes hidden execution to fail. Public completion conceals a parser that cannot process the new value.
Video policy category
Kimi-K3, 64 turns, 14 tests
DeepSeek-V4-Pro, 204 turns, seven tests
A closed vocabulary rejects an unseen audio-layout category. The passing implementation maps unseen categorical values deterministically.
Appendix
Table 15: Four matched pairs of a success and a failure on the same task. Turns and tests summarize interaction effort. The last column states the evidence tied to each outcome.
Figure 16: Same-task numeric generalization. GPT-5.6 Sol passes after 213 turns and 41 candidate tests. Qwen3.8-Max uses 376 turns and 45 feedback calls but only 25 candidate tests. Its hidden mean is 0.291619 rather than 0.267706, and every reported XYZ component is outside tolerance. The failure is joint transform recovery, not a lack of public checks.
Figure 17: Same-task artifact completion. GPT-5.6 Sol turns 110 turns into 11 edits and 17 candidate tests and leaves a passing executable fallback. Grok 4.6 spends 149 turns and 7.54M tokens on evidence search but makes one edit and one test. Its final plan still defers implementation. The workspace contains no executable candidate.
Figure 18: Same-task hidden-input robustness. GPT-5.6 Sol passes in 31 turns with a representation that accepts unseen categorical values. Qwen3.8-Max uses 20.60M tokens and declares completion after all six public scenarios pass, but an invalid integer conversion then prevents execution on a hidden input. The contrast separates hidden-input robustness from completion on public feedback.
Figure 19: Same-task policy generalization. Kimi-K3 maps unseen categorical values deterministically and passes after 64 turns and 14 candidate tests. DeepSeek-V4-Pro reads seven times as much public evidence but runs only seven candidate tests. Its closed vocabulary rejects an unseen audio-layout category. The trajectory then returns to external search instead of adding a fallback for unknown categories.
Terminal agents rely on self-verification to assess and correct their solutions as they solve tasks through interaction with command-line environments. Yet how trustworthy such self-verification is remains poorly understood. To investigate this question systematically, we introduce a diagnostic framework that identifies the first complete solution in each trajectory, determines whether it is objectively correct, and uses this ground truth to quantify the agent's subsequent verification and recovery behavior. Applying it to ten terminal agents on TerminalBench2.1, we find that verification is nearly universal after a complete candidate is formed, yet only 61.43% of incorrect candidates are detected and only 49.36% of detected errors are successfully repaired. These results show that the main weakness in self-verification lies not in initiating verification, but in detecting and repairing errors. Motivated by these findings, we propose Student-Conditioned Verification Distillation (SCVD), which lets the student first produce a candidate solution and distills a stronger teacher's subsequent verification and recovery from the same interaction context. Across three Qwen3.5 backbones, SCVD improves Pass@1 on TerminalBench2.1 by 9.74--16.85 percentage points over the corresponding base models and by 4.49--8.61 points over the standard full-trajectory distillation, while avoiding the pronounced out-of-distribution degradation of full-trajectory distillation on SWE-bench Verified.
Yingfeng Luo, Shaowei Wei, Daixin Wang +7
Northeastern University · Inclusion AI, Ant Group · University of Maryland, College Park
Terminal agents solve tasks in software engineering and scientific research by executing commands in a terminal. Training them via reinforcement learning (RL) requires diverse and verifiable environments, but these environments are still mostly designed by human researchers, so their supply grows only as fast as researchers can work. We introduce AutoEnvScaling, which treats environment design as a terminal task and uses agents to automate the data flywheel. A proposer agent builds new RL environments from the rollouts of the solver being trained, and the solver's new rollouts guide the next round. By turning environment generation into a terminal task, AutoEnvScaling increases the valid-environment rate by up to 25 times compared to prompt-only generation, at up to 75% lower cost per valid environment. We show that training Qwen3.5-35B-A3B on AutoEnvScaling environments from GPT-5.6-sol improves its Terminal-Bench 2.1 score by 4.3 points over the matched Tmax baseline. The broader goal of AutoEnvScaling is to enable recursive self-improvement (RSI): by making both environment generation and solving terminal tasks, it lets one model co-evolve in both roles. We show that Qwen3.6-35B-A3B, acting in both roles, gains 12.2 points on Terminal-Bench 2.1 and raises its Terminal-Bench 4.0 score from 1.6 to 3.2. The pipeline also generalizes across agentic domains, generating environments for software engineering, computer use, professional work, and GPU kernel programming. Overall, these results show a path toward agents building their own training environments, reducing human reliance and taking a core step toward RSI.
Simon Yu, Baolin Peng, Bo Liu +8
Northeastern University · Microsoft · Stanford University +2
We propose SWE-Universe, a scalable and efficient framework for automatically constructing real-world software engineering (SWE) verifiable environments from GitHub pull requests (PRs). To overcome the prevalent challenges of automatic building, such as low production yield, weak verifiers, and prohibitive cost, our framework utilizes a building agent powered by an efficient custom-trained model. This agent employs iterative self-verification and in-loop hacking detection to ensure the reliable generation of high-fidelity, verifiable tasks. Using this method, we scale the number of real-world multilingual SWE environments to a million scale (807,693). We demonstrate the profound value of our environments through large-scale agentic mid-training and reinforcement learning. Finally, we applied this technique to Qwen3-Max-Thinking and achieved a score of 75.3% on SWE-Bench Verified. Our work provides both a critical resource and a robust methodology to advance the next generation of coding agents.