cs.AIAug 23, 2026
SaveDisagree to Explore, Agree to Commit: Routing-Guided Test-Time Scaling for Software Agents
Organizations: Fudan University · Shanghai Innovation Institute
Abstract
Software-engineering agents solve repository-level tasks through long, stochastic tool-use trajectories, and repeated attempts often find fixes missed by one run. Test-time scaling is difficult because patches lack canonical answer forms, while sibling actions from a shared prefix are correlated. We study whether native MoE router traces can guide steering and selection without an external judge or selection-time test execution. Our analysis shows that routing provides a robust behavioral role signal; token-granular readouts and decision-matched comparison sets turn it into effective control. We therefore introduce Risa (Routing-Informed Steering and Arbitration): within trajectories, routing encourages diverse exploration and controlled convergence during patch commitment; across separately sampled trajectories, agreement at informative patch positions selects a final candidate. We evaluate on SWE-bench Verified using open-weight sparse MoE agents across scales and reasoning-effort settings. Risa's routing arbitration raises the macro-average resolved rate from 44.9% under uniform sampling to 48.2% on the gpt-oss family, matching text consensus without answer-string matching, and it transfers to Qwen3.6, where it improves on uniform choice and matches text consensus on the full 500-task benchmark.
Figures & tables
Figure 1: Routing supports decision-matched control. (a) Patch-writing fingerprints overlap across outcomes, motivating relational use. (b) Behavioral roles separate cleanly. (c) Across final-step generations, the least-probable quarter carries of total token surprisal (median and interquartile range; dotted line: equal contribution).
Figure 2: Saturation, decision-token localization, and window choice. Left: layer–expert occupancy (the fraction of cells receiving nonzero mass) grows with span length. Middle: splitting each generation into ten deciles by next-token probability, routing agreement between separately sampled attempts is highest in the lowest-probability decile and falls as probability rises ( attempt pairs; mean and interquartile range); the final decile rebounds, consistent with highly predictable tokens being routed alike. Right: the context-identity AUC (same generation context versus different contexts) decays with length; whole-span routing retains coarse context, while opening and decision-token windows preserve finer separation.
| candidate length | ||
|---|---|---|
| fingerprint read at | shorter half | longer half |
| whole span | .59 | .52 |
| opening 64 tokens | .49 | .51 |
| trailing 64 tokens | .51 | .52 |
| decision tokens | .69 | .52 |
Table 1: Pairwise AUC for separating resolving from non-resolving sibling branches across 3,660 pairs from 423 mixed-outcome steps, split at the median candidate length. Decision tokens give the strongest readout, with their clearest advantage on shorter candidates.
Figure 3: Why the reference set matters. Left: decision tokens yield higher cross-attempt agreement than other positions. Middle: same-prefix outcome distributions overlap, motivating history-relative novelty. Right: across separately sampled attempts, resolving patches receive higher mean agreement with their attempt group.
Figure 4: Risa : one MoE routing representation, three decision-matched comparisons. Exploratory actions are compared with recent executed history; patch-writing actions with their role-gated peers; and accumulated final patches with separately sampled attempts at decision tokens.
| Attempt pool | Terminal arbitration | Upper bound | |||||||
| Model | Effort | Eligible | Yield | Steps | Uniform | Text | Risa | Risa-H | Oracle |
| gpt-oss-20b | low | 496 | 79.1 | 9.5 | 31.8 | 34.5 (+ 2.7) | 34.1 (+ 2.3) | 34.7 (+ 2.9) | 48.4 |
| medium | 498 | 94.8 | 26.5 | 45.7 | 47.8 (+ 2.1) | 48.2 (+ 2.5) | 47.6 (+ 1.9) | 63.3 | |
| high | 498 | 80.0 | 43.1 | 45.3 | 49.2 (+ 3.9) | 51.0 (+ 5.7) | 50.0 (+ 4.7) | 62.2 | |
| gpt-oss-120b | low | 497 | 93.0 | 14.8 | 38.7 | 40.0 (+ 1.3) | 41.2 (+ 2.5) | 40.4 (+ 1.7) | 53.3 |
| medium | 497 | 99.5 | 28.8 | 50.6 | 54.5 (+ 3.9) | 53.5 (+ 2.9) | 54.9 (+ 4.3) | 66.6 | |
Table 2: Final-patch arbitration on SWE-bench Verified (% resolved) over common Risa -steered pools. Rates condition on at least two graded patches. The full -task gpt-oss grid yields – eligible tasks per condition; the full -task Qwen3.6-35B-A3B condition has . Risa-H uses Text-first routing tie-breaking; Uniform is expected random choice and Oracle the success union. Parentheses in deployable columns show gains over Uniform; bold shading and underlining mark the best and second-best deployable estimates.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
| Terminal selector | Macro (%) | vs. Risa |
|---|---|---|
| Uniform | 44.9 | — |
| patch-length prior | 46.7 | |
| file-set Jaccard | 46.9 | |
| changed-line multiset | 46.9 | |
| TF–IDF cosine centrality | 47.5 | |
| character -gram Jaccard | 47.5 |
Table 3: Terminal selection-rule ablations on the six gpt-oss conditions ( – eligible tasks each; macro-average of the six). The right column is the paired difference against Risa , with bootstrap resampling clustered on SWE-bench task. Surface variants change how patch centrality is measured; routing concentration uses the identical fingerprint but no cross-attempt comparison. The -point separation identifies cross-attempt agreement, rather than intrinsic concentration, as the useful routing signal.
| Score | AUC (95% CI) |
|---|---|
| peer similarity alone | .558 |
| mode fraction alone | .523 |
| entropy alone | .512 |
| concentration alone | .486 |
| deployed | .520 |
| equal weights | .522 |
Table 4: Components of the write score on the strong-label bank. Mean peer similarity provides the clearest standalone signal, and removing it produces the largest degradation among the combination ablations. The and terms act as stability guards; all coefficients were frozen before the campaign and remain fixed across tasks, models, and effort settings.
| Model | Pool | Unif. | Text | Rout. | Hyb. | Oracle |
|---|---|---|---|---|---|---|
| 20b med | steered | 45.5 | 49.5 | 50.5 | 50.0 | 63.5 |
| uncontrolled | 45.4 | 50.0 | 48.0 | 51.0 | 63.0 | |
| 120b med | steered | 49.9 | 54.3 | 53.3 | 55.3 | 65.3 |
| uncontrolled | 53.5 | 56.3 | 54.8 | 55.3 | 71.9 |
Table 5: Pipeline composition and sampling-budget comparison on the fixed -task subset (% resolved, paper grading convention; tasks for 20b and with four completed attempts for 120b ). End to end, the Risa (steered Routing) reaches against for the no-controller pipeline (uncontrolled Uniform) at 20b , a task-paired bootstrap difference of points, CI ; at 120b , Risa-H (steered Hybrid) reaches against for uncontrolled Uniform ( points, CI ). Within the uncontrolled pools, which cost about of the candidate tokens per step, routing arbitration still adds and points over Uniform and hybrid arbitration and points, so the terminal level is effective at a small fraction of the generation budget.
| Selector | Resolved (%) | Gain vs. Uniform |
|---|---|---|
| Uniform | 41.7 | — |
| Text | 45.0 | |
| Risa | 45.2 | |
| Risa-H | 45.6 | |
| Oracle | 50.6 |
Table 6: Cross-family reproduction: final-patch selection (% resolved) over separately sampled Qwen3.6-35B-A3B attempts on the full -task benchmark ( eligible tasks). Eligibility requires at least two graded patches. Brackets are task-bootstrap confidence intervals for paired gains over Uniform. Bold and underlining mark the best and second-best deployable estimates.
| parameter | value |
|---|---|
| attempts per task | 4 |
| candidates per step | 16 |
| sampling temperature | 1.0, top- 1.0 ( gpt-oss ); 0.6, top- 0.95, top- 20 ( Qwen3.6 ) |
| decision-token fraction | lowest 25% of patch tokens by teacher-forced probability |
| step fingerprint span | tool-call segment of the candidate (fallback: trailing 64 generated tokens) |
| gpt-oss reasoning efforts | native low / medium / high setting |
Table 7: Final configuration. All arms of an ablation share every value except the selection rule under study.
| decision-token fraction | 5% | 10% | 25% | 50% | 100% |
|---|---|---|---|---|---|
| offline AUC (cross-attempt) | .644 | .648 | .657 | .657 | .639 |
| realized macro-avg (%) | — | 48.0 | 48.2 | 48.2 | — |
| per-condition realized spread across fractions: at most points; the selected 25% setting attains the top macro-average. | |||||
| write-rule threshold : pairwise AUC .548/.572/.536/.580 | |||||
| step-level fraction : peer-similarity AUC .554/.545/.544 | |||||
| exploration , pool : .546/.545/.548/.551; pool : .533/.537/.547/.551 | |||||
Table 8: Sensitivity of the selected routing-window and neighborhood hyperparameters. Candidate count, attempt count, step budget, and the four write-rule coefficients are held fixed throughout.
Figure 5: Similarity to one’s own recent routing history. Left: within-step outcome balance supports the diversity-prior interpretation ( ). Right: across rollouts, mean history similarity strongly separates resolved from unresolved attempts ( ), motivating a trajectory-level diversity prior.
More in this topic
Software Engineering Agents
Software Engineering Agents are AI systems that use code, tools, and feedback to perform software development tasks.
See all 349 papers