Coding agents can invest substantial effort in repository exploration and testing yet still produce unsuccessful patches. Existing online steering methods review trajectories periodically or invoke an advisor when a monitor detects behavioral problems; frequent reviews are costly, and the absence of anomalous behavior does not mean that an agent is on the right track. We propose PAUSE, a framework for steering coding agents with a single planning intervention. Agents reach their first source-code edit at a median of 60% of trajectory progress, after exploration, bug reproduction, and testing. PAUSE intercepts this edit request before it executes, which requires no learned monitor, no search over checkpoints, and no extra model call (search-free). A separate Planner reads the task and the pre-edit transcript and writes one repair plan; the same agent, the Executor, resumes with this plan, keeping its context and workspace. PAUSE applies online, during a live run, and offline, from a recorded trajectory prefix. We evaluate PAUSE on SWE-bench Verified and SWE-bench Multilingual with two Executors and three agent harnesses. With Qwen3-coder as Executor and GPT-5.6 Sol as Planner, online steering improves resolution by 16.8% on Verified in OpenHands and by 14.7% on Multilingual in SWE-agent. In SWE-agent, PAUSE outperforms LivePlan, a rule-triggered advisor, by 3.2% on Verified and 3.7% on Multilingual at similar Planner cost, and is comparable to SICA's periodic overseer at about 11-15% of its Planner cost. Offline, PAUSE recovers 35.4% of eligible failed Verified tasks, and resuming from the recorded prefix uses 33-41% fewer Executor tokens than restarting.
Figures & tables
Method
Trigger
Timing model call
Needs detected problem
Interventions
SWE-PRM ( Gandhi et al., 2025b )
Fixed step interval
Yes
Yes
Repeated
SICA ( Robeyns et al., 2025 )
Overseer-chosen interval
Yes
Yes
Repeated
LivePlan ( Liu et al., 2026b )
Rule-detected anomaly
No
Yes
Repeated
PAUSE (Ours)
First source-code edit
No
No
One
Table 1. Online steering methods compared by how they decide when to guide. Trigger : the event that starts an intervention. Timing model call : whether a model must be called to decide when to intervene. Needs detected problem : whether guidance is given only after a problem in the trajectory is detected. Interventions : how many times guidance is inserted into a single run.
Figure 1. Actions in 500 Qwen3-coder runs on SWE-bench Verified. The curve marks first source-code edits (median 60.0%); 24 runs without one lie below the divider. A heatmap of 500 trajectories over normalized action progress from 0 to 100 percent. Yellow denotes think actions, blue file reads, green shell commands, red file-editor actions, and gray other or finish actions. The first 476 rows are sorted by their first source-code edit, traced by a black curve; a marker on the top edge labels their median position, 60.0 percent. The final 24 rows have no source-code edit and appear below a horizontal divider.
Figure 2. Overview of PAUSE . The Executor investigates the repository by reading code, writing scratch scripts, and running them. When the Executor issues its first source-code edit, PAUSE blocks the request and passes the task and the observed evidence to the Planner. The Planner returns one repair plan, which PAUSE appends to the Executor’s conversation, and the same Executor resumes with its context and workspace preserved. A reconstructed coding trajectory with small tool windows, bold action borders, and ellipses between representative investigation steps. Source reads and a scratch reproduction show that successive evaluations using the same cross-validation generator obtain two folds and then zero folds. Candidate scoring in the sequential feature selector repeatedly passes the original cv object through validation and splitter normalization. PAUSE blocks the first source-code edit. A lavender Planner panel combines these observations into a plan to normalize CV once per fit and propagate the reusable splitter to each candidate. A purple arrow appends the plan to context; the same Executor edits the sequential feature selector and checks that both evaluations retain two folds. The displayed fold counts come from local execution of the actual splitter functions.
Temperature 0.99; top- p 0.95; seed 0; 81,920 output tokens
LivePlan advisor
Temperature 0; 3,000 output tokens
SICA overseer
Temperature 0; 8,192 output tokens
Table 2. Decoding settings and execution budgets.
Method
Planner
Resolved
Resolution (%)
Tokens (M)
Cost ($)
Planner
Executor
SWE-bench Verified, 500 tasks
Baseline
–
251/500
50.2
0
595.5
0
LivePlan ( Liu et al., 2026b )
GPT-5.6 Sol
309/500
61.8 ( +11.6 )
9.2
561.9
26.10
SICA overseer ( Robeyns et al., 2025 )
GPT-5.6 Sol
338/500
67.6 ( +17.4 )
80.9
1056.4
231.71
PAUSE
GPT-5.6 Sol
325/500
65.0 ( +14.8 )
9.0
564.2
26.13
Table 3. Steering compared with monitoring-based methods. Qwen3-coder Executor and GPT-5.6 Sol Planner in SWE-agent on all 500 SWE-bench Verified and all 300 SWE-bench Multilingual tasks. Resolution : resolved tasks over the full benchmark, with the gain over each block’s baseline. Tokens are in millions; Cost covers Planner inference only.
Figure 6
Intervention
Qwen3-coder Executor
Devstral-small Executor
N: no plan
9/223 (4.0%)
14/174 (8.0%)
S: self-reconsideration
13/223 (5.8%)
17/174 (9.8%)
P: Executor Model
12/223 (5.4%)
20/174 (11.5%)
P: Gemini 3.5 Flash Lite
22/223 (9.9%)
12/174 (6.9%)
P: Gemini 3.8 Flash
31/223 (13.9%)
29/174 (16.7%)
P: GPT-5.6 Sol
79/223 (35.4%)
53/174 (30.5%)
Table 4. Offline recovery in OpenHands on SWE-bench Verified across Executors. Each failed trajectory resumes from its restored checkpoint, the last think before the first source-code edit, with its remaining budget. Recovered reports solved eligible failures: 223 for Qwen3-coder and 174 for Devstral-small. N: no plan; S: self-reconsideration; P: plan from the named Planner.
Method
Planner
Resolved
Resolution (%)
Tokens (M)
Cost ($)
Planner
Executor
Baseline
–
228/500
45.6
0
791.9
0
SHERLOC ( Tamoyan et al., 2026 )
Qwen3-30B-Thinking
211/500
42.2 ( − 3.4)
65.7
703.9
0
PAUSE (restart)
Qwen3-30B-Thinking
207/500
41.4 ( − 4.2)
16.0
641.6
0
PAUSE (resume)
Qwen3-30B-Thinking
217/500
43.4 ( − 2.2)
14.5
375.7
0
PAUSE (restart)
GPT-5.6 Sol
293/500
58.6 ( +13.0 )
8.7
519.1
25.75
Table 5. Full-benchmark offline evaluation on SWE-bench Verified. Qwen3-coder in OpenHands, 500 tasks. Resume retains history and workspace with the remaining budget; restart uses the same plan from the initial repository with the full budget. Resume counts add recoveries to original successes and subtract observed regressions. Tokens (M) exclude the original run for continuations; costs cover Planner inference.
Harness
Method
Resolved
Resolution (%)
Tokens (M)
Cost ($)
Planner
Executor
SWE-agent no think tool
Baseline
251/500
50.2
0
595.5
0
PAUSE
325/500
65.0 ( +14.8 )
9.0
564.2
26.13
Pi no think tool
Baseline
185/500
37.0
0
341.9
0
PAUSE
215/500
43.0 ( +6.0 )
6.7
358.8
18.57
OpenHands think tool
Baseline
227/500
45.4
0
372.5
0
Table 6. Transfer across agent harnesses. Online PAUSE with Qwen3-coder Executor and GPT-5.6 Sol Planner on 500 Verified tasks. Think stripped hides think content from the Planner; units follow Table 3 .
Pause position
Common
Paired with PAUSE
Resolved (%)
Tasks
Arm (%)
PAUSE (%)
Only arm / PAUSE
p
PAUSE checkpoint
56/161 (34.8)
–
–
–
–
–
After 20% of actions
43/161 (26.7)
207
27.5
32.9
15 / 26
0.12
Immediately before the edit
50/161 (31.1)
196
30.1
33.2
14 / 20
0.39
After 90% of actions
50/161 (31.1)
167
32.3
34.7
16 / 20
0.62
Table 7. Recovery under different pause positions on SWE-bench Verified. Each arm restores its own prefix of the same recorded failures and obtains its own GPT-5.6 Sol plan. Common : the 161 tasks graded in every arm. Paired : each arm against the PAUSE checkpoint on their largest shared set, with the tasks solved only by the arm or only by PAUSE and the exact McNemar p -value.
Autonomous coding agents solve repository issues by reading code, running commands, editing files, and submitting patches. Extra inference-time compute yields gains only when it produces a useful repair and supplies reliable evidence for choosing one. Three behaviors decide both, and we argue they are teachable rather than byproducts of scale, so a policy can carry them instead of a scaffold. Location diversity remains narrow, since attempts return to the same site and extra samples add no coverage. Edit diversity is left unexploited, since methodologies that differ resolve complementary issues no single run reaches. Verification misleads, since a test the agent writes for its own patch accepts many incorrect ones. Directing search by execution feedback and scoring each patch against its own reverted tree resolves 52.8% of SWE-bench Verified using 48.1% of the agent-steps an eight-sample baseline spends. Training moves these behaviors into the policy. On the 270 issues held out from SFT and RL training, weighted supervised fine-tuning raises pass@1 from 31.9% to 35.2% and pass@8 from 46.7% to 51.1%. A reinforcement objective then trains the verifier against gold-labeled repairs and incorrect variants, crediting the assertions that detect them. It raises pass@1 to 43.0% and pass@8 to 60.7%, lifts verifier precision from 26.8% to 41.7%, and more than halves false acceptance. Resolution improves on two of three out-of-distribution suites and verifier precision on all three, and the gains hold at 7B, 14B, and 30B against published coder baselines.
Muhammad Ahmed Mohsin, Myeongsoo Kim, Kangrui Ruan +3
Code agents resolve 65-70% of SWE-bench Verified issues, but Pass@1 cannot tell us why the rest fail, and, as we show, capable-model failures are systematically misdiagnosed without trajectory data. We introduce TRAJEVAL, a training-free decomposition of agent trajectories into reference-patch-aligned search, read, and edit stages, and apply it across 16,758 trajectories spanning three architectures and seven models. The dominant failure of capable models is not localization: 60-69% of failures on SWE-Agent and OpenHands reach and edit the correct functions yet still produce incorrect patches, and the pattern persists for most models on the bash-only LiveSWEAgent. Within this Edit-Quality residual, we identify Coherence Collapse, where the agent reaches correct code and then overwrites or thrashes it, as the largest theme, replicating across SWE-bench Verified and the multilingual PolyBench Verified. In 5 cases, the agent produces a patch bit-identical to the gold reference mid-trajectory and destroys it later; an edit-commit checkpoint recovers all 5 against the SWE-bench Docker harness. A reference-free consensus-driven variant yields a directional +3.0 pp Pass@1 measurement on GPT-5 (p=0.08).
Current code-agent benchmarks primarily evaluate localized issue resolution within a single target repository, leaving under-tested many software engineering tasks that require external knowledge or broader repository-level changes. We introduce BeyondSWE, a 500-instance benchmark drawn from 246 real-world GitHub repositories to evaluate code agents beyond single-repository bug fixing. BeyondSWE covers four representative settings: cross-repository issue resolution, domain-specific issue resolution, dependency-driven migration, and document-to-repository generation, spanning both broader knowledge scope and broader resolution scope. Our evaluation shows that BeyondSWE remains far from saturated: the best OpenHands-based agent reaches 46.12 average score, while the strongest Codex harness with GPT-5.4 (xhigh) reaches 56.65 under a search-aware prompt. To study whether external information access closes this gap, we use SearchSWE as a controlled diagnostic baseline for search-augmented coding. Search access improves most models and substantially helps some tasks, but the gains remain limited and uneven, showing that current agents still struggle to convert retrieved information into precise, version-compatible, and locally actionable code changes. These results suggest that deep search for coding remains an open problem: progress requires agents that can reliably combine external evidence with repository-local reasoning and execution-based verification.