A coding agent repeatedly makes decisions as it repairs code. These decisions shape repair trajectories and affect the token cost of successive model requests. We ask which repair decisions can be delegated to a low-cost probabilistic judge to reduce token use while maintaining repair quality. The judges answer questions with probabilities over predefined answers, without generating explanations. We evaluate judgment quality across repair stages and test how individual interventions change subsequent repair work. We compare these interventions with unguided repair, which uses no judge, and with the same workflow using the repair model itself as the judge. The benefit depends on the work an intervention changes, rather than the cost of the judgment alone. Based on these studies, we propose JevCoder, a workflow built around Jev, a low-cost probabilistic judge. It combines code-snippet recommendations with action-type guidance and uses judgments to determine when to submit. We evaluate three repair models, each on the same 400 tasks: 200 from SWE-bench Verified and 200 from SWE-Bench Pro Verified. Compared to unguided repair, JevCoder reduces mean repair-model token use by 21.6%-50.6%, with higher observed repair rates across all three models.
Figures & tables
Figure 1: In a simplified recorded SWE-Bench Pro Verified case, JevCoder uses Jev’s relevance probabilities to rank candidate snippets and supply selected code as the coding agent’s initial context.
Figure 2: JevCoder delegates judgments to a shared probabilistic judge: Jev. Before repair, a program retrieves candidate code for the issue, Jev scores each snippet’s relevance, and the highest-scoring snippets enter the repair model’s initial context. After each command, program rules select relevant state questions for Jev and use its answers to suggest an action type. The repair model chooses the specific command. The stopping policy depends on the model class. For the larger API models (DeepSeek, Gemini), a Choice question on the latest check output (Pass: a previously failing check now passes; Rest: other outcomes) triggers direct submission. For the smaller model (Qwen), repeated No answers to whether the next step adds information trigger a reminder and then submission.
Judgment
Metric
Jev
Comparator
Δ
Comparator model
Opening — Initial context
File relevance (Binary)
Top-1 hit (%) ↑
75.9
29.0
+46.9
Qwen3.5-2B
File relevance (Score)
Top-1 hit (%) ↑
94.9
88.1
+6.8
Jev (Choice) ‡
Snippet recommendations
Selection recall (%) ↑
81.6
78.0
+3.6
DeepSeek
During repair — State and action judgments
Code located
AUROC ↑
0.738
0.701
+0.038
DeepSeek/Qwen
Table 1: Judgment quality and snippet selection. Higher is better; bold marks the better point estimate. Δ is Jev minus the comparator (percentage points for percentage metrics; absolute differences for AUROC), computed before rounding. File relevance uses 100 candidate files for Binary and 10 for Score. Selection recall is measured over reference lines covered by candidate snippets. State AUROCs pool DeepSeek and Qwen inputs, each evaluated against its same-model judge. ‡ Jev (Choice) returns probabilities over the same ten candidate files.
Table 2: Component comparisons relative to matched unguided repair on N tasks. Values are changes in repair rate (pp) and in mean repair-model tokens (M); bars show the direction and size of each change. Green and red mark improvement and degradation; grey marks small changes (absolute change below 1 pp or 0.01M). A check mark denotes adoption in JevCoder. 95% intervals and full comparisons appear in Appendix D .
Condition
Repair (%) ↑
Repair-model tokens (M) ↓
Runtime (s) ↓
Repair
Judging
Total
DeepSeek-V4.1-Flash
Baseline
76.25
1.978
—
1.978
181.7
Same-model judging
77.00 ( +0.75 )
1.441
0.265
1.706 ( −13.7% )
287.6 ( +58.2% )
JevCoder
77.25 ( +1.00 )
1.550
†
1.550 ( −21.6% )
162.7 ( −10.5% )
Qwen3.8-27B
Table 3: JevCoder evaluation on 400 matched tasks per condition. Tokens are per-task means; runtimes are medians. Parentheses indicate changes from Baseline (repair: pp; token means and runtime medians: %). Bold marks the best repair rate, Total tokens and runtime within each model. † : Jev tokens are excluded from repair-model totals and reported separately (§ 7 ).
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Judgment
Answers
Question
Snippet relevance
yes / no
Considering only state.blocks.{id} : does this exact source directly implement, branch on, carry data for, call, or test the behavior described in state.issue ? The current buggy implementation counts; shared vocabulary alone does not. Source in other blocks is not evidence for this block.
Code located
yes / no
Has the agent opened code the issue names or clearly implies (function, class, file)?
Issue reproduced
yes / no
Does the latest output show the misbehaviour the issue describes?
Post-edit check
yes / no
Since the last source edit, does any output show the issue’s scenario behaving as the issue expects?
Stalled repair
yes / no
Do the last 6 commands pursue the same goal without new information?
Issue-related activity
yes / no
Do the agent’s last few commands still work on code related to the behavior the issue describes?
Appendix
Table 4: Fixed judge questions. Stalled repair, code located, issue reproduced, and post-edit check drive action-type guidance; the proposed-command question drives command blocking and Qwen’s no-progress submission; the successful-check question drives automatic submission; snippet relevance drives code-snippet recommendations. Issue-related activity and patch outcome are evaluated offline only. Field names in typewriter font refer to fields of the evidence payload.
DeepSeek trajectories
Qwen trajectories
Judgment
Positive/total
Jev
Same-model
Positive/total
Jev
Same-model
Code located
1,525/1,598
0.751
0.687
1,494/1,599
0.731
0.776
Issue reproduced
115/1,234
0.832
0.845
130/1,599
0.816
0.771
Stalled repair
44/1,578
0.914
0.810
322/1,599
0.971
0.972
Post-edit check
158/443
0.725
0.735
56/508
0.845
0.831
Appendix
Table 5: Repair-state AUROC on identical inputs for each repair model. Counts give positive labels and total sampled states; each model’s states combine both benchmarks. The pooled results appear in Table 1 .
Comparison
Repair model
Tasks
Δ Repair rate (pp)
Mean Δ Tokens (M)
File recommendations (Binary)
DeepSeek
118
+0.50
−0.354
[−4.97,+5.78]
—
File recommendations (Score)
DeepSeek
118
−3.58
−0.372
[−10.06,+2.89]
[−0.730,−0.050]
Snippet recommendations
DeepSeek
118
+3.89
−0.206
[−2.28,+10.47]
[−0.566,+0.167]
Appendix
Table 8: Paired component comparisons behind Table 2 , with 95% intervals below the estimates. Each change is the component condition minus matched unguided repair, except snippet withdrawal, which is withdrawal minus full retention. The 118- and 116-task SWE-Bench Pro Verified samples are reweighted to the original failure/success proportions; the 72-task samples are unweighted. For binary file recommendations, intervals are reported separately for input and output tokens in the text.
When code-repair agents fail, final tests reveal that a patch is wrong but not which earlier commitment caused it or which software obligation failed. Poison Valleys traces this path in 1,844 model requests on two synthetic Python cases: PersistQueue adds atomic batch claiming to a disk-backed queue, while Dijkstra adds source-sensitive filtering to shortest-path search. We define plausible but incorrect software mechanisms, place each in a model's hypothesis, implementation plan, or both, and evaluate the generated code with an Oracle and targeted diagnostic coordinates. Moving the misconception from hypothesis to plan increased failure by 32.2% in PersistQueue and 28.9% in Dijkstra after matching the experimental mix. Different misconceptions produced distinct, model-dependent patterns of violated obligations, although several could converge on the same behavior. This creates a measured connection from upstream misconception to code mechanism and failure form. Code, data, and demo: https://poisonvalleys.org.
Large language model agents can repair real repository issues, but they often spend large context budgets on whole-file reads, broad searches, and long terminal outputs where useful evidence is mixed with irrelevant code and logs. This paper presents ContextSniper, AntTrail's code-repair module for precision evidence selection in repository-level program repair, part of AntTrail's broader agent-memory engine. AntTrail is available at https://gitcode.com/datagallery/AntTrail. ContextSniper indexes code and action memory as three abstract levels, retrieves candidates with a hybrid ranker, filters long tool output through an intention-aware context gate, and returns compact evidence packets while keeping full source recoverable on demand. In a matched 50-task-per-condition comparison on SWE-bench Lite (same tasks, baseline vs.\ ContextSniper), ContextSniper reduces total token use by 51.5% and logged cost by 36.4% for OpenClaw, and by 38.9% and 27.3% for Claude Code, with submitted-resolution rates essentially unchanged in both host-agent settings. In a separate five-task comparison, ContextSniper beats existing memory- and RAG-style integrations on token efficiency. These results suggest ContextSniper can substantially cut token and cost overhead for repository-level repair agents without a measurable loss in repair quality. The evaluation harness for this study is available at https://gitcode.com/lukchiwang/ContextSniper.
Chiwang Luk, Matin Mohammad Najafi, Zhifeng Jia +6
Huawei · HKUST(GZ) · Nanyang Technological University
Large language model (LLM) code-repair systems fix faulty programs by repeatedly generating and testing revisions. Each revision is costly, making the choice of which repair to pursue important for efficiency. Existing search policies rely on test pass rates and search statistics, ignoring the details of test failures and proposed edits. We introduce CodeTriage, a learned policy that prioritizes edits before their revised programs are generated and tested. Its 0.6B-parameter model reads the test output, failure diagnosis, and proposed edit, then predicts a distribution over the number of repair steps needed to reach a fix through that edit. These predictions prioritize all pending edits across programs before generating and testing the revised code. On held-out repair tasks, CodeTriage finds fixes with 21% fewer program revisions and 12% fewer output tokens than the strongest baseline on each metric. Without retraining, it achieves the highest average success across search budgets for each of 11 unseen repair LLMs. Ablations show that both the failure context and edit text, as well as comparisons across programs, improve search. We also release a corpus of 12.2 million program versions from four LLMs, organized into repair trees with proposed edits and test results.