SWE-VERIFIER: Selective Verification for Cost-Efficient Coding Agents

Kaijian Zou; Vaibhav Chennareddy; Yunxiang Zhang; Naihao Deng; Longju Bai; Jiaxin Pei; Rada Mihalcea; Lu Wang

cs.LGOct 21, 2026

SWE-VERIFIER: Selective Verification for Cost-Efficient Coding Agents

Authors: Kaijian Zou, Vaibhav Chennareddy, Yunxiang Zhang, Naihao Deng, Longju Bai, Jiaxin Pei, Rada Mihalcea, Lu Wang

Organizations: University of Michigan · The University of Texas at Austin

Abstract

Coding agents spend substantial computation generating actions and processing long interaction histories. Poor action choices can steer agents into low-value states, wasting computation and increasing API costs. Sampling and verifying alternative actions can improve the agent's decisions, but applying this procedure at every step adds substantial overhead. We introduce SWE-Verifier, a lightweight generative model that selectively triggers candidate sampling and verification. It scores the coding agent's proposed action to determine when additional candidate sampling is needed, then ranks the resulting actions before execution. We train SWE-Verifier from Qwen3.5-4B using preference data that groups contrasting actions with their resulting environment observations and rubric-guided teacher scores. Using the mini-swe-agent harness, SWE-Verifier improves resolution from 75.47% to 77.20% on MiniMax-M2.7 at 1.09 times cost and generalizes to DeepSeek-V4-Flash without training on its actions, improving resolution from 72.60% to 74.27% at 1.13 times cost. Across both models, SWE-Verifier verifies fewer than 4% of steps and achieves comparable or higher resolution than the trajectory and all-step baselines at substantially lower cost. These results demonstrate that selective candidate sampling and lightweight verification provide complementary gains in coding-agent performance and inference efficiency. Our model, code, and data are available at https://github.com/zkjzou/SWE-Verifier.

Figures & tables

Figure 1: Verification strategies for coding agents on the full 500-task SWE-Bench Verified set. (a) Trajectory-level verification selects among completed rollouts, all-step verification compares candidates at every step, and selective verification compares candidates only at selected steps. (b) Resolution gains and (c) total costs on MiniMax-M2.7: SWE-Verifier improves resolution by 1.73 percentage points at 1.09

\times

total cost. Both are relative to the no-verifier baseline.

Figure 2: Overview of SWE-Verifier . (a) We construct execution-aware preference data by collecting successful trajectories, sampling and filtering candidate actions, replaying them for environment feedback, and obtaining rubric-guided teacher labels. (b) We train a Qwen3.5-4B verifier to generate outcome predictions, progress assessments, rubric evaluations, and action scores. (c) At inference, the verifier scores the initial action to decide whether to generate alternatives, then ranks candidates when verification is invoked.

Policy	Strategy	Verifier Setting	Resolve (%) $\uparrow$	Total Cost ( $\times$ ) $\downarrow$	Steps (Verified %)
MiniMax-M2.7	None	No verification	75.47 $\pm$ 1.40	1.00	46.38 (0.00%)
	Trajectory	LLM-as-a-Verifier@3	75.85 $\pm$ 0.87	3.50	–
		Oracle@3	82.20	3.00	–
	All-step	Self-Verification	75.27 $\pm$ 1.86	3.35	38.51 (100.00%)
		Qwen3.5-4B	76.40 $\pm$ 0.69	2.26	39.98 (100.00%)
	Selective	Untrained	76.93 $\pm$ 1.21	1.20	44.05 (12.61%)

Table 1: Verification strategies on the full 500-task SWE-Bench Verified set. DeepSeek-V4-Flash uses correct-high reasoning. Untrained uses the base Qwen3.5-4B model. Steps are mean executed actions, with verification rates in parentheses. Costs are normalized to each policy’s no-verifier baseline; e.g., 1.09

\times

means 9% more cost than no verification. Among non-baseline, non-oracle settings, bold/underline mark the best/second-best resolution, steps, and costs within each policy. Across both policies, selective verification improves resolution with substantially lower cost than trajectory-level or all-step verification.

Gate	Selector	Resolve (%) $\uparrow$	Steps (Verified %)	Policy Cost ( $\times$ )	Total Cost ( $\times$ )
No verification	–	68.48 $\pm$ 3.48	57.89 (0.00%)	1.00	1.00
Threshold $\tau=0.2$
SWE-Verifier (Untrained)	MiniMax-M2.7	71.12 $\pm$ 3.48	55.50 (5.41%)	1.05	1.18
	SWE-Verifier (Untrained)	70.63 $\pm$ 0.57	59.43 (5.54%)	1.16	1.16
SWE-Verifier (Score Only)	MiniMax-M2.7	73.27 $\pm$ 2.62	59.21 (5.13%)	1.14	1.29
	SWE-Verifier (Score Only)	71.95 $\pm$ 3.78	61.45 (5.79%)	1.24	1.24

Table 2: Gate/selector ablation for MiniMax-M2.7 on 202 tasks, comparing Untrained and Score Only gates paired with either MiniMax-M2.7 or the corresponding lightweight verifier as selector at two thresholds. The trained Score Only gate attains the highest resolution at both thresholds when paired with MiniMax-M2.7, indicating that a lightweight learned gate can reserve a stronger selector for the decisions where it is most useful.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

	Resolve	$\Delta$ Resolve	Steps	Verify Rate	Policy Cost	Total Cost
MiniMax-M2.7 baseline	75.5%	+0.0	46.4	0%	1.00 $\times$	1.00 $\times$
Trajectory-level upper bound
Oracle@3	82.2%	+6.7	139.1	–	3.00 $\times$	3.00 $\times$
Step-level verifier
All Steps
verifier ( $n=3$ )	76.2%	+0.7	38.2	100%	1.15 $\times$	2.16 $\times$

Appendix

Table 3: MiniMax-M2.7 results on the full 500-task SWE-Bench Verified set. Each step-level and small-model verification setting reports a single run on all 500 tasks. The no-verifier baseline averages three runs, and Oracle@3 selects a successful rollout whenever one exists among those three runs.

\Delta

Resolve is measured against the averaged no-verifier baseline. Verify Rate is the fraction of trajectory steps where candidate actions are verified. Policy Cost counts only MiniMax-M2.7 policy tokens; Total Cost includes verifier-selection calls but excludes label-gate cost for selective-step rows. Small trained verifier calls are treated as near-zero relative to frontier policy calls, so their Total Cost equals Policy Cost.

Figure 5: Phase composition and cross-setting phase stability over normalized trajectory position. Colored areas show the baseline phase distribution. Lines show the fraction of steps whose phase matches the baseline within each position bin. The verifier largely preserves the baseline workflow structure, while verifier-as-tool differs most in the early and middle bins before converging later.

Verification strategy	Policy input cost	Policy output cost	Verifier input cost	Verifier output cost
Baseline	$x$	$y$	$0$	$0$
Trajectory-level	$nx$	$ny$	$V_{\mathrm{traj}}^{\mathrm{in}}$	$V_{\mathrm{traj}}^{\mathrm{out}}$
Step-level	$(1-z)x[1+\alpha c(n-1)]$	$(1-z)y[1+c(n-1)]$	$(1-z)cx$	$(1-z)c\beta y$
Small step-level verifier	$(1-z)x[1+\alpha c(n-1)]$	$(1-z)y[1+c(n-1)]$	$\approx 0$	$\approx 0$

Appendix

Table 4: Cost decomposition for the verification strategies. Here

x

and

y

are the baseline policy input and output costs,

n

is the number of trajectories or candidate actions,

z

is the fractional trajectory-length reduction,

c

is the realized verification rate with

c=1

for every-step verification,

\alpha

is the cached-input price multiplier,

\beta

is the verifier scoring cost ratio, and

V_{\mathrm{traj}}^{\mathrm{in}}

and

V_{\mathrm{traj}}^{\mathrm{out}}

are the outcome-verifier costs for trajectory-level scaling. The first candidate incurs input cost

x

; each later candidate incurs cached-input cost

\alpha x

Panel of Figure 6 — Figure 6: cookiecutter trajectory prefix before candidate selection. Continued parts use the same figure number.

Panel of Figure 7 — Figure 7: cookiecutter candidate-action slate shown to the teacher verifiers before selection, continued.

Panel of Figure 8 — Figure 8: cookiecutter teacher outputs from the basic blind and observation-aware verifiers.

Panel of Figure 9 — Figure 9: cookiecutter checklist-based verifier summary, checklist, candidate scores, and final choice.

Panel of Figure 11 — Figure 11: Resolved and unresolved MiniMax evidence around the post-validation patch-construction decision.

Coding Agents

Coding agents use language models, tools, and feedback to modify software and complete engineering tasks.

See all 139 papers

SWE-Critix: Post-Training Generalist Verifiers for Assessing Coding-Agent Trajectories
Oct 9, 2026Jiajun Hu, Chun Yong ChongLLM Agent Verification AI Coding Agents
Leibniz Research Center, Huawei, Hong Kong SAR, China · School of Information Technology, Monash University Malaysia, Subang Jaya, Malaysia
The Verification Horizon: No Silver Bullet for Coding Agent Rewards
Jun 24, 2026Binghai Wang, Chenlong Zhang, Dayiheng Liu +10Reward Modeling Reward Hacking
Qwen Team
Teaching Agents to Code Reliably
Oct 2, 2026Muhammad Ahmed Mohsin, Myeongsoo Kim, Kangrui Ruan +3Repository-Level Program Repair AI Coding Agents
Stanford University · AWS AI Labs

Oct 9, 2026J/K move · Enter open · S save

SWE-Critix: Post-Training Generalist Verifiers for Assessing Coding-Agent Trajectories

Jiajun Hu, Chun Yong Chong

Leibniz Research Center, Huawei, Hong Kong SAR, China · School of Information Technology, Monash University Malaysia, Subang Jaya, Malaysia

LLM Agent Verification AI Coding Agents

Open paper AskPDF arXiv

Coding agents are increasingly deployed for end-to-end software engineering tasks, where a readily available oracle evaluator that defines the success criteria is often absent. This motivates verifications for coding agents that can reliably judge and explain their behavior. However, verification is non-trivial as coding agents are increasingly capable of solving long-horizon software engineering tasks. Their execution trajectories can be extremely long, containing hundreds of iterative interactions with code repositories and external tools. Existing verifiers either rely on general-purpose LLMs without specialized training or train dedicated critics that predominantly focus on producing scalar rewards, highlighting the relatively limited exploration of training-based generalist verification. We introduce SWE-Critix, a post-training pipeline for training generalist verifiers of coding-agent trajectories. Given a task and its execution trajectory, the learned verifier predicts the completion label and generates a natural-language analysis that explains its judgment. Our pipeline combines supervised fine-tuning (SFT) with reinforcement learning (RL) and trains a series of 30B models on trajectories from CodeForge-Preview. On a held-out test set, SWE-Critix improves classification accuracy from 57% for the base Qwen3-30B-A3B reasoning model to 66% after SFT and 78% after RL. We release our models, datasets, and training code to facilitate further research.

Abstract

Coding Agents

Explore similar work

SWE-Critix: Post-Training Generalist Verifiers for Assessing Coding-Agent Trajectories

The Verification Horizon: No Silver Bullet for Coding Agent Rewards

Teaching Agents to Code Reliably