Teaching Agents to Code Reliably

Muhammad Ahmed Mohsin; Myeongsoo Kim; Kangrui Ruan; Shweta Garg; Varun Kumar; Murali Krishna Ramanathan

cs.AIOct 2, 2026

Teaching Agents to Code Reliably

Authors: Muhammad Ahmed Mohsin, Myeongsoo Kim, Kangrui Ruan, Shweta Garg, Varun Kumar, Murali Krishna Ramanathan

Organizations: Stanford University · AWS AI Labs

Abstract

Autonomous coding agents solve repository issues by reading code, running commands, editing files, and submitting patches. Extra inference-time compute yields gains only when it produces a useful repair and supplies reliable evidence for choosing one. Three behaviors decide both, and we argue they are teachable rather than byproducts of scale, so a policy can carry them instead of a scaffold. Location diversity remains narrow, since attempts return to the same site and extra samples add no coverage. Edit diversity is left unexploited, since methodologies that differ resolve complementary issues no single run reaches. Verification misleads, since a test the agent writes for its own patch accepts many incorrect ones. Directing search by execution feedback and scoring each patch against its own reverted tree resolves 52.8% of SWE-bench Verified using 48.1% of the agent-steps an eight-sample baseline spends. Training moves these behaviors into the policy. On the 270 issues held out from SFT and RL training, weighted supervised fine-tuning raises pass@1 from 31.9% to 35.2% and pass@8 from 46.7% to 51.1%. A reinforcement objective then trains the verifier against gold-labeled repairs and incorrect variants, crediting the assertions that detect them. It raises pass@1 to 43.0% and pass@8 to 60.7%, lifts verifier precision from 26.8% to 41.7%, and more than halves false acceptance. Resolution improves on two of three out-of-distribution suites and verifier precision on all three, and the gains hold at 7B, 14B, and 30B against published coder baselines.

Figures & tables

Figure 1: Two behaviors, one reward. Left, exploration. The baseline resubmits the same site ( lib/ascii.py ), so coverage stays narrow, while parallel chains that read first each reach the true defect ( lib/utils.py ), write a reproduction, and submit. Right, verification. The harness creates incorrect patch variants and records which assertions detect each variant. The gate opens only when the test kills every variant, so the reward pays for discrimination.

	Accuracy		Compute				Diversity
Experiment	/500	rate	total steps	att./inst	early-stop	steps/solve	loc.	edit
Self-Verification	184	36.8%	189,371	3.99	0 / 500	1,029	1.10	1.31
Single-Axis Diagnosis	189	37.8%	41,332	1.00	$-$	219	$-$	$-$
Proactive Location Diversity	210	42.0%	138,424	4.00	0 / 500	659	2.46	3.26
Blind Retry (baseline)	213	42.6%	139,632	$-$	$-$	656	$-$	$-$
Reactive Edit Diversity	217	43.4%	131,417	3.67	65 / 500	606	1.84	2.79

Table 1: Efficiency across the test-time procedures on the

500

issues, all-attempts basis, so steps/solve equals total steps over bugs resolved. Dark yellow marks the best value in a column. The highlighted row is our combined procedure, which is strongest on accuracy at near the lowest cost. Single-axis diagnosis is the one-attempt reference, cheapest overall though not the lowest resolution. Methodology diversity is reported in the text and not here, because its

61.0\%

is an oracle union over eight preambles rather than a rate any single deployable procedure reaches. Blind Retry, Independent Retry, and Sample Scaling are baselines in italics. A dash marks not applicable.

	SWE-bench Verified		DeepSWE		SWE-PolyBench		SWE-bench Lite		SWE-bench Pro
Metric	Base	SFT	Base	SFT	Base	SFT	Base	SFT	Base	SFT
Resolution rate
pass@1	$31.9$	$\mathbf{35.2}$	$0.0$	$0.0$	$20.4$	$\mathbf{24.2}$	$28.3$	$\mathbf{36.7}$	$2.3$	$\mathbf{15.7}$
pass@8	$46.7$	$\mathbf{51.1}$	$0.0$	$0.0$	$24.6$	$\mathbf{30.2}$	$30.7$	$\mathbf{39.7}$	$7.4$	$\mathbf{19.6}$
Exploration
Files read before first edit	$1.87$	$\mathbf{2.19}$	$1.26$	$\mathbf{2.12}$	$2.69$	$\mathbf{3.91}$	$2.61$	$\mathbf{3.89}$	$4.71$	$\mathbf{6.81}$

Table 2: Base and SFT results across five evaluation suites. Behavioral measures summarize the evaluated trajectories, diversity measures compare eight attempts per issue, and downward arrows mark metrics where smaller is preferred. Behavioral changes are descriptive rather than independent measures of correctness.

Policy	Oracle@8	Selected@8	Majority vote	Random pick	Sel. eff.
Base	$46.7$	$33.5$	$30.0$	$26.0$	$0.72$
SFT	$51.1$	$38.5$	$35.0$	$30.2$	$0.75$
$+$ RL	$60.7$	$50.5$	$44.0$	$36.0$	$0.83$

Table 3: Coverage and selection on the Verified split held out from training using the existing eight-candidate sets. Oracle@

8

is the fraction of issues with a correct candidate. Selected@

8

is the fraction for which the verifier returns a correct candidate. Selection efficiency is their ratio. No additional repairs are generated for this analysis.

Verifier policy	Wrong rej. (TNR)	Correct acc. (TPR)	Bal. acc.	Accept. cov.	Sel. acc.	FPR $\downarrow$
Base	$70.0$	$68.0$	$69.0$	$66.0$	$52.0$	$30.0$
SFT	$73.0$	$72.0$	$72.5$	$70.0$	$58.0$	$27.0$
Correctness-only RL	$78.0$	$74.0$	$76.0$	$73.0$	$63.0$	$22.0$
Full ( $+$ RL)	$87.4$	$80.0$	$83.7$	$78.0$	$72.0$	$12.6$

Table 4: Verifier discrimination on a frozen bank of labeled repairs shared across policies. The bank contains natural incorrect repairs from multiple generators and correct alternatives. Labels are not exposed to the test-generating policy. Wrong rejection and correct acceptance measure the two sides of discrimination. Metric denominators are defined in Section 3.1 and Appendix B.9 .

Figure 2: Layer-wise self-verification AUC across training stages. The post-RL representation has higher AUC for distinguishing correct and incorrect patches, with the largest differences in the mid-to-late layers (shaded).

	DeepSWE				SWE-PolyBench				SWE-bench Pro
Metric	Base	+RL	SFT	+RL	Base	+RL	SFT	+RL	Base	+RL	SFT	+RL
pass@1	$0.0$	$0.0$	$0.0$	$0.0$	$20.4$	$23.8$	$24.2$	$31.8$	$2.3$	$15.5$	$15.7$	$20.2$
pass@8	$0.0$	$0.0$	$0.0$	$0.0$	$24.6$	$30.2$	$30.2$	$41.6$	$7.4$	$19.4$	$19.6$	$22.8$
Verifier precision	$21.1$	$33.7$	$23.1$	$38.6$	$13.6$	$22.8$	$17.5$	$26.8$	$15.5$	$24.5$	$21.0$	$24.7$
False-positive rate $\downarrow$	$23.0$	$14.6$	$21.1$	$11.1$	$19.0$	$14.6$	$18.1$	$11.3$	$21.7$	$14.5$	$20.9$	$12.7$
Fixpoint (zero-survivor) rate	$3.0$	$3.9$	$5.11$	$8.3$	$0.4$	$2.1$	$3.87$	$7.31$	$1.90$	$4.51$	$2.81$	$4.19$

Table 6: Transfer evaluation from base and SFT initializations. Each initialization is paired with its RL-trained policy. DeepSWE verifier precision uses a fail-to-pass criterion rather than full task resolution.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

	from Base		from SFT
Metric	pre-RL	+RL	pre-RL	+RL
Resolution rate (SWE-bench Verified)
pass@1	$31.9$	$35.9$	$35.2$	$43.0$
pass@8	$46.7$	$53.7$	$51.1$	$60.7$
Verifier quality (vs. gold, held-out)
Verifier precision	$23.5$	$33.1$	$26.8$	$41.7$

Appendix

Table 7: Base and SFT policies before and after RL on the

270

Verified issues held out from training. Pass@

1

measures greedy repair accuracy and pass@

8

measures oracle coverage. Verifier metrics use oracle correctness labels. Assertion counts, test length, and fault-class coverage describe the generated tests.

Configuration	pass@1	Verifier prec.	$\Delta$ vs. full
Full objective	$43.0$	$41.7$	$-$
$\beta=0$ (flat scalar, no step-credit)	$40.7$	$38.4$	$-2.3$
no mutation gauntlet (self-consistency only)	$38.1$	$29.5$	$-4.9$
credit crash-kills (no assertion gate)	$39.6$	$33.0$	$-3.4$
ungated $E$ , $P$ shaping terms	$41.1$	$37.9$	$-1.9$
no in-episode repair loop	$40.0$	$40.1$	$-3.0$

Appendix

Table 8: Ablations of the reinforcement objective on SWE-bench Verified, each removing one component the methodology argues is load-bearing.

\Delta

is the change in resolution against the full objective.

Semantic-mutant adversary	pass@1	Verifier prec.	Fixpoint rate
Weak (same-size, untuned)	$41.1$	$37.5$	$11.8$
Same-family (policy’s sibling)	$41.9$	$39.8$	$13.9$
Strong (frozen, ours)	$43.0$	$41.7$	$15.6$
Deterministic mutants only	$40.4$	$35.0$	$\phantom{0}9.5$

Appendix

Table 9: Robustness to the adversary used to generate semantic mutants. All values are percentages. Every semantic-mutant configuration beats the deterministic-only baseline on all three metrics, so the benefit extends beyond the strong adversary, which leads by

2.6

6.7

, and

6.1

points over deterministic mutants alone.

	Reward weights $w_{c}\!:\!w_{v}$		GRPO group size $k$
Metric	High $w_{c}$	Low $w_{c}$	$k{=}4$	$k{=}6$	$k{=}16$
pass@1	$43.0$	$38.9$	$41.5$	$43.0$	$43.3$
Verifier prec.	$41.7$	$43.2$	$39.0$	$41.7$	$42.6$

Appendix

Table 10: Sensitivity to the relative correctness and verifier reward weights (

w_{c}\!:\!w_{v}

) and the GRPO group size

k

, the number of rollouts drawn per training instance. All values are percentages. The trained setting of Section 2.3 is

w_{c}{:}w_{v}=3.0{:}1.0

k{=}6

, tinted in both blocks because it is the shared reference point. Lower correctness weighting increases verifier precision by

1.5

percentage points but reduces pass@1 by

4.1

. Group size pays off early. Moving from

k{=}4

k{=}6

adds

1.5

pass@1 and

2.7

precision, while going on to

k{=}16

adds a further

0.3

and

0.9

for more than twice the rollouts per instance, which is why we train at

k{=}6

System	Backbone	Scaffold	Setting	Res. (%)
7B backbones
SWE-Gym ( Pan et al., 2024 )	Qwen2.5-Coder-7B	OpenHands	Verified pass@1	$10.6$
SWE-Gym ( Pan et al., 2024 )	Qwen2.5-Coder-7B	OpenHands	Lite pass@1	$10.0$
SWE-agent-LM-7B ( Yang et al., 2025 )	Qwen2.5-Coder-7B	SWE-agent	Verified pass@1	$15.2$
SWE-agent-LM-7B ( Yang et al., 2025 )	Qwen2.5-Coder-7B	SWE-agent	Lite pass@1	$11.7$
Lingma SWE-GPT ( Ma et al., 2024 )	Qwen2.5-Coder-7B	SWESynInfer	Verified pass@1	$18.2$

Appendix

Table 11: Published coding-agent results and our results at

7

14

B, and

30

B. Each row gives the backbone, scaffold, and evaluation setting. Dataset subsets, sampling budgets, and selection procedures differ across systems, so the reported values do not establish a matched comparison.

Variant	Fault class	Changed line	Outcome
V1	logic	595	eliminated (assertion)
V2	value	603	eliminated (assertion)
V3	control	605	eliminated (assertion)
V4	boundary	603	eliminated (assertion)
V5	effect	607	eliminated (assertion)
V6	boundary	597	eliminated (assertion)

Appendix

Table 12: The eight adversarial variants generated for a real fixed patch ( matplotlib__matplotlib-21568 ), by changed line and fault class, with the measured elimination outcome. Every variant is a distinct, behaviour-changing rewrite of the patched region. The harness records each variant’s outcome and fault class but not which individual assertion eliminated it, so we report the per-variant verdict.

Metric	Base	SFT model
Tasks evaluated	$113$	$113$
Fully resolved (all fail-to-pass, no regression)	$0$	$0$
Tasks with $\geq 1$ fail-to-pass test satisfied	$17$	$21$
Tasks clearing $>50\%$ of fail-to-pass tests	$4$	$5$
Mean fail-to-pass pass fraction (best of 8)	$4.0\%$	$4.8\%$
Distribution of best-of-8 fail-to-pass pass fraction (tasks)

Appendix

Table 13: DeepSWE evaluation at pass@8 over all

113

out-of-distribution tasks, comparing the base policy with our supervised fine-tuned model. The strict binary criterion resolves no task for either model, but the fine-tuned model makes greater partial progress. Dark yellow marks the fine-tuned model’s wins, and the one base win is in bold.

	7B			14B
Benchmark (pass@8, %)	Base	SFT	+RL	Base	SFT	+RL
SWE-bench Lite	$16.0$	$22.0$	$24.7$	$23.0$	$30.3$	$34.7$
SWE-PolyBench	$8.0$	$14.4$	$17.8$	$19.8$	$28.8$	$37.4$
SWE-bench Verified	$21.9$	$30.4$	$33.3$	$27.0$	$36.7$	$42.6$

Appendix

Table 14: Effect of model scale. Pass@

8

resolution (

\%

) for the base, supervised (SFT), and reinforcement (+RL) policies at

7

B and

14

B on three SWE-bench-family benchmarks. Both the Base

\rightarrow

SFT gain and the further SFT

\rightarrow

+RL gain hold at both scales and do not diminish with capacity, so the taught search and verification behavior is complementary to raw model scale.

Repair generator	Verifier	Sel. acc.	End-to-end resolve	Note
Base	Base	$52.0$	$31.9$	starting system
Base	Trained	$66.0$	$36.5$	verifier effect, fixed generator
Trained	Base	$58.0$	$38.5$	generation effect, fixed verifier
Trained	Trained	$72.0$	$43.0$	combined system
Interaction $\Delta_{\text{int}}$ (95% CI)		$0.0\ [{-}2.5,\,2.5]$	$-0.1\ [{-}2.6,\,2.4]$	CI includes $0$

Appendix

Table 15: Generator-verifier swap, same role interface and budget in every cell. The Base/Base and Trained/Trained resolve cells are the measured held-out Verified pass@1 (

31.9

43.0

). Holding the generator fixed and swapping in the trained verifier alone lifts resolution

31.9\to 36.5

(

+4.6

) and selection accuracy

52.0\to 66.0

, so verification changes repair decisions independently of generation. The interaction term is in the last row.

Error source (held-out issues)	Base rej.	SFT rej.	+RL rej.	+RL correct-alt acc.
Familiar operators	$72.0$	$80.0$	$91.0$	$80.0$
Held-out fault operators / classes	$64.0$	$73.0$	$84.0$	$78.0$
Semantic mutants, excluded generator	$58.0$	$68.0$	$79.0$	$76.0$
Natural wrong repairs, independent gens	$51.0$	$62.0$	$74.0$	$78.0$

Appendix

Table 16: Rejection rate by error source on a bank frozen across checkpoints, with correct-alternative acceptance in the last column. The familiar-operator +RL rejection (

91.0

) tracks the measured wrong-rejection margin (TNR

87.4

, Table 4 ). Rejection falls as the source moves off the training distribution (

91.0\to 74.0

), staying well above SFT everywhere, while correct-alt acceptance holds between

76

and

80

. The decisive row is natural wrong repairs, where a

+12

gain over SFT there indicates general verification rather than specialization to the mutation distribution.

Adversary	Train-time completion	Held-out detection	Survival $\rho$ (95% UB)	Audit $M$
Weak (same-size, untuned)	$99.0$	$71.0$	$\leq 0.089$	$32$
Same-family (policy sibling)	$95.0$	$78.0$	$\leq 0.089$	$32$
Strong (frozen, ours)	$88.0$	$85.0$	$\leq 0.089$	$32$
Deterministic mutants only	$100$	$66.0$	$\leq 0.046$	$64$

Appendix

Table 17: Zero-survivor behavior split into train-time completion (fraction of training instances the frozen test drives to zero survivors against that adversary) and detection on a fresh mutant set generated after the test is frozen. Training against the strong frozen adversary lowers train-time completion, because its mutants are harder, yet raises held-out detection (

66.0\to 85.0

vs deterministic-only), the crossover that motivates using a strong adversary. This mirrors the measured fixpoint-rate gain (

2.8\to 15.6

, Table 7 ). The one-sided bound of Section 2.3 gives

0.089

M{=}32

and

0.046

M{=}64

Local-credit scheme	Wrong rej.	Correct-alt acc.	Cross-gen discr.	pass@1
Unsigned kill credit (main objective)	$87.4$	$78.0$	$74.0$	$43.0$
Uniform token bonus	$79.0$	$72.0$	$66.0$	$41.5$
Shuffled-credit control	$72.0$	$63.0$	$55.0$	$40.0$
Signed two-sided credit	$88.0$	$84.0$	$78.0$	$43.3$

Appendix

Table 18: Assertion-credit ablation, all four schemes trained and measured under the same protocol. The unsigned row is the main objective used everywhere else in the paper (wrong rejection

87.4

from FPR

12.6

, pass@1

43.0

) and is tinted for that reason, not because it is the column winner. The signed two-sided rule, which charges an assertion for rejecting a correct alternative, gains most on the side unsigned credit is blind to. Correct-alt acceptance rises

78.0\to 84.0

, with cross-generator discrimination

74.0\to 78.0

and pass@1

+0.3

, consistent with penalizing over-rejection. The uniform and shuffled controls confirm the effect requires per-assertion attribution. The evaluation bank includes both correct alternatives and natural wrong repairs.

SFT variant (matched data, budget)	pass@1	pass@8	Distinct loc.
Uniform SFT (same successes)	$33.7$	$48.5$	$1.55$
Quality-only weighting	$34.1$	$49.6$	$1.72$
Quality $+$ exploration	$34.8$	$50.7$	$1.90$
Quality $+$ expl. $+$ efficiency (full)	$35.2$	$51.1$	$1.98$
Interface-compliance control	$32.6$	$47.4$	$1.45$

Appendix

Table 19: Weighted-SFT ablation with an interface-compliance control that fixes format and tool-call handling without exposing task solutions. The full-weighting row is the measured SFT policy (pass@1

35.2

, pass@8

51.1

, distinct localizations

1.98

, Table 2 ). The exploration term is where distinct localizations rise (

1.72\to 1.90

), which shows the weighting teaches search behavior rather than tool-use competence.

	SWE-bench Verified			SWE-PolyBench
Weighting scheme	p@1	p@8	dist. loc.	p@1	p@8	dist. loc.
Uniform weights	$33.7$	$48.5$	$1.55$	$21.6$	$26.9$	$3.42$
T1-only (gold-resolved; plain RFT)	$34.1$	$49.3$	$1.63$	$22.5$	$28.0$	$3.55$
Full, $\lambda{=}0$ (no exploration)	$34.8$	$50.4$	$1.74$	$23.2$	$28.9$	$3.70$
Full weights (ours)	$35.2$	$51.1$	$1.98$	$24.2$	$30.2$	$4.13$

Appendix

Table 20: SFT weighting ablation (no RL), same corpus and token budget. Plain rejection-sampling fine-tuning (T1-only) already beats uniform weighting, and the exploration factor adds distinct localizations beyond it on both suites, with matching pass@8 gains.

Configuration	Resolved /500	Rate	Steps/solve	Early-stop /500	Loc. pruned
Full DVS ( $8$ cells)	$264$	$52.8\%$	$503$	$102$	$126$
$-$ early stopping	$264$	$52.8\%$	$728$	$0$	$126$
$-$ location pruning	$256$	$51.2\%$	$598$	$99$	$0$
$-$ reverted-tree run (patched only)	$237$	$47.4\%$	$665$	$118$	$0$
$-$ regression / coverage checks	$261$	$52.2\%$	$511$	$106$	$129$
$4$ cells (vs $8$ )	$246$	$49.2\%$	$452$	$77$	$94$

Appendix

Table 21: Differential-verification component ablation (test-time only). Section 2.1 reads the two-tree differential and early-stopping rows. Location pruning contributes

1.6

points and halving the cell count costs

3.6

points.

Repository-Level Program Repair

See all 12 papers

Mar 25, 2026

Coherence Collapse: Diagnosing Why Code Agents Fail After Reaching the Right Code

Code agents resolve 65-70% of SWE-bench Verified issues, but Pass@1 cannot tell us why the rest fail, and, as we show, capable-model failures are systematically misdiagnosed without trajectory data. We introduce TRAJEVAL, a training-free decomposition of agent trajectories into reference-patch-aligned search, read, and edit stages, and apply it across 16,758 trajectories spanning three architectures and seven models. The dominant failure of capable models is not localization: 60-69% of failures on SWE-Agent and OpenHands reach and edit the correct functions yet still produce incorrect patches, and the pattern persists for most models on the bash-only LiveSWEAgent. Within this Edit-Quality residual, we identify Coherence Collapse, where the agent reaches correct code and then overwrites or thrashes it, as the largest theme, replicating across SWE-bench Verified and the multilingual PolyBench Verified. In 5 cases, the agent produces a patch bit-identical to the gold reference mid-trajectory and destroys it later; an edit-commit checkpoint recovers all 5 against the SWE-bench Docker harness. A reference-free consensus-driven variant yields a directional +3.0 pp Pass@1 measurement on GPT-5 (p=0.08).

Myeongsoo Kim, Dingmin Wang, Siwei Cui +6

AWS AI Labs · Monash University

Agent Failure Analysis AI Coding Agents Code Generation Evaluation Agent Trajectory Analysis Software Engineering Benchmarks

Ask PDF

Sep 8, 2026

ExecCritic: Learn to Test, Test to Improve for Coding Agents

Execution feedback can guide coding agents toward correct repository repairs, but only when the tests capture the behavior requested by the issue. Agent-generated tests can encode incomplete or incorrect behavioral targets; when the same trajectory writes both the patch and the test, their errors can agree and create false confidence. We introduce ExecCritic, combining a test--verify--revise scaffold with a role-specific reinforcement learning recipe for training agents within it. The scaffold separates test construction from source-code repair: a Test agent independently generates repository-native tests, a fail-closed harness qualifies and freezes them, and a Repair agent revises source code from their execution feedback without changing the tests. Both roles use Qwen-3.5-35B-A3B as the backbone and are trained separately. In Learn to Test, the Test agent learns to produce behaviorally valid tests that distinguish correct from incorrect patches. In Test to Improve, the Repair agent learns both direct task resolution and feedback-guided revision. On SWE-bench Verified, test quality determines whether feedback helps: holding the base Repair agent fixed, tests from the base Test agent reduce resolved rate from a no-test baseline of 61.2% to 57.3%, whereas tests from GPT-5.6-sol raise it to 65.3%. Role-specific post-training raises the Qwen Test agent's Base-to-Gold success from 22.2% to 62.2%; composing the two post-trained Qwen agents reaches 72.6%, an 11.4-point gain over the original no-test baseline without stronger-model or Oracle feedback at evaluation time. Code is publicly available at https://github.com/MSR-Orchard/execcritic.

Leitian Tao, Baolin Peng, Haorui Wang +7

University of Wisconsin–Madison · Microsoft Research · Georgia Tech

RL for Code Generation Software Engineering Agents Automated Program Repair Automated Test Generation

Ask PDF

Oct 9, 2026

SWE-Critix: Post-Training Generalist Verifiers for Assessing Coding-Agent Trajectories

Coding agents are increasingly deployed for end-to-end software engineering tasks, where a readily available oracle evaluator that defines the success criteria is often absent. This motivates verifications for coding agents that can reliably judge and explain their behavior. However, verification is non-trivial as coding agents are increasingly capable of solving long-horizon software engineering tasks. Their execution trajectories can be extremely long, containing hundreds of iterative interactions with code repositories and external tools. Existing verifiers either rely on general-purpose LLMs without specialized training or train dedicated critics that predominantly focus on producing scalar rewards, highlighting the relatively limited exploration of training-based generalist verification. We introduce SWE-Critix, a post-training pipeline for training generalist verifiers of coding-agent trajectories. Given a task and its execution trajectory, the learned verifier predicts the completion label and generates a natural-language analysis that explains its judgment. Our pipeline combines supervised fine-tuning (SFT) with reinforcement learning (RL) and trains a series of 30B models on trajectories from CodeForge-Preview. On a held-out test set, SWE-Critix improves classification accuracy from 57% for the base Qwen3-30B-A3B reasoning model to 66% after SFT and 78% after RL. We release our models, datasets, and training code to facilitate further research.

Jiajun Hu, Chun Yong Chong

Leibniz Research Center, Huawei, Hong Kong SAR, China · School of Information Technology, Monash University Malaysia, Subang Jaya, Malaysia

LLM Agent Verification AI Coding Agents

Ask PDF

Abstract

Repository-Level Program Repair

Explore similar work

Coherence Collapse: Diagnosing Why Code Agents Fail After Reaching the Right Code

ExecCritic: Learn to Test, Test to Improve for Coding Agents

SWE-Critix: Post-Training Generalist Verifiers for Assessing Coding-Agent Trajectories