ANTE: Anticipate, Then Execute -- Learning Internal World Models for Coding Agents from Verified Beliefs

Yi Ge; Kangrui Ruan; Jifan Chen; Manish Prajapat; Christian Bock; Xiaoyang Wang; Rein Houthooft; Aritra Sengupta

cs.LGOct 11, 2026

ANTE: Anticipate, Then Execute -- Learning Internal World Models for Coding Agents from Verified Beliefs

Authors: Yi Ge, Kangrui Ruan, Jifan Chen, Manish Prajapat, Christian Bock, Xiaoyang Wang, Rein Houthooft, Aritra Sengupta,

Organizations: University of Maryland, College Park · AWS AI Labs

Abstract

Rapid advances in large language models have made software engineering agents capable of tackling complex coding tasks. Within a containerized environment, a coding agent receives tool responses or execution results, and determines its later actions by those earlier results. However, most existing coding agents never look ahead: their policies do not anticipate the consequences of actions, and the terminal reward barely attributes a failure to certain earlier steps, so an ill-judged edit can corrupt the repository state irreversibly before any signal arrives. Recent code world models aim to address these challenges, but they are either trained and then frozen before reinforcement learning (RL), or reduced to an auxiliary next-token loss on the environment's outputs. We present ANTE (Anticipate, Then Execute), an agentic RL framework that verifies the policy's pre-action predictions by real execution and turns them into per-step credit for its actions and trust-or-abstain decisions. With an internal belief stated before each action, Belief-Anchored Policy Optimization (BAPO) turns the settled predictions into per-step advantages by anchoring credit on the policy's own forecasts. To turn the trained forecasts into better actions at inference, thereby facilitating effective test-time scaling, ANTE-TTO (test-time optimization) selects among candidate actions without an external scorer. ANTE raises Qwen3-32B from 25.0% to 57.2% pass@1 on SWE-bench Verified, 6.2% above a matched GRPO baseline trained without the belief state; its world model's accuracy climbs from 44% to 69% during RL, and the learned foresight capability transfers across harnesses and out-of-distribution domains. In summary, we aim to build a code world model that knows what has happened, what holds now, and what the next action will achieve.

Figures & tables

Figure 1: Anticipation, verified by execution. Top: the same task with and without a belief state; only ANTE forecasts the edit’s effect and lets execution settle it before committing. Bottom: under RL, resolve rate and prediction hit rate rise together.

Figure 2: Overview of ANTE. In Stage I, the model learns the belief state from execution-labeled BeliefTraj; Stage II verifies every forecast against execution and turns it into a prediction loss, a decision reward, and BAPO step credit; ANTE-TTO resamples at an abstention or a predicted fail and executes the best-forecast candidate.

Figure 3: BAPO credit. Left: three rollouts of one task; a conclusive step shows its forecast (F or P), its decision (tr or ab), and its decision reward

r_{i,t}

; a bracket marks one decision return

W_{i,t}

, and the bar under every step is its credit

A_{i,t}={\hbox{\pagecolor{abg}{\color[rgb]{0.4141,0.3164,0.6211}

A_{i}

}}}+{\hbox{\pagecolor{dbg}{\color[rgb]{0.1797,0.4883,0.4297}

\Delta_{i,t}

}}}

(Eq. 6 ); pragmatic steps carry

A_{i}

alone. Right: each white strip is a belief-decision group

G(\tilde{y},d)

inside its belief group

G(\tilde{y})

; for the ringed step, the gap to the belief value is its belief-relative advantage and the gap to the belief-decision value its decision-relative advantage.

Table 4

Figure 5: The world model sharpens and the trust decision is learned. (a) The prediction hit rate of ANTE’s world model on conclusive steps rises throughout RL at every scale. (b)–(d) BAPO analysis: (b) the trust rate doubles, (c) the accuracy of trusted predictions rises, and (d) abstention concentrates on the steps that remain uncertain.

Figure 6: Case study. The internal belief helps reinforce ANTE’s world model reasoning for coding agents (task: instance psf__requests-1724 of SWE-bench Verified).

Appendix figures & tables22 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Meaning	Definition or reference
Environment and rollouts
$(\mathcal{S},\mathcal{A},\mathcal{O},\mathcal{T},\Omega,\mathcal{R})$	POMDP of a harness	Appendix A
$P_{\mathrm{env}}(o_{t+1}\mid h_{t},a_{t})$	Kernel of the history process	Appendix A
$b_{t}$	Posterior belief	$P(s_{t}\mid h_{t})$
$\mathcal{Y}=\mathcal{Y}^{\mathrm{tst}}\times\{0,1\}$	Outcome space	$\mathcal{Y}^{\mathrm{tst}}=\{\textsc{pass},\textsc{fail},\textsc{na}\}$ ; § 3.1
$x\sim\mathcal{D}$ ; $\mathcal{D}_{\mathrm{BT}}$	Task; BeliefTraj corpus	§ 3.3 ; § 3.2

Appendix

Table 4: Notation. Hats mark estimates;

\tilde{y}

denotes a value of the predicted key field (a group key) and

\tilde{A}

a standardized advantage.

Figure 7: Belief as a point summary of the history. The forecast $\hat{y}_{t}$ predicts the one coordinate of the next observation that the verifier $\psi$ reads; the prediction loss checks it against $y_{t}$ , and the decision reward scores $d_{t}$ against the same outcome.

Figure 8: BeliefTraj curation. Screening, collection in containers, and label derivation; counts in Table 5 .

Stage	Count
Source pool of repository-issue tasks	13,558
Reproduce in their containers (execution preflight)	10,662
Solved by the trajectory collector	6,746
Verified solving trajectories (quality gates)	12,500 over 6,729 tasks
Training corpora (ANTE, Retrospective, Reactive)	3 $\times$ 12,500 trajectories

Appendix

Table 5: Task screening counts.

Pool	Executable tasks	Solved in collection	Solve rate	In RL pool
R2E-Gym	4,555	3,919	86.0%	3,880
SWE-rebench	4,456	1,829	41.0%	1,767
SWE-Gym	1,651	998	60.4%	919
Total	10,662	6,746	63.3%	6,566

Appendix

Table 6: RL pool composition. Per-pool composition of the repository-issue tasks after the execution preflight.

Panel of Figure 9 — Figure 9: Trajectory samples from the SFT corpora. Steps 1, 5, and 15 (the submission) of one recorded run, with long passages elided.

Figure 10: Training corpus distributions. (a) Assistant steps per trajectory. (b) Tokens per trajectory (characters/4 estimate). (c) Predicted pass and fail counts. (d) Steps with a definite predicted check outcome, per trajectory.

Figure 11: Anatomy of a step message. The belief state $\beta_{t}=(z_{t},\hat{y}_{t},d_{t})$ is between the reasoning and the tool call in one pass. The prediction loss reaches the forecast’s value tokens, and the policy gradient reaches the reasoning, decision, and action tokens. The verifier $\psi$ maps $o_{t+1}$ to the realized outcome $y_{t}$ , which checks the forecast and settles the decision.

	Repository issues (R2E-Gym)	Terminal tasks
Check field predicts	Outcome of the check the step runs	Outcome of the hidden task verifier
Values	pass / fail / na	pass / fail
na is definitional on	Pragmatic steps	–
Infeasible pair	( pass , exit field $0$ ): a passing test implies a clean run	–
Exit field predicts	Command exits with code zero	Same
Conclusive iff	Check field in { pass , fail }	After the last step

Appendix

Table 7: Parser rules. The check field and the exit field per scaffold.

Figure 12: Supervised fine-tuning loss of ANTE.

Figure 13: Training pass rate under RL. Training pass rate (%) for Reactive, Retrospective, and ANTE at 7B, 14B, and 32B on the RL task pool.

	Parameter	Value
SFT	Objective	Next-token cross-entropy
	Corpus	One variant of the paired corpora per training run
	Epochs / cutoff	3 / 32,768 tokens
	Learning rate	$10^{-5}$ , cosine, 5% warmup
RL	Rollouts per task	$N=8$ (64 tasks per step)
	Schedule	103 steps

Appendix

Table 8: Training configuration. Settings are shared across model scales except where noted.

Figure 14: ANTE-TTO across the two harnesses. At a trigger step the policy resamples candidate actions and scores each by its own belief; the command predicted to crash never touches the environment.

Figure 15: BAPO worked example.

Figure 16: Belief state on the terminal harness.

Figure 17: Terminal instantiation. Left: repository issues. Right: terminal tasks.

Method	Backbone	Training	Scaffold	Resolved (%)
$<$ 10B
Qwen2.5-Coder-7B	–	–	OpenHands	1.8
Qwen3-8B	–	–	ProRL Agent	9.6
SWE-Dev-7B	Qwen2.5-Coder-7B	SFT	OpenHands	23.4
R2E-Gym-7B	Qwen2.5-Coder-7B	SFT	R2E-Gym	19.0
Lingma-SWE-GPT-7B	Qwen2.5-Coder-7B	SFT	SWESynInfer	18.2

Appendix

Table 9: Open-weight methods on SWE-bench Verified (pass@1, %) at all three scales. Bold marks the best and underline the second-best task-trained method per scale.

Method	Backbone	Training	Scaffold	Resolved (%)
Base models
Qwen2.5-Coder-7B	–	–	Terminus-2	0.0
Qwen2.5-Coder-14B	–	–	Terminus-2	1.2
Qwen2.5-Coder-32B	–	–	Terminus-2	4.5
Qwen3-8B	–	–	Terminus-2	1.6
Qwen3.5-9B	–	–	Tmax	16.9

Appendix

Table 10: Open-weight terminal agents on Terminal-Bench 2.0 (Resolved, %). Bold marks the best and underline the second-best task-trained method within each size band.

\dagger

: our reproduction of ECHO (backbone Qwen3.5-9B, trained on our RL task pool).

Figure 18: Belief protocol stability. (a) The settled share of predictions, the steps whose forecast is scored, for ANTE at 7B, 14B, and 32B; (b) steps and (c) response tokens per trajectory at 32B for the four variants.

Judge	Covered Acc. (%)	MCC	Overall Acc. (%)
SW-RM (w/o gold)	65.5	$+$ 0.359	58.4
SW-RM (w. gold)	72.3	$+$ 0.454	63.2
ANTE	73.2	$\bm{+}$ 0.482	73.2

Appendix

Table 11: Trajectory-outcome judging. SW-RM: SWE-World’s 32B reward model, with or without the gold patch. Covered Acc. and MCC (Matthews correlation coefficient) count only judged trajectories, and Overall Acc. also counts the rest.

Figure 19: Failure modes without foresight.

Prompt	Stage	Applies to
Repository-issue base system prompt	SFT, RL, evaluation	R2E-Gym scaffold
Belief protocol (repository issues)	SFT, RL, evaluation	ANTE variants
Memory protocol (Retrospective)	SFT, RL, evaluation	Retrospective
Belief protocol (terminal tasks)	SFT, RL, evaluation	ANTE on terminal tasks
Collection-time belief protocol	Corpus construction	The trajectory collector

Appendix

Table 12: Prompt contents. Green-framed prompts carry the belief protocol; blue-framed prompts show the shared scaffold.

Code World Models

See all 15 papers

Before They Can Solve: Predicting Post-Training Coding-Agent Performance from Base Models
Oct 7, 2026Tan Yu, Alexander Bukharin, Khushi Bhardwaj +19AI Coding Agents AI Agent Evaluation
NVIDIA · University of Minnesota – Twin Cities · University of California, Berkeley
Belief-Calibrated Optimization: An Explicit World Model for Agentic Optimization
Sep 1, 2026Yuhan Chen, Zhihua Tian, Mahavir Dabas +7Large Language Model-Guided Optimization Action-Conditioned World Models
1Virginia Tech · 2Amazon · 3Independent Researcher
SWE-Critix: Post-Training Generalist Verifiers for Assessing Coding-Agent Trajectories
Oct 9, 2026Jiajun Hu, Chun Yong ChongLLM Agent Verification AI Coding Agents
Leibniz Research Center, Huawei, Hong Kong SAR, China · School of Information Technology, Monash University Malaysia, Subang Jaya, Malaysia

Oct 7, 2026J/K move · Enter open · S save

Before They Can Solve: Predicting Post-Training Coding-Agent Performance from Base Models

Tan Yu, Alexander Bukharin, Khushi Bhardwaj, Jennifer Williams+18

NVIDIA · University of Minnesota – Twin Cities · University of California, Berkeley

AI Coding Agents AI Agent Evaluation Agentic Code Generation

Open paper AskPDF arXiv

How can we predict which base checkpoint is worth an expensive round of agentic post-training? End-to-end pass@

K

tests whether successful behavior already appears in a base model's distribution, but it is a poor fit for agentic coding: many base checkpoints cannot reliably produce the well-formed tool invocation required to complete a task end-to-end. Single-shot or short-horizon tasks avoid these tool-calling failures by collapsing a multi-step interaction into a fixed prompt and a single patch, but they sidestep the core capability we care about: maintaining coherent state over many tool-using steps as the repository evolves. To bridge this gap, we treat successful post-trained agent trajectories as a lookahead signal of base-model potential. Replaying each trajectory and rerunning tests after every code-changing step identifies the decisive step: the first step whose cumulative patch flips the repository from failing to passing, certifying that the recorded action solves the task given the prior context. Motivated by a coverage principle for agentic traces, we build three screens at this step that do not require a base checkpoint to drive the harness from a cold start: (i) Decisive-Action BPB (bits per byte) measures the probability mass on the certified action, (ii) Patch MCQ tests the checkpoint's choice between that action and alternatives rejected by the same verifier, and (iii) prefix-conditioned pass@

K

evaluates support for functionally-correct generations and credits any continuation that the tests accept. Across ten pairs of public base and post-trained models, all three screens rank the cohort in close agreement with post-trained SWE-bench Verified pass@

1

. As our methods need only a benchmark's successful trajectories and its verifier, they can be applied to turn future agentic coding benchmarks into base-model evaluations.

Abstract

Code World Models

Explore similar work

Before They Can Solve: Predicting Post-Training Coding-Agent Performance from Base Models

Belief-Calibrated Optimization: An Explicit World Model for Agentic Optimization

SWE-Critix: Post-Training Generalist Verifiers for Assessing Coding-Agent Trajectories