A coding agent may safely modify one repository while silently breaking downstream services, libraries, or datastores that depend on it. Exhaustively running integration tests after every agent action is impractical, so the agent must predict these failures before executing them. Existing software world models predict the agent's own observations, while static change-impact analysis only identifies where a change may propagate. We instead introduce the Software World Model (SWM), which models the broader system affected by a code change and predicts its blast set: the components that the change will break. SWM follows three stages: explore, learn, and act. Explore executes candidate changes from restored system states, prioritizing regions where observed failures contradict the dependency graph. Learn fine-tunes a language model on these execution outcomes to predict downstream breakage. Act converts sampled predictions into per-consumer break probabilities for change ranking, proactive migration, and deciding when another execution is worth its cost. On held-out synthetic systems, SWM improves blast-set F1 from 0.431 for static reachability to 0.571±0.037, more than halves ranking regret, and improves migration return at all nine evaluation checkpoints. The two methods are complementary: reachability is stronger on dependencies represented in the graph, while SWM recovers failures caused by couplings the graph misses. Experiments on held-out real libraries further show that predicting structured failure outcomes, rather than only scalar risk, is important for downstream decision quality.
Figures & tables
Figure 1: SWM closes the loop from exploring executable software worlds, to learning their change-conditioned consequences, to improving the coding agent that acts in them.
Figure 2: Companion evidence across Explore, Learn, and Act. (a) Fractions of locally passing real-library perturbations that break a consumer, contradict static prediction, or expose a hidden channel. (b) Blast-set F1 on 90 held-out synthetic forecasts; dots mark the mean and mean ± three-seed half-range. (c) Explicit-belief planning on 40 migration tasks, with perfect foresight as a reference. The panels use separate evaluation populations.
Figure 3: Exploration, consequence learning, and agent improvement. The prediction study in (b) reports three-seed half-ranges; the agent study in (c) reports paired world-bootstrap 95% intervals and individual seed means.
Agent
Synthetic
Real libraries
Return ↑
Recall ↑
Broken ↓
Return ↑
Recall ↑
Broken ↓
Base agent
0.721
0.444
0.600
0.453
0.276
0.387
CWM
0.740+0.019
0.533
0.499
0.548+0.095
0.381
0.351
Qwen- AgentWorld
0.754+0.033
0.559
0.447
0.571+0.118
0.446
0.360
SWM (Ours)
0.773+0.052
0.586
0.363
0.613+0.160
0.499
0.309
Table 1: Mean agent performance on synthetic and real-library worlds. Return subscripts give gains over the base agent. Standard deviations appear in Table 5 .
Training corpus
Return ↑
Δ return [95% interval]
Base agent
0.721
n/a
Random
0.772
+0.050[+0.021,+0.088]
Coverage
0.768
+0.046[+0.012,+0.089]
Expert
0.780
+0.059[+0.029,+0.095]
Decision-aware
0.765
+0.044[+0.010,+0.087]
Table 2: Consequence training improves the base agent on 40 matched tasks. Trained rows average three seeds at 360 records; brackets give paired world-bootstrap 95% intervals for return differences, computed before rounding.
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
Component
Evaluation
Score
Comparator or context
Gain or evidence
Explore
Real hidden-channel yield
0.244
random: 0.157
+55%
Learn
Synthetic blast-set F1 (supervised recipe)
0.608±0.016
reachability: 0.431
+0.177
Learn
Synthetic hidden-channel F1
0.304
reachability: 0.189
+61%
Learn
Real-library hidden-channel F1
0.198±0.038
reachability: 0.137
+45%
Act
Candidate-ranking regret ↓
0.593
reachability: 1.430
−59%
Act
Belief-plan return ↑
0.757
reachability: 0.685
+0.072
Appendix
Table 3: Exploration, forecast, and explicit-decision results. Each row uses its stated population and comparator.
Figure 4: Exploration affects prediction and ranking differently. (a) Blast-set F1 and (b) ranking regret across execution budgets; points are three-seed means and error bars are half-ranges. (c) The 100-record comparison. Coverage and expert exploration were not evaluated at 360 records; the first 60 decision-aware records are random. Higher F1 and lower regret are better.
Corpus
Records
Seed
Return
Δ return
95% interval
Random
100
0
0.749
+0.028
[−0.009,+0.072]
Random
100
1
0.754
+0.033
[+0.005,+0.067]
Random
100
2
0.731
+0.010
[−0.023,+0.052]
Random
200
0
0.755
+0.034
[+0.011,+0.065]
Random
200
1
0.769
+0.048
[+0.018,+0.086]
Random
200
2
0.778
+0.056
[+0.025,+0.094]
Appendix
Table 4: All 24 trained arms on the same 40 tasks; base-agent return is 0.721 . Intervals are the source campaign’s per-seed paired world-cluster 95% intervals.
Figure 5: All twelve full-budget training seeds improve agent return, candidate recall, and services left broken relative to the base checkpoint. Panel (a) includes paired world-cluster 95% intervals; (b–c) show per-seed mean changes on the same tasks.
Agent
Synthetic worlds
Real libraries
Return ↑
Recall ↑
Broken ↓
Return ↑
Recall ↑
Broken ↓
Base agent
0.721
0.444
0.600
0.453
0.276
0.387
CWM ( Copet et al., 2025 )
0.740+0.019±0.006
0.533±0.007
0.499±0.020
0.548+0.095±0.005
0.381±0.005
0.351±0.014
Qwen-AgentWorld ( Zuo et al., 2026 )
0.754+0.033±0.006
0.559±0.022
0.447±0.027
0.571+0.118±0.005
0.446±0.018
0.360±0.022
SWM (Ours)
0.773+0.052±0.008
0.586±0.017
0.363±0.021
0.613+0.160±0.006
0.499±0.015
0.309±0.018
Appendix
Table 5: Complete agent means and standard deviations for the main comparison. Values are means, with standard deviations in superscripts where available. Return subscripts show the mean difference from the base agent within each dataset. Recall measures affected-consumer coverage; broken services are counted per task.
Figure 6: Real-library prediction and executed exploration. (a) Blast-set F1 on 984 held-out records; hollow markers show F1 on the 137 graph-missed breaking records. Error bars retain the reported three-seed spreads; unmarked learned variants are single runs. (b) Prediction after adding random or blind-spot exploration, with base-corpus seed spreads. (c) Acquisition yields from 450 candidate perturbations per policy on three training worlds, conditional on passing local checks. Consumer AUROC measures discrimination among broken and unbroken consumers.
Interface
Extracted breaks
Silent
Set-valued, record-order context
0.417
0.841
Per-consumer, record-order context
0.000
1.000
Per-consumer, change-focused context
0.586
0.808
+ consumer usage
0.575
0.780
+ usage and blind-spot round
0.576
0.805
Alternative 1.5B backbone
0.689
0.799
Appendix
Table 6: Additional interface diagnostics on the same 984 held-out records. Columns retain the recorded “Extracted breaks” and “Silent” measures; blast-set and hidden-channel F1 appear in Figure 6 a.
Corpus
Records
Return
Δ return [95% interval]
Candidate recall
Broken services
Random
100
0.745
+0.023[−0.008,+0.062]
0.573
0.342
Random
200
0.767
+0.046[+0.018,+0.082]
0.594
0.342
Random
360
0.772
+0.050[+0.021,+0.088]
0.569
0.383
Coverage
360
0.768
+0.046[+0.012,+0.089]
0.573
0.375
Expert
360
0.780
+0.059[+0.029,+0.095]
0.603
0.342
Decision-aware
100
0.765
+0.043[+0.013,+0.081]
0.607
0.342
Appendix
Table 7: All corpus budgets in the agent comparison, averaged over three seeds. Intervals are paired world-bootstrap 95% intervals against base-agent return 0.721 .
Figure 7: Training budget and agent behavior on the same 40 tasks. Return gains use paired world-bootstrap 95% intervals; candidate recall and broken-service counts show three-seed half-ranges, with the base agent as the dashed reference.
Figure 8: Forecast accuracy depends on both propagation mechanism and input information. (a) Baselines on 90 held-out synthetic forecast steps; error bars show three-seed half-ranges, and unmarked learned variants are single diagnostic runs. (b) Direct and hidden-channel F1 across nine checkpoints, with 360, 315, and 819 seed-record pairs in the three groups, respectively. (c) Nested graph inputs under identical labels; the star increases the runtime input budget from 30 to 60. Oracle traversal restores hidden graph channels.
Configuration
Seed 0
Seed 1
Seed 2
Mean ± half-range
Learned configurations
text input, per-module clip
0.642
0.614
0.565
0.607±0.039
graph tokens + text input, per-module clip
0.621
0.549
0.579
0.583±0.036
text input, shared clip (preregistered)
0.565
0.537
0.611
0.571±0.037
Structure
typed reachability
0.431
0.431
0.431
0.431
Appendix
Table 8: Per-module against shared gradient clipping for the companion recipe, three paired training seeds, forecast-step blast-set F1 on the 90 held-out synthetic forecast steps. The graph-token arm uses Kg=8 learned attention queries and per-module clipping.
Figure 9: Explicit-belief decisions and prediction-error diagnostics. (a) Mean return for preventive-migration plans on 40 held-out tasks; N is the number of sampled blast sets. The learned N=64 error bar is the three-seed standard error. (b) Return versus mean expenditure on paid checks: shape identifies the check strategy and filled markers denote calibrated beliefs; the displayed error bar is a standard error. (c) Noise-perturbed oracles matched to learned F1 have lower ranking regret but lower planning return. Ranking and planning use separate populations; lower regret and higher return are better.
World
Composition
Changed interface
Executed signal
Procedural systems
4 – 7 services, shared library, datastore, events, configuration and access control
Nine change families over typed relations
Repository suites, service health, checks, and true blast sets
Acme
Four repositories with three Flask services and one shared library
API, dependency, database, event, configuration, flag and permission changes
Local suites plus end-to-end checks
Online Boutique
Ten repositories containing nine Go, Node and Python microservices
gRPC contracts, service wiring, data and latency changes
Repository tests, service health, direct and web journeys
Real-library worlds
Marshmallow with four dependents; OpenTelemetry semantic conventions with four instrumentation packages
Locally consistent upstream API and contract changes
Frozen upstream and downstream repository test suites
Appendix
Table 9: Software worlds used for exploration, training, and evaluation. Each change must pass the changed repository’s own checks before downstream outcomes are measured.
Layer
Kind
Meaning
System
repo
a version-controlled repository
service
a deployable component
library
a shared importable component
database
a datastore instance
table
a table or collection
topic
an event stream
Appendix
Table 10: Node kinds. Every surface-layer node has exactly one owning system-layer node, which identifies the component that owns each interface.
Relation
src → dst
Recovered from
Consumer
contains
repo → service/library
repository layout
depends_on
repo/service → library
manifest, lockfile
✓
imports_symbol
service → symbol
import statements
✓
provides
service → endpoint
route declarations
calls_api
service → endpoint
client call sites
✓
publishes
service → topic
bus client calls
Appendix
Table 11: Edge relations. The consumer relations are those along which breakage propagates backwards, and are the ones the reachability baseline traverses.
Family
Method
Input and output
Role
Structural
Typed reachability
Extracted graph and changed target; reachable consumers
Reachability baseline
Supervised risk
Action-only logistic
Change kind and target count; consumer probabilities
Action-prior control
Supervised risk
Relation/action logistic
Action features, typed degrees, coverage and hops; consumer probabilities
Strong tabular control
Neural structure
Graph neural network (GNN) only
System-Surface Graph; blast set
Graph encoder without language generation
Generative
Zero-shot code models
Serialized state and change; parsed blast set
Task training against model scale
Bounds
Empty, true-graph traversal, and oracle consequence
No failure, inaccessible structure, or executed outcome
Lower and upper references
Appendix
Table 12: Comparison methods and their roles. Learned controls are fitted only on training worlds. Oracle rows use information unavailable at decision time.
Predictor configuration
input
text input (2 hops; ≤30 edges)
graph mode
none (no graph tokens)
base
Qwen2.5-Coder, 0.5B Instruct
adapters
LoRA r=16 , α=32
target modules
q, k, v, o projections
precision
bfloat16
Appendix
Table 13: Consequence-predictor configuration. The default serialises retrieved state as text; graph-conditioned variants are specified in Appendix D.13 .
Software engineering, whether performed by humans or by AI agents, requires reasoning about how software behaves. We call the internal model that supports such reasoning the software world model, and view current code-execution benchmarks as covering one well-studied slice of it -- control flow. In this paper, we take a step toward a broader evaluation by shifting the observable axis to execution resources: alongside test outcome and exception class, we predict peak memory, wall-clock time, and ranked profiler outputs at method and line granularity. We use SWE-bench Verified as the source of data to hold the test close to real-world software engineering tasks. All tested models, frontier ones included, show modest performance and brittle behaviour, suggesting a notable lack of understanding of how software is executed, as opposed to how its source code is written.
Repository-level coding benchmarks largely assume that tasks and environments remain unchanged throughout execution. In software development, code, dependencies, runtime state, and requirements can change while an agent is still working. We introduce SWE-Shift, a benchmark for evaluating coding agents under such mid-execution changes. From the same saved history and workspace, a Clean continuation proceeds without intervention, while a matched Shifted continuation receives a task-relevant update. Success requires completing the current task, preserving still-valid work, and satisfying the applicable state and evidence checks. The benchmark contains 120 scenarios across 12 change families, each evaluated at nine scheduled intervention positions. We evaluate 17 coding-agent systems, all of which have lower Shifted than Clean success. GPT-6 Astra drops from 67.35% to 24.79%, with 20.34% paired success. Tool/session changes and stale evidence yield zero paired success across all systems. Late-position paired success is lower for some systems and higher for others. These results show that success on the unchanged task can coexist with failures to preserve work, recover state, or refresh evidence after an update.
Victor Shea-Jay Huang, Rongqing Li, Zhangyue Yin +7
MMLab, The Chinese University of Hong Kong · Somewhere on the Earth · Fudan NLP Lab, Fudan University +2
Software engineering (SWE) agents resolve repository-level issues through long trajectories that grow increasingly expensive as context accumulates. Failed runs tend to be longer and exhibit redundant exploration or looping, suggesting that some failures may be detectable before completion. Early termination, however, risks interrupting trajectories that would otherwise succeed; conversely, an unsuccessful trajectory may still contain useful repository edits. We present FailFast-RestartSmart, a two-stage controller for a single active trajectory. FailFast is a lightweight 0.6B monitor trained with terminal and dense fail-to-pass supervision to predict failure from observable prefixes without policy logits or hidden states. Upon an alarm, RestartSmart launches a fresh same-policy rollout without prior prompt history and offers the interrupted repository diff as an optional overlay that the agent may inspect, apply, or discard. On SWE-bench Verified, a monitor trained solely on Qwen3.6-27B trajectories transfers to three other policies, including a closed-API model, and saves 14.6%-20.4% of execution tokens at a target 5% false-positive rate; on Qwen3.6-27B, its 20.4% saving exceeds the 12.5% achieved by our per-step AgentStop adaptation. At a target 25% false-positive rate, RestartSmart raises Qwen3.6-27B resolution from 66.6% to 71.8%, whereas cold restart reaches only 66.8%. Together, these results support early termination with sequential same-policy recovery.
Chenyu Wang, Yunbo Lyu, Junda He +4
Singapore Management University · University of Alberta · Nanjing University of Science and Technology +1