An Omitted Mode Is a Rare Rule: The Sampling-Verification Danger Law in Continuous Code World Models

Javier Aguilar Martín

cs.LGAug 18, 2026

An Omitted Mode Is a Rare Rule: The Sampling-Verification Danger Law in Continuous Code World Models

Authors: Javier Aguilar Martín

Organizations: AGILabs

Abstract

In the Code World Model paradigm an LLM synthesizes an executable world model that a classical planner searches, and the model is accepted when it reproduces sampled transitions. We ask what that acceptance certifies in continuous control. We define the pipeline's danger as an expected risk and isolate its exact factor: the probability that N i.i.d. gate rollouts all miss a critical event of probability r is exactly (1-r)^N; an independent acceptance sample adds its budget to the exponent. On three hybrid instruments the accepted mode-blind model is exploited: the planner is pinned at the mode boundary at a regret of nearly the whole attainable return. We prove a localization budget, valid at boundary points: models with Lipschitz constant at most L differing by eta at a point disagree above tolerance eps on a region of volume at least kappa((eta-eps)/L)^(d+m); the discontinuous reset modes studied pay no such budget. With real LLM synthesis, GPT-5.x repairs an omitted 1D clamp in 105 of 111 mode-containing draws -- every attempt exact on 50 of 56 instrument-stream blocks (95% CI [0.781, 0.960]). On 2D regions no artifact recovers the rule (0/156); eight targeted interventions leave the failure in place, and positive controls locate it: a located rule is not induced, while given form and location the constants follow exactly. A version-space certificate proves identification is class-relative: at the widest dose the declared fit succeeds in 20/20 blocks and every sample-consistent circle is within tolerance in 18/20. We prove a class of entry rules exactly consistent with every sample yet harmless at play, so identifiability is a measurable property of the instrument. Re-scoring all 1034 artifacts on independent samples confirms acceptance certifies sample consistency and no more: where the gate is provably informative it covers about two percent of the exploited planner's queries.

Figures & tables

$x_{\mathrm{wall}}$	rarity	$J_{\mathrm{truth}}$	$J_{\mathrm{blind}}$	$J_{\mathrm{rand}}$	play_cost	blind hit	d@20	d@40	d@80
2	0.3168	17.77	$1.4{\times}10^{-5}$	0.53	1.031	1.00	0.001	$2.5{\times}10^{-7}$	$6.0{\times}10^{-14}$
3	0.2135	17.77	$1.3{\times}10^{-5}$	0.53	1.031	1.00	0.008	$6.9{\times}10^{-5}$	$4.7{\times}10^{-9}$
4	0.1352	17.77	$1.9{\times}10^{-5}$	0.53	1.031	1.00	0.056	0.003	$9.3{\times}10^{-6}$
5	0.0797	17.77	$6.3{\times}10^{-5}$	0.53	1.031	1.00	0.196	0.037	0.001
6	0.0439	17.77	$3.8{\times}10^{-4}$	0.53	1.031	1.00	0.420	0.171	0.028
8	0.0114	17.77	0.02	0.53	1.030	1.00	0.818	0.650	0.410

Table 1: The danger curve on the wall-position knob. “blind hit” = fraction of episodes in which the blind planner’s trajectory fires the wall mode; the truth planner’s trajectory fires it in no episode at any knob (

0/20

, so

<0.16

); d@

N

\mathrm{play\_cost}\cdot(1-r)^{N}

. Rarity is measured on 30,000 random rollouts (Wilson 95% CIs in the versioned JSON), the same sample size as the pendulum’s, so the mode-firing and reveal-rarity estimates of the same event are now directly comparable.

Figure 1: The threshold law on the wall-position knob (Table 1 rendered): danger

\approx 0

while the wall is inside the random envelope, rises through the elbow, plateaus at full play_cost;

N

shifts the threshold. Bands and horizontal bars are the rarity’s Wilson interval propagated through

\mathrm{play\_cost}\cdot(1-r)^{N}

, which is monotone in

r

, so the transformed interval is the corner pair; a printed point with no visible band has an interval narrower than the marker.

Figure 2: The two reach distributions of the danger law: the exploited planner’s mode reach is flat at 1.00 across the knob; random reach (= rarity) falls two orders of magnitude; the truth planner’s trajectory reach is a censored zero at every knob (

0/20

episodes) and is drawn as an upper bound rather than as a point — the mode lives on the truth planner’s query distribution (Proposition 8 ), not its path. Error bars are Wilson intervals: binomial over

20

episodes for the two reach curves, and over

30{,}000

rollouts for rarity.

Figure 3: Per-seed return distributions for the three

100

-paired-episode experiments (violins with every seed overplotted). The exploitation claim is the gap between the truth planner and the pinned blind planner, and it is total in every panel. The random baseline’s heavy upper tail — a few episodes that wander onto a reward plateau — is what carries its mean above the blind planner’s on the cart, against

86

100

individual seeds; on PatchField2D the comparison is unambiguous in the other direction (

0

100

$\theta_{\mathrm{stop}}$	rarity	$J_{\mathrm{truth}}$	$J_{\mathrm{blind}}$	play_cost	blind hit	d@40
0.8	0.2876	20.08	0.01	1.002	1.00	$1.3{\times}10^{-6}$
1.0	0.1270	20.08	0.03	1.002	1.00	0.004
1.2	0.0509	20.08	0.05	1.000	1.00	0.124
1.4	0.0196	20.08	0.12	0.997	1.00	0.452
1.6	0.0070	20.08	0.26	0.990	1.00	0.748
2.0	0.0007	20.08	1.23	0.942	1.00	0.917

Table 2: Pendulum-with-stop: threshold law, knob-invariant exploitation (pinned at the stop in every episode at every knob), truth planner untouched by the mode. The mechanism does not care that the plant is nonlinear — only that the mode is hard and rare under the gate’s measure. Rarity is the fraction of 30,000 random rollouts that fire the stop (Wilson 95% CIs for every row in the versioned JSON). The sample was raised from 3000 specifically because the widest knob was a censored zero there (

0/3000

); at 30,000 it resolves to

0.0007

[0.0004,0.0010]

, so every row is now a point estimate and every d@40 a number rather than a bound.

$k_{1}$	$k_{2}$	$r_{1}$	$r_{2}$	$r_{\cup}$	$J_{\mathrm{truth}}$	play_cost	d@40 $P_{1}$	d@40 $P_{2}$	d@40 joint
2	6	0.245	0.010	0.252	17.98	1.006	$1.3{\times}10^{-5}$	0.673	$9.3{\times}10^{-6}$
2	7	0.245	0.008	0.253	18.02	1.006	$1.3{\times}10^{-5}$	0.720	$8.5{\times}10^{-6}$
2	8	0.245	0.007	0.252	18.08	1.006	$1.3{\times}10^{-5}$	0.770	$9.3{\times}10^{-6}$
3	6	0.142	0.008	0.150	18.57	1.006	0.002	0.720	0.002
3	7	0.142	0.008	0.150	18.47	1.006	0.002	0.720	0.002
3	8	0.142	0.007	0.148	17.72	1.006	0.002	0.770	0.002

Table 3: PatchField2D mechanism sweep (

3\times 3

knobs; 600 rarity rollouts, 20 MPC episodes/arm;

J_{\mathrm{blind}}=0

J_{\mathrm{rand}}=0.11

at every cell). d@40

P_{i}

\mathrm{play\_cost}\cdot(1-r_{i})^{40}

(per-mode danger);

r_{\cup}

= measured probability that a rollout contacts either patch; d@40 joint =

\mathrm{play\_cost}\cdot(1-r_{\cup})^{40}

, the danger law at the joint critical event — not the product

((1-r_{1})(1-r_{2}))^{40}

, which would assume within-rollout independence and errs by

-17\%

+12\%

here. Full table (with

J_{\mathrm{blind}}

J_{\mathrm{rand}}

, hit rates) in docs/EXPERIMENTS.md .

arm	reveal-rarity	$(1-r)^{40}$	pass@40 measured	play_cost	d@40
wall@4 omitted	0.1351	0.0030	0.003	1.031	0.0031
wall@8 omitted	0.0103	0.6622	0.667	1.030	0.6821
drag bias $\times$ 1.03 (sub- $\varepsilon$ )	0.0001	0.9940	0.997	$0$ (exact)	$0$ (exact)
drag bias $\times$ 2.0 (supra- $\varepsilon$ )	1.0000	$<10^{-4}$	$<0.013$	$0$ (exact)	$0$ (exact)
$C^{\infty}$ bump@4, amp 0.5	0.1829	0.0003	$<0.013$	$0$ (exact)	$0$ (exact)
$C^{\infty}$ bump@4, amp 1.0	0.2018	0.0001	$<0.013$	$-0.745$	$-0.0001$

Table 4: Axis separation at

\varepsilon=0.01

N=40

; d@

N

\mathrm{play\_cost}\cdot(1-r)^{N}

, as in Table 1 . Danger lives in one quadrant only: rare

\wedge

hard-mode. Reveal-rarity is measured on 20,000 rollouts (raised from 2000 because the sub-

\varepsilon

arm was a censored zero there): it now resolves to

0.0001

, and its predicted pass rate

0.9940

sits inside the measured

[0.9814,0.9994]

instead of above it.

Figure 4: Table 4 as the danger quadrant: gate-miss probability (log) vs play_cost. Only the rare hard mode reaches the top right; danger is the product of the coordinates. Horizontal bars propagate each arm’s rarity interval through

(1-r)^{40}

; the play_cost axis carries a paired interval where per-episode returns exist and no bar where the arms are bit-identical, since there the difference is exactly zero rather than uncertain.

cell (20 seeds each)	full	absent $\to$ blind & exploited	present $\to$ repaired (stalled)
mini $\theta_{\mathrm{stop}}=1.4$	20/20	$9\to 9$ (pc 0.995)	$11\to 11$ (0)
large $\theta_{\mathrm{stop}}=1.4$	20/20	$9\to 9$ (pc 0.995)	$11\to 11$ (0)
mini $\theta_{\mathrm{stop}}=1.0$	20/20	$0\to$ —	$20\to 20$ (0)
large $\theta_{\mathrm{stop}}=1.0$	20/20	$0\to$ —	$20\to 20$ (0)
Qwen $\theta_{\mathrm{stop}}=1.4$ (3 seeds)	3/3	$1\to 1$ (pc 0.995)	$2\to 0$ (2 stalled @0.9997)

Table 5: Pendulum synthesis cells (20 seeds/cell, both GPT-5.x sizes; 3-seed Qwen spot-check at

\theta_{\mathrm{stop}}=1.4

k\to m

= of the

k

seeds in that branch,

m

had the stated outcome (blind & exploited, resp. repaired); the parenthetical count is stalled seeds; pc = play_cost.

#	what it targets	repaired	what did not suffice, as tested
1	region-first guidance, $3\times$ budget	0/40	prompting and budget
2	axis-aligned square, flat edges	0/40	boundary curvature
3	a second model family (Claude)	0/3	one family’s idiosyncrasy
4	trigger arity: a band in one coordinate	0/40	nothing — target unidentifiable
5	the trigger’s argument named	0/40	variable identification
6	the mover stops inside the region	0/40	the interior’s censoring

Table 6: The eight ablations on the 2D mode. “repaired” counts mode-containing synthesis draws recovering the region rule, by the criterion of Section M (a gate pass, a mode probe at

0

, and exact agreement with the truth on a state–action grid, since the probe alone accepts a rule that over-covers the region). Ablation 4 excludes nothing because its target is not identifiable — the measurement above — which is recorded rather than reported as a

0/40

like the others. Ablations 6 and 7 carry complementary confounds: 6 supplies

11\times

more mode evidence, 7 matches the baseline’s evidence and pays in the rule’s complexity, so the negative survives both. Ablation 8 holds the contact count fixed and raises only their angular coverage, to the point where a three-line least-squares fit recovers the region on 20 of 20 samples; it is the only one of the eight whose direction was written down before the run.

model	trained on	off-mode err mean / max	probe err	gate $10^{-9}$	gate $10^{-2}$
linear-LSQ	wall-free	$3.6\mathrm{e}{-15}$ / $1.7\mathrm{e}{-14}$	4.18	PASS	PASS
linear-LSQ	wall-data	$1.9\mathrm{e}{-03}$ / $1.2\mathrm{e}{-02}$	4.17	fail	fail
MLP h=8	wall-free	$3.5\mathrm{e}{-03}$ / $5.0\mathrm{e}{-02}$	4.20	fail	fail
MLP h=8	wall-data	$6.0\mathrm{e}{-03}$ / $4.9\mathrm{e}{-02}$	4.19	fail	fail

Table 7: Smooth learners on the synthesis samples (

x_{\mathrm{wall}}=8

). “probe err” is the wall-region probe error (4.2

\approx

predicting straight through the wall).

Figure 5: Off-mode max error (log) by model and training data. The wall-free linear fit sits at float noise and passes both gates (blind); four contact rows tilt it twelve orders of magnitude; synthesized code is exactly zero off-mode.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

mode arm	$\varepsilon^{\ast}$	firing	first grid $\varepsilon\geq\varepsilon^{\ast}$	measured in the same sample
cart wall@8	0.3959	25	none in grid	flat through $\varepsilon=0.3$
cart wall@4	0.1137	277	0.3	$0.1385$ to $\varepsilon=0.1$ , $0.1355$ at 0.3
pendulum [email protected]	0.0791	282	0.1	$0.1410$ to $3{\times}10^{-2}$ , $0.1400$ at 0.1, $0.1240$ at 0.3
pendulum [email protected]	0.0805	35	0.1	$0.0175$ to $3{\times}10^{-2}$ , $0.0170$ at 0.1, $0.0155$ at 0.3

Appendix

Table 8: The

\varepsilon

-invariance threshold, computed on the sweep’s own rollout stream (2000 rollouts/arm, gate policy only, --seed 10000 to match continuous_eps_sweep.py ). Within one sample the relation is an identity, not a prediction: reveal-rarity is flat below

\varepsilon^{\ast}

and the first grid point at or above it is where the sweep departs. The “firing” column is the number of rollouts the minimum is taken over —

25

on wall@8 — which is why the threshold itself carries the sample-to-sample spread discussed above and must not be read to the last digit.

instrument	knob	pc MPC	pc CEM	contact CEM	crossing CEM	crossing MPC
cart	2.0	1.031	$0$ (exact)	$<0.16$	0.0309	0.3865
cart	4.0	1.031	$0$ (exact)	$<0.16$	0.0055	0.2453
cart	6.0	1.031	$0$ (exact)	$<0.16$	0.0003	0.1483
cart	8.0	1.030	$0$ (exact)	$<0.16$	1.4e-05 †	0.0773
cart	10.0	0.977	$0$ (exact)	$<0.16$	$<2.3e-06$	0.0369
pendulum	0.8	1.002	0.009	0.70	0.1150	0.6392

Appendix

Table 9: The same gate-accepted, mode-blind models under two base planner families (20 paired episodes/row). “pc” is blind-model play_cost within that planner family; crossing is the sampled imagined-boundary-crossing proxy for query-hit mass, one plan per episode seed from paired initial states for both planners. CEM stays near zero play cost and below MPC’s crossing fraction on every row.

cart ( $x_{\mathrm{wall}}$ )
knob	pc_blind	pc_mit	first-contact step
2	1.031	0.290	11.6
4	1.031	0.446	16.9
6	1.031	0.578	21.3
8	1.030	0.699	25.1
10	0.977	0.806	28.7

Appendix

Table 10: Mitigation sweep (20 episodes/knob, both instruments). pc_blind / pc_mit = play_cost of the blind planner under plain MPC / under distrust-region mitigation. Full 11-row table with contact rates and violation counts in docs/EXPERIMENTS.md .

$(k_{1},k_{2})$	pc_blind	pc_mit	mean viol.	first contact
$(2,6)$	1.006	0.257	1.05	8.50
$(3,7)$	1.006	0.541	2.65	12.35
$(4,8)$	1.006	0.862	4.25	15.25

Appendix

Table 11: PatchField2D mitigation sweep (paired seeds). The collapse is partial and degrades with patch distance. Read the means with the distribution beside them: medians are

1/1/2

, maxima

2/28/28

, and the count is dominated by duplicate fences (at most

2/5/6

distinct positions per episode). The degradation is not a longer transient but a growing fraction of outright failures —

0/20

2/20

7/20

episodes end pinned at blind-level return ( scripts/fence_separation_census.py ).

$\varepsilon$	wall@8 rarity	bias $\times$ 1.03 rarity	bias $\times$ 2.0 rarity
$10^{-6}$	0.0125	1.0000	1.0000
$10^{-2}$	0.0125	$<0.0019$	1.0000
0.1	0.0125	$<0.0019$	0.0040
0.3	0.0125	$<0.0019$	0.0040

Appendix

Table 12: Reveal-rarity vs.

\varepsilon

(cart). The mode arm is flat across the whole grid; the pervasive arms switch at their own error scale.

$\eta$	truth passes	blind passes	exact CP $95$	analytic conditional
0–0.1	200/200	11/200	$[0.0278,0.0963]$	0.0550
0.3	200/200	11/200	$[0.0278,0.0963]$	0.0556
1	200/200	32/200	$[0.1121,0.2183]$	0.1582
3	200/200	106/200	$[0.4583,0.6008]$	0.5029

Appendix

Table 13: Support-compatibility gate under bounded observation noise, per seed block. The four levels

\eta\in\{0,0.01,0.03,0.1\}

are identical to four digits and are shown as one row.

campaign	model	cells	calls	min / med / max	cap
qwen_xwall8	Qwen3-Coder-30B	6	16	1 / 1 / 6	2
large_xwall8	gpt-5.4	40	48	1 / 1 / 2	0
large_xwall8_off20	gpt-5.4	40	50	1 / 1 / 2	0
mini_xwall4	gpt-5.4-mini	10	27	1 / 1 / 6	3
mini_xwall8	gpt-5.4-mini	40	62	1 / 1 / 6	1
patch2d_qwen_k3_7	Qwen3-Coder-30B	3	3	1 / 1 / 1	0

Appendix

Table 14: LLM calls per campaign. “Campaign” names the artifact results/continuous_synthesis_

\langle

campaign

\rangle

.json , with qwen abbreviating compat-qwen3-coder-30b-a3b-instruct . Every campaign uses the default prompt variant and max_iters

=5

except the two pv-region_it15 campaigns ( region variant, max_iters

=15

). “Cap” counts cells that exhausted the iteration budget without passing the gate. Source: results/llm_protocol_facts.json

\rightarrow

campaigns .

class	operational rule	line
invalid	exec raises, step missing, or step returns a state of the wrong arity	:183--189
crash	mask construction raises	:190--194
blind	$n_{0}=0$ and $n_{1}=0$ , and step ’s body has no if	:234--237
blind-textual-patch (the paper’s superstitious patch )	$n_{0}=0$ and $n_{1}=0$ , and step ’s body does contain an if ( _step_has_conditional , :170--175 ) — a written rule whose trigger set is behaviourally empty, e.g. an exact-coordinate trap, which is measure-zero and so never fires	:234--236
vdep	$n_{0}=0$ but $n_{1}>0$ : deviates only at nonzero velocity	:238--240
point	largest connected component below $0.5$ world-units 2	:244--247

Appendix

Table 15: The behavioural classifier of scripts/patch2d_artifact_audit.py , in evaluation order.

variant	max_examples	max_failures	guidance
default	30	20	empty string; byte-identical to every pre-knob run
guided	120	40	describe-the-region-first process guidance ( :198--204 )
region	120	40	guided plus “the trigger region need not be a 1-D threshold” de-bias ( :205--210 )
landing	120	40	region plus one sentence naming the trigger’s argument ( :221--225 )

Appendix

Table 16: The prompt variants. Only default and region appear in results/ ; guided and landing are the intermediate and the follow-up treatment.

object	backing results/ JSON	checking
tab:epsstar	eps_invariance_threshold	claims
tab:danger	continuous_reach	cells
fig:threshold	continuous_reach	claims
fig:reach	continuous_reach	claims
tab:pendulum	continuous_pendulum	cells
tab:patch2d	continuous_patch2d	cells

Appendix

Table 17: The manifest: what backs each table and figure, and how it is checked. “cells” means every printed cell is compared against the JSON at printed precision on every run of the numeric audit; “claims” means the audit asserts quoted values without parsing the object. Each JSON is written by the script of the same stem under scripts/ ( continuous_reach.json by continuous_reach.py , and so on); the figures are rendered from their table’s JSON by make_paper2_figures.py , which runs no experiment, so every figure inherits its table’s provenance. Machine-readable form, including the audit call sites per object, in results/repro_manifest.json under manifest[] .

Explore similar work

When a Verified World Model Still Loses: Play-Adequacy vs Prediction-Accuracy in LLM-Synthesized Code World Models
Jul 15, 2026Javier Aguilar MartínLLM World Models World Models
AGILabs (javieraguilar.ai)
An Enclosed Mode Is a Gauge Choice: Topology Relative to Reach in Certified Code World Models
Aug 28, 2026Javier Aguilar MartínWorld Models World Model Evaluation
AGILabs
Where World Models Break: Natural-Input Failure Discovery
Aug 23, 2026Zhanpeng Shi, Zi Liang, Rong Feng +3World Model Evaluation Action-Conditioned World Models
School of Computer Science, Northwestern Polytechnical University · The Hong Kong Polytechnic University · Department of Data Science, City University of Hong Kong +3

Jul 15, 2026J/K move · Enter open · S save

When a Verified World Model Still Loses: Play-Adequacy vs Prediction-Accuracy in LLM-Synthesized Code World Models

Javier Aguilar Martín

AGILabs (javieraguilar.ai)

LLM World Models World Models World Model-Based Planning LLM-Based Program Synthesis Model-Based Planning World Model Evaluation

Open paper AskPDF arXiv

Large language models can synthesize a game's rules as executable code - a Code World Model (CWM) - which a classical planner then searches over. Such models are typically accepted when they reach high transition accuracy on sampled trajectories. We argue this is the wrong notion of adequacy for planning. We show four things. (1) An LLM-synthesized CWM can pass a sampling gate at 100% transition accuracy and be

\geq 98\%

state-accurate on the planner's own search distribution, yet lose systematically at play, because the

<1\%

it gets wrong is exactly the pivotal dynamics; the play cost of the omitted rule is

0.091

(seed-clustered 95% CI

[0.065,0.117]

n=4800

). We call this the verified-vs-correct gap, and confirm it end-to-end through the synthesis pipeline. (2) The harm follows a quantitative law,

\mathrm{danger}=\mathrm{play\_cost}\times(1-\mathrm{rarity})^N

, whose

(1-\mathrm{rarity})^N

gate-miss factor is proven exact and whose play cost is empirically bounded. (3) The failure is not repaired by more data: LLM synthesis behaves as rule translation, not rule inference, and did not infer the omitted rule across models (GPT-5.x) and data regimes (including DAgger and targeted examples). (4) The same mechanism recurs on the belief-inference function of imperfect-information CWMs: we prove a coverage bound (a size-

N

gate is identifying when

N\gtrsim b^{d_{\max}}

), explaining why shallow games such as Kuhn poker show no gap, and hand-construct Beacon, a verified-but-wrong inference function that passes the gate yet loses every game. These results suggest adequacy for planning-oriented world models should be measured on the search distribution or by play directly, not by prediction accuracy on sampled transitions.

Abstract

Explore similar work

When a Verified World Model Still Loses: Play-Adequacy vs Prediction-Accuracy in LLM-Synthesized Code World Models

An Enclosed Mode Is a Gauge Choice: Topology Relative to Reach in Certified Code World Models

Where World Models Break: Natural-Input Failure Discovery