World models enable agents to plan and learn from imagined experience, but in an unfamiliar environment they must learn its mechanics from limited interaction. A mechanic is a rule for how an action or an interaction changes part of the environment. Each occurrence of a mechanic that the model observes is one exposure. We ask how many exposures a world model needs to predict the effects of a new environment's mechanics. We address this question on ARC-AGI-3, a benchmark of interactive games defined by their programs. We introduce MechEval, which labels 69 recurring mechanics across 20 games from these programs. It scores each prediction on the cells the mechanic changes at each exposure. As a test bed, we build WARC (World model for ARC-AGI-3), a 35M-parameter vision transformer that is pretrained on other games and updates all of its parameters after each transition. On held-out games, WARC learns to predict four in five mechanics with at least 80% accuracy on the cells they change and keeps that accuracy to the end of the recordings. Half of them reach that accuracy within about 80 exposures, in a single play of the game, and without updates only one in seven does at all. In closed-loop play on the public games, WARC-Agent, which pairs WARC with a fixed graph explorer, triples the explorer's official efficiency score from 0.15 to 0.46, mostly through test-time training. On the hidden games of the Kaggle leaderboard, our entry scores 0.20.
Figures & tables
Figure 1: WARC and MechEval. Left: WARC is pretrained on recordings of many games (top). In a new game (bottom), it predicts the next frame from the last three frames and actions, then updates its weights. Right: For each mechanic that fires, MechEval scores the prediction made before the update on the cells it changes, as given by the game program. The exposure k counts the mechanic’s earlier occurrences. It reaches the criterion at the first k from which its accuracy stays at or above 80%. The accuracies shown are illustrative.
Method
k =0 ↑
All ↑
EtC ↓
n.r. ↓
FE ↓
%
%
exposures
%
%
Episodic retrieval
0.1
32.9
109.0
76.8
0.08
WARC (frozen)
1.8
18.4
114.0
85.5
0.11
WARC (scratch)
3.9
35.0
105.0
51.2
1.15
WARC (GRU)
0.8
24.0
87.0
84.1
0.24
DreamerV3
14.2
25.9
114.0
90.8
3.07
Table 1: Learning the mechanics of unseen games (69 mechanics). k =0: mechanic accuracy before any exposure; All: over all exposures; EtC: median over all mechanics, with the last exposure for those not reached; n.r.: the fraction of mechanics not reached; FE: false-edit rate. WARC, WARC (scratch), and DreamerV3 average three test-time seeds; the other rows do not update, so their predictions are the same under every seed. Bold: best per column.
Figure 2: Mechanics learned. Share of the 69 mechanics whose accuracy stays at or above 80% from exposure k to their last exposure. Solid lines update the weights at test time and dashed lines do not.
Figure 3: Generalization to other recordings. WARC adapts on one recording and, after k exposures to a mechanic, predicts the mechanic in the game’s other recordings without updates, for (a) 14 of the 15 direct key and (b) 15 of the 16 direct click mechanics (14 in level 1), those with occurrences to predict. Level-1 predictions are split by whether the adapted recording has shown the same key acting at the same place or the same click. Beyond k=20 , only a few direct click mechanics have predictions.
Figure 4: Generalization to new situations. Each row is the first time the mechanic fires ( k=0 ) or a later step new to the recording ( k≤5 ), with the input frame and action, the observed next frame, and the predictions with their mechanic accuracy. No model predicts the first time, but WARC then predicts the first move to the left after one move up, and a click on a new tile after seeing two, which WARC (frozen) and DreamerV3 miss.
Figure 5: Scaling pretraining data. A variant of WARC with a token per game, pretrained on one five-fold split ( Appendix H ). Exact-frame error, the fraction of next frames predicted with any wrong cell, on its five held-out games after test-time training on 1,000 of their transitions, against the number of pretraining transitions D . Labels give the number of games, or of recordings per game over all 20 games.
Accuracy (%)
Choice
Change
All
k =1–4
WARC
69.1
41.9
Pretraining
none
− 34.1
− 28.2
Update
every 5 transitions
− 1.7
− 10.5
Updated part
LoRA, rank 64
− 1.4
+ 3.8
last two blocks
− 4.6
− 6.7
Table 2: Ablations of WARC. Entries give the change from WARC in points when one choice changes, averaged over three test-time seeds. All, k =1–4: mechanic accuracy over all exposures and the first four. FE is 0.11–0.12% except without pretraining (1.15%). Paired intervals and the augmentation rows are in Tab. 11 .
Figure 6: Where the explorer acts in ft09 and tu93. The first 45 actions of level 1 in three windows of 15, for the explorer alone (Graph) and for WARC-Agent (Graph + WARC). The bar under each board is the share of the window’s actions that reached a game state not seen before in the level. These runs were repeated with logging and differ from those in Tab. 3 , as test-time training on the GPU is not deterministic. For tu93, we show the run with the median level-1 clear among seeds 0 to 9. S and G mark the start and the goal.
Agent
L1@1000 ↑
L2@1000 ↑
RHAE@1000 ↑
Random
10.4
0.0
0.045
From-scratch CNN [ 80 ]
18.4
0.0
0.099
Graph
31.6
5.6
0.149
Graph + WARC (frozen)
34.4
4.4
0.181
WARC-Agent
46.0
8.4
0.455
Graph + oracle world model
61.6
26.8
1.392
Table 3: Closed-loop play on the 25 public games with equal weight per game, over 10 seeds. L1@ N , L2@ N : fraction of runs that clear level 1 or 2 within N actions; RHAE@ N : the official action-efficiency score over the first N actions, on its 0–100 scale [ 3 ] ( Appendix I ). Graph: our explorer, alone or with the world model ranking its actions. The from-scratch CNN is StochasticGoose [ 80 ] , reinitialized at every level. The oracle world model returns the true next frame, an upper bound rather than an agent. Bold: best agent.
Appendix figures & tables23 assets
Supplementary material from the paper’s appendix.
Appendix
Game
Rec.
Trans.
Mech.
Occ.
Med.
Game
Rec.
Trans.
Mech.
Occ.
Med.
ar25
10
1,002
4
1,570
32
s5i5
11
1,942
2
2,460
62
cd82
11
654
5
824
7
sb26
12
824
3
973
22.5
dc22
11
1,632
4
2,064
22.5
sc25
15
694
3
865
13
ft09
10
547
2
636
28
sk48
14
3,730
5
6,280
66
ka59
10
1,163
5
1,611
16
su15
13
844
3
1,262
27
lp85
54
4,266
3
6,834
24
tn36
14
1,462
3
2,351
34
Appendix
Table 4: MechEval data per game, up to the first completion of level 2: recordings (Rec.), transitions (Trans.), scored mechanics (Mech.), their scored occurrences (Occ.), and the median number of occurrences of a mechanic in a recording in which it fires (Med.).
Kind
Definition
Mechanics
Example
Direct key
A key moves, tilts, extends, or changes an object
15
ls20 player moves
Direct click
A click changes the clicked object or the object that a button drives
16
lp85 tile row shifts
Linked
Copies or marks another change
10
re86 piece selection
Contact
An object reaches a place or touches another object
6
ka59 pushed block slides
Counter
A step or budget bar changes
21
ls20 steps bar
Autonomous
An object moves by its own rule
1
wa30 orange agent
Appendix
Table 5: The kinds of the 69 mechanics.
Figure 7: The mechanics of MechEval, game by game: ar25 to lp85 (ls20 to wa30 in Figs. 8 and 9 ). Each mechanic is shown at a representative occurrence: among its scored occurrences in all recordings, one whose number of scored cells is closest to the mechanic’s median, with the fewest other cells changed in the same step. The panels show the frame before, the frame after, and the change, with the scored cells Cm(t) in teal and the other cells changed in the same step in gray. Within a game, the mechanics are ordered by kind, which the label above each panel names ( Appendix B ).
Figure 8: The mechanics of MechEval, game by game: ls20 to sc25 , shown as in Fig. 7 . The ls20 step refill is shown at a refill of the steps bar, since half of its occurrences take the item while the bar is full.
Figure 9: The mechanics of MechEval, game by game: sk48 to wa30 , shown as in Fig. 7 .
VARC
WARC
Task
output grid of a static puzzle
next frame of an interactive game
Input
one grid on a 64×64 canvas
the last three 64×64 frames and actions
Transformer
width 512, 10 blocks, 8 heads, 2×2 patches
the same, with 3D patches over the three frames
Output
a color for every cell
the same
Conditioning
one learned token per puzzle
the actions through adaptive layer normalization and a learned vector at a clicked patch, no task token
Parameters
18M
35.3M, of which 15.8M modulate the blocks by the actions and 3.1M embed the 3D patches
Appendix
Table 6: What WARC takes from VARC and what it changes ( Sec. 5 ). The VARC column follows Hu et al. [39] for their default model.
Pretraining
Test-time training
Optimizer
Muon + AdamW
Muon + AdamW
Learning rate
3×10−4
3×10−4
Weight decay
0
0.01
Schedule
warmup, cosine
constant
Batch
516
8
Length
30 epochs
20 steps per transition
Appendix
Table 7: Optimization settings. In both stages, Muon [ 41 ] updates the two-dimensional weights of the transformer blocks except the modulation layers, with momentum 0.95, and AdamW [ 59 ] updates the remaining parameters. Pretraining warms up the learning rate linearly over six epochs and then decays it with a cosine schedule [ 58 ] . The GPUs are RTX PRO 6000, and test-time training runs on one.
Method
k =0 ↑
All ↑
EtC ↓
n.r. ↓
FE ↓
%
%
exposures
%
%
WARC (frozen)
1.8
18.4
114.0
85.5
0.11
WARC
3.3
69.1
74.0
18.8
0.11
WARC (GRU)
0.8
24.0
87.0
84.1
0.24
WARC (GRU), updated
2.5
70.0
74.0
18.8
0.11
Appendix
Table 8: Gradient updates versus a recurrent state. WARC and WARC (GRU) with and without updates on all three folds (69 mechanics), with the columns of Tab. 1 , in test-time seed 0.
Figure 18
Figure 11: The learning curve of every mechanic: direct key and direct click mechanics (the other kinds in Figs. 12 and 13 ). Mechanic accuracy of WARC and WARC (frozen) against exposure k , averaged over each mechanic’s occurrences with k earlier occurrences in all recordings, at every k , for WARC in test-time seed 0. Lines are the mean over the last five exposures, and the faint lines the per-exposure values, on which EtC is computed. Every test-time seed brings 56 mechanics to criterion, as in Tab. 1 . Mechanics are grouped by kind ( Appendix B ) and ordered by EtC within each kind, the exposures to criterion of WARC (n.r.: not reached). The dashed line marks the median exposure at the mechanic’s first occurrence in level 2, over the recordings in which it fires in both levels, and the gray line the 80% criterion.
Figure 12: The learning curve of every mechanic: linked, contact, and autonomous mechanics , shown as in Fig. 11 .
Figure 13: The learning curve of every mechanic: counters , shown as in Fig. 11 .
Figure 14: Withheld transitions. Mechanic accuracy on 32 target mechanics for WARC with their transitions in the buffer or withheld from it, and for WARC (frozen), which is not updated.
Figure 15: Learning curves by kind of mechanic. Mechanic accuracy against exposure k , at every exposure up to k=100 , with equal weight per mechanic within each kind of Appendix B , without the single autonomous mechanic.
Figure 16: Memorization or generalization. Mechanic accuracy against exposure k , at every exposure up to k=100 , split by what is new about each occurrence within its recording: (a) the 13 moves, (b) the 16 direct click mechanics, and (c) all 69 mechanics by the pattern of changed cells. Each occurrence belongs to one class, so a class can be absent at small k . For example, a move at k=1 is never a repeat. A point is drawn where at least five mechanics have an occurrence, and the line breaks elsewhere.
Figure 17: Generalization to other recordings, by class. Fig. 3 with the unseen occurrences split by what is new about them, as in Fig. 16 , for (a) the 12 moves and (b) the 15 direct click mechanics with occurrences to predict (14 in level 1), and (c) all 64 mechanics with occurrences in other recordings (63 in level 1). Bands show 95% intervals for WARC, and dashed lines show WARC (frozen) on the same occurrences. Every exposure up to k=49 is drawn. Beyond k=20 , only a few direct click mechanics have occurrences.
Figure 18: The split for other world models. Mechanic accuracy at five to nineteen exposures on repeats, new positions, and new directions, objects, or forms, for WARC, WARC (frozen), WARC (scratch), episodic retrieval, and DreamerV3. DreamerV3 uses its recipe tuned on the five games outside MechEval.
Group
Class
Seed 0
Seed 1
Seed 2
Mean
vs. repeat
Moves
repeat
91.5
91.9
91.8
91.7
–
new position
87.2
86.8
86.6
86.9
−4.9
new direction
76.0
77.3
78.5
77.3
−14.5
Direct click
repeat
70.5
73.0
74.3
72.6
–
new location
54.5
54.4
54.9
54.6
−18.0
new object
34.7
36.6
35.6
35.6
−37.0
Appendix
Table 10: The split of Fig. 16 over three test-time seeds: mechanic accuracy (%) at five to nineteen exposures, with equal weight per mechanic, and the mean difference from repeats in the same seed. The differences for new directions, locations, objects, and forms exclude zero in every seed, and the intervals for new positions reach zero in every seed.
Figure 19: Blocked moves. The split of moves in Fig. 16 a for blocked moves (19 types in 12 games), scored on the region’s unchanged cells (predicting no change scores 100), at the first blocked move and at five to nineteen exposures, with 95% intervals.
Figure 20: More examples of generalization to new occurrences. Fig. 4 for three moves (left) and three direct click mechanics (right). The last row of sc25 player is in level 2.
Choice
Change
All
k =1–4
Pretraining
none
−34.1 [ −37.4 , −31.3 ]
−28.2 [ −32.2 , −24.2 ]
Update
every 5 transitions
−1.7 [ −2.6 , −0.9 ]
−10.5 [ −14.2 , −7.0 ]
Updated part
LoRA, rank 64
−1.4 [ −2.6 , +0.2 ]
+3.8 [ +2.1 , +5.6 ]
last two blocks
−4.6 [ −5.6 , −3.7 ]
−6.7 [ −9.3 , −4.1 ]
modulation layers
−9.4 [ −11.6 , −7.4 ]
−7.2 [ −10.0 , −4.5 ]
Augmentation
none
−1.1 [ −1.9 , −0.2 ]
−1.8 [ −3.6 , +0.1 ]
Appendix
Table 11: Paired 95% bootstrap intervals for the ablations: the change from WARC in mechanic accuracy over all exposures and the first four, averaged over three test-time seeds, resampling games and then recordings 2,000 times with draws shared by the seeds and the rows.
Figure 21: Pretraining scale, continued. (a) Changed-cell error after test-time training against pretraining transitions, for the runs of Fig. 5 . Both series flatten. (b) Exact-frame accuracy during test-time training for models pretrained on 2, 8, and 20 games, evaluated at 0, 50, 100, and every 200 transitions. Means over five held-out games, one pretraining seed.
Explorer
L1@1000 ↑
L2@1000 ↑
RHAE@1000 ↑
Graph + WARC (WARC-Agent)
47.2
8.8
0.456
without the mask of action-independent cells
47.2
9.6
0.513
a random sample of 32 actions, no mask
42.4
10.4
0.346
one planning step
44.0
8.0
0.473
no random action
44.8
10.4
0.520
Graph + oracle world model
63.2
27.2
1.326
Appendix
Table 12: The explorer with one component removed, and with an oracle world model, on the 25 public games over seeds 0 to 4, with the measures of Tab. 3 . Every row but the last uses WARC with test-time training, and the last row is the oracle world model of Tab. 3 , on these seeds.
We present a Test-time World-model Inference (Twin) system, in which a frontier coding agent writes an executable world model for completing continual learning tasks, such as ARC-AGI-3 games. Traditional approaches hand-engineer such models, one custom design per task. Each game hides its rules and goal, and our system constructs them from simulation and interaction alone. Its inductive prior over grid games is strong enough to recover the true transitions of the game and the goal on nearly all levels. Replay validation happens in a twin world model. The harness enforces that an action is not made until the program reproduces every previous observed game transition. Each mismatch between a world model prediction and the actual action result becomes a counterexample that is used to repair the world model. Twin clears 179 out of 183 levels (97.8%), and does so more efficiently than humans in 158 out of 179 levels (88.3%). The system infers the goal before any reward on 156 of the levels it clears (87.2%), and in the remaining levels automatically discovers the goal by search. The benchmark scores completion and action efficiency, between 0 and 100, against humans playing each game for the first time. Played directly, the base model scores only 7.8%; an off-the-shelf harness increases it to 61.1%, whereas our twin world model increases the same base model to 93.3%, clearing 23 out of 25 games. Building a usable world model is simpler than anticipated, whereas the harder problem is inferring the right goal.
In model-based learning, the agent learns behaviors by simulating trajectories based on world model predictions. Standard world models typically learn a stationary transition function that maps states and actions to next states, when an action and an outcome frequently co-occur in training data, the model tends to internalize this correlation as a general causal rule while ignoring action preconditions. In interactive environments, however, agent actions can reshape the future affordance space. At each timestep, an action may becomes executable only after its prerequisites are met, or non-executable when they are destroyed. We term such events structure-changing events (SC events). As a result, a conventional world model often fails to determine whether a given action is executable in the current state, especially in multi-step predictions. Each imagined step is conditioned on an incorrect affordance state, and therefore the prediction error compounds over the rollout horizon. In this paper, we propose AGWM (Affordance-Grounded World Model), which learns an abstract affordance structure represented as a DAG of prerequisite dependencies to explicitly track the dynamic executability of actions. Experiments on game-based simulated environments demonstrate the effectiveness of our method by achieving lower multi-step prediction error, better generalization to novel configurations, and improved interpretability.
Qinshi Zhang, Weipeng Deng, Zhihan Jiang +4
University of California, San Diego · University of Hong Kong · Columbia University +2
Predicting how an environment will change before acting is a natural route to better decision making for agents. Recent post-training methods therefore require agents to predict the next observation and turn that prediction into a reward or a direct supervision signal, which is called world model. Existing next-observation training methods help the agent to learn the environmental content. However, they additionally involve an optimization process, which may introduce several effects other than learning to predict the world. Consequently, where the performance gain comes from during the training process remains an open question. We answer this research question through replacing true next-observation targets with in-distribution mismatched observations during the training process. Across two interactive text environments, mismatched targets lower prediction accuracy by 15.3-61.6% relative to ground-truth targets, yet retain substantial task gains over the base model. Compared with the base model, trained models consider more candidate actions and exhibit less looping. We also introduce a setting that replaces prediction-based rewards with independent random signals. This training expands task coverage (pass@64) even when the reward carries no environment information. We also generalize this finding to VisualWebArena, where random-reward training raises pass@64 by 14.3% relative to the base model, without observation-matching rewards or an external multimodal teacher for reward construction.
Xinyu Che, Hang Yan, Yanchen Liu +5
Xi’an Jiaotong University · University of Southern California · University of the Chinese Academy of Sciences +1