World models enable agents to plan and learn from imagined experience, but in an unfamiliar environment they must learn its mechanics from limited interaction. A mechanic is a rule for how an action or an interaction changes part of the environment. Each occurrence of a mechanic that the model observes is one exposure. We ask how many exposures a world model needs to predict the effects of a new environment's mechanics. We address this question on ARC-AGI-3, a benchmark of interactive games defined by their programs. We introduce MechEval, which labels 69 recurring mechanics across 20 games from these programs. It scores each prediction on the cells the mechanic changes at each exposure. As a test bed, we build WARC (World model for ARC-AGI-3), a 35M-parameter vision transformer that is pretrained on other games and updates all of its parameters after each transition. On held-out games, WARC learns to predict four in five mechanics with at least 80% accuracy on the cells they change and keeps that accuracy to the end of the recordings. Half of them reach that accuracy within about 80 exposures, in a single play of the game, and without updates only one in seven does at all. In closed-loop play on the public games, WARC-Agent, which pairs WARC with a fixed graph explorer, triples the explorer's official efficiency score from 0.15 to 0.46, mostly through test-time training. On the hidden games of the Kaggle leaderboard, our entry scores 0.20.
Figures & tables
Figure 1: WARC and MechEval. Left: WARC is pretrained on recordings of many games (top). In a new game (bottom), it predicts the next frame from the last three frames and actions, then updates its weights. Right: For each mechanic that fires, MechEval scores the prediction made before the update on the cells it changes, as given by the game program. The exposure k counts the mechanic’s earlier occurrences. It reaches the criterion at the first k from which its accuracy stays at or above 80%. The accuracies shown are illustrative.
Method
k =0 ↑
All ↑
EtC ↓
n.r. ↓
FE ↓
%
%
exposures
%
%
Episodic retrieval
0.1
32.9
109.0
76.8
0.08
WARC (frozen)
1.8
18.4
114.0
85.5
0.11
WARC (scratch)
3.9
35.0
105.0
51.2
1.15
WARC (GRU)
0.8
24.0
87.0
84.1
0.24
DreamerV3
14.2
25.9
114.0
90.8
3.07
Table 1: Learning the mechanics of unseen games (69 mechanics). k =0: mechanic accuracy before any exposure; All: over all exposures; EtC: median over all mechanics, with the last exposure for those not reached; n.r.: the fraction of mechanics not reached; FE: false-edit rate. WARC, WARC (scratch), and DreamerV3 average three test-time seeds; the other rows do not update, so their predictions are the same under every seed. Bold: best per column.
Figure 2: Mechanics learned. Share of the 69 mechanics whose accuracy stays at or above 80% from exposure k to their last exposure. Solid lines update the weights at test time and dashed lines do not.
Figure 3: Generalization to other recordings. WARC adapts on one recording and, after k exposures to a mechanic, predicts the mechanic in the game’s other recordings without updates, for (a) 14 of the 15 direct key and (b) 15 of the 16 direct click mechanics (14 in level 1), those with occurrences to predict. Level-1 predictions are split by whether the adapted recording has shown the same key acting at the same place or the same click. Beyond k=20 , only a few direct click mechanics have predictions.
Figure 4: Generalization to new situations. Each row is the first time the mechanic fires ( k=0 ) or a later step new to the recording ( k≤5 ), with the input frame and action, the observed next frame, and the predictions with their mechanic accuracy. No model predicts the first time, but WARC then predicts the first move to the left after one move up, and a click on a new tile after seeing two, which WARC (frozen) and DreamerV3 miss.
Figure 5: Scaling pretraining data. A variant of WARC with a token per game, pretrained on one five-fold split ( Appendix H ). Exact-frame error, the fraction of next frames predicted with any wrong cell, on its five held-out games after test-time training on 1,000 of their transitions, against the number of pretraining transitions D . Labels give the number of games, or of recordings per game over all 20 games.
Accuracy (%)
Choice
Change
All
k =1–4
WARC
69.1
41.9
Pretraining
none
− 34.1
− 28.2
Update
every 5 transitions
− 1.7
− 10.5
Updated part
LoRA, rank 64
− 1.4
+ 3.8
last two blocks
− 4.6
− 6.7
Table 2: Ablations of WARC. Entries give the change from WARC in points when one choice changes, averaged over three test-time seeds. All, k =1–4: mechanic accuracy over all exposures and the first four. FE is 0.11–0.12% except without pretraining (1.15%). Paired intervals and the augmentation rows are in Tab. 11 .
Figure 6: Where the explorer acts in ft09 and tu93. The first 45 actions of level 1 in three windows of 15, for the explorer alone (Graph) and for WARC-Agent (Graph + WARC). The bar under each board is the share of the window’s actions that reached a game state not seen before in the level. These runs were repeated with logging and differ from those in Tab. 3 , as test-time training on the GPU is not deterministic. For tu93, we show the run with the median level-1 clear among seeds 0 to 9. S and G mark the start and the goal.
Agent
L1@1000 ↑
L2@1000 ↑
RHAE@1000 ↑
Random
10.4
0.0
0.045
From-scratch CNN [ 80 ]
18.4
0.0
0.099
Graph
31.6
5.6
0.149
Graph + WARC (frozen)
34.4
4.4
0.181
WARC-Agent
46.0
8.4
0.455
Graph + oracle world model
61.6
26.8
1.392
Table 3: Closed-loop play on the 25 public games with equal weight per game, over 10 seeds. L1@ N , L2@ N : fraction of runs that clear level 1 or 2 within N actions; RHAE@ N : the official action-efficiency score over the first N actions, on its 0–100 scale [ 3 ] ( Appendix I ). Graph: our explorer, alone or with the world model ranking its actions. The from-scratch CNN is StochasticGoose [ 80 ] , reinitialized at every level. The oracle world model returns the true next frame, an upper bound rather than an agent. Bold: best agent.
Appendix figures & tables23 assets
Supplementary material from the paper’s appendix.
Appendix
Game
Rec.
Trans.
Mech.
Occ.
Med.
Game
Rec.
Trans.
Mech.
Occ.
Med.
ar25
10
1,002
4
1,570
32
s5i5
11
1,942
2
2,460
62
cd82
11
654
5
824
7
sb26
12
824
3
973
22.5
dc22
11
1,632
4
2,064
22.5
sc25
15
694
3
865
13
ft09
10
547
2
636
28
sk48
14
3,730
5
6,280
66
ka59
10
1,163
5
1,611
16
su15
13
844
3
1,262
27
lp85
54
4,266
3
6,834
24
tn36
14
1,462
3
2,351
34
Appendix
Table 4: MechEval data per game, up to the first completion of level 2: recordings (Rec.), transitions (Trans.), scored mechanics (Mech.), their scored occurrences (Occ.), and the median number of occurrences of a mechanic in a recording in which it fires (Med.).
Kind
Definition
Mechanics
Example
Direct key
A key moves, tilts, extends, or changes an object
15
ls20 player moves
Direct click
A click changes the clicked object or the object that a button drives
16
lp85 tile row shifts
Linked
Copies or marks another change
10
re86 piece selection
Contact
An object reaches a place or touches another object
6
ka59 pushed block slides
Counter
A step or budget bar changes
21
ls20 steps bar
Autonomous
An object moves by its own rule
1
wa30 orange agent
Appendix
Table 5: The kinds of the 69 mechanics.
Figure 7: The mechanics of MechEval, game by game: ar25 to lp85 (ls20 to wa30 in Figs. 8 and 9 ). Each mechanic is shown at a representative occurrence: among its scored occurrences in all recordings, one whose number of scored cells is closest to the mechanic’s median, with the fewest other cells changed in the same step. The panels show the frame before, the frame after, and the change, with the scored cells Cm(t) in teal and the other cells changed in the same step in gray. Within a game, the mechanics are ordered by kind, which the label above each panel names ( Appendix B ).
Figure 8: The mechanics of MechEval, game by game: ls20 to sc25 , shown as in Fig. 7 . The ls20 step refill is shown at a refill of the steps bar, since half of its occurrences take the item while the bar is full.
Figure 9: The mechanics of MechEval, game by game: sk48 to wa30 , shown as in Fig. 7 .
VARC
WARC
Task
output grid of a static puzzle
next frame of an interactive game
Input
one grid on a 64×64 canvas
the last three 64×64 frames and actions
Transformer
width 512, 10 blocks, 8 heads, 2×2 patches
the same, with 3D patches over the three frames
Output
a color for every cell
the same
Conditioning
one learned token per puzzle
the actions through adaptive layer normalization and a learned vector at a clicked patch, no task token
Parameters
18M
35.3M, of which 15.8M modulate the blocks by the actions and 3.1M embed the 3D patches
Appendix
Table 6: What WARC takes from VARC and what it changes ( Sec. 5 ). The VARC column follows Hu et al. [39] for their default model.
Pretraining
Test-time training
Optimizer
Muon + AdamW
Muon + AdamW
Learning rate
3×10−4
3×10−4
Weight decay
0
0.01
Schedule
warmup, cosine
constant
Batch
516
8
Length
30 epochs
20 steps per transition
Appendix
Table 7: Optimization settings. In both stages, Muon [ 41 ] updates the two-dimensional weights of the transformer blocks except the modulation layers, with momentum 0.95, and AdamW [ 59 ] updates the remaining parameters. Pretraining warms up the learning rate linearly over six epochs and then decays it with a cosine schedule [ 58 ] . The GPUs are RTX PRO 6000, and test-time training runs on one.
Method
k =0 ↑
All ↑
EtC ↓
n.r. ↓
FE ↓
%
%
exposures
%
%
WARC (frozen)
1.8
18.4
114.0
85.5
0.11
WARC
3.3
69.1
74.0
18.8
0.11
WARC (GRU)
0.8
24.0
87.0
84.1
0.24
WARC (GRU), updated
2.5
70.0
74.0
18.8
0.11
Appendix
Table 8: Gradient updates versus a recurrent state. WARC and WARC (GRU) with and without updates on all three folds (69 mechanics), with the columns of Tab. 1 , in test-time seed 0.
Figure 18
Figure 11: The learning curve of every mechanic: direct key and direct click mechanics (the other kinds in Figs. 12 and 13 ). Mechanic accuracy of WARC and WARC (frozen) against exposure k , averaged over each mechanic’s occurrences with k earlier occurrences in all recordings, at every k , for WARC in test-time seed 0. Lines are the mean over the last five exposures, and the faint lines the per-exposure values, on which EtC is computed. Every test-time seed brings 56 mechanics to criterion, as in Tab. 1 . Mechanics are grouped by kind ( Appendix B ) and ordered by EtC within each kind, the exposures to criterion of WARC (n.r.: not reached). The dashed line marks the median exposure at the mechanic’s first occurrence in level 2, over the recordings in which it fires in both levels, and the gray line the 80% criterion.
Figure 12: The learning curve of every mechanic: linked, contact, and autonomous mechanics , shown as in Fig. 11 .
Figure 13: The learning curve of every mechanic: counters , shown as in Fig. 11 .
Figure 14: Withheld transitions. Mechanic accuracy on 32 target mechanics for WARC with their transitions in the buffer or withheld from it, and for WARC (frozen), which is not updated.
Figure 15: Learning curves by kind of mechanic. Mechanic accuracy against exposure k , at every exposure up to k=100 , with equal weight per mechanic within each kind of Appendix B , without the single autonomous mechanic.
Figure 16: Memorization or generalization. Mechanic accuracy against exposure k , at every exposure up to k=100 , split by what is new about each occurrence within its recording: (a) the 13 moves, (b) the 16 direct click mechanics, and (c) all 69 mechanics by the pattern of changed cells. Each occurrence belongs to one class, so a class can be absent at small k . For example, a move at k=1 is never a repeat. A point is drawn where at least five mechanics have an occurrence, and the line breaks elsewhere.
Figure 17: Generalization to other recordings, by class. Fig. 3 with the unseen occurrences split by what is new about them, as in Fig. 16 , for (a) the 12 moves and (b) the 15 direct click mechanics with occurrences to predict (14 in level 1), and (c) all 64 mechanics with occurrences in other recordings (63 in level 1). Bands show 95% intervals for WARC, and dashed lines show WARC (frozen) on the same occurrences. Every exposure up to k=49 is drawn. Beyond k=20 , only a few direct click mechanics have occurrences.
Figure 18: The split for other world models. Mechanic accuracy at five to nineteen exposures on repeats, new positions, and new directions, objects, or forms, for WARC, WARC (frozen), WARC (scratch), episodic retrieval, and DreamerV3. DreamerV3 uses its recipe tuned on the five games outside MechEval.
Group
Class
Seed 0
Seed 1
Seed 2
Mean
vs. repeat
Moves
repeat
91.5
91.9
91.8
91.7
–
new position
87.2
86.8
86.6
86.9
−4.9
new direction
76.0
77.3
78.5
77.3
−14.5
Direct click
repeat
70.5
73.0
74.3
72.6
–
new location
54.5
54.4
54.9
54.6
−18.0
new object
34.7
36.6
35.6
35.6
−37.0
Appendix
Table 10: The split of Fig. 16 over three test-time seeds: mechanic accuracy (%) at five to nineteen exposures, with equal weight per mechanic, and the mean difference from repeats in the same seed. The differences for new directions, locations, objects, and forms exclude zero in every seed, and the intervals for new positions reach zero in every seed.
Figure 19: Blocked moves. The split of moves in Fig. 16 a for blocked moves (19 types in 12 games), scored on the region’s unchanged cells (predicting no change scores 100), at the first blocked move and at five to nineteen exposures, with 95% intervals.
Figure 20: More examples of generalization to new occurrences. Fig. 4 for three moves (left) and three direct click mechanics (right). The last row of sc25 player is in level 2.
Choice
Change
All
k =1–4
Pretraining
none
−34.1 [ −37.4 , −31.3 ]
−28.2 [ −32.2 , −24.2 ]
Update
every 5 transitions
−1.7 [ −2.6 , −0.9 ]
−10.5 [ −14.2 , −7.0 ]
Updated part
LoRA, rank 64
−1.4 [ −2.6 , +0.2 ]
+3.8 [ +2.1 , +5.6 ]
last two blocks
−4.6 [ −5.6 , −3.7 ]
−6.7 [ −9.3 , −4.1 ]
modulation layers
−9.4 [ −11.6 , −7.4 ]
−7.2 [ −10.0 , −4.5 ]
Augmentation
none
−1.1 [ −1.9 , −0.2 ]
−1.8 [ −3.6 , +0.1 ]
Appendix
Table 11: Paired 95% bootstrap intervals for the ablations: the change from WARC in mechanic accuracy over all exposures and the first four, averaged over three test-time seeds, resampling games and then recordings 2,000 times with draws shared by the seeds and the rows.
Figure 21: Pretraining scale, continued. (a) Changed-cell error after test-time training against pretraining transitions, for the runs of Fig. 5 . Both series flatten. (b) Exact-frame accuracy during test-time training for models pretrained on 2, 8, and 20 games, evaluated at 0, 50, 100, and every 200 transitions. Means over five held-out games, one pretraining seed.
Explorer
L1@1000 ↑
L2@1000 ↑
RHAE@1000 ↑
Graph + WARC (WARC-Agent)
47.2
8.8
0.456
without the mask of action-independent cells
47.2
9.6
0.513
a random sample of 32 actions, no mask
42.4
10.4
0.346
one planning step
44.0
8.0
0.473
no random action
44.8
10.4
0.520
Graph + oracle world model
63.2
27.2
1.326
Appendix
Table 12: The explorer with one component removed, and with an oracle world model, on the 25 public games over seeds 0 to 4, with the measures of Tab. 3 . Every row but the last uses WARC with test-time training, and the last row is the oracle world model of Tab. 3 , on these seeds.