Compiling Known Rules into Compact Neural Agents: Weight-Space Programming on Unseen Games
Organizations: Amazon, TU Dresden
Abstract
Weight-space programming (WSP) uses language models to construct neural network weights without training. We test whether known game mechanics can be compiled into compact agents that generalise to held-out levels, renderings and start states within a game family. Writers receive simulator source with held-out level data removed for 20 privately authored, unpublished ARC-AGI-3-style game families. With a fixed 57k-parameter recurrent architecture, the strongest writer reaches 56% held-out-level success with feedback, while the four main writers' Python players reach 97-100%. An exploratory free-architecture condition reaches 68-90%. Learning from support-level examples transfers much less well, but these learners lack the source information given to writers. Written networks have a program-like structure: 93% of parameters are zero, and 90% of the rest are small integers or powers of two. NumPy players under comparable computation caps reach 91%, versus the networks' 79%. Training the strongest writer's architectures from scratch yields about 1% success within our budget. Written fixed-network weights are fragile, and standard PPO removes most of their success. Prompts for noise robustness increase tolerance by about four orders of magnitude for two of three writers, but PPO makes larger updates and still removes most success. These results demonstrate rule-to-network compilation in this setting, while showing limited benefits over Python programs and no successful combination with the tested PPO fine-tuning.
Figures & tables
| Writer | W1 | W2 | P1 | P2 | W1 deliv. | W2 final lvl |
|---|---|---|---|---|---|---|
| GPT-6 Astra | 0.40 [0.26,0.54] | 0.56 [0.40,0.71] | 0.95 [0.86,1.00] | 1.00 [1.00,1.00] | 1.00 | 0.40 |
| GPT-6.1 Sol | 0.32 [0.17,0.49] | 0.43 [0.28,0.59] | 0.96 [0.91,1.00] | 1.00 [0.98,1.00] | 0.98 | 0.31 |
| Claude Opus 5.5 | 0.15 [0.06,0.26] | 0.36 [0.20,0.52] | 0.85 [0.73,0.94] | 0.97 [0.91,1.00] | 0.85 | 0.25 |
| GPT-6.1 Terra | 0.21 [0.09,0.36] | 0.33 [0.19,0.48] | 0.97 [0.92,1.00] | 0.98 [0.95,1.00] | 0.98 | 0.18 |
| GPT-6 Luna | 0.05 [0.01,0.11] | 0.11 [0.04,0.19] | 0.45 [0.31,0.60] | – | 0.70 | 0.02 |
| Claude Sonnet 5.5 | 0.06 [0.02,0.13] | 0.19 [0.08,0.33] | 0.72 [0.59,0.84] | – | 1.00 | 0.12 |
| Writer | W1 fixed | W1 FreeArch | W2 fixed | W2 FreeArch | W2 [95% CI] | FreeArch fixed |
| GPT-6 Astra | 0.40 | 0.74 | 0.56 | 0.90 | +0.34 [+0.21,+0.48] | 17/20 |
| GPT-6.1 Sol | 0.32 | 0.52 | 0.43 | 0.84 | +0.41 [+0.25,+0.56] | 17/20 |
| Claude Opus 5.5 | 0.15 | 0.40 | 0.36 | 0.68 | +0.33 [+0.22,+0.45] | 18/20 |
| GPT-6.1 Terra | 0.21 | 0.51 | 0.33 | 0.74 | +0.41 [+0.30,+0.52] | 19/20 |
| GPT-6 Luna | 0.05 | 0.09 | 0.11 | 0.14 | +0.04 [-0.01,+0.09] | 8/20 |
| Claude Sonnet 5.5 | 0.06 | 0.17 | 0.19 | 0.39 | +0.20 [+0.11,+0.29] | 14/20 |
| Method | S-start | Q-lvl | Q-tpl | families |
|---|---|---|---|---|
| BC-support | 0.95 | 0.11 | 0.13 | 20 |
| BC-aug | 0.20 | 0.09 | 0.21 | 20 |
| BC-aug (dihedral only) | 0.56 | 0.10 | 0.25 | 13 |
| Distill-aug | 0.17 | 0.07 | 0.17 | 20 |
| Distill (no aug.) | 0.92 | 0.10 | 0.15 | 20 |
| PPO | 0.47 | 0.08 | 0.12 | 20 |
| Relative noise | PPO, first | Random update, | |||
|---|---|---|---|---|---|
| Written weights | 0 | checkpoint | same size | ||
| Fixed actor, noise on all entries | 0.51 | 0.31 | 0.05 | 0.04 | 0.06 |
| Fixed actor, noise on nonzero entries only | 0.51 | 0.45 | 0.08 | ||
| FreeArch networks, noise on all entries | 0.95 | 0.72 | 0.44 | ||
Appendix figures & tables23 assets
Supplementary material from the paper’s appendix.
Appendix
| Tensor | Shape | Entries | Role |
|---|---|---|---|
| , | , | 15,232 | convolution on |
| , | , | 9,248 | convolution |
| , | , | 3,104 | mixing of |
| 2,304 | weight-tied recurrence on | ||
| , | , | 528 | drive from into the recurrence |
| , | , | 12,416 | MLP on pooled features |
| Parameter | Value | |
| DAgger | collection rounds, mixing | 7 (round 0 teacher only), |
| epochs per round, optimiser | 4, Adam (fresh per round, lr ) | |
| minibatch, gradient clip | 64 frames, norm 1.0 | |
| buffer cap | 200,000 samples (whole random episodes dropped) | |
| planner teacher | 5 s per state, no RESET fallback | |
| BPTT window, progressive loss | 16 steps, on ( ) |
| Family | Stratum | Actions | Held-out levels | Levels | S-start | Q-lvl | Q-tpl |
|---|---|---|---|---|---|---|---|
| ztl | easy | A1,A2,A3,A4 | 2,4,6,8 | 8 | 20 | 210 | 50 |
| wih | easy | A1,A5 | 2,4,6 | 6 | 15 | 180 | 45 |
| qcn | easy | A1,A2,A3,A4,A5 | 2,4,6,8 | 8 | 20 | 240 | 60 |
| lys | medium | A1,A2,A3,A4 | 2,4,6,8,10,12 | 12 | 30 | 360 | 90 |
| esm | medium | A6 | 2,4,6,8,10 | 10 | 25 | 300 | 75 |
| aiv | medium | A6 | 2,4,6,8,10,12 | 12 | 30 | 360 | 90 |
| Set | Levels | Templates / seeds | Starts per (level, template) |
| Private tier | |||
| S-start (feedback, training) | support | 0–4 | level start |
| S-dev (training only) | support | 0–4 | 4 perturbed |
| Q-off | support | 0–4 | 6 perturbed |
| Q-lvl (primary) | held out | 0–9 | level start + 5 perturbed |
| Q-tpl | support | 5–9 | level start + 2 perturbed |
| Requests by outcome | No | |||||||||
| Arm | Wr. | Sess. | Req. | Compl. | Trunc. | Ref. | Err. | Interr. | Re-run | artifact |
| Private tier (20 families) | ||||||||||
| W1 | 9 | 540 | 571 | 530 | 0 | 9 | 1 | 31 | 31 | 40 (7%) |
| P1 | 9 | 540 | 560 | 540 | 0 | 0 | 0 | 20 | 20 | 6 (1%) |
| W2 | 9 | 540 | 2,065 | 2,015 | 1 | 8 | 4 | 37 | 38 | 11 (2%) |
| P2 | 4 | 240 | 306 | 289 | 0 | 0 | 0 | 17 | 18 | 0 (0%) |
| Support levels | Held-out levels | |||||
| Policy source | Support | New | appearance [95% CI] | Support | New | Permuted |
| templ. | templ. | templ. | templ. | palette | ||
| GPT-6 Astra W2 | 0.79 | 0.78 | -0.01 [-0.03,+0.01] | 0.57 | 0.56 | 0.54 |
| GPT-6.1 Sol W2 | 0.62 | 0.60 | -0.02 [-0.04,0.00] | 0.44 | 0.43 | 0.39 |
| Claude Opus 5.5 W2 | 0.51 | 0.43 | -0.08 [-0.13,-0.03] | 0.39 | 0.33 | 0.31 |
| GPT-6.1 Terra W2 | 0.52 | 0.48 | -0.04 [-0.09,-0.01] | 0.35 | 0.32 | 0.30 |
| Weights ( actors) | Q-lvl [95% CI] | S-start [95% CI] | ||
| overall | max. tensor | |||
| Written W2 actor (18) | 0 | 0 | 0.51 [0.18, 0.82] | 0.70 [0.50, 0.89] |
| + isotropic noise, | 0.31 [0.11, 0.55] | 0.34 [0.11, 0.63] | ||
| + isotropic noise, | 0.24 [0.08, 0.46] | 0.27 [0.07, 0.55] | ||
| + isotropic noise, | 0.10 [0.02, 0.20] | 0.17 [0.05, 0.33] | ||
| + isotropic noise, | 0.05 [0.00, 0.13] | 0.12 [0.02, 0.26] | ||
| Held-out Q-lvl at relative noise | 0 | |||||||
|---|---|---|---|---|---|---|---|---|
| Fixed actor, all entries | 0.51 | 0.31 | 0.24 | 0.10 | 0.05 | 0.04 | 0.02 | 0.01 |
| Fixed actor, nonzero entries only | 0.51 | 0.45 | 0.31 | 0.20 | 0.08 | 0.06 | 0.05 | 0.02 |
| FreeArch network, all entries | 0.95 | 0.72 | 0.74 | 0.62 | 0.44 | 0.24 | 0.03 | 0.02 |
| FreeArch network, nonzero entries only | 0.95 | 0.71 | 0.72 | 0.62 | 0.54 | 0.24 | 0.05 | 0.02 |
| Prompt | Writer (families) | 0 | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Standard | GPT-6 Astra (6) | 0.63 | 0.35 | 0.24 | 0.10 | 0.05 | 0.05 | 0.03 | 0.01 |
| Robust | GPT-6 Astra (6) | 0.60 | 0.59 | 0.59 | 0.59 | 0.59 | 0.53 | 0.10 | 0.01 |
| Standard | GPT-6.1 Sol (5) | 0.56 | 0.26 | 0.14 | 0.09 | 0.03 | 0.03 | 0.01 | 0.01 |
| Robust | GPT-6.1 Sol (5) | 0.48 | 0.48 | 0.48 | 0.48 | 0.48 | 0.41 | 0.04 | 0.01 |
| Standard | Claude Opus 5.5 (6) | 0.41 | 0.19 | 0.15 | 0.12 | 0.03 | 0.03 | 0.00 | 0.01 |
| Robust | Claude Opus 5.5 (6) | 0.23 | 0.23 | 0.21 | 0.21 | 0.18 | 0.12 | 0.05 | 0.02 |
| Prompt | Writer (families) | 0 | 8k | 16k | 41k | 82k | 164k | 303k |
|---|---|---|---|---|---|---|---|---|
| Standard | GPT-6 Astra (6) | 0.61 | 0.02 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 |
| Robust | GPT-6 Astra (6) | 0.62 | 0.07 | 0.03 | 0.00 | 0.00 | 0.01 | 0.00 |
| Standard | GPT-6.1 Sol (5) | 0.59 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 |
| Robust | GPT-6.1 Sol (5) | 0.44 | 0.05 | 0.05 | 0.04 | 0.03 | 0.03 | 0.03 |
| Standard | Claude Opus 5.5 (4) | 0.48 | 0.03 | 0.03 | 0.03 | 0.03 | 0.03 | 0.02 |
| Robust | Claude Opus 5.5 (4) | 0.33 | 0.09 | 0.09 | 0.08 | 0.09 | 0.10 | 0.10 |
| Writer | Fixed W2 | FreeArch W2 | Bounded | P2 | Bounded FreeArch | P2 Bounded |
|---|---|---|---|---|---|---|
| GPT-6 Astra | 0.56 | 0.90 | 0.98 | 1.00 | +0.08 [+0.02,+0.15] | +0.02 [+0.00,+0.04] |
| GPT-6.1 Sol | 0.43 | 0.84 | 0.91 | 1.00 | +0.07 [-0.02,+0.16] | +0.09 [+0.03,+0.17] |
| Claude Opus 5.5 | 0.36 | 0.68 | 0.87 | 0.97 | +0.18 [+0.11,+0.26] | +0.10 [+0.06,+0.14] |
| GPT-6.1 Terra | 0.33 | 0.74 | 0.87 | 0.98 | +0.13 [+0.02,+0.23] | +0.11 [+0.04,+0.21] |
| Pooled (4 writers) | 0.42 | 0.79 | 0.91 | 0.99 | +0.11 [+0.06,+0.17] | +0.08 [+0.04,+0.12] |
| Writer | Fixed | A | B | FreeArch | A fixed | B A | FreeArch B |
|---|---|---|---|---|---|---|---|
| GPT-6 Astra | 0.56 | 0.58 | 0.88 | 0.90 | +0.02 [-0.03,+0.08] | +0.30 [+0.17,+0.43] | +0.02 [-0.02,+0.06] |
| GPT-6.1 Sol | 0.43 | 0.44 | 0.78 | 0.84 | +0.00 [-0.04,+0.04] | +0.35 [+0.24,+0.46] | +0.06 [-0.03,+0.15] |
| Claude Opus 5.5 | 0.36 | 0.35 | 0.56 | 0.68 | -0.01 [-0.08,+0.06] | +0.22 [+0.12,+0.32] | +0.12 [+0.05,+0.21] |
| GPT-6.1 Terra | 0.33 | 0.37 | 0.68 | 0.74 | +0.03 [-0.00,+0.07] | +0.31 [+0.22,+0.41] | +0.06 [-0.00,+0.13] |
| Pooled (4 writers) | 0.42 | 0.43 | 0.73 | 0.79 | +0.01 [-0.02,+0.04] | +0.29 [+0.21,+0.37] | +0.06 [+0.02,+0.10] |
| Weights in the writer’s FreeArch graph | S-start | Q-lvl | Written learned [95% CI] |
| Written (Astra, FreeArch W2 run 0) | 1.00 | 0.89 | |
| Learned: BC-support | 0.01 | 0.01 | +0.89 [+0.77,+0.98] (19/20) |
| Learned: distill (no aug.) | 0.01 | 0.01 | +0.89 [+0.77,+0.98] (19/20) |
| Fixed actor: BC-support | 0.96 | 0.12 | |
| Fixed actor: distill (no aug.) | 0.90 | 0.09 |
| Arm | Writer | Deliv. | Q-lvl | Cond. deliv. | Excl. undeliv. | Q-pal | runs [Wilson 95% CI] |
|---|---|---|---|---|---|---|---|
| W2 | GPT-6 Astra | 1.00 | 0.56 | 0.56 | 0.56 | 0.54 | 31/60 [0.39, 0.64] |
| GPT-6.1 Sol | 1.00 | 0.43 | 0.43 | 0.43 | 0.39 | 25/60 [0.30, 0.54] | |
| Claude Opus 5.5 | 0.87 | 0.36 | 0.41 | 0.39 (19) | 0.31 | 18/60 [0.20, 0.43] | |
| GPT-6.1 Terra | 1.00 | 0.33 | 0.33 | 0.33 | 0.30 | 21/60 [0.24, 0.48] | |
| P2 | GPT-6 Astra | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 60/60 [0.94, 1.00] |
| GPT-6.1 Sol | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 60/60 [0.94, 1.00] |
| Policy source | easy (3) | medium (11) | hard (5) | very hard (1) | macro |
|---|---|---|---|---|---|
| Written weights, W2 | 0.89 [0.79,0.99] | 0.40 [0.27,0.55] | 0.27 [0.08,0.48] | 0.00 | 0.42 |
| Written weights, W1 | 0.65 | 0.25 | 0.14 | 0.00 | 0.27 |
| Python player, P2 | 0.98 [0.96,1.00] | 0.99 [0.97,1.00] | 0.99 [0.97,1.00] | 0.96 | 0.99 |
| Python player, P1 | 0.97 | 0.93 | 0.94 | 0.84 | 0.93 |
| Reactive Python, P-react | 0.42 | 0.44 | 0.38 | 0.08 | 0.40 |
| Best learned baseline (non-privileged) | 0.32 | 0.16 | 0.05 | 0.00 | 0.15 |
| Main writers pooled | easy (2) | medium (9) | hard (5) | very hard (1) | macro |
| W2 | 0.89 | 0.31 (0.40) | 0.27 | 0.00 | 0.35 (0.42) |
| FreeArch W2 | 0.92 (0.95) | 0.75 (0.78) | 0.76 | 0.64 | 0.77 (0.79) |
| P2 | 0.98 | 0.99 | 0.99 | 0.96 | 0.98 (0.99) |
| Per writer | W2 | FreeArch W2 | P2 | FreeArch W2 W2 [95% CI] | |
| GPT-6 Astra | 0.50 | 0.89 | 1.00 | +0.39 [+0.25,+0.55] (15/17) | |
| GPT-6.1 Sol | 0.35 | 0.82 | 0.99 | +0.47 [+0.30,+0.63] (15/17) | |
| Writer | W1 | W1, M+S | no axial | 8 iter. | effort med. | P1 | P1, M+S |
|---|---|---|---|---|---|---|---|
| GPT-6 Astra | 0.43 | 0.53 | 0.50 | 0.16 | 0.39 | 0.94 | 0.73 |
| GPT-6.1 Sol | 0.39 | 0.41 | 0.33 | 0.28 | 0.29 | 0.96 | 0.73 |
| Claude Opus 5.5 | 0.15 | 0.21 | 0.13 | 0.23 | 0.18 | 0.89 | 0.84 |
| Writer (W2) | Q-lvl | zero frame | first frame | perturbed, no RESET | CF sensitivity |
|---|---|---|---|---|---|
| GPT-6 Astra | 0.56 | 0.00 | 0.07 | 0.65 | 0.75 |
| GPT-6.1 Sol | 0.43 | 0.00 | 0.05 | 0.48 | 0.76 |
| Claude Opus 5.5 | 0.36 | 0.02 | 0.05 | 0.45 | 0.75 |
| GPT-6.1 Terra | 0.33 | 0.00 | 0.06 | 0.41 | 0.71 |
| Replay floor / best open loop | 0.02 / 0.04 |
| Writer | W1 | W2 | P1 | P2 |
|---|---|---|---|---|
| GPT-6 Astra | 0.06 [0.01,0.14] | 0.08 [0.01,0.18] | 0.59 [0.47,0.72] | 0.75 [0.64,0.85] |
| GPT-6.1 Sol | 0.04 [0.00,0.10] | 0.05 [0.01,0.12] | 0.58 [0.46,0.70] | 0.68 [0.55,0.79] |
| Claude Opus 5.5 | 0.03 [0.00,0.10] | 0.06 [0.00,0.15] | 0.31 [0.17,0.45] | 0.43 [0.29,0.56] |
| GPT-6.1 Terra | 0.01 [0.00,0.03] | 0.02 [0.00,0.05] | 0.51 [0.39,0.64] | 0.57 [0.44,0.69] |
| GPT-6 Luna | 0.00 [0.00,0.02] | – | 0.06 [0.01,0.12] | – |
| Claude Sonnet 5.5 | 0.00 [0.00,0.01] | – | 0.18 [0.08,0.30] | – |
| Writer | W2 fixed | W2 FreeArch | P2 | FreeArch fixed [95% CI] |
|---|---|---|---|---|
| GPT-6 Astra | 0.08 | 0.42 | 0.75 | +0.34 [+0.24,+0.44] (25 games) |
| GPT-6.1 Sol | 0.05 | 0.36 | 0.68 | +0.31 [+0.20,+0.42] (25 games) |
| Claude Opus 5.5 | 0.06 | 0.13 | 0.43 | +0.07 [+0.01,+0.16] (25 games) |
| GPT-6.1 Terra | 0.02 | 0.26 | 0.57 | +0.24 [+0.14,+0.36] (25 games) |
| Measure (median [p90]) | Free architecture | Fixed actor | Python player |
|---|---|---|---|
| Parameters | 30,157 [99,383] | 57,289 | – |
| GMAC per step | 0.32 [1.23] | 1.33 | – |
| Node evaluations per step | 2,362 [9,671] | 412 | – |
| GPU, batch 1, eager (ms) | 23.8 [115.3] | 4.8 | – |
| GPU, batch 1, CUDA graph (ms) | 4.6 [22.0] | 4.8 † | – |
| GPU, batch 64, per action ( s) | 525 [2065] | 273 | – |