Automated science needs agents that can work out the rules of an unfamiliar environment by interacting with it. Interactive rule-discovery puzzles offer a controlled setting for studying this ability: an agent infers hidden rules through experimentation and uses what it has inferred to reach a stated goal. We ask what limits current language models on these puzzles and whether reinforcement learning (RL) improves performance on rules held out from training. To study both, we introduce WITNESS, a 2D grid-based puzzle environment with ground-truth ASCII observations and controlled access to rules. An agentic pipeline generates games for WitnessGym, the RL training suite, and WitnessBench, comprising public validation and private test games. The validation set separately tests new compositions of trained rule primitives and primitives absent from training. Under a shared harness, the best of 18 frontier proprietary and open-weight models solves only 24% of private test level slots, with scores sensitive to the observation interface and agent configuration. Providing ground-truth rules raises Opus-5's validation RHAE-L5 (relative human action efficiency over the first five levels) from 59.9 to 97.8, whereas a 27B open-weight model gains only 2.1 points and remains limited even with the rules provided. RL on WitnessGym raises the 27B model's private test RHAE-L5 from 2.1 to 5.4 and yields a mean gain of 4.1 points on four external discovery benchmarks. Together, these results point to rule acquisition as a major difficulty for frontier models like Opus-5 while smaller models further struggle on rule-based execution, and indicate that RL on hidden-rule puzzles transfers to broader rules and real-world tasks beyond training. Benchmark is available at: https://witnessbench.ai
Figures & tables
Figure 1: Overview of W itness- I nspired T estbed of N ovel E nvironments with S ymbolic S tructure (WITNESS).
Figure 2: One level of a WITNESS game as the engine renders it and as the model observes it: the ASCII board with its legend, the current path, and the feedback shown after a rejected submission.
Model
RHAE-L5 mean ± std
Beaten levels
Avg levels per game
OAE L5 / L20
RHAE-uncap L5 / L20
Deepest level
Beaten games
RHAE-L20 mean ± std
Fable-5
36.4 ± 3.7
414/1700(24%)
4.87/20
20.9 / 5.4
156.3 / 34.3
20/20
2/85
8.3 ± 2.7
Opus-5
30.6 ± 2.0
405/1700(24%)
4.76/20
21.2 / 6.6
210.9 / 38.6
20/20
2/85
9.4 ± 2.2
GPT-6-Astra-Pro
29.8 ± 2.0
399/1700(23%)
4.69/20
19.5 / 9.1
224.1 / 59.5
20/20
10/85
12.8 ± 1.1
GPT-6-Astra
27.6 ± 0.9
389/1700(23%)
4.58/20
17.6 / 7.7
182.7 / 50.0
20/20
10/85
11.3 ± 0.9
Kimi-K3
24.6 ± 4.0
249/1700(15%)
2.93/20
14.9 / 1.8
119.2 / 15.2
20/20
1/85
3.1 ± 1.6
GPT-5.6-Sol
23.0 ± 3.1
262/1700(15%)
3.08/20
13.8 / 2.1
92.7 / 10.8
14/20
0/85
3.4 ± 1.0
Table 1: Model Evals using our canonical harness (unseen games w/ novel rules, 20 lvls/game, 5 eval seeds). RHAE-L5 / RHAE-L20: RHAE computed over the first 5 levels (the benchmark number) and over all 20 levels. The first 5 levels are four rule-teaching levels with curriculum design plus a non-teaching testing level; the remaining levels are more complex levels of larger grid or harder. RHAE-uncap: the human-baseline score without the two caps. OAE (objective action efficiency): the same level-weighted efficiency score against the optimal action count. Deepest level: the most levels completed in any single rollout. Evaluated in max-effort where the endpoint exposed effort tiers.
Harness
Opus-5
Opus-4.8
Kimi-K3
Qwen3.8-27B
Our harness
30.58 ± 2.00
22.77 ± 4.27
24.62 ± 4.02
6.42 ± 1.19
Our harness, w/o perception
30.29 ± 4.92
10.95 ± 2.80
8.71 ± 2.91
1.03 ± 0.30
Minimal harness (ARC-AGI-3 alike)
11.33 ± 3.02
3.31 ± 1.11
0.00 ± 0.00
0.01 ± 0.03
[1pt/1.5pt] OSS harness: Prime-Agent
68.11 ± 6.33 ∗
59.88 ± 0.75 ∗
47.99 ± 3.96
26.60 ± 6.39
model-attended solving
55.29 ± 6.68
37.24 ± 4.66
42.22 ± 4.53
23.53 ± 3.87
verbal-only solving
4.37 ± 1.45
0.14 ± 0.05
9.20 ± 4.38
5.86 ± 3.90
Table 2: Harness ablation on the 17 unseen games (5 seeds): how much of each model’s RHAE-L5 score depends on its harness. Our harness : our full harness. w/o perception : same harness without ground-truth ASCII board ( 64×64 pixel input). Minimal harness : a minimalist loop, raw pixel input. Prime-Agent : A self-improving RLM Harness (95% on ARC-AGI-3 public games). Model-attended solving and Verbal-only solving : exclude cases – no model participation in level solving; winning moves computed by code. Details in Appendix A.5 . ∗ These models with Prime-Agent caused frequent refusals from Claude’s content filter; we report 2 seeds to show a spread and repeat no further.
Policy
Extra prompt
RHAE-L5
Δ RHAE-L5
Games cleared
Efficiency
Plan-less calls
Qwen3.8-27B
none
8.0 ± 5.0
–
5/40
0.39
1%
generic text
9.1 ± 2.5
+1.1
6/40
0.44
0%
GT rule card
10.1 ± 2.7
+2.1
6/40
0.48
1%
Opus 5 (frontier)
none
59.9 ± 11.4
–
30/40
0.74
3%
generic text
63.6 ± 3.7
+3.7
34/40
0.72
4%
GT rule card
97.8 ± 0.7
+37.8
40/40
0.99
0%
Table 3: Ground-truth rule cards as a prompt-time probe . A ground-truth rule card is a hand-written natural-language statement of a game’s true rules (win condition and mechanics, transcribed from the game engine), which we supply in the system prompt at evaluation time only, as an oracle probe that separates a policy’s failure to discover the rules from its failure to act on them. Eight held-out validation games, 5 evaluation seeds, 8192-token output cap, thinking off.
Figure 3: (a) RL training dynamics; and (b,c) generalization to test set (unseen rules) and external discovery tasks.
Benchmark
Hidden rules
Observation
Training–evaluation split
ARC-AGI-3 ( ARC Prize Foundation, 2026a )
✓
Pixel grid
Evaluation only
DiG-Bench ( Battleday et al., 2026 )
✓
1D string
Evaluation only
MazeBench ( Pappas & Pappas, 2026 )
✗
3D render (also ASCII/JSON)
Evaluation only
gg-bench ( Verma et al., 2025 )
✗
Text
Evaluation only (LLMs)
lmgame-Bench ( Hu et al., 2026 )
✗
Frames or text
By game
Enigmata ( Chen et al., 2026a )
✗
Text
By instance
Table 4: Comparison of related benchmarks. Hidden rules indicates interactive inference of rules not supplied to the agent. Split identifies the unit held out between training and evaluation in the reported experiments; evaluation only indicates that no training experiment is reported for evaluated models.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Cheatsheet for W itness- I nspired T estbed of N ovel E nvironments with S ymbolic S tructure (WITNESS).
Policy
Extra prompt
RHAE-L5
Δ RHAE-L5
RHAE-uncap
Games cleared
Efficiency
Plan-less calls
Base (no RL)
none
8.0 ± 5.0
–
9.9 ± 6.7
5/40
0.39
1%
generic text
9.1 ± 2.5
+1.1
11.4 ± 5.6
6/40
0.44
0%
GT rule card
10.1 ± 2.7
+2.1
14.8 ± 7.4
6/40
0.48
1%
Broadcast RL, step 10
none
15.9 ± 3.4
–
19.8 ± 2.1
7/40
0.50
2%
generic text
20.2 ± 4.8
+4.3
22.3 ± 6.0
8/40
0.63
1%
GT rule card
47.7 ± 5.0
+31.8
82.6 ± 14.8
31/40
0.69
57%
Appendix
Table 5: Ground-truth (GT) rule cards as extra prompts for test-time ablation. A ground-truth rule card is a hand-written natural-language statement of a game’s true rules (win condition and mechanics, transcribed from the game engine), which we supply in the system prompt at evaluation time only, as an oracle probe that separates a policy’s failure to discover the rules from its failure to act on them. Eight held-out validation games, 5 evaluation seeds, 8192-token output cap, thinking off. A rule card does not bring great improvements on the base model, while the same card improves the RL policies significantly. Frontier models such as Opus-5 are mainly bottle-necked by rule discovery. These hint-assisted scores are only for diagnostic purpose and excluded from all other results.
Figure 5: Why Opus-5 does not achieve perfect RHAE score (100) given GT rules: rare cases where human players complete a level more efficiently than Opus. In these levels, a human player selected a more efficient path, possibly unconsciously as it is a straight-forward path; the opus model selected non-optimal paths in 3/5 and 1/5 seeds, respectively. For most other cases, especially in earlier levels where a human player performed numerous steps to discover rules, Opus given GT rules executes much more efficiently, achieving RHAE-uncap scores that far exceed 100.
Checkpoint
Layout
ft10
ft14
ft15
ft16
( Δ slots 4–5)
( Δ slots 1–2)
( Δ slots 0–4, 7)
( Δ slots 4–7)
Base (no RL)
trained
1.19 ± 0.32 / 0.29 / 0.6
2.25 ± 0.37 / 0.46 / 3.0
2.81 ± 0.32 / 0.45 / 3.4
2.06 ± 0.46 / 0.59 / 4.5
unseen
0.75 ± 0.37 / 0.38 / 1.1
1.31 ± 0.38 / 0.41 / 1.9
2.12 ± 0.38 / 0.43 / 2.7
3.69 ± 0.38 / 0.40 / 4.5
RL, step 10
trained
6.44 ± 0.80 / 0.67 / 40.0
3.12 ± 0.40 / 0.50 / 5.8
2.50 ± 0.48 / 0.59 / 4.6
4.31 ± 0.30 / 0.48 / 8.2
unseen
5.94 ± 0.63 / 0.64 / 25.2
2.88 ± 0.48 / 0.41 / 4.9
2.62 ± 0.38 / 0.44 / 3.9
4.62 ± 0.34 / 0.62 / 13.7
RL, step 30
trained
3.75 ± 0.81 / 0.52 / 12.9
3.44 ± 0.27 / 0.51 / 7.0
1.88 ± 0.32 / 0.58 / 2.7
2.44 ± 0.46 / 0.58 / 5.3
Appendix
Table 6: Transfer of RL gains to unseen layouts of the same rules. For each training game that has an alternate level set we evaluate checkpoints on the trained set and on a reserved set that was never used in training (same rules, different boards; the two layouts differ only in levels). Cells: levels completed / action efficiency on completed levels / official game score. The gains on trained layouts carry over to the unseen layouts; the gap between trained and unseen does not grow with training.
Figure 6: Comparing models of different sizes (9B vs 27B) with identical training settings. (a) 9B model does not experience useful learning signals while 27B learns to beat more levels. These findings corroborate the belief that complex behaviors emerge during model scaling; (b) the validation performance on composite of known primitive rules and unseen rules starts differently but converge. It suggests what model learns during RL training is generalized capability rather than memorizing specific patterns; (c) training on games of primitive rules generalized to test games of unseen rules.
Figure 7: Behavior comparison between 9B and 27B models. (a) In contrast to 9B model, 27B model learns to complete more levels with less LLM calls within a game rollout as the training progresses; (b) 27B model learns to reflect more within each LLM call to plan high-quality actions, while 9B model consistently outputs verbose tokens that do not improve level-solving; (c) 27B model learns to preserve smaller fraction of rules it proposed earlier, while 9B model keeps generating and preserving most low-level rule hypotheses that do not contribute to puzzle solving.
Figure 8: Comparison of RL algorithms: various credit assignment methods. Credit-C is shown with two curves, one with each seed, to exhibit its divergence during training, due to the noise in deterministically keyed rules.
Table 7: Training dynamics of OOD generalization: credit assignment (broadcast vs. credit-C) × training step (10 vs. 20) on the four benchmarks. During RL training, both models have progressive OOD gains against the base model.
Model
Requested identifier
Out cap / reasoning
Evaluated (2026)
Fable-5
claude-fable-5
64k / adaptive-max
Jul 31 – Aug 5
Opus-5
anthropic/claude-opus-5
32k / 16k budget
Jul 29 – Aug 5
GPT-6-Astra-Pro
openai/gpt-6-astra-pro
64k / max
Sep 9 – Sep 17
GPT-6-Astra
openai/gpt-6-astra
64k / max
Sep 4 – Sep 12
Kimi-K3
moonshotai/kimi-k3
32k / max
Jul 30 – Aug 6
GPT-5.6-Sol
openai/gpt-5.6-sol
32k / max
Aug 12 – Aug 22
Appendix
Table 8: Exact model identifiers and sampling settings behind Table 1 . Every model was run with the same canonical harness and protocol: 17 unseen games × 5 evaluation seeds × up to 20 levels; 300 actions per level (an unsolved level ends the rollout) and 1,000 LLM calls per rollout; temperature =1.0 for all models (the client forces 1.0 whenever reasoning is enabled), no top_p , top_k or seed parameter, cached system prompt. Reasoning : “max” sends reasoning:{effort:"max"} ; “16k budget” sends reasoning:{max_tokens:16000} (endpoints without effort tiers); “adaptive-max” is the Anthropic API’s adaptive thinking with output_config.effort = max . Out cap is the per-call output-token limit.
We introduce Pencil Puzzle Bench, a framework for evaluating large language model reasoning through pencil puzzles, a family of constraint-satisfaction problems closely related to NP-complete problems, with deterministic, step-level verification. From a database of 62,231 puzzles across 94 varieties with verified unique solutions, we select a benchmark of 300 puzzles spanning 20 varieties and evaluate 51 models from 11 providers in two modes: direct ask (single-shot) and agentic (multi-turn with iterative verification). A key differentiator of our benchmark is that every intermediate board state can be checked against variety-specific constraints, localizing errors to the exact rule violated, providing the infrastructure for dense, per-move reward signals for process supervision and reinforcement learning. Our evaluation reveals two distinct axes of capability: (1) reasoning effort scaling, where GPT-5.2 improves 81x from no reasoning to maximum effort; and (2) agentic iteration, where Claude Opus 4.6 rises from 0.3% to 30.0% through iterative checking, while GPT-5.2@xhigh improves from 20.2% to 56.0%. Agentic attempts span a median of 29 turns over 17 minutes, with the longest exceeding 1,221 turns and 14.3 hours - a demanding test of long-context utilization, not just reasoning.
We introduce PuzzleJAX, a GPU-accelerated puzzle game engine and description language designed to support rapid benchmarking of tree search, reinforcement learning, and LLM reasoning abilities. Unlike existing GPU-accelerated learning environments that provide hard-coded implementations of fixed sets of games, PuzzleJAX allows dynamic compilation of any game expressible in its domain-specific language (DSL). This DSL follows PuzzleScript, which is a popular and accessible online game engine for designing puzzle games. In this paper, we validate in PuzzleJAX several hundred of the thousands of games designed in PuzzleScript by both professional designers and casual creators since its release in 2013, thereby demonstrating PuzzleJAX's coverage of an expansive, expressive, and human-relevant space of tasks. By analyzing the performance of search, learning, and language models on these games, we show that PuzzleJAX can naturally express tasks that are both simple and intuitive to understand, yet often deeply challenging to master, requiring a combination of control, planning, and high-level insight.
Sam Earle, Graham Todd, Yuchen Li +5
New York University · University of Malta · University of the Witwatersrand +1
Discovery---formulating novel generalizations---is a central part of the scientific process. Despite its importance, there is a gap in the current AI benchmark landscape, with few benchmarks directly probing the capacity for discovering new knowledge with experimentation in controlled environments where the objective is unknown. To address this gap, we release a new benchmark: DiG-bench (Discovery in Games). DiG-bench consists of a set of 70 independent games. Each game is encoded as a short string and has unique transformation rules that must be discovered through interaction and experimentation. The levels of the game present a series of challenges to test whether the rules have been discovered, where the win conditions for each level are also unknown. We provide games at seven tiers of difficulty for AI agents. The lowest tier is routinely solvable by multiple models, while the highest tier challenges the best models in agentic harnesses. All 70 games were solved by at least one human on first attempt. A subset of 21 games is released publicly, and the remainder is held private for secure evaluation.
Ruairidh M. Battleday, Kai Sandbrink, Jimi Cullen-Drohan +13
1Thinking About Thinking · 2Independent · University of Oxford +5