cs.AINov 13, 2026

TReC: Evaluating Move-Induced State Changes in Instruction-Conditioned Go

Authors: Zhuangsheng Li, Yunhan Zhang, Dailin Li, Qingxian Yu, Yu Wu, Youqi Song, Bangyuan He

Organizations: University of California, San Diego · University of Melbourne · University of Virginia · Shanghai Jiao Tong University · 

Abstract

When a Go model predicts a move different from the expert's, coordinate distance measures how far the prediction lands from the reference. We introduce Transition Representation Consistency (TReC) to compare how the two moves change a board-state embedding. Both moves are executed from the same position under the game rules, and TReC is the cosine similarity between the two resulting representation changes. In experiments with a frozen KataGo network on 680 mid-to-late-game positions from five games, the gap between the highest and lowest candidate per position has a median of 0.01 to 0.08 for the direct successor-state cosine and 0.88 to 1.33 for TReC, depending on the evaluator layer. TReC follows coordinate distance only weakly: the rank correlation between a candidate's distance to the reference and its TReC, computed within each position, has a median of -0.19 over these positions.

Figures & tables

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

Oct 16, 2026

Recognizing Atari and Assessing Sente: Controlled Evaluation Tasks for Language-Based Go Agents

Domain terminology provides concise descriptions of game states and strategic relationships, but whether game-playing language models can accurately recognize these concepts and relate them to move evaluation remains unclear. We construct a controlled evaluation set centered on atari and its relationship to sente in Go, examining terminology recognition under local board changes and move evaluation under different strategic conditions. We conduct our tests on LoGos, a Go language model, after fine-tuning it with terminology and commentary supervision. The model struggles to recognize changes in the atari relationship and assess its strategic implications. Its terminology predictions are associated with move preferences even when those preferences contradict the reference evaluation. Our controlled tests provide a framework for examining whether language-based Go agents connect domain terminology with precise board relationships and strategic move values.
May 28, 2026

Chess-World-Model: A 10M-Game Benchmark for Exact State Tracking from Chess Move Sequences

World models require state tracking, which is the ability to maintain a correct latent state across action sequences. Existing benchmarks are often synthetic or language-based, limiting their value as tests of structured state updates in realistic domains. We introduce Chess-World-Model, a large-scale state-tracking benchmark built from 10 million real chess games, where models predict the exact board state reached after a sequence of legal moves. Alongside a held-out real-game split, we include an out-of-distribution test set of uniformly random legal play both at its original trajectory lengths and after matching the human-game length distribution. This tests whether models learn the transition rules rather than shortcuts from common human positions. Prior theoretical and empirical work has shown that Transformers struggle to state-track, while input-dependent linear RNNs require expressive state-transition matrices to do so. We therefore benchmark a causal Transformer, block-diagonal SLiCE, Mamba-3, and Gated DeltaNet with negative eigenvalues under a matched interface and training protocol. The recurrent models strongly outperform the Transformer at approximately 3 and 8 million parameters. Real-game performance largely saturates by the 18 million parameter scale, but length-matched random evaluation continues to distinguish architectures through the largest tested scale of approximately 43 million parameters. Additionally, ablations show that less expressive state-transition mechanisms generally reduce performance on the out-of-distribution splits. Together, these results establish Chess-World-Model as a practical large-scale benchmark for state tracking that exposes failures model scale would otherwise conceal.
Oct 29, 2026

CoToan: Executable Rules and a Representation Contract for Reproducible Game AI Evaluation

Game AI results are comparable only when different implementations expose the same decision problem. Yet seemingly minor representation choices can silently change that problem by altering action probabilities or collapsing states with different outcomes. We study this issue through CoToan, an executable specification and representation contract for Vietnamese Mathematical Chess connecting the Game Description Language (GDL) with OpenSpiel. The game provides a useful case because a single capture may have multiple arithmetic justifications, while outcomes can depend on scoring rules and a finite playing horizon. CoToan therefore represents actions by their state transitions rather than by arithmetic witnesses, and treats time and victory configuration as part of task identity. Cross-representation tests show agreement in legal actions, successor states, and terminal outcomes across hundreds of sampled and constructed positions, complemented by exhaustive checks of all 32 operator masks and 512 opposing-material subsets. Controlled diagnostics further show that uniform witness-level sampling alters endpoint-action probabilities, while omitting remaining time or victory settings can merge states with different terminal behavior. These results show that action semantics, time, and victory configuration are not merely implementation details. They are part of the experimental task itself and must be represented explicitly for reproducible game AI.