Modern chess engines are silent experts: they play at a superhuman level, but do not offer explanations for their play. On the other hand, language models (LMs) can generate plausible-sounding explanations, but their weak playing strength limits the utility of their explanations. We introduce Queen, a 4B-parameter chess-language model that can explain its moves and plans while playing at the level of a typical Grandmaster. Our novel framework enables domain-specific reasoning through complementary components: an encoder-decoder architecture and an iterative distillation algorithm. This architecture integrates a silent expert chess encoder with an instruction-tuned LM through cross-attention, which we train via a question-answering curriculum to extract chess concepts from the encoder's representations. Building on this domain-adapted model, we iteratively improve its explanations with a natural-language analog of the Bellman update: the model analyzes the positions after its top candidate moves and consolidates them into an explanation of the current position, which is then distilled back into the model. Over seven iterations, our model gains over 900 Elo points (1782 to 2697), substantially surpassing all frontier models on both playing strength and puzzle accuracy, despite containing three orders of magnitude fewer parameters. Furthermore, LM-based evaluations show that our explanations are fluent and approach GPT-5.6-Sol (high) in coherence. The generality of our architecture and training procedure suggests a recipe for applying language models to domains where silent expert encoders are available, like games, robotics, and computer use.
Figures & tables
Figure 1: Chess engines play strong moves but can’t explain them. Frontier/finetuned LMs give plausible explanations, but are limited by weak play. Queen offers the best of both worlds.
Figure 2: We use an encoder-decoder architecture inspired by Flamingo ( Alayrac et al., 2022 ) . The encoder is an Lc0 chess network, and the decoder is an instruction-tuned language model. The k -th layer embeddings of Lc0 flow into a Flamingo block injected before the 2k -th LM layer, and all previously trained parameters are frozen during domain adaptation.
Figure 3: An example explanation generated by Queen , snipped for readability. The explanation consists of natural language prose (green) followed by a best move and PV prediction (blue).
Figure 4: Domain adaptation progressively teaches the model to extract static features in the current board state ( static-current ), reason about available moves and interactions ( dynamic-current ), reconstruct a future board state from move sequences ( static-future ), and finally reason about the resulting position ( dynamic-future ).
Figure 5: We repeatedly distill the search-consolidated explanations back into the model for iterative self-improvement. Analogous to the Bellman update, a consolidated explanation combines the explanations at child nodes to explain why one move is the best and the shortcomings of other moves.
Model
Elo
Queen (ours)
2697
Gemini
2201
Sol
2071
Luna
1822
C1-4B
514*
Median Grandmaster
2730
Table 1: Estimated ratings anchored to Lichess Elos after 32 games. All frontier models use high reasoning effort. *C1-4B lost all 32 games.
Model
Tactical ↑
General ↑
NMR (%)
FNMR (%)
NMR (%)
FNMR (%)
Queen
68.8
91.6
66.6
97.1
Gemini
66.9
82.5
63.0
89.0
Sol
65.1
81.9
61.5
86.8
Luna
35.6
53.5
47.1
73.6
C1-4B
—
43.2
—
38.8
Table 2: Substantiation evaluation on 1,000 tactical and 1,000 general positions. NMR is the fraction of predicted PVs containing no mistakes; FNMR is the fraction of positions in which the model’s predicted best move is not a mistake. All frontier models use high reasoning effort. Queen performs the best across all metrics on both tactical and general positions.
Model
Structural Coherence
Conceptual Coherence
Fluency
Queen
3.51
2.76
4.45
Gemini
3.55
3.30
4.98
Sol
3.51
3.61
4.99
Luna
2.82
2.94
4.96
C1-4B
2.48
2.16
4.94
Table 3: Evaluation of the various models on coherence and fluency. Frontier models use high reasoning effort. All models are fluent in their output. Gemini obtains the best structural coherence, though Queen and Sol obtain comparable scores. Queen struggles with conceptual coherence, as iterative search-distillation cannot teach it to verbalize and describe motifs it hasn’t seen.
Model
Elo ↑
Tactics ↑
Pawn -1
1782
73.9
− Encoder
514 ∗
9.7
− Curriculum
514 ∗
1.3
Table 4: Ablations of Queen evaluated after Sol -seeding. Both the encoder and curriculum are essential for a good initialization.
Outcome
Rate (%)
Agrees with Lc0
46.2
Disagrees, in top-5
39.1
Outside top-5
14.7
Table 5: Queen disagrees with Leela on 53.8% of positions with five or more reasonable moves, suggesting that it does not simply reproduce Leela’s policy head.
Model
Elo ↑
P1
P2
P3
P4
Queen
1782
2024
2187
2346
Queen (HCE)
2115
2432
2476
2497
Table 6: Performance of Queen with and without frontier-model distillation. The HCE variant uses an alternative data construction pipeline based on Stockfish’s hand-crafted evaluation.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Question task
Sample Q → A / answer clause
Stages 1 & 3 — Static Board Understanding
Identify the piece on a queried square
What piece is on <SQUARE_28>? → There is <PIECE_MN> on <SQUARE_28>.
Locate all squares holding a queried piece
Where is <PIECE_MR>? → <PIECE_MR> occupies <SQUARE_1> and <SQUARE_8>.
List pieces on a queried file
Pieces on file <SQUARE_5>--<SQUARE_61>? → 1 <PIECE_MP> and 1 <PIECE_OK>.
List pieces on a queried rank
Pieces on rank <SQUARE_1>--<SQUARE_8>? → Walking the rank: …
List pieces on a queried diagonal
Pieces on diagonal <SQUARE_1>--<SQUARE_64>? → …(or ‘‘open diagonal’’)
Appendix
Table 7: Question types used for our four-stage domain adaptation.
Data source
curr-static
curr-dynamic
future-static
future-dynamic
Stage-1 data
100%
–
–
–
Stage-2 data
10%
90%
–
–
Stage-3 data
6%
8%
86%
–
Stage-4 data
6%
6%
8%
80%
Appendix
Table 8: Data composition of the four stages of domain adaptation training. Rows and columns correspond to the final stage data mixture and data sources described above. Earlier-stage data is replayed at a small proportion to prevent forgetting.
Hyperparameter
Value
Training steps
stage 1
50K
stage 2
60K
stage 3
30K
stage 4
100K
Bridge learning rate
Appendix
Table 9: Training hyperparameters for the four-stage curriculum. Hyperparameters shared across all stages are listed once.
Static
Dynamic
Current
Future
Current
Future
0.9997
0.9997
0.9897
0.9607
Appendix
Table 10: Held-out test set accuracies of our model.
Static
Dynamic
Architecture
Model family
Current
Future
Current
Future
Flamingo
SmolLM3
99.97
99.97
98.97
96.07
Qwen3
99.80
99.87
98.42
96.83
Gemma3
99.04
98.51
95.47
79.78
LLaVA
SmolLM3
99.80
99.80
99.00
93.20
Qwen3
99.93
99.89
97.20
57.20 ∗
Appendix
Table 11: Architecture × Model-family ablation: Accuracy (%) on Static and Dynamic on held-out validation sets. ∗ Undertrained: the training run was aborted early, at step counts matched to Flamingo.
Hyperparameter
Value
Training examples
8,402
Validation examples
200
Training steps
2,100
Effective batch size
16
Maximum sequence length
2,048
Bridge learning rate
1×10−5
Appendix
Table 12: Supervised fine-tuning hyperparameters for distillation from GPT-5.6-Sol (low).
Hyperparameter
Value
Training examples
150K–285K
Maximum training steps
30K
Effective batch size
32
Maximum sequence length
2,048
Bridge learning rate
1×10−4
Decoder learning rate
1×10−5
Appendix
Table 13: Supervised fine-tuning hyperparameters for one self-distillation iteration. Each iteration initializes from the selected checkpoint of the preceding iteration.
Opponent
Search budget (nodes)
Anchored Elo
Stockfish
10
1978
Maia-2200 (Lc0)
1
2037
Stockfish
100
2085
TinyGyal-8 (Lc0)
10,000
2422
Stockfish
1,000
2597
11258-16x2-se-4 (Lc0)
10,000
2689
Appendix
Table 14: The opponents we use for accuracy evaluations in Section 4.1 .
Modern chess engines play far beyond human ability, often choosing moves whose underlying ideas are difficult for humans to understand. We propose an automatic metric for evaluating chess explanations, grounded in chess-engine evaluations. Evaluating state-of-the-art language models using this metric, we find that they perform surprisingly poorly, often relying on surface-level board observations rather than robust, transferable chess concepts. We then introduce BinProp, a training-free method that enables these same models to discover reusable chess concepts and use them to explain moves in natural language. BinProp improves explainability for every model tested, nearly tripling the average score compared with direct prompting. Beyond chess, the same method discovers useful concepts in Go, draughts, and shogi without additional tuning.
Large language models can produce fluent explanations for chess moves, but plausible language does not necessarily reflect the reasoning behind a decision. We study this question in chess, where the board state is fully observable, legal actions can be enumerated, and move quality can be evaluated independently. Across 200 Lichess endgame puzzles, we test explanations using move recoverability, decoder-side controls, and token-level scoring of legal candidate moves. Unmasked explanations make generated moves easy to recover, but this advantage drops sharply after explicit move hints are removed. Under strict masking, explanations provide only small and decoder-dependent gains over the board state alone. Token-level scoring shows that explanations can nevertheless alter move preferences: random but plausible explanations from other puzzles reduce the probability of the correct move, indicating that irrelevant reasoning text is not simply ignored. We also find that recognizable endgame motifs can make generated moves easier to recover without reliably improving move correctness. Together, these results show that linguistic plausibility, consistency with a generated action, and solution correctness are distinct properties. Fluent chess explanations can influence action preferences and support a coherent move narrative while providing only limited evidence of faithful reasoning.
Angelina Parfenova
Lucerne University of Applied Sciences and Arts · Technical University of Munich
Language models often lack grounded reasoning capabilities in specialized domains where training data is scarce but bespoke systems excel. We introduce a general framework for distilling expert system reasoning into natural language chain-of-thought explanations, enabling compact models to acquire domain expertise and the ability to generate faithful, grounded explanations. Rather than distilling only final outputs, we capture the full reasoning process, transforming opaque expert computations into transparent, step-by-step explanations. We demonstrate this approach in chess, a canonical reasoning domain where language models continue to underperform. Our 4B parameter model, C1, advances from a near-zero baseline to 48.1% accuracy, outperforming all open-source models and most frontier proprietary systems. Notably, C1 surpasses its distillation teacher and generates solutions in two orders of magnitude fewer tokens than baselines. Unlike prior neural chess approaches that predict only best moves, C1 generates explainable solutions revealing strategic reasoning. Our pipeline combines supervised fine-tuning and reinforcement learning with theme-balanced data sampling for comprehensive tactical coverage. Master Distillation demonstrates how to inject expert-level knowledge into compact models for under-optimized domains, offering a recipe for unlocking RLVR where LLMs lack sufficient base capabilities.
Zhenwei Tang, Qianfeng Wen, Seth Grief-Albert +4
Department of Computer Science, University of Toronto · Queen’s University · Coolwei AI Lab