Searchless chess networks reach human master strength from a single forward pass by imitating a stronger teacher: the strongest, Leela Chess Zero's (Lc0) released Chessformer, distills the visit counts of an AlphaZero-style Monte Carlo Tree Search (MCTS). Imitating a search is a poor proxy for playing without one, so we fine-tune for single-pass strength with self-play reinforcement learning (RL). Its exploration is usually supplied by an entropy bonus, the reverse Kullback-Leibler (KL) divergence to uniform. We replace it with a forward, mass-covering KL toward the network's own MCTS prior (prior-directed exploration), so exploration covers the moves the prior judges promising, and pair it with an entropy-adaptive sampling temperature, set by the value head's outcome uncertainty, that sharpens once a position is decided. In about two thousand steps it raises puzzle accuracy from 93.9% to 94.9% on a 100,000-puzzle suite and mate-in-four accuracy from 77% to 81% while holding searchless strength at or slightly above the base. Measuring tactical accuracy and playing strength together across a matched-compute sweep, we find the two dissociate: accuracy gains fall in a one-point band while ratings straddle the base, and a control fine-tuned on puzzles alone posts the study's largest tactical gains while shedding roughly 260 Elo; a better puzzle-solver is not thereby a stronger player. Distribution-level measurements show what anchoring buys: without a regularizer self-play collapses onto a single line of play, and the puzzles newly solved are the near misses whose winning move the prior kept alive. The forward-KL prior tops the rating ladder, statistically tied with a reverse-KL anchor that concentrates twice as hard and drops the hardest solutions the mass-covering prior keeps in support.
Figures & tables
Figure 1: What self-play fine-tuning does to the base’s mate-finding, depth by depth: geometric-mean probability p⋆ of sampling the full solution line from the model’s output distribution, on the exact mate-in- N suites, relative to the frozen base (dashed, 1× ; ratios of the mate columns of Table 4 ; Section 3.3 ). Unregularized self-play sharpens shallow mates yet loses deep ones ( 0.07× at depth five); the usual entropy bonus erodes every depth; anchoring exploration to the network’s own prior largely preserves them, and the mass-covering forward-KL prior (ours) is the only arm above the base at every depth.
Figure 2: The two asymmetries between the KL directions; probabilities are softmax outputs and hence strictly positive, so the limits describe behavior as a probability becomes arbitrarily small. Rows: the policy either drops a prior move ( πθ(a)→0 where πbase(a) is large) or explores beyond the prior (mass where πbase(a)→0 ). Columns: the forward KL of our penalty and the reverse KL of the standard anchor. The forbidden cells lie on the diagonal: forward KL is zero-avoiding, blowing up when the policy drops a prior move yet free where the prior is small; reverse KL is zero-forcing, the mirror image, permitting collapse onto a single prior mode.
Figure 3: Entropy-adaptive temperature, measured with the frozen base network: the value head’s WDL sets τ=\Ent(p) , and self-play samples from μ∝πθ1/τ . Left (an eight-piece rook-and-knight endgame, White to move, in which Rh6# is the only mate among the 24 legal moves): near-one-hot WDL, τ=0.008 , yet the base still keeps 26.6% on Rg5+ , a check that does not mate, so sampling πθ would miss the finish better than one time in four; tempering discards it and commits to the mate ( 68.7%→100% ). Right (a balanced opening): draw-heavy WDL, τ=0.89 , and μ barely sharpens the policy ( μ≈πθ ). Light bars are the base policy πθ and dark bars the tempered μ (left: the mate, the strongest non-mating move, and the 22 others pooled; right: the five most probable moves plus the rest; per-panel scale, probabilities printed in %).
Figure 4: Prior-directed self-play fine-tuning as a TD(0) one-step backup. Three networks: the frozen prior πbase (the exploration reference), the online πθ , and an EMA copy θˉ supplying slow targets. At state s the policy samples a from μ∝πθ1/τ(s) with τ(s)=\Ent(pθˉ(s)) and steps to s′ . Each update consumes this single transition—no rollouts, no returns: the online net evaluates s , the EMA bootstraps from s′ , and one gradient step minimizes the loss of Equation ( 7 ), after which the EMA tracks θ . The dashed wire is the off-policy gap the truncated weight ρ corrects.
vs. base
Exploration term
Self-play τ
Acc (%)
Acc≥2400 (%)
Elo
W–D–L
LOS
Frozen base ( Monroe et al., 2026 )
–
93.85±0.15
60.01±0.68
2374±37
–
–
No regularizer
adaptive
94.06±0.15
60.50±0.68
Δ+9.8±8.2
326–351–323
0.55
τ≡1
94.04±0.15
60.80±0.68
Δ−0.5±8.3
380–295–325
0.98
Entropy bonus
adaptive
94.90±0.14
62.30±0.67
Δ−5.8±8.3
331–323–346
0.28
τ≡1
94.00±0.15
60.47±0.68
Δ+6.6±8.2
344–335–321
0.81
Table 1: Puzzle accuracy and searchless playing strength after fine-tuning from the same Chessformer base, evaluated single-pass ( argmax moves, EMA weights) at each arm’s selected β⋆ (Table 6 ); higher is better. Acc. : the 100,000 -puzzle suite; Acc≥2400 : the 20,000 -puzzle ≥2400 suite; both multi-solution scored, with 95% Wilson intervals. Accuracies are absolute; of the Elo column only the base row is, anchored to the Chessformer paper’s 2374±37 ( Monroe et al., 2026 ) , with every other row the difference Δ from it—so comparisons rest on the per-row intervals, which the shared anchor leaves unchanged. The last row is the supervised puzzle-tuned control of Section 3.6 .
Optimizer
AdamW, β1=0.9 , β2=0.98
Weight decay
0
Schedule
200-step linear warmup, then constant
Learning rate (backbone / final block)
3×10−6 / 1×10−5
Learning rate (policy head / value head)
1×10−5 / 3×10−5
Total steps
2,000
Parallel games / batch size
1,024
Table 2: Training hyperparameters shared across all runs of Section 3 . The forward-KL weight β is the only quantity that varies across conditions (Section 3.3 ).
Figure 5: Forward-KL coefficient sweep of the headline adaptive-temperature arm, evaluated on the 100,000 -puzzle suite: puzzle error of the EMA policy under temperature sampling versus the forward-KL weight β (lower is better; logβ axis; error bars and shaded bands are 95% Wilson intervals). Dashed lines mark the frozen base and the unregularized β=0 run; red marks β=10−2 , the coefficient selected on the 10,000 -puzzle suite. The four half-decade points are additional resolution for this arm only.
Mate-in- N accuracy (%)
Exploration term
Self-play τ
1
2
3
4
5
Frozen base ( Monroe et al., 2026 )
–
99.94±0.04
98.82±0.15
92.58±0.36
77.16±0.58
58.01±1.45
No regularizer
adaptive
99.92±0.04
98.77±0.15
92.97±0.35
79.23±0.56
61.59±1.43
τ≡1
99.80±0.06
98.90±0.14
93.68±0.34
80.73±0.55
63.81±1.41
Entropy bonus
adaptive
99.95±0.03
99.12±0.13
94.13±0.33
81.62±0.54
64.28±1.40
τ≡1
99.91±0.04
98.76±0.15
93.08±0.35
79.15±0.56
61.55±1.43
Table 3: Searchless accuracy by forced-mate depth: 20,000 Lichess positions per depth for N≤4 , from the suite of Lin et al. (2026) , and for N=5 all 4,465 Lichess puzzles whose forced mate is exactly five moves (the Lichess mate-in-5 theme also contains deeper mates), with the network’s true move history; a puzzle counts as solved only if the argmax at every position is a mate-preserving move (any of several winning moves is credited). Models are the selected self-play configurations of Table 1 . All values are absolute accuracies with 95% Wilson intervals; higher is better.
Figure 6: The difficulty gradient of the two prior-anchored models: solution-NLL reduction over the frozen base (base minus model, in nats—the log of the solution-probability ratio; above zero the model retains solutions better than the base). Left: by mate depth (the mate columns of Table 4 ). Right: by 200 -Elo puzzle rating bin on the 100,000 -puzzle suite (per-puzzle sequence NLL on the solution line); ratings below 600 and above 2600 are pooled into single bins ( 2,496 – 14,167 puzzles per bin). The entropy model is not plotted, its losses lying far below both axes (Table 4 ).
Figure 7: Trajectory coverage ΩKN=4 at the initial position: cumulative trajectory mass covered by K depth- 4 continuations drawn without replacement from each model’s play distribution (mean over 100 sampling seeds; all exploration terms at coefficient 0.01 , the configurations of Table 4 ). Solid: the no-temperature configurations (the frozen base sampled raw); dashed: their adaptive-temperature counterparts (the base sampled through the tempered μ ). Each curve ends where its extraction reaches the 95% target (dotted); across the 100 seeds the standard deviation of the draws needed is at most 2.1% of the mean.
Tactics suite
Mate-in- N
Model
All
≥2400
1
2
3
4
5
Frozen base
0.660
0.199 ( 0.302 )
0.915
0.819
0.615
0.344
0.161
No regularizer
0.536
0.010 ( 0.600 )
0.993
0.896
0.492
0.107
0.011
Entropy bonus
0.222
0.103 ( 0.381 )
0.399
0.187
0.102
0.037
0.011
Reverse-KL anchor
0.765
0.178 ( 0.569 )
0.978
0.930
0.721
0.370
0.126
Forward-KL prior (ours)
0.759
0.236 ( 0.526 )
0.954
0.894
0.712
0.411
0.169
Table 4: Solution probability p⋆ (geometric mean over puzzles; higher is better), defined in Section 3.1 ; mate suites use the network’s true move history. Models are the adaptive-temperature β=10−2 configurations of Table 1 (EMA weights); tactics columns as in Table 1 , mate columns as in Table 3 . Best value per column in bold; fine-tuned values are green above the frozen base, red below it (arithmetic means uncolored). For the ≥2400 suite the arithmetic mean is given in parentheses: the gap between the two means is the collapse signature read in Section 3.3 . Figure 6 (left) plots the mate columns.
Figure 8: Which failed positions fine-tuning recovers, for the two prior-anchored models of Table 4 on the 100,000 -puzzle suite (scoring and totals in the text). Left: share of base-wrong positions recovered, binned by the probability the base retains on the solution at the position. Right: binned by the rank of the solution in the base’s output there; bin sizes annotated.
Configuration
Samp.
Elo
\Ent(p)
DKL(πbasep)
DKL(pπbase)
cov5
Frozen base
raw
0
1.69
0.00
0.00
0.81
μ
361±29
0.39
8.38
0.81
0.98
No regularizer
μ
671±31
0.06
11.04
1.03
0.99
raw
660±31
0.22
7.59
0.89
0.98
Entropy bonus
μ
476±30
0.74
2.38
0.59
0.92
raw
357±29
1.44
1.26
0.26
0.75
Table 5: Sampled-play strength and single-state diagnostics for the ten configurations of Figure 10 in Appendix H (the frozen base raw and tempered, and each exploration term × temperature arm; entropy is the β=10−2 model). Sampling “ μ ” is the adaptive-temperature arm played through its tempered behavior policy, “raw” the τ≡1 arm at temperature one. Elo : sampled-play BayesElo from a closed round-robin (base anchored 0 ; 200 games/pair, 45 pairs; 95% intervals). The remaining columns average, over the positions each model reaches, its played policy p : entropy \Ent(p) (nats), forward and reverse KL to the raw base, and prior coverage cov5 (mass on the base’s top-five moves).
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Coefficient β
Exploration term
Self-play τ
10−4
10−3
10−2
10−1
1
Entropy bonus
adaptive
94.42±0.45
94.19±0.46
94.70±0.44
94.07±0.46
82.77±0.74
τ≡1
93.92±0.47
91.24±0.55
91.59±0.54
90.53±0.57
83.13±0.73
Reverse-KL anchor
adaptive
94.30±0.45
94.22±0.46
94.57±0.44
94.47±0.45
93.93±0.47
τ≡1
94.11±0.46
94.38±0.45
94.63±0.44
94.30±0.45
93.86±0.47
Forward-KL prior (ours)
adaptive
94.37±0.45
94.54±0.45
94.67±0.44
94.52±0.45
93.80±0.47
Appendix
Table 6: Coefficient selection on the 10,000 -puzzle suite (multi-solution scoring): overall accuracy (%) by exploration term, self-play temperature, and coefficient β , with 95% Wilson intervals. Bold marks each row’s maximum, the model carried into Table 1 . For reference, the frozen base scores 93.52±0.48 and the unregularized runs 93.99±0.47 (adaptive τ ) and 94.04±0.46 ( τ≡1 ).
Figure 9: Six exclusive recoveries from the 100,000 -puzzle suite— top row: puzzles the forward prior newly solves and the reverse anchor does not; bottom row: the mirror. Each panel shows the first solver move (side to move at the bottom): green arrow, the winning move; red arrow, the move kept by the base and by the fine-tuned model that fails; bars, the three models’ probabilities of the labeled moves; under each move, the mover’s expected score after it—gray from the base’s own value head 1−vbase(s′) , black from a 1.5 -second Stockfish oracle; beneath each panel, the solution. Left to right: a queen sacrifice into a three-move mate the base’s value head already scores as certain (rating 1326 ); an endgame where the base’s two favorite captures both lose and only the quiet 34.f5 wins ( 2613 ); a knight sacrifice no value head sees ( 2178 ); a rook sacrifice the recovering model commits to at 0.88 ( 2036 ); knight discoveries of which only the double check wins ( 2245 ); both rooks for the queen, the solution ranked fifth ( 1096 ).
Figure 10: Trajectory coverage ΩKN=4 beyond the initial position, under the protocol of Figure 7 : cumulative depth- 4 trajectory mass of the first K draws without replacement, mean over 100 sampling seeds; every curve ends at the 95% extraction target (dotted). Solid lines are the no-temperature configurations (the frozen base sampled raw), dashed their adaptive-temperature counterparts (the base sampled through the tempered μ ); the dot marks the unregularized configurations, whose first draw covers 95% . Each position is shown above its panel, full move history supplied (all White to move): left , the Ruy Lopez mainline (ply 8 , 27 legal moves, ≈7.0×105 depth- 4 continuations); middle , the Najdorf Sicilian English Attack (ply 16 , 47 legal moves, ≈2.6×106 ); right , the Queen’s Gambit Declined Tartakower (ply 14 , 38 legal moves, ≈1.5×106 ).
How strong can an AlphaZero-style chess system become under limited training compute when its entire learning loop is engineered for efficiency? We train from random initialization through searched self-play on a single eight-GPU node for 2.5 days. The resulting 6.32-million-parameter model reaches 3,251 benchmark Elo [3,206, 3,297] at 100,000 searches per move (estimated at under five seconds of thinking time) against a fixed-node Stockfish 13 ladder. The run ingests 3.25 million completed games, involves an estimated 100 billion search simulations, and makes 836.6 million training presentations. We investigate search allocation, replay and restart-state selection, policy representation, progressive model sizing, quantized inference, and throughput engineering. Alongside the retained design, we document plausible alternatives that failed to improve the complete learning loop or did not justify their cost. The reported strength is a result of the integrated system, not an isolated Elo gain attributable to any single choice.
Learning and skill mastery require extensive and deliberate practice. In many learning settings, producing high-quality pedagogical materials can require a high level of domain expertise and be very time-consuming. Pedagogical materials often need to train students to engage in different thinking patterns. In some domains, such as chess, puzzles are used to help students practice their skills in calculating the next moves and recognizing known patterns on a board. Giving students a practice set of puzzles to help them learn different modes of thinking is challenging because the teacher needs to carefully balance between different motifs and how many look-ahead steps a student needs to perform. Popular online platforms like Chess.com and Lichess offer players millions of puzzles. Unlike chess tactics puzzles procured by human experts, where chess beginners can learn valuable insights, these puzzles are automatically generated and often regarded as having low pedagogical value. These platforms also rely on a heuristic to recommend puzzles to users for practice. Using the user history data over an entire year, a total of 1.5 billion puzzle-solving histories, we learn the pedagogical value of a puzzle and how to automatically choose a set of puzzles to better support chess learners using insights from offline reinforcement learning. We show that using offline policy evaluation, our trained policy has significant impact on beginners with puzzle-solving Elo range of 100--1000, particularly for the group of beginners whose learning growth was stagnant. We also performed a qualitative analysis of the puzzles discovered by our model by collecting annotation ratings from expert chess players. The success of our pipeline shows promise for a future where we can understand the pedagogical values of practice items given general user interaction data.
Allen Nie, Anirudhan Badrinath, Nicholas Tomlin +5
A human-like chess engine should mimic the style, errors, and consistency of a strong human player rather than maximize playing strength. We show that training from move sequences alone forces a model to learn two capabilities: state tracking, which reconstructs the board from move history, and decision quality, which selects good moves from that reconstructed state. These impose contradictory data requirements: low-rated games provide the diversity needed for tracking, while high-rated games provide the quality signal for decision learning. Removing low-rated data degrades performance. We formalize this tension as a dual-capability bottleneck, P <= min(T,Q), where overall performance is limited by the weaker capability. Guided by this view, we scale the model from 28M to 120M parameters to improve tracking, then introduce Elo-weighted training to improve decisions while preserving diversity. A 2 x 2 factorial ablation shows that scaling improves tracking, weighting improves decisions, and their combination is superadditive. Linear weighting works best, while overly aggressive weighting harms tracking despite lower validation loss. We also introduce a coverage-decay formula, t* = log(N/kcrit)/log b, as a reliability horizon for intra-game degeneration risk. Our final 120M-parameter model, without search, reached Lichess bullet 2570 over 253 rated games. On human move prediction it achieves 55.2% Top-1 accuracy, exceeding Maia-2 rapid and Maia-2 blitz. Unlike position-based methods, sequence input naturally encodes full game history, enabling history-dependent decisions that single-position models cannot exhibit.
Quanhao Li, Wei Jiang
Abbey Park High School · Shanghai Soong Ching Ling School