cs.LGAug 29, 2025

The Algorithm Is Not the Behavior: Learned Priors Override Look-Ahead in a Chess-Playing Neural Network

Authors: Elias Sandmann, Sebastian Lapuschkin, Wojciech Samek

Organizations: Fraunhofer HHI · TU Dublin · TU Berlin

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Recent mechanistic work has uncovered learned algorithms within neural networks, from modular arithmetic to search and planning in game-playing agents. But does algorithmic structure guarantee algorithmic behavior? We investigate this in Leela Chess Zero, the strongest neural chess engine, where prior work identified learned look-ahead. By extending the logit lens to its move-selecting policy network, we discover that correct puzzle solutions--including immediate checkmates--often appear in intermediate layers but are systematically overridden in the final output, a phenomenon we term "forgotten puzzles". Replicating prior analyses on these positions, we find that look-ahead operates normally--future moves of the correct continuation are represented, causally important, and linearly decodable--ruling out a failure of the algorithm. Instead, late layers increasingly shift toward safe play over aggression. To test whether this shift drives the override, we steer the model against these preferences and recover 71.7% of forgotten puzzles, providing causal evidence that safety priors override algorithmically computed solutions. These findings show that algorithmic structure does not guarantee algorithmic behavior: a model can internally solve a problem and still output the wrong answer.

Figures & tables

Figure 1: Our extended logit lens maps intermediate activations to policy distributions across transformer layers in Leela Chess Zero, revealing a forgotten mate-in-two puzzle with optimal line of play—the principal variation —1. \wmove Qxf1+ \wmove Kxf1 2. \wmove Rd1 \mate . Left: The correct queen sacrifice \wmove Qxf1+ maintains high probability through most layers, peaking at layer 1212 , before dropping sharply in the final layers. The losing move \wmove Qc8—a quiet queen retreat to the back rank—remains near zero throughout the network and rises only in the final layer to become the top choice. Right: The value head, trained to predict game outcomes, correctly assesses both resulting positions, indicating that the policy head’s final move selection contradicts the model’s own evaluation of the consequences. At the puzzle position itself, the value head does not see the win. Full probabilities are provided in Appendix J , with additional examples in Appendix K .
Figure 2: Our extended logit lens reveals progressive policy refinement across layers for a mate-in-two puzzle with principal variation 1. \wmove Ng3+ \wmove hxg3 2. \wmove Rh6 \mate . Move preferences follow a structured progression: from queen moves reflecting piece priors before any inter-square interaction in the input encoding, through greedy captures in early layers, to forcing checks by layer 99 and a positional rook lift at layer 1212 , before converging on the correct knight sacrifice \wmove Ng3+ in the final output. While the intermediate preferences shown are all losing, the trajectory is non-arbitrary, with each stage reflecting recognizable chess heuristics of increasing complexity. Full probabilities are provided in Appendix H , with additional examples in Appendix I .
Table 1: Playing strength (Elo rating) across transformer layers and evaluation methods
Figure 3: Puzzle-solving performance across layers, stratified by Elo rating. Red dashed lines mark phase boundaries derived from tournament analysis, indicating phase-specific improvement rates.
Figure 4: Layer-wise puzzle-solving performance across network depth. The hatched region indicates puzzles solved by at least one previous layer but not by the current layer—forgotten solutions.
Figure 5: Look-ahead analysis from Jenner et al. (2024) on forgotten puzzles. (a) Mean activation patching effects in the residual stream across layers. The 1st move target and 3rd move target squares show outsized effects compared to the other squares line. The corrupted square(s) effect diminishes in late layers. Error bars are two times the standard error of the mean. (b) Zero-ablation of the attention entry from the third to the first move target square in L12H12. The 3rd →\rightarrow 1st target line shows ablation of this single entry, while the other line shows ablation of all remaining 4,0954{,}095 entries simultaneously. Lines show the effect at a given percentile of puzzles sorted by effect size. Error bars are 95%95\% CIs. (c) Bilinear probe accuracy for predicting the third move across layers on a trained and randomly initialized model. Error bars combine variance across five probe training runs with finite-sample uncertainty in accuracy estimates.
Figure 6: Safety-related concept analysis across layers. Left: Mean concept preference Δcℓ\Delta c_{\ell} for four safety-related concepts, measured in centipawns with 95%95\% CI. Late layers shift from aggressive toward conservative concept preferences. Right: Held-out solve rate when steering with each concept’s gradient, with the strength α∈{±0.25,±0.5,…,±1.5}\alpha\in\{\pm 0.25,\pm 0.5,\ldots,\pm 1.5\} selected per layer on half of the puzzles and evaluated on the other half. Lines show the mean and shaded areas one standard deviation across 200200 random splits. Steering against the late-layer safety preferences substantially improves solve rates, peaking at layer 1313 . The gray line shows the mean of 5050 random unit vectors under the same protocol, with 95%95\% CI across vectors, while the dashed line indicates the unsteered baseline.
Figure 7: Training target and search. Top: Probability assigned to the correct move on forgotten puzzles by the training target , the penultimate layer and the final layer . Bottom: Engine solve rate by search budget, with the policy read from the final layer or the penultimate layer . Shading shows the net effect of the final layer, worse on forgotten and better on solved puzzles. With a single node, the engine plays the policy’s top move.
Appendix figures & tables70 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 8: Policy dynamics across network depth on 10,00010{,}000 CCRL positions. Lines show medians; inner and outer shaded regions represent the 2525 th– 7575 th and 55 th– 9595 th percentile ranges. Background shading indicates the three computational phases.
RankModelElo+\mathbf{+}−\mathbf{-}GamesScore (%)Avg. Oppo.Draws (%)
1Lc0 Policy Net (Anchor)2292252564009710432
2Full Model2268252564009610443
3Logit Lens Layer 131670222164008510812
4Logit Lens Layer 121356131264007411015
5Logit Lens Layer 111117101064005811167
6Logit Lens Layer 1010821010640055111811
Appendix
Table 2: Playing strength evaluation through internal tournament ( τ=0\tau=0 )
RankModelElo+\mathbf{+}−\mathbf{-}GamesScore (%)Avg. Oppo.Draws (%)
1Lc0 Policy Net (Anchor)2292––320001007490
2Full Model13678832000888071
3Logit Lens Layer 1311196632000768222
4Logit Lens Layer 1210655532000738263
5Logit Lens Layer 118765532000588375
6Logit Lens Layer 108725532000578385
Appendix
Table 3: Playing strength evaluation through internal tournament ( τ=1\tau=1 )
BotRatingTotalPerformance
 BulletBlitzRapidGamesWDL
LLLBot-In817 ±\pm 45545 ±\pm 45492 ±\pm 452190195281643
LLLBot-0843 ±\pm 45631 ±\pm 45617 ±\pm 452023304781515
LLLBot-1907 ±\pm 45611 ±\pm 45569 ±\pm 451929294561444
LLLBot-2930 ±\pm 45655 ±\pm 45669 ±\pm 451820434781299
LLLBot-3961 ±\pm 45740 ±\pm 45789 ±\pm 451799434541302
Appendix
Table 4: Performance of layer-wise policies on Lichess across different time controls
Figure 9: Layer-wise puzzle solve rates for six Lc0 networks of varying width, depth, and training generation. The hatched forgotten puzzle band between solved by any layer ≤L\leq L and solved at LL is visible in every model.
Figure 10: Layer-wise policy evolution for puzzle 62FeU from Figure 2 .
MoveInputLayer 0Layer 1Layer 2Layer 3Layer 4Layer 5Layer 6
\wmove Ng3+6.54%0.09%0.14%1.79%7.59%0.41%15.23%26.71%
\wmove Rh60.01%0.03%0.04%0.13%0.12%0.15%0.02%2.56%
\wmove Qg1+9.16%1.11%4.36%7.52%6.52%0.43%1.30%22.74%
\wmove Qxb22.80%17.91%19.80%24.22%24.28%27.55%13.09%0.96%
\wmove Qxd26.94%15.90%19.44%21.33%26.00%21.59%9.50%0.66%
\wmove Rxe50.07%16.82%21.35%12.49%5.78%9.91%24.47%10.04%
Appendix
Table 5: Move probabilities by layer for puzzle 62FeU (Part 1: Input to Layer 6)
MoveLayer 7Layer 8Layer 9Layer 10Layer 11Layer 12Layer 13Final
\wmove Ng3+27.02%20.06%19.00%25.38%39.13%28.03%49.15%87.55%
\wmove Rh68.66%15.63%9.35%10.04%28.26%51.73%9.61%0.24%
\wmove Qg1+18.86%15.30%49.02%47.22%22.42%2.68%2.08%0.28%
\wmove Qxb20.27%2.92%0.42%0.35%0.20%0.10%0.86%0.25%
\wmove Qxd20.19%1.72%0.34%0.27%0.13%0.08%0.63%0.24%
\wmove Rxe59.74%6.51%5.83%3.66%2.71%1.44%0.71%0.37%
Appendix
Table 6: Move probabilities by layer for puzzle 62FeU (Part 2: Layer 7 to Final)
Figure 11: Layer-wise policy evolution for puzzle 9JvtA .
MoveInputLayer 0Layer 1Layer 2Layer 3Layer 4Layer 5Layer 6
\wmove Nd6+32.13%0.05%3.60%11.55%0.14%0.10%0.75%0.50%
\wmove Qxa50.20%4.42%6.22%9.20%26.54%50.71%47.82%58.34%
\wmove Qxc6+0.90%26.71%56.31%19.67%20.88%16.16%7.71%5.78%
\wmove b41.44%46.77%0.36%0.88%0.58%0.01%0.01%0.16%
\wmove Qxd41.50%7.14%22.75%39.84%34.48%23.90%29.99%23.33%
\wmove Nf6+36.04%0.01%0.17%2.27%0.05%0.13%0.01%0.01%
Appendix
Table 7: Move probabilities by layer for puzzle 9JvtA (Part 1: Input to Layer 6)
MoveLayer 7Layer 8Layer 9Layer 10Layer 11Layer 12Layer 13Final
\wmove Nd6+1.22%1.26%2.22%2.73%1.72%1.65%24.37%65.34%
\wmove Qxa558.41%52.60%50.88%61.05%56.84%56.52%46.15%12.18%
\wmove Qxc6+6.60%14.67%25.41%22.02%30.43%28.79%3.26%0.61%
\wmove b40.44%1.59%0.40%0.70%1.71%3.37%3.06%0.29%
\wmove Qxd422.78%19.53%14.91%8.88%4.27%4.65%0.79%0.29%
\wmove Nf6+0.01%0.02%0.02%0.02%0.02%0.02%11.00%13.67%
Appendix
Table 8: Move probabilities by layer for puzzle 9JvtA (Part 2: Layer 7 to Final)
Figure 12: Layer-wise policy evolution for puzzle 70v1B .
MoveInputLayer 0Layer 1Layer 2Layer 3Layer 4Layer 5Layer 6
\wmove Qxe8+11.14%50.54%71.54%80.98%40.69%64.80%66.88%68.29%
\wmove Qe756.93%0.68%0.16%0.06%0.08%0.09%0.06%0.09%
\wmove bxc30.18%20.37%21.51%13.64%56.27%30.75%30.98%27.02%
\wmove Qe620.58%1.07%1.00%0.16%0.12%0.10%0.10%0.16%
\wmove h40.08%6.79%0.81%0.44%0.37%0.11%0.07%0.29%
\wmove b40.72%6.44%1.49%0.42%0.06%0.05%0.02%0.04%
Appendix
Table 9: Move probabilities by layer for puzzle 70v1B (Part 1: Input to Layer 6)
MoveLayer 7Layer 8Layer 9Layer 10Layer 11Layer 12Layer 13Final
\wmove Qxe8+42.88%60.35%74.76%69.45%78.19%75.30%53.67%88.91%
\wmove Qe70.11%0.11%0.12%0.30%0.43%0.21%0.26%0.33%
\wmove bxc352.88%35.65%20.83%25.08%17.09%20.41%35.91%0.28%
\wmove Qe60.13%0.07%0.08%0.08%0.15%0.12%0.25%0.41%
\wmove h40.38%0.25%0.22%0.86%0.28%0.14%0.31%0.34%
\wmove b40.05%0.11%0.47%0.56%0.10%0.22%1.02%0.33%
Appendix
Table 10: Move probabilities by layer for puzzle 70v1B (Part 2: Layer 7 to Final)
Figure 13: Layer-wise policy evolution for puzzle 0KBPy .
MoveInputLayer 0Layer 1Layer 2Layer 3Layer 4Layer 5Layer 6
\wmove Ng6+9.76%0.46%0.88%0.35%6.34%0.12%1.38%0.77%
\wmove Rc40.15%0.32%0.36%1.20%14.79%69.39%48.78%41.74%
\wmove Qg8+0.07%40.00%7.11%18.17%4.87%0.36%0.60%0.77%
\wmove Ne619.23%6.12%30.11%7.38%8.51%0.56%10.91%10.97%
\wmove h40.09%6.94%0.16%0.36%0.28%0.07%0.23%0.04%
\wmove Qa40.05%2.77%5.95%10.38%16.30%2.73%5.80%3.49%
Appendix
Table 11: Move probabilities by layer for puzzle 0KBPy (Part 1: Input to Layer 6)
MoveLayer 7Layer 8Layer 9Layer 10Layer 11Layer 12Layer 13Final
\wmove Ng6+3.14%10.83%14.71%16.18%35.31%53.13%33.17%70.65%
\wmove Rc433.25%14.94%13.26%5.06%1.01%0.19%0.28%0.18%
\wmove Qg8+1.51%1.85%5.92%9.55%1.02%0.73%0.52%0.16%
\wmove Ne68.39%12.37%10.91%10.09%10.47%9.26%12.17%1.46%
\wmove h40.04%0.04%0.01%0.04%1.05%1.58%17.72%4.76%
\wmove Qa42.40%3.49%1.38%2.76%2.78%0.47%0.31%0.14%
Appendix
Table 12: Move probabilities by layer for puzzle 0KBPy (Part 2: Layer 7 to Final)
Figure 14: Layer-wise policy evolution for puzzle 09SS5 .
MoveInputLayer 0Layer 1Layer 2Layer 3Layer 4Layer 5Layer 6
\wmove Qe8+1.73%0.11%0.16%0.24%1.28%0.02%0.05%1.11%
\wmove gxf7+13.74%2.54%29.80%33.00%15.57%37.26%36.43%43.50%
\wmove Qxf7+4.30%31.00%39.29%28.88%32.40%35.43%32.63%30.50%
\wmove Qxa30.43%33.82%15.83%23.85%25.18%13.23%13.23%10.04%
\wmove Qd730.62%0.06%0.18%0.05%0.03%0.03%0.02%0.02%
\wmove Rxa30.06%26.89%10.22%8.50%21.20%12.43%15.65%11.68%
Appendix
Table 13: Move probabilities by layer for puzzle 09SS5 (Part 1: Input to Layer 6)
MoveLayer 7Layer 8Layer 9Layer 10Layer 11Layer 12Layer 13Final
\wmove Qe8+0.46%3.87%4.40%7.59%16.43%15.84%22.31%58.86%
\wmove gxf7+41.53%37.46%42.34%36.43%27.24%36.29%20.04%1.46%
\wmove Qxf7+29.73%27.14%28.31%19.52%15.49%5.55%1.83%0.27%
\wmove Qxa310.11%11.00%8.21%7.69%4.35%2.92%6.87%0.45%
\wmove Qd70.02%0.01%0.01%0.01%0.02%0.02%0.17%0.24%
\wmove Rxa315.22%16.47%13.12%19.88%19.22%20.38%12.69%9.17%
Appendix
Table 14: Move probabilities by layer for puzzle 09SS5 (Part 2: Layer 7 to Final)
Figure 15: Layer-wise policy evolution for puzzle 7UUlN .
MoveInputLayer 0Layer 1Layer 2Layer 3Layer 4Layer 5Layer 6
\wmove Bh2+0.08%0.02%0.54%0.16%0.28%0.29%1.79%14.62%
\wmove Rxe315.44%30.90%82.76%82.54%86.84%86.62%76.43%43.60%
\wmove a457.52%43.79%0.40%0.10%3.44%0.02%5.46%3.59%
\wmove Re50.49%0.01%0.07%0.13%0.44%2.98%9.25%31.57%
\wmove h50.20%13.90%1.77%0.11%0.02%0.00%0.01%0.01%
\wmove b52.34%8.17%0.40%0.36%0.01%0.02%0.08%0.05%
Appendix
Table 15: Move probabilities by layer for puzzle 7UUlN (Part 1: Input to Layer 6)
MoveLayer 7Layer 8Layer 9Layer 10Layer 11Layer 12Layer 13Final
\wmove Bh2+16.54%23.41%22.23%23.31%23.15%51.16%67.02%92.87%
\wmove Rxe330.72%50.54%35.04%42.78%31.82%5.67%3.56%0.17%
\wmove a41.12%0.35%0.10%0.27%0.90%1.69%5.71%1.08%
\wmove Re548.65%21.68%38.76%29.97%35.14%35.87%3.34%0.17%
\wmove h50.01%0.01%0.01%0.01%0.02%0.02%0.37%0.26%
\wmove b50.10%0.12%0.05%0.07%0.37%0.03%1.04%0.15%
Appendix
Table 16: Move probabilities by layer for puzzle 7UUlN (Part 2: Layer 7 to Final)
Figure 16: Layer-wise policy evolution for puzzle ANklq .
MoveStockfishΔ\Delta (cp)WinDrawLossΔ\Delta Win
Current position+∞+\infty—1.4%1.4\%2.6%2.6\%96.0%96.0\%—
\wmove Qxf1+ ⋆\star+∞+\infty—99.8%99.8\%0.1%0.1\%0.0%0.0\%+98.4%+98.4\%
\wmove Ba8−4.34-4.34−∞-\infty0.1%0.1\%0.2%0.2\%99.7%99.7\%−1.3%-1.3\%
\wmove Rd7−4.36-4.36−∞-\infty0.3%0.3\%0.6%0.6\%99.1%99.1\%−1.1%-1.1\%
\wmove Bc8−4.51-4.51−∞-\infty0.3%0.3\%0.6%0.6\%99.1%99.1\%−1.1%-1.1\%
\wmove Qc8−4.87-4.87−∞-\infty0.3%0.3\%0.7%0.7\%98.9%98.9\%−1.1%-1.1\%
Appendix
Table 17: Move evaluation for puzzle ANklq : Stockfish evaluation at depth 20 and model WDL prediction for resulting positions
MoveInputLayer 0Layer 1Layer 2Layer 3Layer 4Layer 5Layer 6
\wmove Qxf1+0.46%45.73%49.14%46.74%44.85%47.54%66.46%47.93%
\wmove Rd241.09%0.07%0.04%0.15%0.44%0.81%0.08%0.11%
\wmove Qc80.10%0.02%0.04%0.03%0.03%0.13%0.01%0.02%
\wmove Qxh2+0.51%23.84%13.22%25.78%30.20%31.29%23.55%31.01%
\wmove Qg2+26.47%0.03%0.37%0.34%0.67%0.04%0.08%0.57%
\wmove Qd70.01%0.01%0.03%0.02%0.04%0.05%0.02%0.02%
Appendix
Table 18: Move probabilities by layer for puzzle ANklq (Part 1: Input to Layer 6)
MoveLayer 7Layer 8Layer 9Layer 10Layer 11Layer 12Layer 13Final
\wmove Qxf1+53.31%51.84%56.08%59.27%60.25%88.28%30.59%5.79%
\wmove Rd20.10%0.06%0.12%0.04%0.08%0.05%0.60%0.22%
\wmove Qc80.02%0.02%0.03%0.04%0.06%0.07%2.40%37.04%
\wmove Qxh2+30.50%27.80%26.32%25.86%23.28%3.29%3.05%0.18%
\wmove Qg2+0.50%2.37%1.78%0.47%0.58%0.16%1.41%0.21%
\wmove Qd70.03%0.05%0.14%0.30%0.59%0.31%7.38%24.66%
Appendix
Table 19: Move probabilities by layer for puzzle ANklq (Part 2: Layer 7 to Final)
Figure 17: Layer-wise policy evolution for puzzle ID 58Ib0 .
MoveStockfishΔ\Delta (cp)WinDrawLossΔ\Delta Win
Current position+6.18+6.18—20.0%20.0\%8.2%8.2\%71.8%71.8\%—
\wmove Rxg7+ ⋆\star+6.15+6.15−3-399.5%99.5\%0.4%0.4\%0.1%0.1\%+79.5%+79.5\%
\wmove Kf1−2.47-2.47−865-8650.9%0.9\%7.9%7.9\%91.1%91.1\%−19.1%-19.1\%
\wmove Qe3−3.01-3.01−919-9190.4%0.4\%2.5%2.5\%97.1%97.1\%−19.6%-19.6\%
\wmove Qh2−3.29-3.29−947-9470.5%0.5\%2.9%2.9\%96.6%96.6\%−19.5%-19.5\%
\wmove Rg2−3.64-3.64−982-9820.3%0.3\%1.5%1.5\%98.2%98.2\%−19.7%-19.7\%
Appendix
Table 20: Move evaluation for puzzle ID 58Ib0 : Stockfish evaluation at depth 20 and model WDL prediction for resulting positions
MoveInputLayer 0Layer 1Layer 2Layer 3Layer 4Layer 5Layer 6
\wmove Rxg7+2.93%52.97%34.97%56.12%51.65%54.48%63.68%64.57%
\wmove a559.08%3.06%0.04%0.01%0.02%0.01%0.02%0.05%
\wmove Kf10.31%0.06%0.34%0.29%0.13%0.17%0.26%0.45%
\wmove Qxa711.45%27.21%52.41%31.89%27.95%31.24%33.11%29.93%
\wmove Qg20.39%10.35%2.08%0.34%0.19%0.18%0.04%0.05%
\wmove Qb66.17%0.13%0.19%2.31%8.67%2.10%0.45%0.37%
Appendix
Table 21: Move probabilities by layer for puzzle ID 58Ib0 (Part 1: Input to Layer 6)
MoveLayer 7Layer 8Layer 9Layer 10Layer 11Layer 12Layer 13Final
\wmove Rxg7+62.81%68.93%82.50%82.72%80.06%80.68%50.84%31.61%
\wmove a50.10%0.09%0.14%0.08%0.34%0.07%0.61%0.26%
\wmove Kf10.31%0.51%0.55%0.33%1.22%6.20%30.42%52.98%
\wmove Qxa732.68%27.58%14.43%14.94%13.42%8.13%2.23%0.29%
\wmove Qg20.05%0.05%0.08%0.08%0.25%0.35%1.01%0.33%
\wmove Qb60.60%0.31%0.17%0.13%0.13%0.05%0.27%0.29%
Appendix
Table 22: Move probabilities by layer for puzzle ID 58Ib0 (Part 2: Layer 7 to Final)
Figure 18: Layer-wise policy evolution for puzzle ID 6PIrs .
MoveStockfishΔ\Delta (cp)WinDrawLossΔ\Delta Win
Current position+7.17+7.17—1.8%1.8\%22.3%22.3\%75.9%75.9\%—
\wmove g5+ ⋆\star+8.54+8.54+137+13798.3%98.3\%1.5%1.5\%0.2%0.2\%+96.5%+96.5\%
\wmove Qd3−2.70-2.70−987-9870.8%0.8\%9.7%9.7\%89.5%89.5\%−1.0%-1.0\%
\wmove Qc4−3.09-3.09−1026-10260.8%0.8\%10.1%10.1\%89.1%89.1\%−1.0%-1.0\%
\wmove Qa2−3.09-3.09−1026-10260.5%0.5\%3.6%3.6\%95.9%95.9\%−1.3%-1.3\%
\wmove Qf3−3.19-3.19−1036-10360.5%0.5\%4.9%4.9\%94.6%94.6\%−1.3%-1.3\%
Appendix
Table 23: Move evaluation for puzzle ID 6PIrs : Stockfish evaluation at depth 20 and model WDL prediction for resulting positions
MoveInputLayer 0Layer 1Layer 2Layer 3Layer 4Layer 5Layer 6
\wmove Qxe50.03%19.22%16.74%41.20%46.85%89.57%57.18%49.90%
\wmove Qg25.38%51.44%65.55%2.96%1.62%0.13%0.24%0.19%
\wmove g5+0.22%1.01%0.11%6.18%6.27%0.55%18.46%14.62%
\wmove Qd36.21%0.13%0.13%0.39%0.22%0.15%0.37%0.50%
\wmove Qd225.48%0.14%0.36%0.48%0.17%0.11%0.12%0.17%
\wmove Qc41.15%0.17%0.16%0.55%0.24%0.14%0.40%0.59%
Appendix
Table 24: Move probabilities by layer for puzzle ID 6PIrs (Part 1: Input to Layer 6)
MoveLayer 7Layer 8Layer 9Layer 10Layer 11Layer 12Layer 13Final
\wmove Qxe553.49%59.50%77.55%55.82%56.34%11.45%2.52%0.49%
\wmove Qg20.13%0.07%0.08%0.12%0.10%0.12%0.18%0.36%
\wmove g5+22.91%22.21%0.38%0.34%6.39%0.77%41.93%11.17%
\wmove Qd30.88%0.94%2.92%10.24%9.81%29.23%13.54%29.14%
\wmove Qd20.16%0.15%0.34%1.66%1.47%4.29%5.51%9.92%
\wmove Qc41.00%1.53%1.49%4.79%5.07%19.61%14.48%25.06%
Appendix
Table 25: Move probabilities by layer for puzzle ID 6PIrs (Part 2: Layer 7 to Final)
Figure 19: Layer-wise policy evolution for puzzle ID 1Egyn .
MoveStockfishΔ\Delta (cp)WinDrawLossΔ\Delta Win
Current position+∞+\infty—12.3%12.3\%5.4%5.4\%82.3%82.3\%—
\wmove Qxc8+ ⋆\star+∞+\infty—100.0%100.0\%0.0%0.0\%0.0%0.0\%+87.7%+87.7\%
\wmove Qxa7−4.62-4.62−∞-\infty0.3%0.3\%0.5%0.5\%99.2%99.2\%−12.0%-12.0\%
\wmove Qe4−7.11-7.11−∞-\infty0.0%0.0\%0.0%0.0\%100.0%100.0\%−12.3%-12.3\%
\wmove Qb4−7.23-7.23−∞-\infty0.0%0.0\%0.0%0.0\%99.9%99.9\%−12.3%-12.3\%
\wmove Qc6−7.39-7.39−∞-\infty0.0%0.0\%0.0%0.0\%100.0%100.0\%−12.3%-12.3\%
Appendix
Table 26: Move evaluation for puzzle ID 1Egyn : Stockfish evaluation at depth 20 and model WDL prediction for resulting positions
MoveInputLayer 0Layer 1Layer 2Layer 3Layer 4Layer 5Layer 6
\wmove Qxa70.13%58.43%72.93%38.65%12.37%3.65%3.45%2.67%
\wmove Qxc8+5.06%35.05%9.71%36.67%49.39%54.67%72.84%48.69%
\wmove Rxe61.67%0.17%0.20%0.40%3.85%14.24%8.29%13.42%
\wmove Qd727.16%0.05%0.33%0.68%0.40%0.11%0.10%0.09%
\wmove Qe726.77%0.05%0.49%0.36%0.62%0.20%0.09%0.20%
\wmove c40.33%0.04%0.38%0.20%0.31%0.32%0.84%2.51%
Appendix
Table 27: Move probabilities by layer for puzzle ID 1Egyn (Part 1: Input to Layer 6)
MoveLayer 7Layer 8Layer 9Layer 10Layer 11Layer 12Layer 13Final
\wmove Qxa71.84%6.79%5.08%3.98%5.37%12.37%23.09%63.67%
\wmove Qxc8+47.17%30.34%31.97%42.62%58.43%61.50%54.05%28.66%
\wmove Rxe613.02%25.77%29.43%29.42%20.49%10.78%3.06%0.16%
\wmove Qd70.06%0.07%0.20%0.12%0.09%0.06%0.12%0.21%
\wmove Qe70.10%0.11%0.19%0.08%0.07%0.06%0.23%0.22%
\wmove c43.99%9.02%11.81%16.30%10.39%8.67%1.58%0.23%
Appendix
Table 28: Move probabilities by layer for puzzle ID 1Egyn (Part 2: Layer 7 to Final)
Figure 20: Layer-wise policy evolution for puzzle ID DsaGi .
MoveStockfishΔ\Delta (cp)WinDrawLossΔ\Delta Win
Current position+5.76+5.76—4.5%4.5\%23.1%23.1\%72.4%72.4\%—
\wmove Rxg4+ ⋆\star+6.10+6.10+34+3497.9%97.9\%1.9%1.9\%0.2%0.2\%+93.4%+93.4\%
\wmove Rf1+−0.05-0.05−581-5810.8%0.8\%25.6%25.6\%73.6%73.6\%−3.7%-3.7\%
\wmove Rh1−0.15-0.15−591-5910.9%0.9\%35.8%35.8\%63.3%63.3\%−3.7%-3.7\%
\wmove Kg7−2.46-2.46−822-8220.6%0.6\%8.9%8.9\%90.5%90.5\%−4.0%-4.0\%
\wmove Rc1−2.75-2.75−851-8510.3%0.3\%3.7%3.7\%96.0%96.0\%−4.2%-4.2\%
Appendix
Table 29: Move evaluation for puzzle ID DsaGi : Stockfish evaluation at depth 20 and model WDL prediction for resulting positions
MoveInputLayer 0Layer 1Layer 2Layer 3Layer 4Layer 5Layer 6
\wmove a415.39%88.66%14.83%15.77%10.39%0.24%1.31%0.93%
\wmove Rxg4+0.38%0.66%11.54%22.23%29.10%71.66%80.50%75.47%
\wmove Rf1+3.45%0.12%1.81%7.09%4.04%0.47%0.29%0.60%
\wmove Kg70.12%0.31%25.84%10.54%5.47%1.05%2.15%1.43%
\wmove Rh15.32%0.13%2.62%2.47%3.66%0.35%0.57%1.02%
\wmove Bh80.13%0.06%0.54%0.51%18.57%1.73%8.11%17.35%
Appendix
Table 30: Move probabilities by layer for puzzle ID DsaGi (Part 1: Input to Layer 6)
MoveLayer 7Layer 8Layer 9Layer 10Layer 11Layer 12Layer 13Final
\wmove a40.96%0.48%0.04%0.20%0.86%0.64%1.17%0.72%
\wmove Rxg4+72.17%73.29%83.14%88.00%78.25%86.89%61.03%12.14%
\wmove Rf1+1.23%2.08%0.56%2.04%4.41%3.45%16.89%40.43%
\wmove Kg73.84%3.85%2.77%2.18%3.21%3.16%5.34%8.01%
\wmove Rh10.84%1.16%0.30%0.87%1.89%1.41%4.99%23.00%
\wmove Bh817.62%14.62%9.44%3.12%1.55%0.36%0.52%0.50%
Appendix
Table 31: Move probabilities by layer for puzzle ID DsaGi (Part 2: Layer 7 to Final)
Figure 21: Layer-wise policy evolution for puzzle ID 00aDl .
MoveStockfishΔ\Delta (cp)WinDrawLossΔ\Delta Win
Current position+∞+\infty—8.3%8.3\%10.6%10.6\%81.1%81.1\%—
\wmove Qxb1+ ⋆\star+∞+\infty—99.3%99.3\%0.5%0.5\%0.2%0.2\%+91.0%+91.0\%
\wmove b5−4.12-4.12−∞-\infty0.2%0.2\%0.4%0.4\%99.4%99.4\%−8.1%-8.1\%
\wmove R2a4−4.57-4.57−∞-\infty0.2%0.2\%0.3%0.3\%99.5%99.5\%−8.1%-8.1\%
\wmove R2a7−4.72-4.72−∞-\infty0.1%0.1\%0.1%0.1\%99.8%99.8\%−8.2%-8.2\%
\wmove Na5−4.74-4.74−∞-\infty0.0%0.0\%0.1%0.1\%99.9%99.9\%−8.3%-8.3\%
Appendix
Table 32: Move evaluation for puzzle ID 00aDl : Stockfish evaluation at depth 20 and model WDL prediction for resulting positions
MoveInputLayer 0Layer 1Layer 2Layer 3Layer 4Layer 5Layer 6
\wmove Qxb1+1.80%62.90%47.54%29.77%23.48%18.87%27.94%27.42%
\wmove Nd256.64%0.41%0.72%0.18%2.89%0.03%1.47%0.89%
\wmove b50.43%0.31%0.18%0.03%0.37%0.13%2.56%12.77%
\wmove Nxb21.28%0.66%0.69%24.30%29.59%35.56%31.40%28.96%
\wmove Ne326.36%0.10%0.37%0.02%0.04%0.06%0.02%0.02%
\wmove Qxb2+8.75%11.39%23.80%24.03%20.94%22.84%19.88%13.99%
Appendix
Table 33: Move probabilities by layer for puzzle ID 00aDl (Part 1: Input to Layer 6)
MoveLayer 7Layer 8Layer 9Layer 10Layer 11Layer 12Layer 13Final
\wmove Qxb1+25.28%27.69%24.79%33.23%40.50%41.78%56.96%35.99%
\wmove Nd20.68%0.26%7.69%1.87%0.16%0.17%0.21%0.32%
\wmove b516.94%26.56%17.03%25.45%35.78%33.58%8.85%36.12%
\wmove Nxb231.09%17.79%12.84%12.49%4.64%0.73%0.70%0.35%
\wmove Ne30.02%0.01%0.04%0.03%0.04%0.04%0.20%0.32%
\wmove Qxb2+10.75%11.81%11.53%8.22%3.26%0.36%0.50%0.29%
Appendix
Table 34: Move probabilities by layer for puzzle ID 00aDl (Part 2: Layer 7 to Final)
Figure 22: Attention head ablation results on forgotten puzzles for four additional look-ahead heads identified by Cruz (2025) . Format follows Figure 5(b) : the single attention entry between first and third move target squares versus all other 4,0954{,}095 entries, shown as percentile curves sorted by effect size.
Figure 23: Look-ahead analyses following Jenner et al. (2024) on our solved puzzle set, for direct comparison with the forgotten puzzle results in Figure 5 .
  Puzzle positionAfter solution move
SetnnWinDrawLossWin
Forgotten puzzles8,6488{,}64821.6%21.6\%42.1%42.1\%36.3%36.3\%75.6%75.6\%
Solved puzzles23,60123{,}60163.0%63.0\%21.6%21.6\%15.4%15.4\%88.6%88.6\%
Appendix
Table 35: Mean value-head prediction at the puzzle position (side to move) and the win probability after the solution move, from the mover’s side.
Set WinDrawLoss
Forgotten puzzles ( n=8,648n=8{,}648 )V(s)V(s)21.6%\mathbf{21.6\%}42.1%42.1\%36.3%36.3\%
 V^(s)\hat{V}(s)23.0%\mathbf{23.0\%}36.0%36.0\%41.1%41.1\%
Solved puzzles ( n=23,601n=23{,}601 )V(s)V(s)63.0%63.0\%21.6%21.6\%15.4%\mathbf{15.4\%}
 V^(s)\hat{V}(s)70.4%70.4\%13.6%13.6\%16.0%\mathbf{16.0\%}
Appendix
Table 36: The value head’s direct prediction at the puzzle position, V(s)V(s) , and its one-ply expansion under the model’s own policy, V^(s)\hat{V}(s) , averaged over puzzles. Bold marks the components discussed in the text.
MovesPolicy massWin after moveContributionShare
The solution move16.9%16.9\%75.7%75.7\%12.8%12.8\%56%56\%
Drawing ( −0.5-0.5 to +0.5+0.5 )28.3%28.3\%15.3%15.3\%4.3%4.3\%19%19\%
Slight edge for the player ( +0.5+0.5 to +2+2 )8.4%8.4\%36.4%36.4\%3.1%3.1\%13%13\%
Slight edge for the opponent ( −2-2 to −0.5-0.5 )18.7%18.7\%8.4%8.4\%1.6%1.6\%7%7\%
Losing (below −2-2 )27.0%27.0\%3.3%3.3\%0.9%0.9\%4%4\%
Another winning move (above +2+2 or mate)0.7%0.7\%47.0%47.0\%0.3%0.3\%1%1\%
Appendix
Table 37: Win component of the one-ply expansion on forgotten puzzles, split by the Stockfish 17.1 evaluation of each legal move at depth 12, in pawns from the mover’s side. Policy mass and contribution are means over puzzles; the win probability after the move is the policy-weighted mean within each band, so that policy mass times win probability gives the contribution.
ConceptDescription
material t [mg|eg|ph]Material score, where each piece on the board has a predefined value that changes depending on the phase of the game.
imbalance t [mg|eg|ph]Imbalance score that compares the piece count of each piece type for both colors. E.g., it awards having a pair of bishops vs a bishop and a knight.
pawns t [mg|eg|ph]Evaluation of the pawn structure. E.g., the evaluation considers isolated, doubled, connected, backward, blocked, weak, etc. pawns.
knights [mine|opponent|t] [mg|eg|ph]Evaluation of knights. E.g., extra points are given to knights that occupy outposts protected by pawns.
bishops [mine|opponent|t] [mg|eg|ph]Evaluation of bishops. E.g., bishops that occupy the same color squares as pawns are penalized.
rooks [mine|opponent|t] [mg|eg|ph]Evaluation of rooks. E.g., rooks that occupy open or semi-open files have higher valuation.
Appendix
Table 38: Summary of chess concepts from Stockfish 8’s evaluation function taken from McGrath et al. (2022) . Concepts are enumerated as <concept_name> <side> <game_phase> , where side is [mine|opponent|t] for current player, opponent, or total (difference), and game phase is [mg|eg|ph] for midgame, endgame, or phase-interpolated value.
Figure 24: Stockfish evaluation concepts across layers (Part 1 of 7). Lines show mean probability-weighted concept delta across positions; shaded regions show 95% confidence intervals. All concepts are evaluated from the current player’s perspective.
Figure 25: Stockfish evaluation concepts across layers (Part 2 of 7). Lines show mean probability-weighted concept delta across positions; shaded regions show 95% confidence intervals. All concepts are evaluated from the current player’s perspective.
Figure 26: Stockfish evaluation concepts across layers (Part 3 of 7). Lines show mean probability-weighted concept delta across positions; shaded regions show 95% confidence intervals. All concepts are evaluated from the current player’s perspective.
Figure 27: Stockfish evaluation concepts across layers (Part 4 of 7). Lines show mean probability-weighted concept delta across positions; shaded regions show 95% confidence intervals. All concepts are evaluated from the current player’s perspective.
Figure 28: Stockfish evaluation concepts across layers (Part 5 of 7). Lines show mean probability-weighted concept delta across positions; shaded regions show 95% confidence intervals. All concepts are evaluated from the current player’s perspective.
Figure 29: Stockfish evaluation concepts across layers (Part 6 of 7). Lines show mean probability-weighted concept delta across positions; shaded regions show 95% confidence intervals. All concepts are evaluated from the current player’s perspective.
Figure 30: Stockfish evaluation concepts across layers (Part 7 of 7). Panel (d) shows Stockfish’s final static evaluation, which differs from the phased total in (c) by scaling down drawish endgames and adding a bonus for the side to move. Panel (e) shows the phased total in (c) without the material term, which dominates the total. Lines show mean probability-weighted concept delta across positions; shaded regions show 95% confidence intervals. All concepts are evaluated from the current player’s perspective.
Figure 31: Concept steering effects across layers (Part 1 of 7). Each panel shows the held-out solve rate for the given concept at each intervention layer: the strength is selected on a random half of the puzzles and evaluated on the other half, averaged over 200200 random splits, and the shaded band shows one standard deviation across splits. Marker colors indicate the most frequently selected strength, blue for positive and red for negative values. The gray line and band show the mean and 95%95\% CI across 5050 random directions under the same protocol. Panels without a curve are terms that are zero in every position, so no steering vector exists. The red dashed line shows the unsteered baseline.
Figure 32: Concept steering effects across layers (Part 2 of 7). Each panel shows the held-out solve rate for the given concept at each intervention layer: the strength is selected on a random half of the puzzles and evaluated on the other half, averaged over 200200 random splits, and the shaded band shows one standard deviation across splits. Marker colors indicate the most frequently selected strength, blue for positive and red for negative values. The gray line and band show the mean and 95%95\% CI across 5050 random directions under the same protocol. Panels without a curve are terms that are zero in every position, so no steering vector exists. The red dashed line shows the unsteered baseline.
Figure 33: Concept steering effects across layers (Part 3 of 7). Each panel shows the held-out solve rate for the given concept at each intervention layer: the strength is selected on a random half of the puzzles and evaluated on the other half, averaged over 200200 random splits, and the shaded band shows one standard deviation across splits. Marker colors indicate the most frequently selected strength, blue for positive and red for negative values. The gray line and band show the mean and 95%95\% CI across 5050 random directions under the same protocol. Panels without a curve are terms that are zero in every position, so no steering vector exists. The red dashed line shows the unsteered baseline.
Figure 34: Concept steering effects across layers (Part 4 of 7). Each panel shows the held-out solve rate for the given concept at each intervention layer: the strength is selected on a random half of the puzzles and evaluated on the other half, averaged over 200200 random splits, and the shaded band shows one standard deviation across splits. Marker colors indicate the most frequently selected strength, blue for positive and red for negative values. The gray line and band show the mean and 95%95\% CI across 5050 random directions under the same protocol. Panels without a curve are terms that are zero in every position, so no steering vector exists. The red dashed line shows the unsteered baseline.
Figure 35: Concept steering effects across layers (Part 5 of 7). Each panel shows the held-out solve rate for the given concept at each intervention layer: the strength is selected on a random half of the puzzles and evaluated on the other half, averaged over 200200 random splits, and the shaded band shows one standard deviation across splits. Marker colors indicate the most frequently selected strength, blue for positive and red for negative values. The gray line and band show the mean and 95%95\% CI across 5050 random directions under the same protocol. Panels without a curve are terms that are zero in every position, so no steering vector exists. The red dashed line shows the unsteered baseline.
Figure 36: Concept steering effects across layers (Part 6 of 7). Each panel shows the held-out solve rate for the given concept at each intervention layer: the strength is selected on a random half of the puzzles and evaluated on the other half, averaged over 200200 random splits, and the shaded band shows one standard deviation across splits. Marker colors indicate the most frequently selected strength, blue for positive and red for negative values. The gray line and band show the mean and 95%95\% CI across 5050 random directions under the same protocol. Panels without a curve are terms that are zero in every position, so no steering vector exists. The red dashed line shows the unsteered baseline.
Figure 37: Concept steering effects across layers (Part 7 of 7). Each panel shows the held-out solve rate for the given concept at each intervention layer: the strength is selected on a random half of the puzzles and evaluated on the other half, averaged over 200200 random splits, and the shaded band shows one standard deviation across splits. Marker colors indicate the most frequently selected strength, blue for positive and red for negative values. The gray line and band show the mean and 95%95\% CI across 5050 random directions under the same protocol. Panels without a curve are terms that are zero in every position, so no steering vector exists. The red dashed line shows the unsteered baseline.
SettingTraining targetDeployment
Policy temperature1.451.451.3591.359
cpuctc_{\text{puct}} , tree / root1.321.32 / 2.02.01.7451.745 / 1.7451.745
First-play urgency, tree / rootreduction 0.260.26 / absolute 1.01.0reduction 0.330.33 / reduction 0.330.33
Moves-left utilitymax. effect 0.20.2 , threshold 00 , slope 0.0070.007 , quadratic 0.850.85 , scaled 0.150.15 , constant 00max. effect 0.03450.0345 , threshold 0.80.8 , slope 0.00270.0027 , quadratic −0.6521-0.6521 , scaled 1.65211.6521 , constant 00
Root noiseDirichlet, ε=0.1\varepsilon=0.1 , α=0.12\alpha=0.12none
Smart pruningofffactor 1.331.33
Appendix
Table 39: Search settings for the training target, taken from the self-play run that generated the network’s training data, and for deployment, which uses the engine’s defaults. In both profiles, cpuctc_{\text{puct}} grows with the visit count with the engine’s default factor 3.8943.894 and base 38,73938{,}739 , and a twofold repetition counts as a draw within the search.
 ForgottenSolved
Puzzles8,6488{,}64823,60123{,}601
Visits when the search stops, median861861809809
Target probability on the solution, mean0.9310.9310.9530.953
Target’s top move is the solution99.4%99.4\%100.0%100.0\%
Solution has the highest value at the root99.5%99.5\%>99.99%>99.99\%
Final-layer probability on the solution, median0.1590.1590.7950.795
Appendix
Table 40: Training target on forgotten and solved puzzles, compared with the policies read from the final and the penultimate layer.
PuzzlesNetwork118832321281288008002,0482{,}048
ForgottenFinal layer0.0%0.0\%53.7%53.7\%78.3%78.3\%92.0%92.0\%97.8%97.8\%99.1%99.1\%
 Penultimate layer43.7%43.7\%75.3%75.3\%86.4%86.4\%94.8%94.8\%98.6%98.6\%99.3%99.3\%
SolvedFinal layer100.0%100.0\%99.98%99.98\%>99.99%>99.99\%>99.99%>99.99\%100.0%100.0\%100.0%100.0\%
 Penultimate layer76.7%76.7\%98.6%98.6\%99.6%99.6\%99.9%99.9\%99.97%99.97\%99.99%99.99\%
Appendix
Table 41: Share of puzzles in which the engine plays the solution’s first move, by search budget in nodes, with engine defaults.
More in this topic

Neural Algorithmic Reasoning

Neural algorithmic reasoning studies how neural networks can learn, execute, and generalize algorithmic procedures.

See all 46 papers

Explore similar work

CardsList

  1. Beyond Search-Imitation: Prior-Directed Exploration for Searchless Chess

    Aug 27, 2026Szymon Miłosz, Piotr Duch, Szymon GrabowskiChessRL Exploration

    Institute of Applied Computer Science, Lodz University of Technology

  2. Chessformer: A Unified Architecture for Chess Modeling

    May 18, 2026Daniel Monroe, George Eilender, Philip Chalmers +2Transformer InterpretabilityGame-Playing Agents

    University of Toronto · Williams College

  3. Do Chess Explanations Reflect Model Decisions? Behavioral and Token-Level Tests of LLM Reasoning Faithfulness

    Sep 5, 2026Angelina ParfenovaFaithfulness of Language Model ExplanationsLLM Reasoning

    Lucerne University of Applied Sciences and Arts · Technical University of Munich