Modular Discovery of General Game-Playing Algorithms with Large Language Models

Zun Li; John Schultz; Marc Lanctot; Daniel Hennes

cs.AISep 27, 2026

Modular Discovery of General Game-Playing Algorithms with Large Language Models

Authors: Zun Li, John Schultz, Marc Lanctot, Daniel Hennes

Organizations: Google DeepMind · Work done while at Google DeepMind

Abstract

General Game Playing across arbitrary games from rules alone remains challenging due to differing algorithmic requirements across game classes and strict decision-time constraints. Rather than hand-designing search heuristics for specific domains, can we leverage Large Language Models (LLMs) to discover general game-playing algorithms? Because language models can propose and refactor structured code, they provide an expressive proposal engine for exploring the space of algorithmic designs. We introduce a multi-agent LLM meta-learning system to co-evolve game-agnostic procedural search mechanisms in C++ alongside domain heuristics synthesized directly from game rules. Controlling the compute budget, we benchmark the discovered mechanisms across more than 400 diverse environments, including OpenSpiel training and held-out games, procedural simulation engines, and games with deep neural policy-value representations trained via PPO. Evaluated via AlphaRank stationary distributions and Soft Condorcet Optimization (SCO) against 15 established MCTS baselines, the discovered search mechanisms consistently achieve top-tier ratings and pairwise ballot majorities over most baselines across independent evolutionary runs, generalizing to unseen human-designed and procedurally synthesized games and remaining competitive with baselines on frozen neural network representations.

Figures & tables

Panel of Figure 1 — Figure 1: Exploitability across simulation budgets (mean over independent seeds; lower is better). Dashed lines denote uniform random play; dash-dotted lines denote CFR (100 iterations). Numerical values and significance tests are in Appendix Table 4 .

Discovered Search Mechanism (Ours)
Algorithmic Paradigm / Baseline	Training Games (50)		Held-Out Games (19)		Synthesized Games (360)		Neural PPO Games (17)
	SCO Rating	$m^{\star}_{\mathrm{evo}}$ Win %	SCO Rating	$m^{\star}_{\mathrm{evo}}$ Win %	SCO Rating	$m^{\star}_{\mathrm{evo}}$ Win %	SCO Rating	$m^{\star}_{\mathrm{evo}}$ Win %
$m^{\star}_{\mathrm{evo}}$ (Multi-run aggregate)	$\mathbf{509.34\pm 0.39}$	—	$\mathbf{509.36\pm 0.48}$	—	$\mathbf{508.99\pm 0.20}$	—	$\mathbf{508.91\pm 0.67}$	—
$m^{\star}_{\mathrm{BIFV}}$ ( Bifv-Mcts )	$\mathbf{509.53}$	—	$\mathbf{509.93}$	—	$\mathbf{508.58}$	—	$\mathbf{510.31}$	—
Bandit Tree Search
Vanilla UCT ( Kocsis and Szepesvári, 2006 )	$501.22\pm 0.82$	$61.5\%\pm 4.2\%$	$503.84\pm 2.08$	$59.7\%\pm 5.6\%$	$504.19\pm 0.41$	$59.1\%\pm 0.7\%$	$500.77\pm 2.10$	$54.2\%\pm 4.9\%$

Table 1: Continuous SCO Ratings and Pairwise Ballot Win Rates across Four Benchmark Tiers under a 50 ms Budget (mean

\pm

SE across Seeds 1..5, and representative case study

m^{\star}_{\mathrm{BIFV}}

). For each tier, we report both the continuous Soft Condorcet Optimization rating ( SCO , scale

[0,1000]

) and the pairwise ballot win rate ( Win % ) denoting the percentage of games where

m^{\star}_{\mathrm{evo}}

outranks that baseline under AlphaRank dynamics.

Appendix figures & tables29 assets

Supplementary material from the paper’s appendix.

Appendix

Module Interface	Type Signature	Operational Role
Procedural Mechanism Modules (Game-Agnostic Quintet, $m\in\Omega_{\mathrm{mech}}$ )
TreePolicy	$\nu\mapsto a\in A(\nu)$	Balances exploitation vs. exploration during tree descent
ValueBackup	$(\nu,\vec{v},d,p)\mapsto\text{update}$	Propagates leaf payoff vectors along traversed trajectory
RootActionSelector	$(\nu_{\text{root}},t,B)\mapsto a$	Schedules root simulation budget across candidate actions
FinalActionSelector	$\nu_{\text{root}}\mapsto(a^{\ast},\hat{\pi})$	Selects final action and posterior policy distribution
VirtualLoss	$(\nu,d)\mapsto(\Delta Q,\Delta N)$	Coordinates asynchronous parallel search workers

Appendix

Table 2: The eight typed interfaces forming the modular simulation search template. Together, they constitute eight swappable computational degrees of freedom that span the design space of simulation search. Concrete C++ virtual method signatures and class declarations are provided in Section A.2 .

Game	$S_{\text{evolved}}$	$S_{\text{baseline}}$	$N_{\text{eval}}$	Curve	Game	$S_{\text{evolved}}$	$S_{\text{baseline}}$	$N_{\text{eval}}$	Curve
antichess	20	50	500	Mono	hearts	20	50	500	Flat
backgammon	40	100	500	Flat	hex	80	200	500	Flat
banqi	20	50	500	Mono	kriegspiel	40	100	32	Flat
bargaining	320	800	500	Mono	kuhn_poker	20	50	500	Flat
breakthrough	40	100	500	Flat	latent_ttt	40	100	500	Mono
bridge	20	50	500	Flat	leduc_poker	20	50	500	Flat

Appendix

Table 3: Complete calibrated simulation budgets and evaluation episode counts across the 50 extensive-form OpenSpiel training environments. Evolved candidates receive

S_{\text{evolved}}

simulations; reference PUCT opponents receive

S_{\text{baseline}}=2.5\times S_{\text{evolved}}

N_{\text{eval}}

denotes episodes played per evaluation round, calibrated to complete well within a 1-hour wall-clock window. Curve indicates whether the simulation response is strictly monotonic ( Mono ) or flat/saturated ( Flat ).

Configuration	Kuhn poker	Leduc poker	Liar’s dice
Uniform random	0.4583	2.3736	0.7807
Baseline ( $\langle m_{\text{ref}},\kappa_{\text{base}}\rangle$ )	$0.1698\pm 0.0001$	$1.5171\pm 0.0006$	$0.5700\pm 0.0004$
Mechanism ( $\langle m^{\star},\kappa_{\text{base}}\rangle$ )	$\mathbf{0.1237\pm 0.0085}$	$0.9521\pm 0.0213$	$0.5659\pm 0.0085$
PUCT ( $\langle m_{\text{ref}},\kappa^{\star}\rangle$ )	$0.1313\pm 0.0133$	$1.1781\pm 0.1657$	$0.5216\pm 0.0901$
Mechanism ( $m^{\star}+\pi^{\star}+v^{\star},b_{\text{base}}$ )	$0.1508\pm 0.0183$	$\mathbf{0.9074\pm 0.0593}$	$\mathbf{0.4748\pm 0.0383}$
Full system ( $\langle m^{\star},\kappa^{\star}\rangle$ )	$0.1294\pm 0.0197$	$1.2074\pm 0.1711$	$0.5703\pm 0.0937$

Appendix

Table 4: Exploitability

e

(Eq. 13 , lower is better) at

B=1000

, mean

\pm

standard error over independent evolution replications. Uniform-random and CFR-at-

100

-iterations are computed exactly and bracket the achievable range.

m^{\star}

denotes the discovered search mechanism,

m^{\star}+\pi^{\star}+v^{\star}

the evolved mechanism with priors and value estimator alone (with

b_{\text{base}}

), and

\kappa^{\star}=(\pi^{\star},v^{\star},b^{\star})

the complete domain knowledge triad including evolved belief resampler

b^{\star}

. Bold marks the best configuration in the table per game.

		Kuhn poker		Leduc poker		Liar’s dice
Module	$B$	$\Delta$	$p$	$\Delta$	$p$	$\Delta$	$p$
Mechanism ( $\langle m^{\star},\kappa_{\text{base}}\rangle$ )	20	$-0.109$	$<0.001$	$-0.336$	$<0.001$	$-0.043$	$0.013$
	100	$-0.081$	$<0.001$	$-0.465$	$<0.001$	$-0.061$	$<0.001$
	1000	$-0.046$	$0.006$	$-0.565$	$<0.001$	$-0.004$	$0.648$
Domain Knowledge ( $\langle m_{\text{ref}},\kappa^{\star}\rangle$ )	20	$-0.110$	$0.099$	$-0.697$	$0.119$	$-0.076$	$0.378$
	100	$-0.057$	$0.153$	$-0.516$	$0.189$	$-0.094$	$0.280$

Appendix

Table 5: Paired change in

e

relative to the baseline (negative favours the discovered module), with two-tailed paired

t

-test

p

-values over independent replications. The mechanism significantly reduces exploitability in Kuhn poker and Liar’s dice across lower budgets, and in Leduc poker across all budgets (

p<0.001

Game	Configuration	Mean $\pm$ SE	95% CI
Kuhn Poker	PUCT baseline ( $\langle m_{\text{ref}},\kappa_{\text{base}}\rangle$ )	0.2224 $\pm$ 0.0001	[0.2222, 0.2226]
	Evolved mechanism alone ( $\langle m^{\star},\kappa_{\text{base}}\rangle$ )	0.1412 $\pm$ 0.0081	[0.1253, 0.1571]
	PUCT + evolved domain knowledge ( $\langle m_{\text{ref}},\kappa^{\star}\rangle$ )	0.1652 $\pm$ 0.0323	[0.1019, 0.2285]
	Full co-evolved system ( $\langle m^{\star},\kappa^{\star}\rangle$ )	0.1280 $\pm$ 0.0136	[0.1013, 0.1547]
Leduc Poker	PUCT baseline ( $\langle m_{\text{ref}},\kappa_{\text{base}}\rangle$ )	1.8700 $\pm$ 0.0007	[1.8686, 1.8714]
	Evolved mechanism alone ( $\langle m^{\star},\kappa_{\text{base}}\rangle$ )	1.4051 $\pm$ 0.0264	[1.3534, 1.4568]

Appendix

Table 6: Cross-seed exploitability distribution at budget

B=100

simulations across independent seeds (mean

\pm

standard error and 95% confidence intervals). Pure procedural search (

m^{\star}

) exhibits consistent performance with low variance across games, whereas evolved domain heuristics exhibit higher dispersion on Leduc Poker.

Knowledge	Belief	Sims	Evolved Mech	PUCT Mech	Within-Row Mech Contrast	Contrast vs. Baseline
		$B$	(Mean $\pm$ SE)	(Mean $\pm$ SE)	$\Delta_{\text{mech}}$ (%, $p$ )	$\Delta_{\text{base}}$ (%, $p$ )
Seed	Native	20	0.1879 $\pm$ 0.0053	0.2966 $\pm$ 0.0003	-36.6% ( $p<0.001^{***}$ )	-36.6% ( $p<0.001^{***}$ )
Seed	Native	50	0.1595 $\pm$ 0.0071	0.2450 $\pm$ 0.0001	-34.9% ( $p<0.001^{***}$ )	-34.9% ( $p<0.001^{***}$ )
Seed	Native	100	0.1412 $\pm$ 0.0081	0.2224 $\pm$ 0.0001	-36.5% ( $p<0.001^{***}$ )	-36.5% ( $p<0.001^{***}$ )
Seed	Native	500	0.1177 $\pm$ 0.0085	0.1873 $\pm$ 0.0002	-37.1% ( $p=0.001^{**}$ )	-37.1% ( $p=0.001^{**}$ )
Seed	Native	1000	0.1237 $\pm$ 0.0085	0.1698 $\pm$ 0.0001	-27.1% ( $p=0.006^{**}$ )	-27.1% ( $p=0.006^{**}$ )

Appendix

Table 7: Complete exploitability breakdown for Kuhn Poker across all simulation budgets (

B\in\{20,50,100,500,1000\}

). Random floor anchor: 0.4583; CFR@100: 0.0082. Within-Row Mech Contrast reports paired difference (Evolved vs. PUCT under identical knowledge and belief). Contrast vs. Baseline reports paired difference relative to puct+seed+native .

Knowledge	Belief	Sims	Evolved Mech	PUCT Mech	Within-Row Mech Contrast	Contrast vs. Baseline
		$B$	(Mean $\pm$ SE)	(Mean $\pm$ SE)	$\Delta_{\text{mech}}$ (%, $p$ )	$\Delta_{\text{base}}$ (%, $p$ )
Seed	Native	20	1.7759 $\pm$ 0.0067	2.1116 $\pm$ 0.0017	-15.9% ( $p<0.001^{***}$ )	-15.9% ( $p<0.001^{***}$ )
Seed	Native	50	1.5908 $\pm$ 0.0128	1.9799 $\pm$ 0.0014	-19.7% ( $p<0.001^{***}$ )	-19.7% ( $p<0.001^{***}$ )
Seed	Native	100	1.4051 $\pm$ 0.0264	1.8700 $\pm$ 0.0007	-24.9% ( $p<0.001^{***}$ )	-24.9% ( $p<0.001^{***}$ )
Seed	Native	500	1.0711 $\pm$ 0.0263	1.6381 $\pm$ 0.0006	-34.6% ( $p<0.001^{***}$ )	-34.6% ( $p<0.001^{***}$ )
Seed	Native	1000	0.9521 $\pm$ 0.0213	1.5171 $\pm$ 0.0006	-37.2% ( $p<0.001^{***}$ )	-37.2% ( $p<0.001^{***}$ )

Appendix

Table 8: Complete exploitability breakdown for Leduc Poker across all simulation budgets (

B\in\{20,50,100,500,1000\}

). Random floor anchor: 2.3736; CFR@100: 0.0957. Within-Row Mech Contrast reports paired difference (Evolved vs. PUCT under identical knowledge and belief). Contrast vs. Baseline reports paired difference relative to puct+seed+native .

Knowledge	Belief	Sims	Evolved Mech	PUCT Mech	Within-Row Mech Contrast	Contrast vs. Baseline
		$B$	(Mean $\pm$ SE)	(Mean $\pm$ SE)	$\Delta_{\text{mech}}$ (%, $p$ )	$\Delta_{\text{base}}$ (%, $p$ )
Seed	Native	20	0.6854 $\pm$ 0.0105	0.7285 $\pm$ 0.0009	-5.9% ( $p=0.013^{*}$ )	-5.9% ( $p=0.013^{*}$ )
Seed	Native	50	0.6469 $\pm$ 0.0052	0.7165 $\pm$ 0.0010	-9.7% ( $p<0.001^{***}$ )	-9.7% ( $p<0.001^{***}$ )
Seed	Native	100	0.6446 $\pm$ 0.0069	0.7054 $\pm$ 0.0009	-8.6% ( $p<0.001^{***}$ )	-8.6% ( $p<0.001^{***}$ )
Seed	Native	500	0.6041 $\pm$ 0.0060	0.6372 $\pm$ 0.0011	-5.2% ( $p=0.004^{**}$ )	-5.2% ( $p=0.004^{**}$ )
Seed	Native	1000	0.5659 $\pm$ 0.0085	0.5700 $\pm$ 0.0004	-0.7% ( $p=0.648$ )	-0.7% ( $p=0.648$ )

Appendix

Table 9: Complete exploitability breakdown for Liar’s Dice across all simulation budgets (

B\in\{20,50,100,500,1000\}

). Random floor anchor: 0.7807; CFR@100: 0.0224. Within-Row Mech Contrast reports paired difference (Evolved vs. PUCT under identical knowledge and belief). Contrast vs. Baseline reports paired difference relative to puct+seed+native .

Baseline	Algorithmic Paradigm	Hyperparameter	Search Grid	Selected Value ( $p^{\star}$ )
Vanilla UCT	Bandit Tree Search	Exploration constant $c$	$\{0.25,0.5,1.0,2.0,4.0\}$	$c=0.25$
PUCT	Bandit Tree Search	Predictor weight $c_{\mathrm{init}}$	$\{0.31,0.63,1.25,2.5,5.0\}$	$c_{\mathrm{init}}=5.0$
UCB-Tuned	Bandit Tree Search	Variance scale $c^{\prime}$	$\{0.25,0.5,1.0,2.0,4.0\}$	$c^{\prime}=0.5$
Gumbel AlphaZero	Entropy Planning	Improvement scale $c_{\mathrm{scale}}$	$\{0.025,0.05,0.1,0.2,0.4\}$	$c_{\mathrm{scale}}=0.4$
MENTS	Entropy Planning	Temperature $\tau\times$ Exploration $\epsilon$	$\{0.25,\dots,4.0\}\times\{0.025,\dots,0.4\}$	$\tau=0.25,\epsilon=0.05$
RPO	Entropy Planning	Regularization temperature $\tau$	$\{0.25,0.5,1.0,2.0,4.0\}$	$\tau=0.25$

Appendix

Table 10: Hyperparameter tuning search spaces and final selected values (

p^{\star}

) across the 15 MCTS baselines (including canonical reference PUCT). Optimal parameters were identified via offline grid search across the training suite and selected via 95% bootstrap Lower Confidence Bounds (LCB) over paired matches against reference defaults.

Category	Game	Type	Players	Horizon / Complexity
Perfect Info Deterministic	Amazons	Board territory	2	Deep, high branching
	Quoridor	Path maze	2	Medium, topological
	Twixt	Connection	2	Medium, virtual links
	Oware	Mancala variant	2	Medium, pit counting
	Ultimate Tic-Tac-Toe	Nested grid	2	High tactical branching
	Nine Men’s Morris	Mill alignment	2	Moderate, phase shifts

Appendix

Table 11: Taxonomy and structural properties of the 19 held-out evaluation games.

Structural Dimension	Category / Profile	Count ( $N=360$ )	Percentage
Information Structure	Imperfect Information (hidden hands, fog-of-war, private cards)	166	46.1%
	Perfect Information Deterministic	133	36.9%
	Perfect Information Stochastic (dice rolls, chance transitions)	61	16.9%
Player Count	2-Player	317	88.1%
	4-Player	24	6.7%
	3-Player	7	1.9%

Appendix

Table 12: Taxonomy and empirical distribution of the 360 procedurally synthesized evaluation environments derived from natural language Wikipedia rule specifications.

Independent Evolution Run	Synthesized Games	Held-Out Games	Training Games	Neural PPO Games
Bifv-Mcts ( $m^{\star}_{\mathrm{BIFV}}$ , Case Study)	508.58 (Rank 1)	509.93 (Rank 1)	509.53 (Rank 1)	510.31 (Rank 1)
Seed 4	508.52 (Rank 1)	510.15 (Rank 1)	509.92 (Rank 1)	508.05 (Rank 1)
Seed 1	508.96 (Rank 1)	509.63 (Rank 1)	507.83 (Rank 2)	508.30 (Rank 2)
Seed 5	509.45 (Rank 1)	507.50 (Rank 2)	509.93 (Rank 1)	510.66 (Rank 1)
Seed 2	509.45 (Rank 1)	509.59 (Rank 1)	509.48 (Rank 1)	507.25 (Rank 2)
Aggregate Mean $\pm$ SE	$\mathbf{508.99\pm 0.20}$	$\mathbf{509.36\pm 0.48}$	$\mathbf{509.34\pm 0.39}$	$\mathbf{508.91\pm 0.67}$

Appendix

Table 13: Granular Soft Condorcet Optimization (SCO) Ratings across Independent Evolution Seeds under a Fixed 50 ms Search Budget (IsoTime). Ratings are continuous latent SCO values (

r\in[0,1000]

) derived from AlphaRank stationary preference profiles ( use_inf_alpha=True ) across

N

-player empirical payoff tensors comparing

m^{\star}_{\mathrm{evo}}

against 15 competitive baselines. Parenthesized values denote the rank achieved by each evolved mechanism within its own seed evaluation tournament under AlphaRank+SCO. Aggregate performance across runs is reported as continuous SCO rating (

\text{Mean}\pm\text{SE}

Independent Evolution Run	Synthesized Games	Held-Out Games	Training Games	Neural PPO Games
Bifv-Mcts ( $m^{\star}_{\mathrm{BIFV}}$ , Case Study)	508.99 (Rank 1)	510.57 (Rank 1)	510.40 (Rank 1)	510.01 (Rank 1)
Seed 5	508.93 (Rank 1)	510.91 (Rank 1)	508.61 (Rank 1)	509.20 (Rank 1)
Seed 1	508.84 (Rank 1)	508.05 (Rank 2)	510.18 (Rank 1)	505.83 (Rank 3)
Seed 4	507.44 (Rank 1)	507.33 (Rank 1)	508.06 (Rank 2)	506.31 (Rank 3)
Seed 2	509.33 (Rank 1)	501.80 (Rank 7)	507.25 (Rank 2)	502.29 (Rank 6)
Aggregate Mean $\pm$ SE	$\mathbf{508.71\pm 0.33}$	$\mathbf{507.73\pm 1.64}$	$\mathbf{508.90\pm 0.61}$	$\mathbf{506.73\pm 1.37}$

Appendix

Table 14: Granular Soft Condorcet Optimization (SCO) Ratings across Independent Evolution Seeds under a Fixed-Simulation Budget (IsoSim, 1,000 Simulations). Ratings are continuous latent SCO values (

r\in[0,1000]

) derived from AlphaRank stationary preference profiles ( use_inf_alpha=True ) across

N

-player empirical payoff tensors comparing

m^{\star}_{\mathrm{evo}}

against 15 competitive baselines. Parenthesized values denote the rank achieved by each evolved mechanism within its own seed evaluation tournament under AlphaRank+SCO.

Discovered Search Mechanism (Ours)
Algorithmic Paradigm / Baseline	Training Games (50)		Held-Out Games (19)		Synthesized Games (360)		Neural PPO Games (17)
	SCO Rating	$m^{\star}_{\mathrm{evo}}$ Win %	SCO Rating	$m^{\star}_{\mathrm{evo}}$ Win %	SCO Rating	$m^{\star}_{\mathrm{evo}}$ Win %	SCO Rating	$m^{\star}_{\mathrm{evo}}$ Win %
$m^{\star}_{\mathrm{evo}}$ (Multi-run aggregate)	$\mathbf{508.90\pm 0.61}$	—	$\mathbf{507.73\pm 1.64}$	—	$\mathbf{508.71\pm 0.33}$	—	$\mathbf{506.73\pm 1.37}$	—
$m^{\star}_{\mathrm{BIFV}}$ ( Bifv-Mcts )	$\mathbf{510.40}$	—	$\mathbf{510.57}$	—	$\mathbf{508.99}$	—	$\mathbf{510.01}$	—
Bandit Tree Search
Vanilla UCT ( Kocsis and Szepesvári, 2006 )	$504.26\pm 0.67$	$60.3\%\pm 3.9\%$	$501.98\pm 2.07$	$53.5\%\pm 4.9\%$	$504.98\pm 0.32$	$57.1\%\pm 1.6\%$	$500.29\pm 1.85$	$56.2\%\pm 5.4\%$

Appendix

Table 15: Continuous SCO Ratings and Pairwise Ballot Win Rates across Four Benchmark Tiers under Fixed Simulation Budgets (IsoSim, 1,000 Simulations) (mean

\pm

SE across Seeds 1..5, and representative case study

m^{\star}_{\mathrm{BIFV}}

). For each tier, we report both the continuous Soft Condorcet Optimization rating ( SCO , scale

[0,1000]

) and the pairwise ballot win rate ( Win % ) denoting the percentage of games where

m^{\star}_{\mathrm{evo}}

outranks that baseline under AlphaRank dynamics (“—” denotes self-comparison).

	Training (50)	Held-out (19)	Synthesized (360)	Neural PPO (17)
Full Bifv-Mcts ( $m^{\star}_{\mathrm{BIFV}}$ )	71.0	66.8	64.8	78.0
$\setminus\mathbf{B}$ (branching)	$-8.5$	$-0.4$	$+5.3$	$-5.3$
$\setminus\mathbf{V}$ (tie-break)	$+9.0$	$-5.7$	$+4.9$	$-2.4$
$\setminus\mathbf{F}$ (FPU anneal)	$+9.2$	$+13.9$	$+5.9$	$-5.8$
$\setminus\mathbf{I}$ (info scale)	$+10.5$	$+10.6$	$+5.0$	$-1.0$

Appendix

Table 16: Operator ablation of Bifv-Mcts (50 ms IsoTime). Average pairwise ballot win rate (%) against the same 15 baselines; since the baseline pool is identical, rows are directly comparable. First row: full mechanism. Other rows: change when one operator is removed (positive = removal helps, negative = removal hurts).

Algorithm / Run	Exploration Schedule $c(N)$	Leaf Evaluation Policy ( $Q_{\mathrm{FPU}}$ )	Value Backup Operator	Key Algorithmic Trait
Human-Engineered Prior Art
Evolution Seed Template ( $m_{\mathrm{seed}}$ )	Static scalar $c_{\mathrm{puct}}=2.0$	Naive immediate expansion ( $v=0$ )	Arithmetic mean $\frac{1}{N}\sum R$	Minimal baseline template (evolution starting seed)
AlphaZero ( Silver et al., 2018 )	$1.25+\ln\frac{N+19652}{19652}$	Parent return $Q_{\mathrm{parent}}$	Arithmetic mean $\frac{1}{N}\sum R$	Static single-game parameter schedule
KataGo ( Wu, 2019 )	Dynamic visit-dependent schedule	Utility-bounded normalizer	Arithmetic mean $\frac{1}{N}\sum R$	Utility range clamping $[u_{\min},u_{\max}]$
Leela Chess Zero (Lc0)	Tuned logarithmic schedule	$Q_{\mathrm{parent}}-c_{\mathrm{fpu}}\sqrt{1-\sum\pi}$	Arithmetic mean $\frac{1}{N}\sum R$	Static prior-weighted FPU reduction
Discovered Programs from Evolutionary Runs (Ours)

Appendix

Table 17: Comprehensive Taxonomic Comparison of Discovered Search Programs Against Prior Art. Contrasts mathematical operators across exploration schedules, leaf evaluation (FPU), value backups, and action selection.

Design Dimension	Vector / Tabular Architecture (MLP)	Spatial / Board Architecture (CNNResNet)
Target Game Families	Card, trick-taking, imperfect-information, coordination	2D grid board games, connection, spatial territory
Evaluated Games (17 Total)	Contract Bridge, Hearts, Gin Rummy, Leduc Poker,	Chess, Shogi, Go 9 $\times$ 9, Go 13 $\times$ 13,
	Backgammon, Bargaining, Coop Box, Phantom TTT, Dark Hex	Hex, Othello, Checkers, Gomoku
Input Representation	Flat 1D feature vector $\mathbf{x}\in\mathbb{R}^{D}$	3D tensor $\mathbf{X}\in\mathbb{R}^{H\times W\times C}$ (board planes)
Backbone Structure	2 Fully Connected layers (Dense-256)	1 Initial Conv ( $3\times 3$ , 64 ch) + 4 ResNet Blocks
Residual Block Design	—	Two Conv- $3\times 3$ (64 ch) + skip connection

Appendix

Table 18: Neural Network Architectures Tailored by Game Topology. Model hyperparameters and structural specifications across the two architecture families used in the PPO benchmark.

Game (Engine)	Architecture Pairing	100 Sims	200 Sims	400 Sims	800 Sims	1600 Sims
1. Chess ( Stockfish )	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs Engine	6.7% $\pm$ 1.8	6.3% $\pm$ 1.7	6.1% $\pm$ 2.2	5.3% $\pm$ 2.3	4.7% $\pm$ 2.1
	$\mathcal{S}_{3}\text{(Search+PPO)}$ vs Engine	0.0% $\pm$ 0.0	0.2% $\pm$ 0.1	0.1% $\pm$ 0.1	0.0% $\pm$ 0.0	0.0% $\pm$ 0.0
	H1: $\mathcal{S}_{3}$ vs $\mathcal{S}_{1}$ / H2: $\mathcal{S}_{4}$ vs $\mathcal{S}_{2}$	50.0% / 56.5%	50.0% / 57.2%	50.0% / 62.2%	50.0% / 55.0%	50.0% / 60.5%
	H3: $\mathcal{S}_{1}$ vs $\mathcal{S}_{2}$ ( $S=0$ ) / H4: $\mathcal{S}_{3}$ vs $\mathcal{S}_{4}$	0.8% / 15.3%	0.8% / 9.2%	0.8% / 15.9%	0.8% / 18.4%	0.8% / 18.8%
2. Checkers ( Kingsrow )	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs Engine	0.0% $\pm$ 0.0	0.4% $\pm$ 0.1	0.5% $\pm$ 0.2	0.7% $\pm$ 0.4	1.1% $\pm$ 0.2
	$\mathcal{S}_{3}\text{(Search+PPO)}$ vs Engine	0.1% $\pm$ 0.1	0.2% $\pm$ 0.1	0.2% $\pm$ 0.0	0.3% $\pm$ 0.1	0.5% $\pm$ 0.2

Appendix

Table 19: Cross-Game Summary of Five-Strategy Empirical Game Tensors Across All 10 Benchmark Games (

S=100\to 1600

, 5-Seed Mean

\pm

SE). Rows evaluate both neural search (

\mathcal{S}_{3}

) and co-evolved heuristic search (

\mathcal{S}_{4}

) directly against the external domain engine (

\mathcal{S}_{0}

, Floor tier; default setting for WBridge5), followed by the four core internal hypothesis pairings ( H1 – H4 ; H3 is an unsearched

S=0

baseline). Full 3-tier ( Floor , Mid , Ceiling ) and per-seed breakdowns appear in Tables 20 – 29 .

Lineage / Aggregation	Empirical Tensor Entry	100 Sims	200 Sims	400 Sims	800 Sims	1600 Sims
Seed 1	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs Stockfish (Floor)	5.0%	5.0%	2.5%	3.2%	2.5%
	$\mathcal{S}_{3}\text{(Search+PPO)}$ vs Stockfish (Floor)	0.0%	0.5%	0.0%	0.0%	0.0%
	H1: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs $\mathcal{S}_{1}\text{(PPO)}$	50.0%	50.0%	50.0%	50.0%	50.0%
	H2: $\mathcal{S}_{4}\text{(Search+Heur)}$ vs $\mathcal{S}_{2}\text{(Heur)}$	68.1%	67.6%	63.7%	33.2%	39.3%
	H4: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs $\mathcal{S}_{4}\text{(Search+Heur)}$	7.5%	11.0%	12.9%	19.5%	18.5%
Seed 2	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs Stockfish (Floor)	7.3%	7.2%	6.2%	1.8%	1.0%

Appendix

Table 20: Per-Seed and 5-Seed Aggregated Empirical Game Tensor for Chess (vs. Stockfish). Entries report seat-balanced Score% (

\text{Win}\%+0.5\times\text{Draw}\%

; parentheticals show net score margin

\Delta

where applicable) across simulation budgets

S\in\{100,200,400,800,1600\}

. Rows evaluate Search Amplification of PPO (

\text{H1}:\mathcal{S}_{3}\text{ vs }\mathcal{S}_{1}

), Search Amplification of Heuristic (

\text{H2}:\mathcal{S}_{4}\text{ vs }\mathcal{S}_{2}

), Unsearched Prior Strength (

\text{H3}:\mathcal{S}_{1}\text{ vs }\mathcal{S}_{2}

S=0

), Searched Prior Dominance (

\text{H4}:\mathcal{S}_{3}\text{ vs }\mathcal{S}_{4}

), and direct engine matchups for neural search (

\mathcal{S}_{3}\text{ vs Engine}

) and co-evolved heuristic search (

\mathcal{S}_{4}\text{ vs Engine}

Lineage / Aggregation	Empirical Tensor Entry	100 Sims	200 Sims	400 Sims	800 Sims	1600 Sims
Seed 1	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs Kingsrow (Floor)	0.0%	0.0%	0.5%	1.0%	1.7%
	$\mathcal{S}_{3}\text{(Search+PPO)}$ vs Kingsrow (Floor)	0.0%	0.5%	0.2%	0.8%	1.0%
	H1: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs $\mathcal{S}_{1}\text{(PPO)}$	50.0%	50.0%	50.0%	71.2%	75.0%
	H2: $\mathcal{S}_{4}\text{(Search+Heur)}$ vs $\mathcal{S}_{2}\text{(Heur)}$	98.5%	99.8%	100.0%	98.2%	100.0%
	H4: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs $\mathcal{S}_{4}\text{(Search+Heur)}$	18.6%	48.0%	49.3%	48.8%	46.6%
Seed 2	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs Kingsrow (Floor)	0.0%	0.0%	0.2%	0.2%	0.5%

Appendix

Table 21: Per-Seed and 5-Seed Aggregated Empirical Game Tensor for Checkers (vs. Kingsrow). Entries report seat-balanced Score% (

\text{Win}\%+0.5\times\text{Draw}\%

; parentheticals show net score margin

\Delta

where applicable) across simulation budgets

S\in\{100,200,400,800,1600\}

. Rows evaluate Search Amplification of PPO (

\text{H1}:\mathcal{S}_{3}\text{ vs }\mathcal{S}_{1}

), Search Amplification of Heuristic (

\text{H2}:\mathcal{S}_{4}\text{ vs }\mathcal{S}_{2}

), Unsearched Prior Strength (

\text{H3}:\mathcal{S}_{1}\text{ vs }\mathcal{S}_{2}

S=0

), Searched Prior Dominance (

\text{H4}:\mathcal{S}_{3}\text{ vs }\mathcal{S}_{4}

), and direct engine matchups for neural search (

\mathcal{S}_{3}\text{ vs Engine}

) and co-evolved heuristic search (

\mathcal{S}_{4}\text{ vs Engine}

Lineage / Aggregation	Empirical Tensor Entry	100 Sims	200 Sims	400 Sims	800 Sims	1600 Sims
Seed 1	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs Egaroucid (Floor)	65.2%	31.5%	40.0%	44.4%	35.8%
	$\mathcal{S}_{3}\text{(Search+PPO)}$ vs Egaroucid (Floor)	0.0%	0.5%	0.0%	3.4%	0.0%
	H1: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs $\mathcal{S}_{1}\text{(PPO)}$	46.8%	82.0%	89.1%	99.8%	100.0%
	H2: $\mathcal{S}_{4}\text{(Search+Heur)}$ vs $\mathcal{S}_{2}\text{(Heur)}$	89.0%	98.9%	100.0%	100.0%	100.0%
	H4: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs $\mathcal{S}_{4}\text{(Search+Heur)}$	0.0%	0.0%	1.0%	0.0%	0.0%
Seed 2	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs Egaroucid (Floor)	31.5%	74.8%	58.4%	67.2%	86.1%

Appendix

Table 22: Per-Seed and 5-Seed Aggregated Empirical Game Tensor for Othello (vs. Egaroucid). Entries report seat-balanced Score% (

\text{Win}\%+0.5\times\text{Draw}\%

; parentheticals show net score margin

\Delta

where applicable) across simulation budgets

S\in\{100,200,400,800,1600\}

. Rows evaluate Search Amplification of PPO (

\text{H1}:\mathcal{S}_{3}\text{ vs }\mathcal{S}_{1}

), Search Amplification of Heuristic (

\text{H2}:\mathcal{S}_{4}\text{ vs }\mathcal{S}_{2}

), Unsearched Prior Strength (

\text{H3}:\mathcal{S}_{1}\text{ vs }\mathcal{S}_{2}

S=0

), Searched Prior Dominance (

\text{H4}:\mathcal{S}_{3}\text{ vs }\mathcal{S}_{4}

), and direct engine matchups for neural search (

\mathcal{S}_{3}\text{ vs Engine}

) and co-evolved heuristic search (

\mathcal{S}_{4}\text{ vs Engine}

Lineage / Aggregation	Empirical Tensor Entry	100 Sims	200 Sims	400 Sims	800 Sims	1600 Sims
Seed 1	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs Engine (Floor)	7.5%	1.5%	0.0%	3.5%	10.0%
	$\mathcal{S}_{3}\text{(Search+PPO)}$ vs Engine (Floor)	0.0%	0.0%	0.0%	0.0%	0.0%
	H1: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs $\mathcal{S}_{1}\text{(PPO)}$	91.3%	100.0%	100.0%	99.5%	99.5%
	H2: $\mathcal{S}_{4}\text{(Search+Heur)}$ vs $\mathcal{S}_{2}\text{(Heur)}$	94.0%	99.5%	98.0%	99.2%	100.0%
	H4: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs $\mathcal{S}_{4}\text{(Search+Heur)}$	0.0%	0.0%	0.0%	0.0%	0.0%
Seed 2	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs Engine (Floor)	0.0%	0.5%	0.0%	0.5%	0.0%

Appendix

Table 23: Per-Seed and 5-Seed Aggregated Empirical Game Tensor for Gomoku (vs. Gomoku Engine). Entries report seat-balanced Score% (

\text{Win}\%+0.5\times\text{Draw}\%

; parentheticals show net score margin

\Delta

where applicable) across simulation budgets

S\in\{100,200,400,800,1600\}

. Rows evaluate Search Amplification of PPO (

\text{H1}:\mathcal{S}_{3}\text{ vs }\mathcal{S}_{1}

), Search Amplification of Heuristic (

\text{H2}:\mathcal{S}_{4}\text{ vs }\mathcal{S}_{2}

), Unsearched Prior Strength (

\text{H3}:\mathcal{S}_{1}\text{ vs }\mathcal{S}_{2}

S=0

), Searched Prior Dominance (

\text{H4}:\mathcal{S}_{3}\text{ vs }\mathcal{S}_{4}

), and direct engine matchups for neural search (

\mathcal{S}_{3}\text{ vs Engine}

) and co-evolved heuristic search (

\mathcal{S}_{4}\text{ vs Engine}

Lineage / Aggregation	Empirical Tensor Entry	100 Sims	200 Sims	400 Sims	800 Sims	1600 Sims
Seed 1	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs YaneuraOu (Floor)	0.0%	0.0%	0.0%	0.0%	0.5%
	$\mathcal{S}_{3}\text{(Search+PPO)}$ vs YaneuraOu (Floor)	0.0%	0.0%	0.0%	0.0%	0.0%
	H1: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs $\mathcal{S}_{1}\text{(PPO)}$	79.2%	94.5%	95.0%	99.0%	99.5%
	H2: $\mathcal{S}_{4}\text{(Search+Heur)}$ vs $\mathcal{S}_{2}\text{(Heur)}$	97.2%	99.0%	100.0%	99.5%	100.0%
	H4: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs $\mathcal{S}_{4}\text{(Search+Heur)}$	0.0%	0.0%	0.0%	0.0%	0.0%
Seed 2	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs YaneuraOu (Floor)	0.0%	0.0%	0.0%	0.0%	0.0%

Appendix

Table 24: Per-Seed and 5-Seed Aggregated Empirical Game Tensor for Shogi (vs. YaneuraOu). Entries report seat-balanced Score% (

\text{Win}\%+0.5\times\text{Draw}\%

; parentheticals show net score margin

\Delta

where applicable) across simulation budgets

S\in\{100,200,400,800,1600\}

. Rows evaluate Search Amplification of PPO (

\text{H1}:\mathcal{S}_{3}\text{ vs }\mathcal{S}_{1}

), Search Amplification of Heuristic (

\text{H2}:\mathcal{S}_{4}\text{ vs }\mathcal{S}_{2}

), Unsearched Prior Strength (

\text{H3}:\mathcal{S}_{1}\text{ vs }\mathcal{S}_{2}

S=0

), Searched Prior Dominance (

\text{H4}:\mathcal{S}_{3}\text{ vs }\mathcal{S}_{4}

), and direct engine matchups for neural search (

\mathcal{S}_{3}\text{ vs Engine}

) and co-evolved heuristic search (

\mathcal{S}_{4}\text{ vs Engine}

Lineage / Aggregation	Empirical Tensor Entry	100 Sims	200 Sims	400 Sims	800 Sims	1600 Sims
Seed 1	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs KataHex (Floor)	0.0%	0.0%	0.0%	0.0%	0.0%
	$\mathcal{S}_{3}\text{(Search+PPO)}$ vs KataHex (Floor)	0.0%	0.0%	0.0%	0.0%	0.0%
	H1: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs $\mathcal{S}_{1}\text{(PPO)}$	91.0%	96.5%	98.0%	99.0%	97.5%
	H2: $\mathcal{S}_{4}\text{(Search+Heur)}$ vs $\mathcal{S}_{2}\text{(Heur)}$	11.5%	25.0%	37.5%	59.0%	100.0%
	H4: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs $\mathcal{S}_{4}\text{(Search+Heur)}$	0.0%	0.0%	0.0%	0.0%	0.0%
Seed 2	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs KataHex (Floor)	0.0%	0.0%	0.0%	0.0%	0.0%

Appendix

Table 25: Per-Seed and 5-Seed Aggregated Empirical Game Tensor for Hex 13x13 (vs. KataHex). Entries report seat-balanced Score% (

\text{Win}\%+0.5\times\text{Draw}\%

; parentheticals show net score margin

\Delta

where applicable) across simulation budgets

S\in\{100,200,400,800,1600\}

. Rows evaluate Search Amplification of PPO (

\text{H1}:\mathcal{S}_{3}\text{ vs }\mathcal{S}_{1}

), Search Amplification of Heuristic (

\text{H2}:\mathcal{S}_{4}\text{ vs }\mathcal{S}_{2}

), Unsearched Prior Strength (

\text{H3}:\mathcal{S}_{1}\text{ vs }\mathcal{S}_{2}

S=0

), Searched Prior Dominance (

\text{H4}:\mathcal{S}_{3}\text{ vs }\mathcal{S}_{4}

), and direct engine matchups for neural search (

\mathcal{S}_{3}\text{ vs Engine}

) and co-evolved heuristic search (

\mathcal{S}_{4}\text{ vs Engine}

Lineage / Aggregation	Empirical Tensor Entry	100 Sims	200 Sims	400 Sims	800 Sims	1600 Sims
Seed 1	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs KataGo (Floor)	1.8%	8.5%	12.9%	15.5%	14.9%
	$\mathcal{S}_{3}\text{(Search+PPO)}$ vs KataGo (Floor)	0.2%	0.5%	0.0%	0.0%	0.2%
	H1: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs $\mathcal{S}_{1}\text{(PPO)}$	79.8%	86.2%	60.5%	69.8%	77.7%
	H2: $\mathcal{S}_{4}\text{(Search+Heur)}$ vs $\mathcal{S}_{2}\text{(Heur)}$	98.0%	94.0%	98.5%	98.0%	97.5%
	H4: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs $\mathcal{S}_{4}\text{(Search+Heur)}$	1.7%	1.2%	0.5%	1.0%	0.5%
Seed 2	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs KataGo (Floor)	1.0%	1.0%	4.0%	14.5%	15.5%

Appendix

Table 26: Per-Seed and 5-Seed Aggregated Empirical Game Tensor for Go 9x9 (vs. KataGo). Entries report seat-balanced Score% (

\text{Win}\%+0.5\times\text{Draw}\%

; parentheticals show net score margin

\Delta

where applicable) across simulation budgets

S\in\{100,200,400,800,1600\}

. Rows evaluate Search Amplification of PPO (

\text{H1}:\mathcal{S}_{3}\text{ vs }\mathcal{S}_{1}

), Search Amplification of Heuristic (

\text{H2}:\mathcal{S}_{4}\text{ vs }\mathcal{S}_{2}

), Unsearched Prior Strength (

\text{H3}:\mathcal{S}_{1}\text{ vs }\mathcal{S}_{2}

S=0

), Searched Prior Dominance (

\text{H4}:\mathcal{S}_{3}\text{ vs }\mathcal{S}_{4}

), and direct engine matchups for neural search (

\mathcal{S}_{3}\text{ vs Engine}

) and co-evolved heuristic search (

\mathcal{S}_{4}\text{ vs Engine}

Lineage / Aggregation	Empirical Tensor Entry	100 Sims	200 Sims	400 Sims	800 Sims	1600 Sims
Seed 1	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs KataGo (Floor)	0.0%	0.3%	1.2%	2.3%	5.8%
	$\mathcal{S}_{3}\text{(Search+PPO)}$ vs KataGo (Floor)	1.3%	1.1%	0.6%	0.9%	1.8%
	H1: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs $\mathcal{S}_{1}\text{(PPO)}$	72.6%	70.1%	59.9%	66.1%	61.5%
	H2: $\mathcal{S}_{4}\text{(Search+Heur)}$ vs $\mathcal{S}_{2}\text{(Heur)}$	94.0%	96.5%	92.8%	98.5%	98.5%
	H4: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs $\mathcal{S}_{4}\text{(Search+Heur)}$	0.0%	0.0%	0.0%	0.0%	0.0%
Seed 2	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs KataGo (Floor)	0.0%	0.0%	0.2%	0.3%	2.7%

Appendix

Table 27: Per-Seed and 5-Seed Aggregated Empirical Game Tensor for Go 13x13 (vs. KataGo 13x13). Entries report seat-balanced Score% (

\text{Win}\%+0.5\times\text{Draw}\%

; parentheticals show net score margin

\Delta

where applicable) across simulation budgets

S\in\{100,200,400,800,1600\}

. Rows evaluate Search Amplification of PPO (

\text{H1}:\mathcal{S}_{3}\text{ vs }\mathcal{S}_{1}

), Search Amplification of Heuristic (

\text{H2}:\mathcal{S}_{4}\text{ vs }\mathcal{S}_{2}

), Unsearched Prior Strength (

\text{H3}:\mathcal{S}_{1}\text{ vs }\mathcal{S}_{2}

S=0

), Searched Prior Dominance (

\text{H4}:\mathcal{S}_{3}\text{ vs }\mathcal{S}_{4}

), and direct engine matchups for neural search (

\mathcal{S}_{3}\text{ vs Engine}

) and co-evolved heuristic search (

\mathcal{S}_{4}\text{ vs Engine}

Lineage / Aggregation	Empirical Tensor Entry	100 Sims	200 Sims	400 Sims	800 Sims	1600 Sims
Seed 1	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs Xinxin (Floor)	26.7% (-10.6)	25.0% (-11.4)	31.0% (-9.4)	26.8% (-11.3)	21.5% (-12.1)
	$\mathcal{S}_{3}\text{(Search+PPO)}$ vs Xinxin (Floor)	38.8% (-5.4)	39.5% (-4.4)	40.0% (-4.1)	35.5% (-6.3)	41.0% (-3.2)
	H1: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs $\mathcal{S}_{1}\text{(PPO)}$	54.5% (+1.5)	49.5% (+0.4)	56.0% (+2.2)	53.7% (+1.6)	61.4% (+3.3)
	H2: $\mathcal{S}_{4}\text{(Search+Heur)}$ vs $\mathcal{S}_{2}\text{(Heur)}$	42.2% (-4.0)	43.5% (-3.1)	47.0% (-2.6)	45.8% (-3.1)	41.5% (-4.1)
	H4: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs $\mathcal{S}_{4}\text{(Search+Heur)}$	77.5% (+10.5)	78.6% (+11.7)	79.2% (+12.0)	76.1% (+9.5)	81.8% (+12.5)
Seed 2	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs Xinxin (Floor)	29.2% (-10.9)	27.0% (-10.5)	29.8% (-9.7)	22.8% (-12.3)	28.2% (-10.3)

Appendix

Table 28: Per-Seed and 5-Seed Aggregated Empirical Game Tensor for Hearts (vs. Xinxin). Entries report seat-balanced Score% (

\text{Win}\%+0.5\times\text{Draw}\%

; parentheticals show net score margin

\Delta

where applicable) across simulation budgets

S\in\{100,200,400,800,1600\}

. Rows evaluate Search Amplification of PPO (

\text{H1}:\mathcal{S}_{3}\text{ vs }\mathcal{S}_{1}

), Search Amplification of Heuristic (

\text{H2}:\mathcal{S}_{4}\text{ vs }\mathcal{S}_{2}

), Unsearched Prior Strength (

\text{H3}:\mathcal{S}_{1}\text{ vs }\mathcal{S}_{2}

S=0

), Searched Prior Dominance (

\text{H4}:\mathcal{S}_{3}\text{ vs }\mathcal{S}_{4}

), and direct engine matchups for neural search (

\mathcal{S}_{3}\text{ vs Engine}

) and co-evolved heuristic search (

\mathcal{S}_{4}\text{ vs Engine}

Lineage / Aggregation	Empirical Tensor Entry	100 Sims	200 Sims	400 Sims	800 Sims	1600 Sims
Seed 1	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs WBridge5 (Default)	9.1% (-1470)	5.7% (-1528)	11.6% (-1098)	7.7% (-1154)	10.4% (-1030)
	$\mathcal{S}_{3}\text{(Search+PPO)}$ vs WBridge5 (Default)	7.4% (-1356)	5.9% (-1400)	4.4% (-1589)	8.8% (-1266)	4.7% (-1238)
	H1: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs $\mathcal{S}_{1}\text{(PPO)}$	56.0% (+385)	48.0% (-91)	44.6% (-72)	49.0% (-144)	58.5% (+346)
	H2: $\mathcal{S}_{4}\text{(Search+Heur)}$ vs $\mathcal{S}_{2}\text{(Heur)}$	63.5% (+1215)	77.2% (+1497)	81.2% (+1957)	76.6% (+1281)	69.8% (+1606)
	H4: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs $\mathcal{S}_{4}\text{(Search+Heur)}$	43.5% (+48)	42.8% (-472)	31.0% (-1029)	25.5% (-1297)	25.5% (-1314)
Seed 2	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs WBridge5 (Default)	12.7% (-759)	17.4% (-793)	23.9% (-744)	24.0% (-795)	20.2% (-877)

Appendix

Table 29: Per-Seed and 5-Seed Aggregated Empirical Game Tensor for Contract Bridge (2v2 Homogeneous Partnerships) (vs. WBridge5). Entries report seat-balanced Score% (

\text{Win}\%+0.5\times\text{Draw}\%

; parentheticals show net score margin

\Delta

where applicable) across simulation budgets

S\in\{100,200,400,800,1600\}

. Rows evaluate Search Amplification of PPO (

\text{H1}:\mathcal{S}_{3}\text{ vs }\mathcal{S}_{1}

), Search Amplification of Heuristic (

\text{H2}:\mathcal{S}_{4}\text{ vs }\mathcal{S}_{2}

), Unsearched Prior Strength (

\text{H3}:\mathcal{S}_{1}\text{ vs }\mathcal{S}_{2}

S=0

), Searched Prior Dominance (

\text{H4}:\mathcal{S}_{3}\text{ vs }\mathcal{S}_{4}

), and direct engine matchups for neural search (

\mathcal{S}_{3}\text{ vs Engine}

) and co-evolved heuristic search (

\mathcal{S}_{4}\text{ vs Engine}

Partnership Matchup (North-South vs. East-West)	Lineage Scope	400 Sims Score% ( $\Delta$ )	800 Sims Score% ( $\Delta$ )	1600 Sims Score% ( $\Delta$ )
H2: $\mathcal{S}_{4}\text{(Search+Heur)}$ vs. $\mathcal{S}_{2}\text{(HeurPol)}$	All 5 Seeds	70.5% $\pm$ 7.0 ( $+1{,}540$ )	72.5% $\pm$ 5.3 ( $+1{,}341$ )	66.5% $\pm$ 5.9 ( $+1{,}210$ )
H1: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs. $\mathcal{S}_{1}\text{(PPO)}$	All 5 Seeds	50.0% $\pm$ 2.1 ( $+36$ )	50.7% $\pm$ 0.9 ( $+112$ )	53.4% $\pm$ 2.3 ( $+154$ )
H4: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs. $\mathcal{S}_{4}\text{(Search+Heur)}$	All 5 Seeds	38.8% $\pm$ 16.5 ( $-808$ )	36.3% $\pm$ 17.1 ( $-947$ )	37.1% $\pm$ 16.6 ( $-795$ )
$\mathcal{S}_{4}\text{(Search+Heur)}$ vs. $\mathcal{S}_{0}:\text{WBridge5}$ (Default)	All 5 Seeds	13.8% $\pm$ 4.8 ( $-1{,}982$ )	15.7% $\pm$ 6.6 ( $-1{,}930$ )	13.2% $\pm$ 4.8 ( $-2{,}076$ )
$\mathcal{S}_{3}\text{(Search+PPO)}$ vs. $\mathcal{S}_{0}:\text{WBridge5}$ (Default)	All 5 Seeds	5.0% $\pm$ 0.9 ( $-1{,}369$ )	5.6% $\pm$ 1.2 ( $-1{,}273$ )	5.2% $\pm$ 0.7 ( $-1{,}253$ )

Appendix

Table 30: Contract Bridge 2v2 Homogeneous Partnership Tournament Summary (Across All 5 Lineages). North-South (

\text{Seats }\{0,2\}

) and East-West (

\text{Seats }\{1,3\}

) are each populated by a homogeneous partnership of the same agent type. Scores report 5-seed seat-balanced Win/Score% and single-board duplicate contract point margins (

\Delta

) across simulation budgets

S\in\{400,800,1600\}

against WBridge5 evaluated under its default tournament setting.

Explore similar work

Feb 18, 2026

Discovering Multiagent Learning Algorithms with Large Language Models

Much of the advancement in Multi-Agent Reinforcement Learning (MARL) for imperfect-information games has historically depended on the manual, iterative refinement of algorithmic baselines. Recently, evolutionary coding agents powered by Large Language Models (LLMs) have emerged as powerful tools to automate this discovery process. In this work, we deploy one of such agentic frameworks, AlphaEvolve, to navigate the design spaces of two distinct game-theoretic paradigms: counterfactual regret minimization (CFR) and policy-space response oracles (PSRO). This automated search yielded two algorithms: Volatility-Adaptive Discounted (VAD-) CFR and Smoothed Hybrid Optimistic Regret (SHOR-) PSRO, which are consistently competitive with state-of-the-art human-designed baselines across an 18-game evaluation suite spanning Poker, Goofspiel, Liar's Dice, Blotto, and Battleship variants. However, because the LLM optimizes for fitness on a specific training set, it often constructs highly synergistic, complex mechanisms tailored to those environments. Through systematic ablation studies, we demonstrate that while these mechanisms are tightly coupled, the true driver of generalization lies in a minimal algorithmic core. By distilling the LLM's discoveries down to their most fundamental principles, we produce two minimal solvers: Warm-started Optimistic Predictive (WOP-)CFR and Projection Matching (PM-)PSRO. These distilled versions achieve superior performance on generalization with greatly reduced structural complexity, providing a clear methodology for using LLMs in algorithmic discovery.

Zun Li, John Schultz, Daniel Hennes +1

Google DeepMind

Multi-Agent Reinforcement Learning Regret Minimization Generalization in RL Large Language Model-Guided Optimization

Ask PDF

Feb 22, 2026

Reasoning Capabilities of Large Language Models. Lessons Learned from General Game Playing

This paper examines the reasoning capabilities of Large Language Models (LLMs) from a novel perspective, focusing on their ability to operate within formally specified, rule-governed environments. We evaluate four LLMs (Gemini 2.5 Pro and Flash variants, Llama 3.3 70B and GPT-OSS 120B) on a suite of forward-simulation tasks-including next / multistep state formulation, and legal action generation-across a diverse set of reasoning problems illustrated through General Game Playing (GGP) game instances. Beyond reporting instance-level performance, we characterize games based on 40 structural features and analyze correlations between these features and LLM performance. Furthermore, we investigate the effects of various game obfuscations to assess the role of linguistic semantics in game definitions and the impact of potential prior exposure of LLMs to specific games during training. The main results indicate that three of the evaluated models generally perform well across most experimental settings, with performance degradation observed as the evaluation horizon increases (i.e., with a higher number of game steps). Detailed case-based analysis of the LLM performance provides novel insights into common reasoning errors in the considered logic-based problem formulation, including hallucinated rules, redundant state facts, or syntactic errors. Overall, the paper reports clear progress in formal reasoning capabilities of contemporary models.

Maciej Świechowski, Adam Żychowski, Jacek Mańdziuk

Grail Team Warsaw, Poland · Warsaw University of Technology Warsaw, Poland · AGH University of Krakow Cracow, Poland

LLM Reasoning LLM Evaluation

Ask PDF

May 23, 2026

Distilling Game Code World Model Generation into Lightweight Large Language Models

Large Language Models (LLMs) have shown great ability in generating executable code from natural language, opening the possibility of automatically constructing environments for AI agents. Recent work on Code World Models (CWMs) demonstrates that LLMs can translate game rules into Python implementations compatible with solvers like Monte Carlo Tree Search. We study this problem in game settings, where generated environments must implement rules, legal actions, state transitions, observations, and rewards. We refer to these game-specific executable models as Game Code World Models (GameCWMs). However, current approaches to generating code world models rely on frontier models and inference-time refinement loops, limiting accessibility and scalability. This work investigates whether GameCWM generation capabilities can be distilled into smaller models through post-training. We introduce: (1) a curated dataset of 30 games spanning perfect and imperfect information games, (2) a verification framework that evaluates generated code against structural and semantic game properties, and (3) a post-training pipeline combining Supervised Fine-Tuning (SFT) with Reinforcement Learning with Verifiable Rewards (RLVR). We experiment with Qwen2.5-3B-Instruct and find that SFT can increase syntactic correctness, while RLVR can improve execution-level adherence to game rules, thereby improving Qwen's ability to generate valid GameCWMs in both perfect and imperfect information games. Overall, our pipeline makes Qwen2.5-3B-Instruct more capable of generating valid GameCWMs, thereby offering a scalable path toward automatic environment generation from natural language.

Tyrone Serapio, Arjun Prakash, Haoyang Xu +2

Brown University Providence, United States

LLM-Based Program Synthesis

Ask PDF