Modular Discovery of General Game-Playing Algorithms with Large Language Models

Zun Li; John Schultz; Marc Lanctot; Daniel Hennes

cs.AISep 27, 2026

Modular Discovery of General Game-Playing Algorithms with Large Language Models

Authors: Zun Li, John Schultz, Marc Lanctot, Daniel Hennes

Organizations: Google DeepMind · Work done while at Google DeepMind

Abstract

General Game Playing across arbitrary games from rules alone remains challenging due to differing algorithmic requirements across game classes and strict decision-time constraints. Rather than hand-designing search heuristics for specific domains, can we leverage Large Language Models (LLMs) to discover general game-playing algorithms? Because language models can propose and refactor structured code, they provide an expressive proposal engine for exploring the space of algorithmic designs. We introduce a multi-agent LLM meta-learning system to co-evolve game-agnostic procedural search mechanisms in C++ alongside domain heuristics synthesized directly from game rules. Controlling the compute budget, we benchmark the discovered mechanisms across more than 400 diverse environments, including OpenSpiel training and held-out games, procedural simulation engines, and games with deep neural policy-value representations trained via PPO. Evaluated via AlphaRank stationary distributions and Soft Condorcet Optimization (SCO) against 15 established MCTS baselines, the discovered search mechanisms consistently achieve top-tier ratings and pairwise ballot majorities over most baselines across independent evolutionary runs, generalizing to unseen human-designed and procedurally synthesized games and remaining competitive with baselines on frozen neural network representations.

Figures & tables

Panel of Figure 1 — Figure 1: Exploitability across simulation budgets (mean over independent seeds; lower is better). Dashed lines denote uniform random play; dash-dotted lines denote CFR (100 iterations). Numerical values and significance tests are in Appendix Table 4 .

Discovered Search Mechanism (Ours)
Algorithmic Paradigm / Baseline	Training Games (50)		Held-Out Games (19)		Synthesized Games (360)		Neural PPO Games (17)
	SCO Rating	$m^{\star}_{\mathrm{evo}}$ Win %	SCO Rating	$m^{\star}_{\mathrm{evo}}$ Win %	SCO Rating	$m^{\star}_{\mathrm{evo}}$ Win %	SCO Rating	$m^{\star}_{\mathrm{evo}}$ Win %
$m^{\star}_{\mathrm{evo}}$ (Multi-run aggregate)	$\mathbf{509.34\pm 0.39}$	—	$\mathbf{509.36\pm 0.48}$	—	$\mathbf{508.99\pm 0.20}$	—	$\mathbf{508.91\pm 0.67}$	—
$m^{\star}_{\mathrm{BIFV}}$ ( Bifv-Mcts )	$\mathbf{509.53}$	—	$\mathbf{509.93}$	—	$\mathbf{508.58}$	—	$\mathbf{510.31}$	—
Bandit Tree Search
Vanilla UCT ( Kocsis and Szepesvári, 2006 )	$501.22\pm 0.82$	$61.5\%\pm 4.2\%$	$503.84\pm 2.08$	$59.7\%\pm 5.6\%$	$504.19\pm 0.41$	$59.1\%\pm 0.7\%$	$500.77\pm 2.10$	$54.2\%\pm 4.9\%$

Table 1: Continuous SCO Ratings and Pairwise Ballot Win Rates across Four Benchmark Tiers under a 50 ms Budget (mean

\pm

SE across Seeds 1..5, and representative case study

m^{\star}_{\mathrm{BIFV}}

). For each tier, we report both the continuous Soft Condorcet Optimization rating ( SCO , scale

[0,1000]

) and the pairwise ballot win rate ( Win % ) denoting the percentage of games where

m^{\star}_{\mathrm{evo}}

outranks that baseline under AlphaRank dynamics.

Appendix figures & tables29 assets

Supplementary material from the paper’s appendix.

Appendix

Module Interface	Type Signature	Operational Role
Procedural Mechanism Modules (Game-Agnostic Quintet, $m\in\Omega_{\mathrm{mech}}$ )
TreePolicy	$\nu\mapsto a\in A(\nu)$	Balances exploitation vs. exploration during tree descent
ValueBackup	$(\nu,\vec{v},d,p)\mapsto\text{update}$	Propagates leaf payoff vectors along traversed trajectory
RootActionSelector	$(\nu_{\text{root}},t,B)\mapsto a$	Schedules root simulation budget across candidate actions
FinalActionSelector	$\nu_{\text{root}}\mapsto(a^{\ast},\hat{\pi})$	Selects final action and posterior policy distribution
VirtualLoss	$(\nu,d)\mapsto(\Delta Q,\Delta N)$	Coordinates asynchronous parallel search workers

Appendix

Table 2: The eight typed interfaces forming the modular simulation search template. Together, they constitute eight swappable computational degrees of freedom that span the design space of simulation search. Concrete C++ virtual method signatures and class declarations are provided in Section A.2 .

Game	$S_{\text{evolved}}$	$S_{\text{baseline}}$	$N_{\text{eval}}$	Curve	Game	$S_{\text{evolved}}$	$S_{\text{baseline}}$	$N_{\text{eval}}$	Curve
antichess	20	50	500	Mono	hearts	20	50	500	Flat
backgammon	40	100	500	Flat	hex	80	200	500	Flat
banqi	20	50	500	Mono	kriegspiel	40	100	32	Flat
bargaining	320	800	500	Mono	kuhn_poker	20	50	500	Flat
breakthrough	40	100	500	Flat	latent_ttt	40	100	500	Mono
bridge	20	50	500	Flat	leduc_poker	20	50	500	Flat

Appendix

Table 3: Complete calibrated simulation budgets and evaluation episode counts across the 50 extensive-form OpenSpiel training environments. Evolved candidates receive

S_{\text{evolved}}

simulations; reference PUCT opponents receive

S_{\text{baseline}}=2.5\times S_{\text{evolved}}

N_{\text{eval}}

denotes episodes played per evaluation round, calibrated to complete well within a 1-hour wall-clock window. Curve indicates whether the simulation response is strictly monotonic ( Mono ) or flat/saturated ( Flat ).

Configuration	Kuhn poker	Leduc poker	Liar’s dice
Uniform random	0.4583	2.3736	0.7807
Baseline ( $\langle m_{\text{ref}},\kappa_{\text{base}}\rangle$ )	$0.1698\pm 0.0001$	$1.5171\pm 0.0006$	$0.5700\pm 0.0004$
Mechanism ( $\langle m^{\star},\kappa_{\text{base}}\rangle$ )	$\mathbf{0.1237\pm 0.0085}$	$0.9521\pm 0.0213$	$0.5659\pm 0.0085$
PUCT ( $\langle m_{\text{ref}},\kappa^{\star}\rangle$ )	$0.1313\pm 0.0133$	$1.1781\pm 0.1657$	$0.5216\pm 0.0901$
Mechanism ( $m^{\star}+\pi^{\star}+v^{\star},b_{\text{base}}$ )	$0.1508\pm 0.0183$	$\mathbf{0.9074\pm 0.0593}$	$\mathbf{0.4748\pm 0.0383}$
Full system ( $\langle m^{\star},\kappa^{\star}\rangle$ )	$0.1294\pm 0.0197$	$1.2074\pm 0.1711$	$0.5703\pm 0.0937$

Appendix

Table 4: Exploitability

e

(Eq. 13 , lower is better) at

B=1000

, mean

\pm

standard error over independent evolution replications. Uniform-random and CFR-at-

100

-iterations are computed exactly and bracket the achievable range.

m^{\star}

denotes the discovered search mechanism,

m^{\star}+\pi^{\star}+v^{\star}

the evolved mechanism with priors and value estimator alone (with

b_{\text{base}}

), and

\kappa^{\star}=(\pi^{\star},v^{\star},b^{\star})

the complete domain knowledge triad including evolved belief resampler

b^{\star}

. Bold marks the best configuration in the table per game.

		Kuhn poker		Leduc poker		Liar’s dice
Module	$B$	$\Delta$	$p$	$\Delta$	$p$	$\Delta$	$p$
Mechanism ( $\langle m^{\star},\kappa_{\text{base}}\rangle$ )	20	$-0.109$	$<0.001$	$-0.336$	$<0.001$	$-0.043$	$0.013$
	100	$-0.081$	$<0.001$	$-0.465$	$<0.001$	$-0.061$	$<0.001$
	1000	$-0.046$	$0.006$	$-0.565$	$<0.001$	$-0.004$	$0.648$
Domain Knowledge ( $\langle m_{\text{ref}},\kappa^{\star}\rangle$ )	20	$-0.110$	$0.099$	$-0.697$	$0.119$	$-0.076$	$0.378$
	100	$-0.057$	$0.153$	$-0.516$	$0.189$	$-0.094$	$0.280$

Appendix

Table 5: Paired change in

e

relative to the baseline (negative favours the discovered module), with two-tailed paired

t

-test

p

-values over independent replications. The mechanism significantly reduces exploitability in Kuhn poker and Liar’s dice across lower budgets, and in Leduc poker across all budgets (

p<0.001

Game	Configuration	Mean $\pm$ SE	95% CI
Kuhn Poker	PUCT baseline ( $\langle m_{\text{ref}},\kappa_{\text{base}}\rangle$ )	0.2224 $\pm$ 0.0001	[0.2222, 0.2226]
	Evolved mechanism alone ( $\langle m^{\star},\kappa_{\text{base}}\rangle$ )	0.1412 $\pm$ 0.0081	[0.1253, 0.1571]
	PUCT + evolved domain knowledge ( $\langle m_{\text{ref}},\kappa^{\star}\rangle$ )	0.1652 $\pm$ 0.0323	[0.1019, 0.2285]
	Full co-evolved system ( $\langle m^{\star},\kappa^{\star}\rangle$ )	0.1280 $\pm$ 0.0136	[0.1013, 0.1547]
Leduc Poker	PUCT baseline ( $\langle m_{\text{ref}},\kappa_{\text{base}}\rangle$ )	1.8700 $\pm$ 0.0007	[1.8686, 1.8714]
	Evolved mechanism alone ( $\langle m^{\star},\kappa_{\text{base}}\rangle$ )	1.4051 $\pm$ 0.0264	[1.3534, 1.4568]

Appendix

Table 6: Cross-seed exploitability distribution at budget

B=100

simulations across independent seeds (mean

\pm

standard error and 95% confidence intervals). Pure procedural search (

m^{\star}

) exhibits consistent performance with low variance across games, whereas evolved domain heuristics exhibit higher dispersion on Leduc Poker.

Knowledge	Belief	Sims	Evolved Mech	PUCT Mech	Within-Row Mech Contrast	Contrast vs. Baseline
		$B$	(Mean $\pm$ SE)	(Mean $\pm$ SE)	$\Delta_{\text{mech}}$ (%, $p$ )	$\Delta_{\text{base}}$ (%, $p$ )
Seed	Native	20	0.1879 $\pm$ 0.0053	0.2966 $\pm$ 0.0003	-36.6% ( $p<0.001^{***}$ )	-36.6% ( $p<0.001^{***}$ )
Seed	Native	50	0.1595 $\pm$ 0.0071	0.2450 $\pm$ 0.0001	-34.9% ( $p<0.001^{***}$ )	-34.9% ( $p<0.001^{***}$ )
Seed	Native	100	0.1412 $\pm$ 0.0081	0.2224 $\pm$ 0.0001	-36.5% ( $p<0.001^{***}$ )	-36.5% ( $p<0.001^{***}$ )
Seed	Native	500	0.1177 $\pm$ 0.0085	0.1873 $\pm$ 0.0002	-37.1% ( $p=0.001^{**}$ )	-37.1% ( $p=0.001^{**}$ )
Seed	Native	1000	0.1237 $\pm$ 0.0085	0.1698 $\pm$ 0.0001	-27.1% ( $p=0.006^{**}$ )	-27.1% ( $p=0.006^{**}$ )

Appendix

Table 7: Complete exploitability breakdown for Kuhn Poker across all simulation budgets (

B\in\{20,50,100,500,1000\}

). Random floor anchor: 0.4583; CFR@100: 0.0082. Within-Row Mech Contrast reports paired difference (Evolved vs. PUCT under identical knowledge and belief). Contrast vs. Baseline reports paired difference relative to puct+seed+native .

Knowledge	Belief	Sims	Evolved Mech	PUCT Mech	Within-Row Mech Contrast	Contrast vs. Baseline
		$B$	(Mean $\pm$ SE)	(Mean $\pm$ SE)	$\Delta_{\text{mech}}$ (%, $p$ )	$\Delta_{\text{base}}$ (%, $p$ )
Seed	Native	20	1.7759 $\pm$ 0.0067	2.1116 $\pm$ 0.0017	-15.9% ( $p<0.001^{***}$ )	-15.9% ( $p<0.001^{***}$ )
Seed	Native	50	1.5908 $\pm$ 0.0128	1.9799 $\pm$ 0.0014	-19.7% ( $p<0.001^{***}$ )	-19.7% ( $p<0.001^{***}$ )
Seed	Native	100	1.4051 $\pm$ 0.0264	1.8700 $\pm$ 0.0007	-24.9% ( $p<0.001^{***}$ )	-24.9% ( $p<0.001^{***}$ )
Seed	Native	500	1.0711 $\pm$ 0.0263	1.6381 $\pm$ 0.0006	-34.6% ( $p<0.001^{***}$ )	-34.6% ( $p<0.001^{***}$ )
Seed	Native	1000	0.9521 $\pm$ 0.0213	1.5171 $\pm$ 0.0006	-37.2% ( $p<0.001^{***}$ )	-37.2% ( $p<0.001^{***}$ )

Appendix

Table 8: Complete exploitability breakdown for Leduc Poker across all simulation budgets (

B\in\{20,50,100,500,1000\}

). Random floor anchor: 2.3736; CFR@100: 0.0957. Within-Row Mech Contrast reports paired difference (Evolved vs. PUCT under identical knowledge and belief). Contrast vs. Baseline reports paired difference relative to puct+seed+native .

Knowledge	Belief	Sims	Evolved Mech	PUCT Mech	Within-Row Mech Contrast	Contrast vs. Baseline
		$B$	(Mean $\pm$ SE)	(Mean $\pm$ SE)	$\Delta_{\text{mech}}$ (%, $p$ )	$\Delta_{\text{base}}$ (%, $p$ )
Seed	Native	20	0.6854 $\pm$ 0.0105	0.7285 $\pm$ 0.0009	-5.9% ( $p=0.013^{*}$ )	-5.9% ( $p=0.013^{*}$ )
Seed	Native	50	0.6469 $\pm$ 0.0052	0.7165 $\pm$ 0.0010	-9.7% ( $p<0.001^{***}$ )	-9.7% ( $p<0.001^{***}$ )
Seed	Native	100	0.6446 $\pm$ 0.0069	0.7054 $\pm$ 0.0009	-8.6% ( $p<0.001^{***}$ )	-8.6% ( $p<0.001^{***}$ )
Seed	Native	500	0.6041 $\pm$ 0.0060	0.6372 $\pm$ 0.0011	-5.2% ( $p=0.004^{**}$ )	-5.2% ( $p=0.004^{**}$ )
Seed	Native	1000	0.5659 $\pm$ 0.0085	0.5700 $\pm$ 0.0004	-0.7% ( $p=0.648$ )	-0.7% ( $p=0.648$ )

Appendix

Table 9: Complete exploitability breakdown for Liar’s Dice across all simulation budgets (

B\in\{20,50,100,500,1000\}

). Random floor anchor: 0.7807; CFR@100: 0.0224. Within-Row Mech Contrast reports paired difference (Evolved vs. PUCT under identical knowledge and belief). Contrast vs. Baseline reports paired difference relative to puct+seed+native .

Baseline	Algorithmic Paradigm	Hyperparameter	Search Grid	Selected Value ( $p^{\star}$ )
Vanilla UCT	Bandit Tree Search	Exploration constant $c$	$\{0.25,0.5,1.0,2.0,4.0\}$	$c=0.25$
PUCT	Bandit Tree Search	Predictor weight $c_{\mathrm{init}}$	$\{0.31,0.63,1.25,2.5,5.0\}$	$c_{\mathrm{init}}=5.0$
UCB-Tuned	Bandit Tree Search	Variance scale $c^{\prime}$	$\{0.25,0.5,1.0,2.0,4.0\}$	$c^{\prime}=0.5$
Gumbel AlphaZero	Entropy Planning	Improvement scale $c_{\mathrm{scale}}$	$\{0.025,0.05,0.1,0.2,0.4\}$	$c_{\mathrm{scale}}=0.4$
MENTS	Entropy Planning	Temperature $\tau\times$ Exploration $\epsilon$	$\{0.25,\dots,4.0\}\times\{0.025,\dots,0.4\}$	$\tau=0.25,\epsilon=0.05$
RPO	Entropy Planning	Regularization temperature $\tau$	$\{0.25,0.5,1.0,2.0,4.0\}$	$\tau=0.25$

Appendix

Table 10: Hyperparameter tuning search spaces and final selected values (

p^{\star}

) across the 15 MCTS baselines (including canonical reference PUCT). Optimal parameters were identified via offline grid search across the training suite and selected via 95% bootstrap Lower Confidence Bounds (LCB) over paired matches against reference defaults.

Category	Game	Type	Players	Horizon / Complexity
Perfect Info Deterministic	Amazons	Board territory	2	Deep, high branching
	Quoridor	Path maze	2	Medium, topological
	Twixt	Connection	2	Medium, virtual links
	Oware	Mancala variant	2	Medium, pit counting
	Ultimate Tic-Tac-Toe	Nested grid	2	High tactical branching
	Nine Men’s Morris	Mill alignment	2	Moderate, phase shifts

Appendix

Table 11: Taxonomy and structural properties of the 19 held-out evaluation games.

Structural Dimension	Category / Profile	Count ( $N=360$ )	Percentage
Information Structure	Imperfect Information (hidden hands, fog-of-war, private cards)	166	46.1%
	Perfect Information Deterministic	133	36.9%
	Perfect Information Stochastic (dice rolls, chance transitions)	61	16.9%
Player Count	2-Player	317	88.1%
	4-Player	24	6.7%
	3-Player	7	1.9%

Appendix

Table 12: Taxonomy and empirical distribution of the 360 procedurally synthesized evaluation environments derived from natural language Wikipedia rule specifications.

Independent Evolution Run	Synthesized Games	Held-Out Games	Training Games	Neural PPO Games
Bifv-Mcts ( $m^{\star}_{\mathrm{BIFV}}$ , Case Study)	508.58 (Rank 1)	509.93 (Rank 1)	509.53 (Rank 1)	510.31 (Rank 1)
Seed 4	508.52 (Rank 1)	510.15 (Rank 1)	509.92 (Rank 1)	508.05 (Rank 1)
Seed 1	508.96 (Rank 1)	509.63 (Rank 1)	507.83 (Rank 2)	508.30 (Rank 2)
Seed 5	509.45 (Rank 1)	507.50 (Rank 2)	509.93 (Rank 1)	510.66 (Rank 1)
Seed 2	509.45 (Rank 1)	509.59 (Rank 1)	509.48 (Rank 1)	507.25 (Rank 2)
Aggregate Mean $\pm$ SE	$\mathbf{508.99\pm 0.20}$	$\mathbf{509.36\pm 0.48}$	$\mathbf{509.34\pm 0.39}$	$\mathbf{508.91\pm 0.67}$

Appendix

Table 13: Granular Soft Condorcet Optimization (SCO) Ratings across Independent Evolution Seeds under a Fixed 50 ms Search Budget (IsoTime). Ratings are continuous latent SCO values (

r\in[0,1000]

) derived from AlphaRank stationary preference profiles ( use_inf_alpha=True ) across

N

-player empirical payoff tensors comparing

m^{\star}_{\mathrm{evo}}

against 15 competitive baselines. Parenthesized values denote the rank achieved by each evolved mechanism within its own seed evaluation tournament under AlphaRank+SCO. Aggregate performance across runs is reported as continuous SCO rating (

\text{Mean}\pm\text{SE}

Independent Evolution Run	Synthesized Games	Held-Out Games	Training Games	Neural PPO Games
Bifv-Mcts ( $m^{\star}_{\mathrm{BIFV}}$ , Case Study)	508.99 (Rank 1)	510.57 (Rank 1)	510.40 (Rank 1)	510.01 (Rank 1)
Seed 5	508.93 (Rank 1)	510.91 (Rank 1)	508.61 (Rank 1)	509.20 (Rank 1)
Seed 1	508.84 (Rank 1)	508.05 (Rank 2)	510.18 (Rank 1)	505.83 (Rank 3)
Seed 4	507.44 (Rank 1)	507.33 (Rank 1)	508.06 (Rank 2)	506.31 (Rank 3)
Seed 2	509.33 (Rank 1)	501.80 (Rank 7)	507.25 (Rank 2)	502.29 (Rank 6)
Aggregate Mean $\pm$ SE	$\mathbf{508.71\pm 0.33}$	$\mathbf{507.73\pm 1.64}$	$\mathbf{508.90\pm 0.61}$	$\mathbf{506.73\pm 1.37}$

Appendix

Table 14: Granular Soft Condorcet Optimization (SCO) Ratings across Independent Evolution Seeds under a Fixed-Simulation Budget (IsoSim, 1,000 Simulations). Ratings are continuous latent SCO values (

r\in[0,1000]

) derived from AlphaRank stationary preference profiles ( use_inf_alpha=True ) across

N

-player empirical payoff tensors comparing

m^{\star}_{\mathrm{evo}}

against 15 competitive baselines. Parenthesized values denote the rank achieved by each evolved mechanism within its own seed evaluation tournament under AlphaRank+SCO.

Discovered Search Mechanism (Ours)
Algorithmic Paradigm / Baseline	Training Games (50)		Held-Out Games (19)		Synthesized Games (360)		Neural PPO Games (17)
	SCO Rating	$m^{\star}_{\mathrm{evo}}$ Win %	SCO Rating	$m^{\star}_{\mathrm{evo}}$ Win %	SCO Rating	$m^{\star}_{\mathrm{evo}}$ Win %	SCO Rating	$m^{\star}_{\mathrm{evo}}$ Win %
$m^{\star}_{\mathrm{evo}}$ (Multi-run aggregate)	$\mathbf{508.90\pm 0.61}$	—	$\mathbf{507.73\pm 1.64}$	—	$\mathbf{508.71\pm 0.33}$	—	$\mathbf{506.73\pm 1.37}$	—
$m^{\star}_{\mathrm{BIFV}}$ ( Bifv-Mcts )	$\mathbf{510.40}$	—	$\mathbf{510.57}$	—	$\mathbf{508.99}$	—	$\mathbf{510.01}$	—
Bandit Tree Search
Vanilla UCT ( Kocsis and Szepesvári, 2006 )	$504.26\pm 0.67$	$60.3\%\pm 3.9\%$	$501.98\pm 2.07$	$53.5\%\pm 4.9\%$	$504.98\pm 0.32$	$57.1\%\pm 1.6\%$	$500.29\pm 1.85$	$56.2\%\pm 5.4\%$

Appendix

Table 15: Continuous SCO Ratings and Pairwise Ballot Win Rates across Four Benchmark Tiers under Fixed Simulation Budgets (IsoSim, 1,000 Simulations) (mean

\pm

SE across Seeds 1..5, and representative case study

m^{\star}_{\mathrm{BIFV}}

). For each tier, we report both the continuous Soft Condorcet Optimization rating ( SCO , scale

[0,1000]

) and the pairwise ballot win rate ( Win % ) denoting the percentage of games where

m^{\star}_{\mathrm{evo}}

outranks that baseline under AlphaRank dynamics (“—” denotes self-comparison).

	Training (50)	Held-out (19)	Synthesized (360)	Neural PPO (17)
Full Bifv-Mcts ( $m^{\star}_{\mathrm{BIFV}}$ )	71.0	66.8	64.8	78.0
$\setminus\mathbf{B}$ (branching)	$-8.5$	$-0.4$	$+5.3$	$-5.3$
$\setminus\mathbf{V}$ (tie-break)	$+9.0$	$-5.7$	$+4.9$	$-2.4$
$\setminus\mathbf{F}$ (FPU anneal)	$+9.2$	$+13.9$	$+5.9$	$-5.8$
$\setminus\mathbf{I}$ (info scale)	$+10.5$	$+10.6$	$+5.0$	$-1.0$

Appendix

Table 16: Operator ablation of Bifv-Mcts (50 ms IsoTime). Average pairwise ballot win rate (%) against the same 15 baselines; since the baseline pool is identical, rows are directly comparable. First row: full mechanism. Other rows: change when one operator is removed (positive = removal helps, negative = removal hurts).

Algorithm / Run	Exploration Schedule $c(N)$	Leaf Evaluation Policy ( $Q_{\mathrm{FPU}}$ )	Value Backup Operator	Key Algorithmic Trait
Human-Engineered Prior Art
Evolution Seed Template ( $m_{\mathrm{seed}}$ )	Static scalar $c_{\mathrm{puct}}=2.0$	Naive immediate expansion ( $v=0$ )	Arithmetic mean $\frac{1}{N}\sum R$	Minimal baseline template (evolution starting seed)
AlphaZero ( Silver et al., 2018 )	$1.25+\ln\frac{N+19652}{19652}$	Parent return $Q_{\mathrm{parent}}$	Arithmetic mean $\frac{1}{N}\sum R$	Static single-game parameter schedule
KataGo ( Wu, 2019 )	Dynamic visit-dependent schedule	Utility-bounded normalizer	Arithmetic mean $\frac{1}{N}\sum R$	Utility range clamping $[u_{\min},u_{\max}]$
Leela Chess Zero (Lc0)	Tuned logarithmic schedule	$Q_{\mathrm{parent}}-c_{\mathrm{fpu}}\sqrt{1-\sum\pi}$	Arithmetic mean $\frac{1}{N}\sum R$	Static prior-weighted FPU reduction
Discovered Programs from Evolutionary Runs (Ours)

Appendix

Table 17: Comprehensive Taxonomic Comparison of Discovered Search Programs Against Prior Art. Contrasts mathematical operators across exploration schedules, leaf evaluation (FPU), value backups, and action selection.

Design Dimension	Vector / Tabular Architecture (MLP)	Spatial / Board Architecture (CNNResNet)
Target Game Families	Card, trick-taking, imperfect-information, coordination	2D grid board games, connection, spatial territory
Evaluated Games (17 Total)	Contract Bridge, Hearts, Gin Rummy, Leduc Poker,	Chess, Shogi, Go 9 $\times$ 9, Go 13 $\times$ 13,
	Backgammon, Bargaining, Coop Box, Phantom TTT, Dark Hex	Hex, Othello, Checkers, Gomoku
Input Representation	Flat 1D feature vector $\mathbf{x}\in\mathbb{R}^{D}$	3D tensor $\mathbf{X}\in\mathbb{R}^{H\times W\times C}$ (board planes)
Backbone Structure	2 Fully Connected layers (Dense-256)	1 Initial Conv ( $3\times 3$ , 64 ch) + 4 ResNet Blocks
Residual Block Design	—	Two Conv- $3\times 3$ (64 ch) + skip connection

Appendix

Table 18: Neural Network Architectures Tailored by Game Topology. Model hyperparameters and structural specifications across the two architecture families used in the PPO benchmark.

Game (Engine)	Architecture Pairing	100 Sims	200 Sims	400 Sims	800 Sims	1600 Sims
1. Chess ( Stockfish )	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs Engine	6.7% $\pm$ 1.8	6.3% $\pm$ 1.7	6.1% $\pm$ 2.2	5.3% $\pm$ 2.3	4.7% $\pm$ 2.1
	$\mathcal{S}_{3}\text{(Search+PPO)}$ vs Engine	0.0% $\pm$ 0.0	0.2% $\pm$ 0.1	0.1% $\pm$ 0.1	0.0% $\pm$ 0.0	0.0% $\pm$ 0.0
	H1: $\mathcal{S}_{3}$ vs $\mathcal{S}_{1}$ / H2: $\mathcal{S}_{4}$ vs $\mathcal{S}_{2}$	50.0% / 56.5%	50.0% / 57.2%	50.0% / 62.2%	50.0% / 55.0%	50.0% / 60.5%
	H3: $\mathcal{S}_{1}$ vs $\mathcal{S}_{2}$ ( $S=0$ ) / H4: $\mathcal{S}_{3}$ vs $\mathcal{S}_{4}$	0.8% / 15.3%	0.8% / 9.2%	0.8% / 15.9%	0.8% / 18.4%	0.8% / 18.8%
2. Checkers ( Kingsrow )	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs Engine	0.0% $\pm$ 0.0	0.4% $\pm$ 0.1	0.5% $\pm$ 0.2	0.7% $\pm$ 0.4	1.1% $\pm$ 0.2
	$\mathcal{S}_{3}\text{(Search+PPO)}$ vs Engine	0.1% $\pm$ 0.1	0.2% $\pm$ 0.1	0.2% $\pm$ 0.0	0.3% $\pm$ 0.1	0.5% $\pm$ 0.2

Appendix

Table 19: Cross-Game Summary of Five-Strategy Empirical Game Tensors Across All 10 Benchmark Games (

S=100\to 1600

, 5-Seed Mean

\pm

SE). Rows evaluate both neural search (

\mathcal{S}_{3}

) and co-evolved heuristic search (

\mathcal{S}_{4}

) directly against the external domain engine (

\mathcal{S}_{0}

, Floor tier; default setting for WBridge5), followed by the four core internal hypothesis pairings ( H1 – H4 ; H3 is an unsearched

S=0

baseline). Full 3-tier ( Floor , Mid , Ceiling ) and per-seed breakdowns appear in Tables 20 – 29 .

Lineage / Aggregation	Empirical Tensor Entry	100 Sims	200 Sims	400 Sims	800 Sims	1600 Sims
Seed 1	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs Stockfish (Floor)	5.0%	5.0%	2.5%	3.2%	2.5%
	$\mathcal{S}_{3}\text{(Search+PPO)}$ vs Stockfish (Floor)	0.0%	0.5%	0.0%	0.0%	0.0%
	H1: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs $\mathcal{S}_{1}\text{(PPO)}$	50.0%	50.0%	50.0%	50.0%	50.0%
	H2: $\mathcal{S}_{4}\text{(Search+Heur)}$ vs $\mathcal{S}_{2}\text{(Heur)}$	68.1%	67.6%	63.7%	33.2%	39.3%
	H4: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs $\mathcal{S}_{4}\text{(Search+Heur)}$	7.5%	11.0%	12.9%	19.5%	18.5%
Seed 2	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs Stockfish (Floor)	7.3%	7.2%	6.2%	1.8%	1.0%

Appendix

Table 20: Per-Seed and 5-Seed Aggregated Empirical Game Tensor for Chess (vs. Stockfish). Entries report seat-balanced Score% (

\text{Win}\%+0.5\times\text{Draw}\%

; parentheticals show net score margin

\Delta

where applicable) across simulation budgets

S\in\{100,200,400,800,1600\}

. Rows evaluate Search Amplification of PPO (

\text{H1}:\mathcal{S}_{3}\text{ vs }\mathcal{S}_{1}

), Search Amplification of Heuristic (

\text{H2}:\mathcal{S}_{4}\text{ vs }\mathcal{S}_{2}

), Unsearched Prior Strength (

\text{H3}:\mathcal{S}_{1}\text{ vs }\mathcal{S}_{2}

S=0

), Searched Prior Dominance (

\text{H4}:\mathcal{S}_{3}\text{ vs }\mathcal{S}_{4}

), and direct engine matchups for neural search (

\mathcal{S}_{3}\text{ vs Engine}

) and co-evolved heuristic search (

\mathcal{S}_{4}\text{ vs Engine}

Lineage / Aggregation	Empirical Tensor Entry	100 Sims	200 Sims	400 Sims	800 Sims	1600 Sims
Seed 1	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs Kingsrow (Floor)	0.0%	0.0%	0.5%	1.0%	1.7%
	$\mathcal{S}_{3}\text{(Search+PPO)}$ vs Kingsrow (Floor)	0.0%	0.5%	0.2%	0.8%	1.0%
	H1: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs $\mathcal{S}_{1}\text{(PPO)}$	50.0%	50.0%	50.0%	71.2%	75.0%
	H2: $\mathcal{S}_{4}\text{(Search+Heur)}$ vs $\mathcal{S}_{2}\text{(Heur)}$	98.5%	99.8%	100.0%	98.2%	100.0%
	H4: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs $\mathcal{S}_{4}\text{(Search+Heur)}$	18.6%	48.0%	49.3%	48.8%	46.6%
Seed 2	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs Kingsrow (Floor)	0.0%	0.0%	0.2%	0.2%	0.5%

Appendix

Table 21: Per-Seed and 5-Seed Aggregated Empirical Game Tensor for Checkers (vs. Kingsrow). Entries report seat-balanced Score% (

\text{Win}\%+0.5\times\text{Draw}\%

; parentheticals show net score margin

\Delta

where applicable) across simulation budgets

S\in\{100,200,400,800,1600\}

. Rows evaluate Search Amplification of PPO (

\text{H1}:\mathcal{S}_{3}\text{ vs }\mathcal{S}_{1}

), Search Amplification of Heuristic (

\text{H2}:\mathcal{S}_{4}\text{ vs }\mathcal{S}_{2}

), Unsearched Prior Strength (

\text{H3}:\mathcal{S}_{1}\text{ vs }\mathcal{S}_{2}

S=0

), Searched Prior Dominance (

\text{H4}:\mathcal{S}_{3}\text{ vs }\mathcal{S}_{4}

), and direct engine matchups for neural search (

\mathcal{S}_{3}\text{ vs Engine}

) and co-evolved heuristic search (

\mathcal{S}_{4}\text{ vs Engine}

Lineage / Aggregation	Empirical Tensor Entry	100 Sims	200 Sims	400 Sims	800 Sims	1600 Sims
Seed 1	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs Egaroucid (Floor)	65.2%	31.5%	40.0%	44.4%	35.8%
	$\mathcal{S}_{3}\text{(Search+PPO)}$ vs Egaroucid (Floor)	0.0%	0.5%	0.0%	3.4%	0.0%
	H1: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs $\mathcal{S}_{1}\text{(PPO)}$	46.8%	82.0%	89.1%	99.8%	100.0%
	H2: $\mathcal{S}_{4}\text{(Search+Heur)}$ vs $\mathcal{S}_{2}\text{(Heur)}$	89.0%	98.9%	100.0%	100.0%	100.0%
	H4: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs $\mathcal{S}_{4}\text{(Search+Heur)}$	0.0%	0.0%	1.0%	0.0%	0.0%
Seed 2	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs Egaroucid (Floor)	31.5%	74.8%	58.4%	67.2%	86.1%

Appendix

Table 22: Per-Seed and 5-Seed Aggregated Empirical Game Tensor for Othello (vs. Egaroucid). Entries report seat-balanced Score% (

\text{Win}\%+0.5\times\text{Draw}\%

; parentheticals show net score margin

\Delta

where applicable) across simulation budgets

S\in\{100,200,400,800,1600\}

. Rows evaluate Search Amplification of PPO (

\text{H1}:\mathcal{S}_{3}\text{ vs }\mathcal{S}_{1}

), Search Amplification of Heuristic (

\text{H2}:\mathcal{S}_{4}\text{ vs }\mathcal{S}_{2}

), Unsearched Prior Strength (

\text{H3}:\mathcal{S}_{1}\text{ vs }\mathcal{S}_{2}

S=0

), Searched Prior Dominance (

\text{H4}:\mathcal{S}_{3}\text{ vs }\mathcal{S}_{4}

), and direct engine matchups for neural search (

\mathcal{S}_{3}\text{ vs Engine}

) and co-evolved heuristic search (

\mathcal{S}_{4}\text{ vs Engine}

Lineage / Aggregation	Empirical Tensor Entry	100 Sims	200 Sims	400 Sims	800 Sims	1600 Sims
Seed 1	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs Engine (Floor)	7.5%	1.5%	0.0%	3.5%	10.0%
	$\mathcal{S}_{3}\text{(Search+PPO)}$ vs Engine (Floor)	0.0%	0.0%	0.0%	0.0%	0.0%
	H1: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs $\mathcal{S}_{1}\text{(PPO)}$	91.3%	100.0%	100.0%	99.5%	99.5%
	H2: $\mathcal{S}_{4}\text{(Search+Heur)}$ vs $\mathcal{S}_{2}\text{(Heur)}$	94.0%	99.5%	98.0%	99.2%	100.0%
	H4: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs $\mathcal{S}_{4}\text{(Search+Heur)}$	0.0%	0.0%	0.0%	0.0%	0.0%
Seed 2	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs Engine (Floor)	0.0%	0.5%	0.0%	0.5%	0.0%

Appendix

Table 23: Per-Seed and 5-Seed Aggregated Empirical Game Tensor for Gomoku (vs. Gomoku Engine). Entries report seat-balanced Score% (

\text{Win}\%+0.5\times\text{Draw}\%

; parentheticals show net score margin

\Delta

where applicable) across simulation budgets

S\in\{100,200,400,800,1600\}

. Rows evaluate Search Amplification of PPO (

\text{H1}:\mathcal{S}_{3}\text{ vs }\mathcal{S}_{1}

), Search Amplification of Heuristic (

\text{H2}:\mathcal{S}_{4}\text{ vs }\mathcal{S}_{2}

), Unsearched Prior Strength (

\text{H3}:\mathcal{S}_{1}\text{ vs }\mathcal{S}_{2}

S=0

), Searched Prior Dominance (

\text{H4}:\mathcal{S}_{3}\text{ vs }\mathcal{S}_{4}

), and direct engine matchups for neural search (

\mathcal{S}_{3}\text{ vs Engine}

) and co-evolved heuristic search (

\mathcal{S}_{4}\text{ vs Engine}

Lineage / Aggregation	Empirical Tensor Entry	100 Sims	200 Sims	400 Sims	800 Sims	1600 Sims
Seed 1	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs YaneuraOu (Floor)	0.0%	0.0%	0.0%	0.0%	0.5%
	$\mathcal{S}_{3}\text{(Search+PPO)}$ vs YaneuraOu (Floor)	0.0%	0.0%	0.0%	0.0%	0.0%
	H1: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs $\mathcal{S}_{1}\text{(PPO)}$	79.2%	94.5%	95.0%	99.0%	99.5%
	H2: $\mathcal{S}_{4}\text{(Search+Heur)}$ vs $\mathcal{S}_{2}\text{(Heur)}$	97.2%	99.0%	100.0%	99.5%	100.0%
	H4: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs $\mathcal{S}_{4}\text{(Search+Heur)}$	0.0%	0.0%	0.0%	0.0%	0.0%
Seed 2	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs YaneuraOu (Floor)	0.0%	0.0%	0.0%	0.0%	0.0%

Appendix

Table 24: Per-Seed and 5-Seed Aggregated Empirical Game Tensor for Shogi (vs. YaneuraOu). Entries report seat-balanced Score% (

\text{Win}\%+0.5\times\text{Draw}\%

; parentheticals show net score margin

\Delta

where applicable) across simulation budgets

S\in\{100,200,400,800,1600\}

. Rows evaluate Search Amplification of PPO (

\text{H1}:\mathcal{S}_{3}\text{ vs }\mathcal{S}_{1}

), Search Amplification of Heuristic (

\text{H2}:\mathcal{S}_{4}\text{ vs }\mathcal{S}_{2}

), Unsearched Prior Strength (

\text{H3}:\mathcal{S}_{1}\text{ vs }\mathcal{S}_{2}

S=0

), Searched Prior Dominance (

\text{H4}:\mathcal{S}_{3}\text{ vs }\mathcal{S}_{4}

), and direct engine matchups for neural search (

\mathcal{S}_{3}\text{ vs Engine}

) and co-evolved heuristic search (

\mathcal{S}_{4}\text{ vs Engine}

Lineage / Aggregation	Empirical Tensor Entry	100 Sims	200 Sims	400 Sims	800 Sims	1600 Sims
Seed 1	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs KataHex (Floor)	0.0%	0.0%	0.0%	0.0%	0.0%
	$\mathcal{S}_{3}\text{(Search+PPO)}$ vs KataHex (Floor)	0.0%	0.0%	0.0%	0.0%	0.0%
	H1: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs $\mathcal{S}_{1}\text{(PPO)}$	91.0%	96.5%	98.0%	99.0%	97.5%
	H2: $\mathcal{S}_{4}\text{(Search+Heur)}$ vs $\mathcal{S}_{2}\text{(Heur)}$	11.5%	25.0%	37.5%	59.0%	100.0%
	H4: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs $\mathcal{S}_{4}\text{(Search+Heur)}$	0.0%	0.0%	0.0%	0.0%	0.0%
Seed 2	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs KataHex (Floor)	0.0%	0.0%	0.0%	0.0%	0.0%

Appendix

Table 25: Per-Seed and 5-Seed Aggregated Empirical Game Tensor for Hex 13x13 (vs. KataHex). Entries report seat-balanced Score% (

\text{Win}\%+0.5\times\text{Draw}\%

; parentheticals show net score margin

\Delta

where applicable) across simulation budgets

S\in\{100,200,400,800,1600\}

. Rows evaluate Search Amplification of PPO (

\text{H1}:\mathcal{S}_{3}\text{ vs }\mathcal{S}_{1}

), Search Amplification of Heuristic (

\text{H2}:\mathcal{S}_{4}\text{ vs }\mathcal{S}_{2}

), Unsearched Prior Strength (

\text{H3}:\mathcal{S}_{1}\text{ vs }\mathcal{S}_{2}

S=0

), Searched Prior Dominance (

\text{H4}:\mathcal{S}_{3}\text{ vs }\mathcal{S}_{4}

), and direct engine matchups for neural search (

\mathcal{S}_{3}\text{ vs Engine}

) and co-evolved heuristic search (

\mathcal{S}_{4}\text{ vs Engine}

Lineage / Aggregation	Empirical Tensor Entry	100 Sims	200 Sims	400 Sims	800 Sims	1600 Sims
Seed 1	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs KataGo (Floor)	1.8%	8.5%	12.9%	15.5%	14.9%
	$\mathcal{S}_{3}\text{(Search+PPO)}$ vs KataGo (Floor)	0.2%	0.5%	0.0%	0.0%	0.2%
	H1: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs $\mathcal{S}_{1}\text{(PPO)}$	79.8%	86.2%	60.5%	69.8%	77.7%
	H2: $\mathcal{S}_{4}\text{(Search+Heur)}$ vs $\mathcal{S}_{2}\text{(Heur)}$	98.0%	94.0%	98.5%	98.0%	97.5%
	H4: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs $\mathcal{S}_{4}\text{(Search+Heur)}$	1.7%	1.2%	0.5%	1.0%	0.5%
Seed 2	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs KataGo (Floor)	1.0%	1.0%	4.0%	14.5%	15.5%

Appendix

Table 26: Per-Seed and 5-Seed Aggregated Empirical Game Tensor for Go 9x9 (vs. KataGo). Entries report seat-balanced Score% (

\text{Win}\%+0.5\times\text{Draw}\%

; parentheticals show net score margin

\Delta

where applicable) across simulation budgets

S\in\{100,200,400,800,1600\}

. Rows evaluate Search Amplification of PPO (

\text{H1}:\mathcal{S}_{3}\text{ vs }\mathcal{S}_{1}

), Search Amplification of Heuristic (

\text{H2}:\mathcal{S}_{4}\text{ vs }\mathcal{S}_{2}

), Unsearched Prior Strength (

\text{H3}:\mathcal{S}_{1}\text{ vs }\mathcal{S}_{2}

S=0

), Searched Prior Dominance (

\text{H4}:\mathcal{S}_{3}\text{ vs }\mathcal{S}_{4}

), and direct engine matchups for neural search (

\mathcal{S}_{3}\text{ vs Engine}

) and co-evolved heuristic search (

\mathcal{S}_{4}\text{ vs Engine}

Lineage / Aggregation	Empirical Tensor Entry	100 Sims	200 Sims	400 Sims	800 Sims	1600 Sims
Seed 1	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs KataGo (Floor)	0.0%	0.3%	1.2%	2.3%	5.8%
	$\mathcal{S}_{3}\text{(Search+PPO)}$ vs KataGo (Floor)	1.3%	1.1%	0.6%	0.9%	1.8%
	H1: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs $\mathcal{S}_{1}\text{(PPO)}$	72.6%	70.1%	59.9%	66.1%	61.5%
	H2: $\mathcal{S}_{4}\text{(Search+Heur)}$ vs $\mathcal{S}_{2}\text{(Heur)}$	94.0%	96.5%	92.8%	98.5%	98.5%
	H4: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs $\mathcal{S}_{4}\text{(Search+Heur)}$	0.0%	0.0%	0.0%	0.0%	0.0%
Seed 2	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs KataGo (Floor)	0.0%	0.0%	0.2%	0.3%	2.7%

Appendix

Table 27: Per-Seed and 5-Seed Aggregated Empirical Game Tensor for Go 13x13 (vs. KataGo 13x13). Entries report seat-balanced Score% (

\text{Win}\%+0.5\times\text{Draw}\%

; parentheticals show net score margin

\Delta

where applicable) across simulation budgets

S\in\{100,200,400,800,1600\}

. Rows evaluate Search Amplification of PPO (

\text{H1}:\mathcal{S}_{3}\text{ vs }\mathcal{S}_{1}

), Search Amplification of Heuristic (

\text{H2}:\mathcal{S}_{4}\text{ vs }\mathcal{S}_{2}

), Unsearched Prior Strength (

\text{H3}:\mathcal{S}_{1}\text{ vs }\mathcal{S}_{2}

S=0

), Searched Prior Dominance (

\text{H4}:\mathcal{S}_{3}\text{ vs }\mathcal{S}_{4}

), and direct engine matchups for neural search (

\mathcal{S}_{3}\text{ vs Engine}

) and co-evolved heuristic search (

\mathcal{S}_{4}\text{ vs Engine}

Lineage / Aggregation	Empirical Tensor Entry	100 Sims	200 Sims	400 Sims	800 Sims	1600 Sims
Seed 1	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs Xinxin (Floor)	26.7% (-10.6)	25.0% (-11.4)	31.0% (-9.4)	26.8% (-11.3)	21.5% (-12.1)
	$\mathcal{S}_{3}\text{(Search+PPO)}$ vs Xinxin (Floor)	38.8% (-5.4)	39.5% (-4.4)	40.0% (-4.1)	35.5% (-6.3)	41.0% (-3.2)
	H1: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs $\mathcal{S}_{1}\text{(PPO)}$	54.5% (+1.5)	49.5% (+0.4)	56.0% (+2.2)	53.7% (+1.6)	61.4% (+3.3)
	H2: $\mathcal{S}_{4}\text{(Search+Heur)}$ vs $\mathcal{S}_{2}\text{(Heur)}$	42.2% (-4.0)	43.5% (-3.1)	47.0% (-2.6)	45.8% (-3.1)	41.5% (-4.1)
	H4: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs $\mathcal{S}_{4}\text{(Search+Heur)}$	77.5% (+10.5)	78.6% (+11.7)	79.2% (+12.0)	76.1% (+9.5)	81.8% (+12.5)
Seed 2	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs Xinxin (Floor)	29.2% (-10.9)	27.0% (-10.5)	29.8% (-9.7)	22.8% (-12.3)	28.2% (-10.3)

Appendix

Table 28: Per-Seed and 5-Seed Aggregated Empirical Game Tensor for Hearts (vs. Xinxin). Entries report seat-balanced Score% (

\text{Win}\%+0.5\times\text{Draw}\%

; parentheticals show net score margin

\Delta

where applicable) across simulation budgets

S\in\{100,200,400,800,1600\}

. Rows evaluate Search Amplification of PPO (

\text{H1}:\mathcal{S}_{3}\text{ vs }\mathcal{S}_{1}

), Search Amplification of Heuristic (

\text{H2}:\mathcal{S}_{4}\text{ vs }\mathcal{S}_{2}

), Unsearched Prior Strength (

\text{H3}:\mathcal{S}_{1}\text{ vs }\mathcal{S}_{2}

S=0

), Searched Prior Dominance (

\text{H4}:\mathcal{S}_{3}\text{ vs }\mathcal{S}_{4}

), and direct engine matchups for neural search (

\mathcal{S}_{3}\text{ vs Engine}

) and co-evolved heuristic search (

\mathcal{S}_{4}\text{ vs Engine}

Lineage / Aggregation	Empirical Tensor Entry	100 Sims	200 Sims	400 Sims	800 Sims	1600 Sims
Seed 1	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs WBridge5 (Default)	9.1% (-1470)	5.7% (-1528)	11.6% (-1098)	7.7% (-1154)	10.4% (-1030)
	$\mathcal{S}_{3}\text{(Search+PPO)}$ vs WBridge5 (Default)	7.4% (-1356)	5.9% (-1400)	4.4% (-1589)	8.8% (-1266)	4.7% (-1238)
	H1: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs $\mathcal{S}_{1}\text{(PPO)}$	56.0% (+385)	48.0% (-91)	44.6% (-72)	49.0% (-144)	58.5% (+346)
	H2: $\mathcal{S}_{4}\text{(Search+Heur)}$ vs $\mathcal{S}_{2}\text{(Heur)}$	63.5% (+1215)	77.2% (+1497)	81.2% (+1957)	76.6% (+1281)	69.8% (+1606)
	H4: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs $\mathcal{S}_{4}\text{(Search+Heur)}$	43.5% (+48)	42.8% (-472)	31.0% (-1029)	25.5% (-1297)	25.5% (-1314)
Seed 2	$\mathcal{S}_{4}\text{(Search+Heur)}$ vs WBridge5 (Default)	12.7% (-759)	17.4% (-793)	23.9% (-744)	24.0% (-795)	20.2% (-877)

Appendix

Table 29: Per-Seed and 5-Seed Aggregated Empirical Game Tensor for Contract Bridge (2v2 Homogeneous Partnerships) (vs. WBridge5). Entries report seat-balanced Score% (

\text{Win}\%+0.5\times\text{Draw}\%

; parentheticals show net score margin

\Delta

where applicable) across simulation budgets

S\in\{100,200,400,800,1600\}

. Rows evaluate Search Amplification of PPO (

\text{H1}:\mathcal{S}_{3}\text{ vs }\mathcal{S}_{1}

), Search Amplification of Heuristic (

\text{H2}:\mathcal{S}_{4}\text{ vs }\mathcal{S}_{2}

), Unsearched Prior Strength (

\text{H3}:\mathcal{S}_{1}\text{ vs }\mathcal{S}_{2}

S=0

), Searched Prior Dominance (

\text{H4}:\mathcal{S}_{3}\text{ vs }\mathcal{S}_{4}

), and direct engine matchups for neural search (

\mathcal{S}_{3}\text{ vs Engine}

) and co-evolved heuristic search (

\mathcal{S}_{4}\text{ vs Engine}

Partnership Matchup (North-South vs. East-West)	Lineage Scope	400 Sims Score% ( $\Delta$ )	800 Sims Score% ( $\Delta$ )	1600 Sims Score% ( $\Delta$ )
H2: $\mathcal{S}_{4}\text{(Search+Heur)}$ vs. $\mathcal{S}_{2}\text{(HeurPol)}$	All 5 Seeds	70.5% $\pm$ 7.0 ( $+1{,}540$ )	72.5% $\pm$ 5.3 ( $+1{,}341$ )	66.5% $\pm$ 5.9 ( $+1{,}210$ )
H1: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs. $\mathcal{S}_{1}\text{(PPO)}$	All 5 Seeds	50.0% $\pm$ 2.1 ( $+36$ )	50.7% $\pm$ 0.9 ( $+112$ )	53.4% $\pm$ 2.3 ( $+154$ )
H4: $\mathcal{S}_{3}\text{(Search+PPO)}$ vs. $\mathcal{S}_{4}\text{(Search+Heur)}$	All 5 Seeds	38.8% $\pm$ 16.5 ( $-808$ )	36.3% $\pm$ 17.1 ( $-947$ )	37.1% $\pm$ 16.6 ( $-795$ )
$\mathcal{S}_{4}\text{(Search+Heur)}$ vs. $\mathcal{S}_{0}:\text{WBridge5}$ (Default)	All 5 Seeds	13.8% $\pm$ 4.8 ( $-1{,}982$ )	15.7% $\pm$ 6.6 ( $-1{,}930$ )	13.2% $\pm$ 4.8 ( $-2{,}076$ )
$\mathcal{S}_{3}\text{(Search+PPO)}$ vs. $\mathcal{S}_{0}:\text{WBridge5}$ (Default)	All 5 Seeds	5.0% $\pm$ 0.9 ( $-1{,}369$ )	5.6% $\pm$ 1.2 ( $-1{,}273$ )	5.2% $\pm$ 0.7 ( $-1{,}253$ )

Appendix

Table 30: Contract Bridge 2v2 Homogeneous Partnership Tournament Summary (Across All 5 Lineages). North-South (

\text{Seats }\{0,2\}

) and East-West (

\text{Seats }\{1,3\}

) are each populated by a homogeneous partnership of the same agent type. Scores report 5-seed seat-balanced Win/Score% and single-board duplicate contract point margins (

\Delta

) across simulation budgets

S\in\{400,800,1600\}

against WBridge5 evaluated under its default tournament setting.

Explore similar work

Discovering Multiagent Learning Algorithms with Large Language Models
Feb 18, 2026Zun Li, John Schultz, Daniel Hennes +1Multi-Agent Reinforcement Learning Regret Minimization
Google DeepMind
Reasoning Capabilities of Large Language Models. Lessons Learned from General Game Playing
Feb 22, 2026Maciej Świechowski, Adam Żychowski, Jacek MańdziukLLM Reasoning LLM Evaluation
Grail Team Warsaw, Poland · Warsaw University of Technology Warsaw, Poland · AGH University of Krakow Cracow, Poland
Distilling Game Code World Model Generation into Lightweight Large Language Models
May 23, 2026Tyrone Serapio, Arjun Prakash, Haoyang Xu +2LLM-Based Program Synthesis
Brown University Providence, United States

Feb 18, 2026J/K move · Enter open · S save

Discovering Multiagent Learning Algorithms with Large Language Models

Zun Li, John Schultz, Daniel Hennes, Marc Lanctot

Google DeepMind

Multi-Agent Reinforcement Learning Regret Minimization Generalization in RL Large Language Model-Guided Optimization

Open paper AskPDF arXiv

Much of the advancement in Multi-Agent Reinforcement Learning (MARL) for imperfect-information games has historically depended on the manual, iterative refinement of algorithmic baselines. Recently, evolutionary coding agents powered by Large Language Models (LLMs) have emerged as powerful tools to automate this discovery process. In this work, we deploy one of such agentic frameworks, AlphaEvolve, to navigate the design spaces of two distinct game-theoretic paradigms: counterfactual regret minimization (CFR) and policy-space response oracles (PSRO). This automated search yielded two algorithms: Volatility-Adaptive Discounted (VAD-) CFR and Smoothed Hybrid Optimistic Regret (SHOR-) PSRO, which are consistently competitive with state-of-the-art human-designed baselines across an 18-game evaluation suite spanning Poker, Goofspiel, Liar's Dice, Blotto, and Battleship variants. However, because the LLM optimizes for fitness on a specific training set, it often constructs highly synergistic, complex mechanisms tailored to those environments. Through systematic ablation studies, we demonstrate that while these mechanisms are tightly coupled, the true driver of generalization lies in a minimal algorithmic core. By distilling the LLM's discoveries down to their most fundamental principles, we produce two minimal solvers: Warm-started Optimistic Predictive (WOP-)CFR and Projection Matching (PM-)PSRO. These distilled versions achieve superior performance on generalization with greatly reduced structural complexity, providing a clear methodology for using LLMs in algorithmic discovery.

Abstract

Explore similar work

Discovering Multiagent Learning Algorithms with Large Language Models

Reasoning Capabilities of Large Language Models. Lessons Learned from General Game Playing

Distilling Game Code World Model Generation into Lightweight Large Language Models