A Unified Arena for LLM Agents across Diverse Eurogame

A Unified Arena for LLM Agents across Diverse Eurogame

Abstract

Figures & tables

LLM Agent Evaluation

Game Arena: Strategic LLM Evaluation in Competitive Environments

Abstract

LLM Agent Evaluation

Explore similar work

Appendix

Game Arena: Strategic LLM Evaluation in Competitive Environments

Choosing LLMs for Strategic Agent Interactions

OmniGameArena: A Unified UE5 Benchmark for VLM Game Agents with Improvement Dynamics

Fengrui Zhang; Xiang Li; Ruomeng Ding; Jiayi Yuan; Yuchen Huang; Zhaoyang Han; Xiaoliang Wang; Daoyuan Chen

cs.AIOct 26, 2026

Authors: Fengrui Zhang, Xiang Li, Ruomeng Ding, Jiayi Yuan, Yuchen Huang, Zhaoyang Han, Xiaoliang Wang, Daoyuan Chen,

Organizations: Alibaba Group · Nanjing University · National University of Singapore · Department of Computer Science, University of North Carolina at Chapel Hill ·

Interactive environments offer a practical way to evaluate large language model (LLM) agents, and competitive games extend this evaluation to decisions shaped by competing goals and other agents' choices. Yet existing evaluations confined to individual games or execution frameworks leave unclear which competitive advantages persist across settings. We introduce DECE (Diverse Eurogame Competitive Environments), a unified arena spanning ten Eurogames with shared interfaces and evaluation protocols. These games combine lasting strategic commitments, competition over shared resources, and multiple routes to victory. Our primary study evaluates 12 model-harness configurations, spanning nine models and four execution harnesses, in 480 three-player matches across ten games. The main and supplementary studies comprise 1,032 completed matches in total. These comparisons reveal broad cross-game ranking agreement alongside game-specific strengths. Matched studies show that the same model can perform differently across execution harnesses, adding another source of variation to these profiles. These performance advantages do not consistently translate into better token or cost efficiency, making resource use an additional consideration when comparing configurations. Together, the findings support evaluating complete agent configurations by both their competitive performance across games and the resources needed to achieve it.

Figure 1: DECE overview. Shared interfaces connect games and model–harness systems. Seat-balanced matches produce trajectories and telemetry for four analyses. Icons show adapter families; Table 3 lists evaluated configurations. The mascot, pairings, and plots are schematic.

Benchmark	Game scope	Interaction	Main decision demands	Evaluation focus
GameBench ( 2024 )	9 board, card and social games	Players or teams	Spatial tactics, deck building and social inference	Competitive play; scaffold comparisons
MindGames ( 2026 )	4 strategic and social games	2–6 players; rivals or teams	Opponent inference, signaling, trust and deception	Social reasoning; arena validity
FinBoardBench ( 2026b )	Cashflow, Acquire, Monopoly	4-player competition	Cash flow, investment, mergers and negotiation	Financial decision analysis
PTCG-Bench ( 2026 )	Pokémon TCG; 5 decks	2-player competition	Card combinations, resource recovery and action order	Self-evolution; harness and cost analysis
From Code to Play ( 2024 )	12 TAG tabletop games	2–4-player tournaments	Deck building, drafting and tactical card play	Generate evaluators for search agents
LudoBench ( 2026 )	5 strategy board games	Fixed states	Perception, rule integration and local optimization	Fixed-state question answering

Table 1: Game scope, decision demands, and evaluation focus of related tabletop benchmarks. Entries summarize reported tasks and selected analyses. Appendix C gives sources and scope.

Group	Game	Tier	Players	Distinctive strategic demand
Capacity	Splendor	$\star$	2–4	Shared-market timing; engine investment
	Century: Spice Road	$\star$	2–5	Resource conversion; hand cycling
	Wingspan	$\star\star$	2–5	Tableau synergies; activation order
Spatial	Azul	$\star$	2–4	Drafting; spatial scoring; denial
	Harmonies	$\star\star$	2–4	Three-dimensional pattern construction
	Carcassonne	$\star$	2–5	Shared topology; follower commitment

Table 2: The selected ten-game roster and complementary strategic demands. Groups and colors follow Figure 2 . Player ranges describe the selected competitive multiplayer scope; solo modes are excluded. Stars denote introductory (

\star

), intermediate (

\star\star

), and advanced (

\star\star\star

) design tiers.

Figure 2: Complementary strategic demands across ten Eurogames. Each game is annotated along six dimensions. Values 0/1/2 denote absent/limited/prominent mechanisms; colors indicate overlapping strategic emphases. Appendix B gives the rubric and evidence.

		Capacity			Space			Coordination
Participant	ID	SP	CE	WI	AZ	HA	CA	7W	BU	BR	AN	Mean
GPT-5.6-Sol / Codex	1	66.7	66.7	83.3	66.7	75.0	66.7	66.7	70.8	95.8	95.8	75.4
GPT-5.6-Sol / OpenCode	2	54.2	87.5	75.0	66.7	79.2	81.2	75.0	87.5	66.7	83.3	75.6
GPT-5.6-Terra / Codex	3	37.5	70.8	45.8	33.3	37.5	39.6	41.7	41.7	79.2	75.0	50.2
GPT-5.6-Luna / Codex	4	20.8	12.5	16.7	37.5	37.5	29.2	25.0	41.7	33.3	41.7	29.6
DSV4-Pro / DSH	5	58.3	58.3	62.5	58.3	58.3	66.7	66.7	58.3	58.3	58.3	60.4

Table 3: Three-player PWS (%) across ten games. IDs 1–12 use the 480-match main cohort; Mean weights games equally. Bold marks the highest main-cohort estimate per game. IDs 13–15 are separate-cohort non-LLM baselines (Appendix D.8 ).

		Public spatial ( $G$ )				Other games ( $H$ )
Model	Harness A	AZ	HA	CA	BU	SP	CE	WI	7W	BR	AN
Qwen3.8-Max	Qwen Code	$+33.3$	$+10.4$	$+33.3$	$+41.7$	$+20.8$	$+12.5$	$+12.5$	$-12.5$	$0.0$	$-33.3$
GPT-5.6-Sol	Codex	$0.0$	$-4.2$	$-14.6$	$-16.7$	$+12.5$	$-20.8$	$+8.3$	$-8.3$	$+29.2$	$+12.5$
DSV4-Pro	DSH	$+12.5$	$-4.2$	$+16.7$	$-4.2$	$0.0$	$+8.3$	$+25.0$	$+4.2$	$+4.2$	$0.0$

Table 4: Same-model configuration differences across ten games (PWS, percentage points). (a) Harness A minus OpenCode for each game. (b) Equal-game group means, their difference

\Gamma

with pointwise 95% intervals, and the range of

\Gamma

after omitting one game at a time. Comparisons use the 480-match native cohort.

Figure 3: Harness contrasts in two independent studies. (a,b) Paired PWS gains, Qwen Code (QC) minus OpenCode (OC). (c) All six scenario gains per model–game cell in the 48-match within-Qwen-family study; columns match within each game. (d) Differences in gains: Qwen3.8-Flash minus DSV4-Flash (0731 snapshot; 72 matches), and Qwen3.8-Max minus Qwen3.8-Flash (48). Whiskers: pointwise 95% scenario-bootstrap intervals, without multiplicity correction.

Figure 4: Elo–cost Pareto frontiers by game (480 three-player matches; IDs from Table 3 ). Outlined points are nondominated; staircases show maximum estimated Elo per mean-cost budget. Axes are shared; whiskers: pointwise 95% whole-match intervals. Stability and resources: Appendix E ; token frontiers: Figure 10 .

Appendix figures & tables27 assets

Supplementary material from the paper’s appendix.

Game	Rules edition	Configuration scope
Splendor	Classic base rules	Developments, reservations, and nobles.
Century: Spice Road	Plan B base rules (2017)	Merchant hands public in DECE.
Wingspan	Stonemaier base English rules	Base English set; green goal board.
Azul	Next Move base rules (2018)	Colored-wall variant.
Harmonies	Libellud base rules (2024)	Board A; nature spirits disabled.
Carcassonne	Base rules with farmers	Base game with farmers.

Appendix

Table 5: Implemented game configurations. Rule versions come from engine metadata and experiment records; expansion options were checked against all 336 final databases.

ID	Recorded model string	Harness	Effort
1	gpt-5.6-sol	Codex	xhigh
2	openai/gpt-5.6-sol	OpenCode	xhigh
3	gpt-5.6-terra	Codex	xhigh
4	gpt-5.6-luna	Codex	xhigh
5	deepseek-v4-pro-0813	DSH	max
6	deepseek-v4-pro-0813	OpenCode	max

Appendix

Table 6: Registered model strings and effort fields in the completed seven-game expansion. IDs match the primary participant panel in Table 3 .

Figure 5: Integration and execution in DECE. The numbered stages follow one match; the loop repeats while play remains active. Controllers receive seat-specific views and return action IDs. The versioned game plugin applies rules, and the store commits state and events. Simultaneous choices are collected before any action is applied; public decision updates follow the complete group. Recorded calls and usage support resource analysis.

Appendix

Table 7: Explicit values behind Figure 2 . These ordinal mechanism annotations are not added or averaged.

Game	Matches	$D$ : median [IQR]	Engine turns	$D/3$
Splendor	48	81 [77.2, 84]	81	27
Century	48	129 [119.2, 135]	129	43
Wingspan	48	370 [357.8, 411.2]	370	123.3
Azul	48	70 [67.8, 72.2]	70	23.3
Harmonies	48	166.5 [158.8, 171.2]	166.5	55.5
Carcassonne	48	142 [142, 142]	142	47.3

Appendix

Table 8: Recorded decision and turn counts for all ten native games in the 480-match three-player cohort.

D

is the whole-match decision counter; brackets give its interquartile range. Engine turns and

D/3

are medians over matches. Counter granularity is game-specific;

D/3

is a within-match average over seats, not an individual-player distribution.

Game	3p matches	Appear./config.	Each seat	2p matches	4p matches
Splendor	48	12	4	72	36
Century	48	12	4	–	–
Wingspan	48	12	4	–	–
Azul	48	12	4	–	–
Harmonies	48	12	4	–	–
Carcassonne	48	12	4	–	–

Appendix

Table 9: Native match coverage across all ten games. Appearance and seat columns refer to the three-player cohort. The final columns contain the 252 supplementary two-/four-player matches; dashes mean no observed stratum. Each three-player game has the same 12-configuration roster.

Configuration	ID	SP	CE	WI	AZ	HA	CA	7W	BU	BR	AN
GPT-5.6-Sol / Codex	1	1622	1686	1811	1609	1821	1634	1634	1699	2068	2143
GPT-5.6-Sol / OpenCode	2	1552	1908	1728	1629	1888	1789	1695	1884	1673	1902
GPT-5.6-Terra / Codex	3	1419	1746	1511	1385	1328	1385	1445	1435	1794	1752
GPT-5.6-Luna / Codex	4	1275	1105	1170	1414	1329	1327	1297	1413	1328	1432
DSV4-Pro / DSH	5	1567	1646	1603	1561	1561	1641	1620	1590	1591	1614
DSV4-Pro / OpenCode	6	1550	1613	1378	1478	1660	1477	1583	1598	1525	1605

Appendix

Table 10: Three-player Elo point estimates for all ten native games, rounded to integers. Each game is fitted separately over the same 12 configurations and centered at 1500. No average of per-game Elo is used. IDs and game abbreviations follow Table 3 .

Configuration	Mean PWS (%)	95% interval
GPT-5.6-Sol / OpenCode	75.6	[69.7, 81.3]
GPT-5.6-Sol / Codex	75.4	[68.9, 81.7]
Qwen3.8-Max / Qwen Code	70.2	[64.5, 75.7]
DSV4-Pro / DSH	60.4	[53.6, 67.0]
Qwen3.8-Max / OpenCode	58.3	[51.9, 64.7]
DSV4-Pro / OpenCode	54.2	[47.3, 60.8]

Appendix

Table 11: Ten-game equal-weight PWS and pointwise 95% whole-match bootstrap intervals. Each configuration has 120 primary-cohort appearances. These estimates exclude non-LLM baselines and the matched-bridge cohort.

Game	LOEO $\tau_{b}$	Order acc.	Split-half mean	Split 5–95% range
SP	0.49	75.4%	0.18	[-0.07, 0.44]
CE	0.72	86.2%	0.58	[0.38, 0.78]
WI	0.72	86.2%	0.55	[0.33, 0.74]
AZ	0.59	80.6%	0.27	[0.00, 0.53]
HA	0.71	85.9%	0.64	[0.47, 0.80]
CA	0.86	93.8%	0.49	[0.27, 0.71]

Appendix

Table 12: Descriptive cross-game ranking agreement and within-game split-half reliability. Game abbreviations match Table 3 . The final range describes variation across random splits; it is not a confidence interval.

Configuration	SP	CE	WI	AZ	HA	CA	7W	BU	BR	AN
GPT-5.6-Sol / Codex	2	4	2	3	4	4	2	3	1	1
GPT-5.6-Sol / OpenCode	5	1	3	2	2	2	1	1	3	2
GPT-5.6-Terra / Codex	10	3	6	11	10	7	8	8	2	4
Qwen3.8-Max / Qwen Code	1	2	1	1	1	1	9	2	6	8
Qwen3.8-Max / OpenCode	9	5	4	8	3	5	5	10	7	3
DSV4-Pro / DSH	4	6	5	4	6	3	3	5	4	5

Appendix

Table 13: Selected configurations’ three-player Elo ranks among all 12 participants in each game; 1 is highest. Columns follow Table 3 .

Descriptive check (Qwen3.8-Max throughout)	PWS difference (pp)
Public spatial: Qwen Code minus OpenCode	$+29.7$
Same contrast, omit Azul	$+28.5$
Same contrast, omit Harmonies	$+36.1$
Same contrast, omit Burgundy	$+25.7$
Same contrast, omit Carcassonne	$+28.5$
HI=2: Qwen Code minus OpenCode	$-8.3$

Appendix

Table 14: Additional game-deletion diagnostics for Qwen3.8-Max: Qwen Code minus OpenCode PWS. Public spatial means spatial planning = 2 and hidden information = 0; hidden information = 2 denotes central private holdings. These within-group means differ from the between-group contrast in Table 4 .

Figure 6: Same-model PWS contrasts across all ten native games in the 480-match three-player cohort. Positive values favor the first named harness; diamonds show equal-game means. Pointwise 95% intervals reuse the joint whole-match bootstrap draws, preserving covariance between configurations within a game. These descriptive native-system contrasts use different opponent samples and are separate from the matched studies.

Figure 7: Per-game PWS with pointwise 95% whole-match bootstrap intervals for all ten native games. IDs match Table 3 ; every panel orders configurations by their ten-game mean PWS. Each panel contains 48 three-player matches and 12 appearances per configuration. The dashed reference is 50% PWS.

ID	SP	CE	WI	AZ	HA
1	66.7 [40.0, 90.9]	66.7 [38.9, 90.0]	83.3 [68.8, 95.8]	66.7 [43.8, 87.5]	75.0 [54.5, 92.9]
2	54.2 [35.7, 73.1]	87.5 [66.7, 100.0]	75.0 [55.0, 92.9]	66.7 [47.2, 85.0]	79.2 [59.1, 95.8]
3	37.5 [13.6, 62.5]	70.8 [50.0, 88.9]	45.8 [25.0, 68.8]	33.3 [12.5, 56.2]	37.5 [14.3, 62.5]
4	20.8 [4.2, 40.9]	12.5 [0.0, 25.0]	16.7 [0.0, 37.5]	37.5 [14.3, 62.5]	37.5 [20.0, 55.0]
5	58.3 [37.5, 78.1]	58.3 [42.3, 75.0]	62.5 [40.9, 83.3]	58.3 [30.0, 83.3]	58.3 [38.2, 78.6]
6	58.3 [31.8, 85.0]	50.0 [33.3, 66.7]	37.5 [13.6, 62.5]	45.8 [30.6, 60.0]	62.5 [40.0, 83.3]

Appendix

Table 15: Ten-game PWS (%) and pointwise 95% whole-match bootstrap intervals. Panels cover all 120 primary cells; IDs and game abbreviations match Table 3 . Each cell uses 12 appearances. Intervals are computed jointly within each game’s sampled matches and are not simultaneous bounds across cells.

Figure 8: Execution demand and recorded token volume across the ten-game primary cohort. Cells are means over 12 appearances: (a) saved controller decisions; (b) normalized input plus output tokens, in millions. Each panel has its own logarithmic color scale; printed values use the original units. Game abbreviations follow Table 3 .

			Token-usage coverage
Configuration	ID	Decisions	Summary coverage	Combined coverage
GPT-5.6-Sol / Codex	1	8,176	0/120	120/120
GPT-5.6-Sol / OpenCode	2	7,975	114/120	120/120
GPT-5.6-Terra / Codex	3	7,673	0/120	120/120
GPT-5.6-Luna / Codex	4	7,303	0/120	120/120
DSV4-Pro / DSH	5	7,656	114/120	120/120

Appendix

Table 16: Ten-game execution and usage coverage, 120 appearances per configuration. Decisions are totals from saved controller records. Summary coverage counts appearances with numeric token totals in controller summaries; combined coverage includes usage from native session records. For Codex, native session logs provide token usage for all 360 primary-cohort appearances (120 per configuration); the controller summaries omit these totals.

Nominal model	Uncached input	Cached input	Output
GPT-5.6-Sol	4.0000	0.4000	20.0000
GPT-5.6-Terra	2.0000	0.2000	12.0000
GPT-5.6-Luna	0.2000	0.0200	1.2000
DSV4-Flash	0.4424	0.0442	1.3272
DSV4-Pro	1.3272	0.1327	3.9817
GLM-5.2	1.1797	0.2949	4.1291

Appendix

Table 17: Fixed price scenario, USD per million tokens, dated 28 August 2026 and converted at 6.7811 CNY per USD. Rates are fixed across the resource analysis.

Configuration	ID	SP	CE	WI	AZ	HA	CA	7W	BU	BR	AN
GPT-5.6-Sol / Codex	1	4.95	10.29	38.90	3.73	12.06	12.71	3.84	19.19	40.53	37.20
GPT-5.6-Sol / OpenCode	2	3.87	9.64	34.18	3.33	11.74	11.93	2.85	19.89	16.83	34.14
GPT-5.6-Terra / Codex	3	2.31	6.26	17.29	2.37	5.98	6.40	2.01	10.89	25.07	20.63
GPT-5.6-Luna / Codex	4	0.42	0.61	1.48	0.25	0.63	0.62	0.26	1.12	2.00	1.85
DSV4-Pro / DSH	5	2.53	4.92	9.19	2.82	4.20	5.81	2.38	7.25	13.69	11.36
DSV4-Pro / OpenCode	6	1.79	4.74	13.64	2.18	4.60	8.57	1.71	12.42	22.07	23.50

Appendix

Table 18: Recorded API-equivalent USD per participant appearance, twelve appearances in every cell. Games follow Table 3 . Costs use Equation 5 .

Configuration	PWS (%)	USD [95% interval]	Tokens (M)
GPT-5.6-Sol / Codex	75.4	18.34 [17.83, 18.89]	31.89
GPT-5.6-Sol / OpenCode	75.6	14.84 [14.38, 15.35]	27.79
GPT-5.6-Terra / Codex	50.2	9.92 [9.63, 10.20]	31.21
GPT-5.6-Luna / Codex	29.6	0.92 [0.89, 0.96]	27.74
DSV4-Pro / DSH	60.4	6.42 [6.21, 6.62]	32.76
DSV4-Pro / OpenCode	54.2	9.52 [8.81, 10.28]	61.06

Appendix

Table 19: Equal-game resource overview. Each configuration contributes 120 three-player appearances across ten games. Cost intervals are pointwise 95% whole-match percentile intervals conditional on the accounting and price schedule. Tokens include cached input and reasoning exactly once.

Configuration	ID	Games	Sp	Ce	Wi	Az	Ha	Ca	7W	Bu	Br	AN
GPT-5.6-Sol / Codex	1	5/10	44.3	12.2	63.6	29.6	26.3	19.8	18.3	51.0	90.3	94.0
GPT-5.6-Sol / OpenCode	2	9/10	22.4	83.8	54.5	51.7	64.5	88.7	55.5	90.7	69.8	83.5
GPT-5.6-Terra / Codex	3	3/10	4.7	81.2	16.0	3.7	0.7	2.5	7.7	7.2	74.5	85.6
GPT-5.6-Luna / Codex	4	10/10	99.8	100.0	100.0	100.0	100.0	100.0	100.0	100.0	60.4	100.0
DSV4-Pro / DSH	5	10/10	37.0	71.4	81.5	42.6	71.5	98.7	49.6	76.5	93.1	92.0
DSV4-Pro / OpenCode	6	5/10	63.0	70.1	3.7	14.6	83.1	6.2	65.7	44.8	6.7	16.3

Appendix

Table 20: Pareto-frontier coverage and bootstrap membership frequency (%). Games and configuration IDs follow Table 3 . Coverage counts point-estimate frontiers out of ten games; each frequency uses 5,000 joint whole-match draws. Bold entries mark membership in the point-estimate frontier.

Figure 9: Aggregate performance and resource use on the same 480 three-player matches as Figure 4 . Each point averages ten games equally, with 120 appearances per configuration. Logarithmic horizontal axes show mean API-equivalent USD per appearance and input-plus-output tokens. Whiskers are pointwise 95% whole-match intervals; IDs match Table 3 .

Figure 10: Game-specific Elo–token Pareto frontiers on the same 480 three-player matches as Figure 4 . Horizontal axes show mean input-plus-output tokens per appearance in millions, on a log scale. Elo axes, configuration IDs, and plotting conventions match the cost figure. Outlined, labeled points are nondominated at the point estimates; whiskers show pointwise 95% whole-match intervals. Token volume measures recorded usage, independently of the price schedule.

Configuration	ID	USD	Tokens	SP	CE	WI	AZ	HA	CA	7W	BU	BR	AN
GPT-5.6-Sol / Codex	1	5	8	70.4	60.6	79.5	42.3	93.7	90.0	31.1	97.3	96.2	98.8
GPT-5.6-Sol / OpenCode	2	9	9	99.9	92.7	62.2	98.2	73.0	81.6	99.2	88.7	99.4	17.4
GPT-5.6-Terra / Codex	3	3	2	7.7	11.9	31.2	0.4	67.9	59.9	1.1	26.1	49.5	16.5
GPT-5.6-Luna / Codex	4	10	3	0.1	0.1	31.6	0.9	69.1	80.6	0.0	2.3	1.3	77.5
DSV4-Pro / DSH	5	10	2	15.9	1.2	92.3	16.9	31.0	5.7	16.7	1.8	24.0	47.3
DSV4-Pro / OpenCode	6	5	0	21.3	0.0	0.1	0.5	6.4	0.5	13.9	1.5	0.0	0.0

Appendix

Table 21: Dollar and token frontier coverage, followed by token-frontier membership frequencies (%) over 5,000 joint whole-match draws. USD and Tokens count point-estimate frontiers out of ten games. Bold entries mark token-frontier membership at the point estimates. Game abbreviations and IDs follow Table 3 .

Configuration	ID	SP 2p	SP 4p	7W 4p	BR 2p	BR 4p
GPT-5.6-Sol / Codex	1	50.0	55.6	77.8	100.0	91.7
GPT-5.6-Sol / OpenCode	2	58.3	52.8	58.3	83.3	66.7
GPT-5.6-Terra / Codex	3	58.3	29.2	56.9	91.7	77.8
GPT-5.6-Luna / Codex	4	16.7	13.9	36.1	29.2	44.4
DSV4-Pro / DSH	5	66.7	69.4	52.8	29.2	75.0
DSV4-Pro / OpenCode	6	58.3	47.2	62.5	50.0	38.9

Appendix

Table 22: Supplementary player-count PWS (%) from 252 two-/four-player matches, with 12 appearances per configuration and stratum. The common three-player comparison across all ten games is in Table 3 . Only Splendor, 7 Wonders, and Brass have additional observed player counts; coverage is listed in Table 9 .

Game	Focal model	OpenCode	Qwen Code	$\Delta$ PWS [95% CI]
Splendor	DSV4-Flash	50.0	55.6	$+5.6\;[-38.9,44.4]$
Splendor	Qwen3.8-Flash	38.9	55.6	$+16.7\;[-16.7,50.0]$
7 Wonders	DSV4-Flash	66.7	61.1	$-5.6\;[-38.9,27.8]$
7 Wonders	Qwen3.8-Flash	27.8	72.2	$+44.4\;[11.1,77.8]$
Equal-game mean	DSV4-Flash	58.3	58.3	$+0.0\;[-27.8,25.0]$
Equal-game mean	Qwen3.8-Flash	33.3	63.9	$+30.6\;[5.6,52.8]$

Appendix

Table 23: Focal PWS (%) and harness contrasts (percentage points), Qwen Code minus OpenCode. Each game–model cell contains nine matched pairs; the pooled row has 18 pairs with equal game weights. Pointwise 95% percentile intervals use 100,000 scenario-cluster bootstrap draws. These are anchor-relative PWS diagnostics, not Elo or four-way head-to-head rankings.

Focal model	Harness	PWS (%)	First	Second	Third
Qwen3.8-Flash	Qwen Code	63.9	9	5	4
DSV4-Flash	OpenCode	58.3	7	7	4
DSV4-Flash	Qwen Code	58.3	6	9	3
Qwen3.8-Flash	OpenCode	33.3	3	6	9

Appendix

Table 24: Anchor-relative final placements in the matched bridge. Each row has 18 matches, nine per game. Rows follow point-estimate PWS. Equal PWS can conceal different first-place and last-place frequencies.

Game	Model	OpenCode	Qwen Code	$\Delta$ [95% interval]	$+/=/-$
Splendor	Qwen3.8-Flash	25.0	33.3	$+8.3\;[-25.0,41.7]$	3/1/2
Splendor	Qwen3.8-Max	33.3	91.7	$+58.3\;[50.0,75.0]$	6/0/0
7 Wonders	Qwen3.8-Flash	16.7	33.3	$+16.7\;[-25.0,58.3]$	3/1/2
7 Wonders	Qwen3.8-Max	75.0	83.3	$+8.3\;[-25.0,50.0]$	1/4/1

Appendix

Table 25: The 48-match within-Qwen-family study. Each model–game comparison has six matched pairs. PWS is in percent and contrasts are in percentage points;

+/=/-

counts scenarios in which Qwen Code improves, matches, or worsens focal placement relative to OpenCode. Intervals are pointwise 95% scenario-bootstrap percentiles.

Large language model agent evaluation measures whether tool-using systems complete tasks reliably, safely, and efficiently.

See all 1,217 papers

Game Arena: Strategic LLM Evaluation in Competitive Environments
Sep 25, 2026Bovard Doerschuk-Tiberi, Yao Yan, Justin Chiu +59LLM Agent Evaluation AI Agent Benchmarks
Choosing LLMs for Strategic Agent Interactions
Oct 13, 2026Arastun Mammadli, Herbert Woisetschläger, Jawad Fayaz +1Model Selection LLM Decision-Making
University of Exeter United Kingdom · IBM Research Switzerland
OmniGameArena: A Unified UE5 Benchmark for VLM Game Agents with Improvement Dynamics
Jun 8, 2026Mingxian Lin, Shengju Qian, Yuqi Liu +9VLM Evaluation Game-Playing Agents
1The University of Hong Kong · 2LIGHTSPEED · 3The Chinese University of Hong Kong +1

Sep 25, 2026J/K move · Enter open · S save

Bovard Doerschuk-Tiberi, Yao Yan, Justin Chiu, Hann Wang+58

LLM Agent Evaluation AI Agent Benchmarks Game-Playing Agents Strategic Reasoning Risks in Language Models

Open paper AskPDF arXiv

We introduce Kaggle Game Arena, an open and ever-expanding platform to evaluate large language models (LLMs) through competitive games. Different from static benchmarks, game arena enables models to play head-to-head matchups in structured environments where the gameplay strength naturally increases as models evolve, preventing performance saturation. This technical report details the infrastructure behind Game Arena and describes the three pilot game environments: Chess, Poker, and Werewolf. These environments span perfect information, imperfect information, and multiplayer game settings, enabling a systematic study of models' strategic planning, adaptation, and robustness under uncertainty. For each game, we provide a detailed description of the environment, evaluation metrics, and results from running full competitions across models. Through robust infrastructure and large-scale ground-truth based evaluation, Game Arena ensures reproducibility, transparency and generalizability to new games and variants over time.

Century: Spice Road

Carcassonne