Game balancing is a longstanding challenge requiring repeated playtesting, expert intuition, and extensive manual tuning. We introduce RuleSmith, a framework that couples multi-agent LLM self-play with surrogate-based optimization over a parameterized rule space. As a proof of concept, we build CivMini, a simplified civilization-style game with two asymmetric factions governed by 12 tunable parameters. LLM agents interpret textual rulebooks and game states to propose actions that are validated by the game engine, yielding noisy estimates of faction win rates. Our primary implementation uses Gaussian-process Bayesian optimization with Expected-Improvement-based adaptive sampling to allocate more evaluation games to promising candidates. RuleSmith consistently finds near-balanced configurations: three independent optimization runs reach near-even empirical win rates in independent 200- and 300-game evaluations, and additional experiments with NTBEA and TPE show that the same self-play interface supports different search components. Cross-model evaluations and small human studies provide preliminary evidence of transfer across player populations. RuleSmith produces interpretable numerical rule adjustments without task-specific policy training. Project and code: https://adonis-galaxy.github.io/RuleSmith-website/
Figures & tables
Figure 1: Overview of RuleSmith. RuleSmith treats asymmetric game balancing as optimization over game rollouts using multi-agent LLMs given a natural-language rulebook. It optimizes interpretable parameters such as unit HP, attack damage, resource gain, production cost, and scoring weights. For each candidate game parameter setting, two LLM agents play the game zero-shot, and their outcomes estimate the faction win rates and balance loss. A Bayesian optimizer then proposes new settings, while adaptive sampling allocates more games to promising candidates. This enables closed-loop balancing without task-specific policy training, reducing manual playtesting and parameter tuning. Figure generated by Nano Banana Pro.
Figure 2: Overview of the RuleSmith method. We represent an asymmetric, turn-based strategy game (CivMini) as a parameterized rule space θ∈Θ , covering economy, combat, production, and scoring parameters. Given a candidate rule configuration θt , two role-specific LLM agents ( Empire and Nomads ) play Nt self-play games in the CivMini environment, producing a balance loss L(θ)=∣wE(θ)−0.5∣+∣wN(θ)−0.5∣+0.5wD(θ) , where wE , wN , and wD are the Empire win rate, Nomads win rate, and draw rate, respectively. A Bayesian optimizer maintains a surrogate model g(θ) over a continuous relaxation of the rule space and selects new candidates θ~t+1 by maximizing an acquisition function. The number of games Nt is adaptively determined based on the Expected Improvement: promising candidates receive more games for accurate evaluation. Each continuous proposal is mapped to a valid, discrete ruleset via a deterministic discretization operator D(⋅) before evaluation. Cartoons in this figure were generated using ChatGPT-5.2.
Parameter
Range
Prec.
Parameter
Range
Prec.
Economy
Production
Initial resources
[2,10]
int
Empire unit cost
[2,10]
int
Empire farmer gather
[1,5]
int
Nomads unit cost
[2,10]
int
Nomads kill gain
[1,10]
int
Combat
Scoring
Empire damage
[1,5]
int
Resource weight
[0.1,0.5]
0.1
Table 1: Optimizable parameters in CivMini. We optimize 12 parameters governing economy, combat, production, and scoring. Precision indicates the discretization granularity applied before evaluation.
Optimize \ Evaluate
E 2B vs N 2B
E 2B vs N 8B
E 8B vs N 2B
E 8B vs N 8B
E 2B vs N 2B
48 ∣ 52
32 ∣ 68
73 ∣ 27
55 ∣ 45
E 2B vs N 8B
81 ∣ 19
47 ∣ 53
91 ∣ 9
75 ∣ 25
E 8B vs N 2B
37 ∣ 63
0 6 ∣ 94
52 ∣ 48
29 ∣ 71
E 8B vs N 8B
53 ∣ 47
24 ∣ 76
81 ∣ 19
51 ∣ 49
Table 2: Optimized winning chances of Empire and Nomads. Rows specify the agent pairing used to optimize rules; columns specify the evaluation pairing. Each cell reports Empire wins ∣ Nomads wins over 100 held-out games using the best optimization checkpoint (empirical loss ≤0.1 ). E / N denote the factions; subscripts denote model sizes. Greener cells indicate more even outcomes; empirical near-balance ( 50%±5% ) is bolded.
Configuration
Games
Empire (%)
95% CI (%)
p
Default (additional evaluation)
300
11.3
[8.2, 15.4]
2.9×10−45
Optimized configuration (Table 10 )
500
47.6
[43.3, 52.0]
0.30
GP run 1
300
51.7
[46.0, 57.3]
0.60
GP run 2
300
49.3
[43.7, 55.0]
0.86
GP run 3
300
52.7
[47.0, 58.2]
0.39
NTBEA
300
48.0
[42.4, 53.6]
0.53
Table 3: Additional held-out evaluations. Empire win rates, Wilson 95% intervals, and two-sided exact binomial tests against 0.5. Counts are drawn from the evaluation logs; the test compares Empire and Nomads wins among decisive games (Appendix H ). The default batch is separate from the authors’ initial configuration. NTBEA/TPE replace the search component; GP-LCB replaces the acquisition rule.
Optimization LLM
Evaluation LLM
Empire%
Nomads%
L
InternVL3.5-8B
InternVL3.5-8B
51
49
0.02
InternVL3.5-8B
Qwen3.5-9B
50
50
0.00
Qwen3.5-9B
InternVL3.5-8B
49
51
0.02
Qwen3.5-9B
Qwen3.5-9B
52
48
0.04
Table 4: Cross-LLM-family transfer. Empirical win rates meet the near-balance criterion ( 50%±5% ) when parameters optimized with one LLM family are evaluated by the other. All results use 100 evaluation games.
Random Search
(1+1)-ES
BO (adaptive)
Win Rate
13 ∣ 87
26 ∣ 74
51 ∣ 49
BO ( N=16 )
BO ( N=32 )
BO ( N=64 )
Win Rate
34 ∣ 66
61 ∣ 39
48 ∣ 52
Table 5: Original search and replication ablation. Each method runs for 100 iterations and is evaluated on 100 held-out games. Random Search and (1+1)-ES use 64 games per candidate; GP-BO uses either fixed replication or 16–64 games based on EI. These are single optimization runs.
Metric
First 100 games
All 300 games
Distinct three-turn opening sequences
86/100
229/300
Distinct first-turn action combinations
6
11
Action entropy, Empire / Nomads (bits)
2.03 / 1.65
2.04 / 1.64
Mean turn-5 Jaccard distance
0.62
0.62
Ended before the turn cap (a city destroyed)
63%
62%
Game length, mean (SD) in turns
10.8 (4.4)
11.1 (4.4)
Table 6: Trajectory diversity under identical rules (8B self-play, NTBEA configuration; Appendix I ). The 100-game column is a subset of the 300-game column, not an independent replication.
Map size (turn limit)
Three factions
Map size optimized
5 × 5 (16)
7 × 7 (16)
9 × 9 (32)
11 × 11 (32)
8B
2B
8B
2B
Win rate (%)
53 ∣ 47
51 ∣ 49
48 ∣ 52
51 ∣ 49
32 ∣ 33 ∣ 35
34 ∣ 33 ∣ 33
53 ∣ 47
51 ∣ 49
Balance loss
0.06
0.02
0.04
0.02
0.033
0.013
0.06
0.02
Table 7: Extended configurations. Empire ∣ Nomads win rates (with Theocracy third for three factions) and balance loss for four map sizes with InternVL3.5-8B (turn limits in parentheses), three-faction balancing with four added Theocracy parameters ( L3 ), and map size (5–9) as an additional optimizable parameter (Appendix F ). Map-size losses follow Eq. ( 3 ) from the reported win rates.
Appendix figures & tables20 assets
Supplementary material from the paper’s appendix.
Appendix
Empire
Nomads
Empire%
Nomads%
InternVL3.5-8B
Random
100
0
Random
InternVL3.5-8B
0
100
InternVL3.5-2B
Random
100
0
Random
InternVL3.5-2B
0
100
Appendix
Table 8: LLM agent vs. random agent. LLM agents (InternVL3.5-8B and 2B) are evaluated against a uniformly random legal-action agent using parameters optimized by the respective LLM. The prompted LLM agents win 100% of games in every listed configuration.
LLM-optimized params
Rule-optimized params
Empire
Nomads
Empire%
Nomads%
Empire%
Nomads%
InternVL3.5-8B
Rule-based
75
25
73
27
Rule-based
InternVL3.5-8B
22
78
38
62
InternVL3.5-2B
Rule-based
68
32
65
35
Rule-based
InternVL3.5-2B
31
69
46
54
Appendix
Table 9: LLM agent vs. rule-based agent. Win rates when LLM agents (InternVL3.5-8B and 2B) play against the rule-based agent under parameters optimized via LLM self-play (left) and rule-based self-play (right). All results use 100 evaluation games.
Parameter
Initial
Opt.
Parameter
Initial
Opt.
Economy
Production
Initial resources
6
10
Empire unit cost
7
9
Empire farmer gather
2
1
Nomads unit cost
3
8
Nomads kill resource gain
4
7
Combat
Scoring
Empire damage
4
4
Score per resource
0.1
0.1
Appendix
Table 10: Human-designed vs. RuleSmith-optimized parameters. The authors’ initial configuration is heavily skewed toward Empire (balance score 1.00). RuleSmith reduces the balance score to 0.02 through coordinated adjustments across all parameter categories.
Optimization LLM
Evaluation LLM
Empire%
Nomads%
L
InternVL3.5-8B
Gemini 2.5 Flash
51
49
0.02
InternVL3.5-8B
Gemini 2.5 Pro
47
53
0.06
Qwen3.5-9B
Gemini 2.5 Flash
45
55
0.10
Qwen3.5-9B
Gemini 2.5 Pro
53
47
0.06
Appendix
Table 11: Evaluation with Gemini Flash and Pro. Parameters optimized with InternVL3.5-8B or Qwen3.5-9B meet the empirical near-balance criterion when evaluated with Gemini 2.5 Flash or Pro. The comparison varies model identity and does not isolate a reasoning mode.
Empire
Nomads
Empire%
Nomads%
L
InternVL3.5-8B
Gemini 2.5 Flash
45
55
0.10
InternVL3.5-8B
Gemini 2.5 Pro
41
59
0.18
Gemini 2.5 Flash
InternVL3.5-8B
45
55
0.10
Gemini 2.5 Pro
InternVL3.5-8B
49
51
0.02
Appendix
Table 12: Asymmetric cross-model matchups. Parameters optimized with symmetric InternVL3.5-8B self-play are evaluated with different models on the two sides. Faction outcomes vary with the pairing, while both factions retain wins in the tested matchups.
Empire
Nomads
Total
Wins
32
28
60
Win Rate
53.3%
46.7%
N/A
Appendix
Table 13: Human user study results. 4 participants, 6 pairwise matchups, 2 faction assignments each, 5 games per session, 60 games total. Empire win rate 53.3% and Nomads win rate 46.7% both fall within the near-balanced range ( 50%±5% ).
Participant
Human faction
vs. 2B
vs. 8B
P1
Empire
9/10
8/10
P1
Nomads
10/10
8/10
P2
Empire
8/10
7/10
P2
Nomads
9/10
9/10
P3
Empire
9/10
7/10
P3
Nomads
10/10
9/10
Appendix
Table 14: Human wins by participant and faction against InternVL3.5 agents. Each participant/faction cell has 10 games; totals pool the three participants.
Population
Empire wins
E (%)
E 95% CI (%)
p
Human vs. human
32/60
53.3
[40.9, 65.4]
0.70
Human vs. 2B
27/60
45.0
[33.1, 57.5]
0.52
Human vs. 8B
26/60
43.3
[31.6, 55.9]
0.37
Appendix
Table 15: Aggregate faction outcomes across small human-player studies. Wilson intervals and binomial p values summarize game-level totals under a working independence assumption. Repeated participants and learning limit population-level interpretation.
Parameter
LLM Agents
Random Agents
Rule-Based Agents
Economy
Initial resources
10
4
6
Empire farmer gather
1
1
2
Nomads kill resource gain
7
5
5
Combat
Empire damage
4
2
3
Appendix
Table 16: Optimized parameters: LLM agents, random agents, and rule-based agents. The three proxy player types yield substantially different balanced configurations, reflecting their distinct decision dynamics.
Parameter
E 2B vs N 2B
E 2B vs N 8B
E 8B vs N 2B
E 8B vs N 8B
Initial resources
10
2
10
10
Empire farmer gather
1
2
4
1
Nomads kill resource gain
6
9
1
7
Empire damage
4
2
2
4
Nomads damage
3
3
5
3
Empire soldier HP
9
14
16
9
Appendix
Table 17: Optimized game parameters under different LLM agent configurations. We report optimized parameters that achieve approximately 50/50 win rates across different LLM agent configurations. We also include the implied time-to-kill (TTK; number of attacks required to eliminate the opponent’s main combat unit) derived from damage and HP.
Parameter
R94
R98
Reported 8B/8B
Initial resources
8
4
10
Empire farmer gather
1
1
1
Nomads kill resource gain
8
7
7
Empire damage
2
2
4
Nomads damage
4
4
3
Empire soldier HP
14
14
9
Appendix
Table 18: Configurations used in the qualitative rollouts. R94 and R98 are archived optimization checkpoints, distinct from the reported 8B/8B configuration in Table 17 .
Figure 3: Visualization of a CivMini Game. Nomads conquer the Empire’s city on turn 12 (13 rounds, counting from 0). Cavalry bypass the Empire’s defense and sustain city pressure. Both agents use InternVL3.5-8B. This example is R94 game 13 (Table 18 ); city destruction determines victory irrespective of the displayed scores.
Figure 4: A rapid Nomads conquest on turn 6 (7 rounds, counting from 0). Cavalry Unit 0 strikes the Empire’s city on turns 3–6, dealing 4 HP per attack and destroying it. Checkpoint R98, game 4. (Final score: Empire 15.8 | Nomads 27.6)
Figure 5: A Nomads score victory after 16 rounds (turns 0–15). Continued cavalry production builds a numerical advantage, while the Empire deploys Farmers and Soldiers and loses units during the fighting. Checkpoint R98, game 1. (Final score: Empire 14.2 | Nomads 67.6)
Figure 6: The Empire withstands early Nomads aggression and wins at the 16-round turn cap. Soldiers defend through fighting on turns 3–15, while Farmers supply resources. Two Cavalry remain at the end; the Empire wins through the combined final scoring terms. Checkpoint R98, game 5. (Final score: Empire 31.6 | Nomads 23.4)
Figure 7: The Empire defends against a Nomads city rush beginning on turn 5. By turn 15, one Cavalry remains, while the Empire retains two Soldiers and two Farmers. Unit survival and resources contribute to its score victory. Checkpoint R98, game 13. (Final score: Empire 34.4 | Nomads 12.6)
Search / run
n
kE
E (%)
N (%)
L
E 95% CI
p
GP run 1
200
103
51.5
48.5
0.030
[44.6, 58.3]
0.72
GP run 1
300
155
51.7
48.3
0.033
[46.0, 57.3]
0.60
GP run 2
200
98
49.0
51.0
0.020
[42.2, 55.9]
0.83
GP run 2
300
148
49.3
50.7
0.013
[43.7, 55.0]
0.86
GP run 3
200
103
51.5
48.5
0.030
[44.6, 58.3]
0.72
GP run 3
300
158
52.7
47.3
0.053
[47.0, 58.2]
0.39
Appendix
Table 19: Full 200-/300-game re-evaluations. kE is the number of Empire wins from the evaluation logs; none of these batches contains a draw. Losses use counts rather than rounded percentages. Each configuration’s two batches use separate rollout seeds.
Rate (%)
95% CI (%)
Rate (%)
95% CI (%)
Rate (%)
95% CI (%)
0
[0.0, 3.7]
6
[2.8, 12.5]
13
[7.8, 21.0]
22
[15.0, 31.1]
24
[16.7, 33.2]
26
[18.4, 35.4]
29
[21.0, 38.5]
31
[22.8, 40.6]
32
[23.7, 41.7]
33
[24.6, 42.7]
34
[25.5, 43.7]
35
[26.4, 44.7]
37
[28.2, 46.8]
38
[29.1, 47.8]
41
[31.9, 50.8]
45
[35.6, 54.8]
46
[36.6, 55.7]
47
[37.5, 56.7]
Appendix
Table 20: Wilson 95% intervals for every distinct win rate in the 100-game evaluations of Tables 2 , 4 , 5 , 8 , 9 , 10 , 11 , 12 , and 7 , under the 100-game protocol of Section 4.2 . A rate of r % corresponds to r wins. For no-draw two-faction outcomes, the Nomads interval is [100−u,100−l] when the Empire interval is [l,u] ; for three factions, each faction’s rate is listed.
Setting
Attempted actions
Converted to PASS
Rate
8B agents, 500 games
33,834
1,126
3.33%
2B agents, 200 games
21,871
1,012
4.63%
Appendix
Table 21: Execution-level rejection statistics from additional evaluations.
Component
Setting
Agents
InternVL3.5-2B/8B, FP16, one call per player turn, at most 300 new tokens, temperature 0.7, top- p 0.9, no regeneration; rule checker disabled
Retrieval
TF-IDF with cosine similarity over the 10 rulebook entries, query “multi-unit actions turn structure”, top 3
Prompt
Up to 5 own units listed per decision
Game
7×7 map, 16-turn cap, Farmer HP 5, city HP equal to the faction’s combat-unit HP, battle multiplier 1, city production 3 per action, movement 1 (Empire) and 2 (Cavalry), Empire starts with one Farmer and one Soldier and Nomads with two Cavalry
GP optimizer
scikit-optimize 0.10.2: Gaussian process with constant amplitude times ARD Matérn-5/2 kernel, Gaussian noise, normalized targets; 10 random initial points; acquisition optimized with L-BFGS; EI ( ξ=0.01 ); LCB runs use κ=1.96
Evaluating strategic decision-making in LLM-based agents requires generative, competitive, and longitudinal environments, yet few benchmarks provide all three, and fewer still offer evaluation signals rich enough for long-horizon, multi-agent play. We introduce CivBench, a benchmark for LLM strategists (a model under an agentic setup) in multiplayer Civilization V. Because terminal win/loss is too sparse a signal in games spanning hundreds of turns and multiple opponents, CivBench trains estimators of a state-value function, the probability that a player eventually wins given the game state. cross 307 games with 7 LLMs and multiple CivBench agent conditions, we demonstrate CivBench's potential to estimate strategic capabilities in Civilization V as an unsaturated benchmark, reveal model-specific effects of agentic setup, and outline distinct strategic profiles not visible through outcome-only evaluation.
Can LLM agents carry a hard real-world problem from raw data to a system that holds its own against people? We study this in Hearthstone Battlegrounds, an eight-player auto-battler with stochastic shops and battles, hidden information and nearly a thousand interacting cards. We present AutoTavern, a self-improving research system: specialised LLM research agents share a typed research memory, every claim passes verification gates, and live games close the loop, so the agents improve the simulator, opponents and policy and, from the same feedback, their own research process; a human sets goals and rules on ambiguous mechanics. In 20 days AutoTavern built TavernGym, the first faithful, full-content reproduction of the live game, and delivered 23 versions of TavernAgent, trained with PPO against opponents calibrated to real games. TavernAgent plays complete games on the live ladder by itself: its rating rose from 6,000 to about 15,000 and, on the public leaderboard's measure, its last 20 records average a placement of 1.60 with 80% first places, better than the server's ten highest-rated players on every statistic (2.34, 59%). TavernBench, 960 tasks in 40 scenarios, shows that the system the agents built is the strongest player: TavernAgent answers 91% of the play tasks against at most 63% for frontier LLMs, and decides in 0.03 s where they need up to 26 s; within the second a live turn allows, it keeps 90% and no LLM more than 26%. LLM agents build far better than they play. We will open-source TavernBench and TavernGym at https://github.com/MCG-NJU/AutoTavern.
Recent advances in game AI, such as AlphaZero and Athénan, have achieved superhuman performance across a wide range of board games. While highly powerful, these agents are ill-suited for human-AI interaction, as they consistently overwhelm human players, offering little enjoyment and limited educational value. This paper addresses the problem of balanced play, in which an agent challenges its opponent without either dominating or conceding. We introduce Minibal (Minimize & Balance), a variant of Minimax specifically designed for balanced play. Building on this concept, we propose several modifications of the Unbounded Minimax algorithm explicitly aimed at discovering balanced strategies. Experiments conducted across seven board games demonstrate that one variant consistently achieves the most balanced play, with average outcomes close to perfect balance. These results establish Minibal as a promising foundation for designing AI agents that are both challenging and engaging, suitable for both entertainment and serious games.
Quentin Cohen-Solal, Tristan Cazenave
LAMSADE, Universit´e Paris-Dauphine, PSL, CNRS, Paris, France