RuleSmith: Multi-Agent LLMs for Automated Game Balancing

Ziyao Zeng; Hao Wang; Chen Liu; Youheng Yao; Jingcheng Ni; Tianyu Liu; Xiatao Sun; Fengyu Yang

cs.LGFeb 5, 2026

RuleSmith: Multi-Agent LLMs for Automated Game Balancing

Authors: Ziyao Zeng, Hao Wang, Chen Liu, Youheng Yao, Jingcheng Ni, Tianyu Liu, Xiatao Sun, Fengyu Yang,

Organizations: Yale University, New Haven, CT, USA · Texas A&M University, College Station, TX, USA

Abstract

Game balancing is a longstanding challenge requiring repeated playtesting, expert intuition, and extensive manual tuning. We introduce RuleSmith, a framework that couples multi-agent LLM self-play with surrogate-based optimization over a parameterized rule space. As a proof of concept, we build CivMini, a simplified civilization-style game with two asymmetric factions governed by 12 tunable parameters. LLM agents interpret textual rulebooks and game states to propose actions that are validated by the game engine, yielding noisy estimates of faction win rates. Our primary implementation uses Gaussian-process Bayesian optimization with Expected-Improvement-based adaptive sampling to allocate more evaluation games to promising candidates. RuleSmith consistently finds near-balanced configurations: three independent optimization runs reach near-even empirical win rates in independent 200- and 300-game evaluations, and additional experiments with NTBEA and TPE show that the same self-play interface supports different search components. Cross-model evaluations and small human studies provide preliminary evidence of transfer across player populations. RuleSmith produces interpretable numerical rule adjustments without task-specific policy training. Project and code: https://adonis-galaxy.github.io/RuleSmith-website/

Figures & tables

Figure 1: Overview of RuleSmith. RuleSmith treats asymmetric game balancing as optimization over game rollouts using multi-agent LLMs given a natural-language rulebook. It optimizes interpretable parameters such as unit HP, attack damage, resource gain, production cost, and scoring weights. For each candidate game parameter setting, two LLM agents play the game zero-shot, and their outcomes estimate the faction win rates and balance loss. A Bayesian optimizer then proposes new settings, while adaptive sampling allocates more games to promising candidates. This enables closed-loop balancing without task-specific policy training, reducing manual playtesting and parameter tuning. Figure generated by Nano Banana Pro.

Figure 2: Overview of the RuleSmith method. We represent an asymmetric, turn-based strategy game (CivMini) as a parameterized rule space $\theta\in\Theta$ , covering economy, combat, production, and scoring parameters. Given a candidate rule configuration $\theta_{t}$ , two role-specific LLM agents ( Empire and Nomads ) play $N_{t}$ self-play games in the CivMini environment, producing a balance loss $\mathcal{L}(\theta)=|w_{E}(\theta)-0.5|+|w_{N}(\theta)-0.5|+0.5\,w_{D}(\theta)$ , where $w_{E}$ , $w_{N}$ , and $w_{D}$ are the Empire win rate, Nomads win rate, and draw rate, respectively. A Bayesian optimizer maintains a surrogate model $g(\theta)$ over a continuous relaxation of the rule space and selects new candidates $\tilde{\theta}_{t+1}$ by maximizing an acquisition function. The number of games $N_{t}$ is adaptively determined based on the Expected Improvement: promising candidates receive more games for accurate evaluation. Each continuous proposal is mapped to a valid, discrete ruleset via a deterministic discretization operator $D(\cdot)$ before evaluation. Cartoons in this figure were generated using ChatGPT-5.2.

Parameter	Range	Prec.	Parameter	Range	Prec.
Economy			Production
Initial resources	$[2,10]$	int	Empire unit cost	$[2,10]$	int
Empire farmer gather	$[1,5]$	int	Nomads unit cost	$[2,10]$	int
Nomads kill gain	$[1,10]$	int
Combat			Scoring
Empire damage	$[1,5]$	int	Resource weight	$[0.1,0.5]$	0.1

Table 1: Optimizable parameters in CivMini. We optimize 12 parameters governing economy, combat, production, and scoring. Precision indicates the discretization granularity applied before evaluation.

Optimize $\backslash$ Evaluate	E 2B vs N 2B	E 2B vs N 8B	E 8B vs N 2B	E 8B vs N 8B
E 2B vs N 2B	48 $\mid$ 52	32 $\mid$ 68	73 $\mid$ 27	55 $\mid$ 45
E 2B vs N 8B	81 $\mid$ 19	47 $\mid$ 53	91 $\mid$ 9	75 $\mid$ 25
E 8B vs N 2B	37 $\mid$ 63	0 6 $\mid$ 94	52 $\mid$ 48	29 $\mid$ 71
E 8B vs N 8B	53 $\mid$ 47	24 $\mid$ 76	81 $\mid$ 19	51 $\mid$ 49

Table 2: Optimized winning chances of Empire and Nomads. Rows specify the agent pairing used to optimize rules; columns specify the evaluation pairing. Each cell reports Empire wins

\mid

Nomads wins over 100 held-out games using the best optimization checkpoint (empirical loss

\leq 0.1

). E / N denote the factions; subscripts denote model sizes. Greener cells indicate more even outcomes; empirical near-balance (

50\%\pm 5\%

) is bolded.

Configuration	Games	Empire (%)	95% CI (%)	$p$
Default (additional evaluation)	300	11.3	[8.2, 15.4]	$2.9\times 10^{-45}$
Optimized configuration (Table 10 )	500	47.6	[43.3, 52.0]	0.30
GP run 1	300	51.7	[46.0, 57.3]	0.60
GP run 2	300	49.3	[43.7, 55.0]	0.86
GP run 3	300	52.7	[47.0, 58.2]	0.39
NTBEA	300	48.0	[42.4, 53.6]	0.53

Table 3: Additional held-out evaluations. Empire win rates, Wilson 95% intervals, and two-sided exact binomial tests against 0.5. Counts are drawn from the evaluation logs; the test compares Empire and Nomads wins among decisive games (Appendix H ). The default batch is separate from the authors’ initial configuration. NTBEA/TPE replace the search component; GP-LCB replaces the acquisition rule.

Optimization LLM	Evaluation LLM	Empire%	Nomads%	$\mathcal{L}$
InternVL3.5-8B	InternVL3.5-8B	51	49	0.02
InternVL3.5-8B	Qwen3.5-9B	50	50	0.00
Qwen3.5-9B	InternVL3.5-8B	49	51	0.02
Qwen3.5-9B	Qwen3.5-9B	52	48	0.04

Table 4: Cross-LLM-family transfer. Empirical win rates meet the near-balance criterion (

50\%\pm 5\%

) when parameters optimized with one LLM family are evaluated by the other. All results use 100 evaluation games.

	Random Search	(1+1)-ES	BO (adaptive)
Win Rate	13 $\mid$ 87	26 $\mid$ 74	51 $\mid$ 49
	BO ( $N=16$ )	BO ( $N=32$ )	BO ( $N=64$ )
Win Rate	34 $\mid$ 66	61 $\mid$ 39	48 $\mid$ 52

Table 5: Original search and replication ablation. Each method runs for 100 iterations and is evaluated on 100 held-out games. Random Search and (1+1)-ES use 64 games per candidate; GP-BO uses either fixed replication or 16–64 games based on EI. These are single optimization runs.

Metric	First 100 games	All 300 games
Distinct three-turn opening sequences	86/100	229/300
Distinct first-turn action combinations	6	11
Action entropy, Empire / Nomads (bits)	2.03 / 1.65	2.04 / 1.64
Mean turn-5 Jaccard distance	0.62	0.62
Ended before the turn cap (a city destroyed)	63%	62%
Game length, mean (SD) in turns	10.8 (4.4)	11.1 (4.4)

Table 6: Trajectory diversity under identical rules (8B self-play, NTBEA configuration; Appendix I ). The 100-game column is a subset of the 300-game column, not an independent replication.

	Map size (turn limit)				Three factions		Map size optimized
	5 $\times$ 5 (16)	7 $\times$ 7 (16)	9 $\times$ 9 (32)	11 $\times$ 11 (32)	8B	2B	8B	2B
Win rate (%)	53 $\mid$ 47	51 $\mid$ 49	48 $\mid$ 52	51 $\mid$ 49	32 $\mid$ 33 $\mid$ 35	34 $\mid$ 33 $\mid$ 33	53 $\mid$ 47	51 $\mid$ 49
Balance loss	0.06	0.02	0.04	0.02	0.033	0.013	0.06	0.02

Table 7: Extended configurations. Empire

\mid

Nomads win rates (with Theocracy third for three factions) and balance loss for four map sizes with InternVL3.5-8B (turn limits in parentheses), three-faction balancing with four added Theocracy parameters (

\mathcal{L}_{3}

), and map size (5–9) as an additional optimizable parameter (Appendix F ). Map-size losses follow Eq. ( 3 ) from the reported win rates.

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Empire	Nomads	Empire%	Nomads%
InternVL3.5-8B	Random	100	0
Random	InternVL3.5-8B	0	100
InternVL3.5-2B	Random	100	0
Random	InternVL3.5-2B	0	100

Appendix

Table 8: LLM agent vs. random agent. LLM agents (InternVL3.5-8B and 2B) are evaluated against a uniformly random legal-action agent using parameters optimized by the respective LLM. The prompted LLM agents win 100% of games in every listed configuration.

		LLM-optimized params		Rule-optimized params
Empire	Nomads	Empire%	Nomads%	Empire%	Nomads%
InternVL3.5-8B	Rule-based	75	25	73	27
Rule-based	InternVL3.5-8B	22	78	38	62
InternVL3.5-2B	Rule-based	68	32	65	35
Rule-based	InternVL3.5-2B	31	69	46	54

Appendix

Table 9: LLM agent vs. rule-based agent. Win rates when LLM agents (InternVL3.5-8B and 2B) play against the rule-based agent under parameters optimized via LLM self-play (left) and rule-based self-play (right). All results use 100 evaluation games.

Economy			Production
Parameter	Initial	Opt.	Parameter	Initial	Opt.
Initial resources	6	10	Empire unit cost	7	9
Empire farmer gather	2	1	Nomads unit cost	3	8
Nomads kill resource gain	4	7
Combat			Scoring
Empire damage	4	4	Score per resource	0.1	0.1

Appendix

Table 10: Human-designed vs. RuleSmith-optimized parameters. The authors’ initial configuration is heavily skewed toward Empire (balance score 1.00). RuleSmith reduces the balance score to 0.02 through coordinated adjustments across all parameter categories.

Optimization LLM	Evaluation LLM	Empire%	Nomads%	$\mathcal{L}$
InternVL3.5-8B	Gemini 2.5 Flash	51	49	0.02
InternVL3.5-8B	Gemini 2.5 Pro	47	53	0.06
Qwen3.5-9B	Gemini 2.5 Flash	45	55	0.10
Qwen3.5-9B	Gemini 2.5 Pro	53	47	0.06

Appendix

Table 11: Evaluation with Gemini Flash and Pro. Parameters optimized with InternVL3.5-8B or Qwen3.5-9B meet the empirical near-balance criterion when evaluated with Gemini 2.5 Flash or Pro. The comparison varies model identity and does not isolate a reasoning mode.

Empire	Nomads	Empire%	Nomads%	$\mathcal{L}$
InternVL3.5-8B	Gemini 2.5 Flash	45	55	0.10
InternVL3.5-8B	Gemini 2.5 Pro	41	59	0.18
Gemini 2.5 Flash	InternVL3.5-8B	45	55	0.10
Gemini 2.5 Pro	InternVL3.5-8B	49	51	0.02

Appendix

Table 12: Asymmetric cross-model matchups. Parameters optimized with symmetric InternVL3.5-8B self-play are evaluated with different models on the two sides. Faction outcomes vary with the pairing, while both factions retain wins in the tested matchups.

	Empire	Nomads	Total
Wins	32	28	60
Win Rate	53.3%	46.7%	N/A

Appendix

Table 13: Human user study results. 4 participants, 6 pairwise matchups, 2 faction assignments each, 5 games per session, 60 games total. Empire win rate 53.3% and Nomads win rate 46.7% both fall within the near-balanced range (

50\%\pm 5\%

Participant	Human faction	vs. 2B	vs. 8B
P1	Empire	9/10	8/10
P1	Nomads	10/10	8/10
P2	Empire	8/10	7/10
P2	Nomads	9/10	9/10
P3	Empire	9/10	7/10
P3	Nomads	10/10	9/10

Appendix

Table 14: Human wins by participant and faction against InternVL3.5 agents. Each participant/faction cell has 10 games; totals pool the three participants.

Population	Empire wins	E (%)	E 95% CI (%)	$p$
Human vs. human	32/60	53.3	[40.9, 65.4]	0.70
Human vs. 2B	27/60	45.0	[33.1, 57.5]	0.52
Human vs. 8B	26/60	43.3	[31.6, 55.9]	0.37

Appendix

Table 15: Aggregate faction outcomes across small human-player studies. Wilson intervals and binomial

p

values summarize game-level totals under a working independence assumption. Repeated participants and learning limit population-level interpretation.

Parameter	LLM Agents	Random Agents	Rule-Based Agents
Economy
Initial resources	10	4	6
Empire farmer gather	1	1	2
Nomads kill resource gain	7	5	5
Combat
Empire damage	4	2	3

Appendix

Table 16: Optimized parameters: LLM agents, random agents, and rule-based agents. The three proxy player types yield substantially different balanced configurations, reflecting their distinct decision dynamics.

Parameter	E 2B vs N 2B	E 2B vs N 8B	E 8B vs N 2B	E 8B vs N 8B
Initial resources	10	2	10	10
Empire farmer gather	1	2	4	1
Nomads kill resource gain	6	9	1	7
Empire damage	4	2	2	4
Nomads damage	3	3	5	3
Empire soldier HP	9	14	16	9

Appendix

Table 17: Optimized game parameters under different LLM agent configurations. We report optimized parameters that achieve approximately 50/50 win rates across different LLM agent configurations. We also include the implied time-to-kill (TTK; number of attacks required to eliminate the opponent’s main combat unit) derived from damage and HP.

Parameter	R94	R98	Reported 8B/8B
Initial resources	8	4	10
Empire farmer gather	1	1	1
Nomads kill resource gain	8	7	7
Empire damage	2	2	4
Nomads damage	4	4	3
Empire soldier HP	14	14	9

Appendix

Table 18: Configurations used in the qualitative rollouts. R94 and R98 are archived optimization checkpoints, distinct from the reported 8B/8B configuration in Table 17 .

Figure 3: Visualization of a CivMini Game. Nomads conquer the Empire’s city on turn 12 (13 rounds, counting from 0). Cavalry bypass the Empire’s defense and sustain city pressure. Both agents use InternVL3.5-8B. This example is R94 game 13 (Table 18 ); city destruction determines victory irrespective of the displayed scores.

Figure 4: A rapid Nomads conquest on turn 6 (7 rounds, counting from 0). Cavalry Unit 0 strikes the Empire’s city on turns 3–6, dealing 4 HP per attack and destroying it. Checkpoint R98, game 4. (Final score: Empire 15.8 | Nomads 27.6)

Figure 5: A Nomads score victory after 16 rounds (turns 0–15). Continued cavalry production builds a numerical advantage, while the Empire deploys Farmers and Soldiers and loses units during the fighting. Checkpoint R98, game 1. (Final score: Empire 14.2 | Nomads 67.6)

Figure 6: The Empire withstands early Nomads aggression and wins at the 16-round turn cap. Soldiers defend through fighting on turns 3–15, while Farmers supply resources. Two Cavalry remain at the end; the Empire wins through the combined final scoring terms. Checkpoint R98, game 5. (Final score: Empire 31.6 | Nomads 23.4)

Figure 7: The Empire defends against a Nomads city rush beginning on turn 5. By turn 15, one Cavalry remains, while the Empire retains two Soldiers and two Farmers. Unit survival and resources contribute to its score victory. Checkpoint R98, game 13. (Final score: Empire 34.4 | Nomads 12.6)

Search / run	$n$	$k_{E}$	E (%)	N (%)	$\widehat{\mathcal{L}}$	E 95% CI	$p$
GP run 1	200	103	51.5	48.5	0.030	[44.6, 58.3]	0.72
GP run 1	300	155	51.7	48.3	0.033	[46.0, 57.3]	0.60
GP run 2	200	98	49.0	51.0	0.020	[42.2, 55.9]	0.83
GP run 2	300	148	49.3	50.7	0.013	[43.7, 55.0]	0.86
GP run 3	200	103	51.5	48.5	0.030	[44.6, 58.3]	0.72
GP run 3	300	158	52.7	47.3	0.053	[47.0, 58.2]	0.39

Appendix

Table 19: Full 200-/300-game re-evaluations.

k_{E}

is the number of Empire wins from the evaluation logs; none of these batches contains a draw. Losses use counts rather than rounded percentages. Each configuration’s two batches use separate rollout seeds.

Rate (%)	95% CI (%)	Rate (%)	95% CI (%)	Rate (%)	95% CI (%)
0	[0.0, 3.7]	6	[2.8, 12.5]	13	[7.8, 21.0]
22	[15.0, 31.1]	24	[16.7, 33.2]	26	[18.4, 35.4]
29	[21.0, 38.5]	31	[22.8, 40.6]	32	[23.7, 41.7]
33	[24.6, 42.7]	34	[25.5, 43.7]	35	[26.4, 44.7]
37	[28.2, 46.8]	38	[29.1, 47.8]	41	[31.9, 50.8]
45	[35.6, 54.8]	46	[36.6, 55.7]	47	[37.5, 56.7]

Appendix

Table 20: Wilson 95% intervals for every distinct win rate in the 100-game evaluations of Tables 2 , 4 , 5 , 8 , 9 , 10 , 11 , 12 , and 7 , under the 100-game protocol of Section 4.2 . A rate of

r

% corresponds to

r

wins. For no-draw two-faction outcomes, the Nomads interval is

[100-u,100-l]

when the Empire interval is

[l,u]

; for three factions, each faction’s rate is listed.

Setting	Attempted actions	Converted to PASS	Rate
8B agents, 500 games	33,834	1,126	3.33%
2B agents, 200 games	21,871	1,012	4.63%

Appendix

Table 21: Execution-level rejection statistics from additional evaluations.

Component	Setting
Agents	InternVL3.5-2B/8B, FP16, one call per player turn, at most 300 new tokens, temperature 0.7, top- $p$ 0.9, no regeneration; rule checker disabled
Retrieval	TF-IDF with cosine similarity over the 10 rulebook entries, query “multi-unit actions turn structure”, top 3
Prompt	Up to 5 own units listed per decision
Game	$7\times 7$ map, 16-turn cap, Farmer HP 5, city HP equal to the faction’s combat-unit HP, battle multiplier 1, city production 3 per action, movement 1 (Empire) and 2 (Cavalry), Empire starts with one Farmer and one Soldier and Nomads with two Cavalry
GP optimizer	scikit-optimize 0.10.2: Gaussian process with constant amplitude times ARD Matérn-5/2 kernel, Gaussian noise, normalized targets; 10 random initial points; acquisition optimized with L-BFGS; EI ( $\xi=0.01$ ); LCB runs use $\kappa=1.96$
Adaptive budget	Eq. ( 6 ), $N_{\min}=16$ , $N_{\max}=64$ , $\epsilon=10^{-3}$ , integer truncation

Appendix

Table 22: Settings for the additional runs (Sections 4.4 and 4.5 ). The main runs used the same game and agent settings.

Multi-Agent System Optimization

Multi-Agent System Optimization studies how to improve agent teams’ coordination, performance, resource use, and reliability.

See all 161 papers

CivBench: Progress-Based Evaluation for LLMs' Strategic Decision-Making in Civilization V
Apr 9, 2026John Chen, Sihan Cheng, Can Gurkan +1LLM Agent Evaluation AI Agent Benchmarks
Northwestern University
Build, Don't Play: Self-Improving LLM Research Agents Develop a Hearthstone Battlegrounds AI That Rivals Top Players
Oct 24, 2026Changlian Ma, Limin WangLLM Agent Self-Improvement Autonomous Research Agents
Nanjing University · Shanghai AI Laboratory
Minibal: Balanced Game-Playing Without Opponent Modeling
Mar 24, 2026Quentin Cohen-Solal, Tristan CazenaveGame-Playing Agents Human-AI Interaction
LAMSADE, Universit´e Paris-Dauphine, PSL, CNRS, Paris, France

Apr 9, 2026J/K move · Enter open · S save

CivBench: Progress-Based Evaluation for LLMs' Strategic Decision-Making in Civilization V

John Chen, Sihan Cheng, Can Gurkan, Mingyi Lin

Northwestern University

LLM Agent Evaluation AI Agent Benchmarks Game-Playing Agents Long-Horizon Agent Evaluation

Open paper AskPDF arXiv

Evaluating strategic decision-making in LLM-based agents requires generative, competitive, and longitudinal environments, yet few benchmarks provide all three, and fewer still offer evaluation signals rich enough for long-horizon, multi-agent play. We introduce CivBench, a benchmark for LLM strategists (a model under an agentic setup) in multiplayer Civilization V. Because terminal win/loss is too sparse a signal in games spanning hundreds of turns and multiple opponents, CivBench trains estimators of a state-value function, the probability that a player eventually wins given the game state. cross 307 games with 7 LLMs and multiple CivBench agent conditions, we demonstrate CivBench's potential to estimate strategic capabilities in Civilization V as an unsaturated benchmark, reveal model-specific effects of agentic setup, and outline distinct strategic profiles not visible through outcome-only evaluation.

Abstract

Multi-Agent System Optimization

Explore similar work

CivBench: Progress-Based Evaluation for LLMs' Strategic Decision-Making in Civilization V

Build, Don't Play: Self-Improving LLM Research Agents Develop a Hearthstone Battlegrounds AI That Rivals Top Players

Minibal: Balanced Game-Playing Without Opponent Modeling