CoupVisor: Strategy Optimization by Round and Challenge Decision Support

Cris Huynh

Fig. 1: The CoupVisor pipeline. Public events enter from any of three sources, manual turn entry, replay of a recorded game, or the synthetic simulator, and are applied by the engine, which maintains the public state: coins, remaining influence, revealed roles, the current claim, and the last public action. The belief tracker then updates the hidden-role distribution for every player, and the advisor combines public state and belief into a player-facing recommendation. Because all three input sources produce the same event schema, the advice path is identical in live play, in replay, and during policy training.

Fig. 2: Round-by-round strategy optimization loop. Each round begins by estimating hidden roles from the current belief state, then enumerates the actions that are legal given the player’s coins and influence, scores each candidate by expected utility against the risk that it is blocked or challenged, and returns a ranked suggestion. The legality filter runs before scoring, so constraints such as the forced Coup at ten coins remove options rather than merely penalising them.

Fig. 3: Challenge decision logic. On observing a claim the advisor reads

p_{\mathrm{truth}}

from the belief model, checks how many copies of the claimed role remain unaccounted for, and applies the threshold rule. The two branches lead to Challenge and Do not challenge , each of which is turned into a human-readable advice sentence carrying the probability and the threshold that produced it, so the recommendation can be audited rather than merely obeyed.

Fig. 4: Hidden-role belief update pipeline. An incoming event multiplies the score of the role it implicates by the corresponding likelihood ratio, all scores are then damped by deck pressure and normalised to a distribution, and finally reveal constraints are applied. The two modes diverge only at the last step: in Standard mode duplicate roles in one hand are permitted, so a revealed role can still carry non-zero hidden probability, whereas in Strict no-duplicate-hand mode a player down to a single hidden influence has the revealed role forced to zero and the remainder renormalised.

Fig. 5: Single-turn sequence. The turn opens with an action and its implied role claim, after which the table may block, challenge the action, or challenge the block. Each branch produces its own events, and a challenge resolves before the action or counteraction it targets. Every arrow that carries a claim or a reveal triggers a belief update and a fresh advisor recommendation, which is why a single turn can generate several decision points rather than one.

Step	Player	$P(\mathrm{Duke})$	$P(\mathrm{Assassin})$	$P(\mathrm{Captain})$	$P(\mathrm{Ambassador})$	$P(\mathrm{Contessa})$
0	A	0.20	0.20	0.20	0.20	0.20
0	B, C, D	0.20	0.20	0.20	0.20	0.20
1	A	0.55	0.15	0.10	0.10	0.10
1	B, C, D	0.15	0.20	0.20	0.20	0.25
2	A	0.30	0.40	0.10	0.10	0.10
2	B	0.15	0.15	0.20	0.20	0.30

TABLE I: Belief evolution over the worked example. Step 0 is the uniform prior, Step 1 follows A’s Duke claim, Step 2 follows A’s Assassin claim.

Step	Event	Advisor update
1	A claims Duke (Tax)	Increase $P_{A}(\mathrm{Duke})$
2	No challenge	Belief remains uncertain
3	A claims Assassin	Reduce joint-claim consistency
4	Decision point	Compute $p_{\mathrm{truth}}$ , compare EV

TABLE II: Event sequence and corresponding advisor update.

Fig. 6: Training and evaluation pipeline. Simulated games are flattened into state, action, reward, next-state transitions, which feed both the behavior cloning head and the offline DQN. Both learned policies and the non-learning baselines are then replayed through the same evaluation harness on the same transitions, so that reward per episode, reward per step, and action-match rate are computed identically for every policy.

Fig. 7: User interaction flow. The user loads or enters a game, steps through it event by event, and at each step sees the reconstructed public state, the hidden-role belief table, and the advisor recommendation for any claim currently on the table. Advice and validation warnings are presented side by side, so that a probabilistic judgement is never confused with a hard rule violation.

Seed	Aggressive-heavy	Honest-heavy
101	50 games	50 games
202	50 games	50 games
303	50 games	50 games
404	50 games	50 games
505	50 games	50 games
Total	250 games	250 games

TABLE III: Primary balanced matrix (seed

\times

condition).

Metric	Mean	Std	CI low	CI high
Challenge F1	0.4236	0.1409	0.3363	0.5110
Outcome accuracy	0.5694	0.0242	0.5544	0.5844
Outcome score	0.1428	0.0486	0.1127	0.1730

TABLE IV: Advisor metrics across seed-condition cells.

Condition	Challenge F1	Outcome acc.	Outcome score
Aggressive-heavy	0.5641	0.5619	0.1239
Honest-heavy	0.2831	0.5769	0.1617

TABLE V: Condition comparison by bot mix (mean across seeds).

Condition	Weighted score
Aggressive-heavy	0.5628
Honest-heavy	0.4594

TABLE VI: Weighted ranking score by condition.

Mode	Challenge F1	Outcome acc.	Outcome score
Standard	0.4236	0.5694	0.1428
Strict no-duplicate	0.4190	0.5684	0.1408
$\Delta$ (Strict $-$ Std.)	$-0.0046$	$-0.0010$	$-0.0020$

TABLE VII: Standard vs strict no-duplicate-hand belief mode.

Mode	Challenge F1	Outcome acc.	Outcome score
Advisor on	0.4236	0.5694	0.1428
Advisor off	0.0000	0.8192	0.6424
$\Delta$ (Off $-$ On)	$-0.4236$	$+0.2498$	$+0.4996$

TABLE VIII: Advisor policy ablation (off = never-challenge baseline).

Policy	Reward/episode	Reward/step	Action match
DQN	$-0.2868$	$-0.0179$	0.1905
Random	$-0.6004$	$-0.0375$	0.1415
Behavior cloning	$1.0412$	$0.0651$	0.3980

TABLE IX: Policy benchmark under the original shaped reward (

\Delta

coins

+

\Delta

alive

+

action-match).

Policy	Reward/episode	Reward/step	Winner match
DQN (win-reward)	6.74	0.248	0.373
Belief-EV heuristic	3.68	0.135	0.263
Behavior cloning	3.38	0.124	0.330
Honest (baseline)	2.84	0.104	0.244
Rule-based heuristic	2.42	0.089	0.264
Pressure heuristic	2.30	0.085	0.245

TABLE X: Policy benchmark under win-aligned reward (

+1

when the chosen action matches the winner’s logged action). 100 games; 1000 training episodes for DQN.

Episodes	Avg ep. reward	Avg ep. steps	Final $\varepsilon$
10	12.700	22.60	0.95111
50	14.580	24.46	0.77213
100	14.840	24.46	0.60577
200	15.045	24.46	0.36695
500	16.278	24.46	0.08157
1000	17.793	24.46	0.00500

TABLE XI: DQN training summary under the win-aligned reward.

Fig. 8: Advisor metrics across the ten seed-condition cells. Each metric is plotted with its spread over cells, which makes visible the contrast reported in Table IV : outcome accuracy sits in a narrow band while challenge F1 varies widely, so the instability is confined to the challenge decision rather than affecting the advisor as a whole.

Fig. 9: Advisor performance split by opponent condition. The aggressive-heavy and honest-heavy pools are shown side by side for each metric. Challenge F1 separates sharply between the two while outcome accuracy barely moves, identifying opponent composition as the source of the F1 variance in Fig. 8 .

Fig. 10: DQN reward curve under the original shaped reward, 100 episodes. Per-episode reward oscillates between roughly

-6

and

+4

, and the rolling mean stays inside

[-3,+0.5]

with no upward trend, so the policy is not learning. This flat curve is what motivated changing the reward rather than the algorithm. Corresponds to Table IX .

Fig. 11: DQN reward curve under the win-aligned reward, 1000 episodes. The rolling mean rises from about

1.5

at episode 20 to about

5.9

by episode 1000, a steady climb rather than the flat band of Fig. 10 , confirming that the same architecture does learn once the reward credits winner-matching actions. Corresponds to Table X .

Fig. 12: Policy benchmark summary. Every policy is scored on the same transitions, so the bars are directly comparable. Under the shaped reward the learned and heuristic policies cluster near or below zero while behavior cloning leads, the ordering that the win-aligned rerun in Table X subsequently inverts.

Fig. 13: Exploration-mode ablation under identical seeds, architecture, buffer, and reward. Left: running average episode reward. Middle: mean per-step entropy of the action distribution in nats. Right: action-visitation histogram. The

\varepsilon

-greedy entropy curve is monotone in

\varepsilon

and so is fixed entirely by the schedule, whereas the Boltzmann entropy curve also responds to the spread of learned action values and can plateau above the schedule floor when several actions have comparable value. The two mechanisms therefore allocate exploration differently even when their scalar schedules agree, which is why entropy rather than the schedule parameter is the quantity worth reporting.

Case	Evidence	System output	Ground truth and issue
E1	game 339, event 48 (Tax, Duke by P2)	Challenge ; $p_{\mathrm{truth}}=1.93\mathrm{e}{-34}$ , thr. $0.25$	Challenge result is a win for the claimant, so the claim was true. Overconfident false positive.
E2	game 231, event 1 (Steal block, Ambassador by P2)	Do not challenge ; $p_{\mathrm{truth}}=0.20$ , thr. $0.20$	Claimant loses, so the claim was false. Borderline equality missed a profitable challenge.
E3	game 158, event 7 (Steal, Captain by P3)	Do not challenge ; $p_{\mathrm{truth}}=0.2503$ , thr. $0.25$	Claimant loses, so the claim was false. Near-threshold miss.
E4	game 417, event 49 (Foreign Aid block, Duke by P4)	Challenge ; $p_{\mathrm{truth}}=8.98\mathrm{e}{-34}$ , thr. $0.35$	Claim was true. Block-claim handling is calibration-sensitive.
E5	DQN trace, game 106, step 12	DQN chose Exchange holding 11 coins	Legal action is Coup . In 10+ coin states DQN chose non-Coup 151 of 181 times, so forced-Coup alignment is weak.

TABLE XII: Representative failure cases from the evaluation run.

Parameter	Value
Network	2 hidden layers, width 128, ReLU
Optimiser	Adam, learning rate $1\times 10^{-3}$
Loss	Mean squared Bellman error
Discount factor $\gamma$	0.98
Batch size	64
Replay buffer size	10,000

Action	Effect	Can be blocked by
Income	Take 1 coin	Nobody
Foreign Aid	Take 2 coins	Duke
Coup	Pay 7 coins, target loses an influence	Nobody

Character	Action	Effect	Can be blocked by
Duke	Tax	Take 3 coins	Nobody
Assassin	Assassinate	Pay 3 coins, target loses an influence	Contessa
Captain	Steal	Take 2 coins from another player	Captain, Ambassador
Ambassador	Exchange	Draw 2 from the Court deck, return 2	Nobody

Character claimed	Blocks	Result
Duke	Foreign Aid	Target receives no coins that turn
Contessa	Assassination	Assassination fails, fee stays spent
Captain or Ambassador	Steal	Thief receives no coins that turn

CoupVisor: Strategy Optimization by Round and Challenge Decision Support

Abstract

Figures & tables

Appendix

Game-Playing Agents

Scalable Decision Making for Games of Imperfect Information

StratFormer: Adaptive Opponent Modeling and Exploitation in Imperfect-Information Games

Predicting Drafted Deck Strength for "Magic: the Gathering"

Abstract

Game-Playing Agents

Explore similar work

Scalable Decision Making for Games of Imperfect Information

StratFormer: Adaptive Opponent Modeling and Exploitation in Imperfect-Information Games

Predicting Drafted Deck Strength for "Magic: the Gathering"