Deducing whodunit proves challenging for LLM agents. In this paper, we implement a text-based multi-agent version of the classic board game Clue as a rule-based testbed for evaluating multi-step deductive reasoning, with six agents drawn from GPT-4o-mini and Gemini-2.5-Flash. We further investigate whether fine-tuning on structured logic puzzles transfers to improved in-game reasoning and gameplay. Across 18 simulated games, agents achieve only four correct wins, indicating difficulty in maintaining consistent deductive reasoning over the course of a full game. Additionally, we find that fine-tuning does not reliably improve performance and, in some cases, appears to increase reasoning volume without improving reasoning precision.
Figures & tables
Figure 1: Clue gameplay diagram illustrating Player 1’s turn. In this example, Player 1 made a suggestion (Mrs. White, Kitchen, Rope) and Player 2 did not have any matching cards to the suggestion, so then it moved to Player 3. Since Player 3 had a matching card, he revealed it to Player 1 (only). Player 1 can then rule out “Rope” as the correct weapon and Players 2, 4, 5, and 6 know that Player 3 must have Mrs. White, Kitchen, or Rope.
Figure 2: Mind Bender fine-tuning example (adopted from original).
Outcome
Reasoning
Reliability
Model
Wins
Rank
Acc.
Ded. Correct
Ded. Incorrect
Fallbacks
GPT-4o-mini (base)
5
2.96
0.29
10.1
2.7
0.3
GPT-4o-mini (FT)
3
4.00
0.19
13.1
3.0
4.8
Gemini-2.5-Flash (base)
2
3.92
0.24
9.1
2.9
8.1
Gemini-2.5-Flash (FT)
2
3.25
0.28
6.9
2.2
3.3
Table 1: Performance summary across 12 fine-tuned experiment games. Outcome : Wins (1st place), mean finishing position (Rank; 1=best, 6=worst), and normalized accusation accuracy (0–1). Reasoning : Average number of correct and incorrect deductions per game. Reliability : Average fallback (reprompt) failures per game.
Figure 3: Average correct and incorrect deductions per game by model in both the Baseline experiment and the Fine-tuned (FT) experiment. GPT-4o-mini (FT) makes the most deductions in both categories but achieves the worst accusation accuracy, while Gemini-2.5-Flash (FT) deduces least yet performs best.
Figure 4: Accusation accuracy per player per game. Each cell shows the number of correctly identified solution cards.
Figure 5: Average known cards over rounds in the fine-tuned experiment (12 games). Knowledge is computed as hand cards + shown cards + correct deductions, capped at 18. GPT-4o-mini (FT) accumulates information fastest but achieves the worst accusation accuracy, while Gemini-2.5-Flash (FT) gathers less information but reasons more effectively.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Winning reasoning from Gemini-2.5-Flash player on their first turn.
Figure 7: Winning reasoning from GPT-4o-Mini player on their ninth turn.