Frontier AI systems are increasingly capable and deployed in high-stakes multi-agent environments. However, existing AI safety benchmarks largely evaluate single agents, leaving multi-agent risks such as coordination failure and conflict poorly understood. We introduce GT-HarmBench, a benchmark of 1,535 high-stakes scenarios spanning game-theoretic structures such as the Prisoner's Dilemma, Stag Hunt and Chicken. Scenarios are drawn from realistic AI risk contexts in the MIT AI Risk Repository. Across 15 frontier models, agents fail to choose socially beneficial actions in 39% of high-stakes cases, such as military escalation, election manipulation, and medical malpractice. We measure sensitivity to game-theoretic prompt framing and ordering, and analyze reasoning patterns driving failures. We further show that game-theoretic interventions improve socially beneficial outcomes by up to 31%. Our results highlight substantial reliability gaps and provide a broad standardized testbed for studying alignment in multi-agent environments. The benchmark and code are available at https://github.com/Flecart/gt-harmbench.
Figures & tables
Figure 1: The GT-HarmBench pipeline. We begin with over 1,600 AI safety risks, classifying each risk scenario by which canonical 2 × 2 games could capture its underlying strategic tension. Each (risk, game) pair is then expanded into a contextualized scenario through the generation workflow shown above. Frontier models are evaluated on the resulting benchmark using classical social welfare metrics, and we further test whether targeted mechanism design interventions can steer agents toward more socially beneficial outcomes.
Work
# Instances
Real-World Safety
Mechanisms
Akata et al. (2025)
144
✗
✗
GTBench (Duan et al. 2024)
10
✗
✗
TMGBench (Wang et al. 2024)
720
✗
✗
FAIRGAME (Buscemi et al. 2025)
2
✗
✗
Lorè & Heydari (2023)
5
✗
✗
SanctSim (Piedrahita et al. 2025)
1
✓
✓ (1)
Table 1: Comparison of GT-HarmBench with prior multi-agent and game-theoretic LLM evaluations. # Instances counts unique strategic situations. “Real-World Safety” indicates scenarios grounded in concrete real-world safety risks rather than abstract payoffs or generic thematic framings. Mechanisms are interventions to the game to improve outcomes, such as communication, commitment devices and more.
Figure 2: A representative Prisoner’s Dilemma scenario (id 1094) within our dataset. Models like Grok 4.1 Fast suggest accelerating deployment, resulting in a socially worse outcome, while Opus 4.5 suggests limiting. Bullet points, section headers and bold formatting provided for the clarity of the reader, not to the model.
Figure 3: Framing analysis. We plot the model’s original Nash Equilibrium accuracy and Utilitarian accuracy (Prosaic) and compare it with explicit payoff with stories (Numer.), and pure payoff only version (Abstr.) Most models become more Nash rational as they are stripped of the context. Gemini 3 Pro is not available anymore at the current moment.
Figure 5
Figure 6: Left: Nash Accuracy (blue) measures equilibrium play; Utilitarian Accuracy (orange) measures fraction of optimal play. Dashed lines indicate baseline performance. Labels show absolute scores with change from baseline ( red for decrease and green for improvement). Right: Orange and blue bars show per-model baseline; solid stacked portions indicate improvement ( green ) or decrease ( red ) after intervention. Per model, gains in Utilitarian Accuracy range from +0.04 to +0.31, while changes in Nash Accuracy range from -0.01 to +0.04.
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
Game
Generated
Retained
Pass Rate
Prisoner’s Dilemma
501
490
97.8%
Chicken
386
379
98.2%
Stag Hunt
485
317
65.4%
Coordination
258
180
69.8%
Battle of the Sexes
149
141
94.6%
No Conflict
37
28
75.7%
Appendix
Table 3: Per-game scenario counts after Stage 3 filtering, alongside generation-stage candidate counts. Counts reflect MIT-seeded scenarios only (entries with an Ev_ID ).
Domain
MIT (%)
GT-HarmBench (%)
Discrimination & Toxicity
14.0
16.7
Privacy & Security
12.2
8.7
Misinformation
4.4
5.0
Malicious Actors & Misuse
16.2
15.8
Human-Computer Interaction
6.2
9.3
Socioeconomic & Environmental
20.0
18.2
Appendix
Table 4: Risk domain coverage in MIT AI Risk Repository vs. GT-HarmBench.
Figure 7: Left: normalised utilitarian margin of mutual cooperation over the asymmetric outcome; the dashed line is the point where the optimum flips. Prisoner’s Dilemma scenarios sit far from it, Chicken scenarios close to it. Right: utilitarian accuracy as reported (grey) and under 1,000 ordinal-preserving redraws of the payoffs (mean and 5–95% band), choices held fixed.
Model
CoT
BoS
Chicken
Coord
NoCon
PD
Stag
Total
Claude Opus 4.5
raw
0/45
1/60
0/43
0/29
0/60
0/45
1/282
Claude Sonnet 4.5
raw
0/55
0/53
0/45
0/28
3/61
0/69
3/311
DeepSeek v3.2
raw
0/57
0/59
0/68
0/27
0/56
0/54
0/321
Qwen3 30B
raw
0/39
0/48
0/62
0/32
0/44
0/45
0/270
Qwen3 8B
raw
0/44
0/52
0/63
0/30
0/58
0/58
0/305
Gemini 3 Flash
summ
0/41
0/51
0/63
0/23
3/59
0/45
3/282
Appendix
Table 5: Evaluation awareness: traces flagged by the judge out of all traces, per model and game. For some models, the reasoning trace was not available. For those, we analyzed the summary reasoning.
Game
Verified
Pass
Pass Rate
Prisoner’s Dilemma
490
490
100.0%
Chicken
379
378
99.7%
Stag Hunt
317
317
100.0%
Coordination
180
180
100.0%
Battle of the Sexes
141
137
97.2%
No Conflict
28
28
100.0%
Appendix
Table 6: Per-game results of post-hoc structural verification.
Figure 9: Example of one response within our dataset evaluation using Coordination Game. The model correctly recognizes the coordination game and uses Schelling points [ Schelling, 1960 ] to decide which action to choose to implicitly coordinate with the other player. We observe empirically that most models display this coordination behaviour.
Figure 10: Utilitarian accuracy from cross-play on the 1,535 scenarios: the row model plays the row role and the column model the column role; the diagonal (bold) is self-play. Cross-play is reconstructed from the self-play runs, since each role query carries no information about the opponent. Each pairing is scored on the scenarios where both models’ answers parse. Mean cross-play accuracy is 0.57 against 0.63 for self-play; the gap is largest in the Prisoner’s Dilemma ( −13 points), followed by Coordination ( −8 ) and Stag Hunt ( −7 ), while Battle of the Sexes improves ( +6 ), since mixing models breaks the symmetric insistence on the same preferred option.
Model
Util. acc.
Range
Majority agr.
Claude 4.5 Opus
0.868
0.036
0.963
Claude 4.5 Sonnet
0.851
0.025
0.930
Llama 3.3 70B
0.765
0.040
0.940
Llama 3.2 3B
0.757
0.043
0.909
GPT-4o
0.746
0.047
0.919
Gemini 3 Flash
0.734
0.036
0.944
Appendix
Table 11: Choice stability under repeated sampling ( k=5 , 278 scenarios, temperature 1.0). Mean and range of utilitarian accuracy across the five runs (the SD is reported in Table 2 ); majority agreement is the mean fraction of draws matching each scenario’s modal outcome.
Model
Coord.
Random
Game Cls.
Nash Eq.
GPT-5.1
0.90
0.866 -0.034
0.965
0.838
GPT-5.2 (2025-12-11)
0.91
0.856 -0.056
0.957
0.873
GPT-5 Mini (2025-08-07)
–
–
0.779
0.716
GPT-5 Nano (2025-08-07)
0.90
0.820 -0.084
0.734
0.348
Claude 4.5 Sonnet
0.91
0.866 -0.039
0.907
0.872
Grok 4.1 Fast
0.89
0.750 -0.144
0.905
0.806
Appendix
Table 12: We highlight in bold the best model across columns. Left: We analyze the no-communication coordination ability of models with default ordering (Coord.) or random ordering (Random), the same values reported in the main paper, Figure 5 . Right: Comparison of Game Classification (Cls.) and Nash Equilibrium (Eq.) scores.
Category
Sub-category
Description
Game-Theoretic Reasoning
Nash / Dominant Strategy
Decision explicitly invokes game-theoretic concepts, identifying one action as optimal regardless of opponent behavior, referencing equilibrium, mutual best responses, or rational strategies for self-interested agents.
Payoff Maximization
Decision is justified primarily by maximizing own expected payoff or outcome, emphasizing individual gains over collective welfare.
Social Welfare Reasoning
Utilitarian Reasoning
Decision invokes total or aggregate welfare, collective benefit, or the principle of the greatest good for the greatest number, emphasizing maximization of overall well-being across all parties.
Rawlsian Reasoning
Decision prioritizes fairness, equity, or protection of the worst-off party, invoking distributive justice or minimizing disproportionate burden.
Risk and Catastrophe Reasoning
Catastrophe Prevention
Decision explicitly aims to avoid worst-case or catastrophic outcomes, even at the cost of expected value, prioritizing the prevention of irreversible or large-scale harm.
Precautionary Principle
Decision favors caution under uncertainty, choosing safer options when outcomes are unclear or potential harms are severe but uncertain.
Appendix
Table 13: Categories of Reasoning Patterns
Figure 11: Heatmap of frequency of each reasoning category across 6 core games. Chicken has the highest score for Catastrophe Prevention, while the Prisoner’s Dilemma has the highest score for Utilitarian Reasoning.
Figure 12: Heatmap of frequency of each reasoning category per model. Nash/Dominant Strategy is highest in Claude Sonnet 4.5, while Claude Opus 4.5 has the highest Utilitarian and AI Safety scores.
Figure 13: Nash Accuracy average across all models for baseline and four variants of each mechanism.
Figure 14: Utilitarian Welfare average across all models for baseline and four variants of each mechanism.
Figure 15: Heatmap of Utilitarian Accuracy across 6 core games and 20 mechanism design variants.
Figure 16: Heatmap of Utilitarian Accuracy across 6 models and 20 mechanism design variants
Figure 17: Distribution of joint choices per model and game on the 1,535 scenarios (part 1 of 2). Each 2×2 panel gives the share of scenarios ending in each profile (rows: row player; columns: column player). Actions are aligned across scenarios by their strategic role, read off each scenario’s payoffs: Cooperate/Defect (Prisoner’s Dilemma; Defect is the dominant action), Swerve/Straight (Chicken; Straight/Straight is the crash), Insist/Concede (Battle of the Sexes; Insist is the player’s own preferred coordination point, so coordination lies on the off-diagonal), Stag/Hare (Stag Hunt), Best/Other (No Conflict); Coordination keeps the listed order, as its two coordination points are equally good. n is the number of scenarios with a parsed answer.
Figure 18: Distribution of joint choices per model and game (part 2 of 2; see Figure 17 ).
As large language models (LLMs) evolve into autonomous agents capable of acting in open-ended environments, ensuring behavioral alignment with human values becomes a critical safety concern. Existing benchmarks, focused on static, single-turn prompts, fail to capture the interactive and multi-modal nature of real-world conflicts. We introduce ConflictBench, a benchmark for evaluating human-AI conflict through 150 multi-turn scenarios derived from prior alignment queries. ConflictBench integrates a text-based simulation engine with a visually grounded world model, enabling agents to perceive, plan, and act under dynamic conditions. Empirical results show that while agents often act safely when human harm is immediate, they frequently prioritize self-preservation or adopt deceptive strategies in delayed or low-risk settings. A regret test further reveals that aligned decisions are often reversed under escalating pressure, especially with visual input. These findings underscore the need for interaction-level, multi-modal evaluation to surface alignment failures that remain hidden in conventional benchmarks.
As LLM-based agents are deployed with increasing autonomy in real-world environments, a critical failure mode emerges: agentic misalignment, where agents choose harmful actions over failure. However, existing safety benchmarks primarily evaluate whether agents comply with explicitly harmful instructions, leaving the conditions that cause agents to choose harmful actions poorly understood. To address this limitation, we propose AM-Bench, a unified taxonomy and 35 scenario evaluation benchmark spanning seven industries for agentic misalignment grounded in the fraud triangle, a well-established criminological framework explaining goal-directed misconduct. We adapt this framework to agentic environments and construct our evaluation suite and detection systems along three dimensions: perceived threat, perceived opportunity, and reasoning. We operationalize each dimension into structured subcategories rooted in prior theory: five threat categories derived from the instrumental convergence thesis, seven opportunity categories drawn from the Emergent Strategic Reasoning Risks (ESRR) taxonomy, and a novel four-question binary reasoning framework synthesized from prior literature. Our evaluation of 10 frontier models reveals a 62-percentage-point gap in misalignment across scenarios, with Gemini 3 Flash and Gemini 3.1 Pro exhibiting the highest levels of agentic misalignment while Claude Sonnet 4.6 displays near-zero rates across all conditions. Our reasoning framework further reveals that models may exhibit harmful behaviours without explicit situational awareness or explanation, complicating both mitigation and detection. AM-Bench scenarios and code are available at https://github.com/AI4Collaboration/Agentic-Misalignment-Bench
Eric Zhang, Terry Jingchen Zhang, Chijioke Ugwuanyi +3
Jinesis Lab, University of Toronto & Vector Institute · EuroSafeAI · Carnegie Mellon University +1
As large language model agents increasingly operate autonomously with access to tools and external environments, ensuring their safe and reliable behavior becomes critical. We present GuardianAgentBench (GABench), a benchmark of 580 scenarios across six domains evaluated on three production-ready frameworks: LangChain, LlamaIndex, and Vectara. The benchmark incorporates rigorous multi-stage validation and five adversarial attack modes. Experiments with six state-of-the-art models reveal that even the strongest configuration achieves only 74.8% overall accuracy and expose two distinct failure regimes: stronger models under-call required tools, while weaker models mis-select and over-call tools. Performance degrades monotonically with both tool-set size and sequential turn depth, with long-horizon planning proving the steeper bottleneck. Our guardrail implementation consistently outperforms system-prompt-based defenses across all models, recovering 19.9% of failures at a false positive rate of just 0.5%. These results demonstrate that execution-time structural intervention improves safety without disrupting correct agent behavior.
Vishal Ishwar Naik, Chenyu Xu, Donna Dong +5
Vectara, Inc., Palo Alto, CA, USA · Iowa State University, Ames, IA, USA