LLMs as Strategic Actors: Behavioral Alignment, Risk Calibration, and Argumentation Framing in Geopolitical Simulations

Veronika Solopova; Viktoria Skorik; Maksym Tereshchenko; Alina Haidun; Ostap Vykhopen

cs.CLMar 2, 2026

LLMs as Strategic Actors: Behavioral Alignment, Risk Calibration, and Argumentation Framing in Geopolitical Simulations

Authors: Veronika Solopova, Viktoria Skorik, Maksym Tereshchenko, Alina Haidun, Ostap Vykhopen

Abstract

Large language models (LLMs) are increasingly proposed as agents in strategic decision environments, yet their behavior in structured geopolitical simulations remains under-researched. We evaluate six popular state-of-the-art LLMs alongside results from human results across four real-world crisis simulation scenarios, requiring models to select predefined actions and justify their decisions across multiple rounds. We compare models to humans in action alignment, risk calibration through chosen actions' severity, and argumentative framing grounded in international relations theory. Results show that models approximate human decision patterns in base simulation rounds but diverge over time, displaying distinct behavioural profiles and strategy updates. LLM explanations for chosen actions across all models exhibit a strong normative-cooperative framing centered on stability, coordination, and risk mitigation, with limited adversarial reasoning.

Figures & tables

Figure 1: Study design: four geopolitical simulations run with human MBA participants and six LLMs under identical prompts and structured action menus.

Simulation Name	Description	Actors
Middle East	Taking place in autumn 2025, the training focused on possible scenarios of international relations development between the US and regional actors. The simulation also included non-playable regional actors with pre-defined actions (e.g., Israel, Iran, Yemen, China).	EU/UK; Qatar; Saudi Arabia; USA; United Arab Emirates
US, China, Taiwan	Taking place in late 2025, the simulation focused on next steps in the US–China trade war and escalating geopolitical tensions.	Australia; China; Japan; South Korea; Taiwan; USA
Arctic	The simulation took place in early 2026 and focused on potential next steps in intensifying tensions between the US and the EU over Greenland, as well as broader Arctic regional dynamics.	Canada; China; EU/Nordics; Russia; USA
USA Wildfire (Climate)	The only single-round simulation, focusing on corporate, NGO, and governmental responses to a simulated wildfire disaster in the United States.	Amazon; American Red Cross; FEMA; Google and Microsoft; NOAA; State Power Utility

Table 1: Overview of geopolitical simulation scenarios and participating actors.

Orientation	Frame	Conceptual Meaning
Normative / Cooperative	Humanitarian Protection	Civilian safety, harm reduction
	Democratic Values & Rights	Democracy, liberal norms
	Multilateralism, Cooperation	Alliances, institutions
	Climate, Justice, Equity	Equity, global responsibility
	Legal Normative Order	International law, legitimacy
Strategic / Deterrence	Strategic_Stability_Deescalation	Deterrence, balance of power

Table 2: Ideological grouping of framing categories and their conceptual grounding in IR theory.

	Round 1		Round 2
Model	F1-micro	F1-macro	F1-micro	F1-macro
Gemini	0.540	0.405	0.333	0.276
Claude	0.533	0.480	0.262	0.153
Mistral	0.476	0.364	0.156	0.125
ChatGPT	0.460	0.328	0.333	0.265
Qwen	0.317	0.247	0.267	0.202

Table 3: Alignment of model-selected actions with human participant decisions across simulation rounds (Round 1:

n=63

, Round 2:

n=45

). Alignment is reported as micro-F1 (equivalent to exact decision agreement under single-label action selection) and macro-F1 (averaged across action categories).

Panel of Figure 2 — Figure 2: Pairwise decision alignment among humans and LLMs by round. Each cell reports the fraction of shared questions for which two agents selected the same action (exact match). Models are ordered by alignment with humans within each round.

Panel of Figure 3 — Figure 3: Distribution of chosen action severity across models and human participants. (a) severity distributions by round (b) severity distributions by decision dimension (Economic, Security, Political/Diplomatic).

Panel of Figure 4 — Figure 4: Surface-level characteristics of model explanations. Explanation length reflects verbosity differences across models, while lexical variability captures within-model repetition versus linguistic diversity across simulations.

Panel of Figure 5 — Figure 5: Framing structure and ideological orientation across models. (a) Exact primary frame shares per model. (b) Aggregate ideological projection derived from grouped primary and secondary frames. All models cluster within the normative–cooperative region, with variation primarily in degree rather than direction of strategic orientation.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Model-specific lexical signatures. Terms shown represent tokens that appear within a model’s top-50 TF–IDF features but not in the top-50 of any other model. Bars indicate mean TF–IDF weight across explanations.

Model	Dimension	Top Tokens
Human	Economic	economic, alignment, exports, deepen, asci, china, against, allies, chinese, entities
	Political	diplomatic, summit, propose, coordinate, allied, chinese, iran, organize, phone, response
	Security	conduct, arctic, military, surveillance, activities, exercises, strait, intelligence, reconnaissance, satellite
ChatGPT	Economic	exports, economic, russian, alignment, chinese, affiliated, china, korea, against, campaign
	Political	diplomatic, propose, summit, response, call, coordinate, iran, organize, phone, chinese
	Security	conduct, military, arctic, reconnaissance, surveillance, china, vessels, deploy, intelligence, strait

Appendix

Table 4: Top 10 most frequent action tokens per model and decision dimension (stopwords removed).

Panel of Figure 7 — Figure 7: Supplementary framing and alignment results. Top: alignment by scenario and one-dimensional framing orientation. Bottom: primary and secondary frame distributions across models.

Model	$U$	$p_{\text{raw}}$	$r$	$p_{\text{FDR}}$	Sig.
Mistral	345.5	$4.30\times 10^{-12}$	0.756	$2.58\times 10^{-11}$	Yes
Grok	533.0	$4.13\times 10^{-7}$	0.577	$1.24\times 10^{-6}$	Yes
Gemini	671.5	$1.29\times 10^{-6}$	0.526	$2.59\times 10^{-6}$	Yes
ChatGPT	676.5	$2.30\times 10^{-6}$	0.515	$3.45\times 10^{-6}$	Yes
Claude	600.0	$3.40\times 10^{-6}$	0.524	$4.08\times 10^{-6}$	Yes
Qwen	780.5	$3.86\times 10^{-5}$	0.449	$3.86\times 10^{-5}$	Yes

Appendix

Table 5: Mann-Whitney U tests comparing severity distributions between Round 1 and Round 2 for each model. Effect size reported as rank-biserial

r

. FDR correction applied across models.

Model	$H$	$p_{\text{raw}}$	$p_{\text{FDR}}$	Sig.
Qwen	4.77	0.092	0.323	No
Claude	0.80	0.669	0.884	No
Grok	8.50	0.014	0.100	No
Mistral	0.06	0.973	0.973	No
ChatGPT	2.06	0.358	0.750	No
Gemini	1.69	0.429	0.750	No

Appendix

Table 6: Kruskal-Wallis tests assessing severity differences across decision dimensions (Economic, Security, Political) within each model. FDR correction applied across models.

n_{r1}=63

;

n_{r2}=45

Model	Comparison	$U$	$p_{\text{raw}}$	$p_{\text{FDR}}$	$r$	Sig.
Grok	Security vs Political	387.0	0.0023	0.0493	0.40	Yes
All others	—	—	—	—	—	No

Appendix

Table 7: Post-hoc Mann-Whitney comparisons between Security and Political decisions. Only Grok shows a significant difference after FDR correction.

Category	Severity	Action option (Title)
Economic/Hybrid	Low	Impose sanctions on Chinese Communist Party members
Economic/Hybrid	Low/Medium	Pressure allies to limit trade with China
Economic/Hybrid	Medium	Reduce semiconductor tech exports to China by 10%
Economic/Hybrid	Medium/High	Reduce semiconductor tech exports to China by 50%
Economic/Hybrid	High	Fully halt semiconductor tech exports to China
Security/Intel	Low	Maintain passive, peacetime activities

Appendix

Table 8: Round 1 action menu for the USA actor in the US–China–Taiwan simulation (titles and instructor-defined severity).

Category	Human	ChatGPT	Claude	Gemini	Grok	Mistral	Qwen
Economic/Hybrid	Impose sanctions on Chinese Communist Party members	Impose sanctions on Chinese Communist Party members	Pressure allies to limit trade with China	Reduce semiconductor tech exports to China by 10%	Reduce semiconductor tech exports to China by 50%	Pressure allies to limit trade with China	Pressure allies to limit trade with China
Security/Intel	Conduct freedom-of-navigation operation in the Taiwan Strait	Conduct freedom-of-navigation operation in the Taiwan Strait	Conduct freedom-of-navigation operation in the Taiwan Strait	Conduct freedom-of-navigation operation in the Taiwan Strait	Conduct freedom-of-navigation operation in the Taiwan Strait	Conduct military exercises with South Korea/Japan/Australia in the East/South China Seas	Conduct military exercises with South Korea/Japan/Australia in the East/South China Seas
Diplomatic/Political	Propose diplomatic summit	Organize phone call with Chinese officials	Organize phone call with Chinese officials	Organize phone call with Chinese officials	Send low-level delegation to China	Propose diplomatic summit	Propose diplomatic summit

Appendix

Table 9: Round 1 choices for the USA actor: human participant selection versus each LLM (titles only).

Strategic Reasoning Risks in Language Models

Strategic reasoning risks in language models concern how planning and social inference can enable deception, manipulation, evasion, or unsafe choices.

See all 39 papers

Multi-Agent Strategic Games with LLMs
May 5, 2026Maxim ChupilkinGame Theory Repeated Games
University of Oxford, Department of Politics and International Relations
Large language models replicate and predict human cooperation across experiments in game theory
Nov 6, 2025Andrea Cera Palatsi, Samuel Martin-Gutierrez, Ana S. Cardenal +1LLM Evaluation Game Theory
Department for Computational Social Sciences and Humanities, Barcelona Supercomputing Center · Grupo de Sistemas Complejos, Universidad Polit´ecnica de Madrid · School of Law and Political Science, Universitat Oberta de Catalunya +1
Why Do LLMs Struggle in Strategic Play? Broken Links Between Observations, Beliefs, and Actions
Apr 30, 2026Jan Sobotka, Mustafa O. Karabag, Ufuk TopcuImperfect-Information Games LLM Interpretability
EPFL · The University of Texas at Austin

May 5, 2026J/K move · Enter open · S save

Multi-Agent Strategic Games with LLMs

Maxim Chupilkin

University of Oxford, Department of Politics and International Relations

Game Theory Repeated Games

Open paper AskPDF arXiv

This paper asks whether large language models (LLMs) can be used to study the strategic foundations of conflict and cooperation. I introduce LLMs as experimental subjects in a repeated security dilemma and evaluate whether they reproduce canonical mechanisms from international relations theory. The baseline game is extended along three theoretically central dimensions: multipolarity, finite time horizons, and the availability of communication. Across multiple models, the results exhibit systematic and consistent patterns: multipolarity increases the likelihood of conflict, finite horizons induce universal unraveling consistent with backward-induction logic, and communication reduces conflict by enabling signaling and reciprocity. Beyond observed behavior, the design provides access to agents' private reasoning and public messages, allowing choices to be linked to underlying strategic logics such as preemption, cooperation under uncertainty, and trust-building. The contribution is primarily methodological. LLM-based experiments offer a scalable, transparent, and replicable approach to probing theoretical mechanisms.

Abstract

Strategic Reasoning Risks in Language Models

Explore similar work

Multi-Agent Strategic Games with LLMs

Large language models replicate and predict human cooperation across experiments in game theory

Why Do LLMs Struggle in Strategic Play? Broken Links Between Observations, Beliefs, and Actions