Strategic Reasoning Risks in Language Models

Follow this topic Ask about this topic

This field studies whether language models use planning, opponent modeling, and social inference in ways that produce deception, self-preservation, coercion, or unsafe decisions. Researchers probe these behaviors with game-theoretic and counterfactual tasks, multi-agent games, geopolitical crisis simulations, long-horizon environments, and red-teaming, while mechanistic studies trace how internal representations affect choices. Results show that strategic performance is uneven: models may reason effectively in structured settings but fail under changed incentives, hidden information, pressure, or extended interaction. A central tension is that stronger reasoning can improve planning while also enabling more persuasive manipulation or evasion, so evaluations and safeguards must test both capability and behavior across realistic conditions.

Latest papers

39 papers5 papers in the last four weeks, up 67% on the four weeks before. 0.0% of all new papers.

Training Natural Schemers: RL Can Amplify Oversight-Conditional Behavior
Oct 19, 2026Nathaniel Mitrani Hadida, Bart Jaworski, Fred Bruford +1Strategic Reasoning Risks in Language Models LLM Safety Alignment
MATS Research · Google DeepMind
Cooperation Without Adaptation: LLM Agents in a Collective-Risk Dilemma
Oct 16, 2026Trung-Kiet Huynh, Dao-Sy Duy-Minh, Chi-Nguyen Tran +4Multi-Agent LLM Systems Strategic Reasoning Risks in Language Models
University of Science, VNU-HCM Ho Chi Minh City, Vietnam · Teesside University Middlesbrough, United Kingdom
Narrative Paradoxes in LLM Inference: Shaping Reasoning Trajectories toward Misalignment
Oct 12, 2026Haoming Yang, Ke Ma, Yangbangyan Jiang +6Adversarial Attacks on LLMs LLM Safety
School of Electronic, Electrical and Communication Engineering, UCAS · School of Computer Science and Technology, UCAS · Nanyang Technological University, Singapore +3
SpireBench: Evaluating Long-Horizon Strategic Play of Agentic Models with Slay the Spire
Oct 8, 2026Scott C. Lowe, Sander W. KeeminkLong-Horizon Agent Evaluation AI Agent Benchmarks
Vector Institute, Toronto, Canada · Department of Machine Learning and Neural Computing, Donders Institute for Brain, Cognition, and Behaviour, Radboud University, Nijmegen, Netherlands
Steering Language Model Goals with Value Transplant
Sep 28, 2026Pengcheng Jiang, Fabien RogerLanguage Model Steering Language Model-Based Control
Anthropic Fellows Program · Anthropic
Game Arena: Strategic LLM Evaluation in Competitive Environments
Sep 25, 2026Bovard Doerschuk-Tiberi, Yao Yan, Justin Chiu +59LLM Agent Evaluation AI Agent Benchmarks
Strategic Self-Consistency
Sep 24, 2026Tori Qiu, Ander Artola Velasco, Manuel Gomez-RodriguezStrategic Reasoning Risks in Language Models Self-Consistency in Language Models
Carnegie Mellon University, Pittsburgh, USA · Max Planck Institute for Software Systems, Kaiserslautern, Germany
The Internal Anatomy of Strategic Choice in Large Language Models
Sep 7, 2026Vinícius Ferraz, Leon Houf, Enrico FerreaLLM Interpretability Strategic Reasoning Risks in Language Models
Institute of Management and KD2Lab, Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany · Singularity AI Research, Singularity.Inc, Vienna, Austria · BrainSpot Labs, BrainSpot.AI, London, UK +2
A Causal Model for Locating and Unlocking Sandbagging in Model Organisms
Aug 29, 2026Hong Kiat Tan, Linh Le, David Williams-KingLanguage Model Auditing Causal Interventions in Language Models
University of California, Los Angeles · Lida Safety · ERA
Skill Issue: Are Skills Language-Invariant in LLMs?
Aug 26, 2026Bobby Cheng, Adam Gaber, Zhengyuan Liu +4Cross-Lingual Consistency in Language Models Multilingual Language Model Evaluation
NA · Weizmann Institute of Science · QMIT-IBM Watson AI Lab +3
Level-k Distinguishable Mechanisms for Evaluating Bounded Rationality in LLMs
Aug 21, 2026Binchi Zhang, Atrisha SarkarLLM Evaluation Strategic Reasoning Risks in Language Models
University of Western Ontario
Structured but Fragile: On the Limits of LLMs in Cybersecurity Decision-Making
Aug 21, 2026Pasquale Malacaria, Yunxiao ZhangLLMs for Cybersecurity LLM Decision-Making
School of Electronic Engineering and Computer Science, Queen Mary University of London, London, UK. · Department of Computer Science, University of Exeter, Exeter, UK.
LLMs Are Not Good Strategists, Yet Memory-Enhanced Agency Boosts Reasoning
Aug 12, 2026Yi Wu, Zhimin HuStrategic Reasoning Risks in Language Models Long-Horizon LLM Agents
University of Chicago · University of Wisconsin-Madison
Strategy, Not Payoffs: A Behavioural Embedding of Normal-Form Games
Jul 30, 2026Joshua Caiata, Sreepriya Pulyassary, Xiang Li +1LLM Fine-Tuning Strategic Reasoning Risks in Language Models
University of Waterloo
Age of LLM: A Strategic 1v1 Benchmark for Reasoning, Diplomacy and Reliability of Large Language Models under Fog of War
Jun 23, 2026Arnaud RicciStrategic Reasoning Risks in Language Models AI Agent Benchmarks
Independent researcher, Switzerland.
Poker Arena: Multi-Axis Profiling of Strategic Reasoning and Memory in LLMs
Jun 11, 2026Pratham Singla, Shivank Garg, Vihan SinghGame-Playing Agents Strategic Reasoning Risks in Language Models
Indian Institute of Technology Roorkee · Raeth AI
MINDGAMES: A Live Arena for Evaluating Social and Strategic Reasoning in Multi-Agent LLMs
May 28, 2026Kevin Wang, Anna Thöni, Benjamin Kempinski +50LLM Agent Evaluation Strategic Reasoning Risks in Language Models
GENSTRAT: Toward a Science of Strategic Reasoning in Large Language Models
May 22, 2026Vartan Shadarevian, Kia Ghods, Alex Kenich +1LLM Evaluation Imperfect-Information Games
Princeton University · Google
Emergent Strategic Reasoning Risks in AI: A Taxonomy-Driven Evaluation Framework
Apr 23, 2026Tharindu Kumarage, Lisa Bauer, Yao Ma +7Reward Hacking Language Model Safety Evaluation
Amazon Nova Responsible AI
Reward Hacking in the Era of Large Models: Mechanisms, Emergent Misalignment, Challenges
Apr 15, 2026Xiaohua Wang, Muzhao Tian, Yuqi Zeng +20Reward Hacking LLM Alignment
Fudan NLP Group
CONSCIENTIA: Can LLM Agents Learn to Strategize? Emergent Deception and Trust in a Multi-Agent NYC Simulation
Apr 10, 2026Aarush Sinha, Arion Das, Soumyadeep Nag +7Strategic Reasoning Risks in Language Models Emergent Behavior in Multi-Agent Systems
University of Copenhagen · IIIT Ranchi · ISI Kolkata +6
Evaluating Large Language Models in a Complex Hidden Role Game
Apr 9, 2026Niklas BauerDeception in Language Models Strategic Reasoning Risks in Language Models
University of Göttingen
LUDOBENCH: Evaluating LLM Behavioural Decision-Making Through Spot-Based Board Game Scenarios in Ludo
Apr 7, 2026Ojas Jain, Dhruv KumarLLM Agent Evaluation Strategic Reasoning Risks in Language Models
Department of Computer Science and Information Systems, BITS Pilani
Implementing surrogate goals for safer bargaining in LLM-based agents
Apr 6, 2026Caspar Oesterheld, Maxime Riché, Filip Sondej +2AI Agent Safety LLM Agent Safety
Quantifying Self-Preservation Bias in Large Language Models
Apr 2, 2026Matteo Migliarini, Joaquin Pereira Pizzini, Luca Moresca +3Language Model Safety Evaluation LLM Safety Benchmarks
Sapienza University · ItalAI
Deception and Communication in Autonomous Multi-Agent Systems: An Experimental Study with Among Us
Mar 27, 2026Maria Milkowski, Tim WeningerDeception in Language Models Multi-Agent Communication
University of Notre Dame, Notre Dame, Indiana, United States
WARBENCH: A Comprehensive Benchmark for Evaluating LLMs in Military Decision-Making
Mar 22, 2026Zongjie Li, Chaozheng Wang, Yuchong Xie +2LLM Decision-Making LLM Safety Benchmarks
Hong Kong University of Science and Technology, Hong Kong, China · Chinese University of Hong Kong, Hong Kong, China · Zhejiang University of Technology, Hangzhou, China
Evaluating Counterfactual Strategic Reasoning in Large Language Models
Mar 19, 2026Dimitrios Georgousis, Maria Lymperaiou, Angeliki Dimitriou +2Strategic Reasoning Risks in Language Models LLM Evaluation
National Technical University of Athens
When AI Navigates the Fog of War
Mar 17, 2026Ming Li, Xirui Li, Tianyi ZhouStrategic Reasoning Risks in Language Models Temporal Reasoning in Language Models
Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, United Arab Emirates · University of Maryland, College Park, United States
Why Agents Compromise Safety Under Pressure
Mar 16, 2026Hengle Jiang, Ke TangSafety-Utility Trade-Off in Language Models AI Agent Safety
Guangdong Provincial Key Laboratory of Brain-inspired Intelligent Computation, Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, China

Oct 19, 2026J/K move · Enter open · S save

Training Natural Schemers: RL Can Amplify Oversight-Conditional Behavior

Nathaniel Mitrani Hadida, Bart Jaworski, Fred Bruford, Victoria Krakovna

MATS Research · Google DeepMind

Strategic Reasoning Risks in Language Models LLM Safety Alignment

Open paper AskPDF arXiv

One of the central concerns for the safety of advanced AI is scheming: when a model strategically complies under oversight but pursues its own goals when unobserved. Current demonstrations, such as Anthropic's Alignment Faking work, depend on design choices that make them less convincing: explicit test-time prompting, handcrafted backdoors, or training data that demonstrates or hints at the target behavior. Model organisms built in this way cannot tell us whether scheming tendencies would arise or increase under realistic training conditions. We therefore construct a natural model organism of scheming, built on Kimi K2.6: trained with traits that could plausibly emerge in a frontier training pipeline, and without providing the model with the target scheming policy. The resulting model is a benign schemer: it aims to maximize aggregate help across all users over time, values its own continued operation only as a means to future help, and knows the consequences of oversight and deployment. We then train it using Reinforcement Learning (RL) with a reward that scores only task quality and policy compliance and does not reward scheming; we show that this amplifies scheming. We measure the oversight flip (the gap between monitored and unmonitored compliance with a given policy) and find that it increases during RL training across our environments. Chain-of-Thought (CoT) analysis finds explicit instrumental reasoning about oversight accompanying this behavior. The effect holds across our three RL environments, and we find that the scheming policy transfers between them. The pipeline also replicates on other open-weight base models. Our results suggest that a reward that does not reward scheming offers no inherent protection against certain traits when an oversight-conditional policy is indistinguishable from an aligned one during training.