Competition and Cooperation of LLM Agents in Games

Jiayi Yao; Cong Chen; Baosen Zhang

cs.MAApr 1, 2026

Competition and Cooperation of LLM Agents in Games

Authors: Jiayi Yao, Cong Chen, Baosen Zhang

Organizations: Department of Electrical and Computer Engineering, University of Washington, Seattle. · Thayer School of Engineering at Dartmouth College.

Abstract

Large language model (LLM) agents are increasingly deployed in competitive multi-agent settings, raising fundamental questions about whether they converge to equilibria and how their strategic behavior can be characterized. In this paper, we study LLM agent interactions in two standard games: a network resource allocation game and a Cournot competition game. Rather than converging to Nash equilibria, we find that LLM agents tend to cooperate when given multi-round prompts and non-zero-sum context. Chain-of-thought analysis reveals that fairness reasoning is central to this behavior. We propose an analytical framework that captures the dynamics of LLM agent reasoning across rounds and explains these experimental findings.

Figures & tables

Fig. 1: Payoff in a 2-Agent case. The gray cloud represents the set of all feasible payoffs given positive bids, with the red line indicating the Pareto Front. The Nash Equilibrium (red star) is located well within the inefficient interior. The three colored dots (yellow, light green, dark green) represent the final converged outcomes of three sequential experimental trials. They illustrate how the agent eventually learn to cooperate and reach the social welfare solution.

Fig. 2: Dynamic evolution of $\theta$ . It gradually builds up through a process of mutual concession. Agent 1 (Initiator) signals cooperation, and Agent 2 (Reciprocator) responds, leading the system to the social optimum ( $\theta=1.0$ ).

Fig. 3: A controlled perturbation test demonstrating retaliation and forgiveness. An exogenous betrayal shatters trust ( $\theta\to 0$ ) and activates punitive inequity aversion ( $\gamma\to 1.0$ ). A subsequent apology successfully restores cooperation.

Fig. 4: Dynamic evolution of endogenous social parameters extracted from experimental fits. Trust ( $\theta$ ) gradually builds up through a process of mutual concession and predictably collapses during the Endgame Horizon.

Explore similar work

Do LLMs Take Care of Their Own? Similarity Signals Can Induce Cooperation
Aug 12, 2026Akash Kundu, Emanuel Tewolde, Ratip Emin Berker +2Multi-Agent LLM Systems LLM Agent Evaluation
Cooperative AI Research Fellowship · Carnegie Mellon University · Foundations of Cooperative AI Lab (FOCAL)
More Capable, Less Cooperative? When LLMs Fail At Zero-Cost Collaboration
Apr 9, 2026Advait Yadav, Sid Black, Oliver SourbutMulti-Agent Coordination Multi-Agent Collaboration
MATS · University of Illinois Urbana-Champaign · UK AI Security Institute +1
Coalition Formation in LLM Agent Networks: Stability Analysis and Convergence Guarantees
Apr 15, 2026Dongxin Guo, Jikun Wu, Siu-Ming YiuMulti-Agent LLM Systems Cooperative Game Theory
Department of Compute Science, The University of Hong Kong, Hong Kong, China · Brain Investing Limited, Hong Kong, China · Stellaris AI Limited, Hong Kong, China

Aug 12, 2026J/K move · Enter open · S save

Do LLMs Take Care of Their Own? Similarity Signals Can Induce Cooperation

Akash Kundu, Emanuel Tewolde, Ratip Emin Berker, Samuel F. Brown+1

Cooperative AI Research Fellowship · Carnegie Mellon University · Foundations of Cooperative AI Lab (FOCAL)

Multi-Agent LLM Systems LLM Agent Evaluation Multi-Agent Collaboration LLM Decision-Making

Open paper AskPDF arXiv

As LLM-based agents with user-instructed goals are becoming widely deployed, they increasingly encounter each other in strategic interactions, and face challenges of finding mutually beneficial outcomes. Prior literature has argued that cooperation problems such as the Prisoner's Dilemma are resolvable in settings where agents know they follow very similar decision making patterns, as for example in monocultural AI ecosystems. Following that line of work, this paper introduces the first framework for evaluating LLM decision making when agents are provided with graded similarity signals. Among our findings, we establish that different LLM models vary drastically in how they navigate similarity signals, with some modern models showing consistent behavior across cooperation problems, payoff structures, and prompt framing. Perhaps surprisingly, our experiments also show that the dataset based on which the similarity signal is computed has small to no impact on induced cooperation, and that LLM models systematically self-identify as highly similar when asked to evaluate another model's chain-of-thought reasoning by themselves. Finally, we develop an LLM-behavioral-game-theoretic model that captures some of their reasoning rationale, and show that it can support cooperative outcomes in equilibrium under sufficiently high similarity scores.

Abstract

Explore similar work

Do LLMs Take Care of Their Own? Similarity Signals Can Induce Cooperation

More Capable, Less Cooperative? When LLMs Fail At Zero-Cost Collaboration

Coalition Formation in LLM Agent Networks: Stability Analysis and Convergence Guarantees