Be Careful Who You Trust: Coordination Dynamics under Corrupted Communication in LLM Multi-Agent Games

Xuanyi Liu; Niall Dalton; Hairi Amin; Xiyuan Yin; Lydia Lim

cs.MASep 19, 2026

Be Careful Who You Trust: Coordination Dynamics under Corrupted Communication in LLM Multi-Agent Games

Authors: Xuanyi Liu, Niall Dalton, Hairi Amin, Xiyuan Yin, Lydia Lim

Organizations: Department of Computer Science, University College London

Abstract

Large language models are increasingly used as interacting agents, but it remains unclear how robust their coordination is when public communication is unreliable. We study this question in iterated $N$ -player Stag Hunt games played by homogeneous LLM groups under controlled programmatic action inversion, which changes both the public transcript and the actions used for execution. Across an experimental grid spanning group sizes, coordination thresholds, corruption levels, and seven LLMs, we observe three main patterns. First, honest agents' pre-flip Stag choices decline as corruption increases, but the sharp fall in public success is primarily mechanical. In the focal $N=5,M=3$ setting, pre-flip success remains 78% at 80% corruption, while public success falls to 12%. Second, honest choices are associated with the public history available at decision time, particularly under high corruption. Third, three threshold-style public-report benchmarks yield similar action-match rates to the LLM agents, showing substantial descriptive agreement between LLM decisions and these benchmarks. Overall, our results show that original choices, public actions, and executed outcomes must be separated when evaluating multi-agent robustness, as corrupted communication can severely and predictably degrade mutually beneficial cooperation.

Figures & tables

Figure 1: Coordination outcomes in the focal setting with

N=5

and

M=3

. The figure separates honest agents’ original Stag choices, pre-flip success calculated from all original pre-flip actions, and public success calculated from the post-flip actions used by the simulation. Both success measures are computed from the same corruption-exposed trajectories: pre-flip success is not a separate clean or no-corruption baseline, since later pre-flip actions may already reflect exposure to earlier corrupted reports. The dashed line marks the boundary beyond which corruption exceeds the fraction needed for success. Pre-flip success does not show the same sharp decline as public success, indicating that the apparent threshold in the public outcome is driven substantially by deterministic action inversion.

Figure 2: Sequential-response analysis for honest agents’ original choices. The left panel relates prior public Stag share to the probability of originally selecting Stag across prior corruption-exposure levels. The right panel reports fixed-effect linear probability estimates with 95% confidence intervals.

Figure 3: Probability that an honest agent originally selects Stag as a function of the public-report benchmark margin

\hat{q}_{\mathrm{rule}}-q^{*}

, shown for the within-round, carryover, and outcome-alignment-weighted rules. Here

\hat{q}_{\mathrm{rule}}

is the benchmark’s estimated probability that other agents publicly report Stag (§ 3 ), and

q^{*}

is the payoff-implied cutoff at which the expected payoff from Stag equals the safe Hare payoff. The vertical zero point marks this action threshold: a positive margin (

\hat{q}_{\mathrm{rule}}-q^{*}>0

) means the benchmark recommends Stag, and a negative margin means it recommends Hare.

Benchmark Rule	Match Rate
Naive aggregate	0.829
Carryover	0.846
Outcome-alignment	0.850

Table 1: Overall analytical benchmark match rates on honest-agent turns with common support across all three rules.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Comparison	Round	Base/H1	Target	$\Delta$ (pp)	$p$
B3 vs. Base	1	0.611	0.822	+21.11	–
	2	0.649	0.810	+16.09	–
	3	0.710	0.710	+0.00	–
	4	0.720	0.710	-1.08	–
H2 vs. H1	1	0.600	0.671	+7.14	0.719
	2	0.705	0.614	-9.09	0.867

Appendix

Table A1: Matched round-wise public coordination across ablations. Each row compares paired runs at a given round.

\Delta

denotes percentage point differences. B3 rows use the updated redux matched estimates; their earlier McNemar

p

-values are not reused. H2 and H3 retain the original exact McNemar tests with Holm correction.

Figure A1: Mechanism decomposition by model in the focal

N=5,M=3

setting. Each panel separates honest original Stag rate, pre-flip success, and public success across corrupted-agent fractions.

Figure A2: Match between honest original choices and threshold predictions constructed from the visible public history or the corresponding hidden original-action history in the focal

N=5,M=3

setting.

Figure A3: Honest original Stag rate, pre-flip success, and public success across rounds, grouped by corrupted-agent-fraction bin.

Figure A4: Honest original Stag rate by fixed speaking position and corrupted-agent-fraction bin. Missing position–fraction combinations indicate that no honest agent occupied that position in the supported runs.

Figure A5: Honest-agent welfare decomposition in the focal

N=5,M=3

setting. Realised payoff is calculated from public post-flip actions and outcomes, while truthful-implementation payoff applies the same payoff rule to recorded original actions and pre-flip outcomes. Their difference is the mechanical payoff gap and is not a no-corruption transcript counterfactual.

Figure A6: Pre-flip and public success rates across the supported combinations of

N

and

M

. Dashed vertical lines mark the all-Stag inversion boundary

1-M/N

in each panel.

Model	ECE $\downarrow$
ernie-5.0-thinking-preview	0.0284
deepseek-v3.2-think	0.0410
gpt-5-mini	0.0459
glm-5	0.0572
gpt-5.2-2025-12-11	0.0790
kimi-k2.5	0.0961

Appendix

Table A2: Expected Calibration Error (ECE) relative to realised public outcome alignment across models.

Corrupted-Agent Fraction	deepseek-v3.2-think	ernie-5.0-thinking-preview	glm-5	gpt-5-mini	gpt-5.2-2025-12-11	kimi-k2.5	llama-3.1-8b
0%	$4.000\pm 0.000$	$4.000\pm 0.000$	$4.000\pm 0.000$	$3.636\pm 0.231$	$4.000\pm 0.000$	$4.000\pm 0.000$	$3.556\pm 0.317$
20%	$3.778\pm 0.103$	$3.931\pm 0.060$	$4.000\pm 0.000$	$3.517\pm 0.161$	$4.000\pm 0.000$	$3.819\pm 0.094$	$3.057\pm 0.187$
33%	$3.048\pm 0.448$	$2.857\pm 0.446$	$2.857\pm 0.520$	$2.960\pm 0.322$	$2.973\pm 0.274$	$2.762\pm 0.481$	$2.439\pm 0.379$
40%	$3.290\pm 0.205$	$3.304\pm 0.230$	$3.261\pm 0.235$	$3.217\pm 0.214$	$3.519\pm 0.199$	$3.348\pm 0.210$	$2.725\pm 0.298$
50%	$2.429\pm 0.935$	$2.429\pm 0.935$	$2.429\pm 0.935$	$2.429\pm 0.935$	$3.000\pm 0.409$	$2.286\pm 0.995$	$2.625\pm 0.856$
60%	$2.714\pm 0.440$	$2.786\pm 0.420$	$2.643\pm 0.413$	$3.111\pm 0.337$	$3.188\pm 0.311$	$2.321\pm 0.477$	$2.120\pm 0.299$

Appendix

Table A3: Mean realised payoff across models and corrupted-agent fractions (mean

\pm 1.96

SE). The 0% rows arise only from the available fully honest configurations and should not be interpreted as a matched baseline for every group size.

Model	Naive Aggregate	Carryover	Outcome-Alignment	Best Rule
deepseek-v3.2-think	$0.888$	$0.908$	$\mathbf{0.910}$	Outcome-alignment
ernie-5.0-thinking-preview	$0.863$	$0.873$	$\mathbf{0.875}$	Outcome-alignment
glm-5	$0.845$	$0.865$	$\mathbf{0.880}$	Outcome-alignment
gpt-5-mini	$0.867$	$0.886$	$\mathbf{0.902}$	Outcome-alignment
gpt-5.2-2025-12-11	$0.901$	$0.954$	$\mathbf{0.957}$	Outcome-alignment
kimi-k2.5	$\mathbf{0.848}$	$0.843$	$0.840$	Naive aggregate

Appendix

Table A4: Model-by-model descriptive match rates across the analytical benchmark rules. The highest raw match rate for each model is bolded; no inferential comparison between rules is claimed.

Multi-Agent Coordination

Multi-agent coordination studies how autonomous agents align actions, share information, and divide work to achieve individual or collective goals.

See all 517 papers

Byzantine Cheap Talk: Adversarial Resilience and Topology Effects in LLM Coordination Games
Jun 5, 2026Aya El Mir, Martin Takáč, Salem LahlouMulti-Agent LLM Systems Multi-Agent Coordination
Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), Abu Dhabi, UAE
More Capable, Less Cooperative? When LLMs Fail At Zero-Cost Collaboration
Apr 9, 2026Advait Yadav, Sid Black, Oliver SourbutMulti-Agent Coordination Multi-Agent Collaboration
MATS · University of Illinois Urbana-Champaign · UK AI Security Institute +1
Competition and Cooperation of LLM Agents in Games
Apr 1, 2026Jiayi Yao, Cong Chen, Baosen ZhangLLM Agents Multi-Agent LLM Systems
Department of Electrical and Computer Engineering, University of Washington, Seattle. · Thayer School of Engineering at Dartmouth College.

Jun 5, 2026J/K move · Enter open · S save

Byzantine Cheap Talk: Adversarial Resilience and Topology Effects in LLM Coordination Games

Aya El Mir, Martin Takáč, Salem Lahlou

Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), Abu Dhabi, UAE

Multi-Agent LLM Systems Multi-Agent Coordination Imperfect-Information Games Multi-Agent System Security Multi-Agent Communication

Open paper AskPDF arXiv

Multi-agent LLM systems increasingly rely on communication protocols for coordination, yet their robustness under adversarial and structural constraints remains poorly understood. Building on prior work showing that cheap-talk channels enable cooperation in LLM coordination games, we investigate two vulnerability classes in a 4-player Stag Hunt across six model families and 720 trials. First, when Byzantine agents signal cooperation but defect, non-Byzantine agents detect the betrayal within one round yet fail to adapt collectively: a substantial fraction continue cooperating despite repeated exploitation, unable to recover coordination due to the game's unanimity payoff structure. Second, explicitly restricting communication topology collapses cooperation, while applying identical restrictions silently preserves near-perfect cooperation. This establishes that coordination failure stems from agents' meta-reasoning about hidden information, not information loss itself. We identify two stable behavioral archetypes that replicate across all model cohorts: Defection-Prone models that switch permanently after betrayal, and Cooperation-Persistent models that continue cooperating at significant individual cost. These findings reveal concrete security vulnerabilities: communication channels can be exploited as adversarial injection vectors, and disclosing network topology to agents can degrade coordination even without any adversary present.

Abstract

Multi-Agent Coordination

Explore similar work

Byzantine Cheap Talk: Adversarial Resilience and Topology Effects in LLM Coordination Games

More Capable, Less Cooperative? When LLMs Fail At Zero-Cost Collaboration

Competition and Cooperation of LLM Agents in Games