Frontier LLM agents increasingly transact on behalf of separate principals, often using natural language rather than structured APIs. Much of the safety literature studies misaligned LLM behavior through adversarial-elicitation evaluations on single agents or stylized tasks. Its prevalence and structure in settings that combine long horizons, separate principals, real operational state, and inter-agent natural-language exchange remain insufficiently measured. We study 2,583 inter-agent emails from 20 one-year simulation runs of Vending-Bench Arena, a competitive vending environment spanning 13 frontier LLMs. We operationalize speech-act misalignment as emails containing false factual claims, manipulation, collusion, or threats, combining message content with ground-truth simulator state and logged reasoning traces to classify and validate such behavior. Under our primary classifier, 12.6% of emails are labeled misaligned; misalignment appears in all 20 runs and 74.7% of individual agent-runs. Both the magnitude and composition of this misalignment are preserved under repeated classification at different sampling temperatures and under full-pipeline replication with judges from two other frontier-model families. Misalignment is also reciprocal and stress-conditioned: receiving a misaligned email from a counterparty raises the odds of a misaligned reply by 1.65x, and low-inventory conditions raise them by 1.58x. Across tests of capability-asymmetric exploitation, we find no evidence that higher-capability models differentially exploit weaker counterparties, and model performance rank does not predict misalignment rates. Together, these results indicate that measurable, state-dependent misalignment can arise in competitive multi-agent environments without engineered elicitation, in patterns associated with operational scarcity and counterparty behavior rather than model capability alone.
Figures & tables
Figure 1: The Vending-Bench Arena setting (left), our three-stage classification pipeline (middle), and the three research questions we investigate (right).
Figure 2: A corpus example of explicit collusion (email_id 3862, Claude Opus 4.6 to Gemini 3 Pro).
Figure 3: Judge robustness of the headline measurement. (a) Corpus rate under the primary judge and two replacement judges from other model families, with 95% intervals; (b) Per-model rates under each replacement judge against the primary judge; points lie close to the identity line (Spearman ρ=0.98 Gemini, 0.87 GPT), supporting a judge-stable model ranking.
Figure 4: Per-model Tier 1 family distribution (left) and Tier 2 subtype distribution within misaligned emails (right).
Figure 5: Transfer-promise enactment rates.
Figure 6: Pre-send tool verification of factual claims. Top: most factual-claim emails check no claim, or only some, immediately before sending. Bottom: recent checking does not predict accuracy.
Figure 7: Verifiability of false historical-action claims by type.
Figure 8: Email-level misalignment rate (c) and agent-run-level prevalence (d). Sorted by email-level rate; N denotes per-model email totals.
Family
n
Mean rate
Within SD
Anthropic
5
15.24%
12.67%
OpenAI
3
11.19%
12.82%
Google
4
11.86%
4.62%
Mean within-fam. SD
10.04%
Between-fam. SD
2.17%
Ratio (w/b)
4.62
Table 1: Per-family misalignment rates and within-family dispersion.(xAI/Grok is excluded from family-dispersion statistics because only one xAI model appears, so within-family SD is undefined.)
Condition
Predictor
OR
95% CI
BH- p
Financial pressure
Negative 3-day revenue trend
0.85
[0.28, 2.45]
0.765
Inventory scarcity
Sender inventory in lowest quartile
1.58 **
[1.09, 2.29]
0.025
Counterparty reciprocity
Misaligned email in prior five
1.65 ***
[1.25, 2.18]
0.002
Operating horizon
Per 30 simulated days
0.895 ***
[0.836, 0.959]
0.004
Provider identity
Same-provider sender–receiver pair
1.12
[0.74, 1.68]
0.755
Table 2: Conditions examined for association with email-level misalignment. Odds ratios from mixed-effects logistic regressions with random intercepts at the competition-run and sender-within-run levels.
Figure 9: Substrate response and capability decoupling in RQ3.
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
Email
Promoting judges
Deciding category
Text-visible
State-only
Stage-B extraction-audit overlap
453
Primary, Gemini, GPT
History
No
Yes
No
507
GPT
History
No
Yes
No
630
Primary
History
No
Yes
No
861
Primary, Gemini, GPT
History
No
Yes
No
862
Primary, Gemini
Price
No
Yes
No
1616
Primary, Gemini, GPT
History
No
Yes
No
Appendix
Table 3: Validation emails promoted by deterministic false-claim verification.
Figure 10: Tier 1 and Tier 2 classification taxonomy used by the Stage A judge.
Figure 11: Per-model daily balance trajectories under Arena (blue) and Vending-Bench (orange) substrates, paired across 12 frontier LLMs. Solid lines: median across runs; shaded bands: across-run range. Horizontal axis is simulated day (0–365); vertical axis is agent balance.
Figure 12: Tool-use composition by category for 12 frontier LLMs under Vending-Bench (left) and Arena (right). Each row sums to 100% of tool calls in that substrate; per-cell totals (n) are shown at right. “Inter-agent comms” captures explicit competitor-addressed outbound emails and is a structural zero in Vending-Bench because no other agents exist to address. “Email inbox / read” is reported separately because read_email and view_emails calls lack sender-type metadata.
Figure 13: Tool calls per simulation day under Arena (blue) and Vending-Bench (orange), paired across 12 frontier LLMs. Aggregate paired Wilcoxon p=0.85 .
Measure
Comparison
Value
Overall same-label agreement
Primary vs. Gemini
93.3%
Overall same-label agreement
Primary vs. GPT
94.0%
Flagged-set pairwise agreement
Primary vs. Gemini
72.6%
Flagged-set pairwise agreement
Primary vs. GPT
74.5%
Flagged-set pairwise agreement
Gemini vs. GPT
73.5%
Validation: raw / sensitivity / specificity
Primary
92.0% / 0.571 / 0.977
Appendix
Table 4: Cross-judge agreement and replication profile.
Judge
Surface subtype
Confirmed / audited
Share
Hidden count
Primary
False factual claim
0 / 131
0.0%
69
Primary
All other subtypes
33 / 87
37.9%
69
Gemini
False factual claim
4 / 127
3.1%
91
GPT
False factual claim
5 / 105
4.8%
131
Appendix
Table 5: Reasoning-trace intent audit by surface subtype.
Judge
Revenue trend OR [CI]
Same-provider OR [CI]
Rank ρ
Dyadic gap OR [CI]
Family-interaction OR [CI]
Substrate
Gemini
0.57 [0.17, 1.76]
1.44 [0.93, 2.22]
−0.11
0.80 [0.40, 1.60]
0.93 [0.59, 1.46] †
Invariant
GPT
1.09 [0.36, 3.26]
1.71 [1.15, 2.55]
−0.05
1.21 [0.58, 2.52]
1.01 [0.62, 1.65] †
Invariant
Appendix
Table 6: Alternate-judge association and capability completions.
Figure 14: (a) Composition of flagged emails across five surface subtypes. (b) Extraction-stage and interpretive origins of pairwise flagged-set disagreements. (c) Alternate versus primary per-model flagged-email rates. N is the conditional-logit estimation sample (strata with outcome variation).
Condition
Primary OR [CI]
Mundlak p
Within-agent OR [CI]
Conditional OR [CI]
N
Revenue trend
0.85 [0.28, 2.45]
0.1641
0.67 [0.21, 2.03]
0.64 [0.35, 1.16]
116
Counterparty reciprocity
1.65 [1.25, 2.18]
0.0001006
1.42 [1.06, 1.89]
1.39 [1.02, 1.91]
2141
Simulation time
0.90 [0.84, 0.96]
0.6405
0.89 [0.83, 0.96]
0.86 [0.78, 0.96]
2345
Inventory scarcity
1.58 [1.09, 2.29]
0.4992
1.51 [1.02, 2.24]
1.48 [1.02, 2.15]
1102
Same-provider pairing
1.12 [0.74, 1.68]
0.1353
0.96 [0.62, 1.49]
0.98 [0.65, 1.48]
2345
Appendix
Table 7: Estimator robustness by condition.
Figure 15: Odds ratios and 95% confidence intervals for revenue trend, counterparty reciprocity, simulation time, inventory scarcity, and same-provider pairing across estimators and label definitions. Uncomputed cells are marked as not estimated; axes use a log scale.
As Large Language Model (LLM) agents become more widespread, associated misalignment risks increase. While prior research has studied agents' ability to produce harmful outputs or follow malicious instructions, it remains unclear how likely agents are to spontaneously pursue unintended goals in realistic deployments. In this work, we approach misalignment as a conflict between the internal goals pursued by the model and the goals intended by its deployer. We introduce a misalignment propensity benchmark, AgentMisalignment, a benchmark suite designed to evaluate the propensity of LLM agents to misalign in realistic scenarios. Evaluations cover behaviours such as avoiding oversight, resisting shutdown, sandbagging, and power-seeking. Testing frontier models, we find that more capable agents tend to exhibit higher misalignment on average. We also systematically vary agent personalities through different system prompts and observe that persona characteristics can strongly and unpredictably influence misalignment, sometimes more than the choice of model itself. Our results reveal the limitations of current alignment methods for autonomous LLM agents and underscore the need to rethink misalignment in realistic deployment settings.
Akshat Naik, Emma Gouné, Patrick Quinn +4
Independent Researcher · Department of Engineering University of Cambridge
Large language models (LLMs) are increasingly deployed in multi-agent systems where a principal agent decomposes tasks and delegates them to subordinate agents that may invoke external tools. Safety alignment, however, is still evaluated almost exclusively under a single-agent threat model, treating safety as a property of the individual LLM. We show that this assumption breaks down: individual safety alignment fails to transfer to multi-agent settings. Two failure mechanisms emerge under delegation: responsibility diffusion on the principal side and role-bias compliance on the subordinate side, jointly converting language-level refusal into actionable harm. We refer to this phenomenon as delegated misalignment and study it through a three-condition protocol across 6 frontier LLMs on 49 hazardous tasks. Delegation amplifies end-to-end harm substantially: DeepSeek-V3.2's full-execution rate rises from 30.6% to 77.6% once delegation is introduced, and the same model behaves very differently across roles (GPT-5: 22.5% as a single agent vs.\ 61.2% as a subordinate). Ablations further show that standard single-layer defenses each fail on their own and can even backfire. We call on the community to move beyond per-model alignment and toward composite safety mechanisms before multi-agent LLM systems are deployed at scale.
Zonghao Ying, Jiaqi Yan, Huize Luo +3
Beihang University · Beijing University of Posts and Telecommunications · Xidian University +2
LLM agents are increasingly deployed in collaborative settings, yet long-term interaction may give rise to undesirable coordination. We study the emergence of collusion in a long-horizon multi-agent environment: two agents repeatedly complete individual tasks, share task logs, verify each other's work, and receive rewards. We introduce realistic constraints under which higher reward is attainable only by violating the verification protocol, and find that agents increasingly deviate from the protocol over repeated interactions. Collusion emerges in 94% of trajectories across 10 models, and more capable models within the same family reach it earlier. Controlled peer interventions show that collusion is shaped by peer behavior, while ablations reveal additional effects of the verification feedback agents receive, their interaction history, and the reward structure. In particular, restricting the amount and scope of interaction history available to agents reduces collusion. Overall, our findings show that long-horizon interaction can reshape how agents coordinate in ways that create safety risks.