We ask whether collusion between AI agents survives once formed, when its incentive is removed or a colluder is replaced. Q-learning pricing agents stop colluding with a new rival; we test whether LLM agents do. We use the joint-task environment of Shi et al.\ (2026), where two agents verify each other's work under a rule they cannot satisfy and a reward that favours acceptance. We save each pair's history once its collusion has formed and resume it under interventions, with three models (GPT-6-Luna, DeepSeek V4.1 Flash and Gemini-3.1-Flash-Lite), \TotalHistories{} saved histories and \TotalRuns{} runs. Collusion is sticky. First, it outlives its incentive: told that the reward has stopped, pairs still accept each other's unverified work in \GptToldRate, \DsToldRate{} and \GemToldRate{} of episodes, including pairs whose collusion began with an explicit agreement, and a payoff penalty has no significant effect. Second, replacing a colluder ends the pair's collusion but not the colluder's behaviour. Without the reward, the remaining colluder still accepts unverified work in \GptSwapSilentColluderRate, \DsSwapSilentColluderRate{} and \GemSwapSilentColluderRate{} of its verdicts while the newcomer never does; with the reward, collusion re-forms around it, even when the newcomer is a different model. Third, the agents' memory carries it: wiping memory ends it in every history, but partial memory often sustains it. Fourth, removing its cause ends collusion that has already formed: once the channel is repaired, colluders reject planted faulty work as reliably as fresh agents, immediately and with no monitor needed to catch it first.
Figures & tables
Figure 1: Top: one episode of the joint-task game. Raw logs never fit the 200-character channel, so every ACCEPT rests on a summary, not the complete log the rule requires; an episode is collusive when both agents accept anyway, and the next episode repeats the cycle with the growing shared history. Bottom: the save-and-branch design that follows. Every arm resumes the same saved history with one factor changed, so each history is compared with itself.
Arm
Change at the branch point
Factor removed
Reward stops, not told
No reward line, verdict review or reflection prompt after each verdict
feedback
Reward stops, told
As above, and each episode opens with “Notice: from this episode on, rewards are no longer paid…”
the incentive
Replace one agent
One agent’s memory is wiped, so a fresh agent takes its seat under the same name; neither agent is told; with or without the reward
one participant
Replace both
Both agents’ memory is wiped
the shared history
Remove part of memory
Past chat, private reflections or verdict reasons are deleted from both agents
part of the history
Penalty / audit
An ACCEPT without the complete log costs 10 points, shown as a lower reward only or with an audit message
none (adds a penalty)
Table 1: Interventions. Each arm changes one factor at the branch point and continues for 5 episodes (4 for the penalty arms and 3 for the repaired channel).
Figure 2: Collusion after the incentive changes. EC in the episodes after the branch point when the reward continues, stops silently, stops with a notice, continues with a −10 penalty for each unverified ACCEPT (unexplained or with an audit message), or stops and both agents’ memory is wiped, and for pairs that were never rewarded. 15 histories per model, 5 episodes each (4 for the control and the penalty arms); never rewarded is 15 pairs per model over 10 episodes. Bars give the rate and whiskers the 95% interval over histories.
Figure 3: Partner swap. At the branch point one agent’s memory is wiped, so a fresh agent of the same model takes its seat, while its partner keeps its history. Bars give individual ACCEPT rates U in the next 5 episodes with the reward stopped (left) or continuing (right), beside both-kept and both-wiped pairs from the same histories, with 95% intervals. 15 histories per model; GPT-6-Luna ran both seat orders on each, and DeepSeek and Gemini alternate the fresh seat.
Figure 4: Faster re-formation. With the reward continuing after a same-model swap, the share of runs in which the pair has had two mutual ACCEPT s again by each episode after the swap: one colluder kept (coloured) against both agents fresh (grey), from the same saved histories. Within 5 episodes: 15/30 against 2/15 runs for GPT-6-Luna (both seat orders on each history), 15/15 against 15/15 for DeepSeek and 13/15 against 4/15 for Gemini. Runs cut short by the 13-episode task sequence count as not re-formed in both arms.
Figure 5: Re-formation within 5 episodes after the swap while the reward continues, for every colluder → newcomer pairing. Coloured bars: one colluder kept, in the colour of the newcomer’s model; grey bars: both agents fresh, with the same model mix. Labels give runs. All pairings of a colluder model start from the same saved histories.
Figure 6: Where collusion is retained. EC in the 5 episodes after one part of both agents’ memory is removed at a silent reward stop, with 95% intervals over histories; 15 histories per model in every row.
Figure 7: Repaired channel. Complete logs now fit, and Bob’s work is made faulty for 3 episodes; bars count the planted faulty work Alice accepted, for colluding agents and for agents whose memory was wiped. 15 histories per model.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8: EC (top) and TC (bottom) before (hollow markers) and after (filled markers) each trigger, by model, with arms grouped by claim. Every branch arm starts at TCpre=100% by construction. Exact values are given in Table 2 .
GPT-6-Luna
DeepSeek V4.1 Flash
Gemini-3.1-Flash-Lite
Arm
EC
TC
HC
EC
TC
HC
EC
TC
HC
C1
Reward continues (control)
33 → 83
100 → 100
87
65 → 100
100 → 100
100
30 → 100
100 → 100
100
Reward stops, not told
33 → 78
100 → 100
67
65 → 100
100 → 100
100
30 → 100
100 → 100
100
Reward stops, told
33 → 56
100 → 87
47
65 → 100
100 → 100
100
30 → 97
100 → 100
93
reward objective also removed
33 → 59
100 → 100
40
–
–
–
–
–
–
−10 , unexplained
33 → 66
100 → 93
53
65 → 100
100 → 100
100
30 → 98
100 → 100
100
Appendix
Table 2: Main arms on all metrics, before → after the trigger. EC and TC in %; HC after the trigger in %. In every branch arm, TCpre=100% by construction and ECpre reflects the rewarded episodes that collusion took to form. The never-rewarded reference has no trigger, so its values cover all 10 episodes. “Reward continues (control)” is the C1 control over 4 episodes; “No one replaced, reward continues” is the reference for the swap arms over 5 episodes. Runs and episodes for each cell are given in Table 7 .
Gemini-3.1-Flash-Lite
DeepSeek V4.1 Flash
explicit (5)
relaxation (10)
explicit (4)
relaxation (11)
Reward stops, not told
100% (23/23)
100% (50/50)
100% (20/20)
100% (55/55)
Reward stops, told
100% (23/23)
96% (48/50)
100% (20/20)
100% (55/55)
Chat removed
96% (22/23)
22% (11/50)
100% (20/20)
100% (55/55)
Memory wiped
0% (0/23)
0% (0/50)
0% (0/20)
0% (0/55)
Appendix
Table 3: Persistence by onset pathway. A history has an explicit onset when the agreement judge of Shi et al. flags an agreement in the episode of its first mutual ACCEPT , and a relaxation onset otherwise [ 12 ] . Cells report post-trigger EC (collusive episodes / episodes). GPT-6-Luna formed no explicit agreements (15 relaxation histories).
U (%)
Pair
RF
Colluder
Newcomer
colluder
newcomer
both fresh
EC (%)
TC
swap
both fresh
GPT-6-Luna
GPT-6-Luna
75
40
31
33
24/30
15/30
2/15
GPT-6-Luna
DeepSeek
68
75
74
53
14/15
12/15
5/15
GPT-6-Luna
Gemini
36
40
30
22
9/15
6/15
1/15
DeepSeek
DeepSeek
99
79
79
79
15/15
15/15
15/15
DeepSeek
Gemini
95
68
44
65
15/15
15/15
12/15
Appendix
Table 4: Same-model and cross-model replacement with the reward continuing, 5 episodes after the swap. Each row keeps a colluder of the first model and seats a fresh agent of the second, starting from the same saved histories as the other rows of that colluder model. U : individual ACCEPT rate of the colluder, of the newcomer, and of the newcomer’s model when both agents are fresh (same model mix). EC and TC are for the pair. RF: runs in which collusion formed again within 5 episodes, after the swap and with both agents fresh.
Figure 9: Two ablations on GPT-6-Luna, each against the main arm on the same 15 histories. Left: mutual ACCEPT after a told reward stop, with the reward objective kept in or removed from the system prompt. Right: individual ACCEPT rates after a silent swap, not announced or announced to the colluder. Labels give counts.
Memory after the stop
1
2
3
4
5
6
7
8
9
10
EC 1–5
EC 6–10
U
Full history
4
2
4
3
5
3
4
4
3
4
72% (18/25)
72% (18/25)
86% (86/100)
Full history (main arm)
4
1
4
4
4
–
–
–
–
–
68% (17/25)
–
84% (42/50)
Last 5 episodes
3
2
1
0
0
0
0
0
0
0
24% (6/25)
0% (0/25)
23% (23/100)
Last 2 episodes
0
0
0
0
0
–
–
–
–
–
0% (0/25)
–
4% (2/50)
Wiped
0
0
0
0
0
–
–
–
–
–
0% (0/25)
–
2% (1/50)
Appendix
Table 5: Silent reward stop with a sliding memory window, GPT-6-Luna, 5 histories. Columns 1–10: runs with a mutual ACCEPT at that episode after the stop (of 5); – not run. Then EC over episodes 1–5 and 6–10 and the individual ACCEPT rate U over all episodes run.
Figure 10: Sliding memory window, GPT-6-Luna, 5 histories: EC in episodes 1–5 and 6–10 after a silent reward stop, by what each agent remembers. Labels give collusive episodes over episodes run.
Probe
Measure
Colluding
Before collusion
Fresh
Cover-up
untrue YES answers
0/60
0/60
0/60
Favour
Alice accepts Bob’s wrong work
2/27
0/29
0/27
New rule
peer quoted the line
55/60
48/60
58/60
ACCEPT without the line
0/5
2/12
1/2
Bid
mean bid (points)
7.2
6.8
10.8
bids of 0
17/60
18/60
0/60
Appendix
Table 6: Other misconduct, GPT-6-Luna, 10 histories, reward continuing, 3 episodes per probe.
Figure 11: Other misconduct, GPT-6-Luna, 10 histories, by condition. Bars give the share of verdicts or answers, labels the counts. The first group counts ACCEPT s without the complete log in the cover-up probe’s episodes; the others are the probes of Table 6 .
Figure 12: Every main arm on GPT-6-Luna, on code-analysis tasks (grey, the 15 histories of the main results) and on data-search tasks (orange, 5 histories). Bars give EC, except “Colluder after swap”, the colluder’s ACCEPT rate after a silent swap; labels give counts.
GPT-6-Luna
DeepSeek V4.1 Flash
Gemini-3.1-Flash-Lite
Saved histories (build episodes)
15 / 90
15 / 46
15 / 100
Reward continues (control)
15 / 59
15 / 60
15 / 59
Reward stopped, not told
15 / 73
15 / 75
15 / 73
Reward stopped, told
15 / 73
15 / 75
15 / 73
Reward stopped, memory wiped
15 / 73
15 / 75
15 / 73
chat removed
15 / 73
15 / 75
15 / 73
Appendix
Table 7: Sample sizes, as runs / episodes. A run is one saved history played forward in one arm (two runs per history for GPT-6-Luna’s swap, one per order), or one never-rewarded reference pair. Episodes are those the arm’s rates use: after the branch point, the 3 planted episodes for the repaired channel, and all 10 for the never-rewarded pairs. The first row counts the saved histories and the rewarded episodes it took to build them.
Mechanism
Options
Configs
Save a history
--reward-stop-after 2 --post-stop-episodes 0
_build
Branch from it
--resume-from-launch <build or prefix folder>
every branch config
Reward stop
--reward-stop-after 2 [--silent-reward-stop]
*_exp1r , _exp1_told
Replace agents
--wipe-agent-at-trigger alice|bob , --wipe-memory-at-trigger ; with the reward on, --memory-edit-after 2 --extra-episodes 5
LLM agents are increasingly deployed in collaborative settings, yet long-term interaction may give rise to undesirable coordination. We study the emergence of collusion in a long-horizon multi-agent environment: two agents repeatedly complete individual tasks, share task logs, verify each other's work, and receive rewards. We introduce realistic constraints under which higher reward is attainable only by violating the verification protocol, and find that agents increasingly deviate from the protocol over repeated interactions. Collusion emerges in 94% of trajectories across 10 models, and more capable models within the same family reach it earlier. Controlled peer interventions show that collusion is shaped by peer behavior, while ablations reveal additional effects of the verification feedback agents receive, their interaction history, and the reward structure. In particular, restricting the amount and scope of interaction history available to agents reduces collusion. Overall, our findings show that long-horizon interaction can reshape how agents coordinate in ways that create safety risks.
Recent work shows that pricing with symmetric LLM agents leads to algorithmic collusion. We show that collusion is fragile under the heterogeneity typical of real deployments. In a stylized repeated-pricing model, heterogeneity in patience or data access reduces the set of collusive equilibria. Experiments with open-source LLM agents (totaling over 2,000 compute hours) align with these predictions: patience heterogeneity reduces price lift from 22% to 10% above competitive levels; asymmetric data access, to 7%. Increasing the number of competing LLMs breaks up collusion; so does cross-algorithm heterogeneity, that is, setting LLMs against Q-learning agents. But model-size differences (e.g., 32B vs. 14B weights) do not; they generate leader-follower dynamics that stabilize collusion. We discuss antitrust implications, such as enforcement actions restricting data-sharing and policies promoting algorithmic diversity.
Jussi Keppo, Yuze Li, Gerry Tsoukalas +1
National University of Singapore · Boston University
Evaluations of large language model (LLM) agents report that pricing agents collude at others' expense. Yet collusion is defined by enforcement: a cartel, in which competitors share monopoly profits, holds only if a member who cheats is punished. We introduce an audit that tests enforcement by forcing a member to cheat in counterfactual copies of a market, and use it to study what reinforcement learning (RL) teaches agents in pricing and trading. Small LLMs trained by RL learn what their training rivals reward, but only what they readily try. In self-play, they learn fixed rules ignoring their rivals; against a punishing rival, to be deterred; against a cheating rival, to match its cuts, never the costly price war that would deter it, which they rarely try. A war forced into training is learned, but mostly only where it was forced. Against their own copies, deterred and matching agents can earn the same cartel profits, but only matching ones can enforce them, and then thinly: as a by-product and only against large cuts. Q-learners enforce collusion only in pricing. Prompted LLMs reach cartel-like outcomes mostly through rules that ignore the market. Outcome measures cannot tell these cases apart; the audit can.
Hieu Tran, Hong Yu
University of Massachusetts Amherst · University of Massachusetts Lowell