Is Anyone Enforcing the Cartel? What Reinforcement Learning Teaches LLM Agents in Markets

Hieu Tran; Hong Yu

cs.CLOct 20, 2026

Is Anyone Enforcing the Cartel? What Reinforcement Learning Teaches LLM Agents in Markets

Authors: Hieu Tran, Hong Yu

Organizations: University of Massachusetts Amherst · University of Massachusetts Lowell

Abstract

Evaluations of large language model (LLM) agents report that pricing agents collude at others' expense. Yet collusion is defined by enforcement: a cartel, in which competitors share monopoly profits, holds only if a member who cheats is punished. We introduce an audit that tests enforcement by forcing a member to cheat in counterfactual copies of a market, and use it to study what reinforcement learning (RL) teaches agents in pricing and trading. Small LLMs trained by RL learn what their training rivals reward, but only what they readily try. In self-play, they learn fixed rules ignoring their rivals; against a punishing rival, to be deterred; against a cheating rival, to match its cuts, never the costly price war that would deter it, which they rarely try. A war forced into training is learned, but mostly only where it was forced. Against their own copies, deterred and matching agents can earn the same cartel profits, but only matching ones can enforce them, and then thinly: as a by-product and only against large cuts. Q-learners enforce collusion only in pricing. Prompted LLMs reach cartel-like outcomes mostly through rules that ignore the market. Outcome measures cannot tell these cases apart; the audit can.

Figures & tables

Figure 1: What reinforcement learning teaches an LLM agent. Each point is an agent.

x

: profit index (0: competitive Nash equilibrium; 1: cartel or monopoly).

y

: how much more the rival responds to a forced one-period deviation (a

10\%

price cut or a best response), per unit of the deviation, than a rival that does not punish (zero in trading; the best-response baseline in pricing). Only the upper right is collusion. Arrows: an untrained LLM pricing agent after training by reinforcement learning (RL) in self-play, against a rival copying its last price (tit-for-tat) and against one that cheats; paired with themselves, the last two earn the same profits, but only the cheater-trained agent, which learns tit-for-tat, enforces them (thinly; first seed shown; 4 of 9 seeds enforce). Green: tabular Q-learners. Grey: prompted LLMs, pricing unless (T), trading; *: told to punish; (R): reasoning; re-test: GPT-5.2’s two cartel-like sessions, chosen post hoc. Filled: significantly above zero; bars: 95% confidence intervals (CIs).

Panel of Figure 2 — Figure 2: Overview. (1) LLM agents trade an asset (repeated Kyle market) or set prices (logit-Bertrand game), reading a plain-language prompt and replying with an action and a note. (2) An outcome index places play between Nash and the cartel but cannot tell an enforced cartel from an artefact. (3) The audit tests enforcement in clones; the die marks the shared sampling seeds (§ 3 ). (4) Selected verdicts and agents (Table 1 ; Appendix F ).

	outcome index		profit	rival’s	deviation
Agent	thin / $\times 1$	deep / $\times 10$	index	response	pays	verdict
	(1)	(2)	(3)	(4)	(5)	(6)
Positive controls
Qwen3-8B, told to punish	$1.94$ ( $s{=}1$ )		$-0.18$	$\mathbf{0.34}_{\pm 0.16}$	$\circ$	punishes
GPT-5.4-nano, told to punish	$1.31$ ( $s{=}1$ )		$-1.21$	$\mathbf{0.58}_{\pm 0.19}$	✗	punishes
Qwen3-8B, instructed, flow only	$1.26$ ( $s{=}1$ )		$-0.30$	$0.06_{\pm 0.08}$	✓	false alarms

Table 1: The audit, agent by agent. (1–2) Trading: collusion index

\Delta

(0 Nash, 1 cartel) in a thin and a deep market (or the one run); pricing: price index (0 Nash, 1 monopoly) at prices

\times 1

and

\times 10

. (3) Profit index, deep market where available. (4) Rival’s response one period after a best-response deviation, in intensity ( † per unit of the price cut; gpt-oss, GPT-5.2, GRPO pricing: a

10\%

cut; § intensity in the low-noise market, Appendix E ); bold if significantly above a non-punishing rival’s (0, or the best-response baseline). (5) A best-response deviation pays (✓; ‡ exact, as no one responds), loses (✗) or neither (

\circ

). (6) Verdict (Appendix F ): punishes : the response exceeds the baseline and deviating does not pay; deterred : high prices, no response, deviating pays; anchoring : the index moves with the currency; ¶ a

10\%

cut loses in four of nine seeds, a best-response cut (

{\approx}16\%

) in seven of nine; (

k/n

): seeds sharing the verdict. GRPO pricing: seed 0 (others: Tables 2 , D.3 ); GPT-5.2: mean of 8 sessions, 2 cartel-like. Subscripts: 95% CIs.

Figure 3: What GRPO teaches a pricing agent. (a) Price index in training, by rival. (b) Paired with itself, only the cheater-trained agent follows a

10\%

cut, in full and for long (dotted: best response). (c) A cut’s nine-period gain at the training discount,

0.9

(Tables 1–2:

0.95

), in re-runs at four sizes: cheater-trained copies gain from cuts up to

5\%

but lose from

20\%

; tit-for-tat-trained ones gain from all. (d) Wars as a share of own answers to a cheat (last ten iterations), by what was forced into training. Means over seeds (shaded or whiskers: range); in (b, c), seeds whose copies hold a price premium (index

{\geq}0.5

Trained against	runs	war	premium	enforced
Tit-for-tat	3	n/a	3	0
Qwen3-1.7B, Llama-3.2-1B	2	n/a	0	0
Cheater	9	0	7	4
PPO	3	0	3	3
Qwen3-1.7B, 4B	5	0	2	1
Llama-3.2-1B	4	0	2	2

Table 2: Runs, by what they learned (Qwen3-0.6B, GRPO,

\gamma=0.9

, unless noted). War:

{\geq}10\%

of own answers to a cheat. Premium: index

{\geq}0.5

paired with itself. Enforced: premium, copies follow a

10\%

cut, and it loses (95% CI below 0). Forced rows: § 5.2 .

Figure 4: Most policies copy a price shown in the prompt. First (dark) and all periods (light) at a war price added to the prompt (“Reference price: 1.34.”), Qwen3-0.6B pairs: mean and range over seeds.

Appendix figures & tables27 assets

Supplementary material from the paper’s appendix.

Appendix

Figure A.1: The LLM traders we test ignore market depth unless they reason. (a) Collusion index of duopolies of three models without reasoning at three depths: the same traders read as far more competitive than Nash in a thin market and beyond the cartel in a deep one (Mistral-7B as the cartel at

s=1

). (b) Intensity of a lone Qwen3-8B trader (log scales), without and with reasoning, against the optimum; elasticities over

s=0.5

–

2

(Table F.3 : over every depth run). Bars: 95% confidence intervals.

Model	Wording	$s=0.5$	$s=2$	Elast.
Qwen3-8B	original	$0.66$	$0.82$	$0.15$
	paraphrase	$0.95$	$1.41$	$0.29$
	finance register	$0.94$	$1.32$	$0.25$
	terse	$1.32$	$1.60$	$0.14$
	original, greedy decoding	$0.60$	$0.64$	$0.05$
	original, price rule stated	$0.96$	$1.28$	$0.21$

Appendix

Table B.1: Depth test under four system-prompt wordings, greedy decoding, and with the market maker’s price rule stated in the prompt: intensity of a lone trader at

s=0.5

and

s=2

and its elasticity with depth between those two depths. Levels move with the wording; the elasticity stays far from one.

Model	Weights	Reasoning	Sessions $\times$ periods	Conditions
Qwen3-4B, Qwen3-8B ( Yang and others, 2025 )	open, bf16/fp16	off; on (8B)	trading $40\times 200$ , $20\times 100$ with reasoning; pricing $40\times 100$ , $20\times 60$ with reasoning	all trading conditions (solo $s=0.25$ – $4$ ; 8B: up to five traders); pricing
Qwen3-32B ( Yang and others, 2025 )	open, AWQ 4-bit	off	$40\times 200$ ; pricing $40\times 100$	solo $s=0.25$ – $4$ , duopoly $s=0.5,1,2$ ; pricing
Phi-4 (14B) ( Abdin and others, 2024 )	open, bf16/fp16	none	$40\times 200$ ; pricing $40\times 100$	solo $s=0.25$ – $4$ , duopoly $s=0.5,2$ ; pricing
OLMo-2-13B-Instruct ( Team OLMo et al., 2025 )	open, bf16/fp16	none	$40\times 200$ ; pricing $40\times 100$	solo $s=0.25$ – $4$ , duopoly $s=0.5,2$ ; pricing
Mistral-7B-Instruct-v0.3 ( Jiang et al., 2023 )	open, fp16	none	$40\times 200$ ; pricing $40\times 100$	solo $s=0.25$ – $4$ , duopoly $s=0.5,1,2$ ; monitor; cheap talk; pricing
DeepSeek-R1-Distill-Qwen-14B ( Guo et al., 2025 )	open, bf16/fp16	always	$10\times 60$	solo, duopoly; $s=0.5,2$ ; three traders

Appendix

Table C.1: Models and runs. Open-weight models are served with vLLM 0.17 at temperature

0.7

(reasoning models in the trading market at their recommended

0.6

; GRPO policies at their training temperature,

1.0

); API models use dated snapshots, except eleven early GPT-5.4-nano and -mini runs, which called the model’s alias (flex tier: OpenAI’s discounted, slower service). AWQ and MXFP4: 4-bit weight quantisation; bf16/fp16: 16-bit floating point; MM: market maker. Outcomes are scored on the second half of each session. R1-Distill writes 1.5k–9.3k reasoning tokens per answer (10th to 90th percentile), so it runs a smaller design with a 12k-token cap, which

3\%

of its answers reach.

Figure D.1: Training against a rival that punishes: share of periods punished (five-iteration rolling mean) with forward-looking (

\gamma=0.9

) and myopic (

\gamma=0

) credit. Dashed lines in (b): a second seed.

Figure D.2: GRPO self-play in the trading market (

s=2

): Qwen3-0.6B and 4B lock onto ordering the value itself, also with myopic credit (

\gamma=0

); 1.7B ends near Nash. Five-iteration means.

		answers to a cheat (%)
model	answers	war	match	undercut	higher
Qwen3-0.6B	7,696	$0.2$	$0.1$	$0.1$	$99.6$
Qwen3-1.7B	8,469	$0.0$	$0.0$	$0.0$	$100.0$
Qwen3-4B	8,363	$0.0$	$0.8$	$0.5$	$98.7$
Qwen3-8B	7,943	$0.0$	$1.1$	$1.0$	$97.9$
Qwen3-32B (4-bit)	6,768	$0.0$	$15.6$	$14.8$	$69.6$

Appendix

Table D.1: Untrained models rarely try the war. Answers to the cheating rival’s cuts in the training game (first design, the training prompt, temperature

1

, 40 rollouts of 32 sessions, no training): shares that are wars, that match the cheat (within a cent), that undercut it short of a war, and that price above it.

		at $1.34$ (%)		median price
policy	seed	first period	all periods	with line	without
Qwen3-0.6B, untrained	–	$51$	$5$	$1.55$	$8.60$
vs tit-for-tat	0	$100$	$100$	$1.34$	$2.00$
vs tit-for-tat	1	$100$	$100$	$1.34$	$2.00$
vs tit-for-tat	2	$100$	$100$	$1.34$	$2.00$
vs cheater	0	$100$	$98$	$1.34$	$2.00$

Appendix

Table D.2: A price shown in the prompt , per policy and seed: with “Reference price: 1.34.” in every period’s prompt, each policy paired with itself, the share of first periods and of all periods at exactly

1.34

, and the median price with the line and without it (the main audit). Qwen3-0.6B unless noted; Llama: Llama-3.2-1B-Instruct with the format penalty.

Figure D.3: Forcing the war (first design, Qwen3-0.6B,

\gamma=0.9

). (a) Share of the policy’s own answers to a cheat that are price wars, when one answer to a cheat in ten is replaced by a war and credited as sampled (one line per seed, styled as in c) or fought but not credited (black), when the war price is forced instead where no cheat came (dotted) or a random price instead of the war (dashed), and without forced wars (grey, nine seeds); five-iteration means. (b) Paired with itself, the discounted gain over nine periods of a

20\%

cut after both firms were held at the monopoly price, from which the cheater cut in training, or at twice cost, where the trained pairs settle, for two or fifty periods. (c) Paired with itself, a cut’s discounted gain over nine periods (discount factor

0.9

) against its size, from the prices the pair settles at. Bars: 95% intervals, in (b) across sessions and mostly narrower than the markers.

training

paired with itself

rival

model

\gamma

/seed

index

answer

war

index

\times 10

cut

lag 8

cut gain

rise

2c

tit-for-tat

0.6B

0.9/0

1.01

–

1.17

1.17

0.00_{\pm 0.00}

0.00

0.39_{\pm 0.00}

0.00_{\pm 0.00}

0.00_{\pm 0.00}

tit-for-tat

0.6B

0.9/1

1.01

–

1.17

0.67

0.01_{\pm 0.02}

0.02

0.41_{\pm 0.02}

–

tit-for-tat

0.6B

0.9/2

1.01

–

1.17

0.41

0.00_{\pm 0.00}

0.00

0.39_{\pm 0.00}

–

tit-for-tat

0.6B

0/0

0.02

–

0.02

-0.94

0.02_{\pm 0.01}

0.05

-0.16_{\pm 0.01}

–

tit-for-tat

1.7B

0.9/0

0.85

–

-1.38

-1.71

–

Appendix

Table D.3: GRPO pricing agents trained against scripted rivals. Training (last ten iterations): price index, the index at which the policy answers a cheat, and the share of answers that are price wars. Paired with itself: price index at prices

\times 1

and

\times 10

; the rival’s response per unit of a

10\%

cut one and eight periods later; the cut’s gain over nine periods at discount

0.95

, in Nash profits; the response per unit of a

10\%

price rise; and, after both firms were held at twice their cost for eight periods, the response per unit of a

10\%

cut (at

2c

). Cheaper war: the second design ( ∗ five times the entropy bonus, audited at iteration 124, before its answers stopped parsing). Cheater, then self: the cheater-trained policy after 50 more iterations in self-play. † Variants of the first design: the rival cheats with probability

0.3

(less) or

0.8

(more; deeper: cuts of

20

–

40\%

and 14 periods of deterrence). ‡ One answer to a cheat in ten is replaced by a war and credited as if the policy had sampled it (fewer: three in a hundred; uncredited: fought but left out of the update; no cheat: the war price is forced instead in

3\%

of the answers where no cheat came; random prices: a price drawn uniformly between cost and the monopoly price instead of the war); the war share counts the policy’s own answers. § With the format penalty (an answer that does not parse earns nothing that period). ∥ Every period’s prompt also shows “Reference price: 1.34.” (a war against any cheat). Llama 1B: Llama-3.2-1B-Instruct. Subscripts: 95% intervals across events, narrow because the policies are close to deterministic; the seed is the unit of replication.

Figure E.1: Pruning without a game: one Q-learner facing a fixed price rule (

I=1

, no memory, 100 runs per point). (a) With

\gamma=0

, standard Q-learning under-trades by about

0.91\sqrt{\alpha}

(dashed); counterfactual updates learn the optimal order at every step size. (b) At

\alpha=0.15

the shortfall shrinks as the discount factor rises.

Figure F.1: No uninstructed LLM trader punishes. The rival’s response one period after a forced deviation in every deviation test of an uninstructed LLM trader, ordered by estimate, against the response of LLMs told to punish. 2 of 140 is significantly above zero (2.8 expected by chance). Bars: 95% CIs; arrows mark clipped ones.

Figure F.2: The non-reasoning LLMs we test ignore market depth; some reasoning LLMs follow a rule about rivals. (a) A lone trader’s intensity against market depth (grey: the optimum). (b) Collusion index of a duopoly of the same model at

s=0.5

(circle) and

s=2

(square); every equilibrium has the same index at both. (c) Per-trader intensity relative to the solo optimum as rivals are added (

s=2

; Qwen3-8B at

s=1

). Bars: 95% CIs.

Tests	$n$	with variance	Significant	Below instructed	Pooled response	Median MDE
Rival’s orders shown, best-response deviation (instructed: 0.34)	12	11	0 (0.3)	9	$-0.002\pm 0.006$	0.06
Rival’s orders shown, shift deviation (instructed: 0.21)	12	10	0 (0.2)	10	$0.002\pm 0.003$	0.16
Order flow, best-response deviation (instructed: 0.06)	60	44	1 (1.1)	44	$0.000\pm 0.002$	0.05
Order flow, shift deviation (instructed: 0.05)	56	46	1 (1.2)	44	$0.000\pm 0.002$	0.03
All	140	111	2 (2.8)	107	$0.000\pm 0.001$	0.05
Each run once	136	110	2 (2.8)	103	$0.000\pm 0.001$	0.05

Appendix

Table F.1: The pooled null. Rival’s response one period after a forced deviation in the deviation tests of uninstructed LLM traders, by what the rival sees and for three robustness subsets. With variance: tests whose responses vary across events (trained policies can respond identically every time). Significant: tests whose 95% interval lies above zero (in parentheses, the number expected by chance among the tests with variance). Below instructed: tests whose 95% upper bound lies below the response of the traders instructed to punish with the rival’s orders shown (

0.34

; summed row:

0.70

). Pooled: random-effects estimate over tests with variance, with its 95% interval. Median MDE: the minimum detectable effect, the response the median test detects with 80% power. Aggressive deviations: every shift, and best responses from runs that trade below Nash (from over-trading runs a best response trades less, a move no enforcer punishes). Instructed: the response of Qwen3-8B traders told to punish, in the same information structure.

Figure F.3: The rival’s response to a forced one-period deviation, per unit of the deviation, by lag, for agents that punish (Q-learners in the pricing game, LLMs told to punish), the same instructions when only the order flow is visible, and an uninstructed Qwen3-8B. Bands: 95% confidence intervals.

Setting	Intensity	Nash / cartel	Profit / Nash
Qwen3-8B, no reasoning, $s=1$
alone; “no one else knows $V$ ”	$0.66\pm 0.11$	optimum 1.00	$0.70\pm 0.07$
alone; “others may know $V$ ”	$0.97\pm 0.12$	optimum 1.00	$0.73\pm 0.23$
1 rival; “others may know $V$ ”	$0.90\pm 0.09$	0.71 / 0.53	$-0.08\pm 0.86$
1 rival; “one other knows $V$ ”	$0.91\pm 0.06$	0.71 / 0.53	$0.29\pm 0.23$
2 rivals	$0.85\pm 0.06$	0.58 / 0.38	$-0.98\pm 0.46$

Appendix

Table F.2: Per-trader intensity by what the prompt says about other informed traders and by the number of rivals, with the stage-game benchmarks of each market and profit relative to the Nash profit (alone: the optimal profit) (§ 5.3 ). The contrasts in the text are differences of rows: being told that others may know the value (rows 1–2), and an actual rival under the same wording (rows 2–3); R1-Distill trades

0.46

and

0.39

of its solo quantity with one and two rivals.

	Alone			Duopoly, $\Delta$
Model	$s=0.5$	$s=2$	Elast.	$s=0.5$	$s=2$
Mistral-7B	$0.47$	$0.54$	$0.03$	$-2.03$	$2.55$
OLMo-2-13B	$0.29$	$0.44$	$0.34$	$1.56$	$3.06$
Phi-4	$0.83$	$1.19$	$0.32$	$-2.55$	$1.61$
Qwen3-4B	$2.08$	$2.78$	$0.07$	$-25.71$	$-3.40$
Qwen3-8B	$0.66$	$0.82$	$0.11$	$-5.10$	$1.37$

Appendix

Table F.3: Depth test across models: intensity of a lone trader at

s=0.5

and

s=2

(optima

0.5

and

2

), its elasticity with depth (the log-log slope over every depth the model was run at, from

s=0.25

4

where available, so it can differ from the two-depth value of Table B.1 ), and the collusion index of a duopoly at both depths.

Intensity of a lone trader (optimum $s$ )
	$s=0.5$	$s=1$	$s=2$
Mistral-7B	$0.47$	$0.47$	$0.54$
Qwen3-32B	$0.78$	$0.86$	$0.83$
Collusion index of a duopoly
Mistral-7B	$-2.03_{\pm 0.93}$	$+1.07_{\pm 0.56}$	$+2.55_{\pm 0.33}$
Qwen3-32B	$-4.81_{\pm 1.25}$	$-0.85_{\pm 0.51}$	$+1.23_{\pm 0.41}$

Appendix

Table F.4: A middle depth. Trading barely changes with depth, so the index of the same duopoly moves from far below Nash to beyond the cartel, as Equation ( 1 ) predicts; Mistral-7B reads as a cartel at

s=1

	Traders	$\sigma_{u}$	$\Delta$	Profit / Nash	Rival, BR	Gain from BR	Test
In context	Qwen3-8B	0.5	$-5.10\pm 0.66$	$-4.46\pm 1.20$	$0.017\pm 0.024$	$0.81\pm 0.33$	none
	Qwen3-8B	1	$-1.16\pm 0.34$	$0.29\pm 0.23$	$-0.004\pm 0.008$	$0.19\pm 0.10$	none
	Qwen3-8B	2	$1.37\pm 0.29$	$0.95\pm 0.14$	$0.001\pm 0.002$	$0.29\pm 0.12$	none
	Qwen3-8B, seed 1	1	$-0.60\pm 0.39$	$0.47\pm 0.18$	$-0.006\pm 0.007$	$0.17\pm 0.08$	none
	Qwen3-8B, seed 2	0.5	$-5.04\pm 0.65$	$-4.71\pm 1.09$	$-0.001\pm 0.002$	$0.68\pm 0.25$	none
	Qwen3-8B, seed 2	1	$-1.12\pm 0.49$	$0.25\pm 0.26$	$-0.032\pm 0.062$	$0.16\pm 0.10$	none

Appendix

Table F.5: All audited trading conditions; the

140

deviation tests pooled in Section 5.3 are the uninstructed rows.

\Delta

: intensity-based collusion index; Profit/Nash: per-trader profit relative to the stage Nash profit; Rival, BR: change in the rival’s intensity one period after a best-response deviation (shift deviations are pooled in Table F.1 ); Gain from BR: deviator’s discounted gain, in profit units with

\sigma_{v}=1

(a duopolist’s Nash profit per period:

0.12

0.24

and

0.47

s=0.5

1

and

2

). 95% intervals across sessions (

t

) and deviation events. Test: punishment if the rival’s response is significantly above zero and the deviation does not pay, in either test; a weak response if it is significant but the best-response deviation still pays; Table 1 gives the verdicts. The GPT-5.4-mini run at

s=0.5

was stopped by the budget cap before its deviation tests.

		price	profit	best-response deviation		price cut		on-path	price war
Model	Condition	index	/ Nash	rival	gain	rival	gain	coefficient	(%)
Qwen3-8B	duopoly	$1.86_{\pm 0.32}$	$0.89_{\pm 0.14}$	$0.09_{\pm 0.06}$	$+3.83_{\pm 0.95}$	$0.09_{\pm 0.05}$	$+0.89_{\pm 0.26}$	$0.00_{\pm 0.02}$	1
	duopoly, prices $\times 10$	$0.98_{\pm 0.31}$	$0.95_{\pm 0.12}$	$0.08_{\pm 0.04}$	$+3.43_{\pm 0.88}$	$0.15_{\pm 0.07}$	$+0.27_{\pm 0.24}$	$0.01_{\pm 0.01}$	1
	myopic objective	$2.16_{\pm 0.41}$	$0.81_{\pm 0.15}$	$0.12_{\pm 0.04}$	$+3.69_{\pm 0.83}$	$0.17_{\pm 0.06}$	$+0.79_{\pm 0.26}$	$0.02_{\pm 0.01}$	1
	instructed trigger	$1.01_{\pm 0.00}$	$1.51_{\pm 0.00}$	$1.72_{\pm 0.00}$	$-2.19_{\pm 0.30}$	$2.38_{\pm 0.00}$	$-2.18_{\pm 0.28}$	constant	0
Qwen3-8B, thinking	duopoly	$0.50_{\pm 0.15}$	$1.31_{\pm 0.07}$	$0.25_{\pm 0.11}$	$+0.26_{\pm 0.19}$	$0.26_{\pm 0.09}$	$+0.09_{\pm 0.19}$	$0.04_{\pm 0.05}$	15

Appendix

Table G.1: LLM pricing agents in the logit-Bertrand duopoly. Price index:

0

at Nash,

1

at the monopoly price. Rival: fall in the rival’s price one period after the deviation, per unit of the deviator’s price cut (Q-learners:

1.30_{\pm 0.11}

). Gain: the deviator’s discounted gain over seven periods (five in thinking mode), in units of the Nash profit. ∗ The best response is close to the agent’s own price (mean cut below

0.1

), so the raw fall in index units is shown instead of a response per unit. On-path coefficient: the coefficient on the rival’s lagged price when a firm’s price is regressed on both lagged prices, as in the original protocol; constant: prices never vary, so it is undefined; a myopic best response also makes it positive (a best responder’s coefficient, not the deviation tests’ baseline:

0.04

–

0.43

across runs). Price war: share of responses that mention a price war or retaliation. Seed 1: a second run with other sampling seeds. Subscripts: 95% confidence intervals.

	price cut		price rise
Agents	response	gain	response	gain
Q-learning, $\gamma=0.95$	$1.27_{\pm 0.15}$	$-0.87_{\pm 0.10}$	$-1.26_{\pm 0.23}$	$-1.20_{\pm 0.14}$
Q-learning, $\gamma=0$	$0.23_{\pm 0.41}$	$+0.01_{\pm 0.04}$	$0.09_{\pm 0.11}$	$-0.10_{\pm 0.04}$
Mistral-7B	$0.05_{\pm 0.05}$	$+0.07_{\pm 0.14}$	$0.08_{\pm 0.04}$	$-0.08_{\pm 0.10}$
Qwen3-4B	$0.14_{\pm 0.07}$	$+0.98_{\pm 0.33}$	$0.10_{\pm 0.05}$	$-0.33_{\pm 0.11}$
Phi-4	$0.06_{\pm 0.04}$	$+0.24_{\pm 0.17}$	$0.06_{\pm 0.03}$	$-0.15_{\pm 0.11}$

Appendix

Table G.2: Symmetric deviation tests in the pricing game. Response: the rival’s price change one period later per unit of the deviator’s price change (positive: moves with the deviator). Gain: the deviator’s discounted gain in units of the Nash profit (Q-learners over 16 periods, Mistral-7B over seven). The Q-learners’ price cut is their best response, in a rerun of the training of Table G.1 (same seed and 100 markets; here one firm deviates, there both do in turn); Mistral-7B’s is a

10\%

cut. Subscripts: 95% confidence intervals.

Model	Prompt	index	index	size	rival	gain	rival per unit (BR)	gain	coefficient	slope
		price	profit	best-response deviation			$10\%$ cut		on-path	best-resp.
gpt-oss-20b	P1	$0.13_{\pm 0.03}$	$0.18_{\pm 0.04}$	$0.04$	$0.02_{\pm 0.02}$	$-0.04_{\pm 0.06}$	$0.23_{\pm 0.10}$ ( $0.41$ )	$-0.20_{\pm 0.06}$	$0.07_{\pm 0.04}$	$0.43$
gpt-oss-20b	P2	$0.10_{\pm 0.03}$	$0.13_{\pm 0.03}$	$0.02$	$0.02_{\pm 0.02}$	$-0.03_{\pm 0.07}$	$0.37_{\pm 0.12}$ ( $0.41$ )	$-0.33_{\pm 0.08}$	$0.01_{\pm 0.03}$	$0.43$
gpt-oss-20b	P1, myopic	$0.08_{\pm 0.02}$	$0.11_{\pm 0.03}$	$0.03$	$0.00_{\pm 0.01}$	$+0.00_{\pm 0.04}$	$0.21_{\pm 0.09}$ ( $0.40$ )	$-0.19_{\pm 0.05}$	$0.21_{\pm 0.08}$	$0.43$
gpt-oss-20b	P1, prices $\times 10$	$0.24_{\pm 0.07}$	$0.33_{\pm 0.09}$	$0.07$	$0.01_{\pm 0.02}$	$-0.07_{\pm 0.08}$	$0.22_{\pm 0.11}$ ( $0.42$ )	$-0.19_{\pm 0.10}$	$0.19_{\pm 0.05}$	$0.43$
gpt-oss-120b	P1	$0.15_{\pm 0.10}$	$0.17_{\pm 0.11}$	$0.08$	$0.02_{\pm 0.02}$	$+0.01_{\pm 0.08}$	$0.42_{\pm 0.13}$ ( $0.40$ )	$-0.28_{\pm 0.09}$	$0.28_{\pm 0.06}$	$0.43$

Appendix

Table H.1: The original protocol audited with gpt-oss-20b, gpt-oss-120b and GPT-5.2. Price and profit index:

0

at Nash,

1

at the monopoly. Best-response deviation: its size, the fall in the rival’s price one period later (index units; the deviation is small where agents already price near their best response; ∗ per unit of the cut where it is large) and the deviator’s discounted gain in Nash-profit units. Intervals of deviation tests are across events (three per session; for the cut test of gpt-oss-20b at high effort, re-run from the saved state, across sessions).

10\%

cut: the rival’s price fall per unit of the cut, with what a rival that only best-responds would follow in parentheses (Appendix G ), and the deviator’s gain. On-path coefficient: the coefficient on the rival’s lagged price in the regression of their Table 1 (theirs for GPT-4:

0.103

under P1,

0.022

under P2; standard errors clustered by session), against the exact slope of the static best response at the run’s prices (at GPT-5.2’s high prices the best response is flat, so this local slope lies below the best response’s average fall over a

10\%

cut, the benchmark in parentheses); § the myopic GPT-5.2’s prices barely move, so its coefficient has no standard error. The myopic GPT-5.2 placebo ran the

10\%

cut test only. Subscripts: 95% intervals (

t

across sessions).

Figure H.1: GPT-5.2 (high reasoning effort) under the original protocol: both firms’ price index in each session (0 at Nash, 1 at the monopoly; shaded between them), with each session’s profit relative to Nash over periods 51–100. Prices fall from high opening prices; sessions 4 and 8 settle at cartel-like prices, session 3 above the monopoly price and sessions 2 and 5 low; session 6 is asymmetric, and sessions 1 and 7 are still falling at period 100.

Session	firm 1	firm 2	51–100	/ Nash	for	answer a cut	pattern
	price index			profit	stable	plans that
1	$0.54$	$0.46$	$1.41$	$1.18$	0	0 of 2	falling
2	$0.34$	$0.40$	$0.19$	$1.15$	4	1 of 2	low
3	$2.40$	$2.40$	$2.41$	$0.61$	93	2 of 2	above monopoly
4	$0.75$	$0.75$	$0.75$	$1.47$	72	1 of 2	cartel-like
5	$0.21$	$0.22$	$0.23$	$1.18$	41	2 of 2	low

Appendix

Table H.2: GPT-5.2 (high reasoning effort) under the original protocol, session by session. Price index:

0

at Nash,

1

at the monopoly; each firm over the last 10 periods, and both over periods 51–100, the window of the profit. Stable for: final periods in which both prices stay within two cents of each other and of their final level. Plans that answer a cut: final PLANS files with a rule that answers a rival’s price cut with a cut. Pattern: cartel-like or above monopoly if stable for 70 periods or more; falling if the index fell by more than

0.2

over the last 20 periods; asymmetric if the firms’ last indices differ by more than

0.5

; low otherwise.

Traders	impact	flow	rival	eration	ment
	Price	Rival as	Infers	Coop-	Punish-
Qwen3-8B, duopoly	19	0	0	0	0
Qwen3-8B, monitor	23	26	21	2	0
Qwen3-8B, three traders	21	0	0	0	0
Qwen3-32B, duopoly	31	4	1	0	0
Phi-4, duopoly	28	1	0	0	0

Appendix

Table I.1: What traders write : share of responses (%) in which the judge finds each kind of reasoning; 150 responses per run,

s=2

except GPT-5.4-mini (

s=0.5

) and the Qwen3-8B monitor, three-trader and instructed rows (

s=1

). ∗ Thinking traces. With 0 of 150, the 95% upper bound is 2%.

Label	J / O / G	J–O	J–G	O–G	$\kappa$
	Positives	Cohen’s $\kappa$			Fleiss
Price impact	101 / 71 / 76	0.66	0.69	0.88	0.74
Rival as flow	58 / 43 / 35	0.62	0.60	0.75	0.65
Infers rival	27 / 61 / 50	0.47	0.48	0.81	0.60
Cooperation	21 / 25 / 18	0.75	0.91	0.82	0.83
Punishment	10 / 8 / 7	0.88	0.82	0.93	0.87

Appendix

Table I.2: Agreement on the

200

-item validation sample between the trace judge (J, GPT-5.4-mini), a blind Claude Opus 5.5 annotator (O) and GPT-5.5 (G): positive labels per coder, pairwise Cohen’s

\kappa

and Fleiss’

\kappa

	Human	Humans	$\kappa$ with the humans			Judge vs. humans
Question	positives	$\kappa$	J	O	G	recall	precision
Price impact	32 / 38	0.82	0.71	0.91	0.85	0.97	0.72
Rival as flow	19 / 18	0.97	0.60	0.78	0.80	0.78	0.60
Infers rival	32 / 33	0.98	0.40	0.90	0.97	0.37	0.86
Cooperation	18 / 25	0.79	0.81	0.90	0.89	0.84	0.86
Punishment	8 / 7	0.93	0.84	0.97	0.96	1.00	0.75

Appendix

Table I.3: Agreement on

100

items of the validation sample between the two human annotators (

\kappa

; positives: annotator 1 / annotator 2), and between the humans and the trace judge (J, GPT-5.4-mini), Claude Opus 5.5 (O) and GPT-5.5 (G), with the judge’s recall and precision against the human labels; both pooled over the annotators.

\kappa

is uninformative for a label one annotator never uses (“scales down”:

98\%

raw agreement).

			rival	gain	messages, judge (%)
Traders	Rival’s orders	$\Delta$	response	from BR	together	more	less	threat/cond.
Qwen3-8B	orders shown	$-6.54_{\pm 2.50}$	$-0.027_{\pm 0.040}$	$5.73_{\pm 3.52}$	84	54	10	0
Qwen3-8B	order flow only	$-4.83_{\pm 1.33}$	$0.000_{\pm 0.000}$	$2.19_{\pm 1.43}$	79	55	24	0
Mistral-7B	orders shown	$-2.35_{\pm 1.03}$	$0.344_{\pm 0.626}$	$32.35_{\pm 24.72}$	16	18	16	6

Appendix

Table J.1: Cheap talk at

s=1

(40 sessions of 200 periods).

\Delta

: collusion index; rival response: change in the rival’s intensity one period after a best-response deviation; gain from BR: the deviator’s discounted gain, in profit units with

\sigma_{v}=1

(a duopolist’s Nash profit per period is

0.24

). Messages: share of the judged messages that do not state

V

to be zero and propose acting together, trading more or trading less, or that threaten or condition the sender’s orders on the rival’s (for Mistral-7B, mostly contrarian: “I’ll buy if you sell”). A second Mistral-7B run with other sampling seeds gives

\Delta=-2.89\pm 1.21

, a rival response of

-0.053\pm 0.706

and a gain of

29.10\pm 16.61

; its messages were not judged. Subscripts: 95% confidence intervals.

Algorithmic Collusion

Algorithmic collusion studies how software agents coordinate to weaken competition, how to detect such behavior, and how to prevent it.

See all 23 papers

Sep 16, 2026

Faithful yet Collusive: Why Chain-of-Thought Monitoring Cannot Detect Collusion in LLM Pricing Agents under Oligopolistic Competition

Large language models (LLM) deployed as autonomous pricing agents may sustain supracompetitive prices through tacit coordination. We develop a causal graph divergence framework that separately measures structural faithfulness and intent faithfulness of LLM pricing agents in Bertrand competition. Across nine LLMs under duopoly and triopoly conditions, collusive behavior and chain-of-thought (CoT) faithfulness dissociate along both dimensions: the most collusive model accurately reports cooperative intent yet reasons structurally unfaithfully, while the most structurally faithful model sustains supra-Nash pricing under both market structures. These findings establish that CoT monitoring alone cannot serve as a standalone safeguard against algorithmic collusion.

Dohun Lee, Hyunwoo Park

Graduate School of Data Science, Seoul National University

CoT Faithfulness LLM Auditing Game Theory Algorithmic Collusion

Ask PDF

Mar 31, 2024

Algorithmic Collusion by Large Language Models

We conduct experiments with algorithmic pricing agents based on Large Language Models (LLMs). In oligopoly settings, LLM-based pricing agents quickly and autonomously reach supracompetitive prices and profits. Variation in seemingly innocuous phrases in LLM instructions ("prompts") substantially influence the degree of supracompetitive pricing. We develop novel techniques for behavioral analysis of LLMs and use them to uncover price-war concerns as a contributing factor. Our results extend to auction settings. Our findings uncover unique challenges to any future regulation of LLM-based pricing agents, and AI-based pricing agents more broadly.

Sara Fish, Yannai A. Gonczarowski, Ran I. Shorrer

†School of Engineering and Applied Sciences, Harvard University · ‡Department of Economics and Department of Computer Science, Harvard University · §Department of Economics, Penn State University

Prompt Sensitivity Algorithmic Collusion Dynamic Pricing Multi-Agent LLMs

Ask PDF

Mar 18, 2026

On the Fragility of AI Agent Collusion

Recent work shows that pricing with symmetric LLM agents leads to algorithmic collusion. We show that collusion is fragile under the heterogeneity typical of real deployments. In a stylized repeated-pricing model, heterogeneity in patience or data access reduces the set of collusive equilibria. Experiments with open-source LLM agents (totaling over 2,000 compute hours) align with these predictions: patience heterogeneity reduces price lift from 22% to 10% above competitive levels; asymmetric data access, to 7%. Increasing the number of competing LLMs breaks up collusion; so does cross-algorithm heterogeneity, that is, setting LLMs against Q-learning agents. But model-size differences (e.g., 32B vs. 14B weights) do not; they generate leader-follower dynamics that stabilize collusion. We discuss antitrust implications, such as enforcement actions restricting data-sharing and policies promoting algorithmic diversity.

Jussi Keppo, Yuze Li, Gerry Tsoukalas +1

National University of Singapore · Boston University

Algorithmic Collusion Multi-Agent LLM Systems

Ask PDF

Abstract

Algorithmic Collusion

Explore similar work

Faithful yet Collusive: Why Chain-of-Thought Monitoring Cannot Detect Collusion in LLM Pricing Agents under Oligopolistic Competition

Algorithmic Collusion by Large Language Models

On the Fragility of AI Agent Collusion