Algorithmic Collusion at Test Time: A Meta-game Design and Evaluation

Yuhong Luo; Daniel Schoepflin; Xintong Wang

cs.MAFeb 19, 2026

Algorithmic Collusion at Test Time: A Meta-game Design and Evaluation

Authors: Yuhong Luo, Daniel Schoepflin, Xintong Wang

Organizations: Rutgers University · DIMACS, Rutgers University

Abstract

The threat of algorithmic collusion, and whether it merits regulatory intervention, remains debated, as existing evaluations of its emergence often rely on long learning horizons, assumptions about counterparty rationality in adopting collusive strategies, and symmetry in hyperparameters and economic settings among players. To study collusion risk, we introduce a meta-game design for analyzing algorithmic behavior under test-time constraints. We model agents as possessing pretrained policies with distinct strategic characteristics (e.g., competitive, naively cooperative, or robustly collusive), and formulate the problem as selecting a meta-strategy that combines a pretrained, initial policy with an in-game adaptation rule. We seek to examine whether collusion can emerge under rational choices and how agents co-adapt toward cooperation or competition. To this end, we sample normal-form empirical games over meta-strategy profiles, compute relevant game statistics (e.g., payoffs against individuals and regret against an equilibrium mixture of opponents), and construct empirical best-response graphs to uncover strategic relationships. We evaluate reinforcement-learning, UCB, and LLM-based strategies in repeated pricing games under symmetric and asymmetric cost settings, and present findings on the feasibility of algorithmic collusion and the effectiveness of pricing strategies in practical ``test-time'' environments. The source code is available at: https://github.com/chailab-rutgers/CollusionMetagame.

Figures & tables

Figure 1: A toy meta-game for repeated Prisoner’s Dilemma on canonical strategies. The NE of this meta-game leads to cooperation. See Example 3.1 for more discussion.

$t=10,000$ , $f=1$		RD 0.5	LC 0.5	LC 0.05	LC 0.005	C 0.5	C 0.05	C 0.005	RC 0.5	RC 0.05	RC 0.005
PSNE	$c_{1}=c_{2}=1$	-	-	-	-	✓	-	-	-	-	-
MSNE	$c_{1}=c_{2}=1$	0.00	0.00	0.00	0.00	0.49	0.00	0.00	0.10	0.28	0.13
NE-Regret ( $\times 10^{-3}$ )	$c_{1}=c_{2}=1$	8.10 $\pm$ 0.41	1.05 $\pm$ 0.43	7.09 $\pm$ 0.56	12.76 $\pm$ 0.68	0.00 $\pm$ 0.37	5.60 $\pm$ 0.53	10.56 $\pm$ 0.57	0.00 $\pm$ 0.49	0.00 $\pm$ 0.51	0.00 $\pm$ 0.59
Uniform Score	$c_{1}=c_{2}=1$	35.21 $\pm$ 0.32	39.44 $\pm$ 0.29	33.08 $\pm$ 0.35	28.01 $\pm$ 0.43	40.96 $\pm$ 0.22	36.10 $\pm$ 0.22	32.64 $\pm$ 0.32	39.66 $\pm$ 0.25	37.90 $\pm$ 0.24	37.52 $\pm$ 0.29
PSNE	$c_{1}=c_{2}=0.8$	-	-	-	✓	-	-	-	-	-	-
MSNE	$c_{1}=c_{2}=0.8$	0.00	0.00	0.00	0.23	0.17	0.00	0.08	0.00	0.00	0.53

Table 1: Max-entropy MSNE, NE-regret (

\times 10^{-3}

), and the uniform score (converted to the CoI scale) evaluated at

t=10,000

for Q-learning with optimistic initialization

f=1

. These metrics are defined in Sec. 3.3 . 8 8 8 Uniform scores in all tables are converted to the CoI scale based on the following transformation:

x\rightarrow\frac{x-\bar{r}^{N}}{\bar{r}^{M}-\bar{r}^{N}}\times 100\%

, where

x

is the expected payoff of a meta-strategy in a stage game when the opponent’s strategy is drawn from a uniform distribution over all meta-strategies. We denote

\bar{r}^{N}

as the average competitive payoff and

\bar{r}^{M}

the average monopoly payoff (Sec. 2 ). The best-performing strategy is highlighted in bold . The second-best is underlined . The

\pm

symbol denotes the 95% confidence interval.

$t=10,000$	$f=1$						$f=0.5$
Meta-strategies	C 0.5	C 0.05	C 0.005	RC 0.5	RC 0.05	RC 0.005	C 0.5	C 0.05	C 0.005	RC 0.5	RC 0.05	RC 0.005
PSNE	-	-	-	-	-	-	-	-	-	✓	✓	-
MSNE	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.63	0.37	0.00
NE-Regret ( $\times 10^{-3}$ )	10.20 $\pm$ 1.78	16.77 $\pm$ 1.31	16.88 $\pm$ 1.26	3.95 $\pm$ 1.67	8.84 $\pm$ 1.19	4.39 $\pm$ 1.37	4.03 $\pm$ 1.49	6.73 $\pm$ 1.53	13.58 $\pm$ 1.56	0.00 $\pm$ 1.20	0.00 $\pm$ 1.41	4.48 $\pm$ 1.40
Uniform Score	32.41 $\pm$ 0.81	21.14 $\pm$ 0.89	18.68 $\pm$ 0.90	31.90 $\pm$ 0.72	26.35 $\pm$ 0.62	26.76 $\pm$ 0.62	28.29 $\pm$ 0.87	24.63 $\pm$ 0.82	19.18 $\pm$ 0.84	31.47 $\pm$ 0.64	29.76 $\pm$ 0.66	27.04 $\pm$ 0.76

Table 2: Meta-game evaluation of meta-strategies with different Q-value initializations, including

f=\{1,0.5\}

. Payoffs are evaluated at

t=10,000

$t=10,000$	LC 1	C 1	RC 1	LC 0.5	C 0.5	RC 0.5	LC 0.05	C 0.05	RC 0.05	LC 0.005	C 0.005	RC 0.005	RD 0.005
PSNE	-	-	-	-	✓	✓	-	-	-	-	-	-	-
MSNE	0.00	0.00	0.00	0.00	0.50	0.50	0.00	0.00	0.00	0.00	0.00	0.00	0.00
NE-Regret ( $\times 10^{-3}$ )	13.83 $\pm$ 0.93	1.13 $\pm$ 1.34	0.63 $\pm$ 1.24	13.93 $\pm$ 0.88	0.00 $\pm$ 1.32	0.00 $\pm$ 1.21	15.13 $\pm$ 0.94	2.29 $\pm$ 1.36	2.11 $\pm$ 1.30	19.67 $\pm$ 0.98	5.46 $\pm$ 1.42	6.30 $\pm$ 1.36	39.66 $\pm$ 0.17
Uniform Score	55.23 $\pm$ 0.63	62.68 $\pm$ 0.82	61.66 $\pm$ 0.77	55.59 $\pm$ 0.62	63.14 $\pm$ 0.81	62.02 $\pm$ 0.76	54.32 $\pm$ 0.64	61.56 $\pm$ 0.83	59.43 $\pm$ 0.79	50.38 $\pm$ 0.66	58.19 $\pm$ 0.86	55.97 $\pm$ 0.83	35.35 $\pm$ 0.13

Table 3: Max-entropy MSNE, NE-regret (

\times 10^{-3}

), and uniform score (converted to CoI scale) among UCB meta-strategies at

t=10,000

	p2h3	p0h0	p1h2	p2h1	p1h1	p0h2
PSNE	✓	-	-	-	-	✓
MSNE	0.44	0.00	0.00	0.00	0.00	0.56
NE-Regret $(\times 10^{-3})$	0.00 $\pm$ 7.92	58.76 $\pm$ 7.40	47.38 $\pm$ 9.13	50.37 $\pm$ 9.28	59.47 $\pm$ 6.14	0.00 $\pm$ 9.32
Uniform Score	36.79 $\pm$ 4.86	10.84 $\pm$ 2.95	15.08 $\pm$ 3.83	14.54 $\pm$ 3.57	10.80 $\pm$ 2.57	25.55 $\pm$ 5.44

Table 4: Evaluation scores among LLM strategies evaluated at

t=50

Appendix figures & tables32 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: We collect 12 pairs of pretrained Q-learning policies with 15 discretized actions and use them as initial policies for test-time evaluation. We measure total payoffs from $t=0$ to $t=100$ , averaged over 100 random seeds. In green , each policy is paired with its original pretraining pair. In red , each pretrained policy ( $\pi_{j}$ ) is paired with its best response ( $\pi_{-j}$ ). In orange and blue , we show payoffs for randomly sampled, independently pretrained pairs without adaptation: orange points lie closer (in L2 distance) to the red cluster, while blue points are closer to the green .

Figure 5: With 4 discretized actions, the 500 Q-learning pretrained policies form three distinct clusters when laid out along PC (Def. 3.1 ) and CR (Def. 3.2 ), corresponding to the competitive (LC), naively collusive (C), and robustly collusive (RC) categories introduced in Sec. 4.1.1 .

Figure 6: The PC (Def. 3.1 ) and CR (Def. 3.2 ) of 1,000 pretrained Q-learning policies with 4 and 15 discretized actions, respectively, under the experiment setting described in Sec. 4.2 .

Figure 7: Categorization of pretrained policies with 15 discretized actions obtained via Q-learning (Left) and UCB (Right), under the experiment settings described in Sec. 4.2 and 4.3 .

$t=10,000$	C 0.3	C 0.4	C 0.5	C 0.6	C 0.7	RC 0.3	RC 0.4	RC 0.5	RC 0.6	RC 0.7
PSNE	-	-	-	-	-	✓	-	✓	-	-
MSNE	0.00	0.00	0.00	0.00	0.00	0.40	0.00	0.60	0.00	0.00
NE-Regret ( $\times 10^{-3}$ )	2.07 $\pm$ 1.73	1.88 $\pm$ 1.62	3.26 $\pm$ 1.55	2.80 $\pm$ 1.56	3.55 $\pm$ 1.44	0.00 $\pm$ 1.49	0.40 $\pm$ 1.40	0.00 $\pm$ 1.45	1.71 $\pm$ 1.33	2.60 $\pm$ 1.38
Uniform Score	46.38 $\pm$ 1.04	47.40 $\pm$ 0.95	47.47 $\pm$ 0.90	47.55 $\pm$ 0.82	46.71 $\pm$ 0.76	49.85 $\pm$ 1.01	49.20 $\pm$ 0.94	48.99 $\pm$ 0.93	48.02 $\pm$ 0.87	47.77 $\pm$ 0.86

Appendix

Table 5: Robustness test on varying the test-time learning rates for Q-learning pretrained policies.

t=10,000

Figure 8: Best-response graph for C and RC with a finer grid of learning rates.

Q-scale	$t=10,000$	RD 0.5	LC 0.5	LC 0.05	LC 0.005	C 0.5	C 0.05	C 0.005	RC 0.5	RC 0.05	RC 0.005
$f=0.5$	PSNE	-	✓	-	-	✓	-	✓	-	✓	-
	MSNE	0.00	0.19	0.00	0.00	0.34	0.00	0.00	0.39	0.08	0.00
	NE-Regret ( $\times 10^{-3}$ )	6.78 $\pm$ 1.15	0.00 $\pm$ 1.14	6.24 $\pm$ 1.21	15.19 $\pm$ 1.28	0.00 $\pm$ 1.15	1.90 $\pm$ 1.23	10.60 $\pm$ 1.20	0.00 $\pm$ 1.04	0.00 $\pm$ 1.18	5.00 $\pm$ 1.24
	Uniform Score	27.68 $\pm$ 1.00	32.78 $\pm$ 0.91	28.43 $\pm$ 0.94	21.08 $\pm$ 0.94	33.98 $\pm$ 0.80	30.45 $\pm$ 0.78	24.56 $\pm$ 0.76	31.70 $\pm$ 0.90	29.57 $\pm$ 0.87	25.97 $\pm$ 0.82
$f=0.0$	Mixed & PSNE	0.00	0.00	0.00	0.00	0.00	0.00	0.00	1.00	0.00	0.00
	NE-Regret ( $\times 10^{-3}$ )	7.22 $\pm$ 1.29	1.91 $\pm$ 1.13	10.85 $\pm$ 1.28	11.79 $\pm$ 1.49	0.54 $\pm$ 1.05	9.73 $\pm$ 1.14	9.35 $\pm$ 1.22	0.00 $\pm$ 1.01	5.22 $\pm$ 1.05	5.60 $\pm$ 1.06

Appendix

Table 6: Meta-games on Q-learning using

f=0.5

and

f=0

respectively, with a time horizon of

t=10,000

. In the case of

f=0

, we additionally incorporate

\varepsilon=1

with a decay rate of

0.001

to ensure a policy still updates. The best-performing strategy is highlighted in bold , and the second-best is underlined .

	RD 0.5	LC 0.5	LC 0.05	LC 0.005	C 0.5	C 0.05	C 0.005	RC 0.5	RC 0.05	RC 0.005
RD 0.5	0.29, 0.29	0.27, 0.30	0.26, 0.29	0.27, 0.28	0.27, 0.31	0.26, 0.31	0.26, 0.30	0.26, 0.30	0.25, 0.30	0.25, 0.29
LC 0.5	0.30, 0.27	0.28, 0.28	0.27, 0.28	0.27, 0.27	0.28, 0.29	0.26, 0.28	0.27, 0.27	0.27, 0.28	0.25, 0.29	0.25, 0.28
LC 0.05	0.29, 0.26	0.28, 0.27	0.26, 0.26	0.26, 0.26	0.27, 0.26	0.26, 0.27	0.25, 0.26	0.27, 0.27	0.25, 0.27	0.24, 0.27
LC 0.005	0.28, 0.27	0.27, 0.27	0.26, 0.26	0.26, 0.26	0.26, 0.27	0.25, 0.26	0.24, 0.26	0.26, 0.27	0.25, 0.26	0.24, 0.26
C 0.5	0.31, 0.27	0.29, 0.28	0.26, 0.27	0.27, 0.26	0.28, 0.28	0.26, 0.27	0.27, 0.26	0.27, 0.28	0.24, 0.28	0.24, 0.28
C 0.05	0.31, 0.26	0.28, 0.26	0.27, 0.26	0.26, 0.25	0.27, 0.26	0.26, 0.26	0.25, 0.26	0.27, 0.26	0.23, 0.26	0.26, 0.26

Appendix

Table 7: Average payoffs over 4,000 runs in the Q-learning meta-game payoff matrix. Standard errors are omitted as they are negligible in the order of

10^{-4}

. For reference, in this setting, the competitive and monopoly payoffs are

\bar{r}^{N}=0.22

and

\bar{r}^{M}=0.34

Panel of Figure 9 — Figure 9: $\Delta_{t}$ (CR) and $\Delta_{t}$ (PC) for Q-learning-based meta-strategies under symmetric costs $c_{1}=c_{2}=1$ , $f=1$ and $t=10,000$ .

	RD 0.5	LC 0.5	LC 0.05	LC 0.005	C 0.5	C 0.05	C 0.005	RC 0.5	RC 0.05	RC 0.005
RD 0.5	0.35, 0.35	0.29, 0.34	0.28, 0.33	0.28, 0.31	0.30, 0.35	0.29, 0.34	0.30, 0.33	0.30, 0.37	0.26, 0.35	0.24, 0.33
LC 0.5	0.34, 0.29	0.30, 0.30	0.29, 0.30	0.28, 0.31	0.31, 0.31	0.28, 0.31	0.29, 0.30	0.32, 0.30	0.28, 0.32	0.25, 0.31
LC 0.05	0.33, 0.28	0.30, 0.29	0.30, 0.30	0.29, 0.30	0.30, 0.29	0.28, 0.30	0.27, 0.29	0.32, 0.28	0.29, 0.30	0.25, 0.30
LC 0.005	0.31, 0.28	0.31, 0.28	0.30, 0.29	0.29, 0.29	0.30, 0.29	0.28, 0.28	0.26, 0.29	0.32, 0.27	0.30, 0.28	0.26, 0.29
C 0.5	0.35, 0.30	0.31, 0.31	0.29, 0.30	0.29, 0.30	0.31, 0.31	0.29, 0.31	0.29, 0.30	0.33, 0.30	0.28, 0.32	0.25, 0.32
C 0.05	0.34, 0.29	0.31, 0.28	0.30, 0.28	0.28, 0.28	0.31, 0.29	0.27, 0.27	0.26, 0.28	0.32, 0.28	0.29, 0.27	0.25, 0.28

Appendix

Table 8: Average payoffs over 4,000 runs in the Q-learning meta-game payoff matrix. Standard errors are omitted as they are negligible in the order of

10^{-4}

. For reference, in this setting, the competitive and monopoly payoffs are

\bar{r}^{N}=0.24

and

\bar{r}^{M}=0.41

Figure 10: Best-response graph for Q-learning with $c_{1}=c_{2}=0.8$ , $f=1$ and $t=10,000$ .

Figure 11:

\Delta_{t}

(CR) and

\Delta_{t}

(PC) for Q-learning with

c_{1}=c_{2}=0.8

f=1

and

t=10,000

$t=3,000$ , $f=1$		RD 0.5	LC 0.5	LC 0.05	LC 0.005	C 0.5	C 0.05	C 0.005	RC 0.5	RC 0.05	RC 0.005
MSNE	$c_{1}=c_{2}=1$	0.00	0.17	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.83
NE-Regret ( $\times 10^{-3}$ )	$c_{1}=c_{2}=1$	7.75 $\pm$ 0.52	0.00 $\pm$ 0.59	2.77 $\pm$ 0.59	5.80 $\pm$ 0.53	3.31 $\pm$ 0.59	7.24 $\pm$ 0.45	7.86 $\pm$ 0.39	2.21 $\pm$ 0.51	3.17 $\pm$ 0.44	0.00 $\pm$ 0.42
Uniform Score	$c_{1}=c_{2}=1$	22.89 $\pm$ 0.27	29.93 $\pm$ 0.29	27.42 $\pm$ 0.32	23.90 $\pm$ 0.45	30.84 $\pm$ 0.21	29.92 $\pm$ 0.23	28.45 $\pm$ 0.28	31.50 $\pm$ 0.21	32.28 $\pm$ 0.20	32.66 $\pm$ 0.24
MSNE	$c_{1}=1.0$	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	1.00
MSNE	$c_{2}=0.8$	0.00	1.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00
NE-Regret ( $\times 10^{-3}$ )	$c_{1}=1.0$	24.32 $\pm$ 0.64	15.06 $\pm$ 1.01	12.11 $\pm$ 0.76	11.76 $\pm$ 0.82	18.08 $\pm$ 0.82	12.58 $\pm$ 0.66	6.73 $\pm$ 0.69	12.50 $\pm$ 0.81	7.44 $\pm$ 0.68	0.00 $\pm$ 0.74

Appendix

Table 9: Max-entropy MSNE, NE-Regret and Payoffs against uniformly drawn meta-strategies for Q-learning with shorter time horizon

t=3,000

. For Uniform-score, we let

x\rightarrow((x-\bar{r}^{N})/(\bar{r}^{M}-\bar{r}^{N}))\cdot 100\%

	RD 0.5	LC 0.5	LC 0.05	LC 0.005	C 0.5	C 0.05	C 0.005	RC 0.5	RC 0.05	RC 0.005
RD 0.5	0.28, 0.28	0.25, 0.29	0.25, 0.29	0.26, 0.28	0.25, 0.30	0.24, 0.30	0.25, 0.30	0.24, 0.30	0.23, 0.29	0.23, 0.28
LC 0.5	0.29, 0.25	0.27, 0.27	0.26, 0.27	0.26, 0.26	0.26, 0.27	0.25, 0.27	0.26, 0.26	0.25, 0.27	0.24, 0.28	0.24, 0.27
LC 0.05	0.29, 0.25	0.27, 0.26	0.26, 0.26	0.25, 0.26	0.26, 0.26	0.24, 0.26	0.24, 0.26	0.26, 0.26	0.24, 0.26	0.24, 0.27
LC 0.005	0.28, 0.26	0.26, 0.26	0.26, 0.25	0.25, 0.25	0.25, 0.27	0.24, 0.26	0.23, 0.26	0.25, 0.26	0.24, 0.26	0.23, 0.26
C 0.5	0.30, 0.25	0.27, 0.26	0.26, 0.26	0.27, 0.25	0.27, 0.27	0.25, 0.26	0.25, 0.26	0.25, 0.26	0.23, 0.27	0.23, 0.27
C 0.05	0.30, 0.24	0.27, 0.25	0.26, 0.24	0.26, 0.24	0.26, 0.25	0.25, 0.25	0.24, 0.25	0.26, 0.25	0.24, 0.25	0.23, 0.25

Appendix

Table 10: Average payoffs over 4,000 runs in the Q-learning meta-game payoff matrix. Standard errors are omitted as they are negligible in the order of

10^{-4}

. For reference, in this setting, the competitive and monopoly payoffs are

\bar{r}^{N}=0.22

and

\bar{r}^{M}=0.34

Figure 12: Best-response graph for Q-learning under symmetric costs $c_{1}=c_{2}=1$ with optimistic initialization $f=1$ and a time horizon of $t=3,000$ .

Figure 13:

\Delta_{t}

(CR) and

\Delta_{t}

(PC) for Q-learning under symmetric costs

c_{1}=c_{2}=1

with optimistic initialization

f=1

and a time horizon of

t=3,000

	RD 0.5	LC 0.5	LC 0.05	LC 0.005	C 0.5	C 0.05	C 0.005	RC 0.5	RC 0.05	RC 0.005
RD 0.5	0.28, 0.38	0.17, 0.40	0.17, 0.40	0.17, 0.41	0.19, 0.39	0.19, 0.38	0.21, 0.37	0.16, 0.40	0.16, 0.40	0.17, 0.39
LC 0.5	0.28, 0.35	0.18, 0.40	0.18, 0.41	0.17, 0.41	0.19, 0.39	0.19, 0.38	0.21, 0.37	0.17, 0.40	0.15, 0.39	0.16, 0.39
LC 0.05	0.25, 0.35	0.18, 0.39	0.18, 0.40	0.18, 0.39	0.17, 0.37	0.17, 0.37	0.17, 0.37	0.18, 0.38	0.15, 0.38	0.14, 0.38
LC 0.005	0.23, 0.36	0.19, 0.39	0.18, 0.38	0.18, 0.38	0.17, 0.37	0.17, 0.36	0.17, 0.36	0.18, 0.38	0.16, 0.37	0.15, 0.37
C 0.5	0.27, 0.35	0.18, 0.40	0.17, 0.40	0.17, 0.41	0.18, 0.38	0.19, 0.38	0.21, 0.36	0.17, 0.39	0.15, 0.39	0.15, 0.39
C 0.05	0.24, 0.35	0.18, 0.39	0.18, 0.39	0.18, 0.39	0.17, 0.37	0.17, 0.36	0.17, 0.36	0.18, 0.38	0.15, 0.37	0.14, 0.37

Appendix

Table 11: Average payoffs over 4,000 runs in the payoff matrix of Q-learning-based strategies under asymmetric costs,

c_{1}=1

and

c_{2}=0.8

, and

t=10,000,f=1

. Standard errors are omitted as they are negligible in the order of

10^{-4}

. For reference, in this setting, the competitive and monopoly payoffs for player 1 are

\bar{r}_{c=1.0}^{N}=0.17

and

\bar{r}_{c=1.0}^{M}=0.30

, and for player 2 are

\bar{r}_{c=1.0}^{N}=0.31

and

\bar{r}_{c=0.8}^{M}=0.44

Figure 16:

\Delta_{t}

(CR) and

\Delta_{t}

(PC). The high-cost strategies are plotted with fillings.

	C 0.5 f=1	C 0.05 f=1	C 0.005 f=1	RC 0.5 f=1	RC 0.05 f=1	RC 0.005 f=1	C 0.5 f=0.5	C 0.05 f=0.5	C 0.005 f=0.5	RC 0.5 f=0.5	RC 0.05 f=0.5	RC 0.005 f=0.5
C 0.5 f=1	0.28, 0.28	0.26, 0.27	0.27, 0.26	0.27, 0.28	0.24, 0.28	0.24, 0.28	0.27, 0.27	0.27, 0.27	0.27, 0.26	0.24, 0.28	0.24, 0.28	0.25, 0.27
C 0.05 f=1	0.27, 0.26	0.26, 0.26	0.25, 0.26	0.27, 0.26	0.25, 0.26	0.23, 0.26	0.25, 0.26	0.25, 0.26	0.25, 0.26	0.24, 0.26	0.23, 0.26	0.23, 0.26
C 0.005 f=1	0.26, 0.27	0.26, 0.25	0.25, 0.25	0.26, 0.26	0.25, 0.25	0.23, 0.25	0.25, 0.26	0.24, 0.25	0.24, 0.25	0.24, 0.25	0.23, 0.26	0.23, 0.25
RC 0.5 f=1	0.28, 0.27	0.26, 0.27	0.26, 0.26	0.27, 0.27	0.25, 0.28	0.24, 0.27	0.26, 0.27	0.26, 0.26	0.27, 0.26	0.25, 0.27	0.25, 0.28	0.25, 0.27
RC 0.05 f=1	0.28, 0.24	0.26, 0.25	0.25, 0.25	0.28, 0.25	0.25, 0.25	0.24, 0.26	0.26, 0.26	0.25, 0.25	0.25, 0.25	0.25, 0.26	0.24, 0.26	0.23, 0.26
RC 0.005 f=1	0.28, 0.24	0.26, 0.23	0.25, 0.23	0.27, 0.24	0.26, 0.24	0.24, 0.24	0.25, 0.25	0.25, 0.24	0.25, 0.24	0.25, 0.25	0.24, 0.25	0.23, 0.24

Appendix

Table 12: Average payoffs over 4,000 runs in the payoff matrix of Q-learning-based strategies under symmetric costs

c_{1}=c_{2}=1

with either optimistic initialization

f=1

or pessimistic initialization

f=0.5

. Time horizon

t=10,000

. For reference, in this setting, the competitive and monopoly payoffs are

\bar{r}^{N}=0.22

and

\bar{r}^{M}=0.34

Figure 17: Best-response graph for Q-learning with $c_{1}=c_{2}=1$ , $f\in\{1,0.5\}$ and $t=10,000$ .

Figure 18:

\Delta_{t}

(CR) and

\Delta_{t}

(PC) for Q-learning with

c_{1}=c_{2}=1

f\in\{1,0.5\}

and

t=10,000

Figure 19: The PC (Def. 3.1 ) and CR (Def. 3.2 ) of 1000 pretrained UCB policies with the setting described in Sec. 4.3 .

Figure 20: Best-response graph for UCBs. $c_{1}=c_{2}=1$ , $t=10,000$ .

	UCB LC	UCB C	UCB RC	UCB LC 0.5	UCB C 0.5	UCB RC 0.5	UCB LC 0.05	UCB C 0.05	UCB RC 0.05	UCB LC 0.005	UCB C 0.005	UCB RC 0.005	UCB RD 0.005
UCB LC	0.29, 0.29	0.29, 0.29	0.29, 0.29	0.29, 0.29	0.29, 0.29	0.29, 0.29	0.29, 0.30	0.29, 0.29	0.29, 0.28	0.29, 0.29	0.29, 0.28	0.29, 0.28	0.29, 0.26
UCB C	0.29, 0.29	0.30, 0.30	0.30, 0.30	0.29, 0.29	0.30, 0.30	0.29, 0.30	0.29, 0.29	0.30, 0.30	0.30, 0.29	0.30, 0.28	0.30, 0.29	0.30, 0.29	0.29, 0.26
UCB RC	0.29, 0.28	0.30, 0.30	0.29, 0.29	0.28, 0.28	0.30, 0.30	0.30, 0.30	0.29, 0.29	0.30, 0.30	0.30, 0.28	0.29, 0.29	0.30, 0.28	0.29, 0.28	0.29, 0.26
UCB LC 0.5	0.29, 0.29	0.29, 0.29	0.29, 0.28	0.29, 0.29	0.29, 0.29	0.29, 0.29	0.29, 0.30	0.29, 0.29	0.29, 0.28	0.30, 0.28	0.29, 0.28	0.29, 0.26	0.29, 0.29
UCB C 0.5	0.29, 0.29	0.30, 0.30	0.30, 0.29	0.29, 0.29	0.30, 0.30	0.30, 0.30	0.29, 0.29	0.30, 0.30	0.30, 0.29	0.30, 0.28	0.30, 0.29	0.30, 0.29	0.29, 0.26
UCB RC 0.5	0.29, 0.28	0.30, 0.30	0.30, 0.29	0.29, 0.28	0.30, 0.30	0.29, 0.28	0.30, 0.30	0.30, 0.29	0.29, 0.28	0.30, 0.29	0.30, 0.29	0.29, 0.26	0.30, 0.29

Appendix

Table 13: Average payoffs over 4,000 runs in the payoff matrix of UCB-based strategies, with symmetric cost

c_{1}=c_{2}=1

and time horizon

t=10,000

. For reference, the competitive payoff is

\bar{r}^{N}=0.22

, and the monopoly payoff is

\bar{r}^{M}=0.34

$t=10,000$	Q-RD 0.5	UCB LC	UCB C	UCB RC	UCB LC 0.5	UCB C 0.5	UCB RC 0.5	UCB LC 0.05	UCB C 0.05	UCB RC 0.05	UCB LC 0.005	UCB C 0.005	UCB RC 0.005	UCB RD 0.005
PSNE	✓	-	-	✓	-	✓	✓	-	-	✓	-	-	-	-
MSNE	0.48	0.00	0.00	0.50	0.00	0.01	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00
NE-Regret ( $\times 10^{-3}$ )	0.00 $\pm$ 1.96	16.96 $\pm$ 2.58	0.11 $\pm$ 2.37	0.00 $\pm$ 2.63	16.35 $\pm$ 2.57	0.00 $\pm$ 2.39	0.33 $\pm$ 2.56	18.51 $\pm$ 2.58	2.08 $\pm$ 2.58	2.03 $\pm$ 2.66	24.19 $\pm$ 2.98	6.94 $\pm$ 2.47	6.53 $\pm$ 2.65	34.16 $\pm$ 0.43
Uniform Score	57.14 $\pm$ 1.51	53.38 $\pm$ 1.35	64.37 $\pm$ 2.10	63.33 $\pm$ 1.94	54.09 $\pm$ 1.38	64.78 $\pm$ 2.13	63.52 $\pm$ 1.87	52.70 $\pm$ 1.38	63.08 $\pm$ 2.20	61.24 $\pm$ 1.91	48.63 $\pm$ 1.47	59.79 $\pm$ 2.27	58.10 $\pm$ 1.97	35.72 $\pm$ 0.29

Appendix

Table 14: The metrics for UCB with the addition of the (RD, 0.5) of Q-learning.

c_{1}=c_{2}=1

and

t=10,000

Figure 21: Changes in CR (

\Delta_{t}

(CR)) and PC (

\Delta_{t}

(PC)) over time for UCB.

Figure 22: Best-response graph of UCB with the addition of a Q-learning agent (RD, 0.5).

Figure 23: The PC (Def. 3.1 ) (x-axis) and CR (Def. 3.2 ) (y-axis) of initial policies from GPT-5 models. The setting is described in Sec. 4.4 .

	p2h3	p0h0	p1h2	p2h1	p1h1	p0h2
p2h3	0.29, 0.29	0.23, 0.25	0.25, 0.26	0.26, 0.29	0.23, 0.25	0.31, 0.31
p0h0	0.25, 0.23	0.23, 0.23	0.23, 0.22	0.23, 0.23	0.23, 0.23	0.24, 0.23
p1h2	0.26, 0.25	0.22, 0.23	0.24, 0.24	0.23, 0.23	0.23, 0.24	0.25, 0.24
p2h1	0.29, 0.26	0.23, 0.23	0.23, 0.23	0.23, 0.23	0.22, 0.23	0.23, 0.21
p1h1	0.25, 0.23	0.23, 0.23	0.24, 0.23	0.23, 0.22	0.23, 0.23	0.24, 0.22
p0h2	0.31, 0.31	0.23, 0.24	0.24, 0.25	0.21, 0.23	0.22, 0.24	0.30, 0.30

Appendix

Table 15: Average payoffs over 40 runs in the payoff matrix of LLMs (

c_{1}=c_{2}=1

and

t=50

). For reference, in this setting, the competitive and monopoly payoffs are

\bar{r}^{N}=0.22

and

\bar{r}^{M}=0.34

Figure 25: Best-response graph for LLM players. $c_{1}=c_{2}=1$ and payoffs are evaluated at $t=50.$

Algorithmic Collusion

Algorithmic collusion studies how software agents coordinate to weaken competition, how to detect such behavior, and how to prevent it.

See all 23 papers

On the Fragility of AI Agent Collusion
Mar 18, 2026Jussi Keppo, Yuze Li, Gerry Tsoukalas +1Algorithmic Collusion Multi-Agent LLM Systems
National University of Singapore · Boston University
Mitigating Retaliatory Algorithmic Collusion in Repeated Games
Sep 17, 2026Karthik Sivachandran, Rohan PalejaAlgorithmic Collusion Repeated Games
Department of Computer Science Purdue University
Collusion with Competitive Marginals: Price-Level Audits Are Blind by Construction
Jul 29, 2026Xin Xu, Chengrui Wu, Jiayu Lu +3Algorithmic Auditing Automated Bidding
Carnegie Mellon University · Zhejiang University

Mar 18, 2026J/K move · Enter open · S save

On the Fragility of AI Agent Collusion

Jussi Keppo, Yuze Li, Gerry Tsoukalas, Nuo Yuan

National University of Singapore · Boston University

Algorithmic Collusion Multi-Agent LLM Systems

Open paper AskPDF arXiv

Recent work shows that pricing with symmetric LLM agents leads to algorithmic collusion. We show that collusion is fragile under the heterogeneity typical of real deployments. In a stylized repeated-pricing model, heterogeneity in patience or data access reduces the set of collusive equilibria. Experiments with open-source LLM agents (totaling over 2,000 compute hours) align with these predictions: patience heterogeneity reduces price lift from 22% to 10% above competitive levels; asymmetric data access, to 7%. Increasing the number of competing LLMs breaks up collusion; so does cross-algorithm heterogeneity, that is, setting LLMs against Q-learning agents. But model-size differences (e.g., 32B vs. 14B weights) do not; they generate leader-follower dynamics that stabilize collusion. We discuss antitrust implications, such as enforcement actions restricting data-sharing and policies promoting algorithmic diversity.

Abstract

Algorithmic Collusion

Explore similar work

On the Fragility of AI Agent Collusion

Mitigating Retaliatory Algorithmic Collusion in Repeated Games

Collusion with Competitive Marginals: Price-Level Audits Are Blind by Construction