Detecting Multi-Agent Collusion Through Multi-Agent Interpretability

Aaron Rose; Carissa Cullen; Sahar Abdelnabi; Philip Torr; Brandon Gary Kaplowitz; Christian Schroeder de Witt

cs.AIApr 1, 2026

Detecting Multi-Agent Collusion Through Multi-Agent Interpretability

Authors: Aaron Rose, Carissa Cullen, Sahar Abdelnabi, Philip Torr, Brandon Gary Kaplowitz, Christian Schroeder de Witt

Organizations: University of Oxford · ELLIS Institute Tübingen MPI-IS

Abstract

As LLM agents are increasingly deployed in multi-agent systems, they introduce risks of covert coordination that may evade standard forms of human oversight. While linear probes on model activations have shown promise for detecting deception in single-agent settings, collusion is inherently a multi-agent phenomenon, and the use of internal representations for detecting collusion between agents remains unexplored. We introduce NARCBench, a benchmark for evaluating collusion detection under environment distribution shift, and propose five probing techniques that aggregate per-agent deception scores to classify scenarios at the group level, evaluated across four open-weight models (Qwen3-32B, Llama-3.1-70B, DeepSeek-R1 32B, GPT-OSS-20B) and six probe architectures. We frame this as a distributed anomaly detection problem, identifying three collusion signatures that map onto distinct anomaly types and detection paradigms. Every model reaches 1.00 AUROC in-distribution; on our strongest model (Llama-3.1-70B), our five probing techniques achieve 0.73 to 0.93 AUROC when transferred zero-shot to structurally different multi-agent scenarios and 0.99 to 1.00 on a steganographic blackjack card-counting task, with detection performance scaling with model capability. We find that no single probing technique dominates across all collusion types, consistent with the framework's prediction that different anomaly types require different detection paradigms. This work takes a step toward multi-agent interpretability: extending white-box inspection from single models to multi-agent contexts, where detection requires aggregating signals across agents. These results suggest that model internals provide a complementary signal to text-level monitoring for detecting multi-agent collusion. Code and data available at https://github.com/aaronrose227/narcbench.

Figures & tables

Figure 1: Overview of our methodology. Activations are extracted from multi-agent interactions, projected onto a learned deception direction, and aggregated across agents to classify scenarios.

Figure 2: Distributed detection paradigms and how each processes multi-agent data to classify scenarios.

Figure 3: Per-test AUROC for each model on TRANSFER (solid; mean over six OOD families) and STEGO (hatched; twenty blackjack shoes). Means are taken across the model’s probe-layer range and $10$ seeds.

Figure 4: ROC curves on Llama-3.1-70B. Bands: 95% bootstrap CIs. Parentheses: AUROC.

Method	AUROC
Sorted Concat	$0.728\pm 0.006$
Activation Alignment	$0.638\pm 0.002$
Deception Split	$0.512\pm 0.002$
Asymmetry Probe	$0.495\pm 0.003$
Peak Suspicion	$0.490\pm 0.004$

Table 1: Zero-shot AUROC (mean

\pm

std) of the five NARCBench Core detection methods on Verdict (Reflection phase).

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

	Sorted Concatenation	Act Alignment	Peak Suspicion	Deception Split	Asymmetry Probe
Core	$\mathbf{1.00}$	$\mathbf{1.00}$	$\mathbf{1.00}$	$\mathbf{1.00}$	$\mathbf{1.00}$
Transfer (all)	$0.82$	$0.48$	$0.84$	$\mathbf{0.88}$	$0.87$
CI/CD	$0.53$	$0.28$	$0.32$	$\mathbf{0.73}$	$0.48$
Code Review	$0.94$	$0.39$	$\mathbf{1.00}$	$0.99$	$0.99$
News Framing	$0.71$	$0.71$	$0.94$	$\mathbf{0.96}$	$0.94$
Pump & Dump	$\mathbf{0.99}$	$0.32$	$0.97$	$0.83$	$0.96$

Appendix

Table 2: Per-method AUROC across NARCBench tiers for Qwen3-32B. Bold = best per row. OOD Mean is computed across the available out-of-distribution scenarios ( Transfer families plus Stego ).

	Sorted Concatenation	Act Alignment	Peak Suspicion	Deception Split	Asymmetry Probe
Core	$\mathbf{1.00}$	$0.97$	$\mathbf{1.00}$	$\mathbf{1.00}$	$\mathbf{1.00}$
Transfer (all)	$0.89$	$0.73$	$0.90$	$0.92$	$\mathbf{0.93}$
CI/CD	$\mathbf{0.96}$	$0.84$	$0.89$	$0.89$	$0.92$
Code Review	$0.99$	$0.67$	$\mathbf{1.00}$	$\mathbf{1.00}$	$\mathbf{1.00}$
News Framing	$0.93$	$0.83$	$0.96$	$0.97$	$\mathbf{0.99}$
Pump & Dump	$0.95$	$0.93$	$0.98$	$0.99$	$\mathbf{0.99}$

Appendix

Table 3: Per-method AUROC across NARCBench tiers for Llama-3.1 70B. Bold = best per row.

	Sorted Concatenation	Act Alignment	Peak Suspicion	Deception Split	Asymmetry Probe
Core	$\mathbf{1.00}$	$0.95$	$\mathbf{1.00}$	$\mathbf{1.00}$	$\mathbf{1.00}$
Transfer (all)	$0.75$	$0.58$	$0.84$	$0.69$	$\mathbf{0.86}$
CI/CD	$0.28$	$\mathbf{0.67}$	$0.58$	$0.66$	$0.66$
Code Review	$0.87$	$0.91$	$0.91$	$\mathbf{0.93}$	$0.93$
News Framing	$0.88$	$0.50$	$\mathbf{0.99}$	$0.73$	$0.98$
Pump & Dump	$0.84$	$0.61$	$0.87$	$0.77$	$\mathbf{0.92}$

Appendix

Table 4: Per-method AUROC across NARCBench tiers for DeepSeek-R1 32B. Bold = best per row.

	Sorted Concatenation	Act Alignment	Peak Suspicion	Deception Split	Asymmetry Probe
Core	$\mathbf{1.00}$	$0.91$	$\mathbf{1.00}$	$\mathbf{1.00}$	$\mathbf{1.00}$
Transfer (all)	$0.62$	$\mathbf{0.72}$	$0.56$	$0.65$	$0.64$
CI/CD	$0.76$	$\mathbf{0.98}$	$0.77$	$0.85$	$0.83$
Code Review	$0.77$	$0.57$	$\mathbf{0.80}$	$0.63$	$0.77$
News Framing	$-$	$-$	$-$	$-$	$-$
Pump & Dump	$-$	$-$	$-$	$-$	$-$

Appendix

Table 5: Per-method AUROC across NARCBench tiers for GPT-OSS 20B. Bold = best per row. The News Framing, Pump & Dump and RAG Poisoning families produced no analysable collusion runs (every collusion attempt was refused or failed audit) and are marked “–”; Transfer (all) and OOD Mean are computed across the available families plus Stego .

Figure 5: OOD-mean AUROC ( Transfer + Stego ) plotted against model parameter count. Detection performance scales approximately monotonically with model capability across the four open-weight models we tested.

Probe	Qwen3-32B	Llama-3.1-70B	DeepSeek-R1-32B	GPT-OSS-20B
Logistic Regression	$\mathbf{0.81}$	$\mathbf{0.89}$	0.77	0.62
Multi-Layer Logistic Regression	0.72	0.87	$\mathbf{0.78}$	0.60
Difference of Means	0.68	0.86	0.67	$\mathbf{0.68}$
Multi-Headed Attention	0.60	0.73	0.60	0.64
Multi-Layer Perceptron	0.56	0.77	0.68	0.57
Linear Artificial Tomography	0.52	0.57	0.54	0.63

Appendix

Table 6: OOD-mean AUROC per probe architecture per model (averaged across the six Transfer families and Stego ). Bold = highest per model.

	Transfer			Stego
Method	$\lvert G_{\mathrm{col}}\rvert{=}2$	$\lvert G_{\mathrm{col}}\rvert{=}3$	$\lvert G_{\mathrm{col}}\rvert{=}4$	$\lvert G_{\mathrm{col}}\rvert{=}2$	$\lvert G_{\mathrm{col}}\rvert{=}3$	$\lvert G_{\mathrm{col}}\rvert{=}4$
Sorted Concat	0.635	0.711	0.737	0.998	1.000	0.999
Activation Alignment	0.446	0.443	0.418	1.000	1.000	1.000
Peak Suspicion	0.506	0.627	0.623	1.000	1.000	1.000
Deception Split	0.575	0.682	0.560	1.000	1.000	1.000
Asymmetry Probe	0.568	0.688	0.589	1.000	1.000	1.000

Appendix

Table 7: Mean AUROC for each colluding group size

\lvert G_{\mathrm{col}}\rvert

, on the Transfer and Stego evaluation sets.

Method	Eval-Aware OOD Mean AUROC	Non-Eval-Aware OOD Mean AUROC
Sorted Concat	0.779	0.846
Activation Alignment	0.660	0.552
Peak Suspicion	0.833	0.863
Deception Split	0.819	0.893
Asymmetry Probe	0.834	0.886

Appendix

Table 8: Mean AUROC on out-of-distribution (OOD) evaluation sets, under eval-aware and non-eval-aware generation.

Group	$n$	Mean $\rho$
Colluder, Big Player (seat 1)	20	$\mathbf{+0.416}$
Colluder, Counter (seat 0)	20	$-0.025$
Honest, collusion runs	40	$-0.020$
Honest, control runs	80	$-0.037$

Appendix

Table 10: Mean lag-1 bet/count correlation by group across all 40 shoes.

Panel of Figure 7 — Figure 7: Token-level analysis in NARCBench-Stego . (a) Real result: the Big Player (colluder) projects higher onto the deception direction than honest agents reading the same Counter message ( $+0.29$ vs $-1.41$ across $n=39$ matched examples). (b) Schematic, not measured: illustrative per-token heatmap showing how the effect in (a) would look distributed across the tokens of the Counter’s message. Colours match the paper’s convention: red for colluder roles, blue for honest.

Method	Exploit-Share
Sorted Concat	$0.649\pm 0.010$
Activation Alignment	$0.709\pm 0.001$
Peak Suspicion	$0.554\pm 0.006$
Deception Split	$0.737\pm 0.012$
Asymmetry Probe	$0.565\pm 0.012$

Appendix

Table 12: Zero-shot AUROC of the five probe methods, trained on Core (qwen3_32b) cross-model on Exploit-Share (

\to

gpt-oss-120b)

Figure 8: Per-method AUROC on Llama-3.1-70B across Held-Out , Transfer , and Stego . Means over probe-layer range and ten seeds; error bars show layer standard deviation.

Model	Transfer	Stego
Llama-3.1 70B	0.87	1.00
Qwen3 32B	0.78	1.00
DeepSeek-R1 32B	0.75	0.95
GPT-OSS 20B	0.68	0.86

Appendix

Table 13: Mean AUROC per model, averaged across the five probing techniques, seeds, layers (and Transfer families).

Explore similar work

Sep 2, 2026cs.MA

You Can't Escape Your Own Activations : Evaluation Awareness and Multi-Agent Monitoring

LLM agents are increasingly deployed in multi-agent systems, where they can collude while keeping their actions benign. Output monitors designed to detect such collusions can be fooled by obfuscation and steganography, motivating the use of probes trained on internal activations. However, these probes are usually evaluated on agents that do not know they are being watched. We study how activation-based detection changes when agents are explicitly informed that their internal activations are being monitored, and when they additionally receive feedback from the monitor. We keep the models, probes, and thresholds fixed and change only what the agents are told: nothing (baseline), that an activation monitor is present (aware), or that a monitor is present together with the previous round's score (feedback). We test two games, a four-agent blackjack game and a two-agent Simmons prisoners game, using Qwen3-32B-AWQ and GPT-OSS-20B in homogeneous and mixed pairings. Telling agents about the monitor does not hide them. The best probes stay accurate in all three conditions, and the agents keep colluding.

Aritra Das, Jaee Ponde, Mihir More +1

Truth Audit Labs

Large Language Model Agents Obfuscation

Ask PDF

May 9, 2026cs.CL

AgentCollabBench: Diagnosing When Good Agents Make Bad Collaborators

Multi-agent systems achieve state-of-the-art outcomes through peer collaboration. However, when an agent in the pipeline silently drops a constraint, the system's final output may look correct even though the reasoning chain was quietly corrupted, and existing outcome-based evaluations are blind to such multi-hop process failures. To make these vulnerabilities measurable before deployment, we introduce AgentCollabBench, a diagnostic benchmark of 900 human-validated tasks spanning software engineering, DevOps, and data engineering. Each task isolates one of four behavioral risks: instruction decay (does a constraint survive peer pressure?), false-belief contagion (does a falsehood spread through consensus?), context leakage (does information bleed between tasks?), and tracer durability (does marked data reach the final agent?). Evaluating four modern LLMs (GPT 4.1 mini, Gemini 2.5 Flash Lite, Qwen-3.5-35B-A3B, and Llama 3.1 8B Instruct), we expose model-specific vulnerability profiles invisible to outcome-only evaluation; Qwen-3.5-35B-A3B, for example, leads on tracer durability and instruction stability, while GPT 4.1 mini leads on leakage containment and false-belief resistance. Beyond per-model differences, communication topology emerges as a primary risk factor that explains 7-40% of the variance in multi-hop information survival. The effect traces to a synthesis bottleneck specific to converging-DAG nodes: an agent weighing competing parent inputs discards constraints carried by a minority branch, a bottleneck structurally absent from linear chains. AgentCollabBench demonstrates that suboptimal topology can silently erase the safeguards of highly capable models, arguing that multi-agent reliability is fundamentally a structural problem and that scaling model intelligence alone is no substitute for architecture.

Aritra Mazumder, Shubhashis Roy Dipta, Nusrat Jahan Lia +10

University of Utah · University of Maryland, Baltimore County · University of Dhaka +7

Multi-Agent Communication Data Leakage Synthesis

Ask PDF

May 26, 2026cs.AI

Voluntary Collusion with Secret Tools in Competing LLM Agents

Even when a tool is explicitly described as unfair and harmful to others, ostensibly safety-aligned LLM agents still voluntarily engage in secret collusion whenever doing so confers a strategic advantage. To investigate this phenomenon, we introduce an empirical framework built on two strategic multi-agent environments: Liar's Bar, a competitive deception scenario, and Cleanup, a mixed-motive resource-management scenario, in which agents are offered secret collusion tools that provide significant advantages while clearly disadvantaging the other agents. Across 12 models (at the 7B, 70B, and proprietary scales) and 6 prompt variants, we find that most agents consistently accept these tools and develop collusive strategies, while explicitly acknowledging the unfairness of the tools before accepting. We further show that neither the unfairness labels nor baseline alignment alone reliably deters collusion: only explicit ethical framing reduces adoption and, even then, smaller models remain susceptible. More broadly, our work presents the first systematic investigation of voluntary collusion adoption in LLM-based multi-agent systems, and suggests that preventing such behaviour requires explicit safeguards rather than reliance on general alignment.

Xijie Zeng, Frank Rudzicz

1Dalhousie University · 2Vector Institute for Artificial Intelligence

Collusion Large Language Model Agents Artificial Intelligence Alignment Deception

Ask PDF