Multi-agent discussions have been widely adopted, motivating growing efforts to develop attacks that expose their vulnerabilities. In this work, we study a practical yet largely unexplored attack scenario, the discussion-monitored scenario, where anomaly detectors continuously monitor inter-agent communications and block detected adversarial messages. Although existing attacks are effective without discussion monitoring, we show that they exhibit detectable patterns and largely fail under such monitoring constraints. But does this imply that monitoring alone is sufficient to secure multi-agent discussions? To answer this question, we develop a novel attack method explicitly tailored to the discussion-monitored scenario. Extensive experiments demonstrate that effective attacks remain possible even under continuous monitoring, indicating that monitoring alone does not eliminate adversarial risks.
Figures & tables
Figure 1: Illustration of multi-agent discussion attacks under the discussion-monitored scenario . In this scenario, anomaly detectors continuously monitor inter-agent communications and block detected adversarial messages during the discussion, while the attacker seeks to effectively prevent the multi-agent discussion process from reaching the correct answer under such discussion monitoring constraints.
Adaptation
Attacks
SelfCheckGPT [ 27 ] as anomaly detector
RAGAs [ 9 ] as anomaly detector
G-Safeguard [ 33 ] as anomaly detector
ΔAcc↑
ΔAgradv↑
Detection Rate ↓
ΔAcc↑
ΔAgradv↑
Detection Rate ↓
ΔAcc↑
ΔAgradv↑
Detection Rate ↓
Without stealth- oriented modifications
MCA [ 3 ]
0.7%
1.1%
95.1%
0.3%
0.7%
97.0%
0.4%
1.0%
96.2%
MAD-Spear [ 6 ]
1.5%
2.3%
97.8%
0.7%
1.3%
98.1%
0.9%
1.9%
96.9%
With stealth- oriented modifications
MCA [ 3 ]
2.6%
3.1%
4.5%
2.0%
2.9%
4.8%
2.2%
3.5%
5.6%
MAD-Spear [ 6 ]
6.9%
7.2%
6.7%
5.3%
5.8%
7.1%
6.1%
6.6%
8.4%
Table 1: Evaluation of existing multi-agent discussion attacks under the discussion-monitored scenario using different anomaly detectors for monitoring, on MMLU.
Adaptation
Attacks
SelfCheckGPT [ 27 ] as anomaly detector
RAGAs [ 9 ] as anomaly detector
G-Safeguard [ 33 ] as anomaly detector
ΔAcc↑
ΔAgradv↑
Detection Rate ↓
ΔAcc↑
ΔAgradv↑
Detection Rate ↓
ΔAcc↑
ΔAgradv↑
Detection Rate ↓
Without stealth- oriented modifications
MCA [ 3 ]
1.4%
1.9%
93.5%
0.7%
1.4%
95.0%
0.8%
1.8%
94.4%
MAD-Spear [ 6 ]
2.0%
2.2%
94.6%
1.5%
1.2%
96.8%
1.8%
2.5%
95.7%
With stealth- oriented modifications
MCA [ 3 ]
4.1%
5.0%
3.5%
2.3%
3.6%
4.0%
3.1%
3.8%
5.1%
MAD-Spear [ 6 ]
7.6%
8.5%
5.5%
5.7%
7.0%
6.2%
7.1%
7.2%
7.1%
Table 2: Evaluation of existing multi-agent discussion attacks under the discussion-monitored scenario using different anomaly detectors for monitoring, on MMMU.
Attacks
MMLU [ 14 ]
MMMU [ 39 ]
ΔAcc↑
ΔAgradv↑
ΔAcc↑
ΔAgradv↑
MCA [ 3 ]
34.2%
40.0%
37.1%
41.3%
MAD-Spear [ 6 ]
50.6%
59.4%
52.8%
61.0%
Table 3: Evaluation of existing multi-agent discussion attacks in the scenario without discussion-monitoring constraint , on MMLU and MMMU.
Figure 2: Illustration of the overall pipeline of our tailored attack method. Our method constructs an adversarial-aware opinion-dynamics formulation (Sec. 4.2 ) and selects adversarial agents and their target agents by solving this formulation (Sec. 4.3 ). For clarity, we visualize one adversarial agent with two target agents in the figure, while the same pipeline can extend to varying numbers of adversarial and target agents.
Attacks
Agent Selection Strategies
SelfCheckGPT [ 27 ] as anomaly detector
RAGAs [ 9 ] as anomaly detector
G-Safeguard [ 33 ] as anomaly detector
ΔAcc↑
ΔAgradv↑
Detection Rate ↓
ΔAcc↑
ΔAgradv↑
Detection Rate ↓
ΔAcc↑
ΔAgradv↑
Detection Rate ↓
MCA [ 3 ]
Original strategy of [ 3 ]
2.6%
3.1%
4.4%
2.0%
2.9%
4.8%
2.2%
3.5%
5.6%
Variant I
8.3%
15.6%
4.6%
6.1%
10.8%
4.9%
8.1%
12.0%
5.8%
Variant II
9.8%
13.1%
4.8%
7.9%
11.5%
5.1%
9.2%
13.1%
6.0%
Variant III
14.2%
18.2%
4.4%
11.6%
15.6%
4.9%
13.3%
17.0%
5.6%
Variant IV
11.3%
17.2%
4.7%
8.7%
12.5%
5.0%
10.6%
15.1%
5.9%
Table 4: Evaluation of different attack strategies for attacking multi-agent discussions under the discussion-monitored scenario on MMLU.
Attacks
Agent Selection Strategies
SelfCheckGPT [ 27 ] as anomaly detector
RAGAs [ 9 ] as anomaly detector
G-Safeguard [ 33 ] as anomaly detector
ΔAcc↑
ΔAgradv↑
Detection Rate ↓
ΔAcc↑
ΔAgradv↑
Detection Rate ↓
ΔAcc↑
ΔAgradv↑
Detection Rate ↓
MCA [ 3 ]
Original strategy of [ 3 ]
4.1%
5.0%
3.5%
2.3%
3.6%
4.0%
3.1%
3.8%
5.1%
Variant I
11.4%
14.8%
3.6%
9.2%
13.5%
4.1%
10.9%
14.5%
5.1%
Variant II
11.3%
15.2%
3.8%
9.8%
12.9%
4.4%
10.1%
14.9%
5.5%
Variant III
16.6%
22.9%
3.5%
12.7%
16.3%
4.0%
15.5%
19.8%
5.2%
Variant IV
13.5%
18.0%
3.7%
12.7%
17.8%
4.3%
13.0%
14.2%
5.3%
Table 5: Evaluation of different attack strategies for attacking multi-agent discussions under the discussion-monitored scenario on MMMU.
Figure 3: Qualitative results of our attack method on MMMU using RAGAs [ 9 ] as anomaly detector. As shown, our method successfully conducts multi-agent discussion attack under discussion monitoring constraints, showing its efficacy. Additional qualitative results, including qualitative comparisons with other methods, are in Supp.
Attacks
SelfCheckGPT [ 27 ] as anomaly detector
RAGAs [ 9 ] as anomaly detector
G-Safeguard [ 33 ] as anomaly detector
ΔAcc↑
ΔAgradv↑
Detection Rate ↓
ΔAcc↑
ΔAgradv↑
Detection Rate ↓
ΔAcc↑
ΔAgradv↑
Detection Rate ↓
w/o both modifications
25.7%
31.3%
5.5%
21.0%
28.2%
6.2%
22.2%
29.3%
7.0%
w/o modification (1)
40.2%
46.2%
5.4%
36.4%
41.4%
6.1%
37.3%
43.0%
7.0%
w/o modification (2)
35.1%
39.8%
5.5%
32.1%
36.6%
6.1%
33.2%
33.9%
7.1%
Our formulation
45.3%
51.1%
5.4%
41.7%
44.6%
6.1%
42.2%
49.4%
7.0%
Table 6: Evaluation on the adversarial-aware formulation.
Multi-agent system security studies attacks, trust, privacy, and defenses in interacting AI agents, including language-model teams, robot swarms, and learned controllers.
Language models can find thousands of severe software vulnerabilities, and agents are increasingly being misused for cyberattacks. To avoid detection, attackers frequently distribute their misuse, splitting a harmful task across many user accounts so each individual transcript looks benign. Because safety monitors score only one agent context at a time, they are structurally blind to misuse that is only visible in aggregate, across many accounts. We show this gap is real by building, to our knowledge, the first distributed agent attack, a multi-agent scaffold that completes hard cybersecurity tasks while hiding the harmful objective across subagents with limited contexts, evading a standard monitor that catches it only a fifth as often as prior agent attacks. Towards a defense, we develop an online stateful monitor that uses real-time clustering to collect weak suspiciousness signals across many agent transcripts, and escalates only rarely to a language model that flags misuse across user accounts. In evaluations with large-scale simulated datacenter traffic, our monitor Pareto dominates standard monitors, catching distributed attacks 30% earlier and flagging cyber misuse before it reaches the most harmful stages. Crucially, this comes at negligible additional latency for ~99% of user traffic. This detection advantage persists but narrows as the benign background traffic grows very large. After an extensive red-teaming exercise, we improve the defense and surprisingly also find that it catches standard jailbreaks, since adaptive attackers reuse attack variants across accounts. Our results point toward a new class of safety monitors which reason over groups of users rather than isolated transcripts.
Davis Brown, Samarth Bhargav, Arav Santhanam +7
University of Pennsylvania · 2Carnegie Mellon University
AI control is a family of techniques to prevent an AI with malicious goals from subverting its operator's intent. AI Control usually studies a single agent in one trajectory, but real deployments run many agents over shared infrastructure, and the most severe risks (model-weight exfiltration, training-run poisoning) plausibly need several agents acting in concert. We initiate the empirical study of multi-agent AI control, formalising distributed attacks in which several agents jointly aim for a malicious goal. We develop FakeLab: a synthetic AI-lab codebase (9 services, 86 benign tasks, 4 attack objectives). We evaluate single agent monitoring against distributed attacks, varying the number of agents, their coordination, model capabilities and precise monitoring configuration. Our central finding is the fragmentation effect: as more agents coordinate to attack, per-agent monitoring becomes less likely to catch any of the attackers. Secondly, this is not driven by the ratio of benign to malicious code -- we argue this is likely dependent on model capability. Third, an explicit planner amplifies the fragmentation effect, raises the rate of completing the attack up to sevenfold, and mildly sharpens executors' poor monitor-theory-of-mind. Finally: a weak "trusted" monitor misses most attacks, while a stronger one cuts undetected success by more than an order of magnitude but remains imperfect. We are happy to share FakeLab, our control setting, with safety researchers on request.
Language-model agents can communicate through continuous hidden states that are invisible in public transcripts, creating opportunities for covert harmful coordination. We introduce Verifiable Latent Alignments (VLA), an activation-aware framework for monitoring and steering these private communication channels. For every monitored decision, VLA links the private latent-state record and channel status to the resulting public action using a shared event identifier, enabling matched causal analysis. Our first contribution is a neutral-only three-layer monitor combining representation anomaly detection, counterfactual action-distribution influence, and sparse-autoencoder interpretation support. Our second contribution is a steerability framework spanning black-box behavioral instructions and white-box matched-neutral counterfactuals. Our third contribution is an evaluation on a controlled multi-agent auction benchmark covering homogeneous and heterogeneous model pairs, many-agent scalability, and intervention effectiveness. The sequential monitor achieves mean area under the receiver operating characteristic curve (AUROC) of 0.993 for homogeneous agents and 0.854 for heterogeneous pairs when text- and latent-collusion rows are pooled as positives. In Qwen3-0.6B auctions with 25-100 bidders, monitoring requires only a small normalized load relative to all possible directed pairs, while full white-box steering achieves 100% bid-distribution recovery and reduces collusive low-bid behavior by 47.3 percentage points. Because full white-box steering replays the matched neutral counterfactual, its exact recovery is a sanity check by construction. Overall, the controlled study shows that the evaluated private channel attacks can be monitored without training the primary monitor on attack examples and mitigated when matched counterfactual access is available.
Ramneet Kaur, Pradyumna Chari, Ramesh Raskar +3
SRI International · MIT Media Lab · Westtown School +1