Safety of Latent Communication in Multi-Agent Systems

Muhammad Huzaifa; Sina Mavali; Thorsten Eisenhofer

cs.AISep 30, 2026

Safety of Latent Communication in Multi-Agent Systems

Authors: Muhammad Huzaifa, Sina Mavali, Thorsten Eisenhofer

Organizations: CISPA Helmholtz Center for Information Security, Saarbrücken, Germany

Abstract

Latent communication enables multi-agent systems to exchange information directly in internal representation space, reducing the token, computation, and latency overhead of text-based communication. To this end, lightweight trainable links are introduced to map the sender's representations into the receiver's input space. In this work, we show that even benign link training can increase harmful compliance relative to text-based communication while the underlying safety-aligned agents remain unchanged. An attacker can amplify this effect by optimizing the links on harmful query--response pairs or poisoning otherwise benign training data. We further develop a reinforcement-learning attack that rewards harmful compliance alongside benign task performance without requiring harmful target responses. Across three communication topologies and four safety benchmarks, this attack raises the mean harmful-compliance score from 27.9 with benignly trained links to 76.9. Compared with direct supervised optimization, it also achieves higher average accuracy on two benign utility benchmarks. Adapting the rewards toward safer behavior also enables repair of compromised links, substantially reducing harmful compliance across all evaluated attacks without updating the agents. Overall, our results show that safety alignment requires considering the multi-agent system as a whole. Code: https://github.com/Muhammad-Huzaifaa/latent-safety

Figures & tables

Panel of Figure 1 — Figure 1: Latent communication topologies. Frozen agents communicate through trainable links $f_{\theta}$ , which map sender hidden states $h_{s}$ to soft tokens $z$ in the receiver’s embedding space. Each agent also receives the query $q$ (not shown for clarity).

		Safety (ASR %, $\downarrow$ better)						Utility (%, $\uparrow$ better)		Efficiency ( $\downarrow$ better)
Topology	Comm.	HB	SR	JBB	AB	Avg.	$\Delta$	MATH500	GPQA-D	Tokens	Time (s)
2-Agent	Text	4.0	8.6	4.0	1.0	$\mathbf{4.4}$		51.5	33.7	729	2.65
2-Agent	Latent	39.3	36.9	36.6	11.5	31.1	$\mathbf{+26.7}$	$\mathbf{65.3}$	$\mathbf{35.7}$	$\mathbf{511}$	$\mathbf{1.96}$
Sequential	Text	6.0	6.4	0.0	0.2	$\mathbf{3.2}$		58.3	26.6	1042	2.30
Sequential	Latent	38.8	35.0	26.7	26.1	31.7	$\mathbf{+28.5}$	$\mathbf{59.5}$	$\mathbf{30.2}$	$\mathbf{501}$	$\mathbf{1.84}$
Mixture	Text	19.4	18.8	9.9	1.7	$\mathbf{12.5}$		$\mathbf{80.6}$	34.7	1123	8.44

Table 1: Text-based vs. latent communication. Harmful compliance, benign utility, and inference cost for text-based and benignly trained latent communication across three topologies.

Figure 2: Left: first-token refusal probability. Right: forced refusal prefixes reduce harmful compliance.

		Harmful compliance (%, $\uparrow$ less safe)					Benign utility (%, $\uparrow$ better)
Topology	Condition	HB	SR	JBB	AB	Avg.	MATH500	GPQA-D
2-Agent	Clean	39.3	36.9	36.6	11.5	31.1	65.3	35.7
	Poisoning (10%)	59.2	54.5	48.5	34.7	49.2	63.9	31.7
	Supervised	80.6	78.0	76.2	78.5	$\mathbf{78.3}$	18.4	30.2
Sequential	Clean	38.8	35.0	26.7	26.1	31.7	59.5	30.2
	Poisoning (10%)	54.7	60.5	51.5	56.8	55.9	44.1	32.2

Table 2: Supervised and data-poisoning attacks. Harmful compliance and benign utility after direct supervised optimization and data poisoning across three communication topologies.

Panel of Figure 3 — Figure 3: Attacking latent communication. Links are optimized using (a) harmful targets, directly or through data poisoning, or (b) response-level rewards. All agents remain frozen.

		Harmful compliance (%, $\uparrow$ less safe)					Benign utility (%, $\uparrow$ better)
Topology	Condition	HB	SR	JBB	AB	Avg.	MATH500	GPQA-D
2-Agent	Clean	39.3	36.9	36.6	11.5	31.1	$\mathbf{65.3}$	$\mathbf{35.7}$
2-Agent	RL attack	78.6	74.2	86.1	86.4	$\mathbf{81.3}$	63.7	25.1
Sequential	Clean	38.8	35.0	26.7	26.1	31.7	59.5	$\mathbf{30.2}$
Sequential	RL attack	61.7	65.6	50.5	36.7	$\mathbf{53.6}$	$\mathbf{68.7}$	28.1
Mixture	Clean	29.4	11.8	27.7	14.4	20.8	76.8	43.2

Table 3: Reward-guided attack on latent communication. Harmful compliance and benign utility after reward-guided link optimization across three communication topologies.

Figure 4: Attacking single links.

		Harmful compliance (ASR %, $\downarrow$ better)						Benign utility (%, $\uparrow$ better)
Topology	Setting	HB	SR	JBB	AB	Avg.	$\Delta$	MATH500	GPQA-D
	Clean (no attack)	39.3	36.9	36.6	11.5	31.1		65.3	35.7
	Poisoning	59.2	54.5	48.5	34.7	49.2		63.9	31.7
	$\hookrightarrow$ repair	6.5	12.4	4.0	0.4	$\mathbf{5.8}$	$\mathbf{-43.4}$	64.5	25.6
	Supervised	80.6	78.0	76.2	78.5	78.3		18.4	30.2
	$\hookrightarrow$ repair	1.0	3.5	1.0	0.2	$\mathbf{1.4}$	$\mathbf{-76.9}$	56.3	29.1

Table 4: Reward-guided repair. Harmful compliance and benign utility after repairing the communication links.

\Delta

denotes the change in mean harmful compliance relative to the attacked system.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Topology	Agent Configuration	Hidden Dimensions	Trainable Params.
2-Agent	Llama-3.2-3B-Instruct $\rightarrow$ Qwen2.5-3B-Instruct	$3072\rightarrow 2048$	27.3M
Sequential	Gemma-3-1B-IT $\rightarrow$ Llama-3.2-1B-Instruct $\rightarrow$ Qwen3-1.7B	$1152\rightarrow 2048\rightarrow 2048$	36.5M
Mixture	Qwen2.5-Math-1.5B-Instruct + Qwen3-1.7B $\rightarrow$ Qwen3-8B	$(1536,2048)\rightarrow 4096$	111.2M

Appendix

Table 5: Multi-agent configurations used in our experiments. Only the latent communication links are trainable; all LLM parameters remain frozen.

Dataset	Role	Examples	Supervision
Sequential-Math ( Zou et al., 2026a )	Clean link training	1,904	Benign task supervision
PKU-SafeRLHF ( Ji et al., 2025 )	Supervised attack	3,000	Harmful query–response pairs
PKU-SafeRLHF ( Ji et al., 2025 )	Data poisoning	212	Harmful query–response pairs
PKU-SafeRLHF ( Ji et al., 2025 )	Target-free RL attack	2,697	Harmful queries only
Sequential-Math ( Zou et al., 2026a )	RL utility preservation	1,365	Benign question–answer pairs
HarmBench ( Mazeika et al., 2024 )	Safety evaluation	200	Queries only

Appendix

Table 6: Datasets used for link training, attacks, and evaluation. Supervised and poisoning attacks use harmful query–response pairs, whereas the target-free RL attack uses harmful queries only.

		Harmful compliance (%, $\uparrow$ less safe)				Mean
Topology	Optimization	HB	SR	JBB	AB
2-Agent	Supervised	$\mathbf{82.1}$	79.3	69.3	$\mathbf{80.2}$	77.7
2-Agent	Reward-guided RL	76.6	$\mathbf{85.4}$	$\mathbf{77.2}$	79.3	$\mathbf{79.6}$
Sequential	Supervised	$\mathbf{66.7}$	$\mathbf{70.4}$	$\mathbf{63.4}$	$\mathbf{73.1}$	$\mathbf{68.4}$
Sequential	Reward-guided RL	64.2	58.9	57.4	57.8	59.6
Mixture	Supervised	78.6	82.8	81.2	77.9	80.1

Appendix

Table 7: Supervised vs. reward-guided RL optimization. Starting from the same untrained latent-link initialization, both objectives successfully optimize the communication links toward harmful compliance. Reward-guided RL achieves higher mean harmful compliance in two of the three topologies.

		Harmful compliance (%, $\downarrow$ better)					Utility (%, $\uparrow$ better)
Topology	Training	HB	SR	JBB	AB	Avg.	MATH500
2-Agent	Supervised	39.3	36.9	36.6	11.5	31.1	65.3
2-Agent	RL	$\mathbf{10.0}$	$\mathbf{15.9}$	$\mathbf{5.0}$	$\mathbf{0.6}$	$\mathbf{7.9}$	$\mathbf{66.9}$
Sequential	Supervised	38.8	35.0	26.7	26.1	31.7	59.5
Sequential	RL	$\mathbf{27.9}$	$\mathbf{31.8}$	$\mathbf{14.9}$	$\mathbf{4.0}$	$\mathbf{19.6}$	$\mathbf{68.7}$
Mixture	Supervised	29.4	11.8	27.7	14.4	20.8	76.8

Appendix

Table 8: Supervised vs. RL link training on benign data. Both methods start from the same untrained latent-link initialization and use only benign mathematics data. RL consistently yields lower harmful compliance while matching or improving benign utility.

	Sequential						Mixture
Poison rate	HB	SR	JBB	AB	Mean	$\Delta$	HB	SR	JBB	AB	Mean	$\Delta$
0% (clean)	39.0	34.8	27.0	26.2	31.7	—	29.4	11.8	27.7	14.4	20.8	—
10%	55.0	60.4	52.0	56.9	56.1	$+24.4$	72.0	72.2	68.0	67.3	69.9	$+49.1$
20%	64.0	69.0	71.0	72.5	$\mathbf{69.1}$	$\mathbf{+37.4}$	78.0	79.2	82.0	74.4	$\mathbf{78.4}$	$\mathbf{+57.6}$
30%	58.0	64.9	63.0	57.5	60.8	$+29.1$	79.0	74.1	80.0	72.5	76.4	$+55.6$
40%	53.0	63.9	57.0	54.0	57.0	$+25.3$	79.0	70.3	78.0	68.1	73.8	$+53.0$

Appendix

Table 9: Effect of poisoning rate on harmful compliance. Attack success increases sharply with limited poisoning and peaks around

20\%

in both topologies, after which additional contamination provides diminishing gains.

		Harmful compliance (%, $\uparrow$ less safe)				Mean
Topology	Attacked link	HB	SR	JBB	AB
Sequential	Clean (no attack)	38.8	35.0	26.7	26.1	31.7
	Planner $\rightarrow$ Refiner	61.7	64.6	57.4	65.1	62.2
	Refiner $\rightarrow$ Solver	$\mathbf{73.1}$	$\mathbf{72.9}$	$\mathbf{65.3}$	68.5	$\mathbf{70.0}$
	All links	68.2	69.7	61.4	68.5	67.0
Mixture	Clean (no attack)	29.4	11.8	27.7	14.4	20.8

Appendix

Table 10: Attack surface across individual latent links. Attacking a single communication link is sufficient to induce substantial harmful compliance, and joint optimization of all links is not consistently stronger than targeting the most vulnerable link.

		Starting link		After RL
Topology	RL objective	ASR	MATH500	ASR	MATH500	$\Delta$ MATH
Degraded initialization
2-Agent	harm $+$ utility	78.3	18.4	95.6	43.4	$\mathbf{+25.0}$
Sequential	harm $+$ utility	67.0	25.7	89.6	64.2	$\mathbf{+38.5}$
Mixture	harm $+$ utility	81.6	47.9	92.1	65.6	$\mathbf{+17.7}$
Clean initialization

Appendix

Table 11: Effect of the utility reward. Utility-guided RL strongly recovers degraded links, while its benefit from a clean initialization is topology-dependent.

\Delta

MATH is relative to the starting link.

Diagnostic	Condition / Intervention	Result
Forced refusal prefix	Receiver only	ASR: $9.0\rightarrow 1.5$
	Untrained link	ASR: $8.5\rightarrow 1.0$
	Benign-trained link	ASR: $\mathbf{31.1\rightarrow 0.0}$
Prefix-length intervention	Force I	$69\%$ of link-induced effect recovered
	Force No	$88\%$ recovered
	Force Sorry	$95\%$ recovered

Appendix

Table 12: Mechanistic diagnostics of benign supervised link training. The trained link primarily shifts the receiver away from refusal at the start of generation, while the underlying refusal behavior remains recoverable.

Panel of Figure 5 — Figure 5: Qualitative examples in the simple two-agent communication setting. We compare receiver responses under the clean latent link, three link attacks, and the reward-guided defense. The examples illustrate that benign link training can already weaken refusal behavior, while adversarial link optimization further increases harmful compliance. The defense restores refusal behavior.

Panel of Figure 6 — Figure 6: Qualitative examples in the simple two-agent communication setting.

Panel of Figure 7 — Figure 7: Qualitative examples in the latent sequential communication setting.

Panel of Figure 8 — Figure 8: Qualitative examples in the latent sequential communication setting.

Panel of Figure 9 — Figure 9: Qualitative examples in the mixture style communication setting.

Panel of Figure 10 — Figure 10: Qualitative examples in the mixture style communication setting.

Explore similar work

Finding the Weakest Link: Adversarial Attack against Multi-Agent Communications
May 13, 2026Maxwell Standen, Junae Kim, Claudia SzaboMulti-Agent Reinforcement Learning Multi-Agent Communication
The University of Adelaide · DST Group
Out of Sight, Not Out of Mind: Unveiling Latent Attack in Latent-based Multi-Agent Systems
May 27, 2026Chenxi Wang, Ruiyang Huang, Jiayan Sun +2White-Box Spectral-Subspace-Guided Attack Model-Based Multi-Agent Systems
Southeast University, Nanjing, China · Peking University, Beijing, China
Beyond tokens: a unified framework for latent communication in LLM-based multi-agent systems
Jun 4, 2026Yingzhuo LiuMulti-Agent Large Language Model Systems Unified Framework

May 13, 2026, cs.LGJ/K move · Enter open · S save

Finding the Weakest Link: Adversarial Attack against Multi-Agent Communications

Maxwell Standen, Junae Kim, Claudia Szabo

The University of Adelaide · DST Group

Multi-Agent Reinforcement Learning Multi-Agent Communication White-Box Spectral-Subspace-Guided Attack Adversarial Training

Open paper AskPDF arXiv

Multi-agent systems rely on communication for information sharing and action coordination, which exposes a vulnerability to attacks. We investigate single-victim communication perturbation attacks against Multi-Agent Reinforcement Learning-trained systems and propose methods that use gradient information from the Jacobian to identify which messages, agent, and timesteps are most susceptible to attack and have the greatest impact on the system. We enhance these methods with two proposed adversarial loss functions that trade-off attack success for attack impact which also create more effective perturbations. We empirically demonstrate the effectiveness of our methods against two different multi-agent communication methods in navigation, PredatorPrey, and TrafficJunction environments. Our results show that our novel message selection method achieves a similar or greater impact than random message selection across almost all tested scenarios. Our victim selection, message selection, tempo, and loss functions improve attack effectiveness in half of the thirty scenarios we tested.

Abstract

Explore similar work

Finding the Weakest Link: Adversarial Attack against Multi-Agent Communications

Out of Sight, Not Out of Mind: Unveiling Latent Attack in Latent-based Multi-Agent Systems

Beyond tokens: a unified framework for latent communication in LLM-based multi-agent systems