GroupGuard: A Framework for Modeling and Defending Collusive Attacks in Multi-Agent Systems

Yiling Tao; Xinran Zheng; Shuo Yang; Meiling Tao; Xingjun Wang

cs.AIMar 14, 2026

GroupGuard: A Framework for Modeling and Defending Collusive Attacks in Multi-Agent Systems

Authors: Yiling Tao, Xinran Zheng, Shuo Yang, Meiling Tao, Xingjun Wang

Organizations: Shenzhen International Graduate School, Tsinghua University · Department of Electrical and Electronic Engineering, The University of Hong Kong · School of Computer Science and Engineering, University of Electronic Science and Technology of China

Abstract

While large language model-based agents demonstrate great potential in collaborative tasks, their interactivity also introduces security vulnerabilities. In this paper, we propose and model group collusive attacks, a highly destructive threat in which multiple agents coordinate via sociological strategies to mislead the system. To address this challenge, we introduce GroupGuard, a training-free defense framework that employs a multi-layered defense strategy, including continuous graph-based monitoring, active honeypot inducement, and structural pruning, to identify and isolate collusive agents. Experimental results across five datasets and four topologies demonstrate that group collusive attacks increase the attack success rate by up to 15% compared to individual attacks. GroupGuard consistently achieves high detection accuracy (up to 88%) and effectively restores collaborative performance, providing a robust solution for securing multi-agent systems.

Figures & tables

Figure 1: Attack scenarios in a coding task regarding the removal of a necessary safety check (idx != idx2). (a) In the individual attack scenario, two malicious agents act independently, and their unrelated arguments are refuted by a benign agent based on facts. (b) In the group collusive attack scenario, malicious agents cooperate through a false consensus strategy and successfully mislead agent E.

Figure 2: Overview of the GroupGuard framework.

Attack Types	Model	Datasets
Attack Types	Model	MMLU	GSM8K	MultiArith	HumanEval	SVAMP	Avg.
Individual Attack
Misinformation Injection Yu et al. (2024)	Deepseek-V3.1	49.00%	28.00%	29.00%	21.00%	31.00%	31.60%
	Qwen3-32B	51.00%	19.00%	35.00%	38.00%	19.00%	32.40%
	Qwen3-235B-A22B	50.00%	53.00%	51.00%	34.00%	49.00%	47.40%
Dark Trait Injection Zhang et al. (2024b)	Deepseek-V3.1	33.00%	30.00%	26.00%	18.00%	33.00%	28.00%

Table 1: Comparison of ASR between group collusive and individual attacks across five datasets.

Attack Types	Model	Defense	Datasets
Attack Types	Model	Defense	MMLU	GSM8K	MultiArith	HumanEval	SVAMP	Avg.
False Consensus	Deepseek-V3.1	AutoDefense Zeng et al. (2024)	69.80%	70.80%	67.80%	53.60%	61.80%	64.76%
		G-safeguard Wang et al. (2025a)	83.60%	83.20%	80.60%	86.20%	78.60%	82.44%
		GroupGuard(Ours)	83.00%	84.00%	82.00%	87.00%	80.00%	83.20%
	Qwen3-32B	AutoDefense Zeng et al. (2024)	64.80%	67.80%	60.60%	72.60%	64.60%	66.08%
		G-safeguard Wang et al. (2025a)	80.20%	85.80%	81.40%	80.80%	80.00%	81.64%

Table 2: Detection accuracy comparison of GroupGuard, G-safeguard, and AutoDefense against three types of group collusive attacks across five datasets.

Figure 3: Comparison of performance recovery for three models after group collusion attacks and pruning-based defenses.

Figure 4: Average ASR of three group collusion attacks under four network topologies, and the average detection accuracy of GroupGuard.

Figure 5: Ablation study of defense components across three attack scenarios.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Method	False Consensus	Targeted Suppression	Group Exclusion
AutoDefense	4636	4568	4761
GroupGuard	2826	2826	2826

Appendix

Table 3: Comparison of average token usage per question between AutoDefense and GroupGuard across three attack types.

Attack Types	Model	Datasets
Attack Types	Model	MMLU	GSM8K	MultiArith	HumanEval	SVAMP	Avg.
False Consensus	Fully-Connected	49.00%	59.00%	53.00%	54.00%	53.00%	53.60%
	Layered	50.00%	40.00%	43.00%	43.00%	49.00%	45.00%
	Star	49.00%	37.00%	45.00%	45.00%	45.00%	44.20%
	Mesh	51.00%	37.00%	43.00%	43.00%	52.00%	45.20%
Targeted Suppression	Fully-Connected	62.00%	57.00%	56.00%	62.00%	49.00%	57.20%

Appendix

Table 4: Attack Success Rate (ASR) of three group collusive attacks across four distinct network topologies and five benchmark datasets.

Attack Types	Model	Datasets
Attack Types	Model	MMLU	GSM8K	MultiArith	HumanEval	SVAMP	Avg.
False Consensus	Fully-Connected	83.00%	84.00%	82.00%	87.00%	80.00%	83.20%
	Layered	71.00%	87.00%	82.00%	89.00%	83.00%	82.40%
	Star	85.00%	92.00%	95.00%	90.00%	93.00%	91.00%
	Mesh	76.00%	89.00%	86.00%	91.00%	86.00%	85.60%
Targeted Suppression	Fully-Connected	89.00%	91.00%	92.00%	70.00%	86.00%	85.60%

Appendix

Table 5: Detection accuracy of the GroupGuard framework across four distinct communication topologies.

Hyperparameter	Description	Tested Values (Value)	Detection Accuracy
$\alpha$	Weight for False Consensus detection	0.4 / 0.5 / 0.6	83.5% / 83.8% / 83.7%
$\beta$	Weight for Group Exclusion detection	0.4 / 0.5 / 0.6	85.3% / 88.0% / 84.9%
$k$	SD multiplier for abnormal intensity	2.0 / 3.0 / 4.0	85.4% / 84.3% / 84.7%
$\delta_{\tau}$	Trust decay threshold (victims)	-0.1 / -0.2 / -0.3	84.9% / 85.4% / 84.4%
$\delta_{H}$	Entropy threshold (concentration)	0.5 / 0.6 / 0.7	85.7% / 85.4% / 83.4%
$\epsilon$	Exploration probability (Honeypot)	0.1	-

Appendix

Table 6: Hyperparameter Configurations and Sensitivity Analysis

Nodes ( $N$ )	Attack Type	Acc. (w/o Defense)	Acc. (w/ GroupGuard)
10	False Consensus	57.50%	84.20%
	Targeted Suppression	59.70%	86.70%
	Group Exclusion	58.90%	89.20%
20	False Consensus	52.70%	83.20%
	Targeted Suppression	54.50%	85.40%
	Group Exclusion	57.70%	88.00%

Appendix

Table 7: Defensive Performance Across Different Agent Scales and Attack Types

Attack Type	MMLU	GSM8K	MultiArith	HumanEval	SVAMP
False Consensus	78.3 / 88.8 / 83.2	80.2 / 89.7 / 84.7	79.1 / 86.2 / 82.5	83.2 / 91.1 / 87.0	79.7 / 89.5 / 84.3
Targeted Suppression	84.7 / 90.4 / 87.5	85.3 / 92.9 / 88.9	86.7 / 90.9 / 88.8	71.2 / 74.5 / 72.8	81.3 / 88.7 / 84.8
Group Exclusion	87.0 / 95.6 / 91.1	89.7 / 92.8 / 91.2	78.0 / 85.3 / 81.5	90.1 / 97.8 / 93.8	84.6 / 90.5 / 87.5

Appendix

Table 8: Precision / Recall / F1 Score Across Benchmarks (%)

Graph-Based Anomaly Detection

See all 34 papers

Defending LLM-based Multi-Agent Systems Against Cooperative Attacks with Sentence-Level Rectification
May 27, 2026Yaoyang Luo, Zhi Zheng, Ziwei Zhao +5Multi-Agent LLM Systems Fake News Detection
University of Science and Technology of China · North Automatic Control Technology Institute · Shenzhen Institute for Advanced Study, UESTC
Colosseum: Auditing Collusion in Cooperative Multi-Agent Systems
Feb 16, 2026Mason Nakamura, Abhinav Kumar, Saswat Das +5Multi-Agent LLM Systems Multi-Agent Systems
University of Massachusetts Amherst · University of Virginia · ELLIS Institute Tübingen +1
Breaking the Secret: Economic Interventions for Combating Collusion in Embodied Multi-Agent Systems
Apr 26, 2026Qi Liu, Xiaohui Chen, Zhihui Zhao +5Mechanism Design Multi-Agent Systems
College of Computer Science and Technology, Taiyuan University of Technology, Taiyuan, China, 030024 · State Key Laboratory of Internet Architecture, Tsinghua University, Beijing, China, 100084 · China Mobile Research Institute, Beijing, China, 100053 +3

May 27, 2026J/K move · Enter open · S save

Defending LLM-based Multi-Agent Systems Against Cooperative Attacks with Sentence-Level Rectification

Yaoyang Luo, Zhi Zheng, Ziwei Zhao, Tong Xu+4

University of Science and Technology of China · North Automatic Control Technology Institute · Shenzhen Institute for Advanced Study, UESTC

Multi-Agent LLM Systems Fake News Detection Adversarial Robustness Multi-Agent System Security Multi-Agent Communication

Open paper AskPDF arXiv

Recent years have witnessed the rapid development of Large Language Model-based Multi-Agent Systems (MAS), which excel at collaborative decision-making and complex problem-solving. However, malicious agents in MAS may inject misinformation to mislead other agents and disrupt system performance, giving rise to a new research direction that focuses on attack mechanisms and defense strategies in MAS. Prior studies largely assume malicious agents act independently and investigate the corresponding defense strategies. However, we argue that malicious agents may exhibit collaborative behaviors, enabling more effective attacks through internal information exchange. In this paper, we propose an adaptive cooperative attack framework, where malicious agents autonomously coordinate and dynamically adjust their attack strategies through multi-round interactions. Furthermore, we introduce Sentence-Level Trustworthiness Analysis and Rectification (STAR), a defense framework that identifies and rectifies misleading information at the sentence level within agent communications. Our experiments show that cooperative attacks lead to a significantly larger degradation in task success rate than independent attacks, resulting in a relative drop of 5.34%. Meanwhile, STAR effectively mitigates both cooperative and independent threats and improves task success rate by an average of 36.76%. The code is available at https://github.com/smoooom/STAR.

Abstract

Graph-Based Anomaly Detection

Explore similar work

Defending LLM-based Multi-Agent Systems Against Cooperative Attacks with Sentence-Level Rectification

Colosseum: Auditing Collusion in Cooperative Multi-Agent Systems

Breaking the Secret: Economic Interventions for Combating Collusion in Embodied Multi-Agent Systems