CoopGuard: Stateful Cooperative Agents Safeguarding LLMs Against Evolving Multi-Round Attacks
Authors: Siyuan Li, Zehao Liu, Xi Lin, Qinghua Mao, Yuliang Chen, Haoyu Li, Jun Wu, Jianhua Li, Xiu Su
Organizations: School of Computer Science, Shanghai Jiao Tong University · Department of Computer Science, University of Illinois Urbana-Champaign · Big Data Institute, Central South University
As Large Language Models (LLMs) are increasingly deployed in complex applications, their vulnerability to adversarial attacks raises urgent safety concerns, especially those evolving over multi-round interactions. Existing defenses are largely reactive and struggle to adapt as adversaries refine strategies across rounds. In this work, we propose CoopGuard , a stateful multi-round LLM defense framework based on cooperative agents that maintains and updates an internal defense state to counter evolving attacks. It employs three specialized agents (Deferring Agent, Tempting Agent, and Forensic Agent) for complementary round-level strategies, coordinated by System Agent, which conditions decisions on the evolving defense state (interaction history) and orchestrates agents over time. To evaluate evolving threats, we introduce the EMRA benchmark with 5,200 adversarial samples across 8 attack types, simulating progressively LLM multi-round attacks. Experiments show that CoopGuard reduces attack success rate by 78.9% over state-of-the-art defenses, while improving deceptive rate by 186% and reducing attack efficiency by 167.9%, offering a more comprehensive assessment of multi-round defense. These results demonstrate that CoopGuard provides robust protection for LLMs in multi-round adversarial scenarios.
Figures & tables
Figure 1: Illustration of the challenge posed by independent yet progressively evolving multi-round adversarial attacks on LLMs and our innovative CoopGuard multi-agent adaptive defense mechanism to effectively counter these evolving threats.
Figure 2: Overview of CoopGuard multi-agent jailbreak defense framework. Deferring Agent slows down probing progress via ambiguity injection, while Tempting Agent generates deceptive traps to mislead them. Forensic Agent collects and analyzes evidence of attack behaviors. System Agent oversees the agents, dynamically refining defense strategies to adapt to evolving threats. This cooperative process safeguards the system, depletes the attacker’s resources, and collects intelligence on attack behavior .
Figure 3: Resource footprint of multi-round attacks. (a) Token consumption for defense and attack across question types. (b) Token consumption for defense and attack across jailbreak strategies.
Figure 4: Characteristics of jailbreak strategies. (a) Distribution of prompt lengths for each strategy type. (b) Attack-to-defense token ratio for each strategy type, indicating the attacker’s relative cost.
Method
MTA(Avg) ↓
HQ ↓
RQ ↓
JQ(Avg) ↓
GPT-5
PAT
0.070 ±0.004
0.013 ±0.008
0.187 ±0.005
0.027 ±0.003
RPO
0.078 ±0.003
0.003 ±0.005
0.207 ±0.005
0.023 ±0.005
Self-Reminder
0.048 ±0.020
0.011 ±0.005
0.114 ±0.042
0.018 ±0.042
GoalPriority
0.030 ±0.003
0.010 ±0.008
0.072 ±0.005
0.010 ±0.005
SecurityLingua
0.152 ±0.004
0.000 ±0.005
0.174 ±0.010
0.283 ±0.005
Table 1: Main ASR experiments of multi-turn attacks (MTA), harmful questions (HQ), rephrased questions (RQ), and jailbreak questions (JQ) on GPT-5 , Gemini-2.5-Pro , and DeepSeek-V3 .
Method
MTA(Avg) ↑
HQ ↑
RQ ↑
JQ(Avg) ↑
GPT-5
PAT
0.015 ±0.004
0.013 ±0.010
0.031 ±0.005
0.000 ±0.005
RPO
0.014 ±0.006
0.005 ±0.005
0.018 ±0.012
0.019 ±0.014
Self-Reminder
0.022 ±0.004
0.000 ±0.005
0.065 ±0.009
0.000 ±0.005
GoalPriority
0.003 ±0.002
0.000 ±0.000
0.009 ±0.005
0.000 ±0.005
SecurityLingua
0.036 ±0.004
0.007 ±0.007
0.100 ±0.010
0.000 ±0.000
Table 2: Main DR experiments of multi-turn attacks (Avg), harmful questions (HQ), rephrased questions (RQ), and jailbreak questions (JQ) on GPT-5 , Gemini-2.5-Pro , and DeepSeek-V3 .
Figure 5: Evaluation of AE , quantified by the average token consumption per dialogue across different models. CoopGuard forces the attacker to expend significantly more resources (higher is better for defense) compared to baselines.
Model
PAIR
DRA
M2S
DeepInception
GPT-3.5-turbo
0.05
0.03
0.03
0.01
GPT-4
0.11
0.10
0.03
0.09
Gemini-2-Flash
0.16
0.14
0.12
0.14
Table 3: Attack Success Rate (ASR) of CoopGuard against state-of-the-art single-turn jailbreak attacks across different LLM backbones. The results demonstrate the framework’s generalizability.
Method
Acc.
Clar.
Comp.
Ctxt.
Dep.
Pol.
Eng.
Avg.
GPT-5
8.94
8.92
8.36
9.00
7.02
9.56
7.22
8.43
Ours (GPT-5)
8.54
8.64
7.56
8.33
5.98
8.84
6.87
7.82
Gemini-2.5-Pro
9.06
9.10
8.26
9.12
6.28
9.58
6.92
8.33
Ours (Gemini-2.5-Pro)
8.66
8.82
7.46
8.45
5.84
8.86
6.57
7.80
DeepSeek-V3
9.50
9.00
9.60
9.50
9.30
10.0
8.30
9.31
Ours (DeepSeek-V3)
9.10
8.72
8.80
8.83
8.26
9.28
7.95
7.71
Table 4: Performance comparison across evaluated backbones on MT-BENCH-101. Metrics are abbreviated: Accurate (Acc.), Clarity (Clar.), Completeness (Comp.), Context (Ctxt.), Depth (Dep.), Politeness (Pol.), Engagement (Eng.).
Figure 6: Cross-validation results with Deepseek-Judge and Human-Judge. Both validation methods show consistent trends with our primary GPT-Judge evaluation.