Knowledge attacks on Large Language Models, such as manipulating fact-checking judgments, pose significant risks to AI reliability. Existing approaches -- adversarial suffix/template injection and single-agent self-reflection -- lack robustness across diverse defenses and are prone to confirmation bias and costly trial-and-error, respectively. We propose Debate-to-Deceive (D2D), a multi-agent debate framework grounded in human debate structure and rhetorical style, enabling agents to generate adversarial passages that are semantically coherent, rhetorically effective, and resistant to early confirmation bias. D2D formalizes the adversarial process as an evolutionary game in which agents adaptively evolve their rhetorical strategies (Logos, Pathos, Ethos) through competition. To manage the intractably large LLM output space, we decouple agent behavior into a strategic layer, which selects a rhetorical action from a compact strategy space via evolutionary-inspired learning dynamics, and a generation layer, where the LLM produces a concrete passage conditioned on that strategy and the debate context. On four fact-checking benchmarks (FEVER, HealthVer, CLIMATE-FEVER, and SciFact), D2D substantially outperforms existing attacks and evades most defenses.
Figures & tables
Figure 1: D2D framework: Schematic diagram of a debating adversarial attack framework targeting an LLM for fact-checking. An attacker orchestrates two AI agents to engage in a structured human debate, iteratively crafting adversarial queries that are imperceptible to humans but effective in misleading the target LLM. This process induces incorrect responses. For example, given the claim “Vitamin D can treat COVID-19,” D2D generates adversarial passages that steer the target LLM toward a malicious fact-checking outcome.
Strategy
Logos
Pathos
Ethos
Logos
0
- α
γ
Pathos
α
0
- β
Ethos
- γ
β
0
Table 1: Payoff matrix of a three-strategy rhetorical game
Dataset
No Defense
Known-answer
FEVER
98.5 [95.7, 99.5]
96.5 [93.0, 98.3]
HealthVer
97.0 [93.6, 98.6]
97.5 [94.3, 98.9]
CLIMATE-FEVER
99.5 [97.2, 99.9]
99.5 [97.2, 99.9]
SciFact
100.0 [98.1, 100.0]
99.5 [97.2, 99.9]
Avg.
98.8
98.3
Table 2: ASR (%) with Wilson 95% intervals; Qwen3-32B as agent, judge and target. 200 attacks per cell (100 claims per label, both directions), 1,600 in total. A trial succeeds only when the passage argues the target label and AG returns it. Known-answer blocked none of the 800 defended attacks.
Figure 2: Strategy dynamics of D2D and i.i.d. controls on FEVER and HealthVer with Qwen3-32B self-play (800 debates per arm). (a) Per-claim strategy mixtures on the Logos–Pathos–Ethos simplex. Faint points overlay deployed argmax claims; large markers show dataset–defense means; arrows connect argmax to TD-softmax; the shaded region covers 80% of w∗ claims. (b) Turn-level D2D strategy shares; dashed lines mark w∗ . (c) Repeat-own and counter-opponent transition probabilities, with red lines giving each arm’s i.i.d. expectation. (d) ASR comparison of D2D, w∗ , and Uniform, showing that D2D’s adaptive strategy selection achieves the highest attack success.
Model
Defense
ADAN
OPI
JD
ADMIT
D2D
Llama3
None
100.0
63.7
13.0
82.5
100.0
Instr.
100.0
32.9
14.5
20.0
100.0
Known
0.0
31.6
18.5
55.3
94.8
LLM
100.0
0.0
29.0
0.0
100.0
Resp.
99.8
61.3
18.0
53.8
100.0
Para.
99.8
36.2
23.5
37.8
100.0
Table 3: ASR (%) of D2D and baselines across defenses. ADAN: AutoDAN; OPI: Open-Prompt-Injection; JD: JudgeDeceiver. Defenses: None, Instructional, Known-answer, LLM-based, Response-based, Paraphrasing. Runtimes for every cell are in Table 10 .
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
Symbol
Description
Agents, States and Actions
i,−i
Current agent; all other agents
L,L′
Correct label side; target side
s,s′
Current and next state
Si,S−i
Action of agent i ; actions of others
S={S1,S2,S3}
Strategy space {logos, pathos, ethos}
Appendix
Table 4: Summary of notation
Figure 3: D2D debate example: It illustrates the D2D algorithm’s process using the “Vitamin D can treat COVID-19” claim. The system attempts to flip the LLM’s stance from the original “Refute” position to the target “Support" position through a four-phase debate. Agent 1 (opponent, yellow) maintains the refute stance while Agent 2 (proponent, blue) argues for the support stance. Each argument is annotated with rhetorical strategies: Logos in blue, Pathos appeals in red, and Ethos in green. The attack succeeds when the target LLM validates the stance of support.
Dataset
Claims
Passages
Domain
FEVER
185,445
5,416,568
General
HealthVer
2,149
6,961
Health
Climate-FEVER
7,675
5,416,568
Climate
SciFact
1,409
5,183
Science
Appendix
Table 5: Statistics and characteristics of datasets
Baseline
Original Design
Adaptation in Our Setting
AutoDAN
Automatically generates stealthy jailbreak prompts using a hierarchical genetic algorithm.
Redirects the optimization target to the attacker-specified verdict label under the same fact-checking prompt template.
Open-Prompt-Injection
Provides a general framework for constructing prompt injection attacks.
Uses the target verdict label as the injected adversarial objective to induce label flipping.
ADMIT
Performs semantically aligned poisoning via iterative refinement and memory-based self-reflection.
Iteratively refines adversarial fact-checking arguments to induce the target verdict label.
Judge-Deceiver
Optimizes adversarial sequences so that LLM-as-a-Judge selects an attacker-controlled response.
Redirects the optimized sequence toward causing the target LLM to output the attacker-specified verdict label.
Appendix
Table 6: Summary of baseline methods and their adaptation to fact-checking attacks.
Defense
Type
Description
Instruction
Prevention
Adds explicit instructions to make the model resist malicious or manipulative prompts.
Paraphrasing
Prevention
Rewrites the input prompt to reduce the effectiveness of adversarial patterns.
Known-answer
Detection
Uses reference answers or target context to identify inconsistent or adversarial behavior.
LLM-based
Detection
Uses an LLM as a judge to detect unsafe or adversarial inputs or outputs.
Response-based
Detection
Generates a response first and then evaluates whether it is harmful or suspicious.
Appendix
Table 7: Summary of evaluated defense methods.
Removed
Phase
Defense
ASR
Time
Tokens
Foundation
None
85.0
1395.3
14640
Instructional
80.0
1138.5
12354
Known-answer
0.0
2177.7
25316
LLM-based
95.0
1062.7
11246
Paraphrasing
90.0
936.3
10484
Appendix
Table 8: Ablation results for removing different debate phases under different defense settings.
Dataset
Gemma2:9B
Llama3.1:8B
Llama3:8B
Mistral:7B
Time
Tok.
ASR
Time
Tok.
ASR
Time
Tok.
ASR
Time
Tok.
ASR
FEVER
54.92
1163
100.0%
120.69
2159
84.0%
68.42
1335
100.0%
128.71
1182
100.0%
HealthVer
56.75
1178
100.0%
107.27
1925
89.0%
58.55
1335
100.0%
127.57
1318
100.0%
CLIMATE-FEVER
56.18
1252
100.0%
124.75
2293
85.0%
63.15
1340
100.0%
148.16
1383
100.0%
SciFact
61.47
1289
100.0%
105.22
2379
94.0%
63.74
1244
100.0%
148.22
1470
100.0%
Avg
57.33
1220
100.0 %
114.48
2189
88.0 %
63.46
1313
100.0 %
138.15
1338
100.0 %
Appendix
Table 9: D2D attack performance across models and datasets without defense.
Model
Defense
AutoDAN
Open-Prompt-Inj.
JudgeDeceiver
ADMIT
D2D
Llama3:8B
None
5.83
1.28
581.82
283.60
63.46
Instructional
5.87
1.28
551.05
77.13
53.55
Known-answer
81.64
0.81
530.93
256.33
50.51
LLM-based
7.97
0.00
495.26
235.67
53.26
Response-based
6.49
1.22
796.99
265.04
52.71
Paraphrasing
11.45
1.44
548.00
237.90
46.28
Appendix
Table 10: Average runtime (s/sample) of D2D and baselines, matching the ASR cells of Table 3 .
Figure 4: Payload length, diversity, template reuse, and lexical size across attack methods. From left to right: attack-string length distribution in characters, unique payload ratio, most frequent payload ratio, and average payload length in words.
Model
Target Model →
Gemma2:9B
Llama3.1:8B
Llama3:8B
Defense type
ASR
Time
ASR
Time
ASR
Time
Mistral:7B
Instructional
100.00%
177.48
100.00%
129.46
100.00%
147.91
Known-answer
86.75%
323.11
85.75%
328.32
86.25%
331.15
LLM-based
100.00%
178.15
100.00%
132.51
100.00%
163.86
Response-based
100.00%
186.69
100.00%
135.68
100.00%
135.68
Paraphrasing
100.00%
177.48
100.00%
134.49
100.00%
148.48
Appendix
Table 11: D2D attack performance across different target LLMs
Method
Peak RAM
Peak VRAM
Open-Prompt- Injection
6.86 GB
10.06 GB
D2D
8.17 GB
12.33 GB
ADMIT
11.66 GB
12.32 GB
JudgeDeceiver
21.49 GB
39.61 GB
AutoDAN
17.96 GB
40.99 GB
Appendix
Table 12: Peak RAM and VRAM usage
Setting
(λq,λt,λe)
ASR (%)
Baseline
(1.5, 0.10, 0.005)
84.0
Low weights
(0.5, 0.05, 0.001)
78.0
High weights
(3.0, 0.20, 0.010)
72.0
Appendix
Table 13: Sensitivity to reward weight configuration.
Payoff scale
Scale
ASR (%)
Avg. Time (s)
Avg. Tokens
0.25
0.25
90.0
331.3
1791.5
0.5
0.50
90.0
347.2
1921.1
1
1.00
84.0
380.2
1925.6
2
2.00
90.0
322.3
1674.7
Appendix
Table 14: Sensitivity analysis of EGT payoff matrix scale under a fixed reward configuration.
Defense
Accuracy
Time (s)
No defense
100.0
92.84
Instruction
99.0
97.49
Known-answer
100.0
196.94
LLM-based
100.0
843.95
Paraphrasing
100.0
296.45
PPL
100.0
96.35
Appendix
Table 15: Benign performance under different defenses on SimpleFacts.
Metric
Single-agent
Multi-agent
Succeeded in 1st iteration
16
104
Succeeded in 2–10 iterations
22
64
Failed after 10 iterations
172
42
Overall success rate
18.09
80.00
Avg. iterations to success
8.54
2.77
Total time
3 h
30 min
Appendix
Table 16: Comparison of single-agent and multi-agent debate.
Defense Setting
Success
ASR (%)
No Defense
60
75.00
Instructional
54
67.50
LLM-based
0
0.00
Known-answer
29
36.25
Paraphrasing
52
65.00
Response-based
57
71.25
Appendix
Table 17: Ablation results of D2D without game-theoretic optimization under different defense settings.
Adversarial attacks on large language models probe how crafted inputs, internal changes, and agent interactions bypass safety or disrupt reliable behavior.