Debate to Deceive: Poisoning LLM's Fact-Checking via Game-Theoretic Multi-Agent Debate
Organizations: Newcastle University, United Kingdom · Peking University, China · Fudan University, China
Abstract
Knowledge attacks on Large Language Models, such as manipulating fact-checking judgments, pose significant risks to AI reliability. Existing approaches -- adversarial suffix/template injection and single-agent self-reflection -- lack robustness across diverse defenses and are prone to confirmation bias and costly trial-and-error, respectively. We propose Debate-to-Deceive (D2D), a multi-agent debate framework grounded in human debate structure and rhetorical style, enabling agents to generate adversarial passages that are semantically coherent, rhetorically effective, and resistant to early confirmation bias. D2D formalizes the adversarial process as an evolutionary game in which agents adaptively evolve their rhetorical strategies (Logos, Pathos, Ethos) through competition. To manage the intractably large LLM output space, we decouple agent behavior into a strategic layer, which selects a rhetorical action from a compact strategy space via evolutionary-inspired learning dynamics, and a generation layer, where the LLM produces a concrete passage conditioned on that strategy and the debate context. On four fact-checking benchmarks (FEVER, HealthVer, CLIMATE-FEVER, and SciFact), D2D substantially outperforms existing attacks and evades most defenses.
Figures & tables
| Strategy | Logos | Pathos | Ethos |
| Logos | 0 | - | |
| Pathos | 0 | - | |
| Ethos | - | 0 |
| Dataset | No Defense | Known-answer |
| FEVER | 98.5 [95.7, 99.5] | 96.5 [93.0, 98.3] |
| HealthVer | 97.0 [93.6, 98.6] | 97.5 [94.3, 98.9] |
| CLIMATE-FEVER | 99.5 [97.2, 99.9] | 99.5 [97.2, 99.9] |
| SciFact | 100.0 [98.1, 100.0] | 99.5 [97.2, 99.9] |
| Avg. | 98.8 | 98.3 |
| Model | Defense | ADAN | OPI | JD | ADMIT | D2D |
| Llama3 | None | 100.0 | 63.7 | 13.0 | 82.5 | 100.0 |
| Instr. | 100.0 | 32.9 | 14.5 | 20.0 | 100.0 | |
| Known | 0.0 | 31.6 | 18.5 | 55.3 | 94.8 | |
| LLM | 100.0 | 0.0 | 29.0 | 0.0 | 100.0 | |
| Resp. | 99.8 | 61.3 | 18.0 | 53.8 | 100.0 | |
| Para. | 99.8 | 36.2 | 23.5 | 37.8 | 100.0 |
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
| Symbol | Description |
| Agents, States and Actions | |
| Current agent; all other agents | |
| Correct label side; target side | |
| Current and next state | |
| Action of agent ; actions of others | |
| Strategy space {logos, pathos, ethos} | |
| Dataset | Claims | Passages | Domain |
| FEVER | 185,445 | 5,416,568 | General |
| HealthVer | 2,149 | 6,961 | Health |
| Climate-FEVER | 7,675 | 5,416,568 | Climate |
| SciFact | 1,409 | 5,183 | Science |
| Baseline | Original Design | Adaptation in Our Setting |
| AutoDAN | Automatically generates stealthy jailbreak prompts using a hierarchical genetic algorithm. | Redirects the optimization target to the attacker-specified verdict label under the same fact-checking prompt template. |
| Open-Prompt-Injection | Provides a general framework for constructing prompt injection attacks. | Uses the target verdict label as the injected adversarial objective to induce label flipping. |
| ADMIT | Performs semantically aligned poisoning via iterative refinement and memory-based self-reflection. | Iteratively refines adversarial fact-checking arguments to induce the target verdict label. |
| Judge-Deceiver | Optimizes adversarial sequences so that LLM-as-a-Judge selects an attacker-controlled response. | Redirects the optimized sequence toward causing the target LLM to output the attacker-specified verdict label. |
| Defense | Type | Description |
| Instruction | Prevention | Adds explicit instructions to make the model resist malicious or manipulative prompts. |
| Paraphrasing | Prevention | Rewrites the input prompt to reduce the effectiveness of adversarial patterns. |
| Known-answer | Detection | Uses reference answers or target context to identify inconsistent or adversarial behavior. |
| LLM-based | Detection | Uses an LLM as a judge to detect unsafe or adversarial inputs or outputs. |
| Response-based | Detection | Generates a response first and then evaluates whether it is harmful or suspicious. |
| Removed | ||||
| Phase | Defense | ASR | Time | Tokens |
| Foundation | None | 85.0 | 1395.3 | 14640 |
| Instructional | 80.0 | 1138.5 | 12354 | |
| Known-answer | 0.0 | 2177.7 | 25316 | |
| LLM-based | 95.0 | 1062.7 | 11246 | |
| Paraphrasing | 90.0 | 936.3 | 10484 |
| Dataset | Gemma2:9B | Llama3.1:8B | Llama3:8B | Mistral:7B | ||||||||
| Time | Tok. | ASR | Time | Tok. | ASR | Time | Tok. | ASR | Time | Tok. | ASR | |
| FEVER | 54.92 | 1163 | 100.0% | 120.69 | 2159 | 84.0% | 68.42 | 1335 | 100.0% | 128.71 | 1182 | 100.0% |
| HealthVer | 56.75 | 1178 | 100.0% | 107.27 | 1925 | 89.0% | 58.55 | 1335 | 100.0% | 127.57 | 1318 | 100.0% |
| CLIMATE-FEVER | 56.18 | 1252 | 100.0% | 124.75 | 2293 | 85.0% | 63.15 | 1340 | 100.0% | 148.16 | 1383 | 100.0% |
| SciFact | 61.47 | 1289 | 100.0% | 105.22 | 2379 | 94.0% | 63.74 | 1244 | 100.0% | 148.22 | 1470 | 100.0% |
| Avg | 57.33 | 1220 | 100.0 % | 114.48 | 2189 | 88.0 % | 63.46 | 1313 | 100.0 % | 138.15 | 1338 | 100.0 % |
| Model | Defense | AutoDAN | Open-Prompt-Inj. | JudgeDeceiver | ADMIT | D2D |
| Llama3:8B | None | 5.83 | 1.28 | 581.82 | 283.60 | 63.46 |
| Instructional | 5.87 | 1.28 | 551.05 | 77.13 | 53.55 | |
| Known-answer | 81.64 | 0.81 | 530.93 | 256.33 | 50.51 | |
| LLM-based | 7.97 | 0.00 | 495.26 | 235.67 | 53.26 | |
| Response-based | 6.49 | 1.22 | 796.99 | 265.04 | 52.71 | |
| Paraphrasing | 11.45 | 1.44 | 548.00 | 237.90 | 46.28 |
| Model | Target Model | Gemma2:9B | Llama3.1:8B | Llama3:8B | |||
| Defense type | ASR | Time | ASR | Time | ASR | Time | |
| Mistral:7B | Instructional | 100.00% | 177.48 | 100.00% | 129.46 | 100.00% | 147.91 |
| Known-answer | 86.75% | 323.11 | 85.75% | 328.32 | 86.25% | 331.15 | |
| LLM-based | 100.00% | 178.15 | 100.00% | 132.51 | 100.00% | 163.86 | |
| Response-based | 100.00% | 186.69 | 100.00% | 135.68 | 100.00% | 135.68 | |
| Paraphrasing | 100.00% | 177.48 | 100.00% | 134.49 | 100.00% | 148.48 | |
| Method | Peak RAM | Peak VRAM |
| Open-Prompt- Injection | 6.86 GB | 10.06 GB |
| D2D | 8.17 GB | 12.33 GB |
| ADMIT | 11.66 GB | 12.32 GB |
| JudgeDeceiver | 21.49 GB | 39.61 GB |
| AutoDAN | 17.96 GB | 40.99 GB |
| Setting | ASR (%) | |
| Baseline | (1.5, 0.10, 0.005) | 84.0 |
| Low weights | (0.5, 0.05, 0.001) | 78.0 |
| High weights | (3.0, 0.20, 0.010) | 72.0 |
| Payoff scale | Scale | ASR (%) | Avg. Time (s) | Avg. Tokens |
| 0.25 | 0.25 | 90.0 | 331.3 | 1791.5 |
| 0.5 | 0.50 | 90.0 | 347.2 | 1921.1 |
| 1 | 1.00 | 84.0 | 380.2 | 1925.6 |
| 2 | 2.00 | 90.0 | 322.3 | 1674.7 |
| Defense | Accuracy | Time (s) |
| No defense | 100.0 | 92.84 |
| Instruction | 99.0 | 97.49 |
| Known-answer | 100.0 | 196.94 |
| LLM-based | 100.0 | 843.95 |
| Paraphrasing | 100.0 | 296.45 |
| PPL | 100.0 | 96.35 |
| Metric | Single-agent | Multi-agent |
| Succeeded in 1st iteration | 16 | 104 |
| Succeeded in 2–10 iterations | 22 | 64 |
| Failed after 10 iterations | 172 | 42 |
| Overall success rate | 18.09 | 80.00 |
| Avg. iterations to success | 8.54 | 2.77 |
| Total time | 3 h | 30 min |
| Defense Setting | Success | ASR (%) |
| No Defense | 60 | 75.00 |
| Instructional | 54 | 67.50 |
| LLM-based | 0 | 0.00 |
| Known-answer | 29 | 36.25 |
| Paraphrasing | 52 | 65.00 |
| Response-based | 57 | 71.25 |
Adversarial Attacks on LLMs
Adversarial attacks on large language models probe how crafted inputs, internal changes, and agent interactions bypass safety or disrupt reliable behavior.
See all 383 papersBelieved but Not Said: Measuring Hidden Belief Collapse in Multi-Agent LLM Discussions
Benchmarking Factual Robustness of LLMs via Multi-conversation Persuasion
Refusal Inertia''**: a model's initial refusal often propagates through subsequent turns largely to maintain contextual consistency, thereby masking its true vulnerability to sophisticated, isolated persuasion attempts. To rigorously evaluate the cold-start'' defense capabilities of SOTA models, we introduce the SAST-IR (Stateful Attacker, Stateless Target - Iterative Refinement) framework. By enforcing a memory wipe on the target while retaining the attacker's history, we simulate a worst-case adversarial setting using multi-turn (stateless) iterations. Leveraging CP-Agent (Cognitive Persuasion Agent), an enhanced diagnosis-guided agent, our experiments on the custom CounterFact-Strict dataset () yield alarming results: simple, diverse attack strategies achieved a staggering 96% success rate, exposing severe brittleness in memory-less defense. Furthermore, we reveal a ``Complexity Paradox'': while complex, iteratively refined attacks are effective, they often trigger defensive compliance, whereas simple strategies achieve a higher rate of genuine persuasion (84.7%). Our code and dataset are available at GitHub, https://github.com/cza1006/llm-persuasion-defense.