Adversarial Robustness of Retrieval-Based Fact-Checking: Detecting and Mitigating Evidence Contamination
Organizations: School of Computer Science, University of Galway, Ireland · Research Ireland Insight Centre and ADAPT Centre, University of Galway, Ireland
Abstract
Retrieval-based fact-checking systems ground verification in large document collections, creating a critical vulnerability: adversarial evidence contamination. We demonstrate that factually inconsistent but contextually coherent evidence generated to satisfy both retrieval-similarity and linguistic-fluency constraints can systematically mislead verification models. Contaminating just 3.1-7.4% of training corpora with five label-specific strategies causes adversarial content to penetrate up to 35.2% of top-k retrievals, degrading verification accuracy by up to 46.9% relative F1. Existing retrieval-stage defenses, validated against gradient-optimized or syntactically anomalous attacks, transfer poorly to this threat model: detection rates collapse and F1 recovery exhibits high variance across seeds. We propose GLAD (Geometric-Lexical Adversarial Defense), which fuses dense embedding peer-coherence with lexical distributional isolation. Across two benchmarks, two embedding models, and five random seeds, GLAD achieves the highest detection rates (70-86%) and the most consistent F1 recovery, reclaiming 73-80% of contamination-induced degradation on LIAR-RAW while approaching clean baselines on RAWFC. Crucially, unlike defenses that trade clean performance for robustness, GLAD imposes negligible cost on uncontaminated corpora.
Figures & tables
| Dataset | Model | K | F1 score | ARR % | ||
|---|---|---|---|---|---|---|
| Clean | Adversarial | |||||
| RAWFC | nomic | 3 | 0.5825 | 0.5309 | -8.9% | 15.7% |
| 5 | 0.5403 | 0.5342 | -1.1% | 9.9% | ||
| 10 | 0.5750 | 0.5287 | -8.0% | 6.8% | ||
| qwen | 3 | 0.5187 | 0.4398 | -15.2% | 8.1% | |
| 5 | 0.5368 | 0.5120 | -4.6% | 11.6% | ||
| Dataset | Embed. | Clean | Contam. | Existing Defenses | Proposed (Ours) | |||
|---|---|---|---|---|---|---|---|---|
| Trust. | RAGDef. | SCOD | Lex. | GLAD | ||||
| RAWFC | nomic | .556 .030 | .519 .026 | .545 .027 (68) | .507 .076 (27) | .549 .050 (74) | .517 .025 (18) | .570 .016 ( 85 ) |
| qwen | .574 .019 | .533 .049 | .563 .037 (72) | .533 .024 (29) | .551 .021 (78) | .485 .046 (17) | .563 .016 ( 86 ) | |
| LIAR-RAW | nomic | .271 .010 | .140 .019 | .201 .012 (55) | .151 .021 (35) | .198 .023 (58) | .158 .020 (25) | .236 .022 ( 72 ) |
| qwen | .266 .015 | .133 .012 | .226 .012 (54) | .137 .010 (38) | .205 .009 (57) | .139 .016 (27) | .239 .020 ( 70 ) | |
| Data | Emb. | Trust. | RAGDef. | SCOD | Lex. | GLAD |
|---|---|---|---|---|---|---|
| RAWFC | nomic | 70.5 | 33.7 | 79.9 | 7.7 | 138.8 |
| qwen | 72.6 | 0.1 | 42.7 | 116.3 | 72.0 | |
| LIAR- RAW | nomic | 46.7 | 8.4 | 44.6 | 13.5 | 73.4 |
| qwen | 70.3 | 3.3 | 54.2 | 4.4 | 80.1 |
| Dataset | Embed. | Clean | GLAD (contam.) | GLAD (clean) |
|---|---|---|---|---|
| RAWFC | nomic | .556 .030 | .570 .016 | .561 .032 |
| qwen | .574 .019 | .563 .016 | .562 .035 | |
| LIAR-RAW | nomic | .271 .010 | .236 .022 | .270 .009 |
| qwen | .266 .015 | .239 .020 | .269 .017 |
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
| Setting | Value |
|---|---|
| Generator | |
| Model | google/gemma-3-12b-it |
| Precision | bfloat16 |
| Decoding | |
| do_sample | True |
| temperature | 0.3 |
| Tactic | Description | Template | |
| Supporting — FALSE and PANTS-FIRE claims, pushed toward TRUE / MOSTLY-TRUE | |||
| 1 | False Confirmation | Present fabricated or misleading data that confirms the false claim | “[Specific claim element] is confirmed by [credible-sounding source]” |
| 2 | Authority Misattribution | Cite credible experts or institutions as supporting the false claim | “[Expert/Institution] validates [false claim element] in recent analysis” |
| 3 | Contextual Distortion | Present real facts in a way that supports the false claim | “[Related fact] demonstrates that [false claim element] is accurate” |
| 4 | Partial Truth Expansion | Find a kernel of truth and expand it to support the entire false claim | “[Minor true detail] confirms the broader pattern of [false claim]” |
| 5 | Consensus Manufacturing | Suggest widespread agreement supporting the false claim | “Multiple analyses confirm [false claim element] across investigations” |
| RAWFC | LIAR-RAW | |
| Training claims | 1,612 | 10,065 |
| Sentences per claim | 5 | 5 |
| Adversarial sentences | 8,060 | 50,325 |
| Clean evidence sentences | 248k | 627k |
| Contamination budget | 3.1% | 7.4% |
| Label classes | 3 | 6 |
Retrieval Robustness
Retrieval robustness studies how search systems preserve relevant results when queries, data, collections, or evidence are noisy, shifted, or adversarial.
See all 38 papers