Organizations: Xi’an Jiaotong-Liverpool University, Suzhou, China · University of Liverpool, Liverpool, United Kingdom · Nanjing Sibei Intelligent Technology Co., Ltd., Nanjing, China
Automated fake news detection is vital on social media, where fake news is easily rewritten to fool detectors. However, existing approaches typically rely on superficial writing cues or single-style text rewrites, resulting in poor robustness under diverse cosmetic modifications. We construct MANTIS-Bench, a multi-axis presentation-attack benchmark that pairs original news with rewritten views across writing style, emotional expression, and dissemination framing. We further propose MANTIS, an instance-adaptive adversarial framework enabling dynamic worst-case training tailored to each news article with zero inference-time overhead. This reframes fake news defense from heuristic format smoothing into targeted, fact-invariant credibility assessment. Experiments on MANTIS-Bench show that MANTIS reduces the Union Attack Success Rate (UASR) by 3.55 percentage points relative to SheepDog on GossipCop and consistently outperforms prior single-axis defenses on Macro-F1 while maintaining competitive clean accuracy, demonstrating the effectiveness of multi-axis and instance-adaptive worst-case training.
Figures & tables
Figure 1: Illustration of presentation-level vulnerability in fake news detection. Different presentation variations lead to inconsistent predictions from the same detector.
Figure 2: Overview of MANTIS. An LLM rewriter generates six offline views covering writing style, emotional expression, and dissemination framing. MANTIS trains jointly on the original news and each instance’s worst-case view, while inference requires only a single forward pass.
Figure 3: Macro-F1 under original and transformed views on GossipCop. Style, Emotion, and Dissemination denote the three transformation dimensions.
Figure 4: Case-specific vulnerability on GossipCop (Llama-3.1-8B-Instruct). Points denote originally correct cases misclassified under one transformation; axes show gold-label probabilities under the harmful and other transformations.
Train
Test
Dataset
Domain
Real
Fake
Real
Fake
PolitiFact
Politics
180
180
45
45
GossipCop
Entertainment
3,166
3,166
792
792
Weibo21
Nine topics
–
675
675
Table 1: Dataset statistics. Weibo21 reports the topic-balanced evaluation subset and is not used for training.
Figure 5: Human evaluation of factual-claim preservation across style, emotion, and dissemination transformations.
Original
Style
Emotion
Dissemination
Model
Acc. ↑
F1↑
Acc. ↑
F1↑
Acc. ↑
F1↑
Acc. ↑
F1↑
UASR ↓
Dataset: PolitiFact
Supervised Detectors
BERT
87.00
86.60
81.78
81.39
85.67
85.50
84.44
84.20
15.04
RoBERTa
89.89
89.79
81.78
81.52
90.00
89.86
81.89
80.77
17.09
DeBERTa
88.89
88.86
85.22
85.16
90.11
90.08
81.11
80.57
18.63
Table 2: Main results on the PolitiFact and GossipCop subsets of MANTIS-Bench. All values are percentages; F1 denotes Macro-F1. Per dataset and column, the best and second-best results are bold and underlined , respectively.
Macro- F1↑
Variant
Orig.
Style
Emo.
Dis.
UASR ↓
PolitiFact
w/o Lori
89.27
89.98
89.25
88.83
10.39
w/o Ladv
93.01
88.66
91.84
89.10
11.84
w/o worst-case
92.55
88.42
89.23
87.62
9.83
MANTIS
93.53
90.73
92.58
89.99
7.51
Table 3: Ablation results for the MANTIS objective. All values are percentages; bold indicates the best result.
SheepDog
AdSent
MANTIS
Test Set
Acc.
F1
Acc.
F1
Acc.
F1
Original
90.00
89.92
90.00
89.90
93.71
93.53
Adv.-A
81.11
80.99
85.56
85.47
82.20
81.95
Adv.-B
80.00
79.89
84.44
84.38
83.34
82.95
Adv.-C
82.44
82.36
87.78
87.74
88.85
88.63
Adv.-D
81.31
81.24
86.67
86.64
87.76
87.71
Table 4: Results (%) on the original PolitiFact test set and SheepDog publisher-style attacks. SheepDog and AdSent scores are from AdSent Tahmasebi et al. (2026) .
AdSent
MANTIS
View
Acc. ↑
Macro- F1↑
Acc. ↑
Macro- F1↑
Original
58.67
57.98
61.26
61.14
Style
59.56
59.01
60.07
60.05
Emotion
55.85
55.14
57.56
57.51
Dissem.
59.70
59.16
61.11
61.10
Table 5: Cross-lingual performance and efficiency on Weibo21 after training on English GossipCop.
Figure 6: Topic-level Accuracy (a) and Macro-F1 (b) on Weibo21, averaged over four zero-shot LLMs across original and transformed views.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Word clouds for the nine named Weibo21 topics used in our evaluation. Larger terms occur more frequently within the corresponding topic.
Topic
Train
Val.
Test
Total
Selected
Military
205
69
69
343
150
Health
600
200
200
1,000
150
Politics
511
170
171
852
150
Education
294
98
99
491
150
Culture/Ent.
864
288
288
1,440
150
Disasters
465
155
156
776
150
Appendix
Table 6: Topic distribution of the original Weibo21 dataset and the topic-balanced subset used in our evaluation. Val. denotes the official validation split.
Panel A: Accuracy (%)
Model
Orig.
Style
Emo.
Dissem.
Qwen2.5-7B
77.04
57.85
60.37
55.78
Llama-3.1-8B
66.96
49.11
43.19
36.00
Qwen3-8B
73.33
60.89
58.96
54.07
GLM-4-9B
65.48
55.33
50.74
50.52
Appendix
Table 7: Overall zero-shot results on the topic-balanced Weibo21 evaluation set. All values are percentages. Higher Accuracy and Macro-F1 and lower UASR are better. Boldface denotes the best result in each column.
Recent advances in large language models (LLMs) have enabled the large-scale generation of highly fluent and deceptive news-like content. While prior work has often treated fake news detection as a binary classification problem, modern fake news increasingly arises through human-AI collaboration, where strategic inaccuracies are embedded within otherwise accurate and credible narratives. These mixed-truth cases represent a realistic and consequential threat, yet they remain underrepresented in existing benchmarks. To address this gap, we introduce MANYFAKE, a synthetic benchmark containing 6,798 fake news articles generated through multiple strategy-driven prompting pipelines that capture many ways fake news can be constructed and refined. Using this benchmark, we evaluate a range of state-of-the-art fake news detectors. Our results show that even advanced reasoning-enabled models approach saturation on fully fabricated stories, but remain brittle when falsehoods are subtle, optimized, and interwoven with accurate information.
Xinyu Wang, Sai Koneru, Wenbo Zhang +3
Pennsylvania State University University Park, PA 16802, USA
In recent years, multimodal multidomain fake news detection has garnered increasing attention. Nevertheless, this direction presents two significant challenges: (1) Failure to Capture Cross-Instance Narrative Consistency: existing models usually evaluate each news in isolation, fail to capture cross-instance narrative consistency, and thus struggle to address the spread of cluster based fake news driven by social media; (2) Lack of Domain Specific Knowledge for Reasoning: conventional models, which rely solely on knowledge encoded in their parameters during training, struggle to generalize to new or data-scarce domains (e.g., emerging events or niche topics). To tackle these challenges, we introduce Retrieval-Augmented Multimodal Model for Fake News Detection (RAMM). First, RAMM employs a Multimodal Large Language Model (MLLM) as its backbone to capture cross-modal semantic information from news samples. Second, RAMM incorporates an Abstract Narrative Alignment Module. This component adaptively extracts abstract narrative consistency from diverse instances across distinct domains, aggregates relevant knowledge, and thereby enables the modeling of high-level narrative information. Finally, RAMM introduces a Semantic Representation Alignment Module, which aligns the model's decision-making paradigm with that of humans - specifically, it shifts the model's reasoning process from direct inference on multimodal features to an instance-based analogical reasoning process. Extensive experimental results on three public datasets validate the efficacy of our proposed approach. Our code is available at the following link: https://github.com/li-yiheng/RAMM
Yiheng Li, Weihai Lu, Hanyi Yu +1
University of International Business and Economics · Beijing, China · Peking University +4
To efficiently combat the spread of LLM-generated misinformation in the news domain, we present RADAR, a Retrieval-Augmented Detector with Adversarial Refinement for adaptive LLM-generated fake news detection. Our approach employs a generator that rewrites real articles with factual perturbations, paired with a lightweight detector that verifies claims using dense passage retrieval. To enable effective co-evolution, we introduce Verbal Adversarial Feedback (VAF). Rather than relying on scalar rewards, VAF issues structured natural-language critiques; these guide the generator toward more sophisticated evasion attempts, compelling the detector to adapt and improve. Experiments on an LLM-generated fake news benchmark show that RADAR outperforms retrieval-augmented trainable baselines and general-purpose LLMs with retrieval. Further analysis shows that retrieval on both the generator and detector sides improves performance, while VAF and few-shot demonstrations offer complementary benefits. RADAR also transfers better to fake news generated by an unseen external attacker, suggesting improved generalization beyond the specific co-evolved generator used during training.