Fake News Detectors are Biased against Texts Generated by Large Language Models
Authors: Jinyan Su, Terry Yue Zhuo, Jonibek Mansurov, Di Wang, Preslav Nakov
Organizations: Mohamed bin Zayed University of Artificial Intelligence · Monash University and CSIRO’s Data61 · King Abdullah University of Science and Technology
The spread of fake news has emerged as a critical challenge, undermining trust and posing threats to society. In the era of Large Language Models (LLMs), the capability to generate believable fake content has intensified these concerns. In this study, we present a novel paradigm to evaluate fake news detectors in scenarios involving both human-written and LLM-generated misinformation. Intriguingly, our findings reveal a significant bias in many existing detectors: they are more prone to flagging LLM-generated content as fake news while often misclassifying human-written fake news as genuine. This unexpected bias appears to arise from distinct linguistic patterns inherent to LLM outputs. To address this, we introduce a mitigation strategy that leverages adversarial training with LLM-paraphrased genuine news. The resulting model yielded marked improvements in detection accuracy for both human and LLM-generated news. To further catalyze research in this domain, we release two comprehensive datasets, GossipCop++ and PolitiFact++, thus amalgamating human-validated articles with LLM-generated fake and real news.
Table 1: Performance metrics of various fake news detectors on the GossipCop++ and PolitiFact++ datasets. HR : Human-written Real news. HF : Human-written Fake news. MF : LLM-generated Fake news.
Dataset
MF
HF
HR
PolitiFact++
97
97
194
GossipcopCop++
4084
4084
4169
Table 2: Details of PolitiFact++ and GossipcopCop++ .
style
quotes
HF > MF
MF < HR
-
exclaim
HF < MF
MF > HR
-
allpunc
HF < MF
MF > HR
-
allcaps
HF < MF
MF > HR
HF > HR
stops
HF > MF
MF < HR
-
CC
HF > MF
MF < HR
-
CD
HF < MF
MF > HR
-
Table 3: Comparison of content-based features across Human-written Fake news (HF), LLM-generated Fake news (MF), and Human-written Real news (HR) for the GossipCop++ dataset. The table showcases differences in style, complexity, bias, affect, morale, and event features. The colour intensity represents the significance of the difference ( p value), with darker shades indicating higher significance.
Dataset
Model
Accuracy
F1
Recall
Precision
Auroc
HR
HF
MF
Overall
GossipCop++
logistic regression
77.09(0.2)
47.33(1.0)
95.79(0.3)
74.33(0.2)
73.59(0.3)
75.75(0.1)
71.56(0.6)
74.41(0.2)
logistic regression(debiased)
71.51(0.2)
53.89(0.4)
86.51(0.1)
70.86(0.1)
70.66(0.1)
71.13(0.1)
70.20(0.2)
70.86(0.1)
decision tree
70.32(0.5)
54.80(0.9)
86.90(0.5)
70.59(0.2)
70.66(0.3)
70.49(0.2)
70.85(0.6)
70.60(0.2)
decision tree(debiased)
67.43(0.5)
57.91(1.0)
78.70(0.9)
67.87(0.2)
68.00(0.3)
67.72(0.2)
68.30(0.8)
67.88(0.2)
PolitiFact++
logistic regression
63.37(4.7)
70.21(5.1)
93.84(1.9)
72.67(2.4)
75.11(1.8)
69.57(2.9)
81.97(1.8)
73.67(2.2)
Table 4: Performance metrics of logistic regression and decision tree models on the GossipCop++ and PolitiFact++ datasets. HR : Human-written Real news. HF : Human-written Fake news. MF : LLM-generated Fake news.
Dataset
Model
HF
MR
Before
Debiased
Difference
Before
Debiased
Difference
GossipCop++
RoBERTa
Large
77.97
84.46
6.49 ↑
24.24
90.70
66.46 ↑
Base
69.65
78.21
8.57 ↑
31.21
90.58
59.36 ↑
BERT
Large
70.13
77.85
7.71 ↑
52.63
89.47
36.84 ↑
Base
66.59
72.46
5.88 ↑
46.02
93.27
47.25 ↑
ELECTRA
Large
71.73
77.60
5.88 ↑
31.95
90.82
58.87 ↑
Table 5: Performance comparison of various models on the GossipCop++ and PolitiFact++ datasets before and after debiasing. The ‘Difference’ column highlights the performance change post-debiasing. HF :human-written fake news. MR : LLM-generated real news.
Metric
HR
MR
Significance
Cohen’s Kappa
Semantic Consistency
-
0.95
p>0.05
-
Readability Assessment
4.5
4.6
p>0.05
0.85
Authenticity Perception
4.6
4.5
p>0.05
0.88
Stylistic Alignment
4.7
4.6
p>0.05
0.86
Table 6: Evaluation metrics for original (HR) versus LLM-generated (MR) real news. The table presents scores for semantic consistency (scale of 0 to 1), readability, authenticity perception, and stylistic alignment (all on a scale of 0 to 5). Inter-annotator agreement is also provided via Cohen’s Kappa scores.
style
quotes
HF > MF
MF < HR
-
exclaim
HF < MF
-
HF < HR
allpunc
HF < MF
MF > HR
HF > HR
allcaps
HF < MF
MF > HR
-
stops
HF > MF
MF < HR
-
CC
HF > MF
-
HF > HR
CD
HF < MF
MF > HR
-
Table 7: Comparison of content-based features across Human-written Fake news (HF), LLM-generated Fake news (MF), and Human-written Real news (HR) for the PolitiFact++ dataset. The table showcases differences in style, complexity, bias, affect, morale, and event features. The colour intensity represents the significance of the difference, with darker shades indicating higher significance.
Large Language Models (LLMs) are increasingly used for automated fact-checking, yet their susceptibility to gender bias in this context remains underexplored. This study presents the first systematic investigation of gender bias in LLM-based fake news detection using real-world data. We augment the LIAR benchmark with three gender variants of speaker job titles (Neutral, Male, Female) for each statement to test whether veracity judgments vary solely based on gender presentation. Six state-of-the-art LLMs are evaluated across multiple bias and fairness metrics. All models exhibit gender sensitivity: 9.79%-35.13% of statements receive inconsistent labels across the three variants, with Male-Female comparisons showing 6.5%-23.6% flip rates. Two primary bias manifestations are identified: instability (inconsistent judgments) and directionality (systematic favoritism). Five models show statistically significant directional effects, with the strongest effects displaying male-skeptic patterns. These findings demonstrate that gender bias undermines both reliability and fairness in LLM-based fake news detection, highlighting the need for bias-aware evaluation and mitigation strategies. The augmented dataset is publicly released to support future research.
Recent advances in large language models (LLMs) have enabled the large-scale generation of highly fluent and deceptive news-like content. While prior work has often treated fake news detection as a binary classification problem, modern fake news increasingly arises through human-AI collaboration, where strategic inaccuracies are embedded within otherwise accurate and credible narratives. These mixed-truth cases represent a realistic and consequential threat, yet they remain underrepresented in existing benchmarks. To address this gap, we introduce MANYFAKE, a synthetic benchmark containing 6,798 fake news articles generated through multiple strategy-driven prompting pipelines that capture many ways fake news can be constructed and refined. Using this benchmark, we evaluate a range of state-of-the-art fake news detectors. Our results show that even advanced reasoning-enabled models approach saturation on fully fabricated stories, but remain brittle when falsehoods are subtle, optimized, and interwoven with accurate information.
Xinyu Wang, Sai Koneru, Wenbo Zhang +3
Pennsylvania State University University Park, PA 16802, USA
To efficiently combat the spread of LLM-generated misinformation in the news domain, we present RADAR, a Retrieval-Augmented Detector with Adversarial Refinement for adaptive LLM-generated fake news detection. Our approach employs a generator that rewrites real articles with factual perturbations, paired with a lightweight detector that verifies claims using dense passage retrieval. To enable effective co-evolution, we introduce Verbal Adversarial Feedback (VAF). Rather than relying on scalar rewards, VAF issues structured natural-language critiques; these guide the generator toward more sophisticated evasion attempts, compelling the detector to adapt and improve. Experiments on an LLM-generated fake news benchmark show that RADAR outperforms retrieval-augmented trainable baselines and general-purpose LLMs with retrieval. Further analysis shows that retrieval on both the generator and detector sides improves performance, while VAF and few-shot demonstrations offer complementary benefits. RADAR also transfers better to fake news generated by an unseen external attacker, suggesting improved generalization beyond the specific co-evolved generator used during training.