Fake News Detectors are Biased against Texts Generated by Large Language Models
Authors: Jinyan Su, Terry Yue Zhuo, Jonibek Mansurov, Di Wang, Preslav Nakov
Organizations: Mohamed bin Zayed University of Artificial Intelligence · Monash University and CSIRO’s Data61 · King Abdullah University of Science and Technology
The spread of fake news has emerged as a critical challenge, undermining trust and posing threats to society. In the era of Large Language Models (LLMs), the capability to generate believable fake content has intensified these concerns. In this study, we present a novel paradigm to evaluate fake news detectors in scenarios involving both human-written and LLM-generated misinformation. Intriguingly, our findings reveal a significant bias in many existing detectors: they are more prone to flagging LLM-generated content as fake news while often misclassifying human-written fake news as genuine. This unexpected bias appears to arise from distinct linguistic patterns inherent to LLM outputs. To address this, we introduce a mitigation strategy that leverages adversarial training with LLM-paraphrased genuine news. The resulting model yielded marked improvements in detection accuracy for both human and LLM-generated news. To further catalyze research in this domain, we release two comprehensive datasets, GossipCop++ and PolitiFact++, thus amalgamating human-validated articles with LLM-generated fake and real news.
Table 1: Performance metrics of various fake news detectors on the GossipCop++ and PolitiFact++ datasets. HR : Human-written Real news. HF : Human-written Fake news. MF : LLM-generated Fake news.
Dataset
MF
HF
HR
PolitiFact++
97
97
194
GossipcopCop++
4084
4084
4169
Table 2: Details of PolitiFact++ and GossipcopCop++ .
style
quotes
HF > MF
MF < HR
-
exclaim
HF < MF
MF > HR
-
allpunc
HF < MF
MF > HR
-
allcaps
HF < MF
MF > HR
HF > HR
stops
HF > MF
MF < HR
-
CC
HF > MF
MF < HR
-
CD
HF < MF
MF > HR
-
Table 3: Comparison of content-based features across Human-written Fake news (HF), LLM-generated Fake news (MF), and Human-written Real news (HR) for the GossipCop++ dataset. The table showcases differences in style, complexity, bias, affect, morale, and event features. The colour intensity represents the significance of the difference ( p value), with darker shades indicating higher significance.
Dataset
Model
Accuracy
F1
Recall
Precision
Auroc
HR
HF
MF
Overall
GossipCop++
logistic regression
77.09(0.2)
47.33(1.0)
95.79(0.3)
74.33(0.2)
73.59(0.3)
75.75(0.1)
71.56(0.6)
74.41(0.2)
logistic regression(debiased)
71.51(0.2)
53.89(0.4)
86.51(0.1)
70.86(0.1)
70.66(0.1)
71.13(0.1)
70.20(0.2)
70.86(0.1)
decision tree
70.32(0.5)
54.80(0.9)
86.90(0.5)
70.59(0.2)
70.66(0.3)
70.49(0.2)
70.85(0.6)
70.60(0.2)
decision tree(debiased)
67.43(0.5)
57.91(1.0)
78.70(0.9)
67.87(0.2)
68.00(0.3)
67.72(0.2)
68.30(0.8)
67.88(0.2)
PolitiFact++
logistic regression
63.37(4.7)
70.21(5.1)
93.84(1.9)
72.67(2.4)
75.11(1.8)
69.57(2.9)
81.97(1.8)
73.67(2.2)
Table 4: Performance metrics of logistic regression and decision tree models on the GossipCop++ and PolitiFact++ datasets. HR : Human-written Real news. HF : Human-written Fake news. MF : LLM-generated Fake news.
Dataset
Model
HF
MR
Before
Debiased
Difference
Before
Debiased
Difference
GossipCop++
RoBERTa
Large
77.97
84.46
6.49 ↑
24.24
90.70
66.46 ↑
Base
69.65
78.21
8.57 ↑
31.21
90.58
59.36 ↑
BERT
Large
70.13
77.85
7.71 ↑
52.63
89.47
36.84 ↑
Base
66.59
72.46
5.88 ↑
46.02
93.27
47.25 ↑
ELECTRA
Large
71.73
77.60
5.88 ↑
31.95
90.82
58.87 ↑
Table 5: Performance comparison of various models on the GossipCop++ and PolitiFact++ datasets before and after debiasing. The ‘Difference’ column highlights the performance change post-debiasing. HF :human-written fake news. MR : LLM-generated real news.
Metric
HR
MR
Significance
Cohen’s Kappa
Semantic Consistency
-
0.95
p>0.05
-
Readability Assessment
4.5
4.6
p>0.05
0.85
Authenticity Perception
4.6
4.5
p>0.05
0.88
Stylistic Alignment
4.7
4.6
p>0.05
0.86
Table 6: Evaluation metrics for original (HR) versus LLM-generated (MR) real news. The table presents scores for semantic consistency (scale of 0 to 1), readability, authenticity perception, and stylistic alignment (all on a scale of 0 to 5). Inter-annotator agreement is also provided via Cohen’s Kappa scores.
style
quotes
HF > MF
MF < HR
-
exclaim
HF < MF
-
HF < HR
allpunc
HF < MF
MF > HR
HF > HR
allcaps
HF < MF
MF > HR
-
stops
HF > MF
MF < HR
-
CC
HF > MF
-
HF > HR
CD
HF < MF
MF > HR
-
Table 7: Comparison of content-based features across Human-written Fake news (HF), LLM-generated Fake news (MF), and Human-written Real news (HR) for the PolitiFact++ dataset. The table showcases differences in style, complexity, bias, affect, morale, and event features. The colour intensity represents the significance of the difference, with darker shades indicating higher significance.