Beyond the Evidence Coherence Trap: Agent-Driven Evidence Selection for Temporally Disordered Fake News Detection
Authors: Chi Wang, Min Gao, Yifei Zhang, Bincheng Gu, Zongwei Wang, Wentao Li
Organizations: Chongqing University, China · Also with Key Laboratory of Dependable Service Computing in Cyber Physical Society (Chongqing University), Ministry of Education of China. · University of Leicester, England
Fake news undermines public trust and the integrity of online information ecosystems. Temporally disordered fake news is especially elusive because it rearranges factual events into a false chronology. Individual events may remain factually valid even when their temporal relations are misrepresented. When similar events involving the same entity recur, retrieval may return semantically plausible evidence from the wrong period. Consequently, independent event verification may rely on locally coherent but temporally mismatched evidence and incorrectly validate the article. We call this the evidence coherence trap. Because existing benchmarks rarely isolate this setting, we construct ChronoFake, a source-disjoint benchmark of 30,706 articles derived from temporally anchored real events. It distinguishes temporal manipulation from valid narration, including flashbacks. To address this, we propose DelSA, a Document Selection Self-Updating Agent that iteratively refines evidence selection during training and applies a frozen rulebook for one-pass evidence selection at inference. DelSA uses an LLM to characterize candidate evidence, while a verifier-trained reranker balances local relevance against timeline consistency. Finally, the classifier compares the article-implied chronology extracted by the LLM with the evidence-derived chronology to reveal global temporal conflicts. DelSA outperforms existing methods on ChronoFake, while remaining competitive on PolitiFact and GossipCop.
Figures & tables
Figure 1. Comparison of detection paradigms for fake news. (a) Content-based methods rely on text semantics and ignore temporal structure. (b) Evidence-retrieval methods verify each event independently and may predict the article as real when all events are locally supported, but miss global chronological inconsistency. (c) Our DelSA framework leverages a self-updating agent to guide evidence reselection, reconstruct the evidence-grounded timeline, and detect temporal conflicts.
Figure 2. Overview of the DelSA framework for temporally disordered fake news detection. The input article is first decomposed into check-worthy events, and its asserted temporal relations are inferred by an LLM-based extractor. For each event, candidate evidence is retrieved from a Wikipedia-based corpus. During training, a time-aware, self-updating agent iteratively scores and refines evidence assignments using rule-guided evaluation, resolving temporal ambiguities and updating the rulebook based on downstream verification feedback. The classifier compares the article-implied chronology with the resulting evidence-grounded chronology for final veracity prediction.
Dataset
GossipCop
PolitiFact
ChronoFake
Total News
12,252
741
30,706
Fake News
4,084
391
9,925
Real News
8,168
350
20,781
Table 1. Dataset statistics.
Category
Method
PolitiFact
GossipCop
Acc.
Pre.
Rec.
F1
Acc.
Pre.
Rec.
F1
Content-Based
HAN
0.724
0.743
0.724
0.692
0.734
0.722
0.734
0.709
dEFEND
0.821
0.830
0.821
0.823
0.849
0.846
0.849
0.847
BERT
0.850
0.850
0.850
0.849
0.828
0.820
0.828
0.801
BREAK
0.867
0.867
0.867
0.867
0.831
0.817
0.831
0.808
DRES
0.911
0.913
0.911
0.917
0.874
0.882
0.874
0.867
Table 2. Performance comparison on PolitiFact and GossipCop . Best results are in bold , and second best are underlined .
Category
Method
ChronoFake
Acc.
Pre.
Rec.
F1
Content-Based
HAN
0.672
0.452
0.672
0.540
dEFEND
0.681
0.463
0.681
0.551
BERT
0.797
0.763
0.797
0.765
BREAK
0.700
0.686
0.700
0.691
DRES
0.819
0.817
0.819
0.818
Table 3. Performance comparison on ChronoFake . Best results are in bold , and second best are underlined .
Figure 3. Ablation study results of the proposed fake news detection model on three datasets.
Figure 4. Sensitivity analysis of DelSA with respect to the number of update rounds on PolitiFact, GossipCop, and ChronoFake. The x-axis denotes the maximum number of training-time evidence update rounds, and the y-axis reports F1-score.
Domain
Method
Acc.
Pre.
Rec.
F1
PolitiFact
DRES
0.821
0.827
0.821
0.797
SheepDog
0.638
0.727
0.642
0.593
MUSER
0.535
0.678
0.535
0.577
DelSA
0.789
0.756
0.789
0.772
GossipCop
DRES
0.661
0.775
0.661
0.665
SheepDog
0.520
0.604
0.520
0.522
Table 4. Performance comparison in the out-of-distribution setting. We employ a leave-one-dataset-out protocol in which the model is trained on two domains and tested on the third unseen domain shown in the first column. Best results are bolded, and second-best results are underlined.
Figure 5. A case study of DelSA on temporally disordered fake news.