Automated Fact-Checking

Ask about this topic

Overview

Automated fact-checking studies how to identify verifiable claims, find relevant evidence, and judge whether claims are supported, refuted, or unresolved. Systems combine check-worthiness classifiers, web or knowledge-graph retrieval, claim decomposition, and evidence-grounded language models; newer work also handles images, video, speech, and multilingual claims. Benchmarks such as FEVER, SciFact, and AVeriTeC test retrieval and verdict quality, but results often shift across domains, languages, and evidence conditions. A central challenge is ensuring that verdicts truly follow the cited evidence while controlling retrieval errors, uncertainty, cost, and benchmark contamination.

Latest papers

CardsList

139 papers17 papers in the last four weeks, up 6% on the four weeks before. 0.1% of all new papers.

May 3, 2026

Are LLMs More Skeptical of Entertainment News?

Large language models (LLMs) are increasingly used for automated news credibility assessment, yet it remains unclear whether they apply even-handed standards across journalistic genres. We examine whether zero-shot LLMs are more likely to misclassify legitimate entertainment news as fake than legitimate hard news, using a within-dataset design on GossipCop from FakeNewsNet. Across four frontier models, we find a clear but model-specific genre asymmetry: DeepSeek-V3.2 and GPT-5.2 show false-positive-rate gaps of 10.1 and 8.8 percentage points, respectively (both p<.001p < .001), whereas Claude Opus 4.6 and Gemini 3 Flash show no comparable difference. A style-swap experiment yields only limited and inconsistent changes, suggesting that the asymmetry is not reducible to stylistic register alone. Prompt-based mitigation is likewise possible but not generic: framing the model as an entertainment-news fact-checker reduces false positives for DeepSeek-V3.2 by about 50% without detectable recall loss, but offers little improvement for GPT-5.2. Exploratory qualitative coding further suggests two recurring error patterns in sampled false positives: treating private-life claims as inherently unverifiable and discounting entertainment journalism as an epistemically weaker genre. Taken together, these findings show that aggregate performance metrics can obscure structured false positives within legitimate journalism. We argue that LLM-based credibility assessment may not only evaluate truth claims but also differentially recognize the legitimacy of journalistic genres, and that evaluation should therefore include genre-stratified false-positive analysis alongside overall accuracy.

Huiqian Lai

School of Information Studies, Syracuse University

Fake News DetectionText ClassificationAutomated Fact-CheckingLanguage Model Bias Evaluation
AskPDF
May 2, 2026

Grounding Multi-Hop Reasoning in Structural Causal Models via Group Relative Policy Optimization

Multi-Hop Fact Verification requires complex reasoning across disparate evidence, posing significant challenges for Large Language Models , which may suffer from hallucinations and fractured logical chains. Existing methods, while improving transparency via Chain-of-Thought , often lack explicit modeling of the structural dependencies between evidence and claims. In this work, we introduce an SCM-inspired framework that grounds reasoning in explicit directed dependency graphs, treating verification as a constructive structural reasoning process rather than full causal inference with interventions or counterfactual semantics. We empirically identify an "inverted U-shaped" correlation between reasoning-chain length and accuracy, revealing that excessive structural complexity can degrade performance. To address this, we propose a rule-based reinforcement learning strategy using Group Relative Policy Optimization. This approach dynamically optimizes the trade-off between structural depth and conciseness. Extensive experiments on HoVer and EX-FEVER demonstrate that our SCM-GRPO framework outperforms strong baselines while producing more traceable reasoning structures for complex fact verification.

Yunhan Bu, Quan Zhang, Huaping Zhang +9

Beijing Institute of Technology, Beijing, China · School of Computer Science and Technology, Xinjiang University, Urumqi, China · Military Science Information Research Center, Academy of Military Science, Beijing, China

Structural Causal ModelsRL for Language Model ReasoningAutomated Fact-CheckingMulti-Hop ReasoningReasoning Verification
AskPDF
Apr 27, 2026

Propagation Structure-Semantic Transfer Learning for Robust Fake News Detection

Fake news generally refers to false information that is spread deliberately to deceive people, which has detrimental social effects. Existing fake news detection methods primarily learn the semantic features from news content or integrate structural features from propagation. However, in practical scenarios, due to the semantic ambiguity of informal language and unreliable user interactive behaviors on social media, there are inherent semantic and structural noises in news content and propagation. Although some recent works consider the effect of irrelevant user interactions in a hybrid-modeling way, they still suffer from the mutual interference between structural noise and semantic noise, leading to limited performance for robust detection. To alleviate this issue, this paper proposes a novel Propagation Structure-Semantic Transfer Learning framework (PSS-TL) for robust fake news detection under a teacher-student architecture. Specifically, we design dual teacher models to learn semantics knowledge and structure knowledge from noisy news content and propagation structure independently. Besides, we design a Multi-channel Knowledge Distillation (MKD) loss to enable the student model to acquire specialized knowledge from the teacher models, thereby avoiding mutual interference. Extensive experiments on two real-world datasets validate the effectiveness and robustness of our method.

Mengyang Chen, Lingwei Wei, Han Cao +3

Institute of Information Engineering, Chinese Academy of Sciences, Haidian District Beijing 100085, China · School of Cyber Security, University of Chinese Academy of Sciences, Huairou District Beijing 101408, China · State Key Laboratory of Communication Content Cognition, People’s Daily Online, Chaoyang District Beijing 100020, China

Fake News DetectionText ClassificationAutomated Fact-CheckingTransfer Learning
AskPDF
Apr 23, 2026

When Cow Urine Cures Constipation on YouTube: Limits of LLMs in Detecting Culture-specific Health Misinformation

Social media platforms have become primary channels for health information in the Global South. Using gomutra (cow urine) discourse on YouTube in India as a case study, we present a post-facto Large Language Model (LLM)-assisted discourse analysis of 30 multilingual transcripts showing that promotional content blends sacred traditional language with pseudo-scientific claims in ways that sophisticated debunking content itself mirrors, creating a rhetorical register that LLMs, trained predominantly on Western corpora, are systematically ill-equipped to analyse. Varying prompt tone across three LLMs (GPT-4o, Gemini 2.5 Pro, DeepSeek-V3.1), we find that culturally embedded health misinformation does not look like ordinary misinformation, and this cultural obfuscation extends to gendered rhetoric and prompt design, compounding analytical unreliability. Our findings argue that cultural competency in LLM-assisted discourse analysis cannot be retrofitted through prompt engineering alone.

Anamta Khan, Ratna Kandala, Deepti +2

University of Michigan · University of Kansas · IIT Jodhpur

LLM PromptingAutomated Fact-CheckingCultural Bias in Language ModelsLanguage Model Bias EvaluationHealth Misinformation Detection
AskPDF
Apr 23, 2026

Misinformation Span Detection in Videos via Audio Transcripts

Online misinformation is one of the most challenging issues lately, yielding severe consequences, including political polarization, attacks on democracy, and public health risks. Misinformation manifests in any platform with a large user base, including online social networks and messaging apps. It permeates all media and content forms, including images, text, audio, and video. Distinctly, video-based misinformation represents a multifaceted challenge for fact-checkers, given the ease with which individuals can record and upload videos on various video-sharing platforms. Previous research efforts investigated detecting video-based misinformation, focusing on whether a video shares misinformation or not on a video level. While this approach is useful, it only provides a limited and non-easily interpretable view of the problem given that it does not provide an additional context of when misinformation occurs within videos and what content (i.e., claims) are responsible for the video's misinformation nature. In this work, we attempt to bridge this research gap by creating two novel datasets that allow us to explore misinformation detection on videos via audio transcripts, focusing on identifying the span of videos that are responsible for the video's misinformation claim (misinformation span detection). We present two new datasets for this task. We transcribe each video's audio to text, identifying the video segment in which the misinformation claims appears, resulting in two datasets of more than 500 videos with over 2,400 segments containing annotated fact-checked claims. Then, we employ classifiers built with state-of-the-art language models, and our results show that we can identify in which part of a video there is misinformation with an F1 score of 0.68. We make publicly available our annotated datasets. We also release all transcripts, audio and videos.

Breno Matos, Rennan C. Lima, Savvas Zannettou +2

Max Planck Institute for Informatics · Federal University of Minas Gerais · Kunumi Institute +1

Automated Fact-Checking
AskPDF
Apr 21, 2026

Mask-to-Correct+^+: Leveraging Retriever Diversity for Masking-guided Faithful Fact Correction

The rapid spread of misinformation on social media highlights the need for robust, automated fact correction frameworks. However, existing works rely on supervised learning from manually annotated claim-evidence pairs, which are scarce and prone to biases, limiting their generalization across domains. Moreover, these methods overlook semantic faithfulness in their correction process. To address these challenges, we propose Mask-to-Correct (M2_2C), a training-free, inference-only Retrieval Augmented Generation (RAG) based framework that leverages diversity-aware masking to identify erroneous spans of claims and evaluate the faithfulness of corrections using retrieved evidence. However, the effectiveness of RAG heavily depends on the choice of retriever, which may vary across queries. To mitigate this, we further introduce M2_2C+^+, an ensemble-based framework that combines corrections across multiple rankers to reduce retrieval bias and improve robustness. Extensive experiments on the benchmark datasets demonstrate that our proposed frameworks consistently outperform all baselines, achieving up to 14% improvement in SARI scores, without using gold evidence.

Payel Santra, Lavisha Sharma, Madhusudan Ghosh +1

Indian Association for the Cultivation of Science, Jadavpur, Kolkata-700032, India

Retrieval-Augmented GenerationAutomated Fact-Checking
AskPDF
Apr 21, 2026

Debating the Unspoken: Role-Anchored Multi-Agent Reasoning for Half-Truth Detection

Half-truths, claims that are factually correct yet misleading due to omitted context, remain a blind spot for fact verification systems focused on explicit falsehoods. Addressing such omission-based manipulation requires reasoning not only about what is said, but also about what is left unsaid. We propose RADAR, a role-anchored multi-agent debate framework for omission-aware fact verification under realistic, noisy retrieval. RADAR assigns complementary roles to a Politician and a Scientist, who reason adversarially over shared retrieved evidence, moderated by a neutral Judge. A dual-threshold early termination controller adaptively decides when sufficient reasoning has been reached to issue a verdict. Experiments show that RADAR consistently outperforms strong single- and multi-agent baselines across datasets and backbones, improving omission detection accuracy while reducing reasoning cost. These results demonstrate that role-anchored, retrieval-grounded debate with adaptive control is an effective and scalable framework for uncovering missing context in fact verification. The code is available at https://github.com/tangyixuan/RADAR.

Yixuan Tang, Yirui Zhang, Hang Feng +1

School of Computing, National University of Singapore

Multi-Agent DebateAutomated Fact-Checking
AskPDF
Apr 18, 2026

When Misinformation Speaks and Converses: Rethinking Fact-Checking in Audio Platforms

Audio platforms have evolved beyond entertainment. They have become central to public discourse, from podcasts and radio to WhatsApp voice notes and live streams. With millions of shows and hundreds of millions of listeners, audio platforms are now a major channel for misinformation. Yet existing fact-checking pipelines are mostly designed for written claims, overlooking the unique properties of spoken media. We argue that audio misinformation is not merely textual content with transcripts: it is structurally different because it is both spoken - carrying persuasive force through prosody, pacing, and emotion - and conversational - unfolding across turns, speakers, and episodes. These dual properties introduce verification difficulties that traditional methods rarely face. This position paper synthesizes evidence across modalities and platforms, examines datasets and methods, and highlights why existing pipelines fail on audio. We argue that advancing fact-checking requires rethinking verification pipelines around the spoken and conversational realities of audio.

Chaewan Chun, Delvin Ce Zhang, Dongwon Lee

The Pennsylvania State University, USA · University of Sheffield, UK

Automated Fact-Checking
AskPDF
Apr 16, 2026

Fact4ac at the Financial Misinformation Detection Challenge Task: Reference-Free Financial Misinformation Detection via Fine-Tuning and Few-Shot Prompting of Large Language Models

The proliferation of financial misinformation poses a severe threat to market stability and investor trust, misleading market behavior and creating critical information asymmetry. Detecting such misleading narratives is inherently challenging, particularly in real-world scenarios where external evidence or supplementary references for cross-verification are strictly unavailable. This paper presents our winning methodology for the "Reference-Free Financial Misinformation Detection" shared task. Built upon the recently proposed RFC-BENCH framework (Jiang et al. 2026), this task challenges models to determine the veracity of financial claims by relying solely on internal semantic understanding and contextual consistency, rather than external fact-checking. To address this formidable evaluation setup, we propose a comprehensive framework that capitalizes on the reasoning capabilities of state-of-the-art Large Language Models (LLMs). Our approach systematically integrates in-context learning, specifically zero-shot and few-shot prompting strategies, with Parameter-Efficient Fine-Tuning (PEFT) via Low-Rank Adaptation (LoRA) to optimally align the models with the subtle linguistic cues of financial manipulation. Our proposed system demonstrated superior efficacy, successfully securing the first-place ranking on both official leaderboards. Specifically, we achieved an accuracy of 95.4% on the public test set and 96.3% on the private test set, highlighting the robustness of our method and contributing to the acceleration of context-aware misinformation detection in financial Natural Language Processing. Our models (14B and 32B) are available at https://huggingface.co/KaiNKaiho.

Cuong Hoang, Le-Minh Nguyen

Japan Advanced Institute of Science and Technology

LLM Fine-TuningAutomated Fact-CheckingFinance
AskPDF
Apr 15, 2026

BiCon-Gate: Consistency-Gated De-colloquialisation for Dialogue Fact-Checking

Automated fact-checking in dialogue involves multi-turn conversations where colloquial language is frequent yet understudied. To address this gap, we propose a conservative rewrite candidate for each response claim via staged de-colloquialisation, combining lightweight surface normalisation with scoped in-claim coreference resolution. We then introduce BiCon-Gate, a semantics-aware consistency gate that selects the rewrite candidate only when it is semantically supported by the dialogue context, otherwise falling back to the original claim. This gated selection stabilises downstream fact-checking and yields gains in both evidence retrieval and fact verification. On the DialFact benchmark, our approach improves retrieval and verification, with particularly strong gains on SUPPORTS, and outperforms competitive baselines, including a decoder-based one-shot LLM rewrite that attempts to perform all de-colloquialisation steps in a single pass.

Hyunkyung Park, Arkaitz Zubiaga

Queen Mary University of London

Automated Fact-Checking
AskPDF
Apr 15, 2026

Co-FactChecker: A Framework for Human-AI Collaborative Claim Verification Using Large Reasoning Models

Professional fact-checkers rely on domain knowledge and deep contextual understanding to verify claims. Large language models (LLMs) and large reasoning models (LRMs) lack such grounding and primarily reason from available evidence alone, creating a mismatch between expert-led and fully automated claim verification. To mitigate this gap, we posit human-AI collaboration as a more promising path forward, where expert feedback, grounded in real-world knowledge and domain expertise, guides the model's reasoning. However, existing LRMs are hard to calibrate to natural language feedback, particularly in a multi-turn interaction setup. We propose Co-FactChecker, a framework for human-AI collaborative claim verification. We introduce a new interaction paradigm that treats the model's thinking trace as a shared scratchpad. Co-FactChecker translates expert feedback into trace-edits that introduce targeted modifications to the trace, sidestepping the shortcomings of dialogue-based interaction. We provide theoretical results showing that trace-editing offers advantages over multi-turn dialogue, and our automatic evaluations demonstrate that Co-FactChecker outperforms existing autonomous and human-AI collaboration approaches. Human evaluations further show that Co-FactChecker is preferred over multi-turn dialogue, producing higher quality reasoning and verdicts along with relatively easier to interpret and more useful thinking traces.

Dhruv Sahnan, Subhabrata Dutta, Tanmoy Chakraborty +2

MBZUAI, UAE · TU Darmstadt, Germany · IIT Delhi, India

Automated Fact-CheckingHuman-AI CollaborationLarge Reasoning Models
AskPDF
Apr 15, 2026

Calibrated Selective Fact-Checking via Evidence Chain Evaluation

Large language models (LLMs) can achieve strong fact-checking accuracy, yet forced binary decisions conceal a critical reliability problem: systems may issue confident verdicts even when supporting evidence is weak, sparse, or internally inconsistent. We address this issue through Evidence Chain Evaluation (ECE), a selective fact-checking framework that permits abstention via an uncertain verdict instead of requiring a true/false decision for every claim. The evaluated system is a tool-using verification agent that gathers evidence through web search, scholarly search, and executable checks, and then returns a structured verdict with confidence and source-level metadata. On ECE-Bench, ECE achieves 91.6% standard accuracy, 93.7% coverage, and 97.8% selective accuracy on answered claims. Although ECE does not outperform the strongest retrieval baseline on aggregate calibration metrics such as Expected Calibration Error, Brier score, or AURC, it delivers a clear selective-prediction trade-off: the system maintains very high accuracy on answered claims while deferring 6 of 95 cases. These deferred cases are concentrated in lower-reliability evidence settings (5/6 at source level L4), supporting the view that abstention functions as a safety-oriented mechanism for handling epistemically weak evidence. Code is available at https://github.com/ cheshireyang/ECE.git

Dekun Yang

Zhejiang University

Automated Fact-CheckingSelective PredictionConfidence Estimation in Language Models
AskPDF
Apr 14, 2026

TRUST Agents: A Collaborative Multi-Agent Framework for Fake News Detection, Explainable Verification, and Logic-Aware Claim Reasoning

TRUST Agents is a collaborative multi-agent framework for explainable fact verification and fake news detection. Rather than treating verification as a simple true-or-false classification task, the system identifies verifiable claims, retrieves relevant evidence, compares claims against that evidence, reasons under uncertainty, and generates explanations that humans can inspect. The baseline pipeline consists of four specialized agents. A claim extractor uses named entity recognition, dependency parsing, and LLM-based extraction to identify factual claims. A retrieval agent performs hybrid sparse and dense search using BM25 and FAISS. A verifier agent compares claims with retrieved evidence and produces verdicts with calibrated confidence. An explainer agent then generates a human-readable report with explicit evidence citations. To handle complex claims more effectively, we introduce a research-oriented extension with three additional components: a decomposer agent inspired by LoCal-style claim decomposition, a Delphi-inspired multi-agent jury with specialized verifier personas, and a logic aggregator that combines atomic verdicts using conjunction, disjunction, negation, and implication. We evaluate both pipelines on the LIAR benchmark against fine-tuned BERT, fine-tuned RoBERTa, and a zero-shot LLM baseline. Although supervised encoders remain stronger on raw metrics, TRUST Agents improves interpretability, evidence transparency, and reasoning over compound claims. Results also show that retrieval quality and uncertainty calibration remain the main bottlenecks in trustworthy automated fact verification.

Gautama Shastry Bulusu Venkata, Santhosh Kakarla, Maheedhar Omtri Mohan +1

George Mason University

Fake News DetectionAutomated Fact-CheckingLogical Reasoning
AskPDF
Apr 13, 2026

Uncertainty-Aware Web-Conditioned Scientific Fact-Checking

Scientific fact-checking is vital for assessing claims in specialized domains such as biomedicine and materials science, yet existing systems often hallucinate or apply inconsistent reasoning, especially when verifying technical, compositional claims against an evidence snippet under source and cost/latency constraints. We present a pipeline centered on atomic predicate-argument decomposition and calibrated, uncertainty-gated corroboration: atomic facts are aligned to local snippets via embeddings, verified by a compact evidence-grounded checker, and only facts with uncertain support trigger domain-restricted web search over authoritative sources. The system supports both binary and tri-valued classification where it predicts labels from Supported, Refuted, NEI for three-way tasks. We evaluate under two regimes, Context-Only (no web) and Context+Web (uncertainty-gated web corroboration); when retrieved evidence conflicts with the provided context, we abstain with NEI rather than overriding the context. On multiple benchmarks, our framework surpasses the strongest benchmarks. In our experiments, web corroboration was invoked for only a minority of atomic facts on average, indicating that external evidence is consulted selectively under calibrated uncertainty rather than routinely. Overall, coupling atomic granularity with calibrated, uncertainty-gated corroboration yields more interpretable and context-conditioned verification, making the approach well-suited to high-stakes, single-document settings that demand traceable rationales, predictable cost/latency, and conservative.

Ashwin Vinod, Katrin Erk

University of Texas at Austin · University of Massachusetts Amherst

Automated Fact-CheckingScientific Claim Verification
AskPDF
Apr 13, 2026

CFMS: A Coarse-to-Fine Multimodal Synthesis Framework for Enhanced Tabular Reasoning

Reasoning over tabular data is a crucial capability for tasks like question answering and fact verification, as it requires models to comprehend both free-form questions and semi-structured tables. However, while methods like Chain-of-Thought (CoT) introduce reasoning chains, purely symbolic methodes are inherently limited by their blindness to holistic visual patterns. To address this, we propose the Coarse-to-Fine Multimodal Synthesis framework (CFMS), a novel two-stage paradigm that hierarchically decouples high-level visual perception from granular symbolic reasoning. In the Coarse Stage, CFMS leverages the Multimodal Large Language Models (MLLMs) to perform a one-time synthesis of a multi-perspective knowledge tuple. This tuple subsequently serves as a dynamic reasoning map to guide the fine stage, where a symbolic engine executes a targeted and efficient sequence of iterative operations over the table. Extensive experiments on the WikiTQ and TabFact benchmarks demonstrate that CFMS achieves competitive accuracy. The framework exhibits particular robustness when handling large tables and when instantiated with smaller backbone models, validating its effectiveness and generalizability.

Qixian Huang, Hongqiang Lin, Tong Fu +5

Table QAMultimodal ReasoningSymbolic ReasoningAutomated Fact-Checking
AskPDF
Apr 9, 2026

AtomEval: Validity-Aware Atomic Evaluation of Adversarial Claim Rewriting in Fact Verification

Large language models (LLMs) can rewrite refuted claims to evade evidence-based fact verifiers, but conventional attack success rate (ASR) can be inflated when rewrites change, weaken, or correct the false proposition they are supposed to preserve. We introduce AtomEval, a validity-aware evaluation protocol for fixed-evidence adversarial claim rewriting. AtomEval represents claims as subject--relation--object--modifier (SROM) atoms, applies a one-way preservation gate to separate valid verifier evasion from proposition-changing rewrites, and reports validity-aware attack success rate (VASR), which counts only verifier-evasive rewrites that preserve the original false proposition. AtomEval further provides fine-grained diagnostics that explain both proposition-level failures and non-minimal valid rewrites. On FEVER refuted-claim rewriting, AtomEval exposes and explains ASR inflation: many apparent attacks fool the verifier by altering, weakening, or correcting the proposition they should preserve. By making attacked-proposition preservation explicit and measurable, AtomEval provides a stable evaluation target for evaluating adversarial rewriters that must balance verifier evasion with proposition preservation.

Hongyi Cen, Mingxin Wang, Yule Liu +3

Zhejiang University · The Hong Kong University of Science and Technology (Guangzhou)

Adversarial Text AttacksAutomated Fact-Checking
AskPDF
Apr 8, 2026

What Does a Sharing Question Add? Auditing LLM Survey Scores for Misinformation

Evaluating misinformation requires distinguishing whether readers believe content from whether they would share it. Asking large language models (LLMs) both questions yields two scores, but does the sharing answer contribute information beyond the credibility answer? We audit eight model versions on 290 synthetic misinformation articles, using 1,256 paired survey responses with 317 participant identifiers as an external validity criterion. An initial reversal motivates the audit: every model's raw sharing score predicts mean human sharing less accurately than its credibility score. This ordering changes after offset correction, so it does not by itself diagnose missing information. We instead distinguish score reconstructability, persistence across elicitation formats, and incremental human validity. Credibility predicts 30.8-72.5% of model-sharing variation relative to a held-out constant baseline; remaining sharing differences correlate at 0.61-0.75 across question-order and separate-question conditions. Yet adding model sharing to human and model credibility yields only -0.25% to +0.69% error reduction with fixed regression, with all exploratory intervals crossing zero. Flexible prediction and format changes do not establish an improvement. Sharing answers therefore contain structured variation beyond the observed credibility score, without established incremental validity for human sharing in these data. The findings motivate validating the contribution of each elicited outcome, beyond inspecting score differences or agreement across prompts.

Zonghuan Xu, Xiang Zheng, Yutao Wu +1

Shanghai Key Laboratory of Multimodal Embodied AI, Shanghai, China · City University of Hong Kong, Hong Kong SAR, China · Deakin University, Australia

LLM EvaluationLLM AuditingAutomated Fact-Checking
AskPDF
Apr 8, 2026

A Graph-Enhanced Defense Framework for Explainable Fake News Detection with LLM

Explainable fake news detection aims to assess the veracity of news claims while providing human-friendly explanations. Existing methods incorporating investigative journalism are often inefficient and struggle with breaking news. Recent advances in large language models (LLMs) enable leveraging externally retrieved reports as evidence for detection and explanation generation, but unverified reports may introduce inaccuracies. Moreover, effective explainable fake news detection should provide a comprehensible explanation for all aspects of a claim to assist the public in verifying its accuracy. To address these challenges, we propose a graph-enhanced defense framework (G-Defense) that provides fine-grained explanations based solely on unverified reports. Specifically, we construct a claim-centered graph by decomposing the news claim into several sub-claims and modeling their dependency relationships. For each sub-claim, we use the retrieval-augmented generation (RAG) technique to retrieve salient evidence and generate competing explanations. We then introduce a defense-like inference module based on the graph to assess the overall veracity. Finally, we prompt an LLM to generate an intuitive explanation graph. Experimental results demonstrate that G-Defense achieves state-of-the-art performance in both veracity detection and the quality of its explanations.

Bo Wang, Jing Ma, Hongzhan Lin +4

School of Artificial Intelligence, Jilin University, China · Hong Kong Baptist University, China · Guangdong Institute of Smart Education, Jinan University, China +3

Fake News DetectionAutomated Fact-Checking
AskPDF
Apr 7, 2026

DIA-HARM: Dialectal Disparities in Harmful Content Detection Across 50 English Dialects

Harmful content detectors, particularly disinformation classifiers, are predominantly developed and evaluated on Standard American English (SAE), leaving their robustness to dialectal variation unexplored. We present DIA-HARM, the first benchmark for evaluating disinformation detection robustness across 50 English dialects spanning U.S., British, African, Caribbean, and Asia-Pacific varieties. Using Multi-VALUE's linguistically grounded transformations, we introduce D-CUBE (Dialectal Disinformation Detection Corpus), a core corpus component of DIA-HARM comprising 195K samples derived from established disinformation benchmarks. Our evaluation of 16 detection models reveals systematic vulnerabilities: human-written dialectal content degrades detection by 1.4-3.6% F1, while AI-generated content remains stable. Fine-tuned transformers substantially outperform zero-shot LLMs (96.6% vs. 78.3% best-case F1), with some models exhibiting catastrophic failures exceeding 33% degradation on mixed content. Cross-dialectal transfer analysis across 2,450 dialect pairs shows that multilingual models (mDeBERTa: 97.2% average F1) generalize effectively, while monolingual models like RoBERTa and XLM-RoBERTa fail on dialectal inputs. These findings demonstrate that current disinformation detectors may systematically disadvantage hundreds of millions of non-SAE speakers worldwide. We release the DIA-HARM benchmark, including the D-CUBE corpus (https://github.com/jsl5710/dia-harm), and evaluation tools (https://jsl5710.github.io/dia-harm).

Jason Lucas, Matt Murtagh, Ali Al-Lawati +3

The Pennsylvania State University, USA · Trinity College Dublin, Ireland · MIT Lincoln Laboratory, USA

Fake News DetectionDialect Bias in Language ModelsAutomated Fact-CheckingLanguage Model Robustness
AskPDF
Apr 6, 2026

LiveFact: A Dynamic, Time-Aware Benchmark for LLM-Driven Fake News Detection

The rapid development of Large Language Models (LLMs) has transformed fake news detection and fact-checking tasks from simple classification to complex reasoning. However, evaluation frameworks have not kept pace. Current benchmarks are static, making them vulnerable to benchmark data contamination (BDC) and ineffective at assessing reasoning under temporal uncertainty. To address this, we introduce LiveFact a continuously updated benchmark that simulates the real-world "fog of war" in misinformation detection. LiveFact uses dynamic, temporal evidence sets to evaluate models on their ability to reason with evolving, incomplete information rather than on memorized knowledge. We propose a dual-mode evaluation: Classification Mode for final verification and Inference Mode for evidence-based reasoning, along with a component to monitor BDC explicitly. Tests with 22 LLMs show that open-source Mixture-of-Experts models, such as Qwen3-235B-A22B, now match or outperform proprietary state-of-the-art systems. More importantly, our analysis finds a significant "reasoning gap." Capable models exhibit epistemic humility by recognizing unverifiable claims in early data slices-an aspect traditional static benchmarks overlook. LiveFact sets a sustainable standard for evaluating robust, temporally aware AI verification.

Cheng Xu, Changhong Jin, Yingjie Niu +5

University College Dublin · Bebxy · Georgia Institute of Technology +1

LLM EvaluationFake News DetectionAutomated Fact-CheckingTemporal Reasoning in Language ModelsData Contamination in Language Models
AskPDF
Apr 6, 2026

Is a Picture Worth a Thousand Words? Adaptive Multimodal Fact-Checking with Visual Evidence Necessity

Automated fact-checking is a crucial task that supports a responsible information ecosystem. While recent research has progressed from text-only to multimodal fact-checking, a prevailing assumption is that incorporating visual evidence universally improves verification accuracy. In this work, we challenge this assumption and show that the indiscriminate use of visual evidence can reduce accuracy. Building on this finding, we propose AMuFC, a modular fact-checking framework that employs two collaborative vision-language models with distinct roles to enable the adaptive use of visual evidence. Experimental results on three datasets, including WebFC, introduced in this study, demonstrate the effectiveness of adaptive visual evidence use in fact-checking.

Jaeyoon Jung, Yejun Yoon, Kunwoo Park

School of AI Convergence, Soongsil University · MAUM AI Inc. · Department of Intelligent Semiconductors, Soongsil University

Automated Fact-CheckingVision-Language Models
AskPDF
Apr 5, 2026

Schema-Aware Planning and Hybrid Knowledge Toolset for Reliable Knowledge Graph Triple Verification

Knowledge Graphs (KGs) provide a critical foundation for many AI systems, yet their automated construction inevitably introduces noisy triples that undermine data trustworthiness. Reliable KG auditing requires jointly examining internal structural constraints and open-world evidence, whereas existing methods often rely on a single evidence source or static, one-shot inference, limiting adaptive evidence acquisition and auditability. We propose SHARP (Schema-Hybrid Agent for Reliable Prediction), a training-free agent that reformulates triple verification as a dynamic process of strategic planning, active investigation, and evidential reasoning. SHARP uses retrieved historical verification trajectories and entity--relation schema information to construct a task-specific investigation plan. It then iteratively coordinates KG and textual tools to cross-validate structural and semantic evidence, producing a verification verdict with a transparent evidence chain. On independently sampled, balanced test sets constructed from FB15K-237 and Wikidata5M-Ind, SHARP achieves average accuracies of 87.1% and 93.6%, respectively. Controlled comparisons with standard RAG using the same evidence sources, together with ablation studies, further show that the gains arise not merely from external retrieval, but from the combined effects of schema-aware planning, memory augmentation, and iterative hybrid tool use.

Xinyan Ma, Xianhao Ou, Chang Li +6

Harbin Institute of Technology, China · Huawei Technologies Co., Ltd., China

Knowledge GraphsAutomated Fact-CheckingLLM Reasoning with GraphsKG Construction
AskPDF
Apr 5, 2026

Position: Logical Soundness is not a Reliable Criterion for Neurosymbolic Fact-Checking with LLMs

As large language models (LLMs) are increasing integrated into fact-checking pipelines, formal logic is often proposed as a rigorous means by which to mitigate bias, errors and hallucinations in these models' outputs. For example, some neurosymbolic systems verify claims by using LLMs to translate natural language into logical formulae and then checking whether the proposed claims are logically sound, i.e. whether they can be validly derived from premises that are verified to be true. We argue that such approaches structurally fail to detect misleading claims due to systematic divergences between conclusions that are logically sound and inferences that humans typically make and accept. Drawing on studies in cognitive science and pragmatics, we present a typology of cases in which logically sound conclusions systematically elicit human inferences that are unsupported by the underlying premises. Consequently, we advocate for a complementary approach: leveraging human-like reasoning tendencies of LLMs as a feature rather than a bug, and using these models to validate the outputs of formal components in neurosymbolic systems against potentially misleading conclusions.

Jason Chan, Robert Gaizauskas, Zhixue Zhao

School of Computer Science, University of Sheffield, United Kingdom

Automated Fact-CheckingNeuro-Symbolic ReasoningLLM ReasoningPragmatic Reasoning in Language Models
AskPDF
Apr 5, 2026

FactReview: Evidence-Grounded Peer Review with Execution-Based Claim Verification

AI is accelerating scientific research, but providing timely and trustworthy feedback remains challenging. Such feedback requires evaluating specific claims against verified evidence, helping researchers and autonomous research systems determine which findings can guide subsequent work. Thorough verification is time-consuming, and reviewers under time pressure may overlook or misjudge critical issues. Existing AI reviewers have yet to fill this gap because they often assess claims without adequately verifying the supporting evidence. We introduce FactReview, a framework that verifies claims in papers to provide more trustworthy feedback. It checks claims against the literature and examines derivations, while also executing the paper's code to test empirical claims. Across 300 papers, FactReview ranks best among all evaluated systems on five of seven metrics, achieving 6.8% false support and 79.2% classification accuracy. In a study of 36 papers, reviewers using FactReview detect 9.4 issues per paper compared with 5.6 independently, while spending 74.1% less time. FactReview thus supports more reliable peer review, with the potential to accelerate research iteration. The code is public at https://github.com/DEFENSE-SEU/FactReview.

Ling Yue, Chaoqian Ouyang, Ruijun Huang +8

Rensselaer Polytechnic Institute · Sun Yat-sen University · Southeast University +1

Automated Peer ReviewScientific Claim VerificationAutomated Fact-Checking
AskPDF
Apr 3, 2026

AutoVerifier: An Agentic Automated Verification Framework Using Large Language Models

Scientific and Technical Intelligence (S&TI) analysis requires verifying complex technical claims across rapidly growing literature, where existing approaches fail to bridge the verification gap between surface-level accuracy and deeper methodological validity. We present AutoVerifier, an LLM-based agentic framework that automates end-to-end verification of technical claims without requiring domain expertise. AutoVerifier decomposes every technical assertion into structured claim triples of the form (Subject, Predicate, Object), constructing knowledge graphs that enable structured reasoning across six progressively enriching layers: corpus construction and ingestion, entity and claim extraction, intra-document verification, cross-source verification, external signal corroboration, and final hypothesis matrix generation. We demonstrate AutoVerifier on a contested quantum computing claim, where the framework, operated by analysts with no quantum expertise, automatically identified overclaims and metric inconsistencies within the target paper, traced cross-source contradictions, uncovered undisclosed commercial conflicts of interest, and produced a final assessment. These results show that structured LLM verification can reliably evaluate the validity and maturity of emerging technologies, turning raw technical documents into traceable, evidence-backed intelligence assessments.

Yuntao Du, Minh Dinh, Kaiyuan Zhang +1

Purdue University

Claim VerificationScientific Claim VerificationLLM Agent VerificationAutomated Fact-Checking
AskPDF
Apr 2, 2026

ThinknCheck: Grounded Claim Verification with Compact, Reasoning-Driven, and Interpretable Models

We present ThinknCheck, a 1B-parameter verifier for grounded claim verification that first produces a short, structured rationale and then a binary verdict. We construct LLMAggreFact-Think, a 24.1k reasoning-augmented training set derived from LLMAggreFact, and fine-tune a 4-bit Gemma3 model to follow this format. On LLMAggreFact, ThinknCheck attains 78.1 balanced accuracy (BAcc), surpassing MiniCheck-7B (77.4) with 7x fewer parameters; removing the reasoning step reduces BAcc to 57.5. On SciFact, ThinknCheck reaches 64.7 BAcc, a +14.7 absolute gain over MiniCheck-7B. By contrast, zero-shot chain-of-thought on the base Gemma3-1B harms accuracy relative to direct answers, and preference optimization with a simple format+accuracy reward underperforms supervised reasoning. To probe the latter, we introduce GSMClaims and a domain-specialized variant, ThinknCheck-Science, which improves across benchmarks, including 61.0% accuracy on GSMClaims. Overall, explicit, supervised reasoning enables compact verifiers that are competitive while remaining resource-efficient and interpretable.

Delip Rao, Feijiang Han, Chris Callison-Burch

University of Pennsylvania, Philadelphia, PA, USA

Claim VerificationLLM Answer VerificationAutomated Fact-CheckingEfficient Language Model Reasoning
AskPDF
Apr 1, 2026

AfrIFact: Cultural Information Retrieval, Evidence Extraction and Fact Checking for African Languages

Assessing the veracity of a claim made online is a complex and important task with real-world implications. When these claims are directed at communities with limited access to information and the content concerns issues such as healthcare and culture, the consequences intensify, especially in low-resource languages. In this work, we introduce AfrIFact, a dataset that covers the necessary steps for automatic fact-checking (i.e., information retrieval, evidence extraction, and fact checking), in ten African languages and English. Our evaluation results show that even the best embedding models lack cross-lingual retrieval capabilities, and that cultural and news documents are easier to retrieve than healthcare-domain documents, both in large corpora and in single documents. We show that LLMs lack robust multilingual fact-verification capabilities in African languages, while few-shot prompting improves performance by up to 43% in AfriqueQwen-14B, and task-specific fine-tuning further improves fact-checking accuracy by up to 26%. These findings, along with our release of the AfrIFact dataset, encourage work on low-resource information retrieval, evidence retrieval, and fact checking.

Israel Abebe Azime, Jesujoba Oluwadara Alabi, Crystina Zhang +16

Saarland University, Saarland Informatic Campus, Germany · Masakhane NLP · University of Waterloo, Canada +11

Automated Fact-CheckingEvidence RetrievalLow-Resource Language ProcessingMultilingual IRAfrican Language ProcessingMultilingual Language Model Evaluation
AskPDF
Mar 31, 2026

ContextClaim: A Context-Driven Paradigm for Verifiable Claim Detection

Automated fact-checking pipelines typically begin with a filtering stage that decides which claims are worth verifying, given that the later evidence retrieval and verification components are expensive to apply at scale. A central task in this stage is verifiable claim detection, which asks whether a statement is in principle checkable against external evidence. Prior work on this task, as well as on the closely related notion of check-worthiness, conditions its decisions only on the claim sentence itself. We argue that this is restrictive, because deciding whether a statement is checkable often depends on identifying the entities and events it mentions, and on whether external information about them is actually available in the first place. Motivated by how downstream verification systems rely on retrieved evidence, we move retrieval upstream into the detection stage and introduce ContextClaim. Given an input claim, the approach identifies entity mentions, queries Wikipedia as a structured background source, and uses large language models to compress the retrieved material into short contextual summaries that are then passed to a classifier. Experiments are conducted on two domains and genres, namely the CheckThat! 2022 Twitter collection and the PoliClaim corpus of political debates, and cover both encoder and decoder only models under fine-tuning, zero-shot, and few-shot settings. The added context yields gains on verifiable claim detection in several configurations, although the size of the improvement varies with the dataset, the backbone model, and the training setup. We further find that the same retrieved summaries are useful beyond detection. Feeding them into a downstream verification model on FEVER improves verification F1. Component level analyses, human annotation, and error inspection further clarify the conditions under which retrieved context helps, and where it does not.

Yufeng Li, Rrubaa Panchendrarajan, Arkaitz Zubiaga

School of Electronic Engineering and Computer Science, Queen Mary University of London, London E1 4NS, U.K.

Automated Fact-CheckingClaim Verification
AskPDF
Mar 27, 2026

ClimateCheck 2026: Scientific Fact-Checking and Disinformation Narrative Classification of Climate-related Claims

Automatically verifying climate-related claims against scientific literature is a challenging task, complicated by the specialised nature of scholarly evidence and the diversity of rhetorical strategies underlying climate disinformation. ClimateCheck 2026 is the second iteration of a shared task addressing this challenge, expanding on the 2025 edition with tripled training data and a new disinformation narrative classification task. Running from January to February 2026 on the CodaBench platform, the competition attracted 20 registered participants and 8 leaderboard submissions, with systems combining dense retrieval pipelines, cross-encoder ensembles, and large language models with structured hierarchical reasoning. In addition to standard evaluation metrics (Recall@K and Binary Preference), we adapt an automated framework to assess retrieval quality under incomplete annotations, exposing systematic biases in how conventional metrics rank systems. A cross-task analysis further reveals that not all climate disinformation is equally verifiable, potentially implicating how future fact-checking systems should be designed.

Raia Abu Ahmad, Max Upravitelev, Aida Usmanova +2

Deutsches Forschungszentrum für Künstliche Intelligenz GmbH (DFKI), Germany · Technische Universität Berlin, Germany · Leuphana Universität Lüneburg, Germany +1

Scientific Claim VerificationAutomated Fact-CheckingFake News DetectionIR Evaluation
AskPDF
Mar 25, 2026

WhatsApp Vaccine Discourse (WhaVax): An Expert-Annotated Dataset and Benchmark for Health Misinformation Detection

We introduce WhaVax, a new expert-annotated dataset of vaccine-related WhatsApp messages collected from large Brazilian public groups spanning multiple pandemic years. The dataset was constructed through a rigorous, carefully designed pipeline that integrates keyword-based data collection, semantic deduplication to remove near-duplicate content, and a multi-stage annotation protocol conducted by medical specialists. This process produced a high-quality gold-standard corpus, characterized by substantial inter-annotator agreement and strong reliability for downstream analysis. Additionally, we provide a detailed characterization of WhatsApp misinformation, revealing distinctive linguistic, structural, lexical, temporal, and group-level patterns, as well as a meaningful layer of ambiguous cases that reflect the complexity of health discourse in private messaging. We also benchmark classical models, fine-tuned Small Language Models, and zero- or few-shot Large Language Models under realistic data-scarcity constraints, demonstrating that strong embeddings and LLM approaches perform competitively, while domain alignment and data availability remain critical factors. This study provides a rare, high-quality resource to support misinformation research and computational modeling in encrypted communication environments.

Jônatas H. dos Santos, Julio C. S. Reis, Philipe Melo +9

Universidade Federal de Minas Gerais (UFMG), Brasil · Universidade Federal de Viçosa (UFV), Brasil

Health Misinformation DetectionFake News DetectionAutomated Fact-Checking
AskPDF