Claim Verification

Follow this topic Ask about this topic

Claim verification checks statements against documents, databases, tables, images, audio, or other evidence, supporting fact-checking, scientific review, and safer AI-generated answers. Common systems retrieve and rank evidence, break complex statements into smaller claims, then apply natural language inference, language-model reasoning, or structured calculations to judge support, contradiction, or insufficient evidence. Current approaches also use multiple sources or agents, explicit evidence traces, calibration, and abstention to handle conflicts and limit cost or risk. A central challenge is ensuring that benchmark gains reflect robust reasoning rather than dataset shortcuts, while deciding how much evidence is sufficient and when a system should defer to people.

Latest papers

59 papers8 papers in the last four weeks, down 33% on the four weeks before. 0.0% of all new papers.

Rethinking the Evaluation of LLM Fact-Checkers
Oct 21, 2026Ayana Niwa, Miyu Yamada, Kouta Nakayama +3LLM Evaluation Automated Fact-Checking
MBZUAI · Science Tokyo · NII-LLMC
AnomalyJev: Towards Evaluating Image Anomaly Explanations via Claim-Level Verification
Oct 20, 2026Qizhou Wang, Jiawen Zhu, Guansong Pang +1Explainability Evaluation Interpretable Anomaly Detection
The University of Melbourne · Singapore Management University
One-Sided Verification Under Limited Expert Review
Oct 13, 2026Fan Nie, Haotian Ye, Shurui Liu +3Reasoning Verification Human-in-the-Loop Evaluation
Stanford University
MiniVer-V: Identifying Minimal Sufficient Evidence for Short Video Verification
Oct 8, 2026Leran Chen, Lingnan Kong, Zile CaiClaim Verification Automated Fact-Checking
The University of Sydney
More Claims, Less Evidence: Bounded Verification of AI-Generated Digital Knowledge Artifacts
Oct 4, 2026Feliks Bańka, Jarosław A. ChudziakClaim Verification Automated Fact-Checking
Institute of Computer Science, Warsaw University of Technology, Warsaw, Poland
How Robust Is Multimodal Claim Verification to LLM Rewriting?
Oct 2, 2026Yun-Ang Wu, Xanh Ho, Andre Greiner-Petter +4Language Model Robustness VLM Robustness
NII LLMC, Japan
Towards Robust Numerical Claim Verification
Sep 30, 2026Peter Røysland Aarnes, Vinay SettyNumerical Reasoning in Language Models Claim Verification
University of Stavanger · Factiverse AI and University of Stavanger
Actions with Receipts: Jointly Binding Claims, Evidence, and Execution for Replayable Tool-Agent Auditing
Sep 29, 2026Miaobo Hu, Shuhao Hu, Xiaobo Guo +4AI Agent Auditing Citation Verification
School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China · Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China
Counterfactual Self-Evolving Agents for Evidence-Grounded Reasoning
Sep 26, 2026Xing Han, Yuxin Wang, Chen Chen +8Self-Evolving Agents Evidence-Grounded Reasoning
Johns Hopkins University · MIT · University of Texas at Austin +1
AcoustiClaim: A Numeric Claim Benchmark with Instrument Ground Truth
Sep 24, 2026Sheng-Tse Lin, Siyuan Zhai, Chien-Liang Kuo +2Audio-Language Model Evaluation Claim Verification
Carnegie Mellon University
UNMATCH: Selective Unbalanced Token-Patch Matching for Forensic Image-Claim Verification
Sep 24, 2026Xinjin Li, Lian Lian, Yuanzhe Yang +6Claim Verification Digital Image Forensics
Columbia University New York, United States · Rice University Houston, United States
Impact Is Not Invalidation: Ask About the Claim, Not the Diff
Sep 20, 2026Atul AnandClaim Verification Semantic Change Detection
Thomson Reuters Bangalore, India
Euston: Training Away Mathematical Sycophancy Without Losing the Mathematics
Sep 19, 2026Zehua Cheng, Wei Dai, Jiahao SunClaim Verification LLM Sycophancy
University of Oxford
GeoContext: One Context Ladder, Two Failure Modes in Vision-Language Geolocation: Flat Reliance on User-Provided Location Context and False Confirmation of Location Claims
Sep 4, 2026Yifan Zhang, Kai WangGeospatial Reasoning Benchmarks VLM Evaluation
Beyond "Made with AI": Visualizing Provenance Density to Mitigate the Transparency Penalty
Sep 3, 2026Qing Zhang, Yifei Huang, Juyoung Lee +2Claim Verification Trust in AI
The University of Tokyo · Institute of Industrial Science, The University of Tokyo · Korea Advanced Institute of Science and Technology +2
Peerify: Benchmarking Peer-Review Claim Verification
Sep 2, 2026Alireza Daghighfarsoodeh, Sajad Ebrahimi, Ali Ghorbanpour +4Claim Verification Scholarly Peer Review
Reviewerly · University of Toronto · University of California, Berkeley
ClaimReceipt: Verifying Evidence Sufficiency and Coverage in Agent Evaluations
Sep 2, 2026Peiying Zhu, Sidi ChangClaim Verification Data Provenance
Blossom AI Blossom AI Labs San Francisco, CA, USA
InSight: A Benchmark for Agentic Claim Verification in Interactive Visualizations
Sep 1, 2026Maeve Hutchinson, Syed Mahbubul Huq, Mohammad Albinhassan +3VLM Evaluation Claim Verification
City, University of London · Imperial College London · The Alan Turing Institute
CoVer: Conflict-Aware Claim Verification
Sep 1, 2026Shuning Zhang, Dai Shi, Bohao Chu +7Claim Verification Automated Fact-Checking
Tsinghua University · Tongji University · University of Duisburg-Essen +5
Calibrating Small Language Models for Claim Check-Worthiness Detection
Aug 31, 2026Pratuat Amatya, V Venktesh, Vinay SettyClaim Verification Small Language Models
1Factiverse AS · University of Stavanger · 3Stockholm University
MAP: A Benchmark on Multimodal Accessibility Planning for Real World Places
Aug 28, 2026Jason Armitage, Ioannis Tsochantaridis, Linda Mazzone +3Multimodal Model Evaluation Claim Verification
University of Zurich Switzerland · Google DeepMind
HNR-DAC: Hard-Negative Reranking and Distribution-Aligned Classification for Scientific Claim Verification
Aug 7, 2026Zhenchao Wang, Xin Chen, Luoxi Zhang +2Claim Verification Distribution Shift
Southern University of Science and Technology, Shenzhen, China · Institute of Artificial Intelligence, Shenzhen University of Advanced Technology, Shenzhen, China · Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
Decomposed Entailment for Factuality Checking and Hallucination Detection
Aug 6, 2026Achir Oukelmoun, Nasredine Semmar, Gaël De ChalendarLLM Grounding Claim Verification
CEA LIST NANO INNOV 2 Bd Thomas Gobert, 91120 Palaiseau, France
Evidence-Ledger Adjudication for Claim-Evidence Traceability
Jul 29, 2026Gengyu Chen, Yongjie Yu, Weiling WangClaim Verification Citation Verification
Carnegie Mellon University · Syracuse University
DS@GT ARC at CheckThat! 2026: LLM-Based Trace Ranking and Grouped Reward Modeling for Multilingual Numerical Claim Verification
Jul 27, 2026Sagnik Sinha, Shreyas ShresthaNumerical Reasoning in Language Models Reward Modeling
Georgia Institute of Technology, North Ave, Atlanta, GA 30332, United States
DualityCert: Verifier-Gated Language-Model Repair of Broken Duality Claims in Quantum Field Theory
Jul 26, 2026Xingyang YuLLM Self-Correction Claim Verification
Department of Physics Virginia Tech Blacksburg, VA 24061, USA
Quality Action Assurance: Multimodal Verification of Examiner Claims in VR OSCEs
Jul 21, 2026Harry Rogers, Sally Shiels, Ashley Tomlinson +5Claim Verification Action Quality Assessment
Department of Engineering Science, University of Oxford, Oxford, UK · Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK · Department for Continuing Education, University of Oxford, Oxford, UK
How Human Feedback Shapes AI-generated Community Notes
Jun 29, 2026Soham De, Isaac Slaughter, Jiawei Guo +4Claim Verification Automated Fact-Checking
University of Washington
Multi-Agentic System Leveraging Open-Source LLMs to Mitigate Disinformation Threats
Jun 29, 2026Sebastian Kula, Martin TamajkaMulti-Agent LLM Systems Fake News Detection
Kempelen Institute of Intelligent Technologies, Bratislava, Slovakia · West Pomeranian University of Technology in Szczecin, Szczecin, Poland
Small LLMs for Biomedical Claim Verification: Cost-Effective Fine-Tuning, Structural Dataset Shortcuts, and Cross-Domain Generalization
Jun 11, 2026Gaurav KumarClaim Verification LLM Fine-Tuning
Moveworks AI · University of California San Diego

Oct 21, 2026J/K move · Enter open · S save

Rethinking the Evaluation of LLM Fact-Checkers

Ayana Niwa, Miyu Yamada, Kouta Nakayama, Masahiro Kaneko+2

MBZUAI · Science Tokyo · NII-LLMC

LLM Evaluation Automated Fact-Checking Claim Verification Fake News Detection

Open paper AskPDF arXiv

Large language models (LLMs) are widely used to answer factual questions, but their answers can contain errors and must be fact-checked claim by claim. Yet the resulting score depends on three design decisions: which questions the claims answer, under what conditions each claim is judged, and which sources count as evidence. No existing benchmark specifies all three. We introduce JSFactCheckBench, a Japanese benchmark that specifies all three, with 1,913 claims from LLM answers to questions about misleading information that circulated on Japanese social media, each annotated for factuality at a reference time and for importance to the question, using evidence from preselected Web sources. Across six LLMs, all verify claims directly answering the question less accurately than supplementary claims, a gap that aggregate scores obscure. Scores also drop when models must locate the relevant content in longer or multiple sources and, for all four open models, when the surrounding question and response are added. These weaknesses are visible only because the benchmark records the conditions behind each label. A benchmark that leaves them implicit cannot reveal them.