Scientific Claim Verification

Overview

The field studies how to determine whether scientific claims follow from research papers, datasets, figures, code, and other evidence. Common approaches retrieve relevant passages, break claims into atomic parts, and apply natural-language inference or language models; newer systems add multimodal tools, selective full-text search, and code or experiment checks. Benchmarks span biomedical and climate claims, scientific tables and charts, and evidence chains linking data to conclusions. Strong results on familiar tasks do not always transfer across domains or expose errors in complex claims, leaving open how to evaluate genuine synthesis, uncertainty, and end-to-end evidential support.

Latest papers

63 papers12 papers in the last four weeks, up 71% on the four weeks before. 0.1% of all new papers.

Mar 17, 2026

ClaimFlow: Tracing the Evolution of Scientific Claims in NLP

Scientific papers advance claims\textit{claims} that later work supports, extends, or sometimes refutes. Yet existing methods for citation and claim analysis capture only fragments of this dialogue. In this work, we make these interactions explicit at the level of individual scientific claims. We introduce ClaimFlow\texttt{ClaimFlow}, a claim-centric view of the NLP literature, built from 1,6171{,}617 ACL Anthology papers (1979−2025)(1979 - 2025) that are manually annotated with 5,6895{,}689 claims and 4,8714{,}871 cross-paper claim relations, indicating whether a citing paper supports\texttt{supports}, extends\texttt{extends}, qualifies\texttt{qualifies}, refutes\texttt{refutes}, or references a cited claim as background\texttt{background}. Building on ClaimFlow\texttt{ClaimFlow}, we define a new task -- Claim Relation Classification\textit{Claim Relation Classification} -- which requires models to infer the scientific stance toward a cited claim from the text and citation context. Evaluating neural models and large language models on this task, we report baseline performance of 0.810.81 macro-F1, suggesting that the task is tractable while leaving room for improvement. We then scale this framework to \sim$$13k NLP papers to study claim evolution across decades of NLP research. We show that 63.5%63.5\% claims are never reused; only 11.1%11.1\% are ever challenged. Widely propagated claims are more often reshaped\textit{reshaped} through qualification and extension than supported or refuted. Overall, ClaimFlow\texttt{ClaimFlow} offers a lens for examining how ideas shift and mature within NLP.
Mar 6, 2026

Plato's Cave: A Human-Centered Research Verification System

The growing publication rate of research papers has created an urgent need for better ways to fact-check information, assess writing quality, and identify unverifiable claims. We present Plato's Cave as an open-source, human-centered research verification system that (i) creates a directed acyclic graph (DAG) from a document, (ii) leverages web agents to assign credibility scores to nodes and edges from the DAG, and (iii) gives a final score by interpreting and evaluating the paper's argumentative structure. We report the system implementation and results on a collected dataset of 104 research papers.
Mar 5, 2026

Med-V1: Small Language Models for Zero-shot and Scalable Biomedical Evidence Attribution

Assessing whether an article supports an assertion is essential for hallucination detection and claim verification. While large language models (LLMs) have the potential to automate this task, achieving strong performance requires frontier models such as GPT-5 that are prohibitively expensive to deploy at scale. To efficiently perform biomedical evidence attribution, we present Med-V1, a family of small language models with only three billion parameters. Trained on high-quality synthetic data newly developed in this study, Med-V1 substantially outperforms (+27.0% to +71.3%) its base models on five biomedical benchmarks unified into a verification format. Despite its smaller size, Med-V1 performs comparably to frontier LLMs such as GPT-5, along with high-quality explanations for its predictions. We use Med-V1 to conduct a first-of-its-kind use case study that quantifies hallucinations in LLM-generated answers under different citation instructions. Results show that the format instruction strongly affects citation validity and hallucination, with GPT-5 generating more claims but exhibiting hallucination rates similar to GPT-4o. Additionally, we present a second use case showing that Med-V1 can automatically identify high-stakes evidence misattributions in clinical practice guidelines, revealing potentially negative public health impacts that are otherwise challenging to identify at scale. Overall, Med-V1 provides an efficient and accurate lightweight alternative to frontier LLMs for practical, real-world biomedical evidence attribution. Med-V1 is available at https://github.com/NLM-DIR/Med-V1.