cs.AIAug 24, 2026

Generating Biomedical Fact-Checking Reports with RL-Enhanced Agentic Search

Authors: Jiongxiao Wang, Dingli Ma, Chaoqun Ni

Organizations: University of Wisconsin–Madison · University of Washington

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Automated fact-checking is essential for ensuring the reliability of public health information, yet the biomedical domain poses unique challenges. Validating biomedical claims requires rigorous interpretation of scientific literature, assessment of retrieved evidence, and comprehensive justification toward the conclusion. Although Large Language Models (LLMs) enhanced by Retrieval-Augmented Generation (RAG) and agentic search perform automated fact-checking in a retrieve-then-verify paradigm, current methods still output isolated prediction labels, lacking explanatory depth and offers limited utility for human understanding. To bridge this gap, we introduce an LLM-based agent named BioCheck Agent that generates structured biomedical fact-checking reports with agentic search. Rather than merely outputting supported or refuted labels, our agent synthesizes final conclusions with retrieved evidence and rigorous analysis. To ensure domain-specific accuracy, BioCheck Agent exclusively searches high-quality scientific literature in PubMed, utilizing advanced Boolean search operators. Recognizing that direct prompting often results in hallucinations and low-quality reports, especially for lightweight open-source models, we further propose the Evidence-Grounded Group Relative Policy Optimization (EG-GRPO) to perform reinforcement learning on BioCheck Agent with a task-specific reward that incentivizes advanced search behavior and high-quality evidence retrieval while penalizing hallucinations. Our experimental results show that compared to the base model Qwen3.5-4B, BioCheck Agent with EG-GRPO improves label prediction accuracy on SciFact by 9.95%. Furthermore, it achieves a 3.7% higher evidence quality score and a 19.63% lower evidence hallucination rate, demonstrating its ability to generate biomedical fact-checking reports with improved accuracy and quality.

Figures & tables

Figure 1: Illustration for BioCheck Agent and Reward Modeling for EG-GRPO.
Base ModelMethodSciFactHealthFC
AccPrecRecF1AccPrecRecF1
GPT5.2No Retrieval66.4984.4165.4771.7460.8684.2253.6859.52
CER Barone et al. (2025)78.5396.2578.7386.4754.1382.9547.0153.00
FIRE Xie et al. (2025)82.7285.5983.5482.5774.0172.5873.0172.76
PMSearch Agent78.0192.1278.2484.4370.9577.3667.7971.78
BioCheck Agent81.1588.9881.5784.6872.4875.3471.4773.34
Table 1: Label prediction performance of BioCheck Agent against various baselines. For each Base Model, the highest value is bolded and the second highest is underlined.
Figure 2: Sub-figures (a) and (b) evaluate the EQS (%) and EHR (%), respectively, for the generated fact-checking reports across various models.
Agent ModelQwen3.5-4BGPT-4o
SciFactHealthFCSciFactHealthFC
Qwen3.5-4B66.7672.9864.7373.83
w/ GRPO64.0172.4463.6071.65
w/ EG-GRPO70.4674.3869.9275.84
Table 2: EQS (%) evaluation with alternative judge LLM.
Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: LangGraph structure of BioCheck Agent.
HyperparameterValue
total epochs10
global batch size64
mini batch size64
learning rate1e-6
KL loss coefficient0.03
max response length16000
Appendix
Table 3: Hyperparameters for EG-GRPO.
Figure 4: Training Curve for EG-GRPO.
More in this topic

Automated Fact-Checking

Automated fact-checking detects checkable claims, retrieves evidence, and assesses whether that evidence supports or refutes them.

See all 140 papers

Explore similar work

CardsList
Aug 2, 2026

When Retrieval Helps and Distracts: Evaluating Evidence-Generating LLMs for Biomedical Claim Verification

Biomedical fact-checking systems must do more than predict whether a claim is supported, contradicted, or unaddressed: they should also produce evidence that is faithful, complete, and useful for verification. We study this evidence-generation setting on CARE-XAI, a unified benchmark spanning five biomedical and health fact-checking sources. We compare base instruction LLMs, PubMed retrieval-augmented LLMs, fine-tuned LLMs, label-only LLMs, and biomedical encoder classifiers under a shared evaluation protocol. Biomedical classifiers remain strongest for verdict-only prediction, while fine-tuned LLMs are the strongest evidence-generating systems. PubMed retrieval is mixed: it helps PubMed-aligned sources such as PubMedQA and SciFact, but can distract models on broader public-health claims. We introduce Bio-GRACE, a gold-reference-normalized diagnostic for measuring whether retrieved evidence recovers the decision benefit of reference evidence. Bio-GRACE shows that retrieval utility is source-dependent, motivates selective retrieval, and exposes why retrieval recall and lexical evidence overlap are insufficient for biomedical fact-checking.

Pritam Deka, Prabhjot Singh

Queen’s University Belfast United Kingdom · University of Texas at Austin Austin, Texas, USA

Explainable Artificial IntelligenceAutomated Fact-CheckingRAG EvaluationEvidence-Grounded GenerationScientific Claim Verification
AskPDF
Nov 5, 2025

Hybrid Fact-Checking that Integrates Knowledge Graphs, Large Language Models, and Search-Based Retrieval Agents Improves Interpretable Claim Verification

Large language models (LLMs) excel in generating fluent utterances but can lack reliable grounding in verified information. At the same time, knowledge-graph-based fact-checkers deliver precise and interpretable evidence, yet suffer from limited coverage or latency. By integrating LLMs with knowledge graphs and real-time search agents, we introduce a hybrid fact-checking approach that leverages the individual strengths of each component. Our system comprises three autonomous steps: 1) a Knowledge Graph (KG) Retrieval for rapid one-hop lookups in DBpedia, 2) an LM-based classification guided by a task-specific labeling prompt, producing outputs with internal rule-based logic, and 3) a Web Search Agent invoked only when KG coverage is insufficient. Our pipeline achieves an F1 score of 0.93 on the FEVER benchmark on the Supported/Refuted split without task-specific fine-tuning. To address Not enough information cases, we conduct a targeted reannotation study showing that our approach frequently uncovers valid evidence for claims originally labeled as Not Enough Information (NEI), as confirmed by both expert annotators and LLM reviewers. With this paper, we present a modular, opensource fact-checking pipeline with fallback strategies and generalization across datasets.

Shaghayegh Kolli, Richard Rosenbaum, Timo Cavelius +3

Technical University Munich

LLM GroundingAutomated Fact-Checking
AskPDF
Mar 10, 2026

Reason and Verify: A Framework for Faithful Retrieval-Augmented Generation

Retrieval-Augmented Generation (RAG) significantly improves the factuality of Large Language Models (LLMs), yet standard pipelines often lack mechanisms to verify inter- mediate reasoning, leaving them vulnerable to hallucinations in high-stakes domains. To address this, we propose a domain-specific RAG framework that integrates explicit rea- soning and faithfulness verification. Our architecture augments standard retrieval with neural query rewriting, BGE-based cross-encoder reranking, and a rationale generation module that grounds sub-claims in specific evidence spans. We further introduce an eight-category verification taxonomy that enables fine-grained assessment of rationale faithfulness, distinguishing between explicit and implicit support patterns to facilitate structured error diagnosis. We evaluate this framework on the BioASQ and PubMedQA benchmarks, specifically analyzing the impact of dynamic in-context learning and rerank- ing under constrained token budgets. Experiments demonstrate that explicit rationale generation improves accuracy over vanilla RAG baselines, while dynamic demonstration selection combined with robust reranking yields further gains in few-shot settings. Using Llama-3-8B-Instruct, our approach achieves 89.1% on BioASQ-Y/N and 73.0% on Pub- MedQA, competitive with systems using significantly larger models. Additionally, we perform a pilot study combining human expert assessment with LLM-based verification to explore how explicit rationale generation improves system transparency and enables more detailed diagnosis of retrieval failures in biomedical question answering.

Eeham Khan, Luis Rodriguez, Marc Queudot

Concordia University, Montréal, QC, Canada · Centre de recherche informatique de Montréal (CRIM), Montréal, QC, Canada

RAG Citation FaithfulnessRetrieval-Augmented GenerationRAG RobustnessFaithfulness of Language Model ExplanationsBiomedical QARetrieval-Augmented ICL
AskPDF