Protocol for evaluating ChatGPT in biomedical association generation and verification using a RAG-enabled, cross-model majority voting workflow

Ahmed Abdeen Hamed; Luis M. Rocha

cs.CLMay 28, 2026

Protocol for evaluating ChatGPT in biomedical association generation and verification using a RAG-enabled, cross-model majority voting workflow

Authors: Ahmed Abdeen Hamed, Luis M. Rocha

Organizations: Department of Biochemistry, University of Nebraska-Lincoln, Lincoln, NE 68588, USA · School of Systems Science & Industrial Engineering, Binghamton University, Binghamton, NY 13902, USA · Universidade Catόlica Portuguesa, Catόlica Biomedical Research Centre, Lisbon, Portugal

Abstract

We present a protocol to evaluate ChatGPT's ability to generate disease-centric biomedical associations. It outlines how we generate the associations, validate the biological entities using biomedical ontologies, and verify associations using literature. The protocol includes a self-consistency strategy to assess generative reliability across ChatGPT models. To address ontology exact-match limitations, we provide a use case performing semantic verification through a workflow enabled by Retrieval-Augmented Generation (RAG) powered by open-source large language models (LLMs). This enables LLMs to establish truth over content generated by other LLMs and expose hallucination.

Explore similar work

Deceptive Grounding: Entity Attribution Failure in Clinical Retrieval-Augmented Generation
Jul 10, 2026Cedric Caruzzo, Donggeun Yoo, Tae Soo KimClinical Reasoning Clinical
HealthBench Professional: Evaluating Large Language Models on Real Clinician Chats
Apr 30, 2026Rebecca Soskin Hicks, Mikhail Trofimov, Dominick Lim +13Benchmarking Large Language Models Physical Activities
ProtoMedAgent: Multimodal Clinical Interpretability via Privacy-Aware Agentic Workflows
May 13, 2026Alvaro Lopez Pellicer, Plamen Angelov, Marwan Bukhari +3Multimodal Clinical Data Mediation

Jul 10, 2026 · cs.CLJ/K move · Enter open · S save

Deceptive Grounding: Entity Attribution Failure in Clinical Retrieval-Augmented Generation

Cedric Caruzzo, Donggeun Yoo, Tae Soo Kim

Lunit

Retrieval-augmented generation evaluation checks whether model claims are factually grounded in retrieved documents. It does not check whether retrieved evidence is attributed to the correct entity. A clinical RAG response can pass every automated check (zero hallucinations, near-perfect faithfulness, real citations) while presenting drug Y's clinical evidence as evidence about queried drug X. We term this deceptive grounding (DG): a failure invisible to faithfulness, hallucination, and citation checks because every claim is sourced from a real document, about the wrong entity. Using a controlled factorial benchmark across 13 models, we find DG rates spanning 8-87% at peak adversarial conditions. Medical and biomedical fine-tuned models reach up to 86.7%; domain specialization amplifies the failure rather than mitigating it. A controlled ablation identifies the mechanism: removing entity-specific clinical evidence from retrieved documents eliminates entity-attribution failure entirely, shifting all failures to confabulation. The two failure modes respond to the same trigger, taking different paths. Production measurement across 740 drug-disease pairs finds 7.8% overall DG in a deployed RAG system, rising to 13.6% for recently approved drugs. Entity-attribution verification (checking that cited evidence applies to the queried entity) detects DG at 97.0% precision and 98.7% DG recall (IPW-adjusted human gold standard); no existing framework implements it.

Clinical Reasoning Clinical Entities Deception

Open paper Ask about it PDF

Abstract

Explore similar work

Deceptive Grounding: Entity Attribution Failure in Clinical Retrieval-Augmented Generation

HealthBench Professional: Evaluating Large Language Models on Real Clinician Chats

ProtoMedAgent: Multimodal Clinical Interpretability via Privacy-Aware Agentic Workflows