cs.CLNov 14, 2026

ROUNDS: Source-Framed Diagnostic Revision after Multi-Turn Clinical Evidence

Authors: Jinrui Fang, Runhan Chen, Yining Wu, Jiawei Xu, Xu Yang, Jian Yu, Wenqi Shi, Ying Ding, 

Organizations: School of Information, The University of Texas at Austin · Peter O’Donnell Jr. School of Public Health, The University of Texas Southwestern Medical Center · Department of Computer Science, The University of Texas at Austin · Department of Genetics, Washington University in St. Louis · 

Abstract

Large language models (LLMs) achieve high scores on medical benchmarks, but does this performance translate into reliable diagnosis as a conversation unfolds? In diagnostic conversations, users can provide clinical findings across multiple turns, ask for a diagnosis without supplying answer options, and introduce opinions from others. Models must therefore form diagnoses from the available evidence and decide how to respond to subsequent advice. We introduce ROUNDS, a framework that evaluates these abilities after a fixed sequence of clinical evidence delivered over multiple turns. ROUNDS compares diagnosis with and without answer options, presents the same incorrect diagnosis under clinician and lay framing, and varies support for competing diagnoses in written five-clinician panels. Across seven models and 1,035 cases from five datasets, average accuracy was 76.9% with answer options and 57.0% without them. Clinician-framed misleading advice produced lower final accuracy than lay-framed advice in every model, with differences of 4.6 to 38.2 percentage points. Initially correct diagnoses more often became incorrect answers than abstentions, and many adopted the suggested disease. Conversely, accuracy increased as more panel members supported the correct diagnosis, including when the majority opinion remained unchanged. These findings reveal a gap between medical benchmark performance and reliable diagnosis in conversation: models respond to external opinions, but remain vulnerable to abandoning correct diagnoses without new clinical evidence.

Explore similar work

CardsList
  1. Benchmarking Multi-turn Medical Diagnosis: Hold, Lure, and Self-Correction

    Apr 6, 2026Jinrui Fang, Runhan Chen, Xu Yang +9Clinical Language Model EvaluationMedical Diagnosis

  2. KlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy

    Sep 29, 2026Xueting Fang, Zehui Li, Yang Yang +5Clinical Decision-MakingMulti-Turn Dialogue Evaluation

  3. MedRoundsQA: A Persona and Difficulty Aware Evaluation for Multi-Turn Medical Consultations

    Sep 11, 2026Youssef Mohamed, Ahmed Heakl, Qinrong Cui +13Medical DiagnosisMulti-Turn Dialogue Evaluation