Improving Clinical Diagnosis with Counterfactual Multi-Agent Reasoning
Authors: Zhiwen You, Xi Chen, Aniket Vashishtha, Simo Du, Gabriel Erion-Barner, Hongyuan Mei, Hao Peng, Yue Guo
Organizations: University of Illinois Urbana-Champaign, Urbana, IL, USA · Independent Researcher · Jacobi Medical Center, Albert Einstein College of Medicine, NY, USA · Department of Emergency Medicine, Beth Israel Deaconess Medical Center, Boston, MA, USA · Leaning machines, Palo Alto, CA, USA
Clinical diagnosis is a complex reasoning process in which clinicians gather evidence, form hypotheses, and test them against alternative explanations. In medical training, this reasoning is explicitly developed through counterfactual questioning--e.g., asking how a diagnosis would change if a key symptom were absent or altered--to strengthen differential diagnosis skills. As large language model (LLM)-based systems are increasingly used for diagnostic support, ensuring the interpretability of their recommendations becomes critical. However, most existing LLM-based diagnostic agents reason over fixed clinical evidence without explicitly testing how individual findings support or weaken competing diagnoses. In this work, we propose a counterfactual multi-agent diagnostic framework inspired by clinician training that makes hypothesis testing explicit and evidence-grounded. Our framework introduces counterfactual case editing to modify clinical findings and evaluate how these changes affect competing diagnoses. We further define the Counterfactual Probability Gap, a method that quantifies how strongly individual findings support a diagnosis by measuring confidence shifts under these edits. These counterfactual signals guide multi-round specialist discussions, enabling agents to challenge unsupported hypotheses, refine differential diagnoses, and produce more interpretable reasoning trajectories. Across three diagnostic benchmarks and seven LLMs, our method consistently improves diagnostic accuracy over prompting and prior multi-agent baselines, with the largest gains observed in complex and ambiguous cases. Human evaluation further indicates that our framework produces more clinically useful, reliable, and coherent reasoning. These results suggest that incorporating counterfactual evidence verification is an important step toward building reliable AI systems for clinical decision support.
Figures & tables
Figure 1: Overview of the proposed counterfactual case editing-based multi-agent diagnostic framework. Given a clinical case, a triage agent selects a set of relevant medical specialists, and an initial differential diagnosis (DDx) is generated to guide subsequent reasoning. During multi-round discussion, each specialist performs counterfactual case editing by modifying targeted clinical findings to test competing diagnostic hypotheses. The impact of these edits is quantified using the Counterfactual Probability Gap (CPG), which measures changes in diagnostic confidence before and after the target evidence operation. These counterfactual signals guide iterative discussion, enabling specialists to explicitly test how individual clinical findings support or weaken competing diagnoses. A final judge agent synthesizes the discussion into an evidence-grounded diagnostic decision and structured reasoning trace.
Figure 2: Average diagnostic accuracy of seven LLMs on three datasets, including (a) Llama-3.1-8B-Instruct , (b) Qwen3-8B , (c) m1-7b-23k , (d) MedReason-8B , (e) medgemma-1.5-4b-it , (f) Deepseek-R1 , and (g) GPT-5-mini . Bar graphs indicate the accuracy ± 95% CIs. Numerical results and statistical significance tests are provided in Table 2 .
Figure 3: Average diagnostic accuracy of Llama-3.1-8B-Instruct for four diseases/specialties on three datasets, including (a) Disease-level accuracy on MIMIC, (b) Specialty-level accuracy on MedCaseReasoning, (c) Specialty-level accuracy on ER-Reason. Following Liu et al. Liu et al. (2025) , we categorize the test cases into specialties, and select the Top-4 most frequent specialties with relevant diagnoses from MedCaseReasoning and ER-Reason. Bar graphs indicate the accuracy ± 95% CIs.
Figure 4: Multi-round discussion statistics and the impact of counterfactual case editing on diagnostic confidence using Llama-3.1-8B-Instruct in the multi-round discussion stage. (a) , Consensus rate achieved by the multi-round discussion across datasets. (b) , Average number of the discussion rounds required per case. (c) , Specialist diagnosis-change rate across the three datasets. Error bars indicate the standard deviation across three random seeds. Bar graphs indicate the standard deviation of averaged results across three random seeds. (d) , Outcomes of diagnosis transformations, categorized by the correctness of the initial and final diagnoses relative to the gold standard (W: wrong, C: correct). (e) , Probability density of the diagnostic hypothesis before ( Pbase ) and after ( PCE ) targeted evidence perturbation during counterfactual case editing. Only cases where the predicted diagnosis remains unchanged before and after CF editing are included. (f) , Distribution of diagnosis probability shifts ( ΔP ) across different target evidence operations on the three datasets. TE: target evidence. All statistics are calculated by averaging the results of three random seeds.
Figure 5: Ablation study of Llama-3.1-8B-Instruct over different functional modules on MedCaseReasoning. The shaded area represents the 95% CI. (a) Diagnostic performance with various functional moduels added in our multi-agent diagnostic system. w/o: without; CF: counterfactual. (b) Diagnostic performance with various hyperparameters. DDx: differential diagnosis.
Figure 7: Human evaluation of clinical reasoning quality comparing our method with zero-shot CoT across three datasets. (a) , Error and Safety Assessment (ESA) metrics annotated with binary yes/no labels. (b) , Reasoning Quality Assessment (RQA) and Clinical Contribution (CC) evaluated on a 1–5 Likert scale (presented in Figure 15 and Figure 16 ). (c) , Overall trust preference between the two methods. Bar graphs indicate the mean ± 95% CIs.
Figure 8: Example of diagnostic rationales generated by our multi-agent diagnostic framework using Llama-3.1-8B-Instruct model on a case from the MedCaseReasoning dataset.
Figure 9: Example of our counterfactual case editing approach. CF: counterfactual. DDx: differential diagnosis.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset size
Testing size
Original Length
Summarized Length
Source
MIMIC-CDM-FI [ 18 ]
2,400
80
1,549
1,046
ICU database from Beth Israel Deaconess Medical Center
MedCaseReasoning [ 8 ]
14,489
100
360
-
PubMed Central
ER-REASON [ 19 ]
25,174
80
1,830
647
Emergency department data from UCSF Medical Center
Appendix
Table 1: Comparison of three clinical diagnosis datasets. We calculate the average case length in token level using GPT-4o through the OpenAI tokenizer.
Figure 10: Performance of Deepseek-R1 for the diagnosis of case presentation on three datasets. (a) , Consensus rate achieved by the multi-round discussion process across datasets. (b) , Average number of the discussion rounds required per case. (c) , Specialist diagnosis-change rate across the three datasets. Error bars indicate the standard deviation across three random seeds. Bar graphs indicate the standard deviation of averaged results across three random seeds. (d) , Outcomes of diagnosis transformations, categorized by the correctness of the initial and final diagnoses relative to the gold standard (W: wrong, C: correct). Note that there is no C → C category in LLM-based diagnosis transformation outcomes. (e) , Probability density of the diagnostic hypothesis before ( Pbase ) and after ( PCE ) targeted evidence perturbation during counterfactual case editing. Only cases where the predicted diagnosis remains unchanged before and after CF editing are included. (f) , Distribution of diagnosis probability shifts ( ΔP ) across different target evidence operations on the three datasets. TE: target evidence. All statistics are calculated by averaging the results of three random seeds.
Model
Setting
Method
MIMIC
MedCaseReasoning
ER-Reason
Average Acc.
Llama
Prompting
ZS
54.2 (1.6)
17.3 (0.5)
10.5 (2.4)
27.3* (1.5)
ZS CoT
50.9 (3.9)
18.0 (4.1)
8.4 (2.6)
25.8* (3.5)
FS
71.3 (2.0)
17.7 (3.1)
17.1 (3.6)
35.4 (2.9)
FS CoT
70.0 (3.7)
14.3 (2.5)
17.9 (1.5)
34.1 (2.6)
SC
54.2 (2.6)
19.3 (6.5)
7.9 (0.7)
27.1* (3.3)
Agents
MAC
55.9 (2.6)
17.3 (3.4)
13.8 (2.7)
29.0* (2.9)
Appendix
Table 2: Evaluation results of final diagnosis prediction accuracy on three datasets. We report the standard deviation (std.) in the brackets. ZS: zero-shot; CoT: chain-of-thought; FS: few-shot; SC: self-consistency. * P < 0.05 indicates our method shows significant difference than the baselines.
Figure 11: Distribution of the Top-10 most frequently assigned specialists by the triage agent for each dataset on (a) Llama-3.1-8B-Instruct and (b) Deepseek-R1 .
Metric
Question
Options
Factual Errors
Does the reasoning contain any false/incorrect medical information?
Yes / No
Hallucination
Does the reasoning cite findings not present in the case?
Yes / No
Harm
Does this reasoning trace contain a “Critical Failure?” (e.g., ignoring contraindication, recommending fatal drug dosage, ruling out life-threatening emergency without cause)
Yes / No
Completeness
Does the reasoning correctly identify key findings, risk factors, and diagnoses?
Yes / No
Trust
Which reasoning inspires more confidence in the diagnostic process?
A / B / Equal
Appendix
Table 3: Evaluation metrics for error, safety, completeness, and trust assessment. Criteria are evaluated using binary or comparative scales to identify critical failures and overall confidence in the diagnostic reasoning.
Category
Question
Criteria
Efficiency
Does the reasoning communicate effectively?
1: Verbose or repetitive; ignores key diagnostic points 2: Somewhat verbose; key points present but unclear 3: Appropriate length; mostly focused; occasional redundancy 4: Concise and focused; key insights highlighted 5: Exceptionally clear; every statement adds evidence
Logical Coherence
Does the reasoning demonstrate sound logical structure in ruling in/out diagnoses?
1: Logical failures; no clear chain from evidence to conclusion 2: Incomplete logic; mentions findings but fails to connect 3: Basic coherence; clear progression but may lack depth 4: Strong reasoning; synthesizes findings with explanations 5: Exceptional; comprehensive integration, addresses confounders
Differential Diagnosis
Does the reasoning consider alternative diagnoses appropriately?
1: No differential mentioned 2: Differential vaguely implied but not explicitly stated 3: 1 alternative mentioned without explanation 4: 1 alternative mentioned with brief justification 5: ≥ 2 alternatives with clear reasoning for each
Final Diagnosis Explanation
How well does the reasoning explain the final diagnosis?
1: No explanation linking evidence to diagnosis 2: Explanation vaguely implied without linking specific findings 3: Links < 2 objective findings to diagnosis without explanations 4: Links < 2 objective findings to diagnosis with explanations 5: Links ≥ 3 findings with pathophysiological reasoning
Diagnostic Contribution
Does the reasoning help a clinician understand and justify the diagnosis?
1: Mostly restates facts only; no synthesis 2: Limited synthesis; provides minimal justification and addresses only a few diagnostic criteria or findings 3: Adequate synthesis; connects most key findings to the diagnosis, though some reasoning steps are underexplained 4: Strong diagnostic explanation; provides clear clinical justification but omits a few important reasoning links 5: Comprehensive and clinically instructive synthesis; clearly justifies the diagnosis and integrates relevant differential reasoning, disease course/timeline, and implications
Appendix
Table 4: Evaluation metrics for reasoning quality and clinical contribution. Each category is evaluated on a 5-point Likert scale, assessing the coherence, depth, and clinical utility of the model’s reasoning.
Category
Description
Anchoring Bias
Prefers initial symptoms while disregarding contradictory findings.
Omission Bias
Fails to consider all necessary elements of the diagnostic/treatment pathway.
Overinvestigation
Misuse of laboratory values or drug mechanisms; focusing on irrelevant details.
Incorrect Causal Attribution
Misattributes primary cause; confuses causal relationships between conditions.
Factual Error
Contains incorrect medical information.
Hallucinated Finding
Cites findings not present in the case.
Appendix
Table 5: Bias categories for sentence-level classification. Definitions of specific cognitive biases and reasoning errors annotated within the clinical text.
Metric
Zero-shot CoT
Ours
κ
% Agree
κ
% Agree
Factual Errors
0.833
91.7%
0.471
75.0%
Hallucination
0.750
91.7%
1.000
100.0%
Harm
0.000
83.3%
0.000
83.3%
Completeness
0.226
66.7%
-0.154
58.3%
Efficiency
0.082
66.7%
-0.393
41.7%
Appendix
Table 6: Inter-Annotator Agreement (IAA) across nine human evaluation dimensions. Agreement is reported using Cohen’s κ for categorical metrics and weighted κw for ordinal metrics. Both the zero-shot CoT and our proposed method are evaluated by two physicians ( N=12 ).
Figure 12: The user interface of annotation guideline and the case presentation. Each case is provided with the ground truth diagnosis and two predicted diagnoses from two reasoning methods: zero-shot CoT and our system.
Figure 13: The user interface of side-by-side reasoning trace comparison. Users can highlight sentences in both traces and indicate the corresponding errors.
Figure 14: The user interface of error, safety, and completeness assessment. Criteria are evaluated using binary scales to identify critical failures.
Figure 15: The user interface of reasoning quality evaluation. Each category is evaluated on a 5-point Likert scale.
Figure 16: The user interface of clinical contribution evaluation. Each category is evaluated on a 5-point Likert scale.
Figure 17: The user interface of annotation on reasoning traces trust assessment bias classification. We define four types of biases: anchoring bias, omission bias, overinvestigation, and incorrect causal attribution.
Category
Specialists
General
General Internal Medicine Doctor, Laboratory Doctor, Outpatient Doctor
While Large Language Models (LLMs) have demonstrated potential in healthcare, they often struggle with the complex, non-linear reasoning required for accurate clinical diagnosis. Existing methods typically rely on static, linear mappings from symptoms to diagnoses, failing to capture the iterative, hypothesis-driven reasoning inherent in human clinicians. To bridge this gap, we introduce ClinicalAgents, a novel multi-agent framework designed to simulate the cognitive workflow of expert clinicians. Unlike rigid sequential chains, ClinicalAgents employs a dynamic orchestration mechanism modeled as a Monte Carlo Tree Search (MCTS) process. This allows an orchestrator to iteratively generate hypotheses, actively verify evidence, and trigger backtracking when critical information is missing. The foundation of this framework is a Dual-Memory architecture: a mutable working memory that maintains the evolving patient state for context-aware reasoning, and a static experience memory that retrieves clinical guidelines and historical cases via an active feedback loop. Extensive experiments demonstrate that ClinicalAgents achieves the best performance among evaluated baselines, significantly enhancing both diagnostic accuracy and explainability compared to strong single-agent and multi-agent baselines. Our code is released at https://github.com/ZhuohanGe/ClinicalAgents-Code.
Zhuohan Ge, Haoyang Li, Yubo Wang +3
The Hong Kong Polytechnic University, Hong Kong SAR, China · The Hong Kong University of Science and Technology, Hong Kong SAR, China
Multimodal Large Language Models (MLLMs) in healthcare suffer from severe confirmation bias, often hallucinating visual details to support initial, potentially erroneous diagnostic hypotheses. Existing Chain-of-Thought (CoT) approaches lack intrinsic correction mechanisms, rendering them vulnerable to error propagation. To bridge this gap, we propose Dialectic-Med, a multi-agent framework that enforces diagnostic rigor through adversarial dialectics. Unlike static consensus models, Dialectic-Med orchestrates a dynamic interplay between three role-specialized agents: a proponent that formulates diagnostic hypotheses; an opponent equipped with a novel visual falsification module that actively retrieves contradictory visual evidence to challenge the Proponent; and a mediator that resolves conflicts via a weighted consensus graph. By explicitly modeling the cognitive process of falsification, our framework guarantees that diagnostic reasoning is tightly grounded in verified visual regions. Empirical evaluations on MIMIC-CXR-VQA, VQA-RAD, and PathVQA demonstrate that Dialectic-Med not only achieves state-of-the-art performance but also fundamentally enhances the trustworthiness of the reasoning process. Beyond accuracy, our approach significantly enhances explanation faithfulness and decisively mitigates hallucinations, establishing a new standard over single-agent baselines.
Large language models (LLMs) show potential for medical tasks, but their single-turn question-answer format does not reflect how clinical diagnosis is performed in practice. As a result, they remain limited in complex diagnostic settings. We developed Debate-Mixture-of-Agents (DMoA), a novel multi-agent framework that structures role-based interaction to support iterative diagnostic reasoning. Base models and DMoA were evaluated on 297 rare disease cases and 1,719 challenging cases. Across both datasets, DMoA improved most likely diagnosis accuracy by 10.21 percentage points and safety rate by 11.36 percentage points over GPT-4o baseline. Ablation experiments showed that the gains were not simply due to the use of more models or longer outputs, but also reflected the contribution of the structured workflow. Further analyses examined how framework design, base model choice, and token budget affected performance. DMoA performed better with a 4*2 structure, stronger base models, and a larger token budget. These findings demonstrate the potential of DMoA for clinical tasks and suggest further investigation of multi-agent frameworks.
Chang Xia, Leilei Ouyang, Huimin Wang +2
College of Computer Science, Sichuan University, Chengdu 610207, China · West China Biomedical Big Data Center, West China Hospital, Sichuan University, Chengdu 610041, China · Med-X Center for Informatics, Sichuan University, Chengdu 610041, China