ClinicalAgents: Multi-Agent Orchestration for Clinical Decision Making with Dual-Memory

Zhuohan Ge; Haoyang Li; Yubo Wang; Nicole Hu; Chen Jason Zhang; Qing Li

cs.CLMar 27, 2026

ClinicalAgents: Multi-Agent Orchestration for Clinical Decision Making with Dual-Memory

Authors: Zhuohan Ge, Haoyang Li, Yubo Wang, Nicole Hu, Chen Jason Zhang, Qing Li

Organizations: The Hong Kong Polytechnic University, Hong Kong SAR, China · The Hong Kong University of Science and Technology, Hong Kong SAR, China

Abstract

While Large Language Models (LLMs) have demonstrated potential in healthcare, they often struggle with the complex, non-linear reasoning required for accurate clinical diagnosis. Existing methods typically rely on static, linear mappings from symptoms to diagnoses, failing to capture the iterative, hypothesis-driven reasoning inherent in human clinicians. To bridge this gap, we introduce ClinicalAgents, a novel multi-agent framework designed to simulate the cognitive workflow of expert clinicians. Unlike rigid sequential chains, ClinicalAgents employs a dynamic orchestration mechanism modeled as a Monte Carlo Tree Search (MCTS) process. This allows an orchestrator to iteratively generate hypotheses, actively verify evidence, and trigger backtracking when critical information is missing. The foundation of this framework is a Dual-Memory architecture: a mutable working memory that maintains the evolving patient state for context-aware reasoning, and a static experience memory that retrieves clinical guidelines and historical cases via an active feedback loop. Extensive experiments demonstrate that ClinicalAgents achieves the best performance among evaluated baselines, significantly enhancing both diagnostic accuracy and explainability compared to strong single-agent and multi-agent baselines. Our code is released at https://github.com/ZhuohanGe/ClinicalAgents-Code.

Figures & tables

Notation	Definition
$\mathcal{M}_{\text{work}}^{t}$	Working Memory at step $t$ , defined as $\langle\mathcal{E}_{t},\mathcal{H}_{t},\tau_{t}\rangle$ .
$\mathcal{M}_{\text{exp}}$	Experience Memory, defined as $\langle\mathcal{D}_{\mathrm{guide}},\mathcal{D}_{\mathrm{cdc}}\rangle$ .
$\mathcal{D}_{\text{guide}}$	Structured guideline database.
$\mathcal{D}_{\text{cdc}}$	Historical case database.
$\mathcal{K}_{t}$	Retrieved guideline knowledge at step $t$ .
$\mathcal{E}_{t}$	Accumulated evidence set at step $t$ .

Table 1. Important Notations.

Figure 1. Overview of the ClinicalAgents framework. A Clinical Orchestrator coordinates a pool of specialized agents and a dual-memory system (working memory and experience memory) to execute an iterative Perceive–Hypothesize–Verify–Update loop with evidence-triggered backtracking.

Dataset	# Samples	Modality	Task
MedChain ( Liu et al., 2025 )	12,163	T+I	5-stage pipeline
MedDG ( Liu et al., 2022 )	17,864	T	dialogue generation
MedQA ( Jin et al., 2021 )	1,273	T	multiple QA
PubMedQA ( Jin et al., 2019 )	500	T	y/n/maybe QA
MedBullets ( Chen et al., 2025 )	308	T	multiple QA
PathVQA ( He et al., 2020 )	3,362	I	binary QA

Table 2. Dataset statistics in our experiments. T=Text, I=Image; “5-step pipeline” denotes a five-stage workflow.

Category	Methods	MedChain						Average
		Specialty Referral		Test Ordering	Examination	Diagnosis	Treatment
		Acc. (Lv1)	IoU (Lv2)	IoU	DocLens	LLM-judge	IoU
Single LLM	GPT-5.2	0.5467	0.3045	0.3870	0.4276	0.5754	0.4716	0.4521
	DeepSeek-R1	0.5744	0.3124	0.3475	—	0.4968	0.4296	0.4321
	Qwen3-VL-235B	0.5719	0.3604	0.2867	0.3915	0.5798	0.5017	0.4487
	Llama-4-Maverick-17B	0.5620	0.3386	0.3075	0.3579	0.5675	0.4048	0.4231

Table 3. Overall results on the MedChain benchmark. We compare ClinicalAgents with single-LLM, single-agent, and multi-agent baselines across five tasks representing workflow stages, and report score for each task and the average score. The symbol ‘—’ indicates that the model is a text-only LLM and not applicable to a visual examination task, and average score is computed over applicable non-visual metrics only. The LLM-judge score was normalized to 0–1 by dividing by 5 before averaging.

Agreement Pair	Cohen’s $\kappa$	Tol. Acc. ( $\Delta\!\leq\!1$ )
Clinician A vs. LLM Judge	0.77	88%
Clinician B vs. LLM Judge	0.82	92%
Clinician A vs. Clinician B	0.88	94%

Table 4. Agreement between the LLM judge (Gemini 2.5 Pro) and two clinicians on the Task 4 diagnostic score. Cohen’s

\kappa

corrects for chance agreement; Tolerance Accuracy (

\Delta\!\leq\!1

) is the percentage of cases where two raters’ scores differ by no more than one point on the 1–5 scale.

Metric	Value
Backtracking trigger rate (initially wrong cases)	83%
Recovery success rate (initially wrong cases)	64%
Cases with at least one excluded hypothesis	94%

Table 5. Reliability statistics of the orchestrator’s backtracking mechanism. The trigger and recovery rates are computed over cases whose initial hypothesis was judged incorrect.

Ablation	SP		TO	EX	Diag	Treat	Average
Ablation	Acc	IoU	IoU	DocLens	LLM-judge	IoU	Average
Backbone	0.5467	0.3045	0.3870	0.4276	0.5754	0.4716	0.4521
+ Dual-Memory	0.6095	0.3187	0.4157	0.4460	0.5811	0.4863	0.4762
+ Orchestrator	0.6125	0.3391	0.4649	0.4673	0.5914	0.5022	0.4962
ALL	0.6243	0.3543	0.4820	0.4800	0.5976	0.5259	0.5107

Table 6. Ablation experiments, where SP denotes Specialty Referral, TO denotes Test Ordering, EX denotes Examination, Diag denotes Diagnosis, Treat denotes Treatment. The backbone model is GPT-5.2.

Backbone	SP		TO	EX	Diag	Treat	Average
Backbone	Acc	IoU	IoU	DocLens	LLM-judge	IoU	Average
Qwen3-VL	0.6273	0.3779	0.3717	0.4369	0.5908	0.5229	0.4879
HuaTuoGPT	0.4948	0.2839	0.3034	—	0.5746	0.5082	0.4330
DeepSeek-R1	0.6322	0.3621	0.4561	—	0.5427	0.4695	0.4925
GPT-5.2	0.6243	0.3543	0.4820	0.4800	0.5976	0.5259	0.5107

Table 7. Backbone sensitivity results. We test ClinicalAgents with different backbone models and report the performance. The symbol ‘—’ indicates that the model is a text-only LLM and not applicable to a visual examination task.

Method	Avg. Score	Tokens	Latency (s)	Cost (USD)
Backbone + SC-3	0.4573	45,265	731.51	0.1215
Backbone + SC-5	0.4579	76,058	1243.62	0.2148
Backbone + SC-7	0.4583	104,733	1681.43	0.2962
MedAgents	0.4547	44,599	581.07	0.1264
MedChain-Agents	0.4880	30,892	433.28	0.0879
ClinicalAgents	0.5107	23,438	369.07	0.0658

Table 8. Efficiency comparison across representative methods under the GPT-5.2 backbone. Latency is reported as serial wall-clock time per case. Best results are in bold .

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Category	Methods	MedQA	PubMedQA	MedBullets	PathVQA	Average
Category	Methods	Acc.	Acc.	Acc.	Acc.	Average
Single LLM	GPT-5.2	0.8845	0.6820	0.7890	0.6835	0.7598
	DeepSeek-R1	0.8987	0.6200	0.7987	—	0.7725
	Qwen3-VL-235B	0.8209	0.5620	0.6721	0.7377	0.6982
	Intern-S1	0.8256	0.7580	0.7240	0.7374	0.7613
	Llama-4-Maverick-17B	0.8798	0.6880	0.7695	0.6880	0.7563

Appendix

Table 9. Generalization results on four open-domain medical QA datasets. For text-only models, the average is computed over applicable text-only datasets. The best result for each metric is highlighted in bold, and the second-best result is underlined. The symbol ‘—’ indicates that the model is a text-only LLM and not applicable to the visual QA task.

Method	BLEU-1	BLEU-4	ROUGE-1	ROUGE-2	Entity-F1
GPT-5.2	39.88	21.25	13.14	4.56	15.59
MedAgents	40.87	21.60	14.85 $\pm$ 0.79	4.63	16.74
MedChain-Agents	41.10 $\pm$ 1.06	22.04 $\pm$ 0.74	14.40	6.60 $\pm$ 0.71	16.86 $\pm$ 1.06
ClinicalAgents	42.67 $\pm$ 0.87	23.36 $\pm$ 0.62	18.43 $\pm$ 0.86	8.42 $\pm$ 0.63	20.72 $\pm$ 1.17

Appendix

Table 10. Performance comparison across representative methods on MedDG. The best result for each metric is highlighted in bold, and the second-best result is underlined.

Phase	Description
Perceive (Initial)	Symptoms: recurrent cough, purulent sputum, low-grade fever, and weight loss.
Hypothesize 1	Based on symptoms, the system initially hypothesizes Typical Bacterial Pneumonia .
Verify 1	Orders Chest X-ray, CBC, CRP. Elevated markers (CRP/neutrophils) support infection, but X-ray unexpectedly reveals a spiculated mass and apical cavitation. Unexplained structural abnormalities trigger backtracking. $\rightarrow$ Backtrack.
Hypothesize 2	The single-infection belief is insufficient. The system pivots to co-evaluate malignancy and chronic infection, updating its hypothesis to: Lung Cancer, Tuberculosis, and Pneumonia .
Verify 2 (Final)	Orders targeted tests (high-resolution CT, biopsy, GeneXpert, sputum culture). Results definitively confirm Lung Cancer alongside tuberculosis and pneumonia.

Appendix

Table 11. Success case. Ground truth: Lung Cancer, Tuberculosis, Pneumonia .

Phase	Description
Perceive (Initial)	Symptoms: Stage IV cervical cancer after chemo-radiotherapy, rectal bleeding, history of HBV infection, ureteral stricture, and hydronephrosis.
Hypothesize 1	Broad hypothesis set: Sinus Tachycardia, Pleural Effusion, Hydronephrosis, Pulmonary Infection, Viral Hepatitis Spectrum, Pericardial Effusion, Radiation Proctitis , and Acute Blood Loss Anemia .
Verify 1	Orders ECG, chest X-ray, CBC, liver panel, renal panel. Sinus Tachycardia and Pleural Effusion are supported, and Acute Blood Loss Anemia is excluded by CBC, but others remain unconfirmed. $\rightarrow$ Backtrack.
Hypothesize 2	Updates hypothesis set: Pulmonary Infection , Hydronephrosis , Viral Hepatitis Spectrum , Pericardial Effusion , and Radiation Proctitis remain active.
Verify 2	Orders chest CT, abdominal ultrasound, HBV-DNA, hepatitis serology panel. Pulmonary Infection is strengthened, but others remain unresolved due to partial evidence return. $\rightarrow$ Backtrack.
Hypothesize 3	Updates hypothesis set: Hydronephrosis and Viral Hepatitis Spectrum remain unresolved.

Appendix

Table 12. Failure case. Ground truth: Viral Hepatitis Spectrum, Sinus Tachycardia, Pleural Effusion, Hydronephrosis, Pulmonary Infection .

Explore similar work

Improving Clinical Diagnosis with Counterfactual Multi-Agent Reasoning
Mar 29, 2026Zhiwen You, Xi Chen, Aniket Vashishtha +5Medical Diagnosis Clinical Decision Support
University of Illinois Urbana-Champaign, Urbana, IL, USA · Independent Researcher · Jacobi Medical Center, Albert Einstein College of Medicine, NY, USA +2
Sparse Multi-Stage Expert-Agent Routing for Complex Clinical Reasoning
Aug 22, 2026Sike Xiang, Shuang Chen, Qian sun +3Clinical Reasoning LLM Agent Routing
Department of Computer Science Durham University
Thinking Like a Clinician: A Cognitive AI Agent for Clinical Diagnosis via Panoramic Profiling and Adversarial Debate
Apr 26, 2026Zhiqi Lv, Duofan Tu, Jun Li +4Tree of Thoughts Clinical Decision Support
School of Biomedical Engineering, Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, Anhui, 230026, P.R. China · Suzhou Institute for Advanced Research, University of Science and Technology of China, Suzhou, Jiangsu, 215123, P.R. China · Key Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS), Institute of Computing Technology, CAS, Beijing, PR China +2

Mar 29, 2026J/K move · Enter open · S save

Improving Clinical Diagnosis with Counterfactual Multi-Agent Reasoning

Zhiwen You, Xi Chen, Aniket Vashishtha, Simo Du+4

University of Illinois Urbana-Champaign, Urbana, IL, USA · Independent Researcher · Jacobi Medical Center, Albert Einstein College of Medicine, NY, USA+2

Medical Diagnosis Clinical Decision Support Clinical Reasoning Counterfactual Reasoning in Language Models Multi-Agent Reasoning

Open paper AskPDF arXiv

Clinical diagnosis is a complex reasoning process in which clinicians gather evidence, form hypotheses, and test them against alternative explanations. In medical training, this reasoning is explicitly developed through counterfactual questioning--e.g., asking how a diagnosis would change if a key symptom were absent or altered--to strengthen differential diagnosis skills. As large language model (LLM)-based systems are increasingly used for diagnostic support, ensuring the interpretability of their recommendations becomes critical. However, most existing LLM-based diagnostic agents reason over fixed clinical evidence without explicitly testing how individual findings support or weaken competing diagnoses. In this work, we propose a counterfactual multi-agent diagnostic framework inspired by clinician training that makes hypothesis testing explicit and evidence-grounded. Our framework introduces counterfactual case editing to modify clinical findings and evaluate how these changes affect competing diagnoses. We further define the Counterfactual Probability Gap, a method that quantifies how strongly individual findings support a diagnosis by measuring confidence shifts under these edits. These counterfactual signals guide multi-round specialist discussions, enabling agents to challenge unsupported hypotheses, refine differential diagnoses, and produce more interpretable reasoning trajectories. Across three diagnostic benchmarks and seven LLMs, our method consistently improves diagnostic accuracy over prompting and prior multi-agent baselines, with the largest gains observed in complex and ambiguous cases. Human evaluation further indicates that our framework produces more clinically useful, reliable, and coherent reasoning. These results suggest that incorporating counterfactual evidence verification is an important step toward building reliable AI systems for clinical decision support.

Abstract

Explore similar work

Improving Clinical Diagnosis with Counterfactual Multi-Agent Reasoning

Sparse Multi-Stage Expert-Agent Routing for Complex Clinical Reasoning

Thinking Like a Clinician: A Cognitive AI Agent for Clinical Diagnosis via Panoramic Profiling and Adversarial Debate