One Panel Does Not Fit All: Case-Adaptive Multi-Agent Deliberation for Clinical Prediction

Yuxing Lu; Yushuhong Lin; Jason Zhang

cs.AIMar 31, 2026

One Panel Does Not Fit All: Case-Adaptive Multi-Agent Deliberation for Clinical Prediction

Authors: Yuxing Lu, Yushuhong Lin, Jason Zhang

Organizations: Georgia Institute of Technology · Peking University

Abstract

Large language models applied to clinical prediction exhibit case-level heterogeneity: simple cases yield consistent outputs, while complex cases produce divergent predictions under minor prompt changes. Existing single-agent strategies sample from one role-conditioned distribution, and multi-agent frameworks use fixed roles with flat majority voting, discarding the diagnostic signal in disagreement. We propose CAMP (Case-Adaptive Multi-agent Panel), where an attending-physician agent dynamically assembles a specialist panel tailored to each case's diagnostic uncertainty. Each specialist evaluates candidates via three-valued voting (KEEP/REFUSE/NEUTRAL), enabling principled abstention outside one's expertise. A hybrid router directs each diagnosis through strong consensus, fallback to the attending physician's judgment, or evidence-based arbitration that weighs argument quality over vote counts. On diagnostic prediction and brief hospital course generation from MIMIC-IV across four LLM backbones, CAMP consistently outperforms strong baselines while consuming fewer tokens than most competing multi-agent methods, with voting records and arbitration traces offering transparent decision audits.

Figures & tables

Figure 1: Overview of the CAMP framework. Given a clinical note and candidate diagnoses, the attending physician first renders an initial diagnostic judgment, then assembles a case-adaptive specialist panel with directed focus areas. Each specialist quotes evidence from the note and casts a three-valued vote. A hybrid router directs each diagnosis to one of three resolution paths: strong consensus decisions are applied directly, weak consensus falls back to the attending physician’s initial judgment, and conflicts are escalated for evidence-based arbitration.

Model	Method	MIMIC-IV Diagnosis $\uparrow$		BHC Generation $\downarrow$
		Macro F1	Perfect Rate	Reasoning	Readability	Utility
Llama-3.1-70B ( Grattafiori et al., 2024 )	Single Agent	78.68	44.25	3.24	3.61	3.67
	Self-Consistency ( Wang et al., 2022b )	78.91	45.05	3.78	4.06	4.18
	Chain-of-Thought ( Wei et al., 2022 )	78.99	45.55	3.47	3.80	3.92
	Majority Voting ( Kaesberg et al., 2025 )	85.84	66.55	3.43	3.53	3.55
	LLM-as-a-Judge ( Zheng et al., 2023 )	83.13	57.40	4.02	4.16	4.22

Table 1: Performance comparison across methods and LLM backbones. Best results within each model are bold.

Figure 2: Alignment between specialists selected by CAMP and patients’ actual hospital services. Each row is normalized to sum to 100%. Specialist-specific services show strong concentration on the matching specialist, while general services exhibit broader distributions.

# Agents	Macro F1	Macro Precision	Macro Recall
1	64.84	62.42	70.83
2	67.77	66.33	72.33
3	68.97	67.50	74.33
4	67.37	67.00	71.33
5	68.17	67.83	72.33
6	67.22	66.83	70.83

Table 2: Diagnostic prediction performance as a function of specialist panel size, evaluated on a 200-case subset using Llama-3.1-70B.

Figure 3: Conflict resolution via attending-physician arbitration. The attending physician overrides a 2-to-1 majority Refuse vote by weighing the quality of competing specialist rationales rather than counting votes.

Figure 4: Token consumption vs. Macro F1. CAMP achieves the highest F1 while consuming fewer tokens than most of other multi-agent baselines.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Metric	Mean	Std	Min	Median	Max
Options per case	7.239	1.921	6	6	12
Correct labels per case	1.505	0.692	1	1	3
Distractors per case	5.734	1.253	5	5	9
Input words	621	382	46	537	3353
Input characters	3991	2447	325	3434	22391
Correct option ratio (%)	19.92	4.07	16.67	16.67	25.00

Appendix

Table 3: Descriptive statistics of the dataset.

Model	Method	Correct.	Coverage	Coherence	Linkage	Priority	Tracking	Density
Llama-3.1-70B ( Grattafiori et al., 2024 )	Single Agent	3.73	3.08	3.24	3.27	3.55	3.45	3.92
	Self-Consistency	4.00	3.78	3.88	3.88	3.98	3.84	4.00
	Chain-of-Thought	3.96	3.73	3.69	3.67	3.84	3.67	4.10
	Majority Voting	2.88	3.78	3.18	3.53	3.63	3.16	3.49
	LLM-as-a-Judge	3.65	4.80	3.63	3.71	4.39	3.82	3.94
	Devil’s Advocate	3.98	4.57	4.00	4.31	4.45	4.37	4.59

Appendix

Table 4: Supplementary BHC evaluation dimensions across LLM backbones. All scores are pooled ranks (lower is better). Best results within each model are bold.

Figure 5: Effect of specialist panel size on diagnostic prediction (evaluated on a 200-case subset). The dashed line marks where performance peaks.

Method	Tokens	Macro F1	Perfect Rate
CAMP (Ours)	15,458	0.8750	67.10
Majority Voting	6,252	0.8584	66.55
LLM-as-a-Judge	17,249	0.8313	57.40
Devil’s Advocate	18,673	0.8306	57.20
MedAgents	19,721	0.8277	56.70
Chain-of-Thought	1,854	0.7899	45.55

Appendix

Table 5: Per-note token consumption and diagnostic prediction performance on Llama-3.1-70B.

Explore similar work

A Structured Debate-Mixture-of-Agents Framework for Complex Clinical Diagnostic Decision Support
Sep 4, 2026Chang Xia, Leilei Ouyang, Huimin Wang +2Multi-Agent LLM Systems Clinical Decision Support
College of Computer Science, Sichuan University, Chengdu 610207, China · West China Biomedical Big Data Center, West China Hospital, Sichuan University, Chengdu 610041, China · Med-X Center for Informatics, Sichuan University, Chengdu 610041, China
ClinicalAgents: Multi-Agent Orchestration for Clinical Decision Making with Dual-Memory
Mar 27, 2026Zhuohan Ge, Haoyang Li, Yubo Wang +3Clinical Decision-Making Multi-Agent LLM Systems
The Hong Kong Polytechnic University, Hong Kong SAR, China · The Hong Kong University of Science and Technology, Hong Kong SAR, China
MeDxAgent: Multi-Agent Consultation for Interactive Medical Diagnosis
Jun 2, 2026Akshat Sanghvi, Naren Akash, Raza Imam +2Multi-Agent LLM Systems Medical Diagnosis
Microsoft Research India · Institute of AI and Innovation, AIG Hospitals · Mohamed bin Zayed University of Artificial Intelligence

Sep 4, 2026J/K move · Enter open · S save

A Structured Debate-Mixture-of-Agents Framework for Complex Clinical Diagnostic Decision Support

Chang Xia, Leilei Ouyang, Huimin Wang, Yong Zhao+1

College of Computer Science, Sichuan University, Chengdu 610207, China · West China Biomedical Big Data Center, West China Hospital, Sichuan University, Chengdu 610041, China · Med-X Center for Informatics, Sichuan University, Chengdu 610041, China

Multi-Agent LLM Systems Clinical Decision Support Multi-Agent Debate Clinical Reasoning LLM Agent Reliability Differential Diagnosis

Open paper AskPDF arXiv

Large language models (LLMs) show potential for medical tasks, but their single-turn question-answer format does not reflect how clinical diagnosis is performed in practice. As a result, they remain limited in complex diagnostic settings. We developed Debate-Mixture-of-Agents (DMoA), a novel multi-agent framework that structures role-based interaction to support iterative diagnostic reasoning. Base models and DMoA were evaluated on 297 rare disease cases and 1,719 challenging cases. Across both datasets, DMoA improved most likely diagnosis accuracy by 10.21 percentage points and safety rate by 11.36 percentage points over GPT-4o baseline. Ablation experiments showed that the gains were not simply due to the use of more models or longer outputs, but also reflected the contribution of the structured workflow. Further analyses examined how framework design, base model choice, and token budget affected performance. DMoA performed better with a 4*2 structure, stronger base models, and a larger token budget. These findings demonstrate the potential of DMoA for clinical tasks and suggest further investigation of multi-agent frameworks.

Abstract

Explore similar work

A Structured Debate-Mixture-of-Agents Framework for Complex Clinical Diagnostic Decision Support

ClinicalAgents: Multi-Agent Orchestration for Clinical Decision Making with Dual-Memory

MeDxAgent: Multi-Agent Consultation for Interactive Medical Diagnosis