Miscalibrated confidence scores are a practical obstacle to deploying AI in clinical settings. A model that is always overconfident offers no useful signal for deferral. We present a multi-agent framework that combines domain-specific specialist agents with Two-Phase Verification (Wu et al., 2024) and S-Score Weighted Fusion to improve both calibration and discrimination in medical multiple-choice question answering. Four specialist agents (respiratory, cardiology, neurology, gastroenterology) generate independent diagnoses using Qwen2.5-7B-Instruct. Each diagnosis undergoes a two-phase self-verification process that measures internal consistency and produces a Specialist Confidence Score (S-score). The S-scores drive a weighted fusion strategy that selects the final answer and calibrates the reported confidence. We evaluate on high-disagreement subsets of MedQA-USMLE and MedMCQA (100 and 250 questions). All results are specific to this filtered regime. On MedQA-250, the full system achieves ECE = 0.091 (74.4% reduction over the single-specialist baseline) and AUROC = 0.630 (+0.056) at 59.2% accuracy. Calibration gains of 49-74% hold across all four settings. Ablation analysis reveals that Two-Phase Verification drives ECE reduction while multi-agent reasoning drives AUROC improvement, suggesting that consistency checking and ensemble aggregation address different failure modes of LLM uncertainty. Whether the resulting confidence signal is sufficient to support clinical deferral decisions in practice remains a direction for future investigation.
Figures & tables
Figure 1: The MARC framework pipeline. A medical question is distributed to four domain-specific specialist agents (Component 1), each producing an answer and reasoning chain. Two-Phase Consistency Verification (Component 2; Wu et al. 2024 ) extracts factual claims, answers them independently and with the original reasoning in context, and measures cross-condition inconsistency to derive an S-score Sk per specialist. S-Score Weighted Fusion (Component 3) selects the final answer by vote-count-weighted mean S-score and derives calibrated confidence C^ from the vote fraction and S-score distribution.
Dataset
Configuration
Acc
Δ Acc
ECE
Δ ECE
AUROC
Δ AUROC
Avg Conf
Time (min)
MedQA-100
C1: Single Specialist
52.0%
—
0.374
—
0.537
—
0.886
39
Avg. 4 Specialists
54.2%
—
0.360
—
0.552
—
—
—
C2: Single + Two-Phase
52.0%
+0.0
0.185
− 51%
0.678
+ 0.141
0.430
67
C3: Multi + S-Score (No 2P)
55.0%
+3.0pp
0.356
− 5%
0.578
+ 0.041
0.896
152
C4: Full System
59.0%
+7.0pp
0.098
− 73%
0.645
+ 0.108
0.549
249
MedQA-250
C1: Single Specialist
54.4%
—
0.355
—
0.574
—
0.899
116
Table 1: Performance comparison across all four datasets and configurations. Best value in each column group is bold . Δ columns show change relative to Config 1 baseline. The italic Avg. 4 Specialists row reports the mean performance across all four individual specialists running independently (see Table 4 for per-specialist details). Avg Conf = mean predicted confidence score. Time = wall-clock inference time in minutes on a single NVIDIA RTX 5090 GPU.
Figure 2: Accuracy (top row), ECE (middle row), and AUROC (bottom row) for each of the four datasets and all four configurations. Green-outlined bars indicate the best value in each panel. Config 4 (Full System, dark blue) achieves the best ECE across all four datasets, and the best accuracy on the MedQA subsets. On MedMCQA, Config 3 achieves the best accuracy at the cost of calibration, illustrating the knowledge-recall challenge for verification-based methods.
Figure 3: Reliability diagrams for all four datasets. Each panel shows calibration curves for all four configurations against the perfect-calibration diagonal. Across all datasets, Config 4 (solid blue squares) lies closest to the diagonal, confirming that Two-Phase Verification is the dominant calibration driver regardless of question type or dataset size.
Figure 4: ROC curves for all four datasets. Each panel plots the four configuration curves with the random baseline (diagonal). Config 4 achieves the highest AUROC on MedQA-250 and MedMCQA-250. On MedQA-100, Config 2 (single specialist with verification) achieves the highest AUROC, and on MedMCQA-100, Config 3 (multi-agent without verification) is best, illustrating that discrimination benefits from multi-agent fusion scale more reliably with dataset size and question type.
Figure 5: Calibration analysis of Qwen2.5-7B across all four evaluation sets. Columns represent configurations C1–C4; each dataset occupies two sub-rows. Top sub-row : stacked confidence frequency histogram (blue = correct answer, red = wrong answer). Bottom sub-row : calibration histogram with 5%-wide bins for visual resolution, where bar height represents observed accuracy within that bin; the dashed diagonal represents perfect calibration. Config 4 (Full System) consistently produces the most evenly spread confidence distributions and aligns most closely with the perfect-calibration diagonal across all datasets.
Component Added
Δ Acc
Δ ECE
Δ AUROC
Two-Phase Verif. (C1 → C2)
0.0 pp
− 0.177 ( − 49.9%)
− 0.018
Multi-Agent (C1 → C3)
+2.8 pp
− 0.019 ( − 5.4%)
+0.013
Two-Phase added to Multi (C3 → C4)
+2.0 pp
− 0.245 ( − 72.9%)
+0.043
Full System (C1 → C4)
+4.8 pp
− 0.264 ( − 74.4%)
+0.056
Table 2: Ablation summary on MedQA-250 ( n=250 ). Each row reports the marginal effect of adding one component.
Figure 6: (A) ECE reduction percentage and (B) AUROC change for each component transition across all four datasets. Adding the full system (C1 → C4, dark blue) consistently delivers the largest ECE reduction (49–74%) across all settings. Multi-agent reasoning (C1 → C3, orange) drives the largest AUROC gains on MedQA subsets, while verification components dominate calibration improvement throughout.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
Config
Accuracy 95% CI
ECE 95% CI
AUROC 95% CI
MedQA-100
C1: Single Specialist
[42.0, 62.0]
[0.269, 0.463]
[0.470, 0.605]
C2: Single + Two-Phase
[42.0, 62.0]
[0.103, 0.272]
[0.572, 0.781]
C3: Multi + S-Score (No 2P)
[45.0, 65.0]
[0.244, 0.438]
[0.502, 0.652]
C4: Full System
[49.0, 69.0]
[0.050, 0.194]
[0.531, 0.755]
MedQA-250
C1: Single Specialist
[48.4, 60.4]
[0.291, 0.412]
[0.533, 0.616]
C2: Single + Two-Phase
[48.4, 60.4]
[0.133, 0.245]
[0.486, 0.626]
Appendix
Table 3: Bootstrap confidence intervals (95%, n=10,000 resamples) for all metrics across all four datasets and configurations (C1–C4).
MedQA-100
MedQA-250
MedMCQA-100
MedMCQA-250
Specialist
Acc.
ECE
AUROC
Acc.
ECE
AUROC
Acc.
ECE
AUROC
Acc.
ECE
AUROC
Respiratory
52.0%
0.374
0.537
54.4%
0.355
0.574
45.0%
0.469
0.501
42.8%
0.469
0.536
Cardiology
57.0%
0.343
0.541
57.6%
0.322
0.537
47.0%
0.451
0.557
42.8%
0.469
0.562
Neurology
55.0%
0.343
0.587
58.4%
0.309
0.555
48.0%
0.418
0.514
44.8%
0.448
0.543
Gastroenterology
53.0%
0.380
0.543
58.4%
0.315
0.552
51.0%
0.368
0.589
48.4%
0.399
0.570
Average
54.2%
0.360
0.552
57.2%
0.325
0.555
47.8%
0.426
0.540
44.7%
0.446
0.553
Appendix
Table 4: Per-specialist performance (Accuracy, ECE, AUROC) across all four evaluation sets. Each specialist runs independently with no verification and no fusion (equivalent to Config 1 setup). The Average row is the mean across all four specialists and serves as an alternative single-specialist baseline. Respiratory corresponds to the Config 1 baseline in Table 1 .
Multimodal large language models (MLLMs) applied to Medical Visual Question Answering (VQA) tend to produce overconfident outputs regardless of actual correctness, and existing verbalized confidence calibration methods, developed primarily for text only LLMs, do not account for the multimodal nature of medical image understanding. This work proposes a training based framework that finetunes MLLMs to improve their calibration using a composite loss function combining a Brier style calibration term, an anchor regularizer that prevents confidence collapse toward extreme values, a contrastive image text alignment term, and a KL based model stabilization term. The alignment signal is derived from a 2×2 factorial perturbation design that crosses image presence with text integrity, probing the reliance of the model on visual modality input versus language priors. Finally, a top K KL divergence regularizer is used to protect the answering ability of the model during finetuning. Across three Medical VQA benchmarks and two architectures (MedGemma 4B IT and Qwen2 VL 7B Instruct), our method reduces calibration error by 60% or more, and improves discrimination by 26% or more, while preserving predictive accuracy. On average across benchmarks, the technique outperforms prompting based, sampling based, and training based approaches, and ablation experiments confirm that each component of the loss function is indeed necessary for improving the calibration. All code for the experiments is publicly available.
Multimodal Large Language Models (MLLMs) show great potential in medical tasks, but their elicited confidence often misaligns with actual accuracy, potentially leading to misdiagnosis or overlooking correct advice. This study presents the first comprehensive analysis of the relationship between accuracy and confidence in medical MLLMs. It proposes a novel method that combines Multi-Strategy Fusion-Based Interrogation (MS-FBI) with auxiliary expert LLM assessment, aiming to improve confidence calibration in Medical Visual Question Answering (VQA). Experiments demonstrate that our method reduces the Expected Calibration Error (ECE) by an average of 40% across three Medical VQA datasets, significantly enhancing MLLMs' reliability. The findings highlight the importance of domain-specific calibration for MLLMs in healthcare, offering a more trustworthy solution for AI-assisted diagnosis.
Yuetian Du, Yucheng Wang, Ming Kong +4
College of Computer Science and Technology, Zhejiang University · School of Computer Science and Technology, Xidian University · Zhihui Medical Technology (Shanghai) Co., Ltd., Shanghai, China +1
Multi-agent LLM systems for medical question answering often treat consensus as a reliability signal: if multiple agents agree on an answer, it is presumed trustworthy. However, answer-level consensus does not entail reasoning-level alignment. We introduce CARA (Cross-Agent Reasoning Alignment), a family of automated metrics that measure whether agents who agree on an answer also agree on the reasoning. Applying CARA to a standard debate system on two medical QA benchmarks, MedQA-USMLE and MedThink-Bench, we identify the consistency illusion: a failure mode where debate reduces detectable contradictions between agents while simultaneously decreasing the semantic similarity of their reasoning chains; agents appear to agree more but reason less consistently. To improve this misalignment, we propose the Grounded Debate Protocol (GDP), a prompt-level intervention that requires agents to commit to named medical facts and take explicit stances on other agents' claims. GDP produces large, consistent alignment improvements, with Cohen's d ranging from +1.43 to +1.99, across two datasets and two backbone models, without adding LLM calls or modifying system architecture. Our results motivate cross-agent reasoning alignment as a quantity to audit alongside accuracy in safety-critical domains.