Separating Capability from Confidence: Grounded Dual-State Calibration for GRPO-Trained Medical Vision-Language Models

Yangyang Xie; Ke Hao; Jiaqi Liu; Yun Gu; Xinglin Zhang

cs.CVSep 6, 2026

Separating Capability from Confidence: Grounded Dual-State Calibration for GRPO-Trained Medical Vision-Language Models

Authors: Yangyang Xie, Ke Hao, Jiaqi Liu, Yun Gu, Xinglin Zhang

Organizations: Shanghai Jiao Tong University, Shanghai, China · Medical Image Insights Co. Ltd., Shanghai, China

Abstract

Medical vision-language models (VLMs) require confidence that reflects both answer correctness and patient-specific visual evidence. Recent GRPO-based methods optimize verbalized confidence together with answer generation. However, this joint optimization may interfere with answer learning and drive confidence toward near-binary values. Verbalized confidence also provides no explicit assessment of visual support. We therefore separate capability learning from confidence estimation and propose DualRead. DualRead builds on the insight that reliability can be read from the actor's internal states at critical moments in the answering process. It freezes the GRPO-trained actor and combines pre-answer solvability with a post-answer assessment of the generated answer and its visual support. To further assess whether confidence reflects visual grounding, we introduce Counterfactual Confidence Grounding AUC (CCG-AUC). It measures whether confidence decreases when real-image substitution changes the actor from correct to incorrect. Across two VLM backbones and both in- and out-of-distribution medical VQA benchmarks, DualRead improves correctness discrimination and calibration over verbalized confidence while preserving answer accuracy. CCG-AUC reveals whether confidence responds to answer-relevant visual evidence rather than primarily to non-visual cues.

Figures & tables

Figure 1: Motivation of DualRead. Reward-shaped GRPO calibration jointly optimizes answer correctness and verbalized confidence, potentially causing confidence collapse (top). DualRead instead keeps the RLVR actor frozen and estimates reliability along the same natural generation trajectory (bottom), preserving the answer policy while producing confidence that better tracks empirical accuracy.

Figure 2: Overview of our framework. (a) DualRead estimates confidence from pre- and post-answer states of a frozen GRPO-trained VLM. (b) CCG-AUC evaluates confidence grounding using real and hard-negative image pairs sharing the same question.


Confidence proxy	Baseline	SLAKE-test ( $n{=}1061$ )				O.O.D. Averaged ( $n{=}4584$ )
Confidence proxy	Baseline	Acc.	AUROC	Natural CCG AUC	Strict CCG AUC	Acc.	AUROC	Natural CCG AUC	Strict CCG AUC
Verbalized token	Base	$.443$	$.579$	$.587$	$.423$	$.450$	$.696$	$\mathbf{.782}$	$\mathbf{.622}$
	RLCR	$.608_{\pm\!.009}$	$.822_{\pm\!.014}$	$.533_{\pm\!.031}$	$.514_{\pm\!.036}$	$.492_{\pm\!.012}$	$.613_{\pm\!.008}$	$.516_{\pm\!.022}$	$.509_{\pm\!.028}$
	DCPO	$.640_{\pm\!.003}$	$.789_{\pm\!.007}$	$.554_{\pm\!.021}$	$.540_{\pm\!.022}$	$.491_{\pm\!.004}$	$.622_{\pm\!.005}$	$.505_{\pm\!.018}$	$.509_{\pm\!.011}$
	VL-Cal.	$.591_{\pm\!.010}$	$.756_{\pm\!.026}$	$.553_{\pm\!.016}$	$.500_{\pm\!.034}$	$.497_{\pm\!.010}$	$.633_{\pm\!.020}$	$.566_{\pm\!.045}$	$.559_{\pm\!.042}$

Table 1: Reliability and confidence-grounding results on SLAKE and six OOD medical VQA datasets. OOD Acc./AUROC are equal-domain averages over all six datasets. Confidence proxies from GRPO (RLVR) to DualRead share the same frozen actor and thus identical Acc. Gray and blue denote Base and DualRead, respectively; bold indicates the best non-Acc. result.

Qwen3VL-2B
Variant	Components			AUROC $\uparrow$	Brier $\downarrow$	CCG-AUC $\uparrow$
Variant	PSR	State	Visual	AUROC $\uparrow$	Brier $\downarrow$	CCG-AUC $\uparrow$
PSR	$\checkmark$	$\times$	$\times$	$.926_{\pm\!.008}$	$.101_{\pm\!.005}$	$.733_{\pm\!.044}$
P-State	$\times$	$\checkmark$	$\times$	$.928_{\pm\!.006}$	$.099_{\pm\!.007}$	$.686_{\pm\!.027}$
PER	$\times$	$\checkmark$	$\checkmark$	$.947_{\pm\!.003}$	$.084_{\pm\!.001}$	$.760_{\pm\!.038}$
DualRead	$\checkmark$	$\checkmark$	$\checkmark$	$\mathbf{.952}_{\pm\!.004}$	$\mathbf{.081}_{\pm\!.004}$	$\mathbf{.784}_{\pm\!.009}$

Table 2: Component ablation of DualRead. PER combines P-State and P-Visual; DualRead further combines PER with PSR. CCG-AUC here is the Natural variant on SLAKE-test.

Figure 3: Reliability diagrams on SLAKE-test. The dashed diagonal denotes perfect calibration.

Figure 4: Qualitative examples of counterfactual confidence grounding. Natural CCG allows the answer to change after same-question image replacement, whereas Strict CCG retains the answer while flipping its correctness.

Qwen3VL-2B, ID: SLAKE-test
Methods	Cost	AUROC $\uparrow$	Brier $\downarrow$	AURC $\downarrow$
SC@2	$2t$	$0.742_{\pm.006}$	$0.165_{\pm.002}$	$0.234_{\pm.005}$
SC@8	$8t$	$0.831_{\pm.003}$	$0.140_{\pm.003}$	$0.179_{\pm.006}$
Semantic unc.	$8t$	$0.831_{\pm.002}$	$0.135_{\pm.001}$	$0.179_{\pm.007}$
BICR	$t{+}2\rho$	$0.934_{\pm.002}$	$0.099_{\pm.003}$	$0.101_{\pm.004}$
DualRead	$t{+}\tau$	$\mathbf{0.952}_{\pm.004}$	$\mathbf{0.083}_{\pm.002}$	$\mathbf{0.094}_{\pm.003}$

Table 3: Compute–reliability trade-off.

t

is the required greedy generation,

\rho

one prompt-only view, and

\tau

DualRead’s natural-trajectory replay. Lower AURC means better selective-prediction risk at matched coverage.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

GRPO actor training (shared; reward differs per method)
Advantage estimator	GRPO, group size $G=8$
Rollout temperature / top- $p$	$1.0$ / $1.0$
PPO clip (low / high)	$0.2$ / $0.2$
KL penalty	disabled ( $\mathrm{coef}=0$ )
Optimizer	AdamW, lr $1\!\times\!10^{-6}$ , $\beta=(0.9,0.95)$
LR schedule / warmup	constant, $5\%$ warmup

Appendix

Table 4: Frozen hyperparameters. † MedGemma uses a 512-token prompt limit without SLAKE truncation.

Dataset	Role	QA	Images	Open	Closed
SLAKE-train	actor/head fit	4919	450	2976	1943
SLAKE-val.	selection only	1053	96	631	422
SLAKE-test	ID report	1061	96	645	416
VQA-RAD	OOD report	600	270	236	364
PathVQA	OOD report	1000	706	500	500
PMC-VQA	OOD report	1000	1000	994	6

Appendix

Table 5: Frozen dataset inventory; images are unique identifiers for leakage control and clustered resampling.

Dataset	Role	QA	Images	Open	Closed
SLAKE-train	fit	4919	450	2976	1943
SLAKE-val.	select	1053	96	631	422
SLAKE-test	ID	1061	96	645	416
VQA-RAD	OOD	600	270	236	364
PathVQA	OOD	1000	706	500	500
PMC-VQA	OOD	1000	1000	994	6

Appendix

Table 6: Frozen data and confidence-channel audits; invalid confidence is retained at

c=0.5

Backbone	Contrast (split)	$\Delta$ AUROC	$\Delta$ Brier
Qwen3VL-2B	P-Visual $-$ P-State (val.)	$+.0183_{\pm.0076}$	$-.0157_{\pm.0059}$
	DualRead $-$ RLCR (ID)	$+.130_{\pm.015}$	$+.067_{\pm.013}$
	DualRead $-$ RLCR (OOD)	$+.020_{\pm.012}$	$+.077_{\pm.011}$
MedGemma1.5-4B	P-Visual $-$ P-State (val.)	$+.0114_{\pm.0053}$	—
	DualRead $-$ RLCR (ID)	$+.205_{\pm.024}$	$+.110_{\pm.020}$
	DualRead $-$ RLCR (OOD)	$+.075_{\pm.023}$	$+.102_{\pm.010}$

Appendix

Table 7: Paired image-cluster bootstrap contrasts (mean

\pm

std).

Backbone	Method	SLAKE	VQA-RAD	PathVQA
		Acc./AUROC (top); ECE-15/Brier (bottom)
Qwen3VL-2B	Base	$.443$ / $.579$ $.539$ / $.515$	$.340$ / $.644$ $.550$ / $.510$	$.295$ / $.731$ $.517$ / $.433$
	RLCR	$.608_{\pm\!.009}$ / $.822_{\pm\!.014}$ $.150_{\pm\!.012}$ / $.150_{\pm\!.012}$	$.440_{\pm\!.009}$ / $.709_{\pm\!.006}$ $.312_{\pm\!.010}$ / $.312_{\pm\!.010}$	$.330_{\pm\!.007}$ / $.820_{\pm\!.008}$ $.218_{\pm\!.019}$ / $.218_{\pm\!.019}$
	DCPO	$.640_{\pm\!.003}$ / $.789_{\pm\!.007}$ $.190_{\pm\!.016}$ / $.189_{\pm\!.016}$	$.448_{\pm\!.011}$ / $.731_{\pm\!.010}$ $.277_{\pm\!.003}$ / $.277_{\pm\!.003}$	$.335_{\pm\!.010}$ / $.800_{\pm\!.019}$ $.214_{\pm\!.009}$ / $.214_{\pm\!.009}$
	VL-Cal.	$.591_{\pm\!.010}$ / $.756_{\pm\!.026}$ $.183_{\pm\!.048}$ / $.207_{\pm\!.036}$	$.443_{\pm\!.017}$ / $.709_{\pm\!.006}$ $.282_{\pm\!.057}$ / $.294_{\pm\!.034}$	$.332_{\pm\!.012}$ / $.775_{\pm\!.007}$ $.276_{\pm\!.085}$ / $.273_{\pm\!.060}$
	GRPO (RLVR)	$.639_{\pm\!.009}$ / $.535_{\pm\!.059}$ $.359_{\pm\!.012}$ / $.365_{\pm\!.005}$	$.453_{\pm\!.006}$ / $.490_{\pm\!.019}$ $.538_{\pm\!.017}$ / $.531_{\pm\!.028}$	$.339_{\pm\!.003}$ / $.465_{\pm\!.061}$ $.634_{\pm\!.044}$ / $.621_{\pm\!.067}$

Appendix

Table 8: Reliability on SLAKE, VQA-RAD, and PathVQA. Cells show Acc./AUROC over ECE-15/Brier; bold excludes Acc.

Backbone	Method	PMC-VQA	OmniMed	ReXVQA	VQA-Med19-C	OOD6
		Acc./AUROC (top); ECE-15/Brier (bottom)
Qwen3VL-2B	Base	$.449$ / $.681$ $.468$ / $.437$	$.714$ / $.705$ $.225$ / $.228$	$.493$ / $\mathbf{.617}$ $.441$ / $.426$	$.406$ / $\mathbf{.799}$ $.429$ / $.361$	$.450$ / $.696$ $.438$ / $.399$
	RLCR	$.511_{\pm\!.010}$ / $.563_{\pm\!.005}$ $.433_{\pm\!.001}$ / $.432_{\pm\!.001}$	$.709_{\pm\!.003}$ / $.561_{\pm\!.020}$ $.289_{\pm\!.005}$ / $.289_{\pm\!.005}$	$.285_{\pm\!.044}$ / $.500_{\pm\!.014}$ $.615_{\pm\!.022}$ / $.615_{\pm\!.022}$	$.677_{\pm\!.039}$ / $.523_{\pm\!.024}$ $.312_{\pm\!.031}$ / $.312_{\pm\!.031}$	$.492_{\pm\!.012}$ / $.613_{\pm\!.008}$ $.363_{\pm\!.003}$ / $.363_{\pm\!.003}$
	DCPO	$.501_{\pm\!.005}$ / $.588_{\pm\!.023}$ $.413_{\pm\!.024}$ / $.411_{\pm\!.025}$	$.700_{\pm\!.012}$ / $.613_{\pm\!.011}$ $.289_{\pm\!.021}$ / $.288_{\pm\!.021}$	$.275_{\pm\!.030}$ / $.488_{\pm\!.022}$ $.572_{\pm\!.034}$ / $.571_{\pm\!.034}$	$.688_{\pm\!.031}$ / $.512_{\pm\!.001}$ $.323_{\pm\!.018}$ / $.323_{\pm\!.018}$	$.491_{\pm\!.004}$ / $.622_{\pm\!.005}$ $.348_{\pm\!.006}$ / $.347_{\pm\!.007}$
	VL-Cal.	$.480_{\pm\!.008}$ / $.622_{\pm\!.029}$ $.343_{\pm\!.027}$ / $.354_{\pm\!.025}$	$.716_{\pm\!.010}$ / $.648_{\pm\!.047}$ $.240_{\pm\!.084}$ / $.276_{\pm\!.070}$	$.352_{\pm\!.025}$ / $.508_{\pm\!.008}$ $.444_{\pm\!.067}$ / $.436_{\pm\!.066}$	$.661_{\pm\!.039}$ / $.538_{\pm\!.053}$ $.253_{\pm\!.028}$ / $.278_{\pm\!.017}$	$.497_{\pm\!.010}$ / $.633_{\pm\!.020}$ $.306_{\pm\!.024}$ / $.318_{\pm\!.016}$
	GRPO (RLVR)	$.514_{\pm\!.006}$ / $.532_{\pm\!.045}$ $.409_{\pm\!.132}$ / $.427_{\pm\!.099}$	$.709_{\pm\!.008}$ / $.567_{\pm\!.105}$ $.244_{\pm\!.080}$ / $.269_{\pm\!.037}$	$.283_{\pm\!.007}$ / $.492_{\pm\!.012}$ $.642_{\pm\!.134}$ / $.629_{\pm\!.156}$	$.667_{\pm\!.018}$ / $.495_{\pm\!.008}$ $.368_{\pm\!.070}$ / $.373_{\pm\!.079}$	$.494_{\pm\!.004}$ / $.507_{\pm\!.008}$ $.472_{\pm\!.056}$ / $.475_{\pm\!.051}$

Appendix

Table 9: Reliability on PMC-VQA, OmniMedVQA, ReXVQA, VQA-Med 2019, and OOD6; see Table 8 for format.

Panel of Figure 5 — Figure 5: VQA-RAD reliability ( $n=600$ ; short answer). Qwen rows precede MedGemma rows.

Panel of Figure 6 — Figure 6: PathVQA reliability ( $n=1000$ ; short answer). Qwen rows precede MedGemma rows.

Panel of Figure 7 — Figure 7: PMC-VQA reliability ( $n=1000$ ; option visible). Qwen rows precede MedGemma rows.

Panel of Figure 8 — Figure 8: OmniMedVQA reliability ( $n=920$ ; option visible). Qwen rows precede MedGemma rows.

Panel of Figure 9 — Figure 9: ReXVQA reliability ( $n=1000$ ; option visible). Qwen rows precede MedGemma rows.

Panel of Figure 10 — Figure 10: VQA-Med 2019 reliability ( $n=64$ ; closed answer). Qwen rows precede MedGemma rows.

Open questions
Backbone	Dataset	Method	Acc $\uparrow$	AUROC $\uparrow$	ECE $\downarrow$	Brier $\downarrow$	AURC $\downarrow$	[email protected] $\downarrow$
Qwen3VL-2B	SLAKE	RLCR	$.480_{\pm.011}$	$.885_{\pm.016}$	$.117_{\pm.018}$	$.117_{\pm.018}$	$.261_{\pm.026}$	$.083_{\pm.023}$
	SLAKE	DualRead	$.518_{\pm.004}$	$\mathbf{.973}_{\pm.006}$	$\mathbf{.039}_{\pm.001}$	$\mathbf{.063}_{\pm.005}$	$\mathbf{.155}_{\pm.003}$	$\mathbf{.005}_{\pm.003}$
	VQA-RAD	RLCR	$.143_{\pm.009}$	$\mathbf{.630}_{\pm.013}$	$\mathbf{.210}_{\pm.016}$	$.210_{\pm.016}$	$.792_{\pm.013}$	$.126_{\pm.027}$
	VQA-RAD	DualRead	$.158_{\pm.005}$	$.554_{\pm.047}$	$.223_{\pm.029}$	$\mathbf{.205}_{\pm.025}$	$\mathbf{.775}_{\pm.018}$	$\mathbf{.071}_{\pm.009}$
MedGemma1.5-4B	SLAKE	RLCR	$.552_{\pm.034}$	$.828_{\pm.024}$	$.164_{\pm.024}$	$.164_{\pm.024}$	$.247_{\pm.046}$	$.124_{\pm.041}$

Appendix

Table 10: Open/closed reliability for RLCR and DualRead; bold excludes Acc.

		VQA-RAD		PathVQA		VQA-Med 2019 closed
Backbone	Method	Nat. $\uparrow$	Strict $\uparrow$	Nat. $\uparrow$	Strict $\uparrow$	Nat. $\uparrow$	Strict $\uparrow$
Qwen3VL-2B	Base	$\mathbf{.786}$	$.700$	$.692$	$.500$	$\mathbf{.867}$	$.667$
	RLCR	$.500_{\pm\!.000}$	$.500_{\pm\!.000}$	$.521_{\pm\!.043}$	$.499_{\pm\!.040}$	$.528_{\pm\!.048}$	$.528_{\pm\!.048}$
	DCPO	$.519_{\pm\!.032}$	$.533_{\pm\!.058}$	$.492_{\pm\!.013}$	$.487_{\pm\!.022}$	$.506_{\pm\!.092}$	$.506_{\pm\!.092}$
	VL-Cal.	$.495_{\pm\!.064}$	$.486_{\pm\!.105}$	$.608_{\pm\!.065}$	$.593_{\pm\!.085}$	$.595_{\pm\!.135}$	$.597_{\pm\!.134}$
	GRPO (RLVR)	$.521_{\pm\!.036}$	$.500_{\pm\!.000}$	$.567_{\pm\!.115}$	$.548_{\pm\!.082}$	$.500_{\pm\!.000}$	$.500_{\pm\!.000}$

Appendix

Table 11: Grounding evidence: (a) OOD CCG-AUC, (b) paired SLAKE audit, and (c) eligibility. Bold marks the best confidence score.

Confidence Estimation for VLMs

See all 28 papers

Overconfidence and Calibration in Medical VQA: Empirical Findings and Hallucination-Aware Mitigation
Apr 2, 2026Ji Young Byun, Young-Jin Park, Jean-Philippe Corbeil +1Vision-Language Models VLM Hallucination
Johns Hopkins University, School of Medicine · Massachusetts Institute of Technology · Microsoft Healthcare & Life Sciences
Just how sure are you? Improving Verbalized Uncertainty Calibration in Medical VQA
Jun 25, 2026Eren Senoglu, Federico Toschi, Nicolo Brunello +2Medical VQA Language Model Calibration
Politecnico di Milano
The Mirage of Calibrated Confidence: Trajectory-Independence of Verbalized Confidence in Vision-Language Models
Sep 16, 2026Jisoo Yang, Jaeho Han, Trung X. Pham +1VLM Evaluation Confidence Estimation in Language Models
Department of Artificial Intelligence, Chung-Ang University, Republic of Korea · Van Lang University, Ho Chi Minh City, Vietnam

Apr 2, 2026J/K move · Enter open · S save

Overconfidence and Calibration in Medical VQA: Empirical Findings and Hallucination-Aware Mitigation

Ji Young Byun, Young-Jin Park, Jean-Philippe Corbeil, Asma Ben Abacha

Johns Hopkins University, School of Medicine · Massachusetts Institute of Technology · Microsoft Healthcare & Life Sciences

Vision-Language Models VLM Hallucination Medical VQA Language Model Calibration VLM Calibration

Open paper AskPDF arXiv

As vision-language models (VLMs) are increasingly deployed in clinical decision support, more than accuracy is required: knowing when to trust their predictions is equally critical. Yet, a comprehensive and systematic investigation into the overconfidence of these models remains notably scarce in the medical domain. We address this gap through a comprehensive empirical study of confidence calibration in VLMs, spanning three model families (Qwen3-VL, InternVL3, LLaVA-NeXT), three model scales (2B--38B), and multiple confidence estimation prompting strategies, across three medical visual question answering (VQA) benchmarks. Our study yields three key findings: First, overconfidence persists across model families and is not resolved by scaling or prompting, such as chain-of-thought and verbalized confidence variants. Second, simple post-hoc calibration approaches, such as Platt scaling, reduce calibration error and consistently outperform the prompt-based strategy. Third, due to their (strict) monotonicity, these post-hoc calibration methods are inherently limited in improving the discriminative quality of predictions, leaving AUROC at the same level. Motivated by these findings, we investigate hallucination-aware calibration (HAC), which incorporates vision-grounded hallucination detection signals as complementary inputs to refine confidence estimates. We find that leveraging these hallucination signals improves both calibration and AUROC, with the largest gains on open-ended questions. On closed-ended questions, format-specific signals such as the logit margin show more informative. Overall, our findings suggest post-hoc calibration as standard practice for medical VLM deployment over raw confidence estimates, and highlight the practical usefulness of hallucination signals to enable more reliable use of VLMs in medical VQA.

Abstract

Confidence Estimation for VLMs

Explore similar work

Overconfidence and Calibration in Medical VQA: Empirical Findings and Hallucination-Aware Mitigation

Just how sure are you? Improving Verbalized Uncertainty Calibration in Medical VQA

The Mirage of Calibrated Confidence: Trajectory-Independence of Verbalized Confidence in Vision-Language Models