Aligning Medical Vision Language Models at the Capability Boundary with Entropy Adaptive Rubric Grounding

Hongyu Bai; Jiaqi Zeng; Chunzheng Zhu

cs.CVOct 29, 2026

Aligning Medical Vision Language Models at the Capability Boundary with Entropy Adaptive Rubric Grounding

Authors: Hongyu Bai, Jiaqi Zeng, Chunzheng Zhu

Organizations: Shanghai Jiao Tong University Shanghai, China · Hunan University Changsha, China

Abstract

Medical diagnostic reasoning requires verifiable links between visual evidence and clinical knowledge, yet current medical vision language models are aligned with objectives that reward the wrong behavior. Uniform supervised fine tuning lets easy cases dominate updates while clinically critical boundary cases saturate prematurely. Sparse rewards based only on final-answer correctness cannot assess anatomical grounding or clinical logic, inviting reward hacking through answer shortcuts. We propose MedCBR, a two-stage alignment framework coupling boundary-aware preference optimization with rubric-grounded reinforcement learning. First, we construct counterfactual boundary preferences at the model's capability frontier using empirical pass rates and preference pairs that share reasoning prefixes but diverge at decisive diagnostic branches. Difficulty Weighted Boundary Preference Optimization (DBPO) concentrates learning on this calibrated boundary region. Second, we distill reusable clinical rubrics from boundary trajectories and activate them according to prediction entropy. Entropy Gated Grounded Policy Optimization (EGPO) rewards answer correctness, anatomical grounding, and reasoning fidelity while suppressing rubric noise on reliable samples. A reference policy anchored to the boundary-aligned model couples population-level boundary priors with instance-level uncertainty, making diagnostic reasoning more accurate and clinically grounded. Experiments on six medical benchmarks show consistent improvements over strong general, specialized, and reinforcement learning baselines, reaching 61.2% average accuracy with the largest gains on challenging differential diagnosis tasks. Ablations show super-additive gains: combining the two stages exceeds the sum of their individual improvements.

Figures & tables

Figure 1. Motivation of MedCBR . Standard SFT distributes gradient uniformly across all tokens, causing loss saturation on clinically critical hard samples (left). Sparse binary rewards in conventional RL provide no signal on whether reasoning is grounded in correct anatomical evidence, enabling reward hacking via answer shortcuts (right). MedCBR addresses both via boundary aware preference alignment and entropy adaptive rubric grounding.

Figure 2. Overview of MedCBR . Stage 1 ( CBP , off policy): from the SFT model, $N$ trajectories are sampled per query, empirical pass rates define the capability boundary band, and counterfactual preference pairs diverging at critical diagnostic branch points are mined for difficulty weighted DPO. Stage 2 ( MRG , on policy): meta rubrics distilled from stage one trajectories are dynamically selected via entropy gating, composing structured process rewards for GRPO optimization anchored to $\pi_{\textsc{cbp}}$ .

Method	RAD	SLK	Path	PMC	MMU	MedX	Avg
General Vision Language Models
LLaVA Med ( Li et al., 2023 )	61.5	64.2	52.1	41.3	38.6	18.7	46.1
Qwen2.5 VL 7B ( Shen et al., 2025 )	66.8	70.4	56.3	47.9	45.2	22.4	51.5
InternVL2.5 8B ( Chen et al., 2024b )	68.1	72.0	57.8	49.5	47.0	23.6	53.0
Medical Specialized Models
RadFM ( Wu et al., 2025b )	63.9	68.7	54.6	44.0	40.1	19.9	48.5

Table 1. Comprehensive comparison on six medical multimodal benchmarks (accuracy %). Columns: VQA-RAD (RAD), SLAKE (SLK), PathVQA (Path), PMC-VQA (PMC), MMMU-Med (MMU), MedXpertQA (MedX). Bold best, underlined second best.

Configuration	RAD	SLK	Path	PMC	MMU	MedX	Avg	$\bm{\Delta_{\textit{syn}}}$
SFT Baseline (Qwen2.5 VL 7B)	66.8	70.4	56.3	47.9	45.2	22.4	51.5	–
+ CBP only	72.3	78.1	62.8	53.6	53.8	26.8	57.9	–
+ MRG only (anchor $\pi_{\textsc{sft}}$ )	71.5	77.2	61.9	52.8	52.4	26.1	57.0	–
+ CBP $\to$ MRG (anchor $\pi_{\textsc{cbp}}$ )	74.6	82.3	65.7	56.8	58.4	29.1	61.2	+2.8
+ CBP $\to$ MRG (anchor $\pi_{\textsc{sft}}$ )	73.8	80.9	64.8	55.9	56.9	28.4	60.1	+1.7
CBP component ablation:

Table 2. Component ablation of MedCBR on the 7B backbone (accuracy %). Column abbreviations follow Table 1 . “

\Delta_{\textit{syn}}

” denotes the synergy gain confirming super-additivity.

Figure 3. Stage-wise data composition and super-additive gain decomposition. Left: specialty distribution shifts from radiology-dominated boundary pairs in Stage 1 toward over-sampled differential-diagnosis cases in Stage 2. Right: the combined CBP

\to

MRG pipeline exceeds the sum of individual stage gains, with surplus

\Delta_{\textit{syn}}

largest on the hardest benchmarks.

Figure 4. Training dynamics of MedCBR . Left: the full model’s composite reward climbs to

{\sim}

0.80 while the no-CBP ablation saturates near 0.38, a

2\times

gap. Middle: trajectory length peaks early then compresses as reasoning sharpens, whereas the ablation stays verbose. Right: per-benchmark rubric-gain of entropy-driven selection, largest on MMMU-Med.

Method	Time	Mem	Trainable	Avg.
Vanilla GRPO ( Shao et al., 2024 )	28	62	42M	57.1
Med R1 ( Lai et al., 2026 )	32	64	42M	57.6
MedVLM R1 ( Pan et al., 2025 )	36	68	42M	58.6
V RFT PRA ( Yang et al., 2026e )	40	72	42M	59.9
MedCBR Stage 1	18	58	42M	57.9
MedCBR Stage 2	24	66	42M	–

Table 3. Computational comparison with RL baselines on 8

\times

A100 GPUs (Time in GPU hours, Mem in GB).

Medical VLMs

Medical vision-language models connect clinical images with text for diagnosis, reporting, question answering, and image grounding.

See all 479 papers

Mar 1, 2026

ClinCoT: Clinical-Aware Visual Chain-of-Thought for Medical Vision Language Models

Medical Vision-Language Models have shown promising potential in clinical decision support, yet they remain prone to factual hallucinations due to insufficient grounding in localized pathological evidence. Existing medical alignment methods primarily operate at the response level through preference optimization, improving output correctness but leaving intermediate reasoning weakly connected to visual regions. Although chain-of-thought (CoT) enhances multimodal reasoning, it remains largely text-centric, limiting effective integration of clinical visual cues. To address this gap, we propose ClinCoT, a clinical-aware visual chain-of-thought framework that transforms preference optimization from response-level correction to visual-driven reasoning. We introduce an automatic data generation pipeline that constructs clinically grounded preference pairs through reasoning with hypotheses-driven region proposals. Multiple Med-LLMs evaluators rank and assign scores to each response, and these rankings serve as supervision to train the target model. We further introduce a scoring-based margin-aware optimization strategy that incorporates both preference ranking and score difference to refine region-level reasoning trajectories. To maintain alignment as the model's policy evolves during training, we adopt an iterative learning scheme that dynamically regenerates preference data. Extensive experiments on three medical VQA and report generation benchmarks demonstrate that ClinCoT consistently improves factual grounding and achieves superior performance compared with existing preference-based alignment methods.

Xiwei Liu, Yulong Li, Xinlin Zhuang +5

Mohamed bin Zayed University of Artificial Intelligence · East China Normal University

Vision-Language Models Medical VLMs VLM Hallucination Mitigation Medical VQA

Ask PDF

Oct 3, 2026

TrustMed-RL: Long-Horizon Reinforcement Learning for Evidence-Grounded Clinical Diagnosis

Medical language models can produce correct diagnoses despite incomplete investigations and unsupported reasoning. To support long-horizon, evidence-grounded diagnosis, we introduce TrustMed-RL. Built from PubMed rare-disease cases and over 24,000 manually annotated image panels, it integrates interviews, examinations, testing, specialist consultation, and literature search through state-dependent actions. Our 8B vision--language policy, trained with clinically adapted GiGPO and coverage-adjusted diagnostic rewards, achieves 37.1% diagnostic accuracy on 2,500 evaluation cases, outperforming all evaluated open-weight baselines and improving over supervised fine-tuning by 12.4 percentage points.. When success additionally requires acquiring at least 50% of supporting test evidence, TrustMed-RL achieves 32.5%, exceeding GPT-4o by 6.8 percentage points. Furthermore, it surpasses all evaluated baselines on MTMedDialog and multiple larger 27--32B models on AgentClinic. In physician review of 200 diagnostically accepted test-set trajectories, 83.0% receive evidential-grounding scores of 4--5 out of 5. Physicians' assessments suggest that these diagnostic trajectories are trustworthy and aligned with human diagnostic reasoning.

Wenxin Zhan, Yizheng Jiao, Haifeng Song +4

Department of Information Systems and Statistics, Baruch College · Department of Computer Science, University of North Carolina at Chapel Hill · Tsinghua University +2

Medical Diagnosis Clinical Decision-Making Clinical Reasoning Medical VLMs RL for VLMs

Ask PDF

Aug 11, 2026

CARE: Confidence-Aware Reasoning for Reliable Medical VQA

Reinforcement Fine-Tuning (RFT) has enabled medical Multimodal Large Language Models (MLLMs) to produce Chain-of-Thought (CoT) reasoning for visual question answering, yet these models suffer from

\textit{confidence miscalibration}

---a systematic gap between expressed certainty and actual diagnostic accuracy that undermines clinical trust. We propose

\textbf{CARE}

, a

\textbf{C}

onfidence-

\textbf{A}

ware medical

\textbf{RE}

asoning framework that jointly optimizes accuracy and calibration through a dual-stage pipeline. First, a scalable Medical-CoT synthesis provides structured cold-start data for Supervised Fine-Tuning. Second, Group Relative Policy Optimization (GRPO) with a novel

\textbf{Confidence-Aware Reward (CAR)}

mechanism ties the model's confidence to diagnostic correctness within the reward signal. Across three Medical VQA benchmarks,

\textbf{CARE}

achieves the highest diagnostic accuracy while obtaining the lowest Expected Calibration Error and Hallucination Rate, establishing a foundation for trustworthy clinical decision support. Our code is available at https://github.com/anotherbricki/CARE.

Yuetian Du, Yucheng Wang, Zhenyuan Chen +9

Zhejiang University · Ant Group · University of Michigan +1

Medical VQA Medical VLMs VLM Reasoning VLM Calibration VLM Hallucination Mitigation RL for VLMs

Ask PDF

Abstract

Medical VLMs

Explore similar work

ClinCoT: Clinical-Aware Visual Chain-of-Thought for Medical Vision Language Models

TrustMed-RL: Long-Horizon Reinforcement Learning for Evidence-Grounded Clinical Diagnosis

CARE: Confidence-Aware Reasoning for Reliable Medical VQA