Aligning Medical Vision Language Models at the Capability Boundary with Entropy Adaptive Rubric Grounding

Hongyu Bai; Jiaqi Zeng; Chunzheng Zhu

cs.CVOct 29, 2026

Aligning Medical Vision Language Models at the Capability Boundary with Entropy Adaptive Rubric Grounding

Authors: Hongyu Bai, Jiaqi Zeng, Chunzheng Zhu

Organizations: Shanghai Jiao Tong University Shanghai, China · Hunan University Changsha, China

Abstract

Medical diagnostic reasoning requires verifiable links between visual evidence and clinical knowledge, yet current medical vision language models are aligned with objectives that reward the wrong behavior. Uniform supervised fine tuning lets easy cases dominate updates while clinically critical boundary cases saturate prematurely. Sparse rewards based only on final-answer correctness cannot assess anatomical grounding or clinical logic, inviting reward hacking through answer shortcuts. We propose MedCBR, a two-stage alignment framework coupling boundary-aware preference optimization with rubric-grounded reinforcement learning. First, we construct counterfactual boundary preferences at the model's capability frontier using empirical pass rates and preference pairs that share reasoning prefixes but diverge at decisive diagnostic branches. Difficulty Weighted Boundary Preference Optimization (DBPO) concentrates learning on this calibrated boundary region. Second, we distill reusable clinical rubrics from boundary trajectories and activate them according to prediction entropy. Entropy Gated Grounded Policy Optimization (EGPO) rewards answer correctness, anatomical grounding, and reasoning fidelity while suppressing rubric noise on reliable samples. A reference policy anchored to the boundary-aligned model couples population-level boundary priors with instance-level uncertainty, making diagnostic reasoning more accurate and clinically grounded. Experiments on six medical benchmarks show consistent improvements over strong general, specialized, and reinforcement learning baselines, reaching 61.2% average accuracy with the largest gains on challenging differential diagnosis tasks. Ablations show super-additive gains: combining the two stages exceeds the sum of their individual improvements.

Figures & tables

Figure 1. Motivation of MedCBR . Standard SFT distributes gradient uniformly across all tokens, causing loss saturation on clinically critical hard samples (left). Sparse binary rewards in conventional RL provide no signal on whether reasoning is grounded in correct anatomical evidence, enabling reward hacking via answer shortcuts (right). MedCBR addresses both via boundary aware preference alignment and entropy adaptive rubric grounding.

Figure 2. Overview of MedCBR . Stage 1 ( CBP , off policy): from the SFT model, $N$ trajectories are sampled per query, empirical pass rates define the capability boundary band, and counterfactual preference pairs diverging at critical diagnostic branch points are mined for difficulty weighted DPO. Stage 2 ( MRG , on policy): meta rubrics distilled from stage one trajectories are dynamically selected via entropy gating, composing structured process rewards for GRPO optimization anchored to $\pi_{\textsc{cbp}}$ .

Method	RAD	SLK	Path	PMC	MMU	MedX	Avg
General Vision Language Models
LLaVA Med ( Li et al., 2023 )	61.5	64.2	52.1	41.3	38.6	18.7	46.1
Qwen2.5 VL 7B ( Shen et al., 2025 )	66.8	70.4	56.3	47.9	45.2	22.4	51.5
InternVL2.5 8B ( Chen et al., 2024b )	68.1	72.0	57.8	49.5	47.0	23.6	53.0
Medical Specialized Models
RadFM ( Wu et al., 2025b )	63.9	68.7	54.6	44.0	40.1	19.9	48.5

Table 1. Comprehensive comparison on six medical multimodal benchmarks (accuracy %). Columns: VQA-RAD (RAD), SLAKE (SLK), PathVQA (Path), PMC-VQA (PMC), MMMU-Med (MMU), MedXpertQA (MedX). Bold best, underlined second best.

Configuration	RAD	SLK	Path	PMC	MMU	MedX	Avg	$\bm{\Delta_{\textit{syn}}}$
SFT Baseline (Qwen2.5 VL 7B)	66.8	70.4	56.3	47.9	45.2	22.4	51.5	–
+ CBP only	72.3	78.1	62.8	53.6	53.8	26.8	57.9	–
+ MRG only (anchor $\pi_{\textsc{sft}}$ )	71.5	77.2	61.9	52.8	52.4	26.1	57.0	–
+ CBP $\to$ MRG (anchor $\pi_{\textsc{cbp}}$ )	74.6	82.3	65.7	56.8	58.4	29.1	61.2	+2.8
+ CBP $\to$ MRG (anchor $\pi_{\textsc{sft}}$ )	73.8	80.9	64.8	55.9	56.9	28.4	60.1	+1.7
CBP component ablation:

Table 2. Component ablation of MedCBR on the 7B backbone (accuracy %). Column abbreviations follow Table 1 . “

\Delta_{\textit{syn}}

” denotes the synergy gain confirming super-additivity.

Figure 3. Stage-wise data composition and super-additive gain decomposition. Left: specialty distribution shifts from radiology-dominated boundary pairs in Stage 1 toward over-sampled differential-diagnosis cases in Stage 2. Right: the combined CBP

\to

MRG pipeline exceeds the sum of individual stage gains, with surplus

\Delta_{\textit{syn}}

largest on the hardest benchmarks.

Figure 4. Training dynamics of MedCBR . Left: the full model’s composite reward climbs to

{\sim}

0.80 while the no-CBP ablation saturates near 0.38, a

2\times

gap. Middle: trajectory length peaks early then compresses as reasoning sharpens, whereas the ablation stays verbose. Right: per-benchmark rubric-gain of entropy-driven selection, largest on MMMU-Med.

Method	Time	Mem	Trainable	Avg.
Vanilla GRPO ( Shao et al., 2024 )	28	62	42M	57.1
Med R1 ( Lai et al., 2026 )	32	64	42M	57.6
MedVLM R1 ( Pan et al., 2025 )	36	68	42M	58.6
V RFT PRA ( Yang et al., 2026e )	40	72	42M	59.9
MedCBR Stage 1	18	58	42M	57.9
MedCBR Stage 2	24	66	42M	–

Table 3. Computational comparison with RL baselines on 8

\times

A100 GPUs (Time in GPU hours, Mem in GB).

Medical VLMs

Medical vision-language models connect clinical images with text for diagnosis, reporting, question answering, and image grounding.

See all 479 papers

ClinCoT: Clinical-Aware Visual Chain-of-Thought for Medical Vision Language Models
Mar 1, 2026Xiwei Liu, Yulong Li, Xinlin Zhuang +5Vision-Language Models Medical VLMs
Mohamed bin Zayed University of Artificial Intelligence · East China Normal University
TrustMed-RL: Long-Horizon Reinforcement Learning for Evidence-Grounded Clinical Diagnosis
Oct 3, 2026Wenxin Zhan, Yizheng Jiao, Haifeng Song +4Medical Diagnosis Clinical Decision-Making
Department of Information Systems and Statistics, Baruch College · Department of Computer Science, University of North Carolina at Chapel Hill · Tsinghua University +2
CARE: Confidence-Aware Reasoning for Reliable Medical VQA
Aug 11, 2026Yuetian Du, Yucheng Wang, Zhenyuan Chen +9Medical VQA Medical VLMs
Zhejiang University · Ant Group · University of Michigan +1

Mar 1, 2026J/K move · Enter open · S save

ClinCoT: Clinical-Aware Visual Chain-of-Thought for Medical Vision Language Models

Xiwei Liu, Yulong Li, Xinlin Zhuang, Xuhui Li+4

Mohamed bin Zayed University of Artificial Intelligence · East China Normal University

Vision-Language Models Medical VLMs VLM Hallucination Mitigation Medical VQA

Open paper AskPDF arXiv

Medical Vision-Language Models have shown promising potential in clinical decision support, yet they remain prone to factual hallucinations due to insufficient grounding in localized pathological evidence. Existing medical alignment methods primarily operate at the response level through preference optimization, improving output correctness but leaving intermediate reasoning weakly connected to visual regions. Although chain-of-thought (CoT) enhances multimodal reasoning, it remains largely text-centric, limiting effective integration of clinical visual cues. To address this gap, we propose ClinCoT, a clinical-aware visual chain-of-thought framework that transforms preference optimization from response-level correction to visual-driven reasoning. We introduce an automatic data generation pipeline that constructs clinically grounded preference pairs through reasoning with hypotheses-driven region proposals. Multiple Med-LLMs evaluators rank and assign scores to each response, and these rankings serve as supervision to train the target model. We further introduce a scoring-based margin-aware optimization strategy that incorporates both preference ranking and score difference to refine region-level reasoning trajectories. To maintain alignment as the model's policy evolves during training, we adopt an iterative learning scheme that dynamically regenerates preference data. Extensive experiments on three medical VQA and report generation benchmarks demonstrate that ClinCoT consistently improves factual grounding and achieves superior performance compared with existing preference-based alignment methods.

Abstract

Medical VLMs

Explore similar work

ClinCoT: Clinical-Aware Visual Chain-of-Thought for Medical Vision Language Models

TrustMed-RL: Long-Horizon Reinforcement Learning for Evidence-Grounded Clinical Diagnosis

CARE: Confidence-Aware Reasoning for Reliable Medical VQA