Organizations: University of Toronto, Canada · Vector Institute, Canada · KITE Research Institute, University Health Network, Canada · York University, Canada
Reinforcement learning (RL) is increasingly used to post-train medical Vision-Language Models (VLMs), yet it remains unclear whether RL improves medical visual reasoning or mainly sharpens behaviors already induced by supervised fine-tuning (SFT). We present a controlled study that disentangles these effects along three axes: vision, SFT, and RL. Using MedMNIST as a multi-modality testbed, we probe visual perception by benchmarking VLM vision towers against vision-only baselines, quantify reasoning support and sampling efficiency via Accuracy@1 versus Pass@K, and evaluate when RL closes the support gap and how gains transfer across modalities. We find that RL is most effective when the model already has non-trivial support (high Pass@K): it primarily sharpens the output distribution, improving Acc@1 and sampling efficiency, while SFT expands support and makes RL effective. Based on these findings, we propose a boundary-aware recipe and instantiate it by RL post-training an OctoMed-initialized model on a small, balanced subset of PMC multiple-choice VQA, achieving strong average performance across six medical VQA benchmarks.
Figures & tables
Method
Radiology
Microscopy
Visible Light Photography
Breast
Chest
Pneumonia
OrganA
OrganC
OrganS
Blood
Path
Tissue
Derma
OCT
Retina
MedViT-V2 [ 15 ]
91.00
96.70
97.30
97.30
96.10
85.10
98.70
97.10
71.60
81.70
95.20
57.80
M Base
80.95
13.33
92.52
85.27
79.31
65.70
96.19
89.12
48.26
65.22
84.30
44.00
M SFT
84.52
16.51
93.35
89.01
83.59
70.90
96.19
93.35
51.99
67.08
91.10
52.00
M RL
75.00
16.19
92.95
85.52
80.35
64.96
96.86
88.50
52.88
69.57
84.80
51.00
Table 1: Linear probing on frozen vision encoders across MedMNIST-v2 tasks, grouped by modality. MedViT-v2 is included as a strong vision-only reference.
Method
Radiology
Microscopy
Visible Light Photography
Breast
Chest
Pneumonia
OrganA
OrganC
OrganS
Blood
Path
Tissue
Derma
OCT
Retina
M Base
54.99
8.70
50.53
14.39
10.94
13.14
13.16
26.48
10.39
33.54
32.62
20.19
M SFT
79.17
13.22
86.00
18.66
16.03
19.38
76.00
64.53
12.66
52.33
82.78
22.19
M RL
69.79
8.03
62.41
10.91
8.94
10.75
13.95
34.09
8.96
27.48
28.72
20.62
Table 2: VLM evaluation with Accuracy@1 under greedy decoding on MedMNIST-v2 tasks, grouped by modality.
Figure 1: Pass@K curves on MedMNIST-v2, grouped by modality.
Figure 2: Before/after RL changes in Acc@1 and Pass@16 from MBase and MSFT across in-domain, within-modality, and cross-modality evaluations.
Figure 3: Overview of our boundary-aware recipe. We first diagnose support using Pass@K and Acc@1 then decide between bridging versus RL sharpening.
Method
PMC [ 30 ]
MMMU [ 25 ]
MedX-M [ 31 ]
PathVQA [ 7 ]
SLAKE [ 13 ]
VQA-Rad [ 12 ]
Avg.
Qwen2.5-VL-7B-Instruct [ 2 ]
53.00
55.29
23.50
65.00
68.00
74.00
56.47
QoQ-Med-7B [ 5 ]
51.00
61.18
25.00
63.50
68.50
74.00
57.20
MedVLThinker-7B [ 8 ]
55.00
54.12
27.50
57.50
63.50
64.00
53.60
OctoMed-7B [ 17 ]
55.50
56.47
35.00
63.00
84.00
79.00
62.16
MedVLM-R1-2B [ 18 ]
50.50
39.41
22.50
61.50
71.00
63.50
51.40
Med-R1-2B [ 11 ]
53.00
41.18
23.50
66.50
69.50
62.00
52.61
Table 3: Performance of medical VLMs on multimodal medical benchmarks. Best results are in bold; second-best are underlined.
School of Mathematics, Tianjin University, Tianjin, China · Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · Shanghai Advanced Institute of Finance (SAIFS), East China Normal University, Shanghai, China +4