MediX-R1: Open Ended Medical Reinforcement Learning
Authors: Sahal Shaji Mullappilly, Mohammed Irfan Kurpath, Omair Mohamed, Mohamed Zidan, Fahad Khan, Salman Khan, Rao Anwer, Hisham Cholakkal
Organizations: Mohamed Bin Zayed University of Artificial Intelligence (MBZUAI) · Jubilee Mission Medical College and Research Institute · JJM Medical College
We introduce MediX-R1, an open-ended Reinforcement Learning (RL) framework for medical multimodal large language models (MLLMs) that enables clinically grounded, free-form answers beyond multiple-choice formats. MediX-R1 fine-tunes a baseline vision-language backbone with Group Based RL and a composite reward tailored for medical reasoning: an LLM-based accuracy reward that judges semantic correctness with a strict YES/NO decision, a medical embedding-based semantic reward to capture paraphrases and terminology variants, and lightweight format and modality rewards that enforce interpretable reasoning and modality recognition. This multi-signal design provides stable, informative feedback for open-ended outputs where traditional verifiable or MCQ-only rewards fall short. To measure progress, we propose a unified evaluation framework for both text-only and image+text tasks that uses a Reference-based LLM-as-judge in place of brittle string-overlap metrics, capturing semantic correctness, reasoning, and contextual alignment. Despite using only ∼51K instruction examples, MediX-R1 achieves excellent results across standard medical LLM (text-only) and VLM (image + text) benchmarks, outperforming strong open-source baselines and delivering particularly large gains on open-ended clinical tasks. Our results demonstrate that open-ended RL with comprehensive reward signals and LLM-based evaluation is a practical path toward reliable medical reasoning in multimodal models. Our trained models, curated datasets and source code are available at https://medix.cvmbzuai.com
Figures & tables
Model
Diverse Medical
Single-Stage
Interpretable
Open-Ended
Annotation-Free
Composite
Modalities
RL
Reasoning
Responses
Reasoning
RL Reward
MedVLM-R1 ( Pan et al., 2025 )
✗
✓
✓
✗
✓
✗
BiMediX2 ( Mullappilly et al., 2024 )
✓
✗
✗
✓
✗
✗
HuatuoGPT-V ( Chen et al., 2024b )
✓
✗
✗
✓
✗
✗
MedGemma ( Sellergren et al., 2025 )
✓
✗
✓
✓
✗
✗
MedMO ( Deria et al., 2026 )
✓
✗
✓
✓
✗
✗
Table 1: Differences with existing Medical VLMs: MediX-R1 integrates diverse modalities, interpretable reasoning, and composite RL rewards, enabling practical clinical use.
Figure 2: MediX-R1: Overall Architecture The MediX-R1 reinforcement learning framework for open-ended medical reasoning. An input of a medical image and a natural language question is processed by MediX-R1. The model’s policy is trained using Group Based RL, which leverages a multi-faceted reward signal. This reward is composed of: a) an LLM-based reward for evaluating the overall quality and correctness of the output; b) an embedding-based reward to ensure semantic alignment; c) a format reward to enforce the desired output structure ( <think> and <answer> blocks); and d) a modality reward to ensure the response is grounded in the specified imaging modality. This reward-guided approach encourages the model to generate accurate and interpretable reasoning paths.
Figure 3
Benchmarks
MedVLM R1 2B
BiMediX2 8B
Huatuo GPT-V 7B
MedGemma 4B
MedGemma 27B
MedMO 8B
MediX-R1 2B
MediX-R1 8B
MediX-R1 30B
MMLU-Clinical
0.540
0.732
0.721
0.708
0.879
0.864
0.660
0.845
0.894
MMLU-Bio
0.549
0.792
0.708
0.706
0.972
0.951
0.806
0.951
0.993
MMLU-Med
0.451
0.694
0.653
0.605
0.866
0.827
0.699
0.879
0.890
MMLU-Genetics
0.560
0.790
0.710
0.820
0.940
0.900
0.680
0.900
0.980
MMLU-ProfMed
0.500
0.695
0.625
0.713
0.912
0.890
0.581
0.868
0.974
MMLU-Anat
0.519
0.659
0.600
0.556
0.793
0.785
0.563
0.763
0.874
Table 2: Evaluation Benchmark. The top section lists LLM (text-only) tasks and the bottom section lists VLM (image+text) tasks. Our three-stage evaluation setting evaluates both tasks in a unified framework. MediX-R1 achieves the highest average score across this diverse suite, demonstrating state-of-the-art performance among open models. Best and second best results are bold and underlined
Model
MMMU Medical Validation
MedVLM-R1 2B ( Pan et al., 2025 )
39.33
MedGemma 4B ( Sellergren et al., 2025 )
50.00
HuatuoGPT-V 7B ( Chen et al., 2024b )
47.33
BiMediX2 8B ( Mullappilly et al., 2024 )
39.33
Qwen3-VL 8B ( Yang et al., 2025 )
62.66
MedMO 8B ( Deria et al., 2026 )
62.66
Table 3: MMMU Medical Val results. Accuracy on the Health & Medical validation subset of MMMU (image-text).
Figure 6: Qualitative examples of MediX-R1 . (Top, Microscopy) Correctly identifies the optic tract in section G with interpretable reasoning. (Bottom, X-ray) Explains why heart size appears smaller in PA vs. AP view. MediX-R1 generates clinically grounded, open-ended answers across modalities.
Evaluations
Default
Embedding
LLM
LLM +
MediX-R1
Reward
Reward
Reward
Embedding
LLM Tasks
0.660
0.640
0.666
0.686
0.687
VLM Tasks
0.382
0.409
0.400
0.410
0.431
Overall AVG
0.562
0.558
0.572
0.589
0.597
Table 4: Reward ablation across validation benchmarks. Single signals like default string matching, embedding-only or LLM-only are weaker. Combining LLM + embedding improves robustness, and the MediX-R1 composite (LLM accuracy + embedding-based semantics + modality recognition) yields the best overall average.
Model
Baseline
+ Composite Rewards
SmolVLM2-2.2B ( Marafioti et al., 2025 )
0.410
0.432
Qwen3-VL-2B ( Yang et al., 2025 )
0.529
0.554
Qwen3-VL-8B ( Yang et al., 2025 )
0.666
0.688
Qwen3-VL-30B ( Yang et al., 2025 )
0.698
0.736
Table 5: Baseline comparison across backbones (overall AVG). “Baseline” is the original backbone; “+Composite Rewards” applies our RL with composite rewards on the same backbone.
Evaluations
Baseline
GRPO
GSPO
DAPO
LLM Evaluations (text only)
0.675
0.687
0.689
0.701
VLM Evaluations (image + text)
0.376
0.431
0.439
0.445
Overall AVG
0.570
0.597
0.600
0.610
Table 6: RL Method ablation across benchmarks. Using the same composite reward, different RL algorithms (GRPO/GSPO/DAPO) consistently outperform the baseline across both LLM and VLM tasks, with DAPO achieving the highest overall average.
Figure 7: Overall validation reward vs training step across reward designs . Training with individual signals LLM-only or embedding-only shows volatility and reward hacking, while LLM+embedding reduces but does not eliminate instability. MediX-R1 uses a composite reward which stabilizes learning and delivers the highest final reward and best overall performance.
Model
Score (%)
MedVLM-R1
27.57
MedGemma
43.18
LLaVA-Med
44.29
BiMediX2
46.51
HuatuoGPT
48.81
MediX-R1 (Ours)
51.11
Table 7: Performance comparison on the MedPix 2.0 dataset.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
Samples
PMC_VQA_SUBSET
25000
SLAKE
4919
RAD_VQA
1793
PATH
19623
Total
51335
Appendix
Table 8: Modality Breakdown and Source Dataset composition
High-stakes clinical use of large vision-language models (LVLMs) requires reasoning that is grounded in visual evidence and clinical knowledge, not just correct final answers. We introduce OpenMedReason, a large-scale, open multimodal medical reasoning corpus comprising approximately 450K image-question-answer instances whose reasoning traces are primarily derived from curated biomedical, human-authored scientific articles. OpenMedReason provides high-fidelity supervision beyond synthetic chains of thought, covering diverse medical domain vision modalities such as radiological scans, microscopic images, visible light photographs, charts, and others. We complement it with OpenMedReason-Bench, a held-out benchmark that allows fine-grained evaluation of LVLMs along three complementary axes of capability, including perception, medical knowledge, and rationale, enabling diagnostic evaluation beyond final-answer accuracy. OpenMedReason is a rich training resource that exhibits its effectiveness in both supervised fine-tuning (SFT) and reinforcement-based alignment. Training with OpenMedReason yields a 20% average improvement in VQA accuracy over the base model and achieves performance within 4.2% of the strongest comparable-scale medical LVLMs. Fine-grained performance analysis confirms that the gains are not concentrated in any single axis: OpenMedReason improves perception, medical knowledge, and rationale jointly, and its reasoning traces are preferred over those of the base model in 86.1% of pairwise comparisons. We release the code and dataset at huggingface.co/datasets/neginb/OpenMedReason.
Negin Baghbanzadeh, Pritam Sarkar, Michael Colacci +6
York University · Vector Institute · University of British Columbia +5
Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities in understanding common visual elements, largely due to their large-scale datasets and advanced training strategies. However, their effectiveness in medical applications remains limited due to the inherent discrepancies between data and tasks in medical scenarios and those in the general domain. Concretely, existing medical MLLMs face the following critical limitations: (1) limited coverage of medical knowledge beyond imaging, (2) heightened susceptibility to hallucinations due to suboptimal data curation processes, (3) lack of reasoning capabilities tailored for complex medical scenarios. To address these challenges, we first propose a comprehensive data curation procedure that (1) efficiently acquires rich medical knowledge data not only from medical imaging but also from extensive medical texts and general-domain data; and (2) synthesizes accurate medical captions, visual question answering (VQA), and reasoning samples. As a result, we build a multimodal dataset enriched with extensive medical knowledge. Building on the curated data, we introduce our medical-specialized MLLM: Lingshu. Lingshu undergoes multi-stage training to embed medical expertise and enhance its task-solving capabilities progressively. Besides, we preliminarily explore the potential of applying reinforcement learning with verifiable rewards paradigm to enhance Lingshu's medical reasoning ability. Additionally, we develop MedEvalKit, a unified evaluation framework that consolidates leading multimodal and textual medical benchmarks for standardized, fair, and efficient model assessment. We evaluate the performance of Lingshu on three fundamental medical tasks, multimodal QA, text-based QA, and medical report generation. The results show that Lingshu consistently outperforms the existing open-source multimodal models on most tasks ...
Deploying Large Language Models (LLMs) in healthcare requires robust performance across two complementary dimensions - diagnostic reasoning: the convergent, evidence-driven task of inferring a patient's condition from clinical data to produce a diagnosis, and clinical healthcare reasoning: the broader, navigational judgment required to communicate, plan, and adapt across multi-turn clinical interactions where a single correct answer may not exist. Recent benchmarks such as HealthBench and MedXpertQA reveal persistent weaknesses in both areas, exposing failures in complex diagnostic scenarios and limitations in contextual, patient-centered dialogue. We introduce a sequential training framework that targets these facets using synthetic data and rubric-based reinforcement learning. First, we improve diagnostic reasoning using MedBullets-derived questions with rule- and rubric-guided Reinforcement Learning (RL). We then shift to clinical reasoning by generating 5.3k synthetic multi-turn scenarios, each paired with multi-dimensional rubrics to comprehensively assess the response. This approach yields over 10% improvement on MedXpertQA, and our 30B model achieves 50.1% accuracy on HealthBench-Hard, surpassing proprietary baselines including GPT-5 (thinking). Our results show that targeted synthetic datasets and rubric-based training can systematically improve both diagnostic and interactive clinical reasoning in medical LLMs.