MedVR: Annotation-Free Medical Visual Reasoning via Agentic Reinforcement Learning

Zheng Jiang; Heng Guo; Chengyu Fang; Changchen Xiao; Xinyang Hu; Lifeng Sun; Minfeng Xu

cs.CVApr 9, 2026

MedVR: Annotation-Free Medical Visual Reasoning via Agentic Reinforcement Learning

Authors: Zheng Jiang, Heng Guo, Chengyu Fang, Changchen Xiao, Xinyang Hu, Lifeng Sun, Minfeng Xu

Organizations: Tsinghua University · DAMO Academy, Alibaba Group · Hupan Lab, 310023, Hangzhou, China · Zhejiang University · Key Laboratory of Pervasive Computing, Ministry of Education

Abstract

Medical Vision-Language Models (VLMs) hold immense promise for complex clinical tasks, but their reasoning capabilities are often constrained by text-only paradigms that fail to ground inferences in visual evidence. This limitation not only curtails performance on tasks requiring fine-grained visual analysis but also introduces risks of visual hallucination in safety-critical applications. Thus, we introduce MedVR, a novel reinforcement learning framework that enables annotation-free visual reasoning for medical VLMs. Its core innovation lies in two synergistic mechanisms: Entropy-guided Visual Regrounding (EVR) uses model uncertainty to direct exploration, while Consensus-based Credit Assignment (CCA) distills pseudo-supervision from rollout agreement. Without any human annotations for intermediate steps, MedVR achieves state-of-the-art performance on diverse public medical VQA benchmarks, significantly outperforming existing models. By learning to reason directly with visual evidence, MedVR promotes the robustness and transparency essential for accelerating the clinical deployment of medical AI.

Figures & tables

Figure 1: MedVR enables verifiable visual reasoning via tool-use. In contrast to the visual hallucinations prevalent in text-only reasoning (left), MedVR (middle) interleaves textual deliberation with active image manipulation to achieve precise visual grounding. This synergy results in superior accuracy and robustness across multiple public medical VQA benchmarks (right).

Figure 2: Overview of MedVR. The framework employs EVR to explore visual actions based on the model’s intrinsic uncertainty and CCA to create a consensus-based reward from successful trajectories, enabling annotation-free training of medical visual reasoning.

Settings	General-Domain VQA (Multi-choice)				Modality-Specific VQA (Free-text)
Tasks	OMVQA	PMC-VQA ⋄	MedXQA ⋄	Avg.	VQA-RAD	SLAKE	PathVQA ⋄	Avg.
General-Purpose VLMs
Qwen2.5-VL-7B ( Bai et al., 2025a )	59.0	51.2	22.3	44.2	64.5	67.2	44.1	58.6
Qwen2.5-VL-32B ( Bai et al., 2025a )	69.9	54.1	24.5	49.5	71.8	71.2	41.9	61.6
InternVL2.5-8B ( Chen et al., 2024c )	83.9	51.3	21.0	52.1	59.4	69.0	42.1	56.8
InternVL3-8B ( Zhu et al., 2025 )	81.4	53.8	22.1	52.4	65.4	72.8	48.6	62.3

Table 1: Comprehensive performance comparison on medical VQA benchmarks, divided into General-Domain (multiple-choice) and Modality-Specific (free-text) tasks. Out-of-domain (OOD) test sets are marked with ⋄ . The best and second-best results are highlighted in bold and underlined .

Figure 3: Ablation study for key hyperparameters and reward design of MedVR.

Methods	Components			Performance
Methods	Zoom-in	EVR	CCA	OmniMedVQA	PMC-VQA	MedXpertQA
Textual RL	—	—	—	94.50	53.40	21.38
MedVR	✓	—	—	94.31	52.62	22.26
	✓	✓	—	95.38	53.81	24.73
	✓	—	✓	96.55	53.30	23.09
	✓	✓	✓	96.77	54.31	26.38

Table 2: Ablation study for core components of MedVR.

Model	Method	OmniMedVQA	PMC-VQA	MedXpertQA	Avg.
Qwen2.5-VL-3B	Zero-shot	63.9	50.6	21.9	45.5
	Textual RL	93.8	47.5	20.2	53.8
	MedVR (Ours)	92.1	49.1	22.2	54.5
Qwen2.5-VL-7B	Zero-shot	64.3	51.2	22.3	46.0
	Textual RL	94.5	53.4	21.4	56.4
	MedVR (Ours)	96.7	54.3	26.4	59.1

Table 3: Performance of MedVR across model scales and variants.

Figure 7: Qualitative examples of MedVR’s visual reasoning process.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Training Data	Training Methods	Key Benchmarks Involved in Training
Med-R1-2B	OmniMedVQA	RL	OmniMedVQA
MedVLM-R1-2B	HuatuoGPT-Vision Eval	RL	OmniMedVQA, PMC-VQA, VQA-RAD, SLAKE, PathVQA
MedGemma-4B	Proprietary curated data (33M)	Pretraining, SFT, RL	PMC-VQA, VQA-RAD, SLAKE and others
LLaVA-Med-7B	PMC-15M	SFT	PMC-VQA
HuatuoGPT-V-7B	PubMedVision (1.3M)	Pretraining,SFT	PMC-VQA
Lingshu-7B	Proprietary curated data (12M)	Pretraining, SFT, RL	PMC-VQA, MedXpertQA, VQA-RAD, SLAKE, PathVQA and others

Appendix

Table 4: Summary of training data, methodologies, and benchmark overlap for key baselines.

Table 10

Table 7: Performance on visual grounding and multimodal reasoning benchmarks.

Figure 8: Additional Visualization Results of MedVR’s Reasoning Process.

Figure 9: Average number of tool calls per trajectory during training.

Model	Total Tokens	Total Latency	Extra Visual Tokens	Extra Latency
MedVR	169.23	15.21s	84.76	0.23s (~1.5%)

Appendix

Table 8: Inference latency analysis for MedVR on the OmniMedVQA dataset.

Figure 10: System Prompt used in MedVR.

Figure 11: User Prompt for inference.

Figure 12: System Prompt used in Textual RL.

Figure 13: User Prompt used in Textual RL.

Explore similar work

MedLVR: Latent Visual Reasoning for Reliable Medical Visual Question Answering
Apr 10, 2026Suyang Xi, Songtao Hu, Yuxiang Lai +4Medical VQA Medical VLMs
Department of Radiation Oncology and Winship Cancer Institute, Emory University School of Medicine, Atlanta, 30322, GA, USA · Department of Biostatistics and Bioinformatics, Emory University, Atlanta, 30322, GA, USA
When Does RL Help Medical VLMs? Disentangling Vision, SFT, and RL Gains
Mar 1, 2026Ahmadreza Jeddi, Kimia Shaban, Negin Baghbanzadeh +4Medical VLMs RL for VLMs
University of Toronto, Canada · Vector Institute, Canada · KITE Research Institute, University Health Network, Canada +1
Med-R2: An Adversarial Benchmark for Evidence-Grounded Reasoning in Medical VLMs
May 23, 2026Wen Ma, Fucheng Niu, Zhiting Fan +3Adversarial Attacks on VLMs Healthcare
Zhejiang University, Zhejiang, China · Guangdong Institute of Intelligence Science and Technology

Apr 10, 2026J/K move · Enter open · S save

MedLVR: Latent Visual Reasoning for Reliable Medical Visual Question Answering

Suyang Xi, Songtao Hu, Yuxiang Lai, Wangyun Dan+3

Department of Radiation Oncology and Winship Cancer Institute, Emory University School of Medicine, Atlanta, 30322, GA, USA · Department of Biostatistics and Bioinformatics, Emory University, Atlanta, 30322, GA, USA

Medical VQA Medical VLMs Latent Visual Reasoning RL for VLMs

Open paper AskPDF arXiv

Medical vision--language models (VLMs) have shown strong potential for medical visual question answering (VQA), yet their reasoning remains largely text-centric: images are encoded once as static context, and subsequent inference is dominated by language. This paradigm is fundamentally limited in clinical scenarios, where accurate answers often depend on subtle, localized visual evidence that cannot be reliably preserved in static embeddings. We propose MedLVR, a latent visual reasoning framework that introduces an explicit visual evidence state into autoregressive decoding. Instead of relying solely on text-based intermediate reasoning, MedLVR interleaves a short latent reasoning segment within the decoder by reusing hidden states as continuous latent steps, enabling iterative preservation and refinement of query-relevant visual evidence before answer generation. To support effective visual supervision, we adopt a two-stage training strategy: region of interest (ROI)-supervised fine-tuning aligns latent states with clinically relevant image evidence, and Visual-Latent Policy Optimization (VLPO) further optimizes latent reasoning and answer generation under outcome-level rewards. Experiments on OmniMedVQA and five external medical VQA benchmarks show that MedLVR consistently outperforms recent reasoning baselines and improves the average score over the Qwen2.5-VL-7B backbone from 48.3% to 53.4%. These results show that latent visual reasoning provides an effective mechanism for preserving diagnostically relevant visual evidence and improving the reliability of medical VQA.

Abstract

Explore similar work

MedLVR: Latent Visual Reasoning for Reliable Medical Visual Question Answering

When Does RL Help Medical VLMs? Disentangling Vision, SFT, and RL Gains

Med-R2: An Adversarial Benchmark for Evidence-Grounded Reasoning in Medical VLMs