MEDIC-AD: Towards Medical Vision-Language Model's Clinical Intelligence

Woohyeon Park; Jaeik Kim; Sunghwan Steve Cho; Pa Hong; Wookyoung Jeong; Yoojin Nam; Namjoon Kim; Ginny Y. Wong

cs.CVMar 28, 2026

MEDIC-AD: Towards Medical Vision-Language Model's Clinical Intelligence

Authors: Woohyeon Park, Jaeik Kim, Sunghwan Steve Cho, Pa Hong, Wookyoung Jeong, Yoojin Nam, Namjoon Kim, Ginny Y. Wong,

Organizations: AIDAS Laboratory, Seoul National University · Samsung Changwon Hospital · Samsung Medical Center · NVIDIA, Santa Clara, USA

Abstract

Lesion detection, symptom tracking, and visual explainability are central to real-world medical image analysis, yet current medical Vision-Language Models (VLMs) still lack mechanisms that translate their broad knowledge into clinically actionable outputs. To bridge this gap, we present MEDIC-AD, a clinically oriented VLM that strengthens these three capabilities through a stage-wise framework. First, learnable anomaly-aware tokens (<Ano>) encourage the model to focus on abnormal regions and build more discriminative lesion centered representations. Second, inter image difference tokens (<Diff>) explicitly encode temporal changes between studies, allowing the model to distinguish worsening, improvement, and stability in disease burden. Finally, a dedicated explainability stage trains the model to generate heatmaps that highlight lesion-related regions, offering clear visual evidence that is consistent with the model's reasoning. Through our staged design, MEDIC-AD steadily boosts performance across anomaly detection, symptom tracking, and anomaly segmentation, achieving state-of-the-art results compared with both closed source and medical-specialized baselines. Evaluations on real longitudinal clinical data collected from real hospital workflows further show that MEDIC-AD delivers stable predictions and clinically faithful explanations in practical patient-monitoring and decision-support workflows

Figures & tables

Figure 1: Overall performance of VLMs on Medical Anomaly Detection and Medical Symptom Tracking (MMXU [ 42 ] ).

Figure 2: Comparison of VLMs on clinical applications. Medic-AD provides stronger lesion detection, temporal reasoning, and visual grounding than GPT-4o [ 24 ] and Citrus-V [ 53 ] .

Figure 3: Architecture of Medic-AD . (a) Stage 1: <Ano> Token Generation, (b) Stage 2: <Diff> Token Generation, and (c) Stage 3: Heatmap Generation illustrate each stage of the proposed framework. Note that CA denotes Cross-Attention.

Category	Model	Size	Brain MRI			Head CT			Br35h			COVID-19			Avg. F1
Category	Model	Size	Prec.	Rec.	F1	Prec.	Rec.	F1	Prec.	Rec.	F1	Prec.	Rec.	F1	Avg. F1
General	Qwen2.5-VL	7B	76.1	92.9	83.6	61.6	100.0	76.3	66.4	94.5	78.0	88.6	83.4	85.9	81.0
General	InternVL2.5	8B	81.2	97.4	88.6	67.6	96.0	79.3	71.2	99.0	82.8	94.8	60.2	73.6	81.1
Closed	GPT-4o	–	59.3	98.6	74.1	48.7	100.0	65.5	48.9	99.9	65.6	29.1	92.9	44.4	62.4
Closed	Claude-3.5	–	64.0	100.0	78.1	50.0	100.0	66.7	51.2	100.0	67.7	47.2	100.0	64.2	69.2
Anomaly	AnomalyGPT	13B	61.0	96.8	74.8	50.8	97.0	66.7	47.2	88.6	61.6	33.9	57.2	42.5	61.4

Table 1: Results on Zero-shot Medical Anomaly Detection (Brain MRI [ 29 ] , Head CT [ 31 ] , Br35h [ 18 ] , and COVID-19 [ 13 ] ). For each dataset, we report Precision, Recall, and F1. The rightmost column shows the average F1 over all tasks.

Category	Model	Size	Worsen	Improved	No Change	Overall ( $\uparrow$ )
General	InternVL2.5	8B	0.486	0.607	0.402	0.498
General	Qwen2.5-VL	7B	0.499	0.545	0.513	0.519
Closed	Claude-3.5	–	0.494	0.518	0.493	0.502
Closed	GPT-4o	–	0.480	0.675	0.559	0.571
Medical	Citrus-V	8B	0.468	0.713	0.532	0.571
	Lingshu	7B	0.597	0.734	0.529	0.620

Table 2: Results on MMXU [ 42 ] . Models are categorized into general-purpose, closed-source, and medical-domain VLMs.

Model	BraTS2021		RESC		BTCV + LiTs		ChestX-Det
Model	AUC ( $\uparrow$ )	mIoU ( $\uparrow$ )	AUC ( $\uparrow$ )	mIoU ( $\uparrow$ )	AUC ( $\uparrow$ )	mIoU ( $\uparrow$ )	AUC ( $\uparrow$ )	mIoU ( $\uparrow$ )
Citrus-V	98.8	32.6	87.6	2.1	82.1	1.7	98.0	12.4
Medic-AD (Ours)	99.8	87.6	100	97.2	97.2	83.6	99.8	79.8

Table 3: Visual grounding performance on BMAD [ 7 ] (BraTS2021 [ 5 ] , RESC [ 20 ] , BTCV + LiTs [ 32 , 9 ] ) and ChestX-Det [ 39 ] datasets. Each dataset reports AUC and mIoU metrics (higher is better).

Panel of Figure 4 — Figure 4: Visual Grounding comparison between Medic-AD and Citrus-V [ 53 ] on diverse abnormal and normal samples.

	<Ano>	Avg. F1 ( $\uparrow$ )	MMXU ( $\uparrow$ )
Effect of <Ano> Tokens
<Diff> tokens only	$\times$	–	0.635
Medic-AD	$\checkmark$	–	0.655
Feature Selection Strategy
Last layer	$\checkmark$	90.9	0.619
Intermediate 4-layers	$\checkmark$	91.6	0.635

Table 4: Effect of utilizing <Ano> tokens and comparison of visual feature extraction strategies.

Panel of Figure 5 — Figure 5: Hyperparameter sensitivity analysis on (a) query token pooling size and (b) visual soft prompt counts. The red line denotes the baseline performance of Lingshu [ 59 ] .

Model	GREEN	RaTEScore	GPT eval
Lingshu-7B	0.009	0.359	0.177
Medic-AD	0.020	0.430	0.291

Table 5: Evaluation on a real-world clinical dataset of 300 patients using GREEN [ 43 ] , RaTEScore [ 64 ] , and GPT-4o evaluation.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Comparison of chat template between original and Medic-AD , utilizing <Ano> and <Diff> tokens.

Category	Param. Size
Visual soft prompts	51K
<Ano> modules	40M
<Diff> modules	40M
Heatmap modules	48M

Appendix

Table 6: Parameter size of Medic-AD ’s additional modules.

Model	Size	VQA RAD	SLAKE	PathVQA	MMMU Med	PMC VQA
LLaVA-Med	7B	53.7	48.0	32.5	29.3	30.5
Citrus-V	8B	64.3	84.9	62.0	46.4	55.6
Lingshu	7B	67.9	83.1	61.9	54.0	56.0
Lingshu (full finetuning)	7B	40.8	38.4	45.3	24.8	6.4
Medic-AD	7B	64.3	78.5	56.5	54.2	56.1

Appendix

Table 7: Results on medical VQAs (VQA RAD [ 33 ] , SLAKE [ 51 ] , PathVQA [ 19 ] , MMMU Med [ 62 ] ), and PMC VQA [ 63 ] ).

Model	Size	MedMCQA	PubMedQA	MedQA	MedXpertQA
LLaVA-Med	7B	39.4	26.4	42.0	9.9
Citrus-V	8B	55.1	74.8	64.9	16.9
Lingshu	7B	55.9	75.4	63.3	16.5
Lingshu (full finetuning)	7B	1.8	35.4	24.4	10.9
Medic-AD	7B	56.7	75.6	63.6	16.5

Appendix

Table 8: Results on medical QAs (MedMCQA [ 44 ] , PubMedQA [ 28 ] , MedQA [ 27 ] , and MedXpertQA [ 68 ] ).

Category	Config. Value	Avg. F1	MMXU
Pooling Size	1	90.5	0.620
	2	90.2	0.631
	4	91.6	0.635
	8	91.7	0.623
Soft Prompt Counts	0	90.6	0.625
	5	91.8	0.630

Appendix

Table 9: Ablation on pooling size and soft prompt counts. We report average F1 and MMXU (Overall) for each configuration.

Model	AVG. F1	MMXU
Lingshu-7B	88.7	0.620
Medic-AD (Joint)	89.7	0.630
Medic-AD (Stage-wise)	91.6	0.655

Appendix

Table 10: Comparison between Stage-wise and Joint training.

Medical VLMs

Medical vision-language models connect clinical images with text for diagnosis, reporting, question answering, and image grounding.

See all 474 papers

RVLM: Recursive Vision-Language Models with Adaptive Depth
Mar 25, 2026Nicanor Mayumu, Zeenath Khan, Melodena Stephens +2VLM Reasoning Medical VLMs
Department of Computer Science, University of Wollongong in Dubai, Dubai Knowledge Park, Dubai, UAE · Department of Academic Affairs, Mohammed Bin Rashid School of Government, City Walk, Building B02, Dubai, UAE
MedObvious: Exposing the Medical Moravec's Paradox in VLMs via Clinical Triage
Mar 24, 2026Ufaq Khan, Umair Nawaz, L D M S S Teja +5VLM Evaluation Medical VLMs
Mohamed bin Zayed University of Artificial Intelligence, UAE · National Institute of Technology, Silchar · Birmingham City University, UK
Medical Context Distorts Decisions in Clinical Vision Language Models
May 17, 2026David Restrepo, Ira Ktena, Maria Vakalopoulou +2VLM Evaluation VLM Robustness
1MICS, CentraleSup´elec - Universit´e Paris-Saclay, France · 2Cancer Data Science Unit, IHU PRISM, National Institute in Precision Oncology, University Paris-Saclay, CentraleSupelec, Gustave Roussy, INSERM, Gif-sur-Yvette · 3Ellison Institute of Technology (EIT), Oxford, UK +1

Mar 25, 2026J/K move · Enter open · S save

RVLM: Recursive Vision-Language Models with Adaptive Depth

Nicanor Mayumu, Zeenath Khan, Melodena Stephens, Patrick Mukala+1

Department of Computer Science, University of Wollongong in Dubai, Dubai Knowledge Park, Dubai, UAE · Department of Academic Affairs, Mohammed Bin Rashid School of Government, City Walk, Building B02, Dubai, UAE

VLM Reasoning Medical VLMs Explainable Medical Image Analysis Tool Use in VLMs Radiology VLMs Efficient VLM Inference

Open paper AskPDF arXiv

Medical AI systems face two fundamental limitations. First, conventional vision-language models (VLMs) perform single-pass inference, yielding black-box predictions that cannot be audited or explained in clinical terms. Second, iterative reasoning systems that expose intermediate steps rely on fixed iteration budgets wasting compute on simple cases while providing insufficient depth for complex ones. We address both limitations with a unified framework. RVLM replaces single-pass inference with an iterative generate-execute loop: at each step, the model writes Python code, invokes vision sub-agents, manipulates images, and accumulates evidence. Every diagnostic claim is grounded in executable code, satisfying auditability requirements of clinical AI governance frameworks. RRouter makes iteration depth adaptive: a lightweight controller predicts the optimal budget from task-complexity features, then monitors progress and terminates early when reasoning stalls. We evaluate on BraTS 2023 Meningioma (brain MRI) and MIMIC-CXR (chest X-ray) using Gemini 2.5 Flash without fine-tuning. Across repeated runs, RVLM shows high consistency on salient findings (e.g., mass presence and enhancement) and can detect cross-modal discrepancies between Fluid-Attenuated Inversion Recovery (FLAIR) signal characteristics and segmentation boundaries. On MIMIC-CXR, it generates structured reports and correctly recognises view-specific artefacts. Code: https://github.com/nican2018/rvlm.

Abstract

Medical VLMs

Explore similar work

RVLM: Recursive Vision-Language Models with Adaptive Depth

MedObvious: Exposing the Medical Moravec's Paradox in VLMs via Clinical Triage

Medical Context Distorts Decisions in Clinical Vision Language Models