Medical VLMs

VLM: Vision-Language Model

Follow this topic Ask about this topic

Momentum

22 papers in the last four weeks, up 22% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 266

All topics

Med-R2: An Adversarial Benchmark for Evidence-Grounded Reasoning in Medical VLMs
May 23, 2026Wen Ma, Fucheng Niu, Zhiting Fan +3Adversarial Attacks on VLMs Healthcare
Zhejiang University, Zhejiang, China · Guangdong Institute of Intelligence Science and Technology
Structured Visual Evidence Decomposition for Evidence-Grounded Multimodal Screening of Obstructive Sleep Apnea-Hypopnea Syndrome
May 23, 2026Chen Zhan, Yingchen Wei, Xiaoyu Tan +2Healthcare Clinical Triage
School of Electronic and Electrical Engineering, Shanghai University of Engineering Science, Shanghai 200438, China · Tencent Youtu Lab, Shanghai 200233, China · ENT Institute and Department of Otorhinolaryngology, Eye & ENT Hospital of Fudan University, Shanghai 200031, China +1
SliceWorld: A Predictive and Controllable World-State Model for CT Report Generation
May 23, 2026Yuanhe Tian, Yan SongRadiology Report Generation World Model Learning
Zhongguancun Academy · University of Science and Technology of China
JMed48k: A Multi-Profession Japanese Medical Licensing Benchmark for Vision-Language Model Evaluation
May 21, 2026Yue Xun, Junyu Liu, Qian Niu +10VLM Evaluation Healthcare
The Hong Kong Polytechnic University · Kyoto University · The University of Tokyo +3
Universal CT Representations from Anatomy to Disease Phenotype through Agglomerative Pretraining
May 21, 2026Yuheng Li, Yuan Gao, Haoyu Dong +5Medical Imaging Foundation Models Medical VLMs
Wallace H. Coulter Department of Biomedical Engineering, Georgia Institute of Technology and Emory University, Atlanta, GA 30332, USA · Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, GA 30322, USA · Department of Electrical and Computer Engineering, Duke University, Durham, NC 27705, USA +2
SurgOnAir: Hierarchy-Aware Real-Time Surgical Video Commentary
May 20, 2026Jingyi He, Yue Zhou, Long Bai +3Streaming Video Understanding Surgical Video Understanding
Computer Aided Medical Procedures (CAMP), TU Munich, Germany · Munich Center for Machine Learning (MCML), Munich, Germany · The Chinese University of Hong Kong, Hong Kong +1
VIHD: Visual Intervention-based Hallucination Detection for Medical Visual Question Answering
May 20, 2026Jiayi Chen, Benteng Ma, Zehui Liao +3Hallucination Detection in VLMs Healthcare
Department of Data Science & AI, Faculty of Information Technology, Monash University, Melbourne, VIC 3800, Australia · Hong Kong Polytechnic University, Hong Kong SAR, China · Alfred Health Radiology, Alfred Health, Melbourne, VIC 3004, Australia +1
MedExpMem: Adapting Experience Memory for Differential Diagnosis
May 20, 2026Qianhan Feng, Zhongzhen Huang, Yakun Zhu +4Radiology VLMs Healthcare
The Chinese University of Hong Kong · Shanghai Jiao Tong University · Shanghai Innovation Institue
HalluCXR: Benchmarking and Mitigating Hallucinations in Medical Vision-Language Models for Chest Radiograph Interpretation
May 19, 2026Haoyu Wang, Zitong LiVision-Language Models Audio-Language Model Hallucination Detection
Department of Biostatistics & Health Informatics, Institute of Psychiatry, Psychology & Neuroscience, King’s College London, London, UK
Rethinking Visual Attribution for Chest X-ray Reasoning in Large Vision Language Models
May 19, 2026Guangzhi Xiong, Qiao Jin, Sanchit Sinha +2Medical VQA VLM Interpretability
University of Virginia · National Institutes of Health
Concept-Guided Noisy Negative Suppression for Zero-Shot Classification and Grounding of Chest X-Ray Findings
May 19, 2026Chenyu Lian, Hong-Yu Zhou, Chun-Ka Wong +1Contrastive Learning Chest X-Ray Classification
The Center for Smart Health, School of Nursing, the Hong Kong Polytechnic University, Hong Kong, China · Research Institute for Smart Ageing, the Hong Kong Polytechnic University,2026 Hong Kong, China · School of Biomedical Engineering, Tsinghua Medicine, Tsinghua University, Beijing, China +1
MAM-CLIP: Vision-Language Pretraining on Mammography Atlases for BI-RADS Classification
May 19, 2026Halil Ibrahim Gulluk, Olivier GevaertMedical Imaging Medical Image Classification
Department of Electrical Engineering Stanford University Stanford, CA 94305 · Biomedical Informatics Research (BMIR) Stanford University Stanford, CA 94304
MedFM-Robust: Benchmarking Robustness of Medical Foundation Models
May 18, 2026Xiangxiang Cui, Tianjin Huang, Yifang Wang +2Multimodal Robustness Neural Network Robustness
Beijing Normal University, China · University of Exeter, United Kingdom · University College London, United Kingdom +2
Geometry-Aware Uncertainty Coresets for Robust Visual In-Context Learning in Histopathology
May 18, 2026Franciskus Xaverius Erick, Johanna Paula Müller, Bernhard KainzVLM Robustness In-Context Learning
FAU Erlangen-Nürnberg, Erlangen, DE · Department of Computing, Imperial College London, London, UK
Wasserstein Equilibrium Decoding for Reliable Medical Visual Question Answering
May 18, 2026Luca Hagen, Johanna P. Müller, Weitong Zhang +2Healthcare Efficient VLM Inference
Friedrich-Alexander University Erlangen-Nürnberg, Erlangen, Germany · Imperial College London, London, United Kingdom · University College London, London, United Kingdom
How Good LLMs Are at Answering Bangla Medical Visual Questions? Dataset and Benchmarking
May 18, 2026Rafid Ahmed, Intesar Tahmid, Mir Sazzat Hossain +3VLM Evaluation Multilingual Language Model Evaluation
Penta Global Limited, Bangladesh · Center for Computational & Data Sciences, Independent University, Bangladesh
Segmentation, Detection and Explanation: A Unified Framework for CT Appearance Reasoning
May 15, 2026Yuyuan Liu, Can Peng, Yingyu Yang +3Image Segmentation Medical VLMs
Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford
BiomedAP: A Vision-Informed Dual-Anchor Framework with Gated Cross-Modal Fusion for Robust Medical Vision-Language Adaptation
May 15, 2026Huanyang Tong, Kai Liu, Fangjun Kuang +1VLM Robustness Prompt Learning
Wenzhou University · Wenzhou Business College
RoiMAM: Region-of-Interest Medical Attention Model for Efficient Vision-Language Understanding
May 15, 2026Jiayan Yang, Zhuoyu Wu, Wenqi FangHealthcare Efficient VLM Inference
Shenzhen Institutes of Advanced Technology, Chinese Academy of Science · CyPhi(ΨΦ) AI Research Lab, School of IT, Monash University, Malaysia Campus
DermAgent: A Self-Reflective Agentic System for Dermatological Image Analysis with Multi-Tool Reasoning and Traceable Decision-Making
May 14, 2026Yize Liu, Siyuan Yan, Ming Hu +5Medical Image Analysis LLM Agent Orchestration
AIM for Health Lab, Faculty of Information Technology, Monash University, Melbourne, Australia · Faculty of Information Technology, Monash University, Melbourne, Australia · University College London, Institute of Ophthalmology, London, United Kingdom
CA-GCL: Cross-Anatomy Global-Local Contrastive Learning for Robust 3D Medical Image Understanding
May 13, 2026Hanwen Zhang, Yao Liu, Die Dai +4VLM Robustness Medical Image Anomaly Detection
University of Electronic Science and Technology of China, Chengdu, China · Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE
Verification Mirage: Mapping the Reliability Boundary of Self-Verification in Medical VQA
May 11, 2026Ruinan Jin, Beidi Zhao, Myeongkyun Kang +2VLM Evaluation Vision-Language Models
1The University of British Columbia · 2Vector Institute · 3Redmin University of China
Med-StepBench: A Hierarchical Reasoning Framework for Evaluating Hallucinations in Medical Vision-Language Models
May 11, 2026Minh Khoi Nguyen, Dai Lam Le, Amir Reza Jafari +8Healthcare Hallucination Detection
1AI4LIFE, Hanoi University of Science and Technology, Vietnam · 2SAMOVAR, Télécom SudParis, Institut Polytechnique de Paris, France · Military Central Hospital, Vietnam +1
LiteMedCoT-VL: Parameter-Efficient Adaptation for Medical Visual Question Answering
May 10, 2026Runze Ma, Shunbo Jia, Haonan Lyu +2CoT Distillation Healthcare
School of Information Technology Monash University Malaysia · Faculty of Innovation Engineering Macau University of Science and Technology · Department of Bioelectronics Faculty of Biomedical Engineering Shenzhen University of Advanced Technology +1
Cross-Modal Semantic-Enhanced Diffusion Framework for Diabetic Retinopathy Grading
May 10, 2026Yiqun WangDiabetic Retinopathy Grading Medical VLMs
Beijing Jiaotong University · School of Software Engineering Beijing Jiaotong University Beijing, China
CapCLIP: A Vision-Language Representation Alignment Approach for Wireless Capsule Endoscopy Analysis
May 8, 2026Haroon Wahab, Irfan Mehmood, Hassan UgailEndoscopy Medical Image Analysis
School of Computer Science, AI and Electronics Faculty of Engineering and Digital Technologies University of Bradford Bradford, UK · School of Management Faculty of Mgmt, Law & Social Sciences University of Bradford Bradford, UK · Centre for Visual Computing and Intelligent Systems University of Bradford Bradford, UK
MedVIGIL: Evaluating Trustworthy Medical VLMs Under Broken Visual Evidence
May 8, 2026Hanqi Jiang, Junhao Chen, Mingyu Kang +12VLM Evaluation VLM Robustness
University of Georgia · Chungbuk National University · Chungnam National University Hospital +5
Retina-RAG: Retrieval-Augmented Vision-Language Modeling for Joint Retinal Diagnosis and Clinical Report Generation
May 7, 2026Abdelrahman Zaian, Sheethal Bhat, Mohamed Abdalkader +1Multimodal RAG Medical Report Generation
Friedrich-Alexander-Universität, Germany
FairEnc: A Fair Vision-Language Model with Fair Vision and Text Encoders for Glaucoma Detection
May 6, 2026Mohamed Elhabebe, Ayman El-Baz, Qing LiuAlgorithmic Fairness Algorithmic Bias
Center for Machine Vision and Signal Analysis, University of Oulu, Finland · Department of Bioengineering, University of Louisville, USA · The Department of Physics and Engineering, UiT The Arctic University of Norway, Norway
MIRAGE: Retrieval and Generation of Multimodal Images and Texts for Medical Education
May 6, 2026Miguel Diaz Benito, Cecilia Diana Albelda, Alvaro Garcia Martin +3Multimodal IR Educational Technology
Universidad Autónoma de Madrid, 28049 Madrid, Spain

May 23, 2026, cs.CVJ/K move · Enter open · S save

Med-R2: An Adversarial Benchmark for Evidence-Grounded Reasoning in Medical VLMs

Wen Ma, Fucheng Niu, Zhiting Fan, Zikai Xiao+2

Zhejiang University, Zhejiang, China · Guangdong Institute of Intelligence Science and Technology

Adversarial Attacks on VLMs Healthcare Medical VLMs VLM Reasoning Visually Grounded Reasoning Medical Image Grounding+1

Open paper AskPDF arXiv

Vision-language models have demonstrated impressive capabilities in general medical visual question answering, yet due to limited interpretability, it remains unclear whether their predictions reflect evidence-grounded clinical reasoning or reliance on spurious priors. We introduce Med-R2 Bench, a hierarchical benchmark aligned with the clinical workflow to evaluate adversarial robustness with visual grounding. We design stepwise QA tasks to assess whether reasoning chains are strictly grounded in visual evidence across the four clinical stages, and employ adversarial perturbations to test robustness against misleading cues. Med-R2 comprises 42,432 images, 31 task categories, and 110,406 QA pairs. Evaluation across 14 VLMs reveals a sequential performance degradation along the four-stage clinical workflow. Adversarial experiments show that models rely heavily on correct prompts to guess answers. Even when provided with explicit visual cues, the models struggle to accurately align textual descriptions. Finally, we demonstrate stepwise fine-tuning using our hierarchical data significantly improves reasoning robustness, highlighting its potential to drive future improvements in evidence-based medical AI.

Momentum

Related topics

Latest papers 266