Medical VLMs

VLM: Vision-Language Model

Overview

Medical vision-language models study how to link images such as X-rays, CT scans, pathology slides, and surgical videos with clinical language. Common approaches include CLIP-style image-text pretraining, adapting large multimodal language models with medical data, and adding region-level grounding, segmentation, or retrieval of clinical evidence. They are evaluated on tasks such as report generation on MIMIC-CXR, visual question answering on VQA-RAD and SLAKE, and volumetric CT or whole-slide pathology analysis. A central challenge is ensuring that answers genuinely use relevant visual evidence and remain calibrated under prompt changes, rare findings, and shifts between hospitals or populations.

Latest papers

CardsList

515 papers44 papers in the last four weeks, down 4% on the four weeks before. 0.3% of all new papers.

  1. SurgXBench: Explainable Vision-Language Model Benchmark for Surgery

    May 16, 2025Jiajun Cheng, Xianwu Zhao, Sainan Liu +5VLM EvaluationSurgical Video Understanding

  2. CAG-VLM: Fine-Tuning of a Large-Scale Model to Recognize Angiographic Images for Next-Generation Diagnostic Systems

    May 8, 2025Yuto Nakamura, Satoshi Kodera, Haruki Settai +7Medical VLMsRadiology Report Generation

  3. Synergistic Vision-Language Reinforcement Enables Scalable On-Demand Analysis across Diverse Clinical Tasks

    May 6, 2025Haonan Wang, Jiaji Mao, Lehan Wang +11Image SegmentationMedical Imaging Foundation Models

  4. A Large-Scale Vision-Language Dataset Derived from Open Scientific Literature to Advance Biomedical Generalist AI

    Mar 26, 2025Alejandro Lozano, Min Woo Sun, James Burgess +16Medical VLMs