Medical VLMs

VLM: Vision-Language Model

Overview

Medical vision-language models study how to link images such as X-rays, CT scans, pathology slides, and surgical videos with clinical language. Common approaches include CLIP-style image-text pretraining, adapting large multimodal language models with medical data, and adding region-level grounding, segmentation, or retrieval of clinical evidence. They are evaluated on tasks such as report generation on MIMIC-CXR, visual question answering on VQA-RAD and SLAKE, and volumetric CT or whole-slide pathology analysis. A central challenge is ensuring that answers genuinely use relevant visual evidence and remain calibrated under prompt changes, rare findings, and shifts between hospitals or populations.

Latest papers

CardsList

515 papers44 papers in the last four weeks, down 4% on the four weeks before. 0.3% of all new papers.

  1. NativeSeg: Native Vision Language Segmentation of 3D Computed Tomography via Spatial Visual Tokens and Verified Local Zoom

    Nov 18, 2026Chengqian Ma, Gaole Dai, Zijie Meng +53D Medical Image SegmentationText-Guided Image Segmentation

  2. ORGANIC3D: Grounding 3D CT Vision Language Models in Organ Level Image Text Coupling

    Nov 17, 2026Zelong Liu, Hamza Ahmed, Wynn Wingyi Lee +7Medical VLMs3D VLMs

  3. Reference-Guided Data Curation for 3D Medical Vision-Language Pretraining

    Nov 17, 2026Faidra Patsatzi, Ashwin Kumar, Chong Wang +3Training Data CurationMedical VLMs

  4. Visual Harness Scales General VLMs to Medical Images

    Nov 17, 2026Shaowen Wan, Hanqi Jiang, Zihao Wu +6Medical VLMsTool Use in VLMs

  5. OmniTongue: A Comprehensive MLLM Benchmark for Automated Tongue Diagnosis in Traditional Chinese Medicine

    Nov 17, 2026Liuyi Xu, Yun Guo, Shusheng Li +3Medical VLMsMedical Diagnosis

  6. Efficient Self-Supervised Adaptation of 3D Abdominal Vision-Language Model for Institution-Specific HCC Classification via Full Fine-Tuning and PEFT

    Nov 17, 2026Febryan Putra Kartika, Cheng-Yu Ma, Ying-Jia Lin +2VLM AdaptationMedical VLMs

  7. SCAN: Site-Aware Collaborative Privacy-Preserving Adaptation of Vision-Language Models for Neurological Disease Classification

    Nov 16, 2026Nikhil J. Dhinagar, Rishab Pendam, Harmony S. Cen +7Medical Image ClassificationVLM Adaptation

  8. MedJev: A System One Vision-Language Decision Model for Fast and Effective Medical Image Understanding

    Nov 16, 2026Fan Zhang, Hao Wu, Jinyang Wu +5Medical VLMsVLM Inference

  9. What Matters for Cross-Modal Alignment

    Nov 15, 2026Yipeng Sun, Jiajun Wang, Siyuan Mei +4Cross-Modal AlignmentVision-Language Alignment

  10. Multimodal AI in Healthcare: Review of Vision-Language Foundation Models for Real-World Medical Applications

    Nov 15, 2026Taha Razzaq, Murtaza Taj, Asim IqbalMedical VLMsMedical Imaging Foundation Models

  11. EndoComparer: Comparative Surgical Video Question Answering with Pseudo-Siamese Visual Adaptation and Perceive-then-Compare Reasoning

    Nov 14, 2026Shi Li, Lalith Sharan, Joël L. Lavanchy +8Surgical VQAVideo Question Answering

  12. CARET: Clinical Assertion-Aware Residual Calibration for Medical Image-Text Alignment

    Nov 13, 2026Yifan Jin, Tao LiuRadiology VLMsMedical VLMs

  13. MAGIC: Modality-Aware Gini Inspection for Clinical Hallucination Detection in LMMs

    Nov 11, 2026Sachin Acharya, Niyoj Oli, Shishir Adhikari +4Hallucination Detection in VLMsHallucination Detection

  14. Supervision-Aware Evaluation of MedGemma for Reference-Assisted Radiotherapy Contour Ranking

    Nov 11, 2026Jaden Moon, Caiwen Jiang, Songlin Zhao +1VLM EvaluationMedical VLMs

  15. A frontier machine intelligence for human neuroimaging

    Nov 11, 2026Chenhui Zhao, Akshay Rao, Akhil Kondepudi +19Radiology VLMsMedical VLMs

  16. Aggregate Accuracy Hides Diagnosis-Specific Failure in Medical Vision-Language Models

    Nov 11, 2026Shreya Ajay Kale, Angad Ripudaman Singh BajwaMedical VLMsVLM Evaluation

  17. NuJEV: Structured Visual Judgments for Consistent Nuclei Quantification

    Nov 8, 2026Yonghuang Wu, Pengfei Wang, Zhenyang Liang +4Microscopy Image AnalysisMedical VLMs

  18. Anatomy-Activated Mixture-of-Experts for 3D Medical Vision-Language Pre-training

    Nov 7, 2026Szymon Płotka, Gizem Mert, Pedro R. A. S. Bassi +9Medical VLMsMedical Image Representation Learning

  19. CXR-CMFA: A Cross-Modal Feature Aggregation Network for Chest X-Ray Pathology Classification

    Nov 6, 2026Talukder Hasnat Zadid, Jahin Sultana, Felipe Bochini Brisola +4Chest X-Ray ClassificationMedical VLMs

  20. VoluCompose: Constructing Compact Volumetric Visual Representations for Medical Image Understanding

    Nov 5, 2026Zanting Ye, Yunrui Lin, Shengyuan Liu +4Visual Token CompressionMultimodal Token Compression

  21. Multimodal Hierarchical Mixture-of-Experts for 3D Medical Vision-Language Modeling

    Nov 5, 2026Gizem Mert, Szymon Płotka, Pedro R. A. S. Bassi +5Medical VLMs3D VLMs

  22. TACT-KV: Tri-Axis Cosine Transform for Compressing Volumetric KV Caches in Medical VLMs

    Nov 5, 2026Chongyu Qu, Ritchie Zhao, Yufan He +12KV-Cache CompressionKV-Cache Quantization

  23. A 3D foundation model unifying CT interpretation and segmentation

    Nov 4, 2026Yulu Xie, Zilong Li, Shen Xu +14Medical Imaging Foundation ModelsMedical Image Analysis

  24. SurgEye: Instruction Tuning-Free Perception-Reasoning Framework for Surgical Scene Understanding

    Nov 2, 2026Qing Xu, Xiechen Shen, Xianjian He +9Surgical Video UnderstandingMedical VLMs

  25. Every Pixel Is an Option: A System One Model for Medical Ultrasound Biometry

    Nov 2, 2026Moein Heidari, Junbo Rao, Yudan Chen +6Anatomical Landmark DetectionMedical VLMs

  26. SurgPace: Complexity-Aware Progressive Distillation for Surgical Multimodal Large Language Models

    Nov 2, 2026Zhicheng He, Xiaohan Xing, Hongqiu Wang +7VLM DistillationMultimodal Large Language Models

  27. Findings-aware Language-Image Pretraining for CT Volume Interpretation

    Nov 2, 2026Takuro Shimaya, Junya Inoh, Masahiro SaikoMedical VLMsMedical Image Representation Learning