Medical VLMs

VLM: Vision-Language Model

Follow this topic Ask about this topic

Momentum

22 papers in the last four weeks, up 22% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 266

All topics

MedBenchAgent: Towards Systematic Automation of Medical VLM Benchmark Construction
Oct 8, 2026Yulin Fu, Junren Wang, Guangjing Yang +5Medical Image Benchmarks VLM Evaluation
Beijing University of Posts and Telecommunications · West China Hospital · Sichuan Provincial Engineering Research Center of Intelligent Diagnosis and Treatment of Breast Diseases
$Δ$ Representation: Geometry Supervised Representation Learning of Phenotypes via Counterfactual Reasoning for Medical VLMs
Oct 7, 2026Hao Wang, Qiwei Zeng, Jinghao Lin +6Medical Image Representation Learning Medical VLMs
The University of Sydney · Shanghai Jiao Tong University · Northeastern University +1
A Deterministic Evidence Layer for Vision-Language Autism Screening from Naturalistic Home Video
Oct 6, 2026Wenqi Li, Mindi Ruan, Chuanbo Hu +2Autism Spectrum Disorder Medical VLMs
Department of Computer Science, University at Albany, SUNY, Albany, NY, USA · Lane Department of Computer Science and Electrical Engineering, West Virginia University, Morgantown, WV, USA · Department of Radiology, Washington University in St. Louis, St. Louis, MO, USA
Frozen Models, Evolving Expertise: Model-Agnostic Learning from Deployment Experience for Multimodal Medical AI
Oct 6, 2026Yexiao He, Yucheng Tang, Pengfei Guo +6Medical LLMs Continual Learning for VLMs
University of Maryland, College Park · NVIDIA
MedCORE: Criteria-Grounded Clinical Reasoning for Interpretable Medical Image Diagnosis
Oct 6, 2026Asim Khan, Samee Ullah Khan, Dwarikanath MahapatraMedical Image Classification Medical VLMs
Deparment of Computer Science, Khalifa University, Abu Dhabi, UAE · Department of Aerosapce Engineering, Khalifa University, Abu Dhabi, UAE
A doctrine-grounded visual question answering dataset for Tactical Combat Casualty Care
Oct 5, 2026Junseob Kim, Jade Chng, Ayman Ali +5Medical VQA Video QA
Department of Electrical and Computer Engineering, Duke University, Durham, NC, USA · Department of Biomedical Engineering, Duke University, Durham, NC, USA · Department of Surgery, Duke University, Durham, NC, USA +3
Ranking-Aware Prompt Optimization for Multimodal Clinical Diagnosis
Sep 30, 2026Tian Xia, Minghao Liu, Yiqing Liang +2Medical Diagnosis Multimodal Large Language Models
Harvard University · University of California, Santa Cruz · Brown University +2
GateSPINE: Gated Cross-View Fusion for Lumbar Spine MRI Report Generation
Sep 30, 2026Hoang Nguyen Van, Cuong Vuong Tuan, Trang Mai Xuan +3Radiology Report Generation Magnetic Resonance Imaging
Applied AI Lab, Phenikaa University, Hanoi, Vietnam · Medical Imaging & Radiological Technology Department, Faculty of Medical Technology, Phenikaa School of Medicine & Pharmacy, Phenikaa University, Vietnam · Radiology & Functional Exploration Center, Phenikaa University Hospital, Vietnam +1
From Given to Gathered Evidence: Agentic Learning for Longitudinal Medical Reasoning
Sep 30, 2026Minye Shao, Chaohui Yu, Yixuan Wu +3AI Agent Benchmarks Medical VLMs
Durham University · DAMO Academy, Alibaba Group · Hupan Laboratory +1
OmniMed-Jev: Calibrating LVLM Confidence for Trustworthy Medical Multimodal Decisions via System One
Sep 30, 2026Luyao Tang, Cheng ChenProbability Calibration Multimodal Classification
Department of Electrical and Computer Engineering The University of Hong Kong, Hong Kong SAR, China
CRAFT: Causal Responsibility and Failure Tracing in Medical Vision Language Models
Sep 30, 2026Chunzheng Zhu, Jiaqi Zeng, Hongbo Zhao +3Vision-Language Models Medical VQA
Hunan University · University of Hong Kong
SAM Meets VLM: Parameter-Decoupled Full-Parameter Training for Unified Medical Reasoning and Segmentation
Sep 29, 2026Xuyang Cao, Enyou Liu, Jun Zhao +3Interactive Medical Image Segmentation Text-Guided Image Segmentation
JDH Algo, JD Health International Inc.
How Medical VLMs Underutilize Their Vision Encoders: A Dermatology Perspective
Sep 29, 2026Janet Wang, Yunbei Zhang, Xiao Wang +1Medical Diagnosis VLM Interpretability
Tulane University · Oak Ridge National Laboratory
InfiMed2: A Generalist Medical Multimodal Foundation Model from Contextual Evidence and Stability-Aware Supervision
Sep 28, 2026Guanghao Zhu, Zeyu Liu, Zhitian Hou +9Multimodal Pretraining Multimodal QA
The Hong Kong Polytechnic University · PolyU-Daya Bay Technology and Innovation Research Institute
VL-AcneSeg: A Vision-Language Framework for Region-Aware Acne Lesion Segmentation
Sep 28, 2026Sukju Oh, Soo Ick Cho, Dae Hun Suh +1Text-Guided Image Segmentation Medical VLMs
Department of Computer Science and Artificial Intelligence, Dongguk University, Seoul 04620, Republic of Korea · InSkin Lab Inc., Seoul 06193, Republic of Korea · Department of Dermatology, Seoul National University College of Medicine, and also with the Department of Dermatology, Seoul National University Hospital, Seoul 03080, Republic of Korea
Clinical Trajectory Alignment for Medical Vision-Language Pre-training
Sep 28, 2026Huimin Yan, Xian Yang, Zhi Wang +1Multimodal Pretraining Longitudinal Medical Image Analysis
Institute of Intelligent Information Processing, Shanxi University, Taiyuan, China · Alliance Manchester Business School, The University of Manchester, Manchester, UK
See, Measure, and Reason: Learning Visually Grounded Reasoning in Pathology
Sep 28, 2026Chengyang Zhang, Wenchuan Zhang, Bo Li +8Computational Pathology Medical VQA
College of Computer Science, Sichuan University · Department of Pathology and Institute of Clinical Pathology, West China Hospital, Sichuan University · Department of Computer Science, School of Computing, National University of Singapore +1
Prompt-Anchored Residual Adaptation for Biomedical Vision-Language Models
Sep 27, 2026Jingxuan Kang, Qianying Yue, Che Liu +1VLM Adaptation Few-Shot Learning
Imperial College London · The Chinese University of Hong Kong
FFM-CP: Cross-Backbone Fusion of Vision-Language Foundation Models for Few-Shot Computational Pathology
Sep 23, 2026Anh-Tien Nguyen, Trung DQ. Dang, Nghiem Tuong Diep +9Computational Pathology Few-Shot Learning
Institute for Predictive Deep Learning for Medicine and Healthcare, Giessen University, Germany · Department of Medical Informatics, University Medicine Gottingen, Germany · Department of Applied Mathematics and Computer Science, Technical University of Denmark, Denmark +8
Benchmarking Off-the-Shelf Multimodal AI Models Against Dermatologists on Patient-Captured Skin Images
Sep 21, 2026Rian Dolphin, Laura KnowlesSkin Lesion Classification Medical VLMs
Independent Research, Dublin, Ireland · School of Medicine, University of Limerick, Ireland
A Vision-Language Foundation Model for Precise and Comprehensive Brain Tumor Diagnosis from Preoperative Multimodal Data
Sep 15, 2026Yinong Wang, Jianwen Chen, Zhou Chen +28Medical VLMs Brain Tumor Classification
Joyce · School of Computing and Data Science, The University of Hong Kong, China · Department of Neurosurgery, Xiangya Hospital, Central South University, China +23
A Unified Vision-Language Model for PSMA PET/CT Report Generation, Visual Question Answering, and Lesion Segmentation
Sep 14, 2026Yang Xing, Jiong Wu, Savas Ozdemir +11Radiology Report Generation Medical VQA
1J. Crayton Pruitt Family Department of Biomedical Engineering, University of Florida, Gainesville, FL, USA · Department of Radiology, University of Florida, Jacksonville, FL, USA · Department of Applied Mathematics & Statistics, Stony Brook University, Stony Brook, NY, USA +3
Option-Aware Retrieval and Task-Specific VLM Adaptation for Medical VQA
Sep 14, 2026Tristan Kirscher, Niklas C. Koser, Soren PirkVLM Adaptation Multiple-Choice Question Answering
ICube, Institut Strauss · ICube Laboratory, CNRS UMR 7357, University of Strasbourg, Strasbourg, France · CLCC Institut Strauss, Strasbourg, France +2
Rethinking Correctness for Uncertainty Estimation in Clinical Prediction with Vision-Language Models
Sep 14, 2026Mingcheng Zhu, Jinning Liang, Tingting ZhuUncertainty Quantification Medical VLMs
University of Oxford, Oxford, United Kingdom
Bridging Vision Foundation Model Priors with CLIP for Spatial-aware Few-shot Anomaly Detection in Medical Images
Sep 14, 2026Juzheng Miao, Yuchen Yuan, Cheng Chen +1Few-Shot Anomaly Detection Medical Image Anomaly Detection
Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong, China · Department of Electrical and Computer Engineering, The University of Hong Kong, Hong Kong, China · School of Biomedical Engineering, The University of Hong Kong, Hong Kong, China +1
Learning to Adapt and Calibrate: Score Distribution Alignment for Few-Shot Uncertainty Prediction in Medical VLMs
Sep 9, 2026Xuan Cuong Ngo, Ngan LeVLM Adaptation Conformal Prediction
University of Arkansas Fayetteville, Arkansas, USA
Layer Selection in VLMs for Zero-Shot OOD Detection via Multi-Resolution Entropy Estimation
Sep 8, 2026Shyam Nandan Rai, Francesco Di Salvo, Sebastian Doerrich +1Vision-Language Models Medical VLMs
xAILab Bamberg, University of Bamberg, Bamberg, Germany
A visual large language foundational model for medical image recognition using clinician-contributed online resources
Sep 7, 2026Lingxuan Hou, Yuhua Xie, Yue Hu +14Medical VQA Medical VLMs
College of Biomedical Engineering, Sichuan University, Chengdu, Sichuan, China · School of Engineering, Computer & Mathematical Sciences, Auckland University of Technology, Auckland, New Zealand · Guang’anmen Hospital, China Academy of Chinese Medical Sciences, Beijing, China +2
MetaStructAtlas: A Grounded 3D Vision-Language Dataset and Benchmark for Functional and Structural Reasoning in Whole-Body PET/CT
Sep 3, 2026Chenguang Zheng, Le Xue, Yichi Zhang +8PET/CT Imaging Medical VLMs
Huashan Hospital & Human Phenome Institute, Fudan University, Shanghai, China · Shanghai Academy of Artificial Intelligence for Science, Shanghai, China · Artificial Intelligence Innovation and Incubation Institute, Fudan University, Shanghai, China +1
Federated LoRA Adaptation of BiomedCLIP Across Four International Chest X-Ray Cohorts
Sep 2, 2026Sanjaya Poudel, Nirajan Kunwor, Manish Dhakal +2Federated Fine-Tuning Federated Learning
North Carolina A&T State University · Tribhuvan University · University of Tennessee-Knoxville +1

Oct 8, 2026, cs.AIJ/K move · Enter open · S save

MedBenchAgent: Towards Systematic Automation of Medical VLM Benchmark Construction

Yulin Fu, Junren Wang, Guangjing Yang, Zhangyuan Yu+4

Beijing University of Posts and Telecommunications · West China Hospital · Sichuan Provincial Engineering Research Center of Intelligent Diagnosis and Treatment of Breast Diseases

Medical Image Benchmarks VLM Evaluation Medical VLMs

Open paper AskPDF arXiv

Large-scale construction of medical vision-language model (VLM) benchmarks is increasingly feasible with richly annotated imaging datasets and large language models (LLMs), yet existing automation largely focuses on generating evaluation items within predefined benchmark specifications. We study the broader problem of automatically deriving the specification itself: what to evaluate, which annotations support each task, and how to translate this evidence into reliable evaluation items. We formulate benchmark construction as constrained compilation, in which the benchmark specification is progressively derived from evaluation requirements, heterogeneous annotations, and medical knowledge. Based on this formulation, we introduce MedBenchAgent, a multi-agent framework with a Benchmark Intermediate Representation (BIR) that encodes task definitions, evidence mappings, evaluation protocols, and item specifications across construction stages. MedBenchAgent separates planning, which derives and verifies the specification, from instantiation, which constructs and audits items under the locked specification. MedBenchAgent achieves a Task-Space F1 of 90.9%, outperforming direct task induction (79.2-80.0%) and prior-guided induction (85.1%); 994 of 1,000 sampled items from correctly identified tasks pass human audit. We further demonstrate portability to a specialized medical domain and evaluate twelve VLMs, revealing task- and setting-specific variation obscured by aggregate scores. These results establish constrained compilation as a scalable and auditable framework for medical VLM benchmark construction beyond question generation.

Momentum

Related topics

Latest papers 266