Medical VQA

VQA: Visual Question Answering

Follow this topic Ask about this topic

Momentum

10 papers in the last four weeks, up 67% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 115

All topics

Towards Grounded GI Endoscopy VQA via Multi-Task Learning on Small VLMs
Jul 29, 2026Itbaan Safwan, Ramail Khan, Muhammad Annas Shaikh +1Endoscopy Medical VQA
Institute of Business and Administration, Karachi, Pakistan
ClinFusion: A Vision-Centric Multimodal LLM System for Holistic Medical Understanding
Jul 27, 2026Hangjie Yuan, Yichen Qian, Zhiwei Tang +21Medical Report Generation Medical VQA
1DAMO Academy, Alibaba Group, Hangzhou, China · 3Hupan Laboratory, Hangzhou, China · 2DAMO Academy, Alibaba Group, Beijing, China +7
PathScale-R1: Cross-scale Reasoning for Pathological Image Analysis
Jul 26, 2026Chi Phan, Tianyi Zhang, Yufeng Wu +7Computational Pathology Medical VQA
Department of Electrical and Computer Engineering, National University of Singapore, Singapore 117417 · Department of Biomedical Engineering, National University of Singapore, Singapore 117417, Singapore · Department of Pathology, Fujian Medical University Cancer Hospital & Fujian Cancer Hospital, Fuzhou, China +1
Do Pathology Vision-Language Models Truly See Pathology?
Jul 23, 2026Chengyang Zhang, Wenchuan Zhang, Bo Li +10VLM Evaluation Computational Pathology
College of Computer Science, Sichuan University · Department of Pathology and Institute of Clinical Pathology, West China Hospital, Sichuan University · Department of Computer Science, School of Computing, National University of Singapore +2
Pathologist Attention-Aligned Report Generation for Prostate Histopathology
Jul 21, 2026Ruoyu Xue, Suryakant Singh, Souradeep Chakraborty +12Computational Pathology Medical Report Generation
Department of Computer Science, Stony Brook University, USA · Department of Biomedical Informatics, Stony Brook University, USA · Université Paris-Saclay, CentraleSupélec, Gustave Roussy, INSERM, IHU PRISM, Cancer Data Science Unit, France +4
Measuring and Improving Complex-Atomic Answer Consistency in Endoscopic VQA
Jul 20, 2026Yuhao Liu, Cheng Zhao, Guanghui YueVLM Evaluation Endoscopy
School of Biomedical Engineering, Shenzhen University, Shenzhen, China
Can Multimodal Large Language Models Understand OCT?
Jul 18, 2026Baochen Fu, Wenzhi Deng, Baihao Jin +5Medical Imaging Medical Image Analysis
1Shandong University · 4Shandong Provincial Qianfoshan Hospital · University of Science and Technology of China +2
FM $^2$ : Unified Federated Foundation Models for Heterogeneous Multimodal Medical Imaging
Jul 15, 2026Shengchao Chen, Ting ShuMultimodal Federated Learning Cross-Modal Alignment
School of Artificial Intelligence, Shenzhen University Shenzhen, China · Australian AI Institute, University of Technology Sydney Sydney, Australia
Auditing Data Leakage in Whole-Slide Image Multimodal Benchmarks
Jul 14, 2026Wenhao Zhang, Zhongliang Zhou, John Kang +1VLM Evaluation Data Leakage
University of Virginia · Merck & Co., Inc.
An Empirical Analysis of Continual Learning for Heterogeneous Medical Visual Question Answering
Jul 13, 2026Mai A. Shaaban, Tausifa Jan Saleem, Alaa Mohamed +3Medical VQA Continual Learning for VLMs
Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE · Department of Mathematics and Computer Science, Faculty of Science, Alexandria University, Alexandria, Egypt
IRIS: An Intelligent Vision-Language System for Ocular Surface Diseases via Topic Tree and Scene-Driven VQA Generation
Jul 5, 2026Hao Wei, Wenjin Qi, Dasen Dai +2Visual Question Answering Vision-Language Models
The Chinese University of Hong Kong, Shatin, Hong Kong, China · Westlake University, Xihu District, Hangzhou, Zhejiang Province, China
Do Medical Vision Language Models Actually See? A Counterfactual Grounding Framework and Hard-Negative Contrastive Training for Visually-Reliant Medical VLMs
Jul 4, 2026Anas Zafar, Leema Krishna Murali, Siddhant Bharadwaj +2VLM Evaluation Counterfactual Evaluation
The University of Texas MD Anderson Cancer Center · Eisai Inc. · Cohere Labs Community +1
Breaking Failure Cascades: Step-Aware Reinforcement Learning for Medical Multimodal Reasoning
Jun 30, 2026Junha Jung, Minbyul Jeong, Suhyeon Lim +5Process Reward Models Medical VQA
Korea University · AIGEN Sciences · Upstage AI +3
Token-Sparse Medical Multimodal Reasoning via Dual-Stream Reinforcement Learning
Jun 30, 2026Kaitao Chen, Weiqian Zhao, Jiamin Wu +6Efficient Multimodal Inference Medical VQA
1Fudan University · 2Shanghai Artificial Intelligence Laboratory · 3Shanghai Jiao Tong University +1
Synergistic Perception-Reasoning Governance: Grounding Medical MLLMs with Verifiable Anatomical Evidence
Jun 30, 2026Rui Hao, Qiankun Li, Junyuan Mao +4Radiology Report Generation Medical VQA
Huazhong University of Science and Technology, Wuhan, China · Imperial Global Singapore, Imperial College London, London, United Kingdom · Nanyang Technological University, Singapore, Singapore +2
Aloe-Vision: Robust Vision-Language Models for Healthcare
Jun 25, 2026Jaume Guasch-Martí, Enrique Lopez-Cuena, Martín Suárez-Fernández +3VLM Evaluation VLM Robustness
Barcelona Supercomputing Center (BSC)
Just how sure are you? Improving Verbalized Uncertainty Calibration in Medical VQA
Jun 25, 2026Eren Senoglu, Federico Toschi, Nicolo Brunello +2Medical VQA Language Model Calibration
Politecnico di Milano
SurgAtlas: A Large-Scale Surgical Video-Language Dataset with 2,391 Hours of Open and Minimally Invasive Surgery
Jun 24, 2026Filippos Bellos, Andre S. Gala-Garza, Miaowei Wang +8Medical VQA Surgical Video Understanding
University of Michigan Ann Arbor, Michigan, USA · University of Edinburgh Edinburgh, United Kingdom · Vanderbilt University Medical Center Nashville, Tennessee, USA +3
Teach-to-Reason: Competition-Guided Reasoning with a Self-Improving Teacher
Jun 24, 2026Xiao Han, Hao Liu, Zhimin Bao +5Medical VQA Medical VLMs
1AIRC, Midea Group
Multilingual Hematology Visual Question Answering Dataset
Jun 24, 2026Hajra Malik, Hafiza Tooba Aftab, Abdul Rehman +2Medical VQA Medical VLMs
Information Technology University, Lahore · King Edward Medical University, Lahore
Scalable Training of Spatially Grounded 2D Vision-Language Models for Radiology
Jun 18, 2026Yusuf Salcan, Simon Ging, Robin Tibor Schirrmeister +4Radiology Report Generation Radiology VLMs
Computer Vision Group, University of Freiburg, Germany · CRIION-AI Lab, Freiburg, Germany · Adaptive & Agentic AI +3
Evaluation of Medical Vision Language Models HuluMed and MedGemma, and general purpose chatbots Gemma 3, ChatGPT Plus, and Claude Pro on real previously unseen wound images
Jun 16, 2026Yunzhe Xue, Mohammed Saim Ahmed Quadri, Neal Panse +2VLM Evaluation Medical VQA
Department of Data Science, New Jersey Institute of Technology, Newark, NJ, USA · Department of Computer Science, New Jersey Institute of Technology, Newark, NJ, USA · Vascular and Endovascular Surgery, Robert Wood Johnson Hospital, New Brunswick, NJ, USA
Vision-language models for chest radiography do not always need the image
Jun 16, 2026Mahshad Lotfinia, Sebastian Ziegelmayer, Lisa Adams +4VLM Evaluation Medical VQA
Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany · Department of Diagnostic and Interventional Radiology, TUM University Clinic, School of Medicine and Health, Klinikum rechts der Isar, Technical University of Munich, Munich, Germany · Lab for AI in Medicine, RWTH Aachen University, Aachen, Germany +1
Enhancing Pathological VLMs with Cross-scale Reasoning
Jun 16, 2026Chi Phan, Tianyi Zhang, Qiaochu Xue +5Visual Question Answering Computational Pathology
Department of Electrical and Computer Engineering, National University of Singapore, Singapore · PuzzleLogic Pte Ltd, Singapore · Department of Pathology, Fujian Medical University Cancer Hospital & Fujian Cancer Hospital, Fuzhou, China
Training LLMs with Reinforcement Learning over Digital Twin Representations for Reasoning-Intensive Surgical VideoQA
Jun 15, 2026Yiqing Shen, Han Zhang, Mathias UnberathRL for Language Model Reasoning Medical VQA
Johns Hopkins University, Baltimore, MD, USA
Uncertainty Is Not a Safety Net for Clinical VQA, but Can It Anticipate Model Failure?
Jun 15, 2026Arnisa Fazla, Alberto Testoni, Ameen Abu-Hanna +2VLM Evaluation VLM Robustness
Department of Medical Informatics, Amsterdam University Medical Center, University of Amsterdam, Amsterdam, The Netherlands. · Amsterdam Public Health, Methodology, Amsterdam, The Netherlands. · MaiNLP, Center for Information and Language Processing, LMU Munich, Germany. +1
Calibrated Triage, Not Autonomy: Confidence Estimation for Medical Vision-Language Models
Jun 14, 2026Reza Khanmohammadi, Kundan Thind, Mohammad M. GhassemiVLM Evaluation Selective Prediction
Michigan State University East Lansing, MI, USA · Henry Ford Health Detroit, MI, USA
Object Tokens as a Bridge Between Segmentation and Visual Question Answering in Robotic Surgery
Jun 14, 2026Yiping Li, Ronald de Jong, Romy van Jaarsveld +5Medical VQA Surgical Video Understanding
Department of Biomedical Engineering, Eindhoven University of Technology, The Netherlands · Department of Surgery, University Medical Center Utrecht, The Netherlands · Department of Electrical Engineering, Eindhoven University of Technology, The Netherlands
OpenMedQ: Broad Open Pretraining for Medical Vision-Language Models
Jun 11, 2026Ibrahim Gulluk, Max Van Puyvelde, Olivier GevaertMedical VQA Medical Image Classification
Department of Electrical Engineering, Stanford University · Department of Biomedical Data Science, Stanford University School of Medicine · Department of Mathematical Modelling, Statistics and Bioinformatics, Ghent University
OpenMedReason: Scientific Reasoning Supervision for Medical Vision-Language Models
Jun 10, 2026Negin Baghbanzadeh, Pritam Sarkar, Michael Colacci +6Medical VQA Medical VLMs
York University · Vector Institute · University of British Columbia +5

Jul 29, 2026, cs.CVJ/K move · Enter open · S save

Towards Grounded GI Endoscopy VQA via Multi-Task Learning on Small VLMs

Itbaan Safwan, Ramail Khan, Muhammad Annas Shaikh, Muhammad Atif Tahir

Institute of Business and Administration, Karachi, Pakistan

Endoscopy Medical VQA Multi-Task Learning Medical VLMs Medical Image Grounding

Open paper AskPDF arXiv

Gastrointestinal (GI) endoscopic image analysis has shifted from single-label classification toward visual question answering (VQA), where a model must answer free-form clinical questions about an image. While recent vision-language models (VLMs) achieve promising answer accuracy on this task, clinical adoption also requires the model's internal representations to reflect the visual evidence behind its answers. We propose a simple multi-task fine-tuning recipe that constructs auxiliary grounding and description tasks from an existing VQA dataset with minimal additional annotation: expert-annotated polyp masks are reused directly, while a GI-domain pretrained classifier with Grad-CAM localization provides weak supervision for finding categories that lack ground-truth masks. Three small VLM backbones are fine-tuned with low-rank adaptation under matched VQA-only and multi-task recipes on Kvasir-VQA-x1, and we show consistent accuracy gains together with improved implicit alignment between answer tokens and the relevant image region, evaluated on both in-distribution and out-of-distribution data.

Momentum

Related topics

Latest papers 115