Visual Evidence Sensitivity in VLMs

VLM: Vision-Language Model

Follow this topic Ask about this topic

Latest papers

20 papers4 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

When Evidence Vanishes: Distilling Visual-Evidence Dependency in Medical Vision-Language Models
Oct 16, 2026Qizhen Lan, Yiling Ma, Ruiqi Chen +8Medical VLMs Visual Evidence Sensitivity in VLMs
The University of Texas Health Science Center at Houston · Yale University · University of Michigan +3
In-Context Learning Can Help Vision-Language Models Overcome Training Priors
Oct 9, 2026Kun Wang, Xindi Wu, Sanghyuk Chun +2Visual Evidence Sensitivity in VLMs In-Context Learning
Princeton University
Reading Right, Answering Wrong: How Visual Configuration Changes Affect Evidence Use in VLMs
Sep 22, 2026Dingyang Lin, Yingfeng Luo, Chenglong Wang +4Visual Evidence Sensitivity in VLMs VLM Robustness
School of Computer Science and Engineering, Northeastern University, Shenyang, China · NiuTrans Research, Shenyang, China
Look Where It Counts: A Free, Label-Free Visual Evidence Signal for Fine-Grained Vision-Language Reasoning
Sep 21, 2026Santi Ram Tiwari, Nihal Naik, Devbrat Pandey +1Vision-Language Grounding VLM Reasoning
KGraph AI Solutions Pvt. Ltd. Bangalore, India – 560016
ModaLens: Measuring Image Sensitivity in Report-Conditioned Medical VLMs
Sep 14, 2026Sebastián Andrés Cajas Ordóñez, Maximin Lange, Quang Bui +9VLM Evaluation Radiology VLMs
MIT Critical Data, Massachusetts Institute of Technology · King’s College London · American International School Vienna +6
The Visual Insensitivity Gap: Diagnosing When Vision-Language Models Fail to Use Visual Evidence
Sep 1, 2026Genpei ZhangVLM Evaluation Large Vision-Language Models
University of Wisconsin–Madison
Polish Medical Visual Question Answering: Vision-Language Models Underutilize Visual Evidence
Aug 13, 2026Jakub Pokrywka, Łukasz Grzybowski, Antoni Lasik +3Medical VQA Multilingual VLM Evaluation
Adam Mickiewicz University · ARAAI Poland · NASK National Research Institute +2
Seeing or Knowing? Visual Context Sensitivity in Multimodal Large Language Models
Jul 28, 2026Jiaang Li, Chengzu Li, Zhaochong An +4VLM Evaluation Multimodal Large Language Models
University of Copenhagen · University of Cambridge · 3ETH Zürich +1
What Keeps Vision-Language Models Looking at the Image?
Jul 14, 2026Hiroto Osaka, Shohei Taniguchi, Gouki Minegishi +3Vision-Language Models VLM Interpretability
The University of Tokyo
VisualFLIP: Do Predictions Depend on Task-Critical Visual Evidence in Multimodal Reasoning?
Jun 5, 2026Didi Zhu, Changrui Chen, Stefanos Zafeiriou +1VLM Evaluation Multimodal Grounding
Imperial College London
Consistent Yet Wrong: Evidence Insensitivity in Spatial Vision-Language Models
Jun 1, 2026S Divakar Bhat, Toshihiko YamasakiVLM Evaluation Multi-View Consistency
The University of Tokyo, Japan
Real Images, Worse Judgments: Evaluating Vision-Language Models on Concreteness and Imagery
May 26, 2026Yifan Jiang, Ruoxi Ning, Sheng Yao +1VLM Evaluation Vision-Language Models
University of Waterloo · Vector Institute
Seeing without Looking: Do Vision-Language Benchmarks Really Test Vision?
May 21, 2026Zixuan Lan, Luzhe Sun, Matthew R. Walter +1VLM Evaluation VLM Interpretability
University of Chicago · Toyota Technological Institute at Chicago · Stony Brook University
Are VLMs Seeing or Just Saying? Uncovering the Illusion of Visual Re-examination
May 15, 2026Chufan Shi, Cheng Yang, Yaokang Wu +4VLM Interpretability VLM Robustness
University of Southern California · University of California San Diego · Carnegie Mellon Univer +1
Cross-Modal Attention Analysis and Optimization in Vision-Language Models: A Study on Visual Reliability
Apr 19, 2026Lijie ZhouCross-Modal Learning VLM Robustness
School of Computer Science University of Nottingham Ningbo China Ningbo, China
Arbitration Failure, Not Perceptual Blindness: How Vision-Language Models Resolve Visual-Linguistic Conflicts
Apr 10, 2026Farhad Nooralahzadeh, Omid Rohanian, Yi Zhang +2Vision-Language Grounding Visual Evidence Sensitivity in VLMs
Institute of Computer Science, Zurich University of Applied Sciences, Switzerland · University of Oxford, UK.
How Well Do Vision-Language Models Understand Sequential Driving Scenes? A Sensitivity Study
Apr 8, 2026Roberto Brusnicki, Mattia Piccinini, Johannes BetzVisual Evidence Sensitivity in VLMs Temporal Reasoning in VLMs
Professorship of Autonomous Vehicle Systems, TUM School of Engineering and Design, Technical University of Munich, Munich, Germany
CDH-Bench: A Commonsense-Driven Hallucination Benchmark for Evaluating Visual Fidelity in Vision-Language Models
Mar 30, 2026Kesheng Chen, Yamin Hu, Qi Zhou +2Emotional Hallucination in VLMs VLM Hallucination
Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies, Institute of Cyberspace Security, School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen, China
Tinted Frames: Question Framing Blinds Vision-Language Models
Mar 19, 2026Wan-Cyuan Fan, Jiayun Luo, Declan Kutscher +2Vision-Language Models VLM Robustness
University of British Columbia · Vector Institute for AI · University of California, Berkeley
To See or To Please: Uncovering Visual Sycophancy and Split Beliefs in VLMs
Mar 19, 2026Rui Hong, Shuxue QuanSelective Prediction for VLMs Sycophancy in VLMs
George Mason University, Fairfax, VA, USA · Independent Researcher

Oct 16, 2026J/K move · Enter open · S save

When Evidence Vanishes: Distilling Visual-Evidence Dependency in Medical Vision-Language Models

Qizhen Lan, Yiling Ma, Ruiqi Chen, Mengchen Fan+7

The University of Texas Health Science Center at Houston · Yale University · University of Michigan+3

Medical VLMs Visual Evidence Sensitivity in VLMs VLM Distillation Medical VQA

Open paper AskPDF arXiv

Compact medical vision-language models (VLMs) can answer clinical questions correctly without reliably using the image, relying instead on question wording or learned priors. Standard knowledge distillation copies a teacher's answer preference on the original image, but it leaves unchecked whether that preference is conditional on the visual evidence required by the question. We study this missing target as medical visual evidence dependency: support for the correct answer should decrease when the matched evidence is absent. We introduce Evidence-Conditioned Dependency Distillation (ECDD), which pairs each training image with a control matched to the evidence level and distills the teacher's drop in answer margin across the pair. A lower bound objective transfers positive dependency by penalizing only shortfalls, while original-image supervision preserves task accuracy. The same formulation applies when the evidence is a queried clinical region, a surgical instrument, or whole-image visual content. Across three medical VQA settings, ECDD keeps original-image accuracy competitive with distillation and alignment baselines while making answer support more sensitive to evidence removal. Under the DME headline objective, for example, the positive-case margin response to target rather than random masking rises from 0.786 with KD to 2.023 with ECDD, while original-image accuracy differs by 0.11 points. These results suggest that medical VLM distillation should transfer both the teacher's answer preference and its dependence on visual evidence.