Visual Evidence Sensitivity in VLMs

VLM: Vision-Language Model

Follow this topic Ask about this topic

Latest papers

73 papers44 papers in the last four weeks, up 1367% on the four weeks before. 0.3% of all new papers.

InfiniteWorld: Separating Visual Evidence from Familiar Physics
Oct 24, 2026Tong Zhang, Yuxuan Cheng, Yun PengPhysical Reasoning Visual Evidence Sensitivity in VLMs
Fudan University · The Chinese University of Hong Kong, Shenzhen
The Answer Knows Where to Look: Zooming before MLLMs Answer under Visual-Token Budgets
Oct 23, 2026Zhensheng Hu, Guanglong Xu, Guanqiang HuLarge Vision-Language Models Visual Evidence Sensitivity in VLMs
Integrated Hospital of Traditional Chinese Medicine, Southern Medical University, Guangzhou, China · Institute of Medical Engineering Interdisciplinary Research, Xinjiang Medical University, Urumqi, China · School of Mathematics and Statistics, Zhaoqing University, Guangdong 526061, China +1
PEAR: Adaptive Advantage Reshaping for Perception and Exploration in Large Vision-Language Models
Oct 23, 2026Junxin Wang, Yingfeng Luo, Xiangchen Song +12Reasoning for VLMs Multimodal Reasoning
School of Computer Science and Engineering, Northeastern University, Shenyang, China · University of Michigan, Ann Arbor, US · Kunming University of Science and Technology, Kunming, China +1
MAGIC: Margin-gated Contrastive Debiasing with Evidence Restoration for VLM Test-time Robustness Scaling
Oct 23, 2026Shunshun Gu, Enze Zhang, Ming Pei +2VLM Robustness Test-Time Scaling for VLMs
University of Wisconsin–Madison · University of North Carolina at Chapel Hill
Watch and Learn: Teaching Visual In-Context Learning by Removing Textual Shortcuts
Oct 22, 2026Nimrod Shabtay, Asaf Yehudai, Avihu Dekel +2Multimodal ICL In-Context Learning
IBM Research · Tel-Aviv University
RECAP: Looking Once Is Not Enough for Vision-Language Reasoning
Oct 22, 2026Zhaolu Kang, Tailong Luo, Chenxin Li +13Reasoning for VLMs Visual Evidence Sensitivity in VLMs
Peking University · Chengdu Minto Tech
Loop to Look Better: Learning Adaptive Depth for Question-Guided Perception in Vision-Language Models
Oct 22, 2026Wencheng Ye, Anning Hu, Hongye Fang +2Visual Evidence Sensitivity in VLMs
Tongji University · Shanghai Jiao Tong University
Diagnosing and Mitigating Evidence Misgrounding in Multimodal Knowledge Editing
Oct 21, 2026Zhiwu Wang, Yichen Wu, Jinqian Yang +4Multimodal Knowledge Editing Knowledge Editing
School of Mathematics and Statistics, Xi’an Jiaotong University, Xi’an, Shaanxi, China · Harvard Medical School, Boston, MA 02155, U.S. · School of Mathematics and Statistics, Xidian University, Xi’an, Shaanxi, China +2
Encoded but Unused: Bounding the Sources of VLM Errors
Oct 21, 2026Anat Veikherman, Mor Ventura, Nitay Calderon +2VLM Interpretability Visual Evidence Sensitivity in VLMs
Faculty of Electrical and Computer Engineering, Technion · Faculty of Data and Decision Science, Technion
DISTRACT: Degradation-Indexed Study of Textual Reference Across Competing Targets
Oct 20, 2026Dan Zlotnikov, Ohad Ben-ShaharMultimodal Visual Object Tracking VLM Robustness
Ben-Gurion University of the Negev, Israel
Give the Model a Finger: Developmentally Aligned Evaluation of Counting in Vision-Language Models
Oct 20, 2026Yingxin Jia, Hui Geng, Wenxin Yang +1Counting in VLMs Vision-Language Models
Independent Researcher
Steering Vision-Language Models for Visual-Contextual Privacy in Location Disclosure
Oct 19, 2026Wenyue Chen, Yuhan Cheng, Ruixin Yang +5Privacy Protection in VLMs Visual Evidence Sensitivity in VLMs
Duke University · University of Washington · Georgia Institute of Technology +1
What Language Adds, and at What Cost: Four Controlled Probes of Pathology Vision-Language Models
Oct 19, 2026Wenhao Zhang, Zhongliang Zhou, Yi Ping +2Medical VLMs VLM Evaluation
School of Data Science, University of Virginia, Charlottesville, VA 22903, USA · Merck & Co., Inc., Rahway, NJ 07065, USA
Robustness of Internal Signals for Hallucination Detection in Vision-Language Models
Oct 19, 2026Allison Chen, William Yang, Salma Abdel Magid +3Hallucination Detection in VLMs VLM Robustness
Department of Computer Science, Princeton University, Princeton, NJ 08544
ViCoR: Estimating Visual Necessity via Counterfactual Residuals for Multimodal Medical Data Selection
Oct 18, 2026Haotian Gan, Yudong Li, Xintian Li +2Medical VLMs VLM Robustness
Tsinghua University · The University of Hong Kong · China Agricultural University
When Better Policies Leave Harder Failures: Coverage Shift in VLM Success Monitoring
Oct 17, 2026Isabel Guan, Dingyuan Zhang, Shipeng Lyu +1Visual Evidence Sensitivity in VLMs
The Hong Kong University of Science and Technology · Nanyang Technological University · The Hong Kong Polytechnic University
Are We Evaluating VLM Token Pruning Right? Looking Beyond Accuracy and Token Count
Oct 17, 2026Geng Li, Guohao Chen, Shilin Shan +1Visual Token Pruning VLM Evaluation
Nanyang Technological University
VisConf: Quality-Aware Consensus for Best-of-N in Vision-Language Model Reasoning
Oct 17, 2026Muhammad Usman, Muhammad Qasim Ayub, Usman Ahad +1Reasoning for VLMs Confidence Estimation for VLMs
Department of Computer Science, Lahore University of Management Sciences · Department of Electrical Engineering, Lahore University of Management Sciences
Investigating Length Bias and Robustness in LVLMs for Multiple-Choice Question Answering
Oct 17, 2026Md. Atabuzzaman, Hani Alomari, Chris ThomasVLM Robustness Multiple-Choice Question Answering
Department of Computer Science, Virginia Tech
When Evidence Vanishes: Distilling Visual-Evidence Dependency in Medical Vision-Language Models
Oct 16, 2026Qizhen Lan, Yiling Ma, Ruiqi Chen +8Medical VLMs Visual Evidence Sensitivity in VLMs
The University of Texas Health Science Center at Houston · Yale University · University of Michigan +3
RL Teaches VLMs Where to Attend but Not How to Perceive
Oct 14, 2026Ziqian Zhang, Junjie Liao, Zhining Zhang +1RL for VLMs Visual Evidence Sensitivity in VLMs
National University of Singapore · Beijing Normal University · Peking University +1
In-Context Learning Can Help Vision-Language Models Overcome Training Priors
Oct 9, 2026Kun Wang, Xindi Wu, Sanghyuk Chun +2Visual Evidence Sensitivity in VLMs In-Context Learning
Princeton University
Reading Right, Answering Wrong: How Visual Configuration Changes Affect Evidence Use in VLMs
Sep 22, 2026Dingyang Lin, Yingfeng Luo, Chenglong Wang +4Visual Evidence Sensitivity in VLMs VLM Robustness
School of Computer Science and Engineering, Northeastern University, Shenyang, China · NiuTrans Research, Shenyang, China
Look Where It Counts: A Free, Label-Free Visual Evidence Signal for Fine-Grained Vision-Language Reasoning
Sep 21, 2026Santi Ram Tiwari, Nihal Naik, Devbrat Pandey +1Vision-Language Grounding VLM Reasoning
KGraph AI Solutions Pvt. Ltd. Bangalore, India – 560016
ModaLens: Measuring Image Sensitivity in Report-Conditioned Medical VLMs
Sep 14, 2026Sebastián Andrés Cajas Ordóñez, Maximin Lange, Quang Bui +9VLM Evaluation Radiology VLMs
MIT Critical Data, Massachusetts Institute of Technology · King’s College London · American International School Vienna +6
The Visual Insensitivity Gap: Diagnosing When Vision-Language Models Fail to Use Visual Evidence
Sep 1, 2026Genpei ZhangVLM Evaluation Large Vision-Language Models
University of Wisconsin–Madison
Polish Medical Visual Question Answering: Vision-Language Models Underutilize Visual Evidence
Aug 13, 2026Jakub Pokrywka, Łukasz Grzybowski, Antoni Lasik +3Medical Visual Question Answering Multilingual VLM Evaluation
Adam Mickiewicz University · ARAAI Poland · NASK National Research Institute +2
Seeing or Knowing? Visual Context Sensitivity in Multimodal Large Language Models
Jul 28, 2026Jiaang Li, Chengzu Li, Zhaochong An +4VLM Evaluation Multimodal Large Language Models
University of Copenhagen · University of Cambridge · 3ETH Zürich +1
What Keeps Vision-Language Models Looking at the Image?
Jul 14, 2026Hiroto Osaka, Shohei Taniguchi, Gouki Minegishi +3Vision-Language Models VLM Interpretability
The University of Tokyo
VisualFLIP: Do Predictions Depend on Task-Critical Visual Evidence in Multimodal Reasoning?
Jun 5, 2026Didi Zhu, Changrui Chen, Stefanos Zafeiriou +1VLM Evaluation Multimodal Grounding
Imperial College London

Oct 24, 2026J/K move · Enter open · S save

InfiniteWorld: Separating Visual Evidence from Familiar Physics

Tong Zhang, Yuxuan Cheng, Yun Peng

Fudan University · The Chinese University of Hong Kong, Shenzhen

Physical Reasoning Visual Evidence Sensitivity in VLMs Video Reasoning Counterfactual Evaluation of VLMs

Open paper AskPDF arXiv

Physical reasoning requires adapting familiar expectations to the dynamics of the observed world. Existing benchmarks establish broad physical capabilities, yet an overall score leaves unclear whether answers follow visual evidence when familiar rules no longer apply. We introduce InfiniteWorld, a benchmark that pairs physical-reasoning questions with simulator videos whose physical parameters are systematically altered. Its generation pipeline connects simulation, scene screening, paired video translation, question construction, and input checks. Five tasks assess law identification, parameter estimation, temporal reasoning, analytic counterfactual comparison, and law induction. The evaluation defines five matched input settings for each question, comprising text-only evaluation and ordered or shuffled observations from simulator and translated videos. We evaluate a diverse set of video-language models under these matched settings. The results expose a persistent preference for familiar physical rules. Frames improve performance for most models, but gains differ across tasks. Strong accuracy on Earth-normal questions coexists with poor recognition of altered dynamics, and Earth-normal answers remain common when the observed physics changes. Task scores and semantic agreement further distinguish matching physical factors from matching their amount, and correct answers from repeated errors. InfiniteWorld makes these behaviors measurable through paired changes in the evidence supplied for the same question, separating visual evidence from familiar physics.