Visual Evidence Sensitivity in VLMs

VLM: Vision-Language Model

Follow this topic Ask about this topic

Latest papers

73 papers44 papers in the last four weeks, up 1367% on the four weeks before. 0.3% of all new papers.

Did I See It or Think It? Probing Reality Monitoring in Vision-Language Models with Concept Injection
Nov 6, 2026Tina Khezresmaeilzadeh, Seyedarmin Azizi, Mehdi Kamal +1Faithfulness in VLMs Hallucination Detection in VLMs
University of Southern California
Benchmarking Fine-Grained Visual Reasoning for Remote Sensing in Urban Management
Nov 4, 2026Jiawei Zhang, Rizhao Cai, Dayan GuanRemote Sensing Image Understanding Reasoning for VLMs
School of Electronics and Information Engineering, Harbin Institute of Technology, 150000 Harbin, China. · Huawei International PTE LTd, 486066 Singapore, Singapore.
DreamRealm: Exploring and Materializing Hallucination Worlds for Trustworthy Vision-Language Models
Nov 4, 2026Benlei Cui, Chunyang Chai, Yiliang Zhang +6VLM Hallucination VLM Hallucination Mitigation
Yuvion Team, Alibaba Group
Perturb, Repair, Verify: Self-Play Vision-Language Verifiers for Compositional Understanding
Nov 3, 2026Zijun Huang, Yingjun Du, Cees G. M. SnoekCompositional Reasoning in VLMs Visual Evidence Sensitivity in VLMs
University of Southern California · VisLab, University of Amsterdam
VIGOR: Visual Gain Ordering for Hallucination Mitigation in Multimodal Discrete Diffusion Language Models
Nov 3, 2026Junzhe Chen, Tian Qin, Eugenie Shi +3VLM Hallucination Mitigation Diffusion Language Models
Tsinghua University · The University of Sydney · Stanford University +1
Co-Occurrence Priors Override Visual Spatial Evidence: Understanding False Alarms in Vision-Language Models for Construction Struck-By Hazard Detection
Nov 3, 2026Xiaoshan ZhouVisual Evidence Sensitivity in VLMs Counterfactual Evaluation of VLMs
Reviso: Not All Visual Evidence Deserves Equal Attention in Video Captioning
Nov 2, 2026Tengfei Liu, Yang Shi, Yuran Wang +12Video Captioning Visual Evidence Sensitivity in VLMs
PKU · KlingAI Research · HKUST(GZ) +5
MultiInpaintCOCO: Fine-Grained Visual Understanding in Multiple Languages
Nov 2, 2026Andrei-Alexandru Manea, Jindřich Libovický, Philipp J. RöschMultilingual VLM Evaluation Vision-Language Retrieval
Faculty of Mathematics and Physics, Charles University, Czech Republic · Bundeswehr University Munich, Germany
Learning to Look: A Foundation for Visual Understanding
Nov 1, 2026Wenshuai Zhao, Yihao Wang, Benjamin Robson +2Vision-Language Models Visual Evidence Sensitivity in VLMs
Aalto University · ELLIS Institute Finland · University of Oulu
NAS-PO: Native Attention-Strategy Policy Optimization for Vision-Language Models
Nov 1, 2026Ke An, Shizhe Xiang, Yue Liu +5RL for VLMs Attention Intervention in VLMs
Beijing Institute of Technology · Tianjin University · University of Chinese Academy of Sciences +2
Answering Without Arriving: An Evidence Audit for Embodied Question Answering
Nov 1, 2026Shubh Garg, Divyansh Pandey, Niyati Rawal +2Embodied QA Evidence-Grounded QA
Indian AI Research Organisation (IAIRO), GIFT City, Gujarat · Independent Researcher · BITS Pilani +1
Vision-language classification of Torsades de Pointes risk from pseudo-coloured ECG images: do T-wave morphology and colour context help or hinder?
Nov 1, 2026Dario Labrador Alonso, Alaa AlahmadiElectrocardiogram Classification Visual Evidence Sensitivity in VLMs
School of Computing, Newcastle University, Newcastle upon Tyne, NE4 5TG, United Kingdom
See, Think, Use: Diagnosing Perception and Reasoning Failures in VLMs
Nov 1, 2026Jiacheng Ding, Yanliang Qi, Cong Guo +1VLM Evaluation Reasoning for VLMs
University of Memphis · National University of Singapore
CheXPoT: Post-Hoc Visual Verification and Correction of VLM-Generated Radiology Reports
Oct 31, 2026Joey Guo, Rayan Ansari, Alaa YoussefVLM Hallucination Mitigation Radiology VLMs
The Nueva School · Stanford Cardiovascular Institute · Department of Radiology, Stanford School of Medicine
Look at Where You Think: Localization-Head Alignment for Fine-Grained Perception in Thinking VLMs
Oct 29, 2026Yuan Wang, Wei Wu, Hongning WangFine-Grained Visual Perception Reasoning for VLMs
Department of Computer Science and Technology, Tsinghua University · Ant Group
Where Do Vision Language Models Fail at 3D Reasoning?
Oct 29, 2026Khai Le-Duc, Nguyen Kim Hai Bui, An Ngo +11Counting in VLMs Reasoning for VLMs
Telling Is Not Seeing: Textual Personas Rarely Make Vision-Language Models Answer Like the People They Describe
Oct 29, 2026Jiawei ChenVLM Evaluation Visual Evidence Sensitivity in VLMs
Netflix
Grounding Multimodal Reasoning with Evidence-Ablated Negatives
Oct 29, 2026Chenxi Liu, Tianyi Xiong, Yanshuo Chen +1Multimodal Reasoning RL for VLMs
Department of Computer Science, University of Maryland, College Park, College Park, MD 20742
OneScene-Bench: Do Vision-Language Models Integrate Evidence Across Views?
Oct 28, 2026Swathi Suhas, Anurag Das, Bernt Schiele +1VLM Evaluation Reasoning for VLMs
Max Planck Institute for Informatics, Saarland Informatics Campus, Saarbrücken, Germany
Visually Irrelevant Hallucinations in Medical Vision-Language Models
Oct 28, 2026Ke Zou, Nan Zhou, Tao Yang +9VLM Hallucination Hallucination Detection in VLMs
K. Zou is with the Department of Ophthalmology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore 119228. · N. Zhou, T. Yang, M. Liu, L. He, and H. Chen are with the College of Computer Science, Sichuan University, Chengdu 610065, China. · Z. Liao is with the Faculty of Information Technology, Monash University, Clayton, Victoria 3800, Australia. +5
REEL: Same Frames, Different Answers and Different Costs
Oct 28, 2026Xin Liang, Jiangpeng HeVideo-Language Model Evaluation Memory-Augmented Video Understanding
Indiana University Bloomington
When Visual Evidence Is Insufficient: How Instructions, Post-Training, and Internal Activations Shape Refusal in Multimodal Language Models
Oct 27, 2026Ananth K Suresh, Aveen Dayal, Gouthaman KV +2Refusal Mechanisms in Language Models Abstention in VLMs
Indian Institute of Technology Hyderabad
Uncovering Shortcut Amplification in Video RLVR through the Lens of Perceptual Uncertainty Collapse
Oct 27, 2026Jiafeng Liang, Xianhao Ou, Tao Ren +6RL for Video Reasoning Reinforcement Learning with Verifiable Rewards
Harbin Institute of Technology · Peking University · Huawei Technologies Co., Ltd +2
PRISM-BENCH: Diagnosing Grounded and Faithful Multimodal Reasoning Beyond Final-Answer Accuracy
Oct 27, 2026Vansh Bhatia, Mehdi Rezagholizadeh, Parsa Ashrafi Fashi +3MLLM Evaluation Multimodal Model Evaluation
Advanced Micro Devices, Inc. · University of Toronto
Diagnosing and Correcting Bias in MLLM for Long Video Understanding
Oct 26, 2026Xusheng Liang, Jianqiao Sun, Hao Zhang +3Long-Video Understanding Visual Evidence Sensitivity in VLMs
City University of Hong Kong · Shenzhen Loop Institute · Xidian University +2
DecoyCount: Evaluating Visual-Reasoning Misalignment in Counting
Oct 26, 2026Qiuyi, Zhang, Zhen Xu +14Visual Counting Counting in VLMs
Richard · Elorian AI · Mercor
Conviction Without Evidence: Verdict Stability Is Independent of Visual Grounding in Medical Vision--Language Models
Oct 25, 2026Federico Felizzi, Francesco Andrea Causio, Bianca Destro Castaniti +17Medical VLMs Visual Evidence Sensitivity in VLMs
SIIAM, Rome, Italy · Dept. of Computer Science, University of Pennsylvania, USA · UCSC, Rome, Italy +4
Execution Logs Sway VLM Judges of Robot Progress
Oct 25, 2026Weiyan Zhao, Sunny YangVLM Robustness Vision-Language Model-as-a-Judge
NLPQuant AI Inc. · University of Illinois Urbana-Champaign
Humans in Space: Can MLLMs Construct Human-Centric Spatial Understanding via Active Exocentric Exploration?
Oct 25, 2026Tianchen Guo, Tengyue Zhang, Chen Liu +2Spatial Reasoning Benchmarks Multimodal Spatial Reasoning
The University of Queensland · Follow Me AI Pty LTD · Australian Institute for Machine Learning, Adelaide University +1
Grounding on Request: Learning Modality Control in Vision-Language Models
Oct 24, 2026Maxime Dassen, Alexander Martin, Andrew Yates +2Vision-Language Grounding Visual Evidence Sensitivity in VLMs
University of Amsterdam · Johns Hopkins University · Human Language Technology Center of Excellence, Johns Hopkins University

Nov 6, 2026J/K move · Enter open · S save

Did I See It or Think It? Probing Reality Monitoring in Vision-Language Models with Concept Injection

Tina Khezresmaeilzadeh, Seyedarmin Azizi, Mehdi Kamal, Massoud Pedram

University of Southern California

Faithfulness in VLMs Hallucination Detection in VLMs Visual Evidence Sensitivity in VLMs Object Hallucination in VLMs

Open paper AskPDF arXiv

Recent work tests whether language models can introspect by injecting a concept direction into their activations and asking whether they notice it and can name it. These studies are text-only. Vision-language models (VLMs) face an additional problem, known in cognitive psychology as reality monitoring: they must keep internally generated content separate from what they actually see. We therefore ask: can VLMs distinguish injected concepts from image content? We introduce a paradigm that varies the image and the injected concept independently. On each trial, the model reports what it sees, whether a thought was injected, and what that thought is. This design scores both whether the model identifies the concept and which source it assigns to it. Across six VLMs, only Gemma-3-27B reliably identifies injected concepts (35% vs. 3% for steering alone), with almost no false alarms. Even then, the source it reports is fragile. Strong or visual-pathway injections are reported as seen, and a visible object nearly eliminates correct thought reports. The same failure appears in ordinary captioning, where injected objects appear in up to 87% of captions of real photographs. Evaluating multimodal self-reports therefore requires checking both their content and their reported source.