Faithfulness in VLMs

VLM: Vision-Language Model

Follow this topic Ask about this topic

Momentum

5 papers in the last four weeks, up 67% on the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 29

All topics

Visual Evidence Under Cross-Examination: Evaluating and Controlling Decision-Level Evidence Use in Vision-Language Models
Oct 7, 2026Huiyao Zhang, Jin Bai, Zilong Su +6VLM Evaluation Faithfulness in VLMs
Technology and Engineering Center for Space Utilization, Chinese Academy of Sciences · University of Chinese Academy of Sciences · University of Science and Technology of China
SpatialChain: A Benchmark for Auditing Spatial Reasoning Faithfulness in VLMs
Oct 5, 2026Rafael Teixeira Sousa, Vinícius Paulo Lopes de Oliveira, Elisa Ayumi Masasi de Oliveira +5VLM Evaluation Spatial Reasoning Benchmarks
Universidade Federal de Mato Grosso (UFMT) · Universidade Federal de Goiás (UFG) · Advanced Knowledge Center for Immersive Technologies (AKCIT)
Tracing the Evidence: Faithful Token Attribution Through Vision-Language Reasoning
Sep 29, 2026Bowen Yuan, Danny Wang, Ruihong Qiu +2Large Vision-Language Models VLM Interpretability
The University of Queensland
Beyond Saying Less: Fine-Grained Alignment for Informative and Faithful Vision-Language Models
Sep 28, 2026Xingming Long, Jie Zhang, Yuecong Min +2Credit Assignment in RL Step-Level Credit Assignment in RL
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Zhongguancun Academy
When VLMs Trust Context: Evaluating Scene Text Recognition under Misleading Context
Sep 28, 2026Yuxing Cheng, Yuan Wu, Yi ChangVLM Evaluation Counterfactual Evaluation of VLMs
School of Artificial Intelligence, Jilin University · Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, MOE, China · International Center of Future Science, Jilin University
EDCT-Bench: Uncovering Faithfulness Gaps in VLMs via Explanation-Driven Counterfactual Testing
Sep 16, 2026Sihao Ding, Santosh Vasa, Aditi Ramadwar +1VLM Evaluation VLM Robustness
Mercedes-Benz Research & Development North America
I Don't Miss You, but I Do: Self-Explanation Faithfulness of Modality Missingness in Vision-Language Models
Sep 7, 2026Aydin Javadov, Daniel Schoess, Florian von WangenheimVLM Evaluation VLM Interpretability
ETH Zurich
Does Playing it Safe Count as Faithfulness? Reassessing LVLM Hallucination Mitigation Methods
Sep 1, 2026Mehrdad Fazli, Sina Mansouri, Mohit Marvania +1VLM Evaluation Faithfulness in VLMs
Department of Computer Science, George Mason University Fairfax, VA 22030, USA
Visual Attention Faithfulness in Vision-Language Models is Heterogeneous
Sep 1, 2026Xurui Song, Weishi Wang, Zhongqi Yue +5VLM Evaluation Visual Question Answering
SAP · Microsoft · College of Computing and Data Science, Nanyang Technological University, Singapore
LookBack: Where and How to Score LVLM Responses via Visual Reference Usage
Aug 12, 2026Beomsik Cho, Jinhyeong Kim, Dongseok Lee +1VLM Evaluation Hallucination Detection in VLMs
Yonsei University
Messages, Not Tokens: Grounded Coresets for Faithful VLM Compression
Aug 3, 2026Long Qian, Jiaqi Wei, Bingke Zhu +2Efficient VLM Inference Coreset Selection
Foundation Model Research Center, Institute of Automation, Chinese Academy of Sciences, Beijing, China · School of Engineering, Cardiff University, Cardiff, United Kingdom · School of Future Technology, University of Chinese Academy of Sciences, Beijing, China +1
DiffuseAgent-MI: Distributionally-Grounded,Tool-Integrated Self-Evolving Agents for Faithful Visual Reasoning
Aug 1, 2026An Lanji, Dawei Liu, Jin Li +3AI Agent Reliability VLM Interpretability
University of Electronic Science and Technology of China, Chengdu, China.
Role-Break in Attention Heads: Understanding and Detecting Hallucinations in VLMs
Jul 31, 2026Mingyu Wang, Weilin Jin, Wenbo Li +5Hallucination Detection in VLMs Attention Head Analysis
Peking University, Beijing, China · Joy Future Academy, Beijing, China
FaithEyes: Towards Faithful Tool Use via Multi-Agent Process-Image Self-Verification
Jul 30, 2026Haoqing Wang, Xingrun Xing, Ziheng Li +2Tool Use in VLMs VLM Reasoning
Samsung Research, Beijing, China · Peking University
Position, Not Provenance: Separating Reasoning Mediation from Sycophancy in Medical Vision-Language Models
Jul 29, 2026Supratik Bhowal, Subhrajyoti Basu, Aritra Gir Mahanta +1Medical VQA Medical VLMs
Department of Computer Science (AIML) IEM Kolkata, School of UEMK Kolkata, India · Department of Computer Science and Business System Heritage Institute of Technology Kolkata, India · Department of Computer Science and Engineering Indian Institute of Information Technology, Kalyani Kalyani, India
Be Faithful When Response: Returning Fluent and Grounded Answers for Vision-Language Models Reinforcement Learning
Jun 29, 2026Peng, Lee, Yin Zhang +9Vision-Language Grounding VLM Reasoning
GMLab · Hong Kong Polytechnic University · XPENG Robotics
FADE: Mitigating Hallucinations by Reducing Language-Prior Dominance in Large Vision-Language Models
Jun 28, 2026Yichen Guo, Kai Tang, Fenglai Lin +5Vision-Language Models Large Vision-Language Models
1Nanyang Technological University · 2State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · 3Tsinghua University
Dismantling Pathological Shortcuts: A Causal Framework for Faithful LVLM Decoding
Jun 25, 2026Liu Yu, Can Chen, Ping Kuang +3Object Hallucination in VLMs Faithfulness in VLMs
School of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan, China · School of Computer Science, University of Auckland, Auckland, New Zealand
CaVe-VLM-CoT: An Interpretable Vision-Language Model Framework
Jun 16, 2026Sneha Rao, Shaina Raza, Dhanesh RamachandramVLM Evaluation Vision-Language Grounding
Vector Institute, Toronto, ON M5G 0C6, Canada
Mirage Probes: How Vision Models Fake Visual Understanding
Jun 11, 2026Daniel Ben-Levi, Judah Goldfeder, Weiliang Zhao +5Vision-Language Models VLM Interpretability
Columbia University · Intuit · Technion +2
YARD: Y-Architecture Register Decoding for Efficient Hallucination Mitigation in Large Vision-Language Models
May 29, 2026Ting Chen, Geng Li, Guohao Chen +5Efficient VLM Inference Object Hallucination in VLMs
Guangdong University of Technology · Nanyang Technological University · Shenzhen TENCLASS Technology Co., Ltd.
The Abstraction Gap in Vision-Language Causal Reasoning
May 27, 2026Chinh Hoang, Mohammad Rashedul HasanVLM Evaluation Causal Reasoning in Language Models
Department of Electrical and Computer Engineering, University of Nebraska–Lincoln, Lincoln, Nebraska, USA.
Faithful-MR1: Faithful Multimodal Reasoning via Anchoring and Reinforcing Visual Attention
May 21, 2026Changyuan Tian, Zhicong Lu, Huaxing Liu +7Multimodal Large Language Models Multimodal Reasoning
AMAP, Alibaba Group · University of Chinese Academy of Sciences · Tsinghua University +1
Ablate-to-Validate: Are Vision-Language Models Really Using Continuous Thought Tokens?
May 20, 2026Tianyi Zhang, Mahtab Bigverdi, Ranjay KrishnaVLM Evaluation VLM Interpretability
University of Washington
Mitigating Hallucinations in Large Vision-Language Models via Causal Route Gating
May 20, 2026Zhe Cheng, Wenyu Chen, Fode Zhang +1VLM Robustness Hallucination in Language Models
Center of Statistical Research, School of Statistics and Data Science, Southwestern University of Finance and Economics, Chengdu, China. · Department of Biomedical Engineering, College of Design and Engineering, National University of Singapore, Singapore.
When Language Overwrites Vision: Over-Alignment and Geometric Debiasing in Vision-Language Models
May 7, 2026Harshvardhan Saini, Samyak Jha, Yiming Tang +1Vision-Language Models Object Hallucination in VLMs
Indian Institute of Technology Dhanbad · National University of Singapore
Instruction-Evidence Contrastive Dual-Stream Decoding for Grounded Vision-Language Reasoning
Apr 28, 2026Yashwant Pravinrao Bangde, Debaditya RoyObject Hallucination in VLMs Vision-Language Grounding
Department of Computer Science and Engineering Indian Institute of Technology Kharagpur Kharagpur, India 721302
Global Context or Local Detail? Adaptive Visual Grounding for Hallucination Mitigation
Apr 27, 2026Yubo Jiang, Xin Yang, Abudukelimu Wuerkaixi +7Hallucination in Language Models Object Hallucination in VLMs
School of Astronautics, Beihang University, Beijing 102206, China · Longcat Interaction Team, Meituan, Beijing 100102, China · Tianmushan Laboratory, Beihang University, Hangzhou 311115, China
Breaking the Illusion: When Positive Meets Negative in Multimodal Decoding
Apr 22, 2026Yubo Jiang, Yitong An, Xin Yang +7Vision-Language Models Hallucination in Language Models
School of Astronautics, Beihang University, Beijing 102206, China · Longcat Interaction Team, Meituan, Beijing 100102, China · Tianmushan Laboratory, Beihang University, Hangzhou 311115, China
TAR: Temporal Anchor-Constrained Reasoning for Video Temporal Grounding
Aug 11, 2025Chaohong Guo, Xun Mo, Yongwei Nie +3Temporal Video Grounding Vision-Language Models
South China University of Technology · Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ) · Bytedance Inc.

Oct 7, 2026, cs.CVJ/K move · Enter open · S save

Visual Evidence Under Cross-Examination: Evaluating and Controlling Decision-Level Evidence Use in Vision-Language Models

Huiyao Zhang, Jin Bai, Zilong Su, Rui Guo+5

Technology and Engineering Center for Space Utilization, Chinese Academy of Sciences · University of Chinese Academy of Sciences · University of Science and Technology of China

VLM Evaluation Faithfulness in VLMs VLM Interpretability Vision-Language Grounding

Open paper AskPDF arXiv

Vision-language models increasingly reason through crops, regions, and tool-produced observations. Yet an observation can influence the answer without benefiting the candidate it supports. We study candidate-bound visual contribution: valid evidence should help, invalidating its supporting relation should remove its additional effect, and valid rebinding should redirect that effect to the newly supported candidate. We introduce CROSS-Bench, a benchmark of 28,000 decision problems, with matched invalidation and rebinding tests on a dedicated evaluation subset. Our RIVET interface preserves evidence identity and uncertainty, composes a candidate-conditioned response, and separately controls its strength. Shared-evidence experiments show that task accuracy and evidence ownership can diverge. Under matched capacity and training, RIVET increases normalized effect transfer from 0.512 to 0.651 where clean evidence has a positive effect. The advantage persists on common evaluation examples and across repeated decision-layer fits. With evidence predicted from raw inputs, RIVET improves CROSS-Bench accuracy by an average of 5.70 pp across four frozen backbones, relative to the same models without auxiliary evidence. These results separate the utility of visual evidence from the candidate-specific destination of its effect.

Momentum

Related topics

Latest papers 29