VLM Hallucination Mitigation

VLM: Vision-Language Model

Follow this topic Ask about this topic

Momentum

11 papers in the last four weeks, down 15% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 131

All topics

Detect Before You Leap: Mirage Detection in Vision-Language Models
May 29, 2026Md. Shaown Miah, S. M. Taiabul Haque, Syed Ishtiaque Ahmed +1Hallucination Detection in VLMs Vision-Language Models
Indiana University Indianapolis · University of Toronto
Hyperbolic and Evidence-Prioritized Experts for Large Vision-Language Models
May 29, 2026Zijie Zhou, Dandan Zhu, Hangxiangpan Wang +3Vision-Language Models Large Vision-Language Models
China University of Petroleum (Beijing) · Hainan Institute of China University of Petroleum (Beijing) · South China Normal University
YARD: Y-Architecture Register Decoding for Efficient Hallucination Mitigation in Large Vision-Language Models
May 29, 2026Ting Chen, Geng Li, Guohao Chen +5Efficient VLM Inference Object Hallucination in VLMs
Guangdong University of Technology · Nanyang Technological University · Shenzhen TENCLASS Technology Co., Ltd.
Learning from Fine-Grained Visual Discrepancies: Mitigating Multimodal Hallucinations via In-Context Visual Contrastive Optimization
May 29, 2026Haolin Deng, Xin Zou, Zhiwei Jin +3Vision-Language Models Direct Preference Optimization
The Hong Kong University of Science and Technology (Guangzhou) · OPPO AI Center · The Hong Kong University of Science and Technology
Mitigating Hallucination in Vision-Language Models through Barrier-Regulated Adaptive Closed-form Steering
May 28, 2026Soumyadeep Jana, Pulkit Mittal, Sanasam Ranbir SinghHallucination in Language Models Large Vision-Language Models
Indian Institute of Technology Guwahati, India
Risk-aware Selective Prompting for Hallucination Mitigation in Large Vision-Language Models
May 27, 2026Yuang Huang, Yafeng Zhang, Yu ZilanVision-Language Models Hallucination in Language Models
Shanghai Jiao Tong University · iFLYTEK · Tsinghua University
Rethinking Visual Neglect: Steering via Context-Preference for MLLM Hallucination Mitigation
May 27, 2026Jingwen Wu, Xijun Zhang, Ge SongLLM Hallucination Mitigation Hallucination in Language Models
School of Computer and Electronic Information, Nanjing Normal University, China
Adversarial Orthogonal Disentanglement for LVLM Hallucination Mitigation
May 25, 2026Ruoxi Cheng, Haoxuan Ma, Zhengfei Hai +6Adversarial Representation Learning Vision-Language Models
Fudan University · Tencent · Nanjing University +3
Universal Boosts, Specific Suppressors: Sparse Autoencoder Steering of Medical Vision-Language Models
May 24, 2026Farhad Nooralahzadeh, Benjamin Gundersen, Nicolas Deperrois +7Radiology Report Generation Healthcare
University of Zurich and University Hospital of Zurich, Switzerland · Zurich University of Applied Sciences, Switzerland · Kobe University, Japan +3
Mitigating Object Hallucinations in Vision-Language Models through Region-Aware Attention Recalibration
May 24, 2026Yuanzhi Xu, Qian Gao, Jun Fan +4VLM Robustness Vision-Language Models
Qilu University of Technology (Shandong Academy of Sciences) Jinan, China · China Telecom Digital Intelligence Technology Co, Ltd Jinan, China · Shenyang Aerospace University Shenyang, China +1
Correcting Visual Blur Induced by Attention Distraction to Reduce Hallucinations: Algorithm and Theory
May 23, 2026Quanjiang Li, Zhiming Liu, Wei Luo +2LLM Hallucination Mitigation Hallucination in Language Models
National University of Defense Technology, China · Harbin Institute of Technology (Shenzhen), China · Xi’an Jiaotong University, China.
CHASD: Language Increment-Calibrated Contrastive Decoding against Hallucination in LVLMs
May 22, 2026Xiaoyi Huang, Kejia Zhang, Zhiming LuoHallucination Detection in VLMs Efficient VLM Inference
Institute of Artificial Intelligence, Xiamen University · Department of Artificial Intelligence, Xiamen University
Reducing Object Hallucination in LVLMs via Emphasizing Image-negative Tokens
May 20, 2026Meng Shen, Minghao Wu, Deepu RajanVision-Language Models Hallucination in Language Models
Nanyang Technological University · Monash University
Mitigating Hallucinations in Large Vision-Language Models via Causal Route Gating
May 20, 2026Zhe Cheng, Wenyu Chen, Fode Zhang +1VLM Robustness Hallucination in Language Models
Center of Statistical Research, School of Statistics and Data Science, Southwestern University of Finance and Economics, Chengdu, China. · Department of Biomedical Engineering, College of Design and Engineering, National University of Singapore, Singapore.
Finding the Correct Visual Evidence Without Forgetting: Mitigating Hallucination in LVLMs via Inter-Layer Visual Attention Discrepancy
May 20, 2026Yutong Xie, Zhenglin Hua, Ran Wang +3Vision-Language Models Hallucination in Language Models
School of Computer Science and Engineering, Southeast University, Nanjing, China · School of Artificial Intelligence, Shenzhen University, Shenzhen, China · National Engineering Laboratory for Big Data Systems Computing Technology, Shenzhen University, Shenzhen, China +3
HalluCXR: Benchmarking and Mitigating Hallucinations in Medical Vision-Language Models for Chest Radiograph Interpretation
May 19, 2026Haoyu Wang, Zitong LiVision-Language Models Audio-Language Model Hallucination Detection
Department of Biostatistics & Health Informatics, Institute of Psychiatry, Psychology & Neuroscience, King’s College London, London, UK
Causal Evidence for Attention Head Imbalance in Modality Conflict Hallucination
May 19, 2026Jinrui Jiang, Zhangtai Wu, Zhen Wu +1Hallucination in Language Models Multimodal Large Language Models
National Key Laboratory for Novel Software Technology, Nanjing University, China · School of Artificial Intelligence, Nanjing University, China
MHSA: A Lightweight Framework for Mitigating Hallucinations via Steered Attention in LVLMs
May 14, 2026Wei Ding, Yilin Li, Yudong Zhang +4Vision-Language Models LLM Hallucination Mitigation
Tsinghua University Beijing, China · Tsinghua University, Tencent Beijing, China · Tencent Beijing, China +2
Do We Really Need External Tools to Mitigate Hallucinations? SIRA: Shared-Prefix Internal Reconstruction of Attribution
May 14, 2026Tian Qin, Junzhe Chen, Yuqing Shi +3Efficient VLM Inference Hallucination in Language Models
Tsinghua University · The University of Sydney · Stanford University +1
Reducing Hallucination in Vision-Language Models via Stage-wise Preference Optimization under Distribution Shift
May 13, 2026Qinwu XuVLM Robustness Hallucination in Language Models
Meta AI
Instruction Lens Score: Your Instruction Contributes a Powerful Object Hallucination Detector for Multimodal Large Language Models
May 12, 2026Runhe Lai, Xinhua Lu, Yanqi Wu +3Hallucination Detection in VLMs Hallucination Detection
School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China · Peng Cheng Laboratory, Shenzhen, China · Key Laboratory of Machine Intelligence and Advanced Computing, MOE, Guangzhou, China.
When Looking Is Not Enough: Visual Attention Structure Reveals Hallucination in MLLMs
May 12, 2026Fanpu Cao, Xin Zou, Xuming Hu +1Hallucination Detection in VLMs Hallucination in Language Models
Thrust of Artificial Intelligence, HKUST (Guangzhou), China · Department of Computer Science and Engineering, HKUST, Hong Kong SAR, China
Not Blind but Silenced: Rebalancing Vision and Language via Adversarial Counter-Commonsense Equilibrium
May 11, 2026Qingxin Xiao, Peilin Zhao, Yangyang Zhao +2VLM Robustness Hallucination in Language Models
South China University of Technology · Institute for Super Robotics (Huangpu) · Shanghai Jiao Tong University +1
Vocabulary Hijacking in LVLMs: Unveiling Critical Attention Heads by Excluding Inert Tokens to Mitigate Hallucination
May 11, 2026Yangneng Chen, Junlin Li, Weijun Yao +4Hallucination in Language Models VLM Hallucination
1Harbin Institute of Technology (Shenzhen), China · 2Huawei Technologies Co., Ltd. · 3The Hong Kong Polytechnic University +1
When Language Overwrites Vision: Over-Alignment and Geometric Debiasing in Vision-Language Models
May 7, 2026Harshvardhan Saini, Samyak Jha, Yiming Tang +1Vision-Language Models Object Hallucination in VLMs
Indian Institute of Technology Dhanbad · National University of Singapore
CAST: Mitigating Object Hallucination in Large Vision-Language Models via Caption-Guided Visual Attention Steering
May 6, 2026Qiming Li, Zekai Ye, Xiaocheng Feng +9Hallucination in Language Models Large Vision-Language Models
1Harbin Institute of Technology · 2Peng Cheng Laboratory · 3Harbin Institute of Technology, Shenzhen
Query-Dependent Use of Generated Descriptions for Reliable Visual Question Answering
May 3, 2026Zeshang Li, Shuoyang ZhangVisual Question Answering Hallucination in Language Models
University of International Relations
Online Self-Calibration Against Hallucination in Vision-Language Models
May 1, 2026Minghui Chen, Chenxu Yang, Hengjie Zhu +3Vision-Language Models Hallucination in Language Models
Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China
Prefill-Time Intervention for Mitigating Hallucination in Large Vision-Language Models
Apr 28, 2026Chengsheng Zhang, Chenghao Sun, Xinyan Jiang +2Vision-Language Models Hallucination in Language Models
University of Science and Technology of China · Shanghai Advanced Research Institute, Chinese Academy of Sciences, Shanghai, China · University of Chinese Academy of Sciences, Beijing, China
Global Context or Local Detail? Adaptive Visual Grounding for Hallucination Mitigation
Apr 27, 2026Yubo Jiang, Xin Yang, Abudukelimu Wuerkaixi +7Hallucination in Language Models Object Hallucination in VLMs
School of Astronautics, Beihang University, Beijing 102206, China · Longcat Interaction Team, Meituan, Beijing 100102, China · Tianmushan Laboratory, Beihang University, Hangzhou 311115, China

May 29, 2026, cs.CVJ/K move · Enter open · S save

Detect Before You Leap: Mirage Detection in Vision-Language Models

Md. Shaown Miah, S. M. Taiabul Haque, Syed Ishtiaque Ahmed, Sayeed Shafayet Chowdhury

Indiana University Indianapolis · University of Toronto

Hallucination Detection in VLMs Vision-Language Models Hallucination Detection Vision-Language Alignment VLM Hallucination Mitigation

Open paper AskPDF arXiv

Vision-language models (VLMs) can produce confident answers without relevant visual evidence, a failure mode known as mirage reasoning (Asadi et al., 2026). To that end, we study pre-release mirage detection: deciding whether a VLM answer should be released or withheld. Our model-agnostic method, Text-Conditioned Layer-wise Internal Alignment (TC-LIA), tracks question-image alignment across the layers of a frozen CLIP ViT-H/14 encoder, summarizing patch-text alignment by final similarity, late-layer top-k alignment, early-to-late gain, and slope. TC-LIA is purely unsupervised (fixed projections, fixed scoring weights, no labels, no training) and already delivers strong detection independently. Additionally, when combined with blank/noise detection, domain routing, and VLM self-assessment, it forms an ensemble whose supervised training improves performance but is an optional add-on. On 19,004 samples spanning ten VQA domains, fourteen state-of-the-art VLMs exhibit 57.3-75.0% base mirage rates. Our proposed TC-LIA alone cuts this to 7.5% with 83.5% Related/Unrelated/Blank-Noise classification accuracy, and the ensemble reaches 84.3-88.4% accuracy with 5.9-7.2% mirage rates (best joint result: 88.4% accuracy, 6.4% mirage rate). Notably, an ensemble trained on a single backbone transfers well to unseen backbones, with the best-transferring source staying within 1.2% accuracy points of per-backbone training across thirteen held-out VLMs.

Momentum

Related topics

Latest papers 131