Hallucination Detection in VLMs

VLM: Vision-Language Model

Ask about this topic

Overview

Hallucination detection studies when vision-language models produce claims that are not supported by their visual inputs, such as inventing objects, attributes, or events. Approaches include probing hidden states and attention, measuring uncertainty or consistency under image perturbations, and verifying claims against specific regions or with counterfactual evidence. Benchmarks test object hallucinations, caption and answer errors, fine-grained spans, and specialized settings such as medical images and video. A central challenge is that confidence and accuracy do not guarantee visual grounding, while detectors must generalize across models and tasks without excessive abstention or added inference cost.

Latest papers

CardsList

72 papers5 papers in the last four weeks, down 58% on the four weeks before. 0.0% of all new papers.

  1. HalDec-Bench: Benchmarking Hallucination Detector in Image Captioning

    Mar 16, 2026Kuniaki Saito, Risa Shinoda, Shohei Tanaka +3Image Captioning EvaluationHallucination Detection in VLMs

    OMRON SINICX · The University of Tokyo · The University of Osaka

  2. Overthinking Causes Hallucination: Tracing Confounder Propagation in Vision Language Models

    Mar 8, 2026Abin Shoby, Ta Duc Huy, Tuan Dung Nguyen +6Hallucination Detection in VLMsObject Hallucination in VLMs

    Australian Institute for Machine Learning, University of Adelaide · Hanoi University of Science and Technology · Mohamed bin Zayed University of Artificial Intelligence

  3. Lyapunov Probes for Hallucination Detection in Large Foundation Models

    Mar 6, 2026Bozhi Luan, Gen Li, Yalan Qin +6Hallucination DetectionLLM Hallucination Detection

    Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, School of Artificial Intelligence, Beihang University · Shanghai University · School of Electronic and Information Engineering, State Key Laboratory of CNS/ATM, Beihang University +2

  4. HALP: Detecting Hallucinations in Vision-Language Models without Generating a Single Token

    Mar 5, 2026Sai Akhil Kogilathota, Sripadha Vallabha E G, Luzhe Sun +1Hallucination Detection in VLMs

    Stony Brook University · Toyota Technological Institute at Chicago

  5. Beyond Accuracy: Evaluating Visual Grounding In Multimodal Medical Reasoning

    Mar 3, 2026Anas Zafar, Leema Krishna Murali, Ashish VashistVisual GroundingMedical VQA

    The University of Texas MD Anderson Cancer Center · Cohere Labs Community · Eisai Inc. +1

  6. Seeing Clearly without Training: Mitigating Hallucinations in Multimodal LLMs for Remote Sensing

    Mar 3, 2026Yi Liu, Jing Zhang, Di Wang +3Remote Sensing VQAVLM Hallucination Mitigation

    School of Computer Science, Wuhan University, China · Zhongguancun Academy, China · School of Computer Science, Chongqing University, China

  7. Toward Guarantees for Clinical Reasoning in Vision Language Models via Formal Verification

    Feb 27, 2026Vikash Singh, Debargha Ganguly, Haotian Yu +5VLM ReasoningRadiology VLMs

    Case Western Reserve University

  8. Structure and Redundancy in Large Language Models: A Spectral Study via Random Matrix Theory

    Feb 25, 2026Davide EttoriLLM CompressionRepresentation Geometry in Language Models

  9. VAUQ: Vision-Aware Uncertainty Quantification for LVLM Self-Evaluation

    Feb 24, 2026Seongheon Park, Changdae Oh, Hyeong Kyu Choi +2Confidence Estimation for VLMsVLM Evaluation

    University of Wisconsin-Madison · Nanyang Technological University

  10. VIGIL: Tackling Hallucination Detection in Image Recontextualization

    Feb 16, 2026Joanna Wojciechowicz, Maria Łubniewska, Jakub Antczak +4Hallucination Detection in VLMsMultimodal Hallucination

    Wroclaw University of Science and Technology · Tooploox

  11. Detecting Misbehaviors of Large Vision-Language Models by Evidential Uncertainty Quantification

    Feb 5, 2026Tao Huang, Rui Wang, Xiaofei Liu +3VLM SafetyUncertainty Quantification

    State Key Laboratory of Advanced Rail Autonomous Operation, China · Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence, China · School of Computer Science and Technology, Beijing Jiaotong University, China +2

  12. Multi-agent Undercover Gaming: Hallucination Removal via Counterfactual Test for Multimodal Reasoning

    Nov 14, 2025Dayong Liang, Xiao-Yong Wei, Linhui Xiao +1Multi-Agent LLM SystemsMultimodal Reasoning

    School of Future Technology, South China University of Technology, Guangzhou, China · Department of networked intelligence, Peng Cheng Laboratory, Shenzhen, China · College of Computer Science, Sichuan University, Chengdu, China +1