Hallucination Detection in VLMs

VLM: Vision-Language Model

Ask about this topic

Overview

Hallucination detection studies when vision-language models produce claims that are not supported by their visual inputs, such as inventing objects, attributes, or events. Approaches include probing hidden states and attention, measuring uncertainty or consistency under image perturbations, and verifying claims against specific regions or with counterfactual evidence. Benchmarks test object hallucinations, caption and answer errors, fine-grained spans, and specialized settings such as medical images and video. A central challenge is that confidence and accuracy do not guarantee visual grounding, while detectors must generalize across models and tasks without excessive abstention or added inference cost.

Latest papers

CardsList

72 papers5 papers in the last four weeks, down 58% on the four weeks before. 0.0% of all new papers.

Mar 16, 2026

HalDec-Bench: Benchmarking Hallucination Detector in Image Captioning

Hallucination detection in captions (HalDec) assesses a vision-language model's ability to correctly align image content with text by identifying errors in captions that misrepresent the image. Beyond evaluation, effective hallucination detection is also essential for curating high-quality image-caption pairs used to train VLMs. However, the generalizability of VLMs as hallucination detectors across different captioning models and hallucination types remains unclear due to the lack of a comprehensive benchmark. In this work, we introduce HalDec-Bench, a benchmark designed to evaluate hallucination detectors in a principled and interpretable manner. HalDec-Bench contains captions generated by diverse VLMs together with human annotations indicating the presence of hallucinations, detailed hallucination-type categories, and segment-level labels. The benchmark provides tasks with a wide range of difficulty levels and reveals performance differences across models that are not visible in existing multimodal reasoning or alignment benchmarks. Our analysis further uncovers two key findings. First, detectors tend to recognize sentences appearing at the beginning of a response as correct, regardless of their actual correctness. Second, our experiments suggest that dataset noise can be substantially reduced by using strong VLMs as filters while employing recent VLMs as caption generators. Our project page is available at https://dahlian00.github.io/HalDec-Bench-Page/.

Kuniaki Saito, Risa Shinoda, Shohei Tanaka +3

OMRON SINICX · The University of Tokyo · The University of Osaka

Image Captioning EvaluationHallucination Detection in VLMsVLM EvaluationImage Captioning
AskPDF
Mar 8, 2026

Overthinking Causes Hallucination: Tracing Confounder Propagation in Vision Language Models

Vision Language models (VLMs) often hallucinate non-existent objects. Detecting hallucination is analogous to detecting deception: a single final statement is insufficient, one must examine the underlying reasoning process. Yet existing detectors rely mostly on final-layer signals. Attention-based methods assume hallucinated tokens exhibit low attention, while entropy-based ones use final-step uncertainty. Our analysis reveals the opposite: hallucinated objects can exhibit peaked attention due to contextual priors; and models often express high confidence because intermediate layers have already converged to an incorrect hypothesis. We show that the key to hallucination detection lies within the model's thought process, not its final output. By probing decoder layers, we uncover a previously overlooked behavior, overthinking: models repeatedly revise object hypotheses across layers before committing to an incorrect answer. Once the model latches onto a confounded hypothesis, it can propagate through subsequent layers, ultimately causing hallucination. To capture this behavior, we introduce the Overthinking Score, a metric to measure how many competing hypotheses the model entertains and how unstable these hypotheses are across layers. This score significantly improves hallucination detection: 78.9% F1 on MSCOCO and 71.58% on AMBER.

Abin Shoby, Ta Duc Huy, Tuan Dung Nguyen +6

Australian Institute for Machine Learning, University of Adelaide · Hanoi University of Science and Technology · Mohamed bin Zayed University of Artificial Intelligence

Hallucination Detection in VLMsObject Hallucination in VLMsVLM Interpretability
AskPDF
Mar 6, 2026

Lyapunov Probes for Hallucination Detection in Large Foundation Models

We address hallucination detection in Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs) by framing the problem through the lens of dynamical systems stability theory. Rather than treating hallucination as a straightforward classification task, we conceptualize (M)LLMs as dynamical systems, where factual knowledge is represented by stable equilibrium points within the representation space. Our main insight is that hallucinations tend to arise at the boundaries of knowledge-transition regions separating stable and unstable zones. To capture this phenomenon, we propose Lyapunov Probes: lightweight networks trained with derivative-based stability constraints that enforce a monotonic decay in confidence under input perturbations. By performing systematic perturbation analysis and applying a two-stage training process, these probes reliably distinguish between stable factual regions and unstable, hallucination-prone regions. Experiments on diverse datasets and models demonstrate consistent improvements over existing baselines.

Bozhi Luan, Gen Li, Yalan Qin +6

Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, School of Artificial Intelligence, Beihang University · Shanghai University · School of Electronic and Information Engineering, State Key Laboratory of CNS/ATM, Beihang University +2

Hallucination DetectionLLM Hallucination DetectionHallucination Detection in VLMsMultimodal Large Language Models
AskPDF
Mar 5, 2026

HALP: Detecting Hallucinations in Vision-Language Models without Generating a Single Token

Hallucinations remain a persistent challenge for vision-language models (VLMs), which often describe nonexistent objects or fabricate facts. Existing detection methods typically operate after text generation, making intervention both costly and untimely. We investigate whether hallucination risk can instead be predicted before any token is generated by probing a model's internal representations in a single forward pass. Across a diverse set of vision-language tasks and eight modern VLMs, including Llama-3.2-Vision, Gemma-3, Phi-4-VL, and Qwen2.5-VL, we examine three families of internal representations: (i) visual-only features without multimodal fusion, (ii) vision-token representations within the text decoder, and (iii) query-token representations that integrate visual and textual information before generation. Probes trained on these representations achieve strong hallucination-detection performance without decoding, reaching up to 0.93 AUROC on Gemma-3-12B, Phi-4-VL 5.6B, and Molmo 7B. Late query-token states are the most predictive for most models, while visual or mid-layer features dominate in a few architectures (e.g., ~0.79 AUROC for Qwen2.5-VL-7B using visual-only features). These results demonstrate that (1) hallucination risk is detectable pre-generation, (2) the most informative layer and modality vary across architectures, and (3) lightweight probes have the potential to enable early abstention, selective routing, and adaptive decoding to improve both safety and efficiency.

Sai Akhil Kogilathota, Sripadha Vallabha E G, Luzhe Sun +1

Stony Brook University · Toyota Technological Institute at Chicago

Hallucination Detection in VLMs
AskPDF
Mar 3, 2026

Beyond Accuracy: Evaluating Visual Grounding In Multimodal Medical Reasoning

Recent work shows that text-only reinforcement learning with verifiable rewards (RLVR) can match or outperform image-text RLVR on multimodal medical VQA benchmarks, suggesting current evaluation protocols may fail to measure causal visual dependence. We introduce a counterfactual evaluation framework using real, blank, and shuffled images across four medical VQA benchmarks: PathVQA, PMC-VQA, SLAKE, and VQA-RAD. Beyond accuracy, we measure Visual Reliance Score (VRS), Image Sensitivity (IS), and introduce Hallucinated Visual Reasoning Rate (HVRR) to detect cases where models generate visual claims despite producing image-invariant answers. Our findings reveal that RLVR improves accuracy while degrading visual grounding: text-only RLVR achieves negative VRS on PathVQA (-0.09), performing better with mismatched images, while image-text RLVR reduces image sensitivity to 39.8% overall despite improving accuracy. On VQA-RAD, both variants achieve 63% accuracy through different mechanisms: text-only RLVR retains 81% performance with blank images, while image-text RLVR shows only 29% image sensitivity. Models generate visual claims in 68-74% of responses, yet 38-43% are ungrounded (HVRR). These findings demonstrate that accuracy-only rewards enable shortcut exploitation, and progress requires grounding-aware evaluation protocols and training objectives that explicitly enforce visual dependence.

Anas Zafar, Leema Krishna Murali, Ashish Vashist

The University of Texas MD Anderson Cancer Center · Cohere Labs Community · Eisai Inc. +1

Visual GroundingMedical VQACounterfactual Evaluation of VLMsRL for VLMsHallucination Detection in VLMs
AskPDF
Mar 3, 2026

Seeing Clearly without Training: Mitigating Hallucinations in Multimodal LLMs for Remote Sensing

Multimodal large language models (MLLMs) suffer from pronounced hallucinations in remote sensing visual question-answering (RS-VQA), primarily caused by visual grounding failures in large-scale scenes or misinterpretation of fine-grained small targets. To systematically analyze these issues, we introduce RSHBench, a protocol-based benchmark for fine-grained diagnosis of factual and logical hallucinations. To mitigate grounding-induced factual hallucinations, we further propose Relative Attention-Driven Actively Reasoning (RADAR), a training-free inference method that leverages intrinsic attention in MLLMs to guide progressive localization and fine-grained local reasoning at test time. Extensive experiments across diverse MLLMs demonstrate that RADAR consistently improves RS-VQA performance and reduces both factual and logical hallucinations. Code and data will be publicly available at: https://github.com/MiliLab/RADAR

Yi Liu, Jing Zhang, Di Wang +3

School of Computer Science, Wuhan University, China · Zhongguancun Academy, China · School of Computer Science, Chongqing University, China

Remote Sensing VQAVLM Hallucination MitigationMultimodal Large Language ModelsVision-Language GroundingHallucination Detection in VLMsRemote Sensing Benchmarks
AskPDF
Feb 27, 2026

Toward Guarantees for Clinical Reasoning in Vision Language Models via Formal Verification

Vision-language models (VLMs) show promise in drafting radiology reports, yet they frequently suffer from logical inconsistencies, generating diagnostic impressions unsupported by their own perceptual findings or missing logically entailed conclusions. Standard lexical metrics heavily penalize clinical paraphrasing and fail to capture these deductive failures in reference-free settings. Toward guarantees for clinical reasoning, we introduce a neurosymbolic verification framework that deterministically audits the internal consistency of VLM-generated reports. Our pipeline autoformalizes free-text radiographic findings into structured propositional evidence, utilizing an SMT solver (Z3) and a clinical knowledge base to verify whether each diagnostic claim is mathematically entailed, hallucinated, or omitted. Evaluating seven VLMs across five chest X-ray benchmarks, our verifier exposes distinct reasoning failure modes, such as conservative observation and stochastic hallucination, that remain invisible to traditional metrics. On labeled datasets, enforcing solver-backed entailment acts as a rigorous post-hoc guarantee, systematically eliminating unsupported hallucinations to significantly increase diagnostic soundness and precision in generative clinical assistants.

Vikash Singh, Debargha Ganguly, Haotian Yu +5

Case Western Reserve University

VLM ReasoningRadiology VLMsVLM Hallucination MitigationHallucination Detection in VLMs
AskPDF
Feb 25, 2026

Structure and Redundancy in Large Language Models: A Spectral Study via Random Matrix Theory

This thesis addresses two persistent and closely related challenges in modern deep learning, reliability and efficiency, through a unified framework grounded in Spectral Geometry and Random Matrix Theory (RMT). As deep networks and large language models continue to scale, their internal behavior becomes increasingly opaque, leading to hallucinations, fragile generalization under distribution shift, and growing computational and energy demands. By analyzing the eigenvalue dynamics of hidden activations across layers and inputs, this work shows that spectral statistics provide a compact, stable, and interpretable lens on model behavior, capable of separating structured, causal representations from noise-dominated variability. Within this framework, the first contribution, EigenTrack, introduces a real-time method for detecting hallucinations and out-of-distribution behavior in large language and vision-language models. EigenTrack transforms streaming activations into spectral descriptors such as entropy, variance, and deviations from the Marchenko-Pastur baseline, and models their temporal evolution using lightweight recurrent classifiers, enabling early detection of reliability failures before they appear in model outputs while offering interpretable insight into representation dynamics. The second contribution, RMT-KD, presents a principled approach to compressing deep networks via random matrix theoretic knowledge distillation. By interpreting outlier eigenvalues in activation spectra as carriers of task-relevant information, RMT-KD progressively projects networks onto lower-dimensional subspaces through iterative self-distillation, yielding significantly more compact and energy-efficient models while preserving accuracy and dense, hardware-friendly structure.

Davide Ettori

LLM CompressionRepresentation Geometry in Language ModelsKnowledge DistillationOOD DetectionHallucination Detection in VLMsRandom Matrix Theory
AskPDF
Feb 24, 2026

VAUQ: Vision-Aware Uncertainty Quantification for LVLM Self-Evaluation

Large Vision-Language Models (LVLMs) frequently hallucinate, limiting their safe deployment in real-world applications. Existing LLM self-evaluation methods rely on a model's ability to estimate the correctness of its own outputs, which can improve deployment reliability; however, they depend heavily on language priors and are therefore ill-suited for evaluating vision-conditioned predictions. We propose VAUQ, a vision-aware uncertainty quantification framework for LVLM self-evaluation that explicitly measures how strongly a model's output depends on visual evidence. VAUQ introduces the Image-Information Score (IS), which captures the reduction in predictive uncertainty attributable to visual input, and an unsupervised core-region masking strategy that amplifies the influence of salient regions. Combining predictive entropy with this core-masked IS yields a training-free scoring function that reliably reflects answer correctness. Comprehensive experiments show that VAUQ consistently outperforms existing self-evaluation methods across multiple datasets.

Seongheon Park, Changdae Oh, Hyeong Kyu Choi +2

University of Wisconsin-Madison · Nanyang Technological University

Confidence Estimation for VLMsVLM EvaluationHallucination Detection in VLMs
AskPDF
Feb 16, 2026

VIGIL: Tackling Hallucination Detection in Image Recontextualization

We introduce VIGIL (Visual Inconsistency & Generative In-context Lucidity), a benchmark dataset and framework that provides a fine-grained categorization of hallucinations in the multimodal image recontextualization task for large multimodal models. Most existing methods treat hallucinations as a single undifferentiated error. We instead decompose them into five categories, namely Object Visual Fidelity, Background Fidelity, Spatial and Instructional Fidelity, Physical and Integration Fidelity, and Object Omission. We propose a multi-stage detection pipeline that processes recontextualized images through specialized steps targeting all five categories with a coordinated set of open-source models. We evaluate the pipeline on the VIGIL dataset of 1,269 manually annotated samples across five product domains and report macro-F1 per category. The decomposed pipeline reaches the best macro-F1 among open-source detectors. It returns a textual explanation for each detected error, which prior methods for this task do not provide. We openly release VIGIL, along with the detection pipeline and benchmark code, through our GitHub repository: https://github.com/mlubneuskaya/vigil and Data repository: https://huggingface.co/datasets/joannaww/VIGIL.

Joanna Wojciechowicz, Maria Łubniewska, Jakub Antczak +4

Wroclaw University of Science and Technology · Tooploox

Hallucination Detection in VLMsMultimodal Hallucination
AskPDF
Feb 5, 2026

Detecting Misbehaviors of Large Vision-Language Models by Evidential Uncertainty Quantification

%Large vision-language models (LVLMs) have shown substantial advances in multimodal understanding and generation. However, when presented with incompetent or adversarial inputs, they frequently produce unreliable or even harmful content, such as fact hallucinations or dangerous instructions. This misalignment with human expectations, referred to as misbehaviors of LVLMs, raises serious concerns for deployment in critical applications. These misbehaviors are found to stem from epistemic uncertainty, specifically either conflicting internal knowledge or the absence of supporting information. However, existing uncertainty quantification methods, which typically capture only overall epistemic uncertainty, have shown limited effectiveness in identifying such issues. To address this gap, we propose Evidential Uncertainty Quantification (EUQ), a fine-grained method that captures both information conflict and ignorance for effective detection of LVLM misbehaviors. In particular, we interpret features from the model output head as either supporting (positive) or opposing (negative) evidence. Leveraging Evidence Theory, we model and aggregate this evidence to quantify internal conflict and knowledge gaps within a single forward pass. %We extensively evaluate our method across four categories of misbehavior, including hallucinations, jailbreaks, adversarial vulnerabilities, and out-of-distribution (OOD) failures, using state-of-the-art LVLMs, and find that EUQ consistently outperforms strong baselines, showing that hallucinations correspond to high internal conflict and OOD failures to high ignorance. Furthermore, layer-wise evidential uncertainty dynamics analysis helps interpret the evolution of internal representations from a new perspective. The source code is available at https://github.com/HT86159/EUQ.

Tao Huang, Rui Wang, Xiaofei Liu +3

State Key Laboratory of Advanced Rail Autonomous Operation, China · Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence, China · School of Computer Science and Technology, Beijing Jiaotong University, China +2

VLM SafetyUncertainty QuantificationHallucination Detection in VLMs
AskPDF
Nov 14, 2025

Multi-agent Undercover Gaming: Hallucination Removal via Counterfactual Test for Multimodal Reasoning

Hallucination continues to pose a major obstacle in the reasoning capabilities of large language models (LLMs). Although the Multi-Agent Debate (MAD) paradigm offers a promising solution by promoting consensus among multiple agents to enhance reliability, it relies on the unrealistic assumption that all debaters are rational and reflective, which is a condition that may not hold when agents themselves are prone to hallucinations. To address this gap, we introduce the Multi-agent Undercover Gaming (MUG) protocol, inspired by social deduction games like "Who is Undercover?". MUG reframes MAD as a process of detecting "undercover" agents (those suffering from hallucinations) by employing multimodal counterfactual tests. Specifically, we modify reference images to introduce counterfactual evidence and observe whether agents can accurately identify these changes, providing ground-truth for identifying hallucinating agents and enabling robust, crowd-powered multimodal reasoning. MUG advances MAD protocols along three key dimensions: (1) enabling factual verification beyond statistical consensus through counterfactual testing; (2) introducing cross-evidence reasoning via dynamically modified evidence sources instead of relying on static inputs; and (3) fostering active reasoning, where agents engage in probing discussions rather than passively answering questions. Collectively, these innovations offer a more reliable and effective framework for multimodal reasoning in LLMs. The source code can be accessed at https://github.com/YongLD/MUG.git.

Dayong Liang, Xiao-Yong Wei, Linhui Xiao +1

School of Future Technology, South China University of Technology, Guangzhou, China · Department of networked intelligence, Peng Cheng Laboratory, Shenzhen, China · College of Computer Science, Sichuan University, Chengdu, China +1

Multi-Agent LLM SystemsMultimodal ReasoningMultimodal HallucinationHallucination Detection in VLMs
AskPDF