Overthinking Causes Hallucination: Tracing Confounder Propagation in Vision Language Models
Authors: Abin Shoby, Ta Duc Huy, Tuan Dung Nguyen, Minh Khoi Ho, Qi Chen, Anton van den Hengel, Phi Le Nguyen, Johan W. Verjans, Vu Minh Hieu Phan
Organizations: Australian Institute for Machine Learning, University of Adelaide · Hanoi University of Science and Technology · Mohamed bin Zayed University of Artificial Intelligence
Vision Language models (VLMs) often hallucinate non-existent objects. Detecting hallucination is analogous to detecting deception: a single final statement is insufficient, one must examine the underlying reasoning process. Yet existing detectors rely mostly on final-layer signals. Attention-based methods assume hallucinated tokens exhibit low attention, while entropy-based ones use final-step uncertainty. Our analysis reveals the opposite: hallucinated objects can exhibit peaked attention due to contextual priors; and models often express high confidence because intermediate layers have already converged to an incorrect hypothesis. We show that the key to hallucination detection lies within the model's thought process, not its final output. By probing decoder layers, we uncover a previously overlooked behavior, overthinking: models repeatedly revise object hypotheses across layers before committing to an incorrect answer. Once the model latches onto a confounded hypothesis, it can propagate through subsequent layers, ultimately causing hallucination. To capture this behavior, we introduce the Overthinking Score, a metric to measure how many competing hypotheses the model entertains and how unstable these hypotheses are across layers. This score significantly improves hallucination detection: 78.9% F1 on MSCOCO and 71.58% on AMBER.
Figures & tables
Figure 1: Overthinking leads to object hallucination in Vision-Language Models. Each column corresponds to the top predicted object token at a decoder layer, illustrating the model’s layer-wise reasoning progression. We propose Overthinking Score (S-OT) to measure how much the model shifts among objects across layers. Top : the model demonstrates stable reasoning , quickly converges on a consistent concept (cat) across decoder layers, yielding low S-OT. Bottom : the model shows overthinking , hesitating between semantically co-occurring objects or “confounders” (sink, soap) that bias it towards confidently producing a hallucinated answer (dish) captured by high S-OT.
Figure 2: Left: Confounder Propagation examples. In each example, we illustrate the image and its prefix prompt on top. Under the image, we highlight confounders in the intermediate layers prior to the hallucinated final layer token . We also list the values of different hallucination indicators: SVAR [ 8 ] , MetaToken [ 4 ] , Final Layer Entropy, and our proposed Overthinking Score (S-OT) . The attention map of the fifth layer is also presented following [ 8 ] . Existing methods ignore intermediate-layer token dynamics, missing hallucinations. In contrast, our approach detects by tracing confounder propagation. Right: Histograms of SVAR, MetaToken, Final-Layer Entropy, and our Overthinking Score are shown, with the example positions highlighted. Existing indicators show high overlapping between Real and Hallucinated object distribution, while our Overthinking Score offers noticeably better separation.
LLaVA-1.5
Gemma-3
Qwen3-VL
Hallu.
40.6%
47.9%
58.6%
Real
41.6%
42.4%
55.1%
All
41.5%
43.2%
55.5%
Table 1: Semantic alignment between intermediate-layer tokens and the final-layer decoded token for LLaVA-1.5, Gemma-3, and Qwen3-VL. We report the mean alignment for hallucinated tokens, real tokens, and all tokens. The high alignment implies a semantic influence from the intermediate layers to the final layer, indicating confounder propagation .
Figure 3: Distribution of final layer entropy in LLaVA-1.5, Gemma-3 and Qwen3-VL for hallucinated and real tokens.The strong overlap shows the weakness of entropy as a hallucination predictor.
Figure 4: Left: Correlation between per-layer average entropy and hallucination rate. Most layers exhibit positive correlation, suggesting that increased uncertainty across depth contributes to hallucination. Right: Confounder propagation rate versus the number of unique tokens in the intermediate layers. A higher number of unique tokens is associated with an increased likelihood of confounder propagation. Both measured on LlaVA-1.5
Figure 5: Hallucination detection pipeline. 1) We begin with Prefix Prompting , where the model is asked to predict the next token given an image and a partial prompt. 2) We apply LogitLens to extract the top- p tokens at each decoder layer, revealing how the model’s intermediate hypotheses evolve. 3) In the Feature Extraction step, the top- p tokens distributions are used to compute the Overthinking Score and layer-wise entropy, while image and text attention are measured with respect to the generated token. All features are concatenated into a single feature vector. 4) A Hallucination Detector is trained on these features to identify hallucinated cases.
Figure 6: Qualitative comparison of our method with MetaToken and SVAR. Each row presents an example, and each column shows the prediction made by each method. Correct hallucination detections are highlighted with green boxes, while missed detections are shown in red .
Figure 7: Illustration of an instance where an attention-based mechanism (SVAR) can highly attend to inconsistent objects and fails to detect the hallucinated token laptop . LLaVA highly attends to the book object and leads to confused thought keyboard → laptop → laptop. Our method successfully captures the hallucination by tracing this thought process.
LLaVA-1.5
Gemma-3
Qwen3-VL
Avg.
Method
AUC
AP
F1
AUC
AP
F1
AUC
AP
F1
AUC
AP
F1
SVAR [ 8 ]
85.12
50.68
69.35
74.11
32.75
47.84
75.56
35.69
50.20
78.26
39.71
55.80
HalLoc [ 18 ]
80.38
53.61
73.68
79.27
49.96
67.11
83.85
59.50
74.75
81.17
54.36
71.85
MetaToken (LR) [ 4 ]
85.41
52.59
72.88
73.91
28.78
48.27
79.49
27.60
47.27
79.60
36.32
56.14
MetaToken (GB) [ 4 ]
88.95
61.03
75.95
77.23
34.75
67.15
84.21
38.06
74.43
83.46
44.61
72.51
MetaToken (MLP) [ 4 ]
86.81
55.72
73.89
78.40
33.63
60.03
86.29
47.00
68.03
83.83
45.45
67.32
Table 2: Object hallucination detection performance (in %) on MSCOCO dataset measured using AUC, AP and F1 across three different VLMs.
Method
AUC
AP
F1
SVAR [ 8 ]
79.51
12.78
56.87
HalLoc [ 18 ]
50.00
2.27
49.44
MetaToken (LR) [ 4 ]
79.50
11.55
54.76
MetaToken (GB) [ 4 ]
82.15
23.48
65.54
MetaToken (MLP) [ 4 ]
79.41
14.04
58.54
Ours (LR)
70.76
7.97
53.92
Table 3: OOD object hallucination detection performance (in %) on AMBER dataset and LLaVA-1.5 VLM measured using AUC, AP and F1.
Layers
0-4
5-18
19-31
31
0-31
AUC
85.14
87.37
88.93
83.79
89.73
F1
67.67
71.61
74.75
68.76
75.37
Table 4: AUC and F1 scores using different layer subsets for hallucination detection.
Figure 8: SHAP analysis indicating that both mean entropy and number of unique token are informative for detecting hallucination, while the compound Overthinking Score offers the strongest overall contribution.
Figure 9: Hallucinations concentrate in regions of both high entropy and token diversity (i.e., with MSCOCO & LLaVA-1.5).
LR
Maximum depth
No. of estimators
F1 (%)
0.1
3
100
71.39
0.1
3
200
72.43
0.1
3
300
72.72
0.1
5
100
73.59
0.1
5
200
74.25
0.1
5
300
73.06
Table 5: Results of hyperparameter tuning with GB model.
Hidden units
LR
Optimizer
F1 (%)
32
0.1
Adam
69.50
32
0.1
SGD
77.81
32
0.01
Adam
73.97
32
0.01
SGD
73.48
32
0.001
Adam
76.04
32
0.001
SGD
72.99
Table 6: Results of hyperparameter tuning with MLP model.
Figure 10: Semantic alignment of hallucinated (hallu) and real (real) token instances across the layers (relative index) for LLaVA-1.5, Gemma-3 and Qwen3-VL models.
w/o. Overthinking Score
w. Overthinking Score
Method
AUC
F1
AUC
F1
SVAR [ 8 ]
85.12
69.35
86.67
75.06
HalLoc [ 18 ]
80.38
73.68
88.53
72.97
MetaToken (LR) [ 4 ]
85.41
72.88
87.67
70.09
MetaToken (GB) [ 4 ]
88.95
75.95
89.15
76.14
MetaToken (MLP) [ 4 ]
86.81
73.89
89.23
72.35
Table 7: Impact of overthinking score integration in existing hallucination detection methods.
Figure 11: Semantic Alignment vs Confounding Proppagation Rate in LLaVA-1.5, Qwen3-VL and Gemma-3 models.
Method
AUC
F1
SVAR
76.92
48.86
Ours (GB)
86.36
82.59
Table 8: Performance comparison (%) of SVAR and overthinking-based approach on strong scene prior cases.
AUC
Full features
83.33
w/o. entropy
82.91
w/o. image attention
82.42
w/o. text attention
80.49
w/o. Overthinking Score
79.93
Table 9: Comparison of performance (%) drop by ablating individual features.
Method
Inference time (s)
Greedy Search
4.21
SVAR [ 8 ]
8.07
HalLoc [ 18 ]
5.03
MetaToken (LR) [ 4 ]
5.35
MetaToken (GB) [ 4 ]
5.42
MetaToken (MLP) [ 4 ]
5.53
Table 10: Inference time of various hallucination detection methods.
Figure 12: The above examples illustrates that SVAR fails to detect hallucinated objects when the attention values are active in the intermediate layers.
Figure 13: Some examples of confounding propagation in LLaVA-1.5.
Hallucinations remain a persistent challenge for vision-language models (VLMs), which often describe nonexistent objects or fabricate facts. Existing detection methods typically operate after text generation, making intervention both costly and untimely. We investigate whether hallucination risk can instead be predicted before any token is generated by probing a model's internal representations in a single forward pass. Across a diverse set of vision-language tasks and eight modern VLMs, including Llama-3.2-Vision, Gemma-3, Phi-4-VL, and Qwen2.5-VL, we examine three families of internal representations: (i) visual-only features without multimodal fusion, (ii) vision-token representations within the text decoder, and (iii) query-token representations that integrate visual and textual information before generation. Probes trained on these representations achieve strong hallucination-detection performance without decoding, reaching up to 0.93 AUROC on Gemma-3-12B, Phi-4-VL 5.6B, and Molmo 7B. Late query-token states are the most predictive for most models, while visual or mid-layer features dominate in a few architectures (e.g., ~0.79 AUROC for Qwen2.5-VL-7B using visual-only features). These results demonstrate that (1) hallucination risk is detectable pre-generation, (2) the most informative layer and modality vary across architectures, and (3) lightweight probes have the potential to enable early abstention, selective routing, and adaptive decoding to improve both safety and efficiency.
Sai Akhil Kogilathota, Sripadha Vallabha E G, Luzhe Sun +1
Stony Brook University · Toyota Technological Institute at Chicago
Large Vision-Language Models (LVLMs) often suffer from object hallucination, generating objects that are absent from the image. Prior work largely attributes this to insufficient visual attention. However, we find that both real and hallucinated objects receive equally strong visual attention in the model's mid-to-late layers, suggesting that the key issue may not be how much the model attends, but what it attends to and why. To this end, we decode the visual features of high-attention regions using Logit Lens, and observe that regions corresponding to real objects can be correctly decoded to the target object tokens, whereas those for hallucinated objects cannot. Building on this, we identify two hallucination mechanisms: (i) visual uncertainty, triggered by semantically similar or confusable regions; masking these regions eliminates the hallucination. (ii) contextual prior, triggered by strong co-occurrence priors; even when the initially attended region is masked, the hallucination persists and attention drifts to other regions. Based on these findings, we propose a simple yet effective training-free Detect-Mitigate framework comprising a Logit-Lens Consistency Check to detect hallucination and targeted remedies: High-Attention Regions Masking (HARM) for visual uncertainty hallucination, and Visual Evidence Enhanced Decoding (VEED) for contextual prior hallucination. Our approach achieves state-of-the-art results on multiple hallucination benchmarks. Code will be available.
Zichuan Wang, Songlin Yang, Bo Peng +4
School of Artificial Intelligence, University of Chinese Academy of Sciences · 3Hong Kong University of Science and Technology · 2New Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Science
Large vision-language models (LVLMs) achieve strong performance on visual reasoning tasks but remain highly susceptible to hallucination. Existing detection methods predominantly rely on coarse, whole-image measures of how an object token relates to the input image. This global strategy is limited: hallucinated tokens may exhibit weak but widely scattered correlations across many local regions, which aggregate into deceptively high overall relevance, thus evading the current global hallucination detectors. We begin with a simple yet critical observation: a faithful object token must be strongly grounded in a specific image region. Building on this insight, we introduce a patch-level hallucination detection framework that examines fine-grained token-level interactions across model layers. Our analysis uncovers two characteristic signatures of hallucinated tokens: (i) they yield diffuse, non-localized attention patterns, in contrast to the compact, well-focused attention seen in faithful tokens; and (ii) they fail to exhibit meaningful semantic alignment with any visual region. Guided by these findings, we develop a lightweight and interpretable detection method that leverages patch-level statistical features, combined with hidden-layer representations. Our approach achieves up to 90% accuracy in token-level hallucination detection, demonstrating the superiority of fine-grained structural analysis for detecting hallucinations.
Tuan Dung Nguyen, Minh Khoi Ho, Qi Chen +8
Hanoi University of Science and Technology · Australian Institute for Machine Learning, University of Adelaide · Mohamed bin Zayed University of Artificial Intelligence +2