HALP: Detecting Hallucinations in Vision-Language Models without Generating a Single Token

Sai Akhil Kogilathota; Sripadha Vallabha E G; Luzhe Sun; Jiawei Zhou

cs.CVMar 5, 2026

HALP: Detecting Hallucinations in Vision-Language Models without Generating a Single Token

Authors: Sai Akhil Kogilathota, Sripadha Vallabha E G, Luzhe Sun, Jiawei Zhou

Organizations: Stony Brook University · Toyota Technological Institute at Chicago

Abstract

Hallucinations remain a persistent challenge for vision-language models (VLMs), which often describe nonexistent objects or fabricate facts. Existing detection methods typically operate after text generation, making intervention both costly and untimely. We investigate whether hallucination risk can instead be predicted before any token is generated by probing a model's internal representations in a single forward pass. Across a diverse set of vision-language tasks and eight modern VLMs, including Llama-3.2-Vision, Gemma-3, Phi-4-VL, and Qwen2.5-VL, we examine three families of internal representations: (i) visual-only features without multimodal fusion, (ii) vision-token representations within the text decoder, and (iii) query-token representations that integrate visual and textual information before generation. Probes trained on these representations achieve strong hallucination-detection performance without decoding, reaching up to 0.93 AUROC on Gemma-3-12B, Phi-4-VL 5.6B, and Molmo 7B. Late query-token states are the most predictive for most models, while visual or mid-layer features dominate in a few architectures (e.g., ~0.79 AUROC for Qwen2.5-VL-7B using visual-only features). These results demonstrate that (1) hallucination risk is detectable pre-generation, (2) the most informative layer and modality vary across architectures, and (3) lightweight probes have the potential to enable early abstention, selective routing, and adaptive decoding to improve both safety and efficiency.

Figures & tables

Figure 1: Pre-generation hallucination prediction scores on examples across 8 different VLM task domains (benchmarking data distribution in Fig. 3 (a)) for Qwen2.5-VL-7B. The probes scores reflect how likely the probe thinks the model will hallucinate based on different pre-generation features (VF, VT, QT as defined in Sec. 3.3 ).

Figure 2: HALP detection pipeline. We extract three types of representations: (1) visual feature vectors from the encoder, (2) vision token states from decoder layers at the final vision patch position, and (3) query token states from the same layers at the final query position. These representations are used independently to probe for hallucination prediction prior to decoding.

Figure 3: Hallucination detection dataset distributions of task domains, answer formats, and hallucination question types.

Dataset	Focus	Samples
AMBER ( An et al., 2023 )	discriminative tasks	3,926
HaloQuest ( Cheng et al., 2023 )	adversarial challenges	2,784
POPE ( Li et al., 2023a )	object hallucination	1,230
MME ( Fu et al., 2023 )	multimodal reasoning	885
HallusionBench ( Guan et al., 2023 )	visual illusions	617
MathVista ( Lu et al., 2023 )	mathematical reasoning	558

Table 1: Our hallucination detection dataset composition.

Model Name	VF	VT	QT	Average
Gemma3-12B	0.6736	0.5956	0.9349	0.7347
FastVLM-7B	0.6830	0.7028	0.6136	0.6665
LLaVa-Next-8B	0.6108	0.6270	0.9026	0.7135
Molmo-V1-7B	0.6830	0.6867	0.9193	0.7630
Qwen2.5-VL-7B	0.7873	0.6683	0.9150	0.7902
Llama-3.2-11B-Vision	0.7703	0.7377	0.8959	0.8013

Table 2: AUROC performance for hallucination detection probes across different vision-language models and different model representations.

Figure 4: AUROC scores for the final Vision Token (VT) representation at different decoder layers.

Figure 5: AUROC scores for the final Query Token (QT) representation at different decoder layers.

Hallucination Type	Count	FastVLM-7B	Gemma3-12B	LLaVA-Next-8B	Llama-3.2-11B	Molmo-V1	Phi4-VL	Qwen2.5-VL-7B	SmolVLM2-2.2B
Attribute-Related	3493	7.58%	11.84%	19.73%	9.58%	14.30%	8.51%	4.87%	20.47%
Object-Related	1622	5.81%	10.42%	8.93%	7.27%	10.91%	6.80%	6.38%	8.33%
Relationship	1720	6.74%	5.12%	15.29%	4.91%	7.44%	3.73%	0.87%	2.62%
Other	3165	13.52%	13.08%	14.66%	14.46%	14.66%	16.85%	9.13%	10.21%

Table 3: Hallucination rates by basic hallucination type across eight vision–language models. Bold values indicate the lowest hallucination rate among tested models per category.

Hallucination Type	Rate (%)	VT AUROC	QT AUROC	$\Delta$ AUROC
Attribute-Related	11.92	0.567	0.804	0.237
Object-Related	7.94	0.734	0.878	0.144
Relationship	6.06	0.559	0.708	0.149
Other	13.47	0.755	0.827	0.072

Table 4: Hallucination rates and AUROC performance by error type averaged across models.

\Delta

AUROC shows the performance gain from query token (QT) over visual features (VF) representations.

Figure 6: Hallucination types and probe scores to predict possible hallucinations before decoding, with Qwen2.5-VL-7B. Higher scores indicate greater chance of answer hallucination.

Domain	Rate (%)	QT AUROC
Temporal & Video	46.90	0.456
Knowledge & Identity	21.19	0.694
Math & Calculation	16.79	0.819
Visual Understanding	10.26	0.859
Attribute Recognition	7.14	0.863
General QA	7.04	0.797

Table 5: Hallucination rates and query token AUROC performance by application domain, ordered by hallucination rate.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Model Name	Token	Vision	Decoder Layers
Model Name	Type	Encoder	Layer 0	Layer $L/4$	Layer $L/2$	Layer $3L/4$	Layer $L$
Gemma3-12B	VF	0.6736	-	-	-	-	-
	VT	-	0.6538	0.6698	0.6146	0.5995	0.5956
	QT	-	0.7165	0.8119	0.9247	0.9315	0.9349
FastVLM-7B	VF	0.6830	-	-	-	-	-
	VT	-	0.6935	0.6957	0.7093	0.6879	0.7028

Appendix

Table 6: Hallucination Detection (AUROC) for probes built on visual features (VF), vision token (VT), and query token (QT) representations at different decoder layers using Gemma3-12B, FastVLM-7B, and LLaVA-Next-8B.

Model Name	Token	Vision	Decoder Layers
Model Name	Type	Encoder	Layer 0	Layer $L/4$	Layer $L/2$	Layer $3L/4$	Layer $L$
Molmo-V1-7B	VF	0.6830	-	-	-	-	-
	VT	-	0.6936	0.6982	0.6931	0.6862	0.6867
	QT	-	0.7675	0.8588	0.9365	0.9345	0.9193
Qwen2.5-VL-7B	VF	0.7873	-	-	-	-	-
	VT	-	0.6543	0.6593	0.6539	0.6625	0.6683

Appendix

Table 7: Hallucination Detection (AUROC) for probes built on visual features (VF), vision token (VT), and query token (QT) representations at different decoder layers using Molmo-V1-7B, Qwen2.5-VL-7B, Llama3.2-11B-Vision, and Phi4-VL.

Figure 7: AUROC scores for visual features detection.

Main Domain	Task Categories	Count
Attribute Recognition	$\rightarrow$ States (wet/dry, clean/messy)	3,014
	$\rightarrow$ Colors, numbers, counts
	$\rightarrow$ Physical properties, actions
Visual Understanding	$\rightarrow$ Object recognition	2,984
	$\rightarrow$ Existence questions
	$\rightarrow$ Visual challenges, illusions

Appendix

Table 8: Domain type distribution showing main categories and their specific task subcategories.

Main Category	Specific Error Types	Count
Object-Related (3,493)	$\rightarrow$ Object Hallucination	2,810
	$\rightarrow$ Object Existence/Count	683
	$\rightarrow$ Visual Challenge (Complex)	699
	$\rightarrow$ Scene Hallucination	250
	$\rightarrow$ Action Hallucination	123
Attribute-Related (1,622)	$\rightarrow$ Attribute Hallucination (State)	1,169

Appendix

Table 9: Hallucination type distribution with grouped main categories and specific error subcategories.

Answer Category	Definition	Count
Yes/No	$\rightarrow$ Binary answers: yes, no	6,568
	$\rightarrow$ Simple true/false responses
Open-Ended	$\rightarrow$ Descriptive text answers	2,008
	$\rightarrow$ Detailed explanations
Unanswerable	$\rightarrow$ "Can’t tell"/"Don’t know"	731
Number	$\rightarrow$ Numeric answers, counts	657

Appendix

Table 10: Answer type distribution showing response categories and their specific characteristics.

Metric	Rep	Qwen2.5	Gemma3	LLaVA	Molmo	Llama3.2	Phi4	FastVLM	SmolVLM
AUROC	VF	0.787	0.674	0.611	0.683	0.770	0.617	0.683	0.724
	VT	0.668	0.596	0.627	0.687	0.738	0.774	0.703	0.689
	QT	0.915	0.935	0.903	0.919	0.896	0.903	0.614	0.901
Threshold	VF	0.200	0.100	0.200	0.100	0.300	0.100	0.300	0.100
	VT	0.200	0.200	0.100	0.200	0.100	0.300	0.100	0.100
	QT	0.200	0.200	0.200	0.100	0.100	0.200	0.100	0.400

Appendix

Table 11: Complete early refusal analysis at optimal F1 threshold across all models and representation types. VT and QT use layer

L

Model	Best F1	Recall	Precision
	Threshold
Gemma3-12B	0.2	70.8%	57.2%
Molmo-7B	0.1	80.1%	54.2%
LLaVA-1.5-7B	0.2	68.8%	53.9%
SmolVLM-2.2B	0.4	69.2%	49.8%
Phi4-VL	0.2	66.8%	51.4%

Appendix

Table 12: Last Query Token (QT) early refusal performance at optimal F1 threshold.

Model	Best F1	Recall	Precision
	Threshold
Qwen2.5-VL-7B	0.2	38.0%	18.8%
Llama-3.2-11B	0.3	29.0%	40.6%
SmolVLM-2.2B	0.1	56.1%	19.8%
Molmo-7B	0.1	54.8%	20.8%
FastVLM	0.3	30.5%	25.2%

Appendix

Table 13: Vision Feature (VF) (before LLM decoder) early refusal performance at optimal F1 threshold.

Model	Best F1	Recall	Precision
	Threshold
Phi4-VL	0.3	47.9%	35.5%
Llama-3.2-11B	0.1	42.5%	21.1%
FastVLM	0.1	65.5%	14.7%
SmolVLM-2.2B	0.1	56.6%	20.6%
Molmo-7B	0.2	48.5%	22.2%

Appendix

Table 14: Last Vision Token (VT) early refusal performance at optimal F1 threshold (layer

L

Hallucination Detection in VLMs

Hallucination detection in vision-language models identifies claims that are unsupported by images, video, or other visual evidence.

See all 70 papers

UniProbe: A Learnable Token-Level Hallucination Detector for Large VLMs using Multi-Structural Internal Representations
Aug 11, 2026Dvir Samuel, Guy Bar-Shalom, Fabrizio Frasca +4Hallucination Detection in VLMs VLM Hallucination
NVIDIA Research, Tel Aviv, Israel · Technion, Haifa, Israel · Bar-Ilan University, Ramat Gan, Israel +1
Overthinking Causes Hallucination: Tracing Confounder Propagation in Vision Language Models
Mar 8, 2026Abin Shoby, Ta Duc Huy, Tuan Dung Nguyen +6Hallucination Detection in VLMs Object Hallucination in VLMs
Australian Institute for Machine Learning, University of Adelaide · Hanoi University of Science and Technology · Mohamed bin Zayed University of Artificial Intelligence
Role-Break in Attention Heads: Understanding and Detecting Hallucinations in VLMs
Jul 31, 2026Mingyu Wang, Weilin Jin, Wenbo Li +5Hallucination Detection in VLMs Attention Head Analysis
Peking University, Beijing, China · Joy Future Academy, Beijing, China

Aug 11, 2026J/K move · Enter open · S save

UniProbe: A Learnable Token-Level Hallucination Detector for Large VLMs using Multi-Structural Internal Representations

Dvir Samuel, Guy Bar-Shalom, Fabrizio Frasca, Ethan Fetaya+3

NVIDIA Research, Tel Aviv, Israel · Technion, Haifa, Israel · Bar-Ilan University, Ramat Gan, Israel+1

Hallucination Detection in VLMs VLM Hallucination Object Hallucination in VLMs VLM Hallucination Mitigation

Open paper AskPDF arXiv

Large Vision-Language Models (LVLMs) achieve impressive visual reasoning and dialogue capabilities, yet frequently hallucinate content unsupported by the visual input. Effective mitigation requires token-level localization, enabling targeted intervention without discarding the entire response. Existing detectors require expensive full-model fine-tuning, rely on external verifiers that ignore the model's generation process, or reduce internal signals to isolated features and hand-crafted statistics, discarding spatial, sequential, and relational structure. We introduce UniProbe, a lightweight, unified, learnable detector that models a frozen LVLM's heterogeneous computational trace from a single forward pass. UniProbe constructs a directed graph over image patches, query tokens, and generated tokens, with attention weights encoding their relations. It processes this trace with alternating structure-aware modules: a GNN for relational evidence, a ViT for 2-D visual geometry, and a GRU for response order. Interleaving them allows spatial, relational, and sequential evidence to interact throughout the detector. We further develop a streaming variant for hallucination-aware decoding, which detects and resamples hallucinated tokens during generation, and a self-adaptation strategy aligning the detector with the LVLM's own generations. Across diverse LVLM backbones, UniProbe achieves state-of-the-art token-level and object-hallucination detection. During decoding, it reduces object hallucinations by up to 55% at

1.06\times

the latency of standard generation.

Abstract

Hallucination Detection in VLMs

Explore similar work

UniProbe: A Learnable Token-Level Hallucination Detector for Large VLMs using Multi-Structural Internal Representations

Overthinking Causes Hallucination: Tracing Confounder Propagation in Vision Language Models

Role-Break in Attention Heads: Understanding and Detecting Hallucinations in VLMs