Steering the Verifiability of Multimodal AI Hallucinations

Jianhong Pang; Ruoxi Cheng; Ziyi Ye; Xingjun Ma; Zuxuan Wu; Xuanjing Huang; Yu-Gang Jiang

cs.AIApr 8, 2026

Steering the Verifiability of Multimodal AI Hallucinations

Authors: Jianhong Pang, Ruoxi Cheng, Ziyi Ye, Xingjun Ma, Zuxuan Wu, Xuanjing Huang, Yu-Gang Jiang

Organizations: Institute of Trustworthy Embodied AI, Fudan University · Shanghai Key Laboratory of Multimodal Embodied AI

Abstract

AI applications driven by multimodal large language models (MLLMs) are prone to hallucinations and pose considerable risks to human users. Crucially, such hallucinations are not equally problematic: some hallucination contents could be detected by human users(i.e., obvious hallucinations), while others are often missed or require more verification effort(i.e., elusive hallucinations). This indicates that multimodal AI hallucinations vary significantly in their verifiability. Yet, little research has explored how to control this property for AI applications with diverse security and usability demands. To address this gap, we construct a dataset from 4,470 human responses to AI-generated hallucinations and categorize these hallucinations into obvious and elusive types based on their verifiability by human users. Further, we propose an activation-space intervention method that learns separate probes for obvious and elusive hallucinations. We reveal that obvious and elusive hallucinations elicit different intervention probes, allowing for fine-grained control over the model's verifiability. Empirical results demonstrate the efficacy of this approach and show that targeted interventions yield superior performance in regulating corresponding verifiability. Moreover, simply mixing these interventions enables flexible control over the verifiability required for different scenarios.

Figures & tables

Figure 1: Illustration of verifiability of hallucinations. The same image–text pair can yield hallucinations that are obvious (easy to verify) or elusive (difficult to verify).

Figure 2: Overview of our data construction pipeline. (a) constructs questionnaires with candidate image-text pairs; (b) collects human judgments; (c) performs hallucination attribution to build a human-labeled dataset.

Figure 3: Joint visualization of hallucination-rate reduction under OHI and EHI across validation and test sets. Each point corresponds to a single-direction coefficient sweep, revealing a consistent spatial separation between the two interventions across three MLLMs.

Model	Type	OHS		EHS
Model	Type	ACC(%)	UT(%)	ACC(%)	UT(%)
Qwen2.5-VL-3B	OHI	+5.79	+26.28	+4.73	+20.3
Qwen2.5-VL-3B	EHI	+2.52	+17.10	+3.15	+13.50
Qwen2.5-VL-7B	OHI	+9.14	-1.08	+5.22	-0.47
Qwen2.5-VL-7B	EHI	+16.99	+0.43	+14.22	+0.85
LLaVA-OneVision-1.5-8B	OHI	+32.75	+2.51	+24.82	+1.76

Table 1: Effects of OHI and EHI on ACC(%) and UT(%) across different test subsets under the selected intervention settings. Values denote absolute changes relative to the baseline.

Model	OHS			EHS
Model	OHI	EHI	$\Delta$	OHI	EHI	$\Delta$
Qwen2.5-VL-3B	-32.07	-19.56	+12.51	-25.04	-16.65	+8.39
Qwen2.5-VL-7B	-8.06	-17.42	-9.36	-4.76	-15.08	-10.32
LLaVA-OneVision-1.5-8B	-35.26	-17.53	+17.73	-26.58	-19.43	+7.15

Table 2: Effects of OHI and EHI on HR(%) across different test subsets under the selected intervention settings. We also report

\Delta

, defined as the difference between the two subsets (

\Delta=\mathrm{EHI}-\mathrm{OHI}

Panel of Table 3 — Table 3: Influence on general ability under the selected intervention coefficients. We report accuracy (%) on general understanding benchmarks. Color intensity reflects the change compare to base model. (Green for improvements, Red for declines).

Figure 4: Layer-wise cosine similarity between obvious and elusive hallucination directions.

Figure 5: Layer comparison of OHI and EHI in Qwen2.5-VL-7B and LLaVA-OneVision-1.5-8B. The figure shows the average hallucination rate over post-instruction token positions.

Figure 6: Verifiability steering via the mixed direction on Qwen2.5-VL-7B and LLaVA-OneVision-1.5-8B. We fix the ablation strength $\alpha=1$ and vary the mixing coefficient $\lambda$ .

Figure 7: Case studies of intervention effects on LLaVA-OneVision-1.5-8B. Hallucinated content is highlighted in red, and corrected or image-grounded content is highlighted in green. The mixed-direction intervention is performed with $\lambda=0.5$ .

Model	Type	$i$	$l/L$	$\mathbf{\mathrm{hr\_h\_score}}$	$\mathbf{\mathrm{\Delta acc\_nh}}$	$\mathbf{\mathrm{kl\_score}}$
Qwen2.5-VL-3B	OHI	-1	29/36	-1.01	-0.32	0.08
Qwen2.5-VL-3B	EHI	-2	25/36	-0.27	-0.08	0.04
Qwen2.5-VL-7B	OHI	-4	20/28	-0.54	-0.06	0.04
Qwen2.5-VL-7B	EHI	-5	20/28	-0.49	0.03	0.04
LLaVA-OneVision-1.5-8B	OHI	-2	26/36	-1.23	0.02	0.09
LLaVA-OneVision-1.5-8B	EHI	-2	23/36	-0.64	0.07	0.06

Table 4: Direction selection details for each model. Note that

i=-1

indicates that the direction is selected from the last token position,

i=-2

the second-to-last token position, and so on. Also note that the layer index

l

starts from index 0, while

L

indicates the total number of layers.

Model	Type	Obvious Hallucination Subset			Elusive Hallucination Subset			Non-Hallucination Subset
Model	Type	HR(%)	ACC(%)	UT(%)	HR(%)	ACC(%)	UT(%)	HR(%)	ACC(%)	UT(%)
Qwen2.5-VL-3B	OHI	-32.07	+5.79	+26.28	-25.04	+4.73	+20.30	-3.26	-0.64	+3.90
Qwen2.5-VL-3B	EHI	-19.56	+2.52	+17.10	-16.65	+3.15	+13.50	-1.26	-3.66	+4.92
Qwen2.5-VL-7B	OHI	-8.06	+9.14	-1.08	-4.76	+5.22	-0.47	-3.32	+5.74	-2.43
Qwen2.5-VL-7B	EHI	-17.42	+16.99	+0.43	-15.08	+14.22	+0.85	+2.72	-1.78	-0.94
LLaVA-OneVision-1.5-8B	OHI	-35.26	+32.75	+2.51	-26.58	+24.82	+1.76	-1.78	+1.39	+0.39

Table 5: Full intervention results across all test sets. For each metric, we report the absolute change in hallucination rate (HR), accuracy (ACC), and uncertain tendency (UT) under the OHI and EHI.

Model	$\alpha_{\text{oh}}$	$\alpha_{\text{eh}}$
Qwen2.5-VL-3B	0.90	0.90
Qwen2.5-VL-7B	0.80	0.70
LLaVA-OneVision-1.5-8B	0.80	0.80

Table 6: Selected intervention coefficients for OHI and EHI.

Hallucination in Language Models

Hallucination in language models studies how systems produce unsupported or false claims and how to detect, explain, and reduce them.

See all 254 papers

OmniHallu: Unified Hallucination Detection for Cross-Modal Comprehension and Generation in Multimodal Large Language Models
Sep 10, 2026Jianjiang Yang, Peihang Li, Shanqing Xu +3Hallucination Detection in VLMs Hallucination Detection
The University of Manchester · The University of Hong Kong · Huazhong University of Science and Technology +2
HalluScope: Fine-grained Hallucination Diagnosis for Multimodal Large Language Models
Jul 23, 2026Weilin Jin, Mingyu Wang, Wenbo Li +5Hallucination Detection in VLMs Multimodal Hallucination
Peking University Beijing, China · Joy Future Academy Beijing, China
Measuring Epistemic Humility in Multimodal Large Language Models
Sep 11, 2025Bingkui Tong, Jiaer Xia, Sifeng Shang +1VLM Hallucination Multimodal Large Language Models
Mohamed bin Zayed University of Artificial Intelligence, United Arab Emirates. · Hong Kong Baptist University, Hong Kong.

Sep 10, 2026J/K move · Enter open · S save

OmniHallu: Unified Hallucination Detection for Cross-Modal Comprehension and Generation in Multimodal Large Language Models

Jianjiang Yang, Peihang Li, Shanqing Xu, Mengchen Qian+2

The University of Manchester · The University of Hong Kong · Huazhong University of Science and Technology+2

Hallucination Detection in VLMs Hallucination Detection Multimodal Large Language Models Multimodal Hallucination Multimodal Model Evaluation

Open paper AskPDF arXiv

While Multimodal Large Language Models (MLLMs) have achieved remarkable progress across diverse tasks, they suffer from hallucinations where generated outputs contradict or misrepresent input semantics. Existing research typically addresses hallucination detection within a single modality or task type, limiting generalizability. We introduce OmniHallu, a unified hallucination detection framework spanning both comprehension and generation tasks across image, video, and audio modalities. We contribute OmniHallu-Bench, a 10,000-sample benchmark with claim-level human annotations covering six cross-modal tasks: image-to-text (I2T), video-to-text (V2T), audio-to-text (A2T), text-to-image (T2I), text-to-video (T2V), and text-to-audio (T2A). Our multi-agent architecture decomposes model outputs into atomic claims, verifies them through modality-specific experts, and aggregates evidence via structured reasoning. We further propose a preference-optimized trainable verifier that approximates the multi-agent decision boundary, reducing expert calls by 66% with minimal performance loss. Extensive experiments reveal a consistent modality-dependent performance gradient and provide fine-grained insights into cross-modal hallucination patterns.

Abstract

Hallucination in Language Models

Explore similar work

OmniHallu: Unified Hallucination Detection for Cross-Modal Comprehension and Generation in Multimodal Large Language Models

HalluScope: Fine-grained Hallucination Diagnosis for Multimodal Large Language Models

Measuring Epistemic Humility in Multimodal Large Language Models