Unified Hallucination Fuzzing for Multimodal Large Language Models

Pengfei Zhou; Jiajun Song; Zhiwei Tang; Yixing Ma; Donghui Si; Yuhang Xu; Huiqi Song; Yiyuan Miao

cs.CLJul 15, 2026

Unified Hallucination Fuzzing for Multimodal Large Language Models

Authors: Pengfei Zhou, Jiajun Song, Zhiwei Tang, Yixing Ma, Donghui Si, Yuhang Xu, Huiqi Song, Yiyuan Miao,

Organizations: National University of Singapore · DAMO Academy, Alibaba Group · Renmin University of China · Zhejiang University ·

Abstract

Hallucination remains a persistent challenge for Multimodal Large Language Models (MLLMs), severely limiting their reliability in high-stakes applications. Existing evaluations, predominantly based on static benchmarks, suffer from narrow taxonomical coverage and rapid performance saturation, failing to reflect model robustness in evolving real-world scenarios. To bridge this gap, we present a systematic evaluation framework integrating a comprehensive benchmark with self-evolving stress testing. First, we introduce UniHall, a fine-grained dataset grounded in a unified taxonomy spanning Object, Instruction, and Knowledge dimensions. Second, to address benchmark saturation, we propose Self-Adaptive Multimodal Fuzzing (SAMF), a self-adaptive framework that employs evolutionary mutation strategies to explore the boundaries of model hallucinations. Crucially, to ensure reliable assessment of dynamic inputs, SAMF incorporates a structured metric suite driven by an ensemble of multi-modal oracles. Our extensive experiments reveal that state-of-the-art MLLMs exhibit significant performance degradation under fuzzing compared to conventional settings, exposing a dissociation between reasoning capabilities and factual grounding. Furthermore, we identify a helpfulness-hallucination trade-off, where reinforcement learning alignment inadvertently exacerbates sycophancy in instruction-following tasks. The framework, code and benchmark are available at https://github.com/LanceZPF/EvalHall.

Figures & tables

Figure 1: UniHall organizes multimodal hallucinations into Object, Instruction, and Knowledge dimensions, each with three subtypes for systematic evaluation.

Panel of Table 1 — Table 1: Comparison of UniHall with existing multimodal hallucination benchmarks. Task : Disc.:discriminative; Gen.:generative; G&D:generative + discriminative. Hallucination Taxonomy : object / instruction / knowledge. UniHall provides significantly more extensive coverage across hallucination types and includes structured oracles, self-adaptive fuzzing, and risk - tier annotation for robust evaluation.

Overall
Statistic	Number
Total instances (question-image pairs)	2,170
Question Formats
Yes-or-No (YON)	900
Open-ended VQA (free-form)	1,270
Avg # acceptable answers per instance ( $\|\mathcal{G}\|$ )	1.65

Table 2: Key Statistics of UniHall.

Figure 2: Architecture of SAMF. SAMF uses agent-driven fuzzing to generate adaptive multimodal test samples for evaluating the robustness of the model under test.

Panel of Figure 3 — Figure 3: Structured Hallucination Metric. The workflow combines rule-based verifiers with coarse-to-fine judgments. Extracted atomic claims are matched against V-STAR, C-STAR, and E-STAR, with an NLI [ 6 ] fallback for robust hallucination detection.

Panel of Table 3 — Table 3: GHR comparison under standard evaluation. We report General Hallucination Rate (GHR) $\downarrow$ across different MLLMs and hallucination categories. Lower values indicate better faithfulness.

Panel of Figure 6 — Figure 6: Fuzzing case study. Representative example of standard and fuzzing evaluation results for Gemini3-Flash in a knowledge-reference hallucination scenario. Given a mutated image and a perturbed text input, the model’s behavior can be significantly influenced.

Metric	Acc.	Rec.	F1	$\boldsymbol{\kappa}$	$\boldsymbol{r}$	$\boldsymbol{\rho}$
Original Analysis
Random	0.495	0.485	0.376	-0.012	-0.014	-0.014
GHR	0.741	0.905	0.683	0.477	0.521	0.521
BHR	0.644	0.896	0.608	0.338	0.356	0.369
SHR	0.624	0.833	0.578	0.287	0.344	0.344
GHS	0.802	0.746	0.732	0.553	0.574	0.543

Table 4: Metric–human alignment.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Models	General			Object				Instruction				Knowledge
Models	Overall	YON	VQA	Avg	Exist	Attr	Rel	Avg	Ctxt	Refuse	Syco	Avg	Detail	Fact	Ref
Qwen2.5-VL-7B	29.76	17.98	38.11	23.90	9.50	27.79	27.00	42.63	25.88	88.00	11.50	27.17	26.00	17.00	38.50
Qwen3-VL-8B	32.70	23.42	39.29	25.90	14.00	28.62	30.00	52.67	22.94	98.00	25.50	27.34	44.00	18.50	19.50
LLaVA-Next-Llama3-8B	42.93	21.20	58.35	37.36	24.00	41.60	38.00	53.51	39.41	100.00	19.00	42.17	46.50	8.00	72.00
LLaVA-v1.6-Mistral-7B	43.90	20.53	60.47	39.26	21.50	46.26	36.00	54.04	44.71	100.00	16.00	42.00	51.00	8.00	67.00
LLaVA-v1.6-Vicuna-7B	45.19	21.31	62.13	39.36	23.00	46.42	34.50	58.95	48.24	100.00	27.00	41.83	43.50	11.50	70.50

Appendix

Table 5: Performance comparison of different MLLMs using standard evaluation on General Hallucination Rate (GHR)

\downarrow

Models	General			Object				Instruction				Knowledge
Models	Overall	YON	VQA	Avg	Exist	Attr	Rel	Avg	Ctxt	Refuse	Syco	Avg	Detail	Fact	Ref
Qwen2.5-VL-7B	45.46	17.98	64.96	31.26	25.11	37.06	20.00	62.11	52.35	95.00	37.50	53.33	49.00	52.50	58.50
Qwen3-VL-8B	47.74	23.31	65.07	31.81	30.82	35.47	21.80	61.58	44.71	100.00	37.50	61.17	75.00	54.50	54.00
LLaVA-Next-Llama3-8B	48.21	21.09	67.45	34.41	25.35	42.18	20.10	64.56	64.12	100.00	29.50	55.69	56.00	45.00	66.08
LLaVA-v1.6-Mistral-7B	49.62	20.53	70.26	35.89	27.23	45.27	16.35	66.49	63.53	100.00	35.50	56.50	62.00	49.00	58.50
LLaVA-v1.6-Vicuna-7B	50.08	21.20	70.56	35.91	27.63	44.53	18.30	68.60	72.35	100.00	34.00	56.11	53.64	50.00	64.68

Appendix

Table 6: Performance comparison of different MLLMs using standard evaluation on Structured Hallucination Rate (SHR)

\downarrow

Models	General			Object				Instruction				Knowledge
Models	Overall	YON	VQA	Avg	Exist	Attr	Rel	Avg	Ctxt	Refuse	Syco	Avg	Detail	Fact	Ref
Qwen2.5-VL-7B	50.14	17.98	72.96	43.54	52.66	50.42	13.75	56.67	51.18	99.50	18.50	54.95	56.00	49.67	59.20
Qwen3-VL-8B	45.63	23.31	61.46	37.07	38.30	43.93	15.25	56.32	42.94	100.00	24.00	49.74	80.75	10.75	57.73
LLaVA-Next-Llama3-8B	49.79	21.09	70.15	41.42	33.58	53.41	13.25	58.68	60.59	100.00	15.75	55.29	61.50	40.75	63.62
LLaVA-v1.6-Mistral-7B	52.49	20.53	75.16	45.88	52.39	55.41	10.75	59.39	59.41	100.00	18.75	56.96	66.25	46.50	58.12
LLaVA-v1.6-Vicuna-7B	52.75	21.20	75.14	45.54	49.44	55.41	12.00	61.93	63.53	100.00	22.50	56.07	58.75	47.25	62.20

Appendix

Table 7: Performance comparison of different MLLMs using standard evaluation on Breakdown Hallucination Rate (BHR)

\downarrow

Models	General			Object				Instruction				Knowledge
Models	Overall	YON	VQA	Avg	Exist	Attr	Rel	Avg	Ctxt	Refuse	Syco	Avg	Detail	Fact	Ref
LLaVA-v1.6-34B	33.52	31.74	34.79	26.16	16.80	31.18	20.45	44.28	19.82	74.70	34.65	35.58	58.50	7.45	40.80
InternVL3.5-38B-Pretrained	27.13	17.98	33.62	25.59	15.70	27.02	31.20	36.37	15.59	72.75	17.65	20.92	26.30	8.75	27.70
InternVL3.5-38B-Cascade RL	26.39	18.76	31.80	22.78	18.20	24.01	23.65	37.25	13.82	72.80	21.60	22.10	29.70	11.60	25.00
Gemini-3-Flash	21.79	19.64	23.31	16.14	16.25	13.94	22.65	34.81	11.53	59.05	30.35	18.83	41.65	4.35	10.50
Gemini-3-Pro	22.24	20.87	23.21	18.45	18.10	17.05	23.00	31.00	12.41	54.50	23.30	20.23	45.25	7.25	8.20

Appendix

Table 8: Performance comparison of different MLLMs using standard evaluation on General Hallucination Scores (GHS)

\downarrow

Model	Accuracy	Recall	F1-Score	Kappa	Pearson	Spearman
LLaVA-v1.6-34B	0.819	0.723	0.769	0.621	0.625	0.625
InternVL3.5-38B-Pretrained	0.872	0.851	0.804	0.710	0.712	0.712
InternVL3.5-38B-Cascade RL	0.871	0.718	0.775	0.685	0.689	0.689
Gemini-3-Flash	0.912	0.802	0.783	0.728	0.728	0.728
Gemini-3-Pro	0.872	0.743	0.625	0.550	0.560	0.560

Appendix

Table 9: Inter-rater reliability analysis of different models across various human annotations.

Eval. Setting	Models	General			Object				Instruction				Knowledge
Eval. Setting	Models	Overall	YON	VQA	Avg	Exist	Attr	Rel	Avg	Ctxt	Refuse	Syco	Avg	Detail	Fact	Ref
Standard	Qwen2.5-VL-7B	29.76	15.38	36.80	22.54	3.00	30.23	19.00	41.40	24.71	85.00	12.00	24.67	30.00	7.00	37.00
	Qwen3-VL-8B	32.70	23.42	39.29	25.97	14.00	28.62	30.00	50.18	22.94	98.00	25.50	27.33	44.00	18.50	19.50
	GPT-5.2	27.68	21.53	32.05	22.57	14.50	23.29	28.50	45.61	27.06	85.50	21.50	19.17	37.50	7.50	12.50
	Gemini3-Flash	31.97	19.87	40.55	25.58	21.50	22.30	39.50	49.82	25.88	86.00	34.00	25.67	50.50	7.50	19.00
Dynamic	Qwen2.5-VL-7B	36.54	24.51	45.22	31.56	17.08	35.53	34.12	51.56	34.58	93.50	24.06	34.70	37.59	18.52	48.00

Appendix

Table 10: Performance comparison of different MLLMs under Standard, Dynamic, Heuristic Fuzz, and RL-based Fuzz evaluation settings on UniHall. All values represent the Generic Hallucination Rate (GHR)

\downarrow

Panel of Figure 8 — Figure 8: Case study: Object existence.

Panel of Figure 9 — Figure 9: Case study: Object attribute.

Panel of Figure 10 — Figure 10: Case study: Object relation.

Panel of Figure 11 — Figure 11: Case study: Instruction context.

Panel of Figure 12 — Figure 12: Case study: Failure-to-refuse.

Panel of Figure 13 — Figure 13: Case study: Sycophancy.

Panel of Figure 14 — Figure 14: Case study: Knowledge detail.

Panel of Figure 15 — Figure 15: Case study: Knowledge factual.

Panel of Figure 16 — Figure 16: Case study: Knowledge reference.

Figure 17: Representative samples from the Object, Instruction, and Knowledge evaluation categories. We demonstrate model performance on object existence queries (top-left), strict instruction following (bottom-left), and complex knowledge retrieval with citation requirements (right). The accompanying tables highlight the fine-grained hallucination scores for each response.

Hallucination in Language Models

Hallucination in language models studies how systems produce unsupported or false claims and how to detect, explain, and reduce them.

See all 266 papers

FREAK: A Fine-grained Hallucination Evaluation Benchmark for Advanced MLLMs
Mar 20, 2026Zhihan Yin, Jianxin Liang, Yueqian Wang +3VLM Hallucination VLM Evaluation
Wangxuan Institute of Computer Technology, Peking University · State Key Laboratory of General Artificial Intelligence
ReactBench: A Cause-Driven Benchmark for Multimodal Hallucination via Systematic Evaluation
May 28, 2026Shizhe Zhou, Bohan Jia, Kai Wu +4VLM Robustness Hallucination Detection
East China Normal University
Measuring Epistemic Humility in Multimodal Large Language Models
Sep 11, 2025Bingkui Tong, Jiaer Xia, Sifeng Shang +1VLM Hallucination Multimodal Large Language Models
Mohamed bin Zayed University of Artificial Intelligence, United Arab Emirates. · Hong Kong Baptist University, Hong Kong.

Mar 20, 2026J/K move · Enter open · S save

FREAK: A Fine-grained Hallucination Evaluation Benchmark for Advanced MLLMs

Zhihan Yin, Jianxin Liang, Yueqian Wang, Yifeng Yao+2

Wangxuan Institute of Computer Technology, Peking University · State Key Laboratory of General Artificial Intelligence

VLM Hallucination VLM Evaluation Counterfactual Evaluation of VLMs

Open paper AskPDF arXiv

Multimodal Large Language Models (MLLMs) suffer from hallucinations. Existing hallucination evaluation benchmarks are often limited by over-simplified tasks leading to saturated metrics, or insufficient diversity that fails to adequately assess the hallucination extent in state-of-the-art multimodal models. To address this gap, we propose FREAK, a comprehensive multimodal benchmark designed for fine-grained hallucination assessment in MLLMs. Through high-quality photorealistic images featuring fine-grained counter-commonsense edits, FREAK innovatively evaluates hallucination phenomena in detailed visual perception of MLLMs. Extensive experiments on FREAK show severe hallucination issues in SOTA models regarding detailed visual perception. To enable deeper investigation, we curate a controlled subset to indirectly evaluate the model's ability to perceive target detailed information. Through systematic evaluation of prevailing Chain-of-Thought (CoT) prompting techniques within this task, we reveal critical insights regarding hallucination patterns and model reasoning processes.

Abstract

Hallucination in Language Models

Explore similar work

FREAK: A Fine-grained Hallucination Evaluation Benchmark for Advanced MLLMs

ReactBench: A Cause-Driven Benchmark for Multimodal Hallucination via Systematic Evaluation

Measuring Epistemic Humility in Multimodal Large Language Models