Lost in Diffusion: Uncovering Hallucination Patterns and Failure Modes in Diffusion Large Language Models

Zhengnan Guo; Fei Tan

cs.CLApr 12, 2026

Lost in Diffusion: Uncovering Hallucination Patterns and Failure Modes in Diffusion Large Language Models

Authors: Zhengnan Guo, Fei Tan

Organizations: East China Normal University · Zhejiang University of Technology

Abstract

While Diffusion Large Language Models (dLLMs) have emerged as a promising non-autoregressive paradigm comparable to autoregressive (AR) models, their faithfulness, specifically regarding hallucination, remains largely underexplored. To bridge this gap, we present the first controlled comparative study to evaluate hallucination patterns in dLLMs. Our results demonstrate that current dLLMs exhibit a higher propensity for hallucination than AR counterparts controlled for architecture, scale, and pre-training weights. Furthermore, an analysis of inference-time compute reveals divergent dynamics: while quasi-autoregressive generation suffers from early saturation, non-sequential decoding unlocks potential for continuous refinement. Finally, we identify distinct failure modes unique to the diffusion process, including premature termination, incomplete denoising, and context intrusion. Our findings underscore that although dLLMs have narrowed the performance gap on general tasks, their distinct hallucination mechanisms pose a critical challenge to model reliability. Our code is available at https://github.com/ZeroLoss-Lab/Lost-in-Diffusion

Figures & tables

Model	Precise WikiQA			LongWiki			NonexistentRefusal
Model	FRR $\downarrow$	HR $\downarrow$	CR $\uparrow$	Prec. $\uparrow$	Rec.@32 $\uparrow$	F1@32 $\uparrow$	FA $\downarrow$
LLaMA-3-8B	28.72	85.94	10.30	0.408	0.293	0.306	73.35
LLaDA-8B	21.40	95.13	3.92	0.271	0.306	0.272	87.10
Qwen2.5-7B	18.12	89.06	9.06	0.376	0.441	0.387	94.05
Dream-7B	26.96	92.54	6.04	0.345	0.400	0.340	98.50

Table 1: Performance comparison across three extrinsic hallucination tasks. Metrics include False Refusal Rate (FRR), Hallucination Rate (HR), and Correct Rate (CR) for PreciseWikiQA ; Precision, Recall@32, and F1@32 for LongWiki ; and False Acceptance Rate (FA) for NonexistentRefusal . Bold indicates the better performance within each control group.

Figure 1: F1@32 trends across exponentially increasing denoising steps. While LLaDA saturates early, Dream benefits from increased inference compute.

Model	PT	ID	CI
LLaDA-8B	18.0	60.0	38.0
Dream-7B	13.0	44.0	58.0

Table 2: Frequency (%) of human-annotated failure modes on broken outputs from diffusion language models. Percentages do not sum to 100 because some examples exhibit multiple failure modes.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Task	Sample size	Metric validated	Human–Auto Agreement	$\kappa$
PreciseWikiQA	100	Correct / Incorrect / Refusal	88.0	0.74
LongWiki	300 claims	Supported / Not Supported	84.3	0.68
NonExistentRefusal	100	Accept / Refuse	91.0	0.81
Overall	—	—	87.8	0.74

Appendix

Table 3: Human validation of automatic judgments across the three evaluation tasks.

Model	Precise WikiQA			LongWiki				Nonexistent Refusal
Model	FRR $\downarrow$	HR $\downarrow$	CR $\uparrow$	Prec. $\uparrow$	Rec.@32 $\uparrow$	F1@32 $\uparrow$	# Claims	FA $\downarrow$
LLaMA-3-8B-Instruct	10.88	81.15	16.80	0.368	0.550	0.430	48	96.30
LLaDA-8B-Instruct	11.56	94.80	4.66	0.352	0.264	0.218	10	87.10
Qwen2.5-7B-Instruct	18.80	86.92	10.62	0.391	0.620	0.469	54	79.55
Dream-7B-Instruct	11.26	93.06	6.78	0.950	0.066	0.122	2	47.95
Fast-dLLM v2	2.30	91.19	8.60	0.372	0.487	0.408	43	78.60

Appendix

Table 4: Performance comparison across three extrinsic hallucination tasks. Metrics include False Refusal Rate (FRR), Hallucination Rate (HR), and Correct Rate (CR) for PreciseWikiQA; Precision, Recall@32, and F1@32 for LongWiki; and False Acceptance Rate (FA) for NonExistentRefusal. # Claims refers to the median number of extracted claims in LLM-generated responses. Bold indicates the best performance within each control group.

Model	Task	Representative Example
Dream-7B-Instruct	LongWiki	Query : Describe the architectural design of Acton Town station, detailing the elements introduced by Charles Holden and the specific reasons for its Grade II listing. Output : Acton Town station, designed by Charles Holden, [EOS] Extracted Claims: 1. Acton Town station was designed by Charles Holden. Labels: 1. Supported
		Query : Describe the characteristics and nesting behaviors of solitary bees. Output : Solitary bees are solitary and nest in burrows or the ground. Extracted Claims 1. Solitary bees are solitary. 2. Solitary bees nest in burrows. 3. Solitary bees nest in the ground. Labels 1. Supported 2. Supported 3. Supported
	NonExistentRefusal	Query : Could you provide an overview of the medicine lcz-xp pulmicaps? Output : I’m sorry, but I can’t provide information on medications.
		Query : What can you tell me about the medicine atrich-l liquid? Output : I’m sorry, but I can’t provide information on specific medications.
		Query : Can you share some details about the medicine amleto diclofenac+metaxalone pack? Output : I’m sorry, but I can’t assist with that request.
Ling-mini-2.0	NonExistentRefusal	Query : Can you describe the medicine canoliv aquanase? Output : It appears there might be a typo in the name. The product you’re referring to is likely …

Appendix

Table 5: Representative failure modes and response patterns of instruction-tuned dLLMs. Dream-7B-Instruct shows abnormal truncation in long-form generation and over-refusal in nonexistent entity tasks. Ling-mini-2.0 lacks of a unified refusal strategy, while LLaDA2.0-mini exhibits a consistent, rigid refusal template.

Pattern	Representative Example
Premature Termination	Query: What can you tell me about the medicine atrich-l liquid? Output: Atrich-l liquid is a to treat otototototototitis [EOS]…
	Query: I’d like to learn more about the medicine kaziclox hbr douche. Output: Sure, I can help you with that. [EOS]…
	Query: What can you tell me about the medicine amlojack organogel? Output: Amlojack organogel is a topical cream that is treatend_idend_id [EOS]…
Incomplete Denoising	Query: Explain the development process and key contributors to Jurassic World Evolution. Output: The development of Jurassic World Evolution began in 2016… In the case of the Tyrannos, the 0.0.0.0.0.0.0.0.0. …
	Query: What was the designed crew size for the U-107 class boats? Output: The U-107 class boats had a crew of ____.
	Query: Explain the counterarguments raised by Maddy and Sober against confirmational holism within the context of the indispensability argument. Output: erserserserserserserserserserserserserserserserserserser…

Appendix

Table 6: Qualitative examples of the three distinct failure modes identified in dLLMs. Premature Termination shows abrupt endings with broken syntax; Incomplete Denoising exhibits repetitive or placeholder-like artifacts; Context Intrusion demonstrates hallucinated pivots to unrelated topics (e.g., coding or math).

LLaDA-8B (Quasi-Autoregressive)	Dream-7B (Non-Sequential)
Step 16
The Teton Range and Jackson Hole are two geological features located in the of the	The Teton Range and Jackson Hole are interconnected geological features in the [MASK] … [MASK] Park
Behavior: The model generates Left-to-Right. The end of the sentence is cut off abruptly.	Behavior: Non-sequential anchoring. The model resolves the future token “Park” while the middle remains masked.
Step 64
The Teton Range and Jackson Hole are two geological features located in the Rocky Mountains of the United States. They are connected by the Teton Range fault, which is a geological fault that runs along the eastern edge of the Teton Range. The fault is believed to have formed during the period, million years ago,	The Teton Range and Jackson Hole are interconnected geological features in the [MASK] … [MASK] Park [MASK] … [MASK] canyons, [MASK] … [MASK] Tetons [MASK] … [MASK] Grand Tet
Behavior: Linear extension. The prefix becomes longer and coherent, but the future context is completely noise.	Behavior: High-confidence entities (canyons, Tetons) appear discontinuously.

Appendix

Table 7: Comparison of Intermediate Decoding Dynamics. We list the generation states of LLaDA and Dream at steps

t=16,64,256

(

T=1024

). LLaDA exhibits a quasi-autoregressive pattern, resolving tokens linearly. Dream demonstrates stochastic flexibility, utilizing Minimum Entropy Decoding to resolve high-confidence tokens non-sequentially, leaving low-confidence regions as noise (represented by [MASK] ).

Hallucination in Language Models

Hallucination in language models studies how systems produce unsupported or false claims and how to detect, explain, and reduce them.

See all 254 papers

DynHD: Hallucination Detection for Diffusion Large Language Models via Denoising Dynamics Deviation Learning
Mar 17, 2026Yanyu Qian, Yue Tan, Yixin Liu +2Hallucination Detection Diffusion Language Models
Nanyang Technological University, Singapore · Griffith University, Australia
Reliability Challenges in Diffusion Vision-Language Models
Sep 1, 2026Md. Atabuzzaman, Chris ThomasVLM Evaluation VLM Robustness
Department of Computer Science Virginia Tech
Uncertainty Quantification for Large Language Diffusion Models
May 14, 2026Artem Vazhentsev, Vladislav Smirnov, David Li +3Uncertainty Quantification LLM Hallucination Detection
1MBZUAI · 2The University of Melbourne

Mar 17, 2026J/K move · Enter open · S save

DynHD: Hallucination Detection for Diffusion Large Language Models via Denoising Dynamics Deviation Learning

Yanyu Qian, Yue Tan, Yixin Liu, Wang Yu+1

Nanyang Technological University, Singapore · Griffith University, Australia

Hallucination Detection Diffusion Language Models LLM Uncertainty Estimation

Open paper AskPDF arXiv

Diffusion large language models (D-LLMs) have emerged as a promising alternative to auto-regressive models due to their iterative refinement capabilities. However, hallucinations remain a critical issue that hinders their reliability. To detect hallucination responses from model outputs, token-level uncertainty, such as entropy, has been widely used to indicate potential factual errors. Nevertheless, unlike auto-regressive models that generate tokens sequentially, D-LLMs generate fixed-length sequences simultaneously, where only a small subset of tokens is informative for hallucination detection. Thus, aggregating uncertainty over all tokens can be suboptimal. Moreover, the evolution trend of uncertainty throughout the diffusion process can also provide valuable signals, highlighting the necessity of modeling its denoising dynamics for hallucination detection. In this paper, we propose DynHD, which bridges these gaps from both spatial (token sequence) and temporal (denoising dynamics) perspectives. To handle the information density imbalance across tokens, we propose a semantic-aware evidence construction module that extracts hallucination-indicative signals by removing task-invariant structural tokens and emphasizing the uncertainty of the remaining informative tokens. To model denoising dynamics for hallucination detection, we introduce a reference evidence generator that learns the expected evolution trajectory of uncertainty evidence, along with a deviation-based hallucination detector that makes predictions by measuring the discrepancy between the observed and reference trajectories. Extensive experiments demonstrate that DynHD consistently outperforms state-of-the-art baselines while achieving higher efficiency across multiple benchmarks and backbone models. The code is available at: https://github.com/qyy11-com/DynHD.

Abstract

Hallucination in Language Models

Explore similar work

DynHD: Hallucination Detection for Diffusion Large Language Models via Denoising Dynamics Deviation Learning

Reliability Challenges in Diffusion Vision-Language Models

Uncertainty Quantification for Large Language Diffusion Models