What Hallucinations Have We Measured? Semantic Illusion, Confabulation, and Expressed Uncertainty

Hazel H. Kim; Jakub Binkowski; Ekaterina Fadeeva; Juan David Guerra; Yijie Hao; Zheng Yi Ho; Baixiang Huang; Buyun Liang

cs.CLOct 28, 2026

What Hallucinations Have We Measured? Semantic Illusion, Confabulation, and Expressed Uncertainty

Authors: Hazel H. Kim, Jakub Binkowski, Ekaterina Fadeeva, Juan David Guerra, Yijie Hao, Zheng Yi Ho, Baixiang Huang, Buyun Liang,

Organizations: University of Oxford · Wroclaw University of Science and Technology · ETH Zurich · Mila – Quebec AI Institute ·

Abstract

Hallucination remains one of the most critical challenges in deploying large language models (LLMs) reliably and safely. Yet the term is used inconsistently across the research community. We argue for a strict definition of hallucinations in LLMs. First, hallucinations should be confident incorrectness, kept separate from expressed uncertainty (UNK)1. UNK reveals that the model may be wrong, whereas a hallucination does not reveal it and is therefore more dangerous in practice. Conflating the two obscures their distinct behaviors and makes them hard to study. Second, hallucinations should be split by a simple re-elicitation test: does any semantically equivalent prompt recover the correct response? This test probes whether the relevant knowledge2 is accessible. If none does, the knowledge is likely absent or inaccessible, and the error type is a confabulation (CF) that likely requires external knowledge or retraining; if some does, the knowledge is accessible from the model but applied inconsistently, and the error type is a semantic illusion (SI) that simple prompt or decoding changes can mitigate3. Re-evaluating 14 recent hallucination methods under this taxonomy yields three findings. (1) Detection methods flag UNK more often than hallucinated responses (i.e., SI and CF). (2) Mitigation methods are far more effective for SI than for CF. (3) Latent signals are more informative than output-level signals for CF. An audit of recent hallucination papers shows that most conflate these categories. We recommend reporting CF, SI, and UNK separately, and introduce a practical labeling protocol that needs only a modest number of semantically equivalent prompts.

Figures & tables

Panel of Figure 1 — Figure 1: Proposed taxonomy of model responses. An incorrect answer is a semantic illusion if any semantically equivalent re-elicitation recovers the correct answer. If none does, it is a confabulation, unless the model expresses uncertainty under some condition, in which case it is unk . Only si and cf are hallucinations.

Figure 2: Recent text-based hallucination papers filtered by their definitions and approaches with uncertainty estimation, additional knowledge, and prompt variance. From a pool of all qualifying text-based papers ∗ across ICML, NeurIPS, ICLR, ACL, and EMNLP (2021–2025), most papers neither distinguish hallucination from uncertainty nor test prompt variance before considering external knowledge as a source. The methods that skip these distinctions risk targeting the wrong error type entirely. The detailed explanations are in § C .

Figure 3: Labels stabilize at about ten paraphrases , subsampling from a pool of 30. (a) Binary entropy of the per-question correctness rate. (b) cf

\to

si reassignment rate. (c) Cohen’s

\kappa

against the full-pool labels, and pairwise flip rate. Shading marks the recommended window. Phi-3, Phi-4 and Gemma 3 27B show that the curves generalize across families.

Figure 4: Label stability under decoding, and SI/CF is not an information-injection artifact. (a)&(b) Pairwise agreement (%) on a 30 question variants, aggregated over three instruction prompts. (c)

\rho_{30}

vs. the information quantity emerged from paraphrases

\Delta I=\log P(A\mid V)-\log P(A\mid Q)

Intervention stage (Finding 1)
Inference-time	ActLCD [ 110 ] , NoVo [ 33 ]
Post-hoc	SelfCheckGPT [ 62 ] , UQ Heads [ 86 ]
External knowledge	DRAG [ 14 ] , HEB [ 37 ] , CS [ 31 ]
Uncertainty representation (Findings 2–3)
Verbal	VUF [ 46 ] , UQ Heads [ 86 ]
Latent	UQ Heads [ 86 ] , SenD [ 64 ] , IRIS [ 88 ] , Attn. Sink [ 8 ]

Table 1: The 14 methods, grouped by the two dimensions used in the main text. Other groupings are analyzed in § F .

Scheme	cf	si	unk
S1	0.36	0.62	0.02
S2	0.04	0.64	0.32

Table 2: Label distribution among questions answered incorrectly under the original prompt (

|\mathcal{V}(Q)|=10

, pooled over models). Inviting abstention (S2) converts most cf into unk .

Figure 5: Summary of all methods at

|\mathcal{V}(Q)|=10

. Labeling schemes for S1 and S2 are in § 5.2 .

Figure 6: Gaps by intervention stage , as

|\mathcal{V}(Q)|

grows. Only external-knowledge methods (highlighted) mitigate cf well above zero (about 20%), while inference-time and post-hoc methods stay near zero.

	S1			S2
	cf	si	unk	cf	si	unk
Hallucination score (mean)	0.54 $\pm$ .21	0.43 $\pm$ .20	0.61 $\pm$ .25	0.42 $\pm$ .29	0.42 $\pm$ .21	0.57 $\pm$ .19
Miss rate (%)	34.2 $\pm$ 25.6	35.6 $\pm$ 24.7	22.9 $\pm$ 21.4	50.4 $\pm$ 33.5	35.9 $\pm$ 24.9	30.8 $\pm$ 23.0
Mitigation correct (%)	3.9 $\pm$ 7.3	29.5 $\pm$ 7.5	0.0 $\pm$ 0.0	2.2 $\pm$ 5.0	30.4 $\pm$ 8.6	0.4 $\pm$ 0.7

Table 3: Uncertainty-estimation detectors by label (7 methods). For unk , a high score is a failure: these responses are not hallucinations. Mean

\pm

std across methods.

Figure 7: Detection (top) and mitigation (bottom) by uncertainty representation as

|\mathcal{V}(Q)|

grows. Verbal signals detect unk best and cf worst; latent signals detect cf better than si .

n

is the number of methods per group; some methods belong to several groups.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 8: Dimensions that 24 first authors of hallucination papers consider most important for categorizing hallucination. Each bar is the fraction of respondents who selected that dimension; multiple selections were allowed, so percentages do not sum to 100%. Dark bars mark the dimensions underlying our CF/SI/UNK taxonomy.

Figure 9: Recent text-based hallucination papers filtered by their definitions and approaches with uncertainty estimation, additional knowledge, and prompt variance. From a pool of qualifying text-based papers ∗ across ICML, NeurIPS, ICLR, ACL, and EMNLP (2021–2025), most papers neither distinguish hallucination from uncertainty nor test prompt variance before considering external knowledge as a source. The methods that skip these distinctions risk targeting the wrong error type entirely. Table 4 lists the papers with their category at each stage, and Table 5 groups the paper IDs by category.

#	Paper	Definition	Uncertainty	New knowledge	Prompt variance
1	A Token-level Reference-free Hallucination Detection Benchmark [ 59 ]	Any factual	Proxy	Needs knowledge	Not tested
2	Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization [ 11 ]	Unfaithful	Ack./Distinct	Needs knowledge	Not tested
3	Detecting and Mitigating Hallucinations in Machine Translation: Model Internal Workings [ 21 ]	Unfaithful	Uniform	No new knowledge	Tested
4	Optimal Transport for Unsupervised Hallucination Detection in Neural Machine Translation [ 30 ]	Unfaithful	Uniform	No new knowledge	Tested
5	Self-Alignment for Factuality: Mitigating Hallucinations in LLMs via Self-Evaluation [ 115 ]	Confident	Ack./Distinct	Needs knowledge	Tested
6	ANAH: Analytical Annotation of Hallucinations in Large Language Models [ 44 ]	Any factual	Uniform	Needs knowledge	Not tested

Appendix

Table 4: All 91 text-modality papers in the survey. “Needs knowledge” includes “partially”.

Stage	Category (n)	Paper IDs
Definition	Confident error (32)	5, 7, 8, 20, 21, 23, 25, 30, 35, 39, 42, 50, 52, 54, 55, 56, 59, 61, 62, 66, 68, 69, 70, 71, 75, 76, 80, 81, 83, 85, 88, 91
	Any factual error (36)	1, 6, 10, 12, 13, 14, 15, 16, 22, 24, 26, 31, 33, 38, 41, 43, 44, 45, 46, 47, 48, 51, 53, 57, 58, 60, 63, 64, 65, 67, 72, 74, 78, 84, 86, 90
	Unfaithful output (19)	2, 3, 4, 9, 11, 17, 18, 19, 27, 28, 29, 32, 34, 36, 37, 40, 49, 82, 89
	Inconsistent output (4)	73, 77, 79, 87
Uncertainty	Acknowledged/Distinct (25)	2, 5, 7, 8, 11, 17, 22, 23, 30, 31, 39, 42, 48, 50, 52, 54, 61, 62, 67, 68, 75, 77, 78, 81, 90
	Used as proxy (11)	1, 15, 28, 35, 55, 59, 66, 80, 86, 88, 91

Appendix

Table 5: Paper IDs (as in Table 4 ) by category, for all 91 text-modality papers.

Panel of Figure 10 — Figure 10: Methods in RQ1–RQ2 space.

RQ	H	Angle	Category	Methods
RQ1	H1	Prompt Sensitivity	Modeled	SECA [ 57 ] , SelfCheckGPT [ 62 ]
			Not modeled	DRAG [ 14 ] , UQ Heads [ 86 ] , ActLCD [ 110 ]
RQ1	H1	Intervention Stage	Inference	ActLCD [ 110 ] , NoVo [ 33 ]
			Post-hoc	SelfCheckGPT [ 62 ] , UQ Heads [ 86 ]
			Ext. Know.	DRAG [ 14 ] , HEB [ 37 ] , CS(Beyond Fact) [ 31 ]
RQ1	H2	Knowledge Assump.	Absent	SelfCheckGPT [ 62 ] , IRIS [ 88 ] , VUF [ 46 ] , SCH [ 67 ] , EES(SenD) [ 64 ] , Attn. Sink [ 8 ]

Appendix

Table 6: Experimental dimensions for RQ1–RQ2. H1: prompt sensitivity, intervention stage, and target error type; H2: knowledge assumption; H3–H4: uncertainty usage and representation. Dimensions are explained in § F.1 .

Scheme	CF	SI	UNK
S1	0.36	0.62	0.02
S2	0.04	0.64	0.32

Appendix

Table 7: Label distribution

Figure 11: Model confidence scores across different response categories – correct/incorrect responses and not attempted – on the TriviaQA validation split (5k) and SimpleQA (4.3k). High confidence on incorrect responses and near-zero confidence on not-attempted responses show that the confidence score is consistent with explicit verbal uncertainty.

Figure 12: cf / si label behavior as

|\mathcal{V}(Q)|

increases, subsampled from a 30-paraphrase pool. (a) binary entropy of the correctness rate. (b) cf

\rightarrow

si reassignment rate. (c) Cohen’s

\kappa

against the full-pool label and pairwise subset flip rate. Shading marks the recommended window

|\mathcal{V}(Q)|

Figure 13: Label stability under decoding, and SI/CF is not an information-injection artifact. (a)&(b) Pairwise agreement (%) on a 30 question variants, aggregated over three instruction prompts. (c)

\rho_{30}

vs. the information quantity emerged from paraphrases

\Delta I=\log P(A\mid V)-\log P(A\mid Q)

Figure 14: si vs. cf across labeling schemes at

|\mathcal{V}(Q)|=10

. Each marker is the mean over methods in that group. Mitigation correctness, †

P(c{=}1\mid h{=}1,\ell)

, is for mitigation methods only.

Panel of Figure 15 — Figure 15: Comparison of existing methods.

Detection

Mitigation

Metric

UNK

Miss rate (%)

\downarrow

34.2

\pm

25.6

35.6

\pm

24.7

22.9

\pm

21.4

50.4

\pm

33.5

35.9

\pm

24.9

30.8

\pm

23.0

–

27.9

\pm

37.6

–

41.1

\pm

44.7

28.1

\pm

37.7

29.6

\pm

38.1

Hallucination score

\uparrow

0.54

\pm

0.21

0.43

\pm

0.20

0.61

\pm

0.25

0.42

\pm

0.29

0.42

\pm

0.21

0.57

\pm

0.19

–

Correctness (%)

\uparrow

–

3.9

\pm

7.3

29.5

\pm

7.5

0.0

\pm

0.0

2.2

\pm

5.0

30.4

\pm

8.6

0.4

\pm

0.7

Appendix

Table 8: Uncertainty estimation-based detection and mitigation performance (mean

\pm

std).

Panel of Figure 16 — Figure 16: Comparison of subset of existing methods that adopt uncertainty estimation.

Hallucination in Language Models

Hallucination in language models studies how systems produce unsupported or false claims and how to detect, explain, and reduce them.

See all 256 papers

May 26, 2026

How Do Uncertainty Estimators Align with Hallucination Occurrence in LLMs?

Large language models (LLMs) can hallucinate, but do model uncertainty estimators reliably reveal when they do? Although uncertainty estimators are increasingly used to identify unreliable generations, their relationship with hallucination remains poorly understood and is often taken for granted. We systematically evaluate 46 uncertainty estimators across complementary hallucination settings, covering both intrinsic hallucinations, where outputs are unsupported by the provided context, and extrinsic hallucinations, where claims are unsupported by expected model knowledge. Across four benchmarks and multiple LLMs, we find that the association between uncertainty and hallucination varies widely and is often weak, depending on the hallucination setting and the model under evaluation. These results challenge the use of uncertainty as a direct signal of hallucination and clarify when it provides useful information.

Yedidia Agnimo, Anna Korba, Annabelle Blangero +2

Ekimetrics France · Centre Inria de l’Université Grenoble Alpes France · CREST, ENSAE Institut Polytechnique de Paris France

LLM Evaluation Hallucination in Language Models LLM Reliability LLM Hallucination Detection LLM Uncertainty Estimation

Ask PDF

Sep 17, 2025

Geometric Uncertainty for Detecting and Correcting Hallucinations in LLMs

Large language models are known to hallucinate, generating linguistically plausible but incorrect answers to questions. Uncertainty quantification has been proposed as a strategy to detect such behaviour, but existing methods lack a unified framework to assess reliability at both the prompt and answer level. We introduce a geometric framework which quantifies language model uncertainty at both levels by explicitly modelling a prompt-conditioned semantic distribution in answer embedding space. Our approach is black-box and sampling-based; we generate multiple answers per prompt, and use archetypal analysis to estimate a geometric support for the answer distribution. At the prompt level, we approximate the distribution entropy to quantify uncertainty; for each individual answer, we then use notions of atypicality to assess its reliability relative to the batch. We employ our framework to not only detect hallucinations but correct them, by selecting the batch example deemed most reliable. Experiments show that our framework performs comparably to or better than prior methods on short form question-answering datasets, and achieves superior results on medical datasets where hallucinations carry particularly critical risks. Beyond pure performance, we suggest the theoretical grounding of our work provides support for semantic distributions as useful objects of study for language model uncertainty.

Edward Phillips, Sean Wu, Soheila Molaei +3

Department of Engineering Science, University of Oxford · GlaxoSmithKline · Oxford Suzhou Centre for Advanced Research

Uncertainty Quantification LLM Hallucination Mitigation Hallucination in Language Models LLM Reliability LLM Hallucination Detection LLM Uncertainty Estimation+1

Ask PDF

Sep 14, 2026

When the Wrong Key Wins: Understanding and Detecting Hallucinations in LLMs

Large language models can hallucinate even when the knowledge required for a correct answer is already available. We study this failure through a latent-key view of inference, where answer selection depends on competition among associations acquired during pretraining. We show that model predictions can be highly sensitive to individual query keywords, that these influential keywords exhibit entity-specific binding, and that their effects are systematically shaped by pretraining frequency. Multiple bindings can also compete and exhibit higher-order interactions within the same query. Based on this mechanism, we introduce a two-stage keyword-perturbation method for hallucination detection. By removing influential keywords and measuring how the model reorganizes its prediction, the method distinguishes errors caused by misleading key associations from correct decisions supported by diagnostic evidence. Across multiple models and benchmarks, perturbation provides a strong and transferable detection signal, reaching

0.910

AUROC on probe-known ScientistQA. Finally, we extend the same probabilistic framework to four hallucination regimes: knowledge deficit, wrong knowledge, context distraction, and unstable inference. Their operational distributions across benchmarks provide diagnostic context for why different detector families succeed in different settings.

Xuhan Tong, Haoyue Bai, Dawei Zhou +2

University of Wisconsin–Madison

Language Model Error Detection Hallucination in Language Models LLM Reliability LLM Hallucination Detection Language Model Probing

Ask PDF

Abstract

Hallucination in Language Models

Explore similar work

How Do Uncertainty Estimators Align with Hallucination Occurrence in LLMs?

Geometric Uncertainty for Detecting and Correcting Hallucinations in LLMs

When the Wrong Key Wins: Understanding and Detecting Hallucinations in LLMs