Sample Transform Cost-Based Training-Free Hallucination Detector for Large Language Models

Zeyang Ding; Xinglin Hu; Jicong Fan

cs.LGMar 17, 2026

Sample Transform Cost-Based Training-Free Hallucination Detector for Large Language Models

Authors: Zeyang Ding, Xinglin Hu, Jicong Fan

Organizations: The Chinese University of Hong Kong, Shenzhen

Abstract

Hallucinations in large language models (LLMs) remain a central obstacle to trustworthy deployment, motivating detectors that are accurate, lightweight, and broadly applicable. Since an LLM with a prompt defines a conditional distribution, we argue that the complexity of the distribution is an indicator of hallucination. However, the density of the distribution is unknown and the samples (i.e., responses generated for the prompt) are discrete distributions, which leads to a significant challenge in quantifying the complexity of the distribution. We propose to compute the optimal-transport distances between the sets of token embeddings of pairwise samples, which yields a Wasserstein distance matrix measuring the costs of transforming between the samples. This Wasserstein distance matrix provides a means to quantify the complexity of the distribution defined by the LLM with the prompt. Based on the Wasserstein distance matrix, we derive two complementary signals: AvgWD, measuring the average cost, and EigenWD, measuring the cost complexity. This leads to a training-free detector for hallucinations in LLMs. We further extend the framework to black-box LLMs via teacher forcing with an accessible teacher model. Experiments show that AvgWD and EigenWD are competitive with strong uncertainty baselines and provide complementary behavior across models and datasets, highlighting distribution complexity as an effective signal for LLM truthfulness.

Figures & tables

Figure 1: Overview of distribution-consistency detection. For a given prompt, we sample multiple responses, extract generated-token hidden states, compute pairwise Wasserstein distances between responses, and summarize the resulting structure using AvgWD (average pairwise transform cost) and EigenWD (spectral complexity of the induced cost matrix).

Panel of Figure 2 — Figure 2: Heatmaps of sample-to-sample OT costs on CoQA. For a prompt, $D_{ij}=W_{2}(\mu_{i},\mu_{j})$ is the Wasserstein distance between the $i$ -th and $j$ -th sampled responses (diagonal is zero); darker cells indicate larger transform costs. Hallucinated cases often exhibit larger average costs and/or more fragmented block structure, motivating AvgWD (magnitude) and EigenWD (structure).

Panel of Figure 3 — Figure 3: Heatmaps of sample-to-sample OT costs on SQuAD. For a prompt, $D_{ij}=W_{2}(\mu_{i},\mu_{j})$ is the Wasserstein distance between the $i$ -th and $j$ -th sampled responses (diagonal is zero); darker cells indicate larger transform costs. Hallucinated cases often exhibit larger average costs and/or more fragmented block structure, motivating AvgWD (magnitude) and EigenWD (structure).

Figure 4: Cost-graph visualization from sample transform costs. Each node is a sampled response for a prompt; node positions come from a 2D embedding of the pairwise Wasserstein matrix

D

(precomputed distances), and edges connect nearest-neighbor pairs under

D

(solid: hallucinated cases; dashed: non-hallucinated). Marker shapes (A1–A3) denote different representative prompts within each group.

Model	Dataset	AvgWD	EigenWD	ER	ES	DSE	LNE	LS
Llama-3.2-3b	CoQA	0.811	0.816	0.755	0.749	0.781	0.602	0.773
	SQuAD	0.742	0.753	0.742	0.692	0.713	0.615	0.741
	MATH-500	0.715	0.728	0.683	0.702	0.693	0.606	0.688
	CNN/DailyMail	0.658	0.674	0.652	0.633	0.628	0.629	0.656
	Average	0.732	0.743	0.708	0.694	0.704	0.613	0.715
Llama-2-7b	CoQA	0.815	0.812	0.794	0.799	0.805	0.760	0.802

Table 1: White-box hallucination detection (AUROC; higher is better. Best and second-best per row are marked by bold and underline .). We report both AvgWD (cost magnitude) and EigenWD (cost-structure complexity).

Panel of Figure 5 — Figure 5: Ablation results on Llama-3.1-8B. Left: AUROC versus the number of sampled responses $K$ . Right: AUROC versus temperature $\tau$ . Unless otherwise stated, we keep decoding settings (including top- $k$ and top- $\rho$ ) fixed across runs.

Dataset	EigenWD M1	ER	ES	LNE	LS	DSE
SciQ	0.6480	0.6398	0.5784	0.5916	0.4921	0.5978
NQ-open	0.7861	0.6797	0.6088	0.6585	0.6804	0.7114
Math500	0.8889	0.8760	0.5587	0.5344	0.8335	0.5997
Average	0.7743	0.7318	0.5820	0.5948	0.6687	0.6363

Table 2: Black-box case study: DeepSeek-Chat (target model) with Llama-2-7B (teacher model). AUROC for hallucination detection (higher is better). Best and second-best per row are marked by bold and underline .

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Black model	Teacher model	Dataset	EigenWD M1	ER	ES	LNE	LS	DSE
chatgpt-4o-mini	Llama-2-7b	SciQ	0.6032	0.6382	0.6252	0.6223	0.6145	0.6160
		NQ-open	0.6414	0.6350	0.5680	0.6385	0.6276	0.5803
		Math500	0.7876	0.8099	0.6307	0.6449	0.7795	0.5686
	Llama-3.1-8b	SciQ	0.6251	0.6108	0.6148	0.6038	0.6145	0.6160
		NQ-open	0.6303	0.6263	0.6205	0.6207	0.6276	0.5803
		Math500	0.8329	0.7556	0.6794	0.6176	0.7795	0.5686

Appendix

Table 3: Black-box hallucination detection with teacher forcing, 10 samples (AUROC; higher is better). Best and second-best per row are marked by bold and underline .

Black model	Teacher model	Dataset	EigenWD M1	ER	ES	LNE	LS	DSE
chatgpt-4o-mini	Llama-2-7b	SciQ	0.6302	0.6409	0.6074	0.6154	0.6075	0.6172
		NQ-open	0.6368	0.6338	0.5677	0.6024	0.6199	0.6097
		Math500	0.8652	0.8049	0.6534	0.6389	0.7723	0.6071
	Llama-3.1-8b	SciQ	0.6280	0.6330	0.6406	0.5862	0.6075	0.6172
		NQ-open	0.6344	0.6300	0.6238	0.6060	0.6199	0.6097
		Math500	0.8387	0.7660	0.6993	0.6306	0.7723	0.6071

Appendix

Table 4: Black-box hallucination detection with teacher forcing, 15 samples (AUROC; higher is better). Best and second-best per row are marked by bold and underline .

Black model	Teacher model	Dataset	EigenWD M1	ER	ES	LNE	LS	DSE
chatgpt-4o-mini	Llama-2-7b	SciQ	0.6192	0.6412	0.6073	0.6092	0.6134	0.6167
		NQ-open	0.6465	0.6321	0.5520	0.6043	0.6323	0.6130
		Math500	0.8567	0.8035	0.6540	0.6648	0.7911	0.6010
	Llama-3.1-8b	SciQ	0.6204	0.6429	0.6481	0.5876	0.6134	0.6167
		NQ-open	0.6356	0.6338	0.6250	0.6084	0.6323	0.6130
		Math500	0.8645	0.7548	0.6841	0.6057	0.7911	0.6010

Appendix

Table 5: Black-box hallucination detection with teacher forcing, 20 samples (AUROC; higher is better). Best and second-best per row are marked by bold and underline .

$t$	Dataset	AvgWD M1	AvgWD L1	EigenWD M1	EigenWD L1	ER	ES	DSE	LNE	LS
0.1	CoQA	0.642	0.633	0.645	0.643	0.630	0.620	0.635	0.596	0.631
	SQuAD	0.577	0.561	0.577	0.577	0.565	0.451	0.563	0.555	0.556
	MATH-500	0.682	0.671	0.685	0.685	0.671	0.665	0.668	0.613	0.628
	CNN/DailyMail	0.648	0.640	0.633	0.632	0.641	0.629	0.619	0.588	0.607
	Average	0.637	0.626	0.635	0.634	0.627	0.591	0.621	0.588	0.606
0.3	CoQA	0.669	0.659	0.673	0.670	0.648	0.668	0.670	0.572	0.667

Appendix

Table 6: Ablation on sampling temperature

t

for Qwen3-8B (AUROC; higher is better). Best and second-best per row are marked by bold and underline .

$N$	Dataset	AvgWD M1	AvgWD L1	EigenWD M1	EigenWD L1	ER	ES	DSE	LNE	LS
10	CoQA	0.742	0.704	0.734	0.728	0.735	0.733	0.733	0.557	0.738
	SQuAD	0.656	0.642	0.645	0.640	0.626	0.627	0.647	0.617	0.647
	MATH-500	0.726	0.703	0.712	0.712	0.715	0.718	0.729	0.656	0.689
	CNN/DailyMail	0.679	0.643	0.668	0.668	0.675	0.668	0.654	0.615	0.645
	Average	0.701	0.673	0.690	0.687	0.688	0.686	0.691	0.611	0.680
15	CoQA	0.733	0.693	0.723	0.720	0.725	0.727	0.725	0.545	0.728

Appendix

Table 7: Ablation on number of generations

N

for Qwen3-8B (AUROC; higher is better). Best and second-best per row are marked by bold and underline .

$t$	Dataset	AvgWD M1	AvgWD L1	EigenWD M1	EigenWD L1	ER	ES	DSE	LNE	LS
0.1	CoQA	0.722	0.726	0.726	0.726	0.693	0.717	0.713	0.587	0.713
	SQuAD	0.727	0.729	0.758	0.758	0.740	0.725	0.710	0.618	0.707
	MATH-500	0.665	0.661	0.682	0.682	0.687	0.669	0.676	0.598	0.659
	CNN/DailyMail	0.596	0.600	0.601	0.601	0.598	0.568	0.597	0.529	0.573
	Average	0.678	0.679	0.692	0.692	0.680	0.670	0.674	0.583	0.663
0.3	CoQA	0.775	0.776	0.790	0.790	0.775	0.744	0.774	0.594	0.772

Appendix

Table 8: Ablation on sampling temperature

t

for Llama-3.1-8B (AUROC; higher is better). Best and second-best per row are marked by bold and underline .

$N$	Dataset	AvgWD M1	AvgWD L1	EigenWD M1	EigenWD L1	ER	ES	DSE	LNE	LS
10	CoQA	0.771	0.771	0.789	0.789	0.729	0.712	0.761	0.618	0.769
	SQuAD	0.817	0.815	0.799	0.799	0.767	0.785	0.769	0.613	0.734
	MATH-500	0.713	0.713	0.722	0.722	0.705	0.709	0.726	0.603	0.711
	CNN/DailyMail	0.625	0.622	0.629	0.629	0.624	0.620	0.615	0.548	0.605
	Average	0.732	0.730	0.735	0.735	0.706	0.707	0.718	0.596	0.705
15	CoQA	0.775	0.775	0.801	0.801	0.735	0.754	0.771	0.605	0.788

Appendix

Table 9: Ablation on number of generations

N

for Llama-3.1-8B (AUROC; higher is better). Best and second-best per row are marked by bold and underline .

Figure 6: Ablation on the numerator order

p

in EigenWD.

LLM Hallucination Detection

Large language model hallucination detection identifies unsupported, incorrect, or fabricated claims in model outputs and traces.

See all 208 papers

Entropy Distribution as a Fingerprint for Hallucinations in Generative Models
May 27, 2026Mattia J. Villani, Pranav Deshpande, Akshay Seshadri +2Language Model Error Detection Language Model Generation Evaluation
Global Technology Applied Research, JPMorganChase, New York, NY 10001, USA
LLM Hallucination Detection via Low-rank Multilinear Decomposition
Oct 26, 2026Devansh Sharma, Faizul Rakib Sayem, Shahana IbrahimLLM Hallucination Detection Hallucination in Language Models
Department of Electrical & Computer Engineering, University of Central Florida
Spilled Energy in Large Language Models
Feb 21, 2026Adrian Robert Minut, Hazem Dewidar, Iacopo MasiLLM Reliability LLM Hallucination Detection
Sapienza University of Rome, Italy · OmnAI Lab GLADIA

May 27, 2026J/K move · Enter open · S save

Entropy Distribution as a Fingerprint for Hallucinations in Generative Models

Mattia J. Villani, Pranav Deshpande, Akshay Seshadri, Romina Yalovetzky+1

Global Technology Applied Research, JPMorganChase, New York, NY 10001, USA

Language Model Error Detection Language Model Generation Evaluation Hallucination Detection Hallucination in Language Models LLM Reliability LLM Hallucination Detection+1

Open paper AskPDF arXiv

Large Language Models (LLMs) often generate factually incorrect outputs, commonly termed hallucinations, that undermine trust and limit deployment in high-stakes settings. Existing hallucination detection methods typically require multiple forward passes, or access to model internals. In this work, we provide theoretical background and empirical evidence that the distribution of token-level entropies, beyond the mean captured by perplexity or length-normalised entropy, serves as a fingerprint of hallucination, with distributional shape and tail behaviour carrying independent signal. We formalize hallucination detection as a statistical hypothesis test and propose the Calibrated Entropy Score (CES), a lightweight algorithm requiring only a single forward pass and black-box access to token logits. CES combines the mean signal with the maximum signal of the generated entropy through a calibrated reference CDF, producing scores that are directly comparable across models and tasks. We establish finite-sample calibration guarantees via a novel random-length Dvoretzky--Kiefer--Wolfowitz inequality, and also prove that CES detects hallucinations with probability converging to one exponentially fast in the generation length. Across eight QA benchmarks and ten generator models spanning open-source and API access models, CES achieves the highest detection performance among all single-pass black-box methods while providing formal error guarantees that existing heuristics lack. Remarkably, CES is statistically indistinguishable from multi-sample methods that require far greater computational cost, closing the gap between lightweight and expensive detection and making it suitable for real-time, large-scale deployment.

Abstract

LLM Hallucination Detection

Explore similar work

Entropy Distribution as a Fingerprint for Hallucinations in Generative Models

LLM Hallucination Detection via Low-rank Multilinear Decomposition

Spilled Energy in Large Language Models