Gray-Box Hallucination Detection in LLMs with Token Identities

Shaked Goldshtein; Yitzchak Grunbaum; Fabrizio Frasca; Guy Bar-Shalom; Haggai Maron

cs.CLOct 14, 2026

Gray-Box Hallucination Detection in LLMs with Token Identities

Authors: Shaked Goldshtein, Yitzchak Grunbaum, Fabrizio Frasca, Guy Bar-Shalom, Haggai Maron

Organizations: Technion

Abstract

Large language models often produce fluent but factually incorrect output. Gray-box detectors identify such hallucinations from the token-probability statistics emitted during generation, avoiding the need for white-box access to hidden states that frontier models do not provide. These detectors, however, typically rely only on probability-derived representations and ignore the identities of the candidate tokens. They can therefore observe how uncertain the model is, but not what it is uncertain about. Consequently, they cannot distinguish between a model that is torn between "big" and "small" and one that is torn between "big" and "huge" when the two pairs receive the same probability distribution. We introduce GURI (Gray-box Uncertainty Representations with Identity), a lightweight gray-box hallucination detector that combines candidate-token probabilities with per-token vector identifiers. This exposes which alternatives the model considered while preserving the accessibility and transferability of the gray-box setting. Across three open-weight models and three question-answering benchmarks, GURI increases mean test ROC-AUC from 72.09% to 79.09% over a probability-only baseline. On the closed-source GPT-4.1 model, mean ROC-AUC increases from 67.39% to 71.84%. Among the three open-weight models, GURI outperforms transferred probability-only baseline in 17 of 18 directed cross-model transfers without fine-tuning. For the three open-weight models, cross-dataset transfer between HotpotQA and TriviaQA with the model held fixed improves mean ROC-AUC over transferred LOS-Net by 3.76 points.

Figures & tables

Panel of Figure 1 — Figure 1: Identical numerical profiles can conceal different candidate structures. Both cases share the same token distribution, so a probability-only detector receives the same input. Candidate identities reveal competing factual answers on the left and paraphrastic alternatives on the right. The emitted token is shown in bold.

Figure 2: GURI pipeline. At each generation step, a shared external tokenizer splits each top-

K

candidate into subtokens whose codes are aggregated. Fusion modules combine these identity codes with candidate probabilities, and a response-level Transformer processes the resulting sequence. Both components are trained jointly for answer-level hallucination detection.

Panel of Figure 3 — Figure 3: TDS- $K$ and ATP inputs.

Dataset	Target model	Confidence	LOS-Net	GURI
HotpotQA	Ministral	64.98	$70.68_{\pm 0.97}$	$\mathbf{77.18}_{\pm 1.37}$
	Gemma	61.13	$74.11_{\pm 1.30}$	$\mathbf{83.93}_{\pm 0.66}$
	Llama	60.94	$80.48_{\pm 1.01}$	$\mathbf{86.36}_{\pm 0.58}$
NQ-WC	Ministral	59.28	$65.01_{\pm 1.00}$	$\mathbf{73.32}_{\pm 0.84}$
	Gemma	55.29	$65.88_{\pm 1.19}$	$\mathbf{78.18}_{\pm 0.56}$
	Llama	53.27	$63.36_{\pm 0.58}$	$\mathbf{73.06}_{\pm 0.39}$

Table 1: Hallucination-detection performance on the

3\times 3

evaluation suite. Values are test ROC-AUC (%). Learned detectors report the mean and standard deviation over all 36 hyperparameter–seed combinations. GURI uses a fixed random code table.

Dataset	LOS-Net	GURI	Difference
HotpotQA	$64.20_{\pm 0.43}$	$\mathbf{70.12}_{\pm 0.36}$	$+5.92$
TriviaQA	$\mathbf{72.49}_{\pm 0.39}$	$71.78_{\pm 0.74}$	$-0.71$
NQ-WC	$65.49_{\pm 0.23}$	$\mathbf{73.63}_{\pm 0.61}$	$+8.14$
Mean	67.39	$\mathbf{71.84}$	$+4.45$

Table 2: Test ROC-AUC (%) on the closed-source GPT-4.1 model.

Panel of Figure 4 — Figure 4: Cross-model transfer of GURI with unit-norm random codes. Rows and columns denote source and target models: G, L, and M denote Gemma, Llama, and Ministral. Each row uses one source-validation-selected checkpoint. Bold values show test ROC-AUC (%). Colors and signed values show gains over target-model Confidence (AUC points). Stars and outlines mark scores above target-trained LOS-Net, not statistical significance.

Dataset	Target model	GURI: Random	GURI: Phi	Alternating: Phi	Alternating: Random
HotpotQA	Ministral	$\mathbf{77.18}_{\pm 1.37}$	$76.82_{\pm 0.78}$	$\mathbf{77.18}_{\pm 1.35}$	$76.85_{\pm 1.24}$
	Gemma	$83.93_{\pm 0.66}$	$81.70_{\pm 0.60}$	$82.91_{\pm 0.50}$	$\mathbf{84.61}_{\pm 0.14}$
	Llama	$86.36_{\pm 0.58}$	$85.60_{\pm 0.56}$	$86.19_{\pm 0.18}$	$\mathbf{87.55}_{\pm 0.10}$
NQ-WC	Ministral	$\mathbf{73.32}_{\pm 0.84}$	$70.53_{\pm 1.04}$	$71.14_{\pm 0.40}$	$72.94_{\pm 0.64}$
	Gemma	$\mathbf{78.18}_{\pm 0.56}$	$76.58_{\pm 0.67}$	$76.72_{\pm 0.84}$	$76.99_{\pm 1.25}$
	Llama	$73.06_{\pm 0.39}$	$69.87_{\pm 0.85}$	$69.62_{\pm 0.51}$	$\mathbf{73.43}_{\pm 0.71}$

Table 3: Test ROC-AUC (%) for code-source and architecture variants. Random codes were unit-normalized; Phi-4-mini embeddings were not. Bold indicates the highest mean in each row, including ties at the reported precision.

Panel of Figure 5 — Figure 5: Non-actual identities alone match or exceed the full model in all nine settings. Test ROC-AUC after retraining with unit-norm random codes and actual only , non-actual only , or both identity channels, retaining all probability inputs. Markers and error bars show the mean and standard deviation over three training seeds. LOS-Net was retrained with the same hyperparameters. Actual identities alone recover only part of the gain.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

	HotpotQA		TriviaQA		NQ-WC
Model	Train pool	Test	Train pool	Test	Train pool	Test
Ministral	33.60	29.98	60.99	60.87	54.77	54.62
Llama	28.45	24.97	58.32	60.23	60.74	60.52
Gemma	30.27	26.56	58.69	58.70	62.65	62.82

Appendix

Table 4: Responses labeled factual (%) in each dataset. Train pool denotes the data before the training/validation split.

Panel of Figure 6 — Figure 6: The fitted detector relies on both identity channels. Test ROC-AUC lost when, with the trained GURI frozen, the token-to-code correspondence is permuted at evaluation time (a derangement of all token identities) in every slot, in the actual-token channel only (its ATP entry and its candidate slot at every step), or in the non-actual candidates’ slots only. Unit-norm random codes, three LLMs, three datasets, bars are the mean over 3 permutation seeds and error bars one standard deviation. Both channels cost AUC in every cell, and the joint loss exceeds the sum of the two everywhere, by a wide margin in most cells: neither channel is read in isolation. The fitted detector leans more on the actual token (the larger loss in eight of nine cells), yet a permutation loss measures what this detector depends on, not what a channel is worth: retrained without one channel (Fig. 5 ), the non-actual candidates alone match the full model while the actual token alone does not.

Panel of Figure 7 — Figure 7: The same retraining with pretrained Phi-4-mini embeddings in place of random codes. Layout and hyper-parameters as in Fig. 5 ; each token’s vector is the mean of the Phi-4-mini input embeddings of its sub-tokens. The non-actual-only detector is lower than with random codes in every cell (by 2 to 7 AUC points), so the full model now needs the actual-token channel to reach its best value, and on NQ the actual token alone is worth more than the non-actual candidates. The full model itself is below its random-code counterpart in all nine cells. Mean over 3 training seeds, error bars one standard deviation.

Dataset transfer	Model	$n$	Confidence	LOS-Net	GURI
$H\leftrightarrow T$	Same	6	62.30	69.55	73.31
$H\leftrightarrow T$	Different	12	62.30	57.18	62.11
$N\leftrightarrow\{H,T\}$	Same	12	59.13	52.97	52.94
$N\leftrightarrow\{H,T\}$	Different	24	59.13	51.56	50.99

Appendix

Table 5: Mean test ROC-AUC (%) under dataset shifts.

H

T

, and

N

denote HotpotQA, TriviaQA, and NQ-WC. “Same” and “Different” indicate whether the source and target models coincide;

n

is the number of directed transfers. Confidence uses the target-specific baseline scores, averaged over the same target cells. Means summarize transfer settings, not repeated training runs.

LLM Hallucination Detection

Large language model hallucination detection identifies unsupported, incorrect, or fabricated claims in model outputs and traces.

See all 208 papers

May 8, 2026

Hallucination Detection via Activations of Open-Weight Proxy Analyzers

We introduce a proxy-analyzer framework for detecting hallucinations in large language models. Instead of looking inside the generating model, our system reads already-generated text through a small locally hosted open-weight model and spots hallucinations using the reader's own internal activations. This works just as well when the generator is a closed API like GPT-4 as when it is any open-weight model. We built eighteen features grounded in how transformers process text, covering residual stream norms, per-head source-document attention, entropy, MLP activations, logit-lens trajectories, and three new token-level grounding statistics. We trained a stacking ensemble on 72,135 samples from five hallucination datasets. We tested across seven analyzer architectures from 0.5 billion to 9 billion parameters: Qwen2.5 at 0.5B and 7B, Gemma-2 at 2B and 9B, Pythia at 1.4B, and LLaMA-3 at both 3B and 8B. Across all seven, we consistently beat ReDeEP's token-level AUC of 0.73 on RAGTruth by 7.4 to 10.3 percentage points. Qwen2.5-7B reached an F1 of 0.717, just above ReDeEP's 0.713, while Qwen2.5-0.5B hit 0.706. The most striking finding is how tightly all seven models cluster: AUC spans only 2.3 percentage points across an eighteen-fold difference in model size. Even more surprising, our 3B LLaMA outperforms our 8B LLaMA on RAGTruth, showing that bigger is not always better even within the same model family. Both RAGTruth and LLM-AggreFact include outputs from multiple LLM families, so our results are not skewed toward any particular generator.

Akshita Singh, Prabesh Paudel, Siddhartha Roy

Khoury College of Computer Sciences Northeastern University

Hallucination in Language Models LLM Reliability LLM Hallucination Detection

Ask PDF

May 6, 2026

Detecting Hallucinations in Large Language Models via Internal Attention Divergence Signals

We propose a lightweight and single-pass uncertainty quantification method for detecting hallucinations in Large Language Models. The method uses attention matrices to estimate uncertainty without requiring repeated sampling or external models. Specifically, we measure the Kullback-Leibler divergence between each attention head's distribution and a uniform reference distribution, and use these features in a logistic regression probe. Across multiple datasets, task types, and model families, attention divergence is highly predictive of answer correctness and performs competitively with existing uncertainty estimation methods. We find that this signal is concentrated in middle layers and on factual tokens such as named entities and numbers, suggesting that attention dynamics provides an efficient and interpretable white-box signal of model uncertainty.

Gijs van Dijk

Utrecht University

Language Model Error Detection Confidence Estimation in Language Models LLM Interpretability Hallucination in Language Models LLM Reliability LLM Hallucination Detection+1

Ask PDF

Oct 20, 2026

Detecting Errors in Large Language Model Responses: The Incremental Predictive Value of Hidden States

Although large language models support a wide range of tasks, hallucinations remain a major obstacle to user trust. When a model generates an answer, both its internal processing and final output vary with the question. Internal states are often expected to offer richer signals for hallucination detection. However, studies using these states typically report their standalone discrimination performance, while output-side comparisons often use simple baselines with weaker modeling capacity. We integrate output-side information available at inference time into a combined baseline and compare detection performance before and after adding hidden states under identical data splits and evaluation rules. Experiments cover open-ended generation and constrained multiple-choice tasks across different models. Adding hidden states improves AUROC by approximately 0.004 and 0.039 on the two tasks, respectively. Hidden states thus provide additional predictive information, but strong standalone detection performance can still yield only limited additional gains.

Ningzhi Li, Xiaojun Ma, Xiaofei Wang

Beijing Institute of Computer Technology and Application, Beijing, China

Language Model Error Detection LLM Hallucination Detection

Ask PDF

Abstract

LLM Hallucination Detection

Explore similar work

Hallucination Detection via Activations of Open-Weight Proxy Analyzers

Detecting Hallucinations in Large Language Models via Internal Attention Divergence Signals

Detecting Errors in Large Language Model Responses: The Incremental Predictive Value of Hidden States