Weakly Supervised Distillation of Hallucination Signals into Transformer Representations

Shoaib Sadiq Salehmohamed; Jinal Prashant Thakkar; Hansika Aredla; Shaik Mohammed Omar; Shalmali Ayachit

cs.AIApr 7, 2026

Weakly Supervised Distillation of Hallucination Signals into Transformer Representations

Authors: Shoaib Sadiq Salehmohamed, Jinal Prashant Thakkar, Hansika Aredla, Shaik Mohammed Omar, Shalmali Ayachit

Organizations: LLM Lens

Abstract

Existing hallucination detection methods for large language models (LLMs) rely on external verification at inference time, requiring gold answers, retrieval systems, or auxiliary judge models. We ask whether this external supervision can instead be distilled into the model's own representations during training, enabling hallucination detection from internal activations alone at inference time. We introduce a weak supervision framework that combines three complementary grounding signals: substring matching, sentence embedding similarity, and an LLM as a judge verdict to label generated responses as grounded or hallucinated without human annotation. Using this framework, we construct a 15000-sample dataset from SQuAD v2 (10500 train/development samples and a separate 5000-sample test set), where each example pairs a LLaMA-2-7B generated answer with its full per-layer hidden states and structured hallucination labels. We then train five probing classifiers: ProbeMLP (M0), LayerWiseMLP (M1), CrossLayerTransformer (M2), HierarchicalTransformer (M3), and CrossLayerAttentionTransformerV2 (M4), directly on these hidden states, treating external grounding signals as training-time supervision only. Our central hypothesis is that hallucination detection signals can be distilled into transformer representations, enabling internal detection without any external verification at inference time. Results support this hypothesis. Transformer-based probes achieve the strongest discrimination, with M2 performing best on 5-fold average AUC/F1, and M3 performing best on both single-fold validation and held-out test evaluation. We also benchmark inference efficiency: probe latency ranges from 0.15 to 5.62 ms (batched) and 1.55 to 6.66 ms (single sample), while end-to-end generation plus probe throughput remains approximately 0.231 queries per second, indicating negligible practical overhead.

Figures & tables

Model	AUC	F1	ECE	Acc
M0: ProbeMLP	0.7560 $\pm$ 0.0079	0.4969 $\pm$ 0.0069	0.2370 $\pm$ 0.0583	0.6687
M1: LayerWiseMLP	0.7999 $\pm$ 0.0105	0.5403 $\pm$ 0.0103	0.2258 $\pm$ 0.0434	0.7064
M2: CrossLayerTransformer	0.8515 $\pm$ 0.0054	0.6096 $\pm$ 0.0154	0.1721 $\pm$ 0.0600	0.7851
M3: HierarchicalTransformer	0.8490 $\pm$ 0.0089	0.5898 $\pm$ 0.0219	0.1845 $\pm$ 0.0791	0.7689
M4: CrossLayerAttentionTransformerV2	0.8275 $\pm$ 0.0066	0.5845 $\pm$ 0.0142	0.1231 $\pm$ 0.0461	0.7920

Table 1: 5-fold cross-validation performance (mean

\pm

std) across the five trained architectures (M0–M4). All probes are trained on

y_{\text{judge}}

labels. Higher AUC/F1/Acc and lower ECE are better.

Model	AUC	[email protected]	Best F1	$\tau^{*}$	ECE	Acc
M0: ProbeMLP	0.7271	0.4864	0.4864	0.50	0.2468	0.6590
M1: LayerWiseMLP	0.7921	0.5463	0.5463	0.50	0.1739	0.7343
M2: CrossLayerTransformer	0.8535	0.6250	0.6381	0.75	0.1787	0.7943
M3: HierarchicalTransformer	0.8577	0.6404	0.6445	0.65	0.1051	0.8219
M4: CrossLayerAttentionTransformerV2	0.8377	0.6098	0.6099	0.55	0.1308	0.8038

Table 2: Single-fold validation metrics from best-checkpoint analysis. Here,

\tau^{*}

denotes the threshold maximizing validation F1.

Model	AUC	PR-AUC	F1	Acc	ECE	Best F1	$\tau^{*}$
M0: ProbeMLP	0.7441	0.5124	0.5316	0.6712	0.2092	0.5351	0.5468
M1: LayerWiseMLP	0.8074	0.6070	0.6025	0.7514	0.1430	0.6084	0.4581
M2: CrossLayerTransformer	0.8490	0.6960	0.6475	0.7742	0.1747	0.6600	0.6453
M3: HierarchicalTransformer	0.8577	0.7057	0.6644	0.8040	0.1025	0.6653	0.4729
M4: CrossLayerAttentionTransformerV2	0.8294	0.6454	0.6236	0.7822	0.1136	0.6282	0.3646

Table 3: Final held-out test-set results (5,000 rows) on the held-out test dataset. Here,

\tau^{*}

is the threshold that maximizes F1 on the evaluation sweep.

Model	Batch lat. (ms)	Single lat. (ms)	Samples/s	GPU mem (MB)
M0: ProbeMLP	0.15	1.66	6687.96	584.51
M1: LayerWiseMLP	0.19	1.55	5362.70	589.26
M2: CrossLayerTransformer	5.62	6.66	177.83	629.25
M3: HierarchicalTransformer	2.59	4.02	385.43	628.68
M4: CrossLayerAttentionTransformerV2	1.34	4.68	745.58	603.66

Table 4: Probe-only benchmark with hidden states precomputed. “Batch lat.” denotes amortized latency under batched inference; “Single lat.” denotes one-sample latency.

Model	Gen. (s)	Probe (ms)	Total (s)	Total q/s	Avg prob.
M0: ProbeMLP	4.3313	1.262	4.3325	0.231	0.549
M1: LayerWiseMLP	4.3313	1.584	4.3329	0.231	0.489
M2: CrossLayerTransformer	4.3313	6.709	4.3380	0.231	0.470
M3: HierarchicalTransformer	4.3313	3.637	4.3349	0.231	0.456
M4: CrossLayerAttentionTransformerV2	4.3313	4.758	4.3360	0.231	0.332

Table 5: End-to-end deployment-style benchmark including generation, hidden-state extraction, and probe scoring. “Avg prob.” is the mean predicted hallucination probability over the benchmark set.

Method	AUC	F1	Acc
Cosine Similarity (Max)	0.4110	0.4269	0.5146
Mean Similarity	0.4092	0.4220	0.4900
Softmax Similarity	0.4105	0.4254	0.5068
Token F1 Similarity	0.4222	0.3652	0.3116
Best Probe (M3: HierarchicalTransformer)	0.8577	0.7057	0.6644

Table 6: Output-level similarity baselines and best probe result on the held-out 5,000-row test set.

Model ID	Acc	Best F1	ROC-AUC	PR-AUC	ECE
M0	0.8925	0.8262	0.9528	0.8880	0.1145
M1	0.8127	0.6357	0.8655	0.6760	0.1515
M2	0.8641	0.7520	0.9255	0.8123	0.1754
M3	0.7472	0.6959	0.8954	0.7520	0.2484
M4	0.8702	0.7396	0.9187	0.7976	0.1265

Table 7: Diagnostic metrics computed on the full training-data evaluation run ( eval 10 , 10k samples). These are not held-out test metrics and should not be interpreted as final generalization performance.

Panel of Figure 1 — Figure 1: PR (top left), ROC (top right), and predicted logit distribution (bottom) from full train/development-pool diagnostic inference. These plots are intended for qualitative comparison only and do not reflect held-out test evaluation.

LLM Hallucination Detection

Large language model hallucination detection identifies unsupported, incorrect, or fabricated claims in model outputs and traces.

See all 208 papers

Oct 26, 2026

LLM Hallucination Detection via Low-rank Multilinear Decomposition

Large language models (LLMs) frequently generate fluent yet factually incorrect responses, raising concerns about their reliability. Recent hallucination detectors increasingly exploit internal representations, but extracting informative signals from high-dimensional hidden states remains challenging. Existing approaches often rely on selected activations or pooling-based compression, potentially overlooking discriminative information distributed across Transformer layers, feature modes, and tokens. We propose LOTUS (Low-rank Order-statistic TUcker-based Subspace learning), a multilinear low-rank decomposition framework for hallucination detection that extracts compact representations from LLM hidden states while reducing dependence on labeled data. Our approach first summarizes variable-length token representations using complementary statistical descriptors that capture peak activations, activation distributions, and cross-layer changes. We then leverage low-rank multilinear decomposition to extract compact representations that capture the underlying structure of LLM hidden states, which are subsequently used to train a lightweight hallucination classifier. Experiments across multiple LLMs and question-answering benchmarks demonstrate improved hallucination detection performance over recent state-of-the-art baselines.

Devansh Sharma, Faizul Rakib Sayem, Shahana Ibrahim

Department of Electrical & Computer Engineering, University of Central Florida

LLM Hallucination Detection Hallucination in Language Models

Ask PDF

Oct 1, 2026

External Observers May See More Clearly: Cross-Model Span-Level Hallucination Detection in Large Language Models via Hidden State Probing

As Large Language Models (LLMs) increasingly serve as foundational reasoning engines, their tendency to hallucinate remains a critical vulnerability. While recent internal state probes offer a promising alternative to slow external retrieval systems, they largely reduce hallucination detection to a token-wise binary classification task, failing to capture the structured, sequential boundaries of semantic drift. Here, we introduce an internal hidden state framework for fine-grained, span-level hallucination detection. By inspecting layer-wise activation patterns, we attempt to detect the exact hallucination onset and continuation tokens in an LLM generation. Our experiments show that this approach successfully isolates hallucination onsets, achieving substantial improvements in Precision-Recall AUC over random baselines despite extreme class imbalance. Ultimately, we propose a novel cross-model detection framework in which one model observes the internal representations elicited by another model's generation. We find that an external observer can match or exceed a generator's self-detection of its own hallucination onsets, including when the observer is the smaller model, suggesting that self-detection is not the ceiling for onset localisation.

Kingshuk Gupta, Davide Buscaldi

LIX, École Polytechnique · School of Electrical and Electronic Engineering, Nanyang Technological University · LIPN, Sorbonne Paris Nord

Hallucination Detection LLM Hallucination Detection Language Model Probing

Ask PDF

Feb 2, 2026

HALT: Hallucination Assessment via Log-probs as Time series

Hallucinations remain a major obstacle for large language models (LLMs), especially in safety-critical domains. We present HALT (Hallucination Assessment via Log-probs as Time series), a lightweight hallucination detector that leverages only the top-20 token log-probabilities from LLM generations as a time series. HALT uses a gated recurrent unit model combined with entropy-based features to learn model calibration bias, providing an extremely efficient alternative to large encoders. Unlike white-box approaches, HALT does not require access to hidden states or attention maps, relying only on output log-probabilities. Unlike black-box approaches, it operates on log-probs rather than surface-form text, which enables stronger domain generalization and compatibility with proprietary LLMs without requiring access to internal weights. To benchmark performance, we introduce HUB (Hallucination detection Unified Benchmark), which consolidates prior datasets into ten capabilities covering both reasoning tasks (Algorithmic, Commonsense, Mathematical, Symbolic, Code Generation) and general purpose skills (Chat, Data-to-Text, Question Answering, Summarization, World Knowledge). While being 30x smaller, HALT outperforms Lettuce, a fine-tuned modernBERT-base encoder, achieving a 60x speedup gain on HUB. HALT and HUB together establish an effective framework for hallucination detection across diverse LLM capabilities.

Ahmad Shapiro, Karan Taneja, Ashok Goel

Georgia Institute of Technology · Microsoft

Language Model Error Detection Hallucination Detection Hallucination in Language Models LLM Reliability LLM Hallucination Detection

Ask PDF

Abstract

LLM Hallucination Detection

Explore similar work

LLM Hallucination Detection via Low-rank Multilinear Decomposition

External Observers May See More Clearly: Cross-Model Span-Level Hallucination Detection in Large Language Models via Hidden State Probing

HALT: Hallucination Assessment via Log-probs as Time series