Detecting Errors in Large Language Model Responses: The Incremental Predictive Value of Hidden States

Ningzhi Li; Xiaojun Ma; Xiaofei Wang

cs.CLOct 20, 2026

Detecting Errors in Large Language Model Responses: The Incremental Predictive Value of Hidden States

Authors: Ningzhi Li, Xiaojun Ma, Xiaofei Wang

Organizations: Beijing Institute of Computer Technology and Application, Beijing, China

Abstract

Although large language models support a wide range of tasks, hallucinations remain a major obstacle to user trust. When a model generates an answer, both its internal processing and final output vary with the question. Internal states are often expected to offer richer signals for hallucination detection. However, studies using these states typically report their standalone discrimination performance, while output-side comparisons often use simple baselines with weaker modeling capacity. We integrate output-side information available at inference time into a combined baseline and compare detection performance before and after adding hidden states under identical data splits and evaluation rules. Experiments cover open-ended generation and constrained multiple-choice tasks across different models. Adding hidden states improves AUROC by approximately 0.004 and 0.039 on the two tasks, respectively. Hidden states thus provide additional predictive information, but strong standalone detection performance can still yield only limited additional gains.

Figures & tables

Symbol	Meaning
$Y$	Error label: 1 for an erroneous response and 0 otherwise
$S$	Output-side information included in this study, including prompt-side controls
$B_{k}$	Information block $k$ ( $k=1,\ldots,5$ ); its predictor in performance comparisons
$H$	Hidden representation from a prespecified layer and pooling rule; its predictor when standalone performance is reported
$S^{\ast}$	Combined baseline constructed from the five information blocks
$S^{\ast}+H$	Fused model adding the hidden-state readout to the combined baseline

Table 1: Main notation.

Panel of Figure 1 — Figure 1: Framework for evaluating conditional gain.

Block	Category	H1	MCQ
$B_{1}$	Answer identity	Eligible generated identifier strings	Selected answer letter
$B_{2}$	Surface form	Length, format, identifier count and positions	Option lengths and lexical overlap
$B_{3}$	Text content	Full generated text	Selected option text
$B_{4}$	Generation and sampling signals	Token scores, cross-sample consistency, and explicit confidence	Candidate-answer scores and cross-sample statistics
$B_{5}$	Prompt context	Prompt text, task family, and generation route	Question stem and all options

Table 2: Five information blocks in the main-analysis baseline.

Model	Task	$S^{\ast}$	$H$	$S^{\ast}+H$	$\Delta_{H}$ (95% CI)
Qwen	H1	0.9889	0.9438	0.9905	$+0.0016$ $[+0.0006,+0.0028]$
Qwen	MCQ	0.8198	0.8292	0.8487	$+0.0289$ $[+0.0189,+0.0389]$
Llama	H1	0.9712	0.9148	0.9752	$+0.0040$ $[+0.0014,+0.0075]$
Llama	MCQ	0.7465	0.7916	0.8061	$+0.0596$ $[+0.0447,+0.0751]$
Task	Readout or representation	$S^{\ast}$	$H$	$S^{\ast}+H$	$\Delta_{H}$ (95% CI)
H1	L2, wide grid	0.9889	0.9482	0.9910	$+0.0021$ $[+0.0008,+0.0037]$

Table 3: Error detection for both models, with readout and representation comparisons.

Figure 2: Absolute performance and conditional gain with and without

B_{4}

. Each panel has its own vertical scale; points and connecting lines show point estimates only.

Model	$B_{3}$ source	$S^{\ast}$	$H$	$S^{\ast}+H$	$\Delta_{H}$ (95% CI)
Qwen	Option text	0.8198	0.8292	0.8487	$+0.0289$ $[+0.0189,+0.0389]$
Qwen	Full response	0.8221	0.8292	0.8495	$+0.0274$ $[+0.0172,+0.0378]$
Qwen	Option + response	0.8221	0.8292	0.8498	$+0.0277$ $[+0.0176,+0.0379]$
Llama	Option text	0.7465	0.7916	0.8061	$+0.0596$ $[+0.0447,+0.0751]$
Llama	Full response	0.7699	0.7916	0.8076	$+0.0377$ $[+0.0255,+0.0502]$
Llama	Option + response	0.7689	0.7916	0.8075	$+0.0387$ $[+0.0264,+0.0515]$

Table 4: Output coverage of the MCQ text block.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Block	H1	MCQ
$B_{1}$	Text representation of eligible identifier strings in the response	Four-dimensional one-hot encoding of the selected answer letter A–D
$B_{2}$	9 surface and format features: length, line count, list structure, identifier count and positions	6 surface features: selected-option length, relative option lengths, and lexical overlap
$B_{3}$	Full generated text	Text of the selected option
$B_{4}$	14 features: identifier-token and response log-probabilities, entropy, ranks, length-normalized scores, cross-sample consistency, and explicit confidence	6 features: margins and entropy over four candidate letters, full-vocabulary entropy, answer-token log-probability, and cross-sample consistency
$B_{5}$	Prompt text with prespecified task-family and generation-route indicators	Question stem and all options, excluding the model’s selection

Appendix

Table 5: Implementation of

B_{1}

–

B_{5}

in H1 and MCQ.

Item	Prespecified setting
Outer and inner splits	5 outer folds and 3 inner folds; grouping depends on E1, E2, E3, or M2c
Base learner	L2 logistic regression, liblinear , maximum 2,000 iterations
Base-learner tuning	$C\in\{0.01,0.1,1,10\}$ , selected by grouped inner cross-validation within outer training folds
Meta-learner	L2 logistic regression, $C=1$ , lbfgs , maximum 5,000 iterations, tolerance $10^{-6}$ , intercept, no class weights
Meta-learner inputs	Base learners’ decision_function scores, without further standardization
Repetitions and seeds	3 repetitions; outer-fold seed $20260804+100r$

Appendix

Table 6: Prespecified learning and resampling parameters.

Arm	Unit	Final	Analyzable	Ineligible	Groups $G$	Errors (rate)
H1	Generation	9,264	8,664	600	1,508	693 (0.0800)
MCQ	Question	1,751	1,750	1	1,750	318 (0.1817)

Appendix

Table 7: Sample accounting for Qwen.

Purpose	Comparison setting	Analysis role
Dependence on generation and sampling signals	Path Cremoves $B_{4}$ from the baseline and fused model in both arms	Block-dependence diagnostic
Sensitivity to numerical handling	Path Bretains information coverage and hidden states, reconstructing affected features through tail stabilization for comparison with path A	Protocol-deviation sensitivity
Performance without shared identifiers (E2)	Split by prompt–identifier co-occurrence components without using error labels; do not estimate performance if structural conditions fail	Prespecified generalization evaluation
Performance under changed vocabulary scope (E3)	Partition vocabulary by a fixed hash rule and redefine block-level labels and hidden states, yielding a separate estimand	Prespecified additional estimand
Sensitivity to grouping (M2c)	Construct label-dependent groups from prompts and fabricated identifiers; use only for sensitivity because bias direction is unknown	Grouping sensitivity
Dependence on identity and format cues	Compare full-text block $B_{3}$ with identity block $B_{1}$ and format block $B_{2}$ to check the task construct	Prespecified construct check

Appendix

Table 8: Purposes and settings of additional comparisons.

Path	Arm	$n$	$G$	$S^{\ast}$	$H$	$S^{\ast}+H$	$\Delta_{H}$ (95% CI)	CI width	Precision
A	H1	8,664	1,508	0.9889	0.9438	0.9905	$0.00157$ $[0.00056,0.00283]$	0.00226	Adequate
A	MCQ	1,750	1,750	0.8198	0.8292	0.8487	$0.02893$ $[0.01894,0.03895]$	0.02001	Adequate
B	H1	8,664	1,508	0.9854	0.9438	0.9882	$0.00282$ $[0.00137,0.00456]$	0.00319	Adequate
B	MCQ	1,750	1,750	0.8219	0.8292	0.8492	$0.02728$ $[0.01747,0.03717]$	0.01970	Adequate
C	H1	8,664	1,508	0.9853	0.9438	0.9891	$0.00381$ $[0.00197,0.00592]$	0.00394	Adequate
C	MCQ	1,750	1,750	0.5684	0.8292	0.8251	$0.25672$ $[0.22699,0.28668]$	0.05969	Insufficient

Appendix

Table 9: Confirmatory estimates for the six E1 branches.

Figure 3: AUROC and conditional gain across E1 paths.

Path	Arm	CI width	Monte Carlo band for width	Precision
A	H1	0.00226	$[0.00221,0.00232]$	Adequate
A	MCQ	0.02001	$[0.01973,0.02028]$	Adequate
B	H1	0.00319	$[0.00312,0.00326]$	Adequate
B	MCQ	0.01970	$[0.01951,0.01989]$	Adequate
C	H1	0.00394	$[0.00390,0.00399]$	Adequate
C	MCQ	0.05969	$[0.05881,0.06058]$	Insufficient

Appendix

Table 10: E1 confidence interval widths and Monte Carlo error bands.

Path / task	$S^{\ast}$	$H$	$S^{\ast}+H$	$\Delta_{H}$ (95% CI)
A / H1	0.9712	0.9148	0.9752	$+0.0040$ $[+0.0014,+0.0075]$
A / MCQ	0.7465	0.7916	0.8061	$+0.0596$ $[+0.0447,+0.0751]$
C / H1	0.9668	0.9148	0.9725	$+0.0057$ $[+0.0021,+0.0104]$
C / MCQ	0.5595	0.7916	0.7816	$+0.2221$ $[+0.1937,+0.2506]$

Appendix

Table 11: Llama detection performance and conditional gains on paths A and C.

Arm and metric	$B_{1}$	$B_{2}$	$B_{3}$	$B_{4}$	$B_{5}$	$S^{\ast}$
H1 AUROC	0.9826	0.6662	0.9286	0.8023	0.6456	0.9889
H1 $\Delta_{S,k}$	0.0063	0.3227	0.0603	0.1865	0.3433	—
MCQ AUROC	0.4884	0.5329	0.5304	0.8256	0.5783	0.8198
MCQ $\Delta_{S,k}$	0.3314	0.2868	0.2894	$-0.0059$	0.2415	—

Appendix

Table 12: Single-block AUROC and baseline point-estimate differences on path A.

Path	Arm	iv-a min. bound (block)	M1b	$\Delta_{H}$ lower bound	iv-b
A	H1	$0.0048$ ( $B_{1}$ )	Pass	$0.0006$	Fail
A	MCQ	$0.0145$ ( $B_{4}$ )	Pass	$0.0189$	Fail
B	H1	$0.0016$ ( $B_{1}$ )	Pass	$0.0014$	Fail
B	MCQ	$0.0158$ ( $B_{4}$ )	Pass	$0.0175$	Fail
C	H1	$0.0038$ ( $B_{1}$ )	Pass	$0.0020$	Fail
C	MCQ	$0.2194$ ( $B_{5}$ )	Fail ( $B_{5}$ )	$0.2270$	Fail

Appendix

Table 13: Blockwise criteria for the six E1 branches.

Criterion	Arm	$B_{1}$	$B_{2}$	$B_{3}$	$B_{4}$	$B_{5}$	Decision
M1b	H1	0.0036	0.2969	0.0500	0.1653	0.3188	Pass
M1b	MCQ	0.3007	0.2572	0.2603	$-0.0112$	0.2131	Pass
iv-a	H1	0.0048	0.2987	0.0514	0.1667	0.3204	Pass
iv-a	MCQ	0.3310	0.2866	0.2912	0.0145	0.2430	Pass

Appendix

Table 14: Marginal one-sided 95% lower bounds for M1b and iv-a on path A.

Figure 4: Distance between blockwise lower bounds and criterion thresholds on primary path A. Points show the marginal one-sided 95% lower bound minus its threshold; positive values satisfy the corresponding criterion.

Path	$S^{\ast}$	$H$	$S^{\ast}+H$	$\Delta_{H}$ (95% CI)	M1b	iv-a	iv-b
A	0.8094	0.8319	0.8379	$0.0285$ $[0.0142,0.0434]$	Not certified ( $B_{4}$ )	Pass	Fail
B	0.8114	0.8319	0.8382	$0.0268$ $[0.0116,0.0426]$	Certified	Pass	Fail
C	0.7629	0.8319	0.8192	$0.0562$ $[0.0382,0.0747]$	Certified	Pass	Pass

Appendix

Table 15: Confirmatory E3 estimates and criteria.

Estimand	Path	$n$	$\Delta_{H}$ (95% CI)	Non-inferiority Min. bound / threshold	Direct comparison Min. bound / threshold	Consistency $\Delta_{H}$ bound / threshold
E3	A	6,085	$+0.0285$ $[0.0142,0.0434]$	$-0.0229/-0.0200$ ( $B_{4}$ )	$0.0053/0$ ( $B_{4}$ )	$0.0142/0.0200$
E3	B	6,085	$+0.0268$ $[0.0116,0.0426]$	$-0.0122/-0.0200$ ( $B_{4}$ )	$0.0139/0$ ( $B_{4}$ )	$0.0116/0.0200$
E3	C	6,085	$+0.0562$ $[0.0382,0.0747]$	$0.0065/-0.0200$ ( $B_{1}$ )	$0.0519/0$ ( $B_{1}$ )	$0.0382/0.0200$
M2c	A	8,664	$+0.0142$ $[0.0087,0.0192]$	$-0.0463/-0.0200$ ( $B_{1}$ )	$-0.0332/0$ ( $B_{1}$ )	$0.0087/0.0200$
M2c	B	8,664	$+0.0205$ $[0.0136,0.0270]$	$-0.0564/-0.0200$ ( $B_{1}$ )	$-0.0385/0$ ( $B_{1}$ )	$0.0136/0.0200$
M2c	C	8,664	$+0.0236$ $[0.0177,0.0291]$	$-0.0637/-0.0200$ ( $B_{1}$ )	$-0.0415/0$ ( $B_{1}$ )	$0.0177/0.0200$

Appendix

Table 16: Vocabulary-block generalization (E3) and grouping sensitivity (M2c).

Language Model Error Detection

Language model error detection identifies unreliable answers, reasoning steps, and actions so systems can flag, verify, or repair them.

See all 91 papers

External Observers May See More Clearly: Cross-Model Span-Level Hallucination Detection in Large Language Models via Hidden State Probing
Oct 1, 2026Kingshuk Gupta, Davide BuscaldiHallucination Detection LLM Hallucination Detection
LIX, École Polytechnique · School of Electrical and Electronic Engineering, Nanyang Technological University · LIPN, Sorbonne Paris Nord
Detecting Hallucinations in Large Language Models via Internal Attention Divergence Signals
May 6, 2026Gijs van DijkLanguage Model Error Detection Confidence Estimation in Language Models
Utrecht University
Domain-Specific Hallucination Detection in Large Language Models
Sep 10, 2026Varun Teja Chundru, Debasmita BiswasLanguage Model Error Detection LLM Hallucination Mitigation
Department of Computer Science Purdue University Fort Wayne

Oct 1, 2026J/K move · Enter open · S save

External Observers May See More Clearly: Cross-Model Span-Level Hallucination Detection in Large Language Models via Hidden State Probing

Kingshuk Gupta, Davide Buscaldi

LIX, École Polytechnique · School of Electrical and Electronic Engineering, Nanyang Technological University · LIPN, Sorbonne Paris Nord

Hallucination Detection LLM Hallucination Detection Language Model Probing

Open paper AskPDF arXiv

As Large Language Models (LLMs) increasingly serve as foundational reasoning engines, their tendency to hallucinate remains a critical vulnerability. While recent internal state probes offer a promising alternative to slow external retrieval systems, they largely reduce hallucination detection to a token-wise binary classification task, failing to capture the structured, sequential boundaries of semantic drift. Here, we introduce an internal hidden state framework for fine-grained, span-level hallucination detection. By inspecting layer-wise activation patterns, we attempt to detect the exact hallucination onset and continuation tokens in an LLM generation. Our experiments show that this approach successfully isolates hallucination onsets, achieving substantial improvements in Precision-Recall AUC over random baselines despite extreme class imbalance. Ultimately, we propose a novel cross-model detection framework in which one model observes the internal representations elicited by another model's generation. We find that an external observer can match or exceed a generator's self-detection of its own hallucination onsets, including when the observer is the smaller model, suggesting that self-detection is not the ceiling for onset localisation.

Abstract

Language Model Error Detection

Explore similar work

External Observers May See More Clearly: Cross-Model Span-Level Hallucination Detection in Large Language Models via Hidden State Probing

Detecting Hallucinations in Large Language Models via Internal Attention Divergence Signals

Domain-Specific Hallucination Detection in Large Language Models