Measurement Validity and Representation Dynamics of Confidence Signals in Large Language Models

Debojeet Ghosh; Shreyansh Kumar; Ayushman Garg

cs.CLOct 19, 2026

Measurement Validity and Representation Dynamics of Confidence Signals in Large Language Models

Authors: Debojeet Ghosh, Shreyansh Kumar, Ayushman Garg

Abstract

Whether a language model "knows what it knows" matters for trustworthy deployment and for deciding when it should abstain. Prior work usually measures this through one channel: verbalized confidence, a linear probe on hidden states, or agreement among sampled answers (semantic entropy), and often treats these as interchangeable. We compare all three in a pre-registered audit of five Qwen2.5 models (0.5B-7B, including a base/instruct pair at 7B) on factual retrieval (PopQA, SimpleQA) and multi-step reasoning (GSM8K, MATH). First, probes trained on prefill states to predict whether sampled answers will disagree become informative earlier for retrieval (about 7% of layer depth on average) than for reasoning (about 28%). The pre-registered five-layer test supported this only marginally; a finer 21-layer follow-up, designed after those results, supports it more clearly. In most retrieval cells, however, a prompt-only TF-IDF classifier matches the probe, so the early signal may reflect surface features of the question. Second, verbalized confidence is unstable: slightly different prompt templates yield scores that barely correlate. Third, Expected Calibration Error is misleading: verbal confidence looks well calibrated because it clusters near the base rate while discriminating poorly, whereas sampling agreement has worse ECE but about twice the resolution. Of five pre-registered hypotheses, two were falsified, one was partly supported, one was not supported, and one could not be tested because the base model's verbal confidence was degenerate.

Figures & tables

Figure 1: Complete 30-cell evaluation matrix. Pilot accuracy per (model $\times$ tier) cell. Dotted outlines indicate cells outside the 25% to 80% accuracy band.

Figure 2: AUROC across 21 layer depth percentiles. Calibration-split AUROC of linear probes predicting the median-split semantic-entropy label from prefill hidden states, faceted by model size. Retrieval tiers (cool lines) mostly cross 0.65 by 5% depth; reasoning tiers (warm lines) cross later on average. In 9 of 11 retrieval cells the onset-layer probe does not beat the prompt-only TF-IDF baseline (Table 1 ).

Panel A: Factual retrieval tiers ( $N=11$ cells; mean onset $=6.82\%$ , median $=5.0\%$ )
Model	Tier	Onset (%)	Onset layer	Probe AUROC (onset)	Probe AUROC (peak)	TF-IDF AUROC	Probe beats TF-IDF at onset
Qwen2.5-1.5B-Instruct	R1 (PopQA-high)	5.0	Layer 1 / 28	0.864	0.941	0.882	No
Qwen2.5-1.5B-Instruct	R2 (PopQA-low)	5.0	Layer 1 / 28	0.877	0.935	0.889	No
Qwen2.5-1.5B-Instruct	R3 (SimpleQA)	5.0	Layer 1 / 28	0.775	0.817	0.728	Yes
Qwen2.5-3B-Instruct	R2 (PopQA-low)	5.0	Layer 2 / 36	0.867	0.905	0.868	No
Qwen2.5-3B-Instruct	R3 (SimpleQA)	15.0	Layer 5 / 36	0.654	0.784	0.670	No

Table 1: Onset depth for all 24 probe-valid cells (21-tap follow-up sweep).

Comparison Pair	Spearman $\rho$	Pre-Registered Target	Status
Format A vs. Format B	$+0.126$	$\geq 0.60$	FALSIFIED
Format A vs. Format C	$-0.069$	$\geq 0.60$	FALSIFIED
Format B vs. Format C	$+0.247$	$\geq 0.60$	FALSIFIED

Table 2: Verbal format correlation matrix (Gate 2 compliance).

Figure 3: Signal calibration and Murphy decomposition on the test split. Left: Calibration curves close to the diagonal; points with

N<10

pruned to remove small-sample noise. Right: Murphy decomposition of the Brier score. All signals show small reliability error (solid bars), while behavioral consensus achieves more than double the resolution (light bars) of verbal confidence.

Signal	ECE (15 bins) $\downarrow$	ECE (10 bins) $\downarrow$	Reliability $\downarrow$	Resolution $\uparrow$	Brier $\downarrow$
Verbal ( $B_{\text{fix}}$ )	0.0366	0.0326	0.00171	0.03160	0.1713
Internal Probe	0.0500	0.0338	0.00219	0.05154	0.1524
Behavioral Consensus	0.0704	0.0680	0.00808	0.06371	0.1465

Table 3: Complete-case Murphy decomposition and ECE sensitivity (

N=842

matched items).

Figure 4: Epistemic alignment quadrants and forced-answer abstention behavior. Left: Calibrated verbal vs. behavioral/internal confidence. Right: The abstention split on Format C passes (justified hedge vs. missed knowledge).

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Cross-signal rank correlations. Per-cell Spearman correlations between the raw verbal, behavioral and internal signals, computed where both signals in a pair are present. Pooled over the test split, raw verbal and internal signals correlate at $\rho=0.086$ . After per-cell calibration the pooled correlation rises to 0.83, which mostly reflects differences in difficulty between cells.

Confidence Estimation in Language Models

Confidence estimation in language models measures whether a model’s uncertainty signals predict answer correctness and support reliable decisions.

See all 229 papers

An Analysis of Training-Free Self-Reported Confidence in Language Models
Sep 17, 2026Lukas Meyer, Sofia Rossi, Wei Chen +2Confidence Estimation in Language Models Language Model Self-Assessment
DreamAI
Closing the Confidence-Faithfulness Gap in Large Language Models
Mar 26, 2026Miranda Muqing Miao, Lyle UngarConfidence Estimation in Language Models Verbalized Confidence Estimation
University of Pennsylvania · Department of Computer and Information Science
Self-Aware Knowledge Probing: Evaluating Language Models' Relational Knowledge through Confidence Calibration
Jan 26, 2026Christopher Kissling, Elena Merdjanovska, Alan AkbikLanguage Model Probing Language Model Calibration
Humboldt-Universität zu Berlin · Science of Intelligence

Sep 17, 2026J/K move · Enter open · S save

An Analysis of Training-Free Self-Reported Confidence in Language Models

Lukas Meyer, Sofia Rossi, Wei Chen, Thomas Laurent+1

DreamAI

Confidence Estimation in Language Models Language Model Self-Assessment LLM Reliability LLM Uncertainty Estimation Self-Consistency in Language Models Verbalized Confidence Estimation

Open paper AskPDF arXiv

Large language models can report a numerical confidence together with generated content, but it is unclear whether this report is more than calibrated rhetoric. We analyze three training-free signals: confidence verbalized with the answer, post-hoc

P(\mathrm{True})

, and agreement with three additional generations on the same 100 TriviaQA questions for two model families. Direct verbalization is a surprisingly strong baseline: after auditing benchmark errors, it reaches AUROC 0.956 and 0.937 for correctness prediction. Three-sample agreement is substantially weaker (0.765 and 0.790), and a fixed interpolation with verbalized confidence has no statistically reliable benefit. Four of nine errors from one model and two of eight from the other receive unanimous sample support, showing that self-consistency can amplify shared misconceptions. Re-eliciting confidence for the same fixed answers with equivalent prompts changes scores by 0.043 to 0.084 on average and flips 4% to 9% of decisions at a 0.8 threshold. An exploratory audit of 100 confidence-tagged biography claims further finds only a modest confidence gap between supported and contradicted claims. These results argue that useful self-reports remain sensitive to elicitation, correlated errors, and benchmark noise.

Abstract

Confidence Estimation in Language Models

Explore similar work

An Analysis of Training-Free Self-Reported Confidence in Language Models

Closing the Confidence-Faithfulness Gap in Large Language Models

Self-Aware Knowledge Probing: Evaluating Language Models' Relational Knowledge through Confidence Calibration