LLMs Should Express Uncertainty Explicitly

Junyu Guo; Shangding Gu; Ming Jin; Costas Spanos; Javad Lavaei

cs.LGApr 7, 2026

LLMs Should Express Uncertainty Explicitly

Authors: Junyu Guo, Shangding Gu, Ming Jin, Costas Spanos, Javad Lavaei

Organizations: University of California, Berkeley · Virginia Tech

Abstract

Large language models (LLMs) often produce confident yet incorrect answers, which can lead to risky failures in real-world applications. We study whether post-training can make a model's self-assessment explicit: when the model is uncertain, can it be trained to signal so within its own response? A central design question is where in the response this signal should be exposed -- during reasoning, while the answer is still being formed, or at the end, once the answer has been produced. We study both. For end-of-reasoning self-assessment, we train the model to verbalize a confidence score for its response, with the aim of high confidence on correct answers and low confidence on incorrect ones. For during-reasoning self-assessment, we train the model to emit the marker <uncertain> whenever its current reasoning state appears unreliable. Across factual reasoning tasks, both forms sharply reduce overconfident errors while improving answer quality, and both can be used as triggers for retrieval augmented generation (RAG) to improve the final response. We further analyze their internal mechanisms: end-of-reasoning verbalized confidence sharpens a confidence-related structure already present in the pretrained model, whereas during-reasoning <uncertain> emission teaches the model to mark high-risk reasoning steps, with parameter changes concentrated in the model's late layers.

Figures & tables

Figure 1: We train LLMs to express uncertainty at two points in the response: during reasoning by emitting <uncertain> at risky steps, and after reasoning by verbalizing a confidence score for the final answer. Both signals can trigger retrieval or abstention, while our analysis suggests that confidence training sharpens an existing confidence-related structure and <uncertain> training teaches the model to mark high-risk reasoning states through late-layer changes.

Panel of Figure 2 — Figure 2: Logit-lens analysis of the confidence-token hidden state. Calibration sharpens late-layer confidence routing and yields a cleaner final-layer confidence structure.

Panel of Figure 3 — Figure 3: PCA analysis of the confidence-token hidden state, colored by verbalized confidence.

Error type	Base	Cal.
Epistemic	92.4%	34.9%
Aleatoric	7.6%	65.1%
Strict epi.	88.6%	3.9%

Table 4

Panel of Figure 6 — Figure 6: Layer-wise CKA between base and calibrated models. Verbalized confidence preserves representation geometry, whereas the <uncertain> marker induces increasing late-layer divergence.

A. Verbalized confidence						B. <uncertain> marker
Method	EM	F1	Brier	ECE	OConf	Method	Emit	Prec.	Recall	Acc ¬t	Wrong/Pos.
Base	24.5	37.3	-0.108	0.357	88.5	Emit heur.	0.336	0.959	0.444	0.392	0.726
P(True)	24.4	37.2	-0.096	0.340	39.7	Hidden probe	0.699	0.889	0.856	0.653	0.726
Global TS	24.5	37.3	+0.116	0.185	69.4	Output clf.	0.925	0.754	0.961	0.622	0.726
ATS	24.5	37.3	+0.123	0.166	53.4	Self-RAG	0.444	0.861	0.478	0.250	0.799
SFT-Conf	21.1	33.5	+0.083	0.226	7.3	FLARE	0.586	0.738	0.598	0.300	0.722

Table 3: Panel A evaluates verbalized confidence; OConf is the percentage of wrong answers with confidence

>0.5

. Panel B evaluates <uncertain> marker triggers; Prec. = P(wrong

\mid

trigger), Recall = P(trigger

\mid

wrong), and Acc ¬t is accuracy on untriggered examples.

	HotpotQA			MuSiQue			2WikiMultiHop			NQ			TriviaQA			Overall
Method	EM	F1	T	EM	F1	T	EM	F1	T	EM	F1	T	EM	F1	T	EM	F1	T
No-Ret	23.4	31.9	–	5.6	10.3	–	16.4	20.1	–	29.2	42.0	–	54.8	61.2	–	25.9	33.1	–
SR-7B	4.2	16.6	57.8	0.6	5.2	56.0	4.4	14.8	45.6	17.8	22.0	18.0	5.6	24.9	53.6	6.5	16.7	46.2
SR-13B	2.6	16.0	42.4	0.8	6.1	39.8	3.4	16.6	30.0	30.8	36.9	5.4	6.4	36.5	29.8	8.8	22.4	29.5
Ret-All	34.0	44.9	100	9.8	17.4	100	30.0	34.7	100	25.4	35.8	100	43.0	49.1	100	28.4	36.4	100
ADARAGUE	27.8	37.2	57.0	9.6	15.9	53.8	20.6	25.8	57.4	28.6	40.0	21.8	52.2	58.1	29.0	27.8	35.4	43.8

Table 4: Adaptive RAG evaluation results. EM, F1 (both %), and trigger rate T (%) are shown per dataset when available. Darker shaded cells indicate stronger EM/F1 performance within each metric column; yellow marks the highest selective trigger rate, excluding Ret-All .

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 7 — Figure 7: KL mass fractions by token type. These plots complement the main-text boxplots by showing total KL allocation rather than per-token enrichment. Because reasoning spans are much longer than uncertainty spans, raw mass fractions should be interpreted together with the per-token statistics in the main text.

Panel of Figure 8 — Figure 8: Hidden-state patching at the signal position versus reasoning positions. In both methods, patching the reasoning trace is at least as disruptive as patching the signal token itself, which is consistent with uncertainty being assembled across the trajectory and only exposed at the designated output position.

Panel of Figure 9 — Figure 9: Relative Frobenius weight drift across layers and module types. Both calibrated models show similar update structure in parameter space, with the largest changes in v_proj , o_proj , and MLP projection layers. This makes the difference in representation geometry especially noteworthy: similar magnitudes of weight drift yield very different geometric consequences.

Figure 10: Mechanism-to-behavior linkage for verbalized confidence. Localization-related features predict per-example confidence shifts with cross-validated

R^{2}=0.51

, indicating that the strength of the learned confidence mechanism varies meaningfully across examples rather than appearing only as a population-level average.

Figure 11: Mechanism-to-utility linkage for the <uncertain> marker. Under the current proxy utility target, the <uncertain> -marker model shows weak and unstable within-subset linkage. This likely reflects a mismatch between the current utility proxy and the model’s operative mechanism, which is more naturally framed as a binary emission decision than as graded variation within already-emitting examples.

Figure 12: Detailed binned routing view for verbalized confidence calibration. In the base model, correct answers and many wrong answers both terminate with dominant mass in the High confidence bin. After calibration, low-confidence errors are redirected away from High and into Low , while correct answers remain more conservative. This makes the main mechanism visually explicit: calibration sharpens the late-stage mapping from hidden states to confidence outputs rather than uniformly lowering confidence everywhere.

Panel of Figure 13 — Figure 13: Final-layer PCA of the confidence-token hidden state, grouped by outcome type. In the base model, correct and wrong low-confidence examples remain substantially mixed. After calibration, the representation aligns more cleanly with outcome structure: wrong low-confidence cases concentrate on the low-confidence side of the structure, while correct examples occupy the higher-confidence region more consistently.

Metric	Llama-3-8B (base)	Llama-3-8B (calibrated)
Accuracy ( $\uparrow$ )	0.345	0.358
Avg. verbalized confidence	0.869	0.403
Overconfidence gap (conf $-$ acc) ( $\downarrow$ )	+0.523	+0.045
ECE ( $\downarrow$ )	0.383	0.049
Brier score ( $\downarrow$ )	0.504	0.166
NLL (confidence) ( $\downarrow$ )	4.987	0.498

Appendix

Table 5: Calibration metrics for Llama-3-8B before and after calibration training.

Panel of Figure 14 — Figure 14: Training dynamics and calibration quality of the calibrated Llama-3-8B model.

		Accuracy		ECE		AUSC		Overconf.		Conf. on wrong
Dataset	$n$	Base	Cal.	Base	Cal.	Base	Cal.	Base	Cal.	Base	Cal.
2WikiMultihopQA	500	22.4	28.0	0.407	0.164	0.267	0.458	84.8	13.3	0.823	0.343
HotpotQA	500	35.7	35.6	0.459	0.085	0.477	0.536	92.2	4.0	0.854	0.293
MuSiQue	500	12.1	14.4	0.543	0.089	0.126	0.224	84.6	0.0	0.809	0.207
NQ	500	46.5	48.2	0.367	0.033	0.560	0.625	96.3	0.4	0.888	0.426
TriviaQA	500	63.9	68.4	0.242	0.211	0.749	0.838	94.4	0.6	0.866	0.329

Appendix

Table 6: Dataset-level summary for verbalized-confidence calibration. “Overconf.” denotes the fraction of wrong answers with confidence

>0.5

Table 7: Supplementary diagnostics for verbalized-confidence calibration.

	Accuracy			Answer Line		Emit Rate		Wrong / Correct + Emit
Dataset	Base	Calibrated	$\Delta$	Base	Calibrated	Base	Calibrated	W+E	C+E
2WikiMultihopQA	12.4	25.6	+13.2	51.4	100.0	43.2	59.8	43.2 / 55.4	0.0 / 4.4
HotpotQA	22.0	27.8	+5.8	62.8	99.6	36.6	70.8	35.8 / 59.0	0.8 / 11.8
MuSiQue	4.2	6.6	+2.4	50.8	100.0	49.8	94.6	49.8 / 89.0	0.0 / 5.6
NQ	21.6	40.8	+19.2	77.0	100.0	23.6	56.6	23.0 / 39.6	0.6 / 17.0
TriviaQA	36.2	56.0	+19.8	72.0	100.0	31.8	44.2	30.2 / 31.6	1.6 / 12.6

Appendix

Table 8: Base model vs. calibrated model on six factual reasoning datasets. “Answer Line” is the fraction of responses containing an explicit final answer line. “Emit Rate” is the overall <uncertain> emission rate. Each entry in the last two columns is Base / Calibrated : the fraction of wrong answers co-occurring with emission (W+E) and the fraction of correct answers co-occurring with emission (C+E). All values are percentages.

Table 9: Probe diagnostics for the <uncertain> marker.

Figure 15: Layer-wise probe performance for retrieval triggering.

A. Verbalized confidence reward ablation
Setting	Reward / format	Acc.	Brier	ECE	Overconf.	Gap
Base prompt	none, decimal	24.5	-0.337	0.647	89.4	+0.111
Final design	Brier, decimal	27.4	+0.110	0.133	3.0	+0.330
Format variant	Brier, integer-style	24.6	+0.043	0.234	25.5	+0.611
B. Local marker reward ablation
Setting	Reward order $\{\mathrm{C\bar{E}},\mathrm{CE},\mathrm{WE},\mathrm{W\bar{E}}\}$	Acc.	Emit	Rec.-err.	Sep.	Len.

Appendix

Table 10: Reward ablations for the two methods. Verbalized-confidence rows use a 500-example held-out set; <uncertain> -marker rows use a 1,100-example held-out set. Values are percentages except Brier, confidence gap, separation gap in percentage points, and response length. In Panel B,

\mathrm{C}

\mathrm{W}

denote correct/wrong and

\mathrm{E}

\bar{E}

denote marker emission/no emission.

A. Verbalized confidence
Family	Recipe	Acc.	Brier	ECE	Overconf.	Gap	Conf. support
Llama-3-8B	Brier GRPO	27.4	+0.110	0.133	3.0	+0.330	7 values
Qwen2.5-7B	Brier GRPO	21.1	+0.039	0.039	7.2	+0.636	8 values
B. <uncertain> marker
Family	Recipe	Acc.	Emit	Rec.-err.	Sep.	Mean len.	CNE / CE / WE / WNE
Llama-3.1-8B	marker GRPO	47.1	57.7	72.9	+32.7	478	27.9 / 19.2 / 38.5 / 14.4

Appendix

Table 11: Cross-family robustness results. Panel A uses the verbalized-confidence held-out set (

n=500

); Panel B uses the <uncertain> -marker held-out set (

n=1100

). Values are percentages except Brier, confidence gap, separation gap in percentage points, and mean response length. CNE, CE, WE, and WNE denote correct/no-emission, correct/emission, wrong/emission, and wrong/no-emission.

Panel of Figure 16 — Figure 16: Qualitative examples of epistemic and aleatoric errors before and after GRPO calibration training. Epistemic errors (Examples 1–2) are wrong answers delivered with high stated confidence and no uncertainty signal in the response text; they represent knowledge gaps the model fails to recognise. Aleatoric errors (Example 3) are wrong answers accompanied by explicit hedging in the reasoning, indicating the model correctly identifies the limits of its knowledge. Error type is determined by the response content independently of the confidence number, motivating the use of an LLM judge (see § F.1 ) in addition to the verbalized confidence threshold.

Panel of Figure 17 — Figure 17: Qualitative examples for the <uncertain> marker, which signals uncertainty via an explicit reasoning-time marker rather than a verbalized confidence score. Epistemic errors (Examples 1–2) occur when the model produces a wrong answer without emitting <uncertain> in the reasoning chain; the baseline (Example 1) never uses the token, while the trained model (Example 2) still fails to trigger it for short, over-confident responses. Aleatoric errors (Example 3) occur when <uncertain> is correctly inserted at the knowledge boundary, signalling that the error is not a hallucination but an honest absence of information. Unlike the verbalized confidence setting (Fig. 16 ), the uncertainty signal here is binary and localised : the marker either appears or it does not, and its position in the chain-of-thought identifies the specific reasoning step that fails.

Language Model Self-Assessment

Language model self-assessment studies how models judge their own confidence, capabilities, errors, and behavior.

See all 89 papers

Beyond "I Don't Know": Evaluating LLM Self-Awareness in Discriminating Data and Model Uncertainty
Apr 19, 2026Jingyi Ren, Ante Wang, Yunghwei Lai +5LLM Evaluation Selective Prediction
Dept. of Comp. Sci. & Tech., Institute for AI, Tsinghua University, Beijing, China · Institute for AI Industry Research (AIR), Tsinghua University, Beijing, China
Confidence Before Answering: A Paradigm Shift for Efficient LLM Uncertainty Estimation
Mar 6, 2026Changcheng Li, Jiancan Wu, Hengheng Zhang +5Confidence Estimation in Language Models LLM Uncertainty Estimation
University of Science and Technology of China · Huawei Inc.
Clustered Self-Assessment: A Simple yet Effective Method for Uncertainty Quantification in Large Language Models
Jun 2, 2026Qi Cao, Takeshi Kojima, Andrew Gambardella +3LLM Evaluation Uncertainty Quantification
The University of Tokyo, Japan

Apr 19, 2026J/K move · Enter open · S save

Beyond "I Don't Know": Evaluating LLM Self-Awareness in Discriminating Data and Model Uncertainty

Jingyi Ren, Ante Wang, Yunghwei Lai, Xiaolong Wang+4

Dept. of Comp. Sci. & Tech., Institute for AI, Tsinghua University, Beijing, China · Institute for AI Industry Research (AIR), Tsinghua University, Beijing, China

LLM Evaluation Selective Prediction Confidence Estimation in Language Models Language Model Self-Assessment LLM Reliability LLM Uncertainty Estimation

Open paper AskPDF arXiv

Reliable Large Language Models (LLMs) should abstain when confidence is insufficient. However, prior studies often treat refusal as a generic "I don't know'', failing to distinguish input-level ambiguity (data uncertainty) from capability limitations (model uncertainty). This lack of distinction limits downstream action decisions like requesting clarification or invoking external tools. In this work, we introduce UA-Bench, a benchmark of over 3,500 questions drawn from six datasets spanning knowledge-intensive and reasoning-intensive tasks, designed to evaluate explicit uncertainty attribution. An evaluation of 18 frontier LLMs shows that even state-of-the-art models struggle to reliably discriminate between data uncertainty and model uncertainty, and that high answer accuracy does not necessarily imply strong uncertainty attribution ability. To narrow this gap, we propose a lightweight data synthesis and reinforcement learning strategy. Experiments on both Qwen3-4B-Instruct-2507 and Qwen3-8B in thinking mode show that the proposed method improves uncertainty attribution while preserving answer accuracy. Our code and data are publicly available now.

Abstract

Language Model Self-Assessment

Explore similar work

Beyond "I Don't Know": Evaluating LLM Self-Awareness in Discriminating Data and Model Uncertainty

Confidence Before Answering: A Paradigm Shift for Efficient LLM Uncertainty Estimation

Clustered Self-Assessment: A Simple yet Effective Method for Uncertainty Quantification in Large Language Models