Understanding Reasoning in LLMs through Strategic Information Allocation under Uncertainty

Jeonghye Kim; Xufang Luo; Minbeom Kim; Sangmook Lee; Dongsheng Li; Yuqing Yang

cs.AIMar 16, 2026

Understanding Reasoning in LLMs through Strategic Information Allocation under Uncertainty

Authors: Jeonghye Kim, Xufang Luo, Minbeom Kim, Sangmook Lee, Dongsheng Li, Yuqing Yang

Organizations: Microsoft Research · KAIST · Seoul National University

Abstract

LLMs often exhibit Aha moments such as self-correction after tokens like "Wait," yet the underlying mechanism remains unclear. Standard LLMs collapse mainly through silent divergence, where trajectories drift from the correct answer yet remain locally coherent, so no explicit error triggers reactive self-correction. We introduce an information-theoretic framework that separates reasoning into procedural advancement and epistemic verbalization, the token-level externalization of uncertainty, and prove that sporadic verbalization restores convergence toward the correct answer even without explicit error triggers. Empirically, a minimal doubt cue recovers failed trajectories, and small-scale SFT suffices to instill or suppress this capability, suggesting that strong reasoning hinges less on an extraordinary inner mechanism than on the linguistic habit of externalizing uncertainty. Our framework recasts reasoning as strategic information allocation under uncertainty, offering a new lens for understanding and advancing LLM reasoning.

Figures & tables

Figure 1: Two pathways for self-correction. The Procedural channel reacts only when an explicit violation triggers an alarm, whereas the Epistemic channel continuously monitors and prompts self-checking even without any visible contradiction.

Figure 2: Reasoning collapse analysis across models and benchmarks. (a) Collapse rate among incorrect responses per model and dataset. (b) Breakdown of collapse types per model. Model names omit the “Qwen” prefix (e.g., 2.5-7B refers to Qwen2.5-7B).

Type	Model	Reactive	Proactive
		$n$ (%)	$n$ (%)
LLM	Qwen3-1.7B-Base †	12 (100.0)	0 (0.0)
	Qwen3-8B-Base †	19 (86.4)	3 (13.6)
	Qwen3-14B-Base †	23 (92.0)	2 (8.0)
	Qwen2.5-7B †	11 (100.0)	0 (0.0)
	Qwen2.5-Math-7B †	35 (79.5)	9 (20.5)

Table 1: Reactive and proactive self-correction counts across models. For each generation, a GPT-5-based filter flags traces with a wrong path

\rightarrow

correction

\rightarrow

correct answer pattern;

n

is the number of such traces per model. Proactive correction is largely absent in LLMs and emerges consistently in LRMs. † Small sample (

N<50

). Details can be found in Appendix D.2 .

Figure 3: Capability map across model families. The

x

-axis reports direct fluency , the share of responses answered correctly without invoking any self-correction, computed as

(\text{correct}-\text{SC})/N

. The

y

-axis reports the proactive correction rate , the share of responses containing a suspicion-based self-correction, i.e., a correction triggered without explicit evidence of error.

Model	Precision (%)
DeepSeek-R1-Distill-Qwen-7B	37.3
DeepSeek-R1-Distill-Qwen-32B	23.9
Qwen3-8B	20.5
Qwen3-14B	15.8

Table 2: Precision of proactive self-correction signals across LRMs. Precision is the fraction of suspicion-based corrections issued on genuinely incorrect trajectories, as judged by GPT-5 (See Appendix D.2 for details). The remainder second-guess already-correct chains.

#	Example
1	Wait, is that correct?
2	Wait, 2023 is 7 multiplied by 17 squared, right?
3	Wait, maybe $f(n)$ is related to the Möbius function but scaled differently.
4	Wait, perhaps I can write it as $(f*(n/d))(n)=1$ , but that doesn’t seem helpful.

Table 3: Examples of epistemic verbalizations co-occurring with “Wait” in reasoning traces from DeepSeek-R1-Distill-Qwen-1.5B/7B/32B.

Condition	Phrase
None	(no injection)
Hmm-Doubt	“Hmm, I’m not sure this is right.”
Hmm-Doubt-Revisit	“Hmm, I’m not sure this is right. Let me reconsider.”
Wait-Doubt	“Wait, is that correct?”
Wait-Revisit	“Wait, let me double-check.”

Table 4: Injection conditions. Four epistemic phrases span two phrase types ( Hmm , Wait ) and vary in whether they express doubt alone, signal an intent to revisit, or both. None controls for the effect of mere re-sampling from the truncated prefix.

Figure 4: Recovery rate vs. truncation fraction

\alpha

across injection conditions and benchmarks, for Qwen3-8B-Base (top) and Qwen3-14B-Base (bottom).

Figure 5: Comparison of avg@8 between baseline inference and inference with epistemic tokens suppressed on DeepSeek-R1-Distill-Qwen-14B/32B.

	Base	SFT (no EV)
Qwen2.5-7B	13.3	6.7
Qwen3-8B-Base	16.7	3.3
Qwen3-14B-Base	16.7	10.0
DeepSeek-R1-Distill-7B	50.0	30.0
DeepSeek-R1-Distill-32B	80.0	43.3

Table 5: AIME24 pass@1 across base models and SFT variants trained on traces with epistemic verbalization (EV) suppressed.

Figure 6: Per-sample epistemic token counts in LIMO. "Wait" alone appears 77 times per response on average.

Panel of Figure 7 — Figure 7: Comparison of AIME24 pass@1 scores between base models and models trained with SFT on the LIMO-v2 dataset.

Figure 8: Distributions of token-level log-probability and entropy for all tokens versus the subset of "Wait" and "Alternative" tokens. Vertical blue and red lines indicate the dataset-level averages for "Wait" and "Alternative" tokens, respectively.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 9 — Figure 9: Token-level entropy over reasoning steps for Qwen2.5-Math-7B and Qwen3-14B-Base on AIME24 decreases similarly in both correct and incorrect solutions, suggesting that entropy alone does not reliably reflect uncertainty toward the correct answer.

Model	#Inc.	Err. Rate	#Col.	Col./Inc.
Qwen2.5-7B	2,568	53.5%	1,452	56.5%
Qwen2.5-32B	1,202	25.0%	717	59.7%
Qwen3-1.7B-Base	2,486	51.8%	1,534	61.7%
Qwen3-8B-Base	1,987	41.4%	1,091	54.9%
Qwen3-14B-Base	1,401	29.2%	839	59.9%
Total	9,644	40.2%	5,633	58.4%

Appendix

Table 6: Reasoning collapse statistics. Each model is evaluated on

600

problems with

8

sampled responses. "Inc." = incorrect responses, "Col." = collapsed responses.

Temperature (Top-P)	0.0	0.7 (1.0)	0.7 (0.8)
DeepSeek-R1-Distill-32B	50.0	70.0	80.0
DeepSeek-R1-Distill-14B	56.67	56.67	70.0
DeepSeek-R1-Distill-1.5B	20.0	30.0	30.0
DeepSeek-Math-7B-Instruct	3.33	0.0	0.0
Qwen2.5-Math-7B	16.67	10.0	6.67
Qwen2.5-7B	13.33	6.67	3.33

Appendix

Table 7: AIME24 pass@1 performance across various base models with varying temperature and top-p configurations.

Figure 10: On AIME24 #7, both models initially fail, but only Qwen3-8B-SFT sustains information gain through epistemic verbalization and recovers the correct answer.

Figure 11: Token-level MI analysis: high MI aligns with evaluative behavior, not with the thinking tokens themselves.

Figure 12: Acc@16 (avg. score) and Len@16 (avg. response length) of DeepSeek-Distill 1.5B–14B on math benchmarks.

Panel of Figure 13 — Figure 13: Token occurrence counts for DeepSeek-R1-Distill-Qwen-{1.5B, 7B, 14B} models on the AIME24, AIME25, AMC23, and MATH500 benchmarks.

Panel of Figure 14 — Figure 14: Token occurrence counts for Qwen3-{4B, 8B, 14B}-Base models on the AIME24, AIME25, AMC23, and MATH500 benchmarks.

Baseline surface form	Suppression substitute
Wait, let me check…	But hold on, let me…
Hmm,	No, that’s not quite right / Hold on, no
Maybe / Perhaps	It’s possible that… / Suppose…
Alternatively	Another way to see this
Actually	I realize / On closer look
Let me check	Let me verify

Appendix

Table 8: Bypass patterns observed when the epistemic tokens are suppressed during generation.

Model	Preceding computation	Trigger verbalization
LLM — reactive correction only
Qwen2.5-32B-Instruct	Diagonal midpoint equation yields $x=6$ , which explicitly violates the given constraint $x>7$ .	“However, we are given that $x>7$ . Therefore, we need to re-evaluate the vertices.”
Qwen2.5-7B	Substitution gives $y=\tfrac{5}{2z}\big\|_{z=5/2}=1$ , violating the condition $y>1$ .	“We made an error in the interpretation of the equations.”
Qwen2.5-Math-7B	solve(...) raises TypeError: list indices must be integers or slices, not Symbol .	“It seems there was an issue with how the solution was being accessed.”
LRM — reactive correction
DeepSeek-R1-Distill-Qwen-32B	$AP=\dfrac{2025-4225}{286}=\dfrac{-2200}{286}<0$	“Wait, negative length? That can’t be.”

Appendix

Table 9: Representative examples of reactive and proactive correction in LLMs and LRMs. LLMs exhibit predominantly reactive correction, triggered by explicit computational failures or contradiction. LRMs additionally exhibit proactive correction, where doubt arises without any surfaced error.

LLM Reasoning

Large language model reasoning studies how models solve multi-step problems, verify intermediate steps, and use computation or tools reliably.

See all 926 papers

LLMs Should Express Uncertainty Explicitly
Apr 7, 2026Junyu Guo, Shangding Gu, Ming Jin +2Language Model Self-Assessment LLM Uncertainty Estimation
University of California, Berkeley · Virginia Tech
Uncertainty Localization in LLM Reasoning via Embedding Perturbations
Feb 2, 2026Qihao Wen, Jiahao Wang, Yang Nan +3LLM Reliability LLM Uncertainty Estimation
University of Arizona · Michigan State University
Dissecting Failure Dynamics in Large Language Model Reasoning
Apr 16, 2026Wei Zhu, Jian Zhang, Lixing Yu +2Inference-Time Guidance LLM Reasoning
School of Information Science and Engineering, Yunnan University, Kunming, China · Yunnan Key Laboratory of Intelligent Systems and Computing, Kunming, China

Apr 7, 2026J/K move · Enter open · S save

LLMs Should Express Uncertainty Explicitly

Junyu Guo, Shangding Gu, Ming Jin, Costas Spanos+1

University of California, Berkeley · Virginia Tech

Language Model Self-Assessment LLM Uncertainty Estimation Metacognition in Language Models

Open paper AskPDF arXiv

Large language models (LLMs) often produce confident yet incorrect answers, which can lead to risky failures in real-world applications. We study whether post-training can make a model's self-assessment explicit: when the model is uncertain, can it be trained to signal so within its own response? A central design question is where in the response this signal should be exposed -- during reasoning, while the answer is still being formed, or at the end, once the answer has been produced. We study both. For end-of-reasoning self-assessment, we train the model to verbalize a confidence score for its response, with the aim of high confidence on correct answers and low confidence on incorrect ones. For during-reasoning self-assessment, we train the model to emit the marker <uncertain> whenever its current reasoning state appears unreliable. Across factual reasoning tasks, both forms sharply reduce overconfident errors while improving answer quality, and both can be used as triggers for retrieval augmented generation (RAG) to improve the final response. We further analyze their internal mechanisms: end-of-reasoning verbalized confidence sharpens a confidence-related structure already present in the pretrained model, whereas during-reasoning <uncertain> emission teaches the model to mark high-risk reasoning steps, with parameter changes concentrated in the model's late layers.

Abstract

LLM Reasoning

Explore similar work

LLMs Should Express Uncertainty Explicitly

Uncertainty Localization in LLM Reasoning via Embedding Perturbations

Dissecting Failure Dynamics in Large Language Model Reasoning