The Stepwise Informativeness Assumption: When Do Entropy Dynamics Track Reasoning Correctness in LLMs?

Mar Gonzàlez I Català; Haitz Sáez de Ocáriz Borde; George D. Montañez; Pietro Liò

cs.CLMar 11, 2026

The Stepwise Informativeness Assumption: When Do Entropy Dynamics Track Reasoning Correctness in LLMs?

Authors: Mar Gonzàlez I Català, Haitz Sáez de Ocáriz Borde, George D. Montañez, Pietro Liò

Organizations: University of Cambridge · University of Oxford · Harvey Mudd College

Abstract

Recent work uses entropy-based signals at multiple representation levels to study reasoning in large language models, but the field remains largely empirical. From a theoretical perspective, reductions in model uncertainty do not necessarily reflect progress toward correctness. In this work, we characterize when such alignment can be expected through the Stepwise Informativeness Assumption (SIA), which requires entropy-reducing reasoning-prefix extensions increase the probability assigned to valid answers in expectation. We show that next-token prediction can transfer answer-relevant information from training data to the model, but does not, by itself, guarantee alignment with correctness. Instead, we propose that this alignment depends on how the training regime couples reasoning traces to correct answers. Furthermore, we hypothesize observable signatures of this phenomenon and characterize its expected failure regimes. We empirically investigate how entropy dynamics relate to correctness across four reasoning benchmarks (GSM8K, ARC, SVAMP, MATH) and diverse open-weight language models spanning base, instruction-tuned, and RL-trained regimes. Our results show that entropy-correctness alignment strengthens with post-training but can weaken when task difficulty exceeds model capabilities.

Figures & tables

Figure 1: Overview of the proposed framework. We formalize the conditions for entropy-correctness alignment and investigate how different training stages may promote it.

Model	Training	GSM8K	SVAMP	ARC	MATH
Qwen2.5-3B	Base	0.682 [0.649, 0.714]	0.603 [0.512, 0.681]	0.344 [0.300, 0.386]	-0.0975 [-0.1939, -0.0042]
Qwen2.5-3B-it	SFT	0.744 [0.717, 0.770]	0.835 [0.771, 0.893]	0.666 [0.633, 0.701]	0.0082 [-0.0979, 0.1146]
Qwen2.5-Math-1.5B	SFT	0.499 [0.429, 0.567]	0.802 [0.643, 0.922]	0.701 [0.654, 0.747]	-0.5411 [-0.6248, -0.4579]
DeepSeek-Chat-7B	SFT	0.346 [0.305, 0.388]	0.295 [0.190, 0.414]	0.143 [0.090, 0.199]	-0.1277 [-0.1879, -0.0703]
DeepSeek-R1-Distilled	SFT+RL	0.795 [0.763, 0.829]	0.593 [0.436, 0.730]	0.783 [0.744, 0.817]	-0.7746 [-0.8462, -0.6865]
Gemma-2-2B	Base	-0.530 [-0.689, -0.346]	0.169 [-0.489, 0.844]	-0.208 [-0.266, -0.151]	-0.5595 [-0.7502, -0.3559]

Table 1: Training aligns entropy descent with the true answer. We report the correlation between conditional answer entropy and gold surprisal along each trace (SIA alignment coefficient

\rho_{\mathrm{SIA}}

), averaged by model and dataset. Negative or near-zero values indicate failure of alignment.

Panel of Figure 2 — Figure 2: Observable signatures of entropy-correctness alignment on GSM8K. Correct traces accumulate answer-relevant information earlier and conditional answer entropy separates correct from incorrect traces before answer generation. Further results can be found in Appendix C .

Appendix figures & tables22 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 3 — Figure 3: Early information accumulation for aligned model-dataset pairs on GSM8K. Normalized cumulative information gain $G(s)$ versus relative prefix length $s$ , split by correctness. Each panel corresponds to one model. Shaded regions denote 95% bootstrap confidence intervals.

Panel of Figure 4 — Figure 4: Early information accumulation for aligned model-dataset pairs on SVAMP. Normalized cumulative information gain $G(s)$ versus relative prefix length $s$ , split by correctness. Each panel corresponds to one model. Shaded regions denote 95% bootstrap confidence intervals.

Panel of Figure 5 — Figure 5: Early information accumulation for aligned model-dataset pairs on ARC. Normalized cumulative information gain $G(s)$ versus relative prefix length $s$ , split by correctness. Each panel corresponds to one model. Shaded regions denote 95% bootstrap confidence intervals.

Panel of Figure 6 — Figure 6: Early information accumulation for aligned model-dataset pairs on MATH. Normalized cumulative information gain $G(s)$ versus relative prefix length $s$ , split by correctness. Each panel corresponds to one model. Shaded regions denote 95% bootstrap confidence intervals.

Panel of Figure 7 — Figure 7: Early information accumulation for non-aligned model-dataset pairs on GSM8K. Normalized cumulative information gain $G(s)$ versus relative prefix length $s$ , split by correctness. Each panel corresponds to one model. Shaded regions denote 95% bootstrap confidence intervals.

Panel of Figure 8 — Figure 8: Early information accumulation for non-aligned model-dataset pairs on SVAMP. Normalized cumulative information gain $G(s)$ versus relative prefix length $s$ , split by correctness. Each panel corresponds to one model. Shaded regions denote 95% bootstrap confidence intervals.

Panel of Figure 9 — Figure 9: Early information accumulation for non-aligned model-dataset pairs on ARC. Normalized cumulative information gain $G(s)$ versus relative prefix length $s$ , split by correctness. Each panel corresponds to one model. Shaded regions denote 95% bootstrap confidence intervals.

Panel of Figure 10 — Figure 10: Early information accumulation for non-aligned model-dataset pairs on MATH. Normalized cumulative information gain $G(s)$ versus relative prefix length $s$ , split by correctness. Each panel corresponds to one model. Shaded regions denote 95% bootstrap confidence intervals.

Figure 11: Early separability for aligned model-dataset pairs on GSM8K. AUC for using conditional answer entropy to distinguish correct from incorrect traces as a function of relative prefix length $s$ .

Figure 12: Early separability for aligned model-dataset pairs on SVAMP. AUC for using conditional answer entropy to distinguish correct from incorrect traces as a function of relative prefix length $s$ .

Figure 13: Early separability for aligned model-dataset pairs on ARC. AUC for using conditional answer entropy to distinguish correct from incorrect traces as a function of relative prefix length $s$ .

Figure 14: Early separability for aligned model-dataset pairs on MATH. AUC for using conditional answer entropy to distinguish correct from incorrect traces as a function of relative prefix length $s$ .

Figure 15: Early separability for non-aligned model-dataset pairs on GSM8K. AUC for using conditional answer entropy to distinguish correct from incorrect traces as a function of relative prefix length $s$ .

Figure 16: Early separability for non-aligned model-dataset pairs on SVAMP. AUC for using conditional answer entropy to distinguish correct from incorrect traces as a function of relative prefix length $s$ .

Figure 17: Early separability for non-aligned model-dataset pairs on ARC. AUC for using conditional answer entropy to distinguish correct from incorrect traces as a function of relative prefix length $s$ .

Figure 18: Early separability for non-aligned model-dataset pairs on MATH. AUC for using conditional answer entropy to distinguish correct from incorrect traces as a function of relative prefix length $s$ .

Figure 19: LoRA SFT induces entropy-correctness alignment. We fine-tune Gemma-2-2B base on GSM8K chain-of-thought solutions using LoRA and evaluate the SIA alignment coefficient

\rho_{\mathrm{SIA}}

at intermediate checkpoints. The coefficient increases from negative values early in training to consistently positive values after supervised fine-tuning, indicating that reasoning supervision causally strengthens the coupling between conditional answer entropy reduction and increasing probability of the gold answer. Shaded regions denote 95% bootstrap confidence intervals.

Figure 20: Scaling trends in entropy-correctness alignment. Mean

\rho_{\mathrm{SIA}}

across datasets is plotted against model size for base and instruction-tuned variants. Larger models generally exhibit stronger entropy-correctness alignment, and instruction-tuned models consistently outperform base models within the same family. This suggests that model scale and reasoning-oriented supervision jointly determine whether entropy descent becomes aligned with the true answer.

Model	Original mean	Shuffled mean
Qwen2.5-3B	0.682	-0.132
Qwen2.5-3B-it	0.744	-0.005
DeepSeek-R1-Distilled	0.795	0.020
Gemma-2-2B-it	0.522	-0.063

Appendix

Table 2: Shuffle-prefix ablation . Entropy-correctness alignment (

\rho_{\text{SIA}}

) drops sharply when prefix tokens are permuted. Negative or near-zero values indicate coupling misalignment.

Model	Original mean	Ablated mean
Qwen2.5-3B	0.682	0.635
Qwen2.5-3B-it	0.744	0.831
DeepSeek-R1-Distilled	0.795	0.711
Gemma-2-2B-it	0.522	0.506

Appendix

Table 3: Monte-Carlo ablation. Entropy-correctness alignment (

\rho_{\text{SIA}}

) remains qualitatively stable across different fidelity Monte-Carlo approximations.

Model	$T=0.2$	$T=0.4$	$T=0.6$	$T=0.7$	$T=0.8$	$T=1.0$
Qwen2.5-3B	0.644	0.689	0.752	0.682	0.599	0.652
Qwen2.5-3B-it	0.762	0.733	0.905	0.744	0.756	0.821
DeepSeek-R1-Distilled	0.837	0.843	0.744	0.795	0.686	0.781
Gemma-2-2B-it	0.616	0.385	0.543	0.522	0.417	0.613

Appendix

Table 4: Temperature ablation . Entropy-correctness alignment (

\rho_{\text{SIA}}

) remains qualitatively stable across decoding temperatures.

Artifact	Source	License / terms
GSM8K	OpenAI	MIT License
ARC	AllenAI AI2 ARC	CC BY-SA 4.0
SVAMP	Patel et al. / HF mirror	MIT License
MATH	Hendrycks et al.	MIT License
Gemma-2-2B, Gemma-2-9B	Google	Gemma Terms of Use
LLaMA-3.2-3B	Meta	Llama 3.2 Community License

Appendix

Table 5: Licenses and usage terms for datasets and open-weight model checkpoints used in our experiments.

LLM Reasoning

Large language model reasoning studies how models solve multi-step problems, verify intermediate steps, and use computation or tools reliably.

See all 926 papers

When Should LLMs Reason? Adaptive Inference Routing via Entropy Dynamics
May 20, 2026Wei Xia, Haoqing Wang, Zhi-Hong Deng +1Language Model Decoding CoT Reasoning
State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University · Samsung Research, Beijing, China
Answer-Distribution Trajectories: A Stochastic-Dynamics View of LLM Reasoning
Sep 8, 2026Mar Gonzàlez I Català, Haitz Sáez de Ocáriz Borde, Davide Murari +3LLM Uncertainty Estimation LLM Reasoning
University of Cambridge · Harvey Mudd College
InfoDensity: Rewarding Information-Dense Traces for Efficient Reasoning
Mar 18, 2026Chengwei Wei, Jung-jae Kim, Longyin Zhang +2Efficient Language Model Reasoning RL for Language Model Reasoning
Institute for Infocomm Research (I2R), A*STAR, Singapore · Centre for Frontier AI Research (CFAR), A*STAR, Singapore

May 20, 2026J/K move · Enter open · S save

When Should LLMs Reason? Adaptive Inference Routing via Entropy Dynamics

Wei Xia, Haoqing Wang, Zhi-Hong Deng, Yehui Tang

State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University · Samsung Research, Beijing, China

Language Model Decoding CoT Reasoning Efficient Language Model Reasoning Efficient Language Model Inference LLM Reasoning

Open paper AskPDF arXiv

Chain-of-thought (CoT) reasoning can improve LLM performance, but its benefits vary across tasks, models, and individual queries. On some tasks, CoT provides marginal or negative accuracy gains while substantially increasing token consumption. We investigate whether early decoding dynamics can help identify when explicit reasoning is useful. Our analysis shows that tasks with larger CoT gains tend to exhibit decreasing early-stage entropy, while low-gain tasks more often show oscillatory or increasing trajectories. These patterns motivate an empirical regime interpretation of decoding behavior. We propose EDRM (Entropy Dynamics-based Reasoning Manifold), a lightweight routing framework whose default heuristic requires no router training. EDRM summarizes early entropy trajectories with a compact, interpretable representation and selects among Direct, Standard, and CoT decoding. Across 15 benchmarks and 7 LLMs, its global and instance-level variants offer 16--55% token savings at comparable accuracy. The Full instance-level configuration combines routing with a short Direct fallback branch; our heuristic-router ablation reports up to 4.9 percentage points of additional accuracy from oracle answer selection over routing without fallback. These results support selective reasoning based on early decoding signals and demonstrate the potential of entropy-based control for efficient LLM inference.

Abstract

LLM Reasoning

Explore similar work

When Should LLMs Reason? Adaptive Inference Routing via Entropy Dynamics

Answer-Distribution Trajectories: A Stochastic-Dynamics View of LLM Reasoning

InfoDensity: Rewarding Information-Dense Traces for Efficient Reasoning