Entropy trajectory shape predicts LLM reasoning reliability: A diagnostic study of uncertainty dynamics in chain-of-thought

Xinghao Zhao

cs.CLMar 19, 2026

Entropy trajectory shape predicts LLM reasoning reliability: A diagnostic study of uncertainty dynamics in chain-of-thought

Authors: Xinghao Zhao

Organizations: Huazhong University of Science and Technology

Abstract

Understanding uncertainty in chain-of-thought reasoning is critical for reliable deployment of large language models. In this work, we propose a simple yet effective diagnostic approach based on trajectory shape rather than scalar magnitude. We show that this signal is practical, interpretable, and inexpensive to obtain in black-box settings, while remaining robust across models and datasets. Through extensive ablations and cross-domain replications, we demonstrate its utility for selective prediction and triage. Our findings offer a generalizable insight into uncertainty dynamics in reasoning tasks, with particular focus on numeric and discrete-answer settings.

Figures & tables

Panel of Figure 1 — Figure 1: Two key findings from our diagnostic study. (a) Token log-probability confidence becomes increasingly miscalibrated at later reasoning steps. (b) Chains with monotone entropy trajectories achieve substantially higher accuracy than non-monotone chains.

Confidence Proxy	ECE $\downarrow$	AUROC $\uparrow$	$\rho$	Conf. Range
$\sigma(\bar{\ell}+1.5)$ (sigmoid-shifted)	0.186	—	$+0.166^{**}$	$[0.802,0.818]$
$\sigma(\bar{\ell})$ (sigmoid-unshifted)	0.133	—	$+0.166^{**}$	$[0.475,0.500]$
$\bar{\ell}$ (raw-logprob, normalized)	0.265	—	$+0.166^{**}$	$[0.990,1.000]$
$\exp(\bar{\ell})$ (exp-logprob)	0.254	—	$+0.166^{**}$	$[0.906,1.000]$
Majority-vote rate (answer-dist)	0.251	0.547	$+0.084$	$[0.144,0.749]$

Table 1: Confidence proxies for step-0 correctness prediction on the pilot GSM8K split (

n{=}300

). ECE: Expected Calibration Error; AUROC: area under ROC;

\rho

: Spearman correlation with final answer correctness. All token log-probability proxies produce identical Spearman rankings and collapsed confidence ranges. Larger-scale GSM8K and MATH-500 robustness results are reported in Sections 4 and A.16 .

Subset	$n$	Accuracy	95% CI	Fisher’s $p$ / OR
All chains	300	$63.0\%$	—	—
Monotone chains	221	$68.8\%$	$[63.0,74.7]$	$p{=}0.0005$ / OR $=2.50$
Non-monotone chains	79	$46.8\%$	$[35.7,57.9]$	—
Accuracy gap (monotone $-$ non-monotone)			$+21.9$ pp	$[+9.5,+34.3]$

Table 2: Accuracy by entropy-trajectory monotonicity on the pilot GSM8K split (

n{=}300

, Qwen2.5-7B-Instruct). CI is 95% bootstrap;

p

is Fisher’s exact test. Full-scale GSM8K (

n{=}1319

) and MATH-500 (

n{=}500

) robustness is reported in text.

Figure 2: Example per-step answer-distribution entropy trajectories. Left: A monotone trajectory (each step reduces

H_{k}

), corresponding to a correct final answer. Right: A non-monotone trajectory with a mid-chain entropy spike, corresponding to an incorrect answer.

H_{k}

is defined in Equation 1 .

Signal	Acc@ $73.7\%$	$\Delta$	Spearman $\rho$	AURC
Random	$62.9\%$	$-0.1$ pp	$0.000$	$0.398$
Chain length (shorter first)	$65.6\%$	$+2.6$ pp	$+0.101$	$0.310$
Final-step entropy $H_{N}$	$65.2\%$	$+2.2$ pp	$+0.093$	$0.313$
Scalar coherence $H_{0}{-}H_{N}$	$62.4\%$	$-0.6$ pp	$-0.059$	$0.408$
Self-judgment (1 verifier call)	$62.4\%$	$-0.6$ pp	$+0.019$	$0.368$
Yes/No self-eval ( $P(\text{Yes})$ )	$63.3\%$	$+0.3$ pp	$-0.019$	$0.395$

Table 3: Cheap reliability signals on the pilot GSM8K split (

n{=}300

). Coverage fixed at

73.7\%

(

k{=}221

). AURC: area under risk-coverage curve (lower is better; random

\approx 0.398

, oracle

=0.068

). ∗ :

p{<}0.05

; ∗∗ :

p{<}0.01

Figure 3: Accuracy vs. coverage for five cheap reliability signals. Entropy-trajectory monotonicity (solid blue) dominates all scalar baselines. Scalar coherence (dotted, AURC

=0.408

) is below the random baseline. Dashed line: full-coverage accuracy (

63.0\%

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

$\tau$	$n$	Mono. rate	Mono. acc / Non-mono. acc	Gap
$0.3$	$300$	$0.697$	$0.703$ / $0.473$	$+23.1$ pp
$0.5$	$300$	$0.707$	$0.689$ / $0.523$	$+16.6$ pp
$0.7$	$300$	$0.737$	$0.688$ / $0.469$	$+21.9$ pp
$1.0$	$300$	$0.723$	$0.687$ / $0.542$	$+14.4$ pp

Appendix

Table 4: Effect of sampling temperature

\tau

on the entropy-monotonicity signal. Lower

\tau

underestimates uncertainty (more deterministic completions); higher

\tau

adds noise unrelated to model uncertainty. Gap = monotone

-

non-monotone accuracy.

$m$	$n$	Mono. rate	Mono. acc / Non-mono. acc	Gap
$3$	$300$	$0.843$	$0.672$ / $0.447$	$+22.5$ pp
$5$	$300$	$0.737$	$0.688$ / $0.469$	$+21.9$ pp
$10$	$212$	$0.698$	$0.743$ / $0.531$	$+21.2$ pp

Appendix

Table 5: Effect of number of per-step samples

m

on the entropy-monotonicity signal. Monotone and non-monotone accuracy are at

\varepsilon{=}0.01

. Gap = monotone

-

non-monotone accuracy. The

m{=}10

run used

n{=}212

because the

m{=}3

and

m{=}10

experiments ran concurrently on the same GPU; occasional CUDA memory errors at long steps excluded some problems.

Prefix rule	$n$	Coverage	Mono. acc	Non-mono. acc	Gap	Cost ratio
$k{=}1$	300	$91.0\%$	$64.1\%$	$51.9\%$	$+12.3$ pp	$0.32$
$k{=}2$	290	$82.4\%$	$65.7\%$	$49.0\%$	$+16.7$ pp	$0.60$
$k{=}3$	225	$72.4\%$	$63.8\%$	$50.0\%$	$+13.8$ pp	$0.73$
Full (all)	300	$73.7\%$	$68.8\%$	$46.8\%$	$+21.9$ pp	$1.00$
Full (same $n$ as $k{=}2$ )	290	$72.8\%$	$68.7\%$	$46.8\%$	$+21.9$ pp	$1.00$

Appendix

Table 6: Prefix monotonicity on GSM8K from figures/prefix_results.json . Gap = monotone

-

non-monotone accuracy. “Full (same

n

k{=}2

)” controls for subset shift.

Step $k$	$n_{k}$	ECE	95% CI
0	300	0.186	—
1	300	0.186	—
2	288	0.190	—
3	241	0.218	—
4	175	0.222	—
5	118	0.232	—

Appendix

Table 7: ECE by step position (sigmoid-shifted proxy; 95% bootstrap CI,

B{=}500

resamples).

n_{k}

is the number of problems with at least

k

steps.

Variable	Coef.	Odds ratio	$p$ -value
monotone	0.861	2.367	0.003
chain len z	0.020	1.020	0.880
question len z	0.019	1.019	0.933
sc3 agreement z	0.528	1.696	0.003

Appendix

Table 8: Difficulty-proxy controlled logistic regression on GSM8K (

n{=}300

). Bootstrap-based uncertainty estimates are used for the fallback solver.

$\varepsilon$	Mono. rate	Mono. acc	Non-mono. acc	Gap
$0.000$	$0.737$	$0.688$	$0.468$	$+21.9$ pp
$0.005$	$0.737$	$0.688$	$0.468$	$+21.9$ pp
$0.010$	$0.737$	$0.688$	$0.468$	$+21.9$ pp
$0.020$	$0.737$	$0.688$	$0.468$	$+21.9$ pp
$0.050$	$0.737$	$0.688$	$0.468$	$+21.9$ pp
$0.100$	$0.737$	$0.688$	$0.468$	$+21.9$ pp

Appendix

Table 9: Sensitivity of the monotonicity result to

\varepsilon

. Accuracy gap = monotone accuracy

-

non-monotone accuracy. Results are essentially identical across

\varepsilon\in[0,0.10]

, confirming that the choice

\varepsilon{=}0.01

is not critical.

Dataset	Violation bucket	$n$	Accuracy	Spearman $\rho(v,y)$
GSM8K ( $n{=}1319$ )	$0$	$898$	$93.2\%$	$-0.198$
	$1$	$302$	$86.4\%$
	$2$	$83$	$72.3\%$
	$\geq 3$	$36$	$63.9\%$
MATH-500 ( $n{=}500$ )	$0$	$135$	$63.7\%$	$-0.381$
	$1$	$173$	$43.4\%$

Appendix

Table 10: Accuracy by violation count on full GSM8K and MATH-500. Violation count

v

equals the number of steps with

H_{k+1}>H_{k}+\varepsilon

(

\varepsilon{=}0.01

v{=}0

corresponds to monotone chains.

Step $k$	$n_{k}$	ECE (EW)	ECE (EM)
0	300	0.186	0.195
1	300	0.186	0.186
2	290	0.190	0.188
3	225	0.218	0.216
4	160	0.222	0.222
5	89	0.232	0.242

Appendix

Table 11: ECE by step position: equal-width (EW) vs. equal-mass (EM) binning.

n_{k}

= problems with at least

k

steps. Both binning strategies show the same increasing trend; equal-mass binning produces slightly higher ECE at later steps owing to heavier weighting of the tails.

Dataset	$n$	Mono. rate	Mono. acc / Non-mono. acc	Gap
GSM8K (full)	$1319$	$0.681$	$0.932$ / $0.817$	$+11.5$ pp
MATH-500	$500$	$0.270$	$0.637$ / $0.304$	$+33.3$ pp

Appendix

Table 12: Entropy-trajectory monotonicity on full GSM8K and MATH-500, Qwen2.5-7B-Instruct. The monotone group is more accurate on both datasets, with a larger gap on MATH-500.

Model	$n$	Mono. rate	Mono. acc	Non-mono. acc	Gap
Qwen2.5-7B-Instruct	300	$0.737$	$0.688$	$0.468$	$+21.9$ pp
Mistral-7B-Instruct-v0.3	300	$0.397$	$0.723$	$0.376$	$+34.7$ pp

Appendix

Table 13: Cross-model replication on GSM8K. The monotonicity signal remains strong across model families, though coverage differs.

Figure 4: Monotone

-

non-monotone accuracy gap across three additional seeds on GSM8K (

n{=}300

per seed). The gap remains positive for all seeds.

Method	Accuracy	Avg tokens / problem	Coverage
SC@3	$66.0\%$	$831.4$	$100\%$
SC@5 (near-equal budget)	$65.3\%$	$1385.7$	$100\%$
ESC-sim (min 2 chains)	$66.3\%$	$673.6$	$100\%$
Our selective (monotonicity)	$68.8\%$	$\approx 1500$	$73.7\%$

Appendix

Table 14: Empirical SC/ESC comparison on GSM8K (

n{=}300

). “Our selective” uses entropy-trajectory monotonicity and reports answered-set accuracy at 73.7% coverage (main result). SC/ESC report full-coverage accuracy.

Figure 5: Coverage-aware answered-set accuracy on GSM8K (

n{=}300

). SC confidence is vote-agreement fraction for SC@3/SC@5; our curve uses monotonicity-first ranking. Dotted line marks 73.7% coverage.

Figure 6: Unified matched-coverage comparison on GSM8K (

n{=}300

): our monotonicity-first ranking, SC@3/SC@5 vote-agreement rankings, self-judgment confidence, and strict Yes/No self-evaluation confidence. All methods are evaluated as answered-set accuracy versus coverage.

Quantity	Value	Type
Base-chain tokens $\bar{T}_{\text{base-chain}}$	$234.7$	measured
Mean step prefixes $\bar{N}$	$4.91$	measured
Per-prefix samples $m$	$5$	configured
Short completion cap	$150$	configured
Reported total $T_{\text{ours}}$	$\approx 1500$	reported (main text)
SC@3 tokens/problem	$831.4$	measured

Appendix

Table 15: Unified token accounting on GSM8K (

n{=}300

). “Measured” values come from logged generations; “configured” values are fixed decoding settings.

LLM Uncertainty Estimation

Large language model uncertainty estimation measures when outputs may be wrong, ambiguous, or unreliable to guide safer decisions.

See all 399 papers

Answer-Distribution Trajectories: A Stochastic-Dynamics View of LLM Reasoning
Sep 8, 2026Mar Gonzàlez I Català, Haitz Sáez de Ocáriz Borde, Davide Murari +3LLM Uncertainty Estimation LLM Reasoning
University of Cambridge · Harvey Mudd College
Tracing Uncertainty in Language Model "Reasoning"
May 8, 2026Nils Grünefeld, Bertram Højer, Philipp Mondorf +5LLM Evaluation CoT Reasoning
Data Science Section, IT University of Copenhagen · Pioneer Centre for Artificial Intelligence · MaiNLP, Center for Information and Language Processing, LMU Munich +2
LLM Reasoning as Trajectories: Step-Specific Representation Geometry and Correctness Signals
Apr 7, 2026Lihao Sun, Hang Dong, Bo Qiao +3LLM Reasoning Representation Geometry in Language Models
Microsoft

Sep 8, 2026J/K move · Enter open · S save

Answer-Distribution Trajectories: A Stochastic-Dynamics View of LLM Reasoning

Mar Gonzàlez I Català, Haitz Sáez de Ocáriz Borde, Davide Murari, Carola-Bibiane Schönlieb+2

University of Cambridge · Harvey Mudd College

LLM Uncertainty Estimation LLM Reasoning Reasoning Trace Analysis

Open paper AskPDF arXiv

Chain-of-thought reasoning provides a structured computation between a model's input and final answer. Yet it is often evaluated through endpoint accuracy, which ignores the path taken to reach that answer. An emerging line of work addresses this limitation using entropy profiles, which track how uncertainty evolves over the reasoning process but do not reveal which competing hypotheses account for that uncertainty. We introduce answer-distribution trajectories, a stochastic-dynamics-inspired representation that tracks the model's full predictive distribution over answers as reasoning unfolds. As a strictly finer representation than endpoint and entropy summaries, answer-distribution trajectories enable us to characterize a trace through a dynamical reasoning profile spanning exploration, revision, motion, and commitment, and to distinguish different dynamical mechanisms of reasoning success and failure. Across sixteen open-weight language models and four reasoning benchmarks, we show that traces with the same endpoint and similar entropy profiles can exhibit substantially different reasoning dynamics. We further find substantial variation in these dynamics both within and across models and tasks, with different objectives favoring different dynamical profiles. Additionally, we show that training and inference choices systematically reshape these profiles. Our results suggest that answer-distribution trajectories provide a rich framework for analysing and evaluating the dynamics of LLM reasoning.

Abstract

LLM Uncertainty Estimation

Explore similar work

Answer-Distribution Trajectories: A Stochastic-Dynamics View of LLM Reasoning

Tracing Uncertainty in Language Model "Reasoning"

LLM Reasoning as Trajectories: Step-Specific Representation Geometry and Correctness Signals