STEP: Soft Trajectory Estimation of Prediction Correctness via Per-Step Probability Scoring

Chenyang Yu; Zifan Zeng; Qunli Zhang; Feng Liu

cs.CLOct 25, 2026

STEP: Soft Trajectory Estimation of Prediction Correctness via Per-Step Probability Scoring

Authors: Chenyang Yu, Zifan Zeng, Qunli Zhang, Feng Liu

Organizations: Technical University of Munich · RAMS Lab, Huawei Heisenberg Research Center

Abstract

Estimating the correctness of large language model (LLM) predictions without access to ground truth is critical for reliable deployment in high-stakes settings. Existing approaches, such as entropy and perplexity, operate solely on final output statistics, discarding the rich information encoded in intermediate reasoning steps. To address this limitation, we propose STEP (Soft Trajectory Estimation of Prediction correctness), a training-free framework that scores each reasoning step via lightweight forward passes to obtain a soft probability distribution qi over candidate answers, from which we derive a set of soft trajectory features capturing per-step confidence and its progression toward the final answer. Extensive experiments on various benchmarks demonstrate consistent and substantial improvements, particularly in AUROC, over output-based baselines and hard-indicator trajectory methods. Notably, Temporally-Weighted Consistency, which assigns higher importance to later reasoning steps where answer distributions have stabilized, contributes the largest gains, highlighting the importance of temporal structure in reasoning trajectories. These results further validate the advantage of preserving continuous probability signals over binary discretization.

Figures & tables

Figure 1: Overview of the STEP framework. (a) Given a question and a predefined candidate answer set, (b) the LLM generates a $T$ -step chain-of-thought followed by a predicted answer. (c) For each prefix $c_{i}$ , a lightweight forward pass yields a soft probability distribution over candidate answers, from which $q_{i}^{*}$ is extracted. The resulting trajectory $\{q_{1}^{*},\ldots,q_{T}^{*}\}$ reflects how confidently the model commits to its predicted answer across reasoning steps. (d) Three soft trajectory features are computed from this trajectory. (e) The resulting correctness score is used to assess the reliability of the model’s prediction.

Qwen2.5-7B-Instruct
	MMLU			CommonsenseQA			Belebele			ARC-Challenge
Method	AUROC	AUPR-C	AUPR-E	AUROC	AUPR-C	AUPR-E	AUROC	AUPR-C	AUPR-E	AUROC	AUPR-C	AUPR-E
Final-output statistics
Entropy	54.00	72.44	32.43	56.96	83.95	23.42	55.39	91.04	12.04	58.81	90.05	16.64
Perplexity	53.39	72.19	31.91	56.09	83.41	22.74	54.94	90.06	11.80	58.44	90.20	15.84
Max Prob.	53.33	72.11	31.93	56.07	83.35	22.92	54.69	90.88	11.76	58.62	90.27	16.03

Table 1: AUROC, AUPR-C, and AUPR-E on LLM experiments across four benchmarks and two models. Bold : best result per column within each model block. Underline : second best result per column within each model block.

Figure 2: Mean $q^{*}$ trajectory across normalized reasoning steps for correct (blue) and incorrect (orange) predictions, across four datasets and two models. Correct trajectories consistently maintain higher confidence and converge toward near-certainty by the final step, while incorrect trajectories plateau at a lower level. The widening gap toward later steps motivates the temporal weighting in $\text{Con}_{\text{temp}}$ .

Dataset	Qwen2.5-7B	Llama-3-8B
	Acc. (%)	Acc. (%)
MMLU	70.60	61.13
CommonsenseQA	80.67	74.42
Belebele	89.89	89.42
ARC-Challenge	87.63	79.20

Table 2: Task accuracy per benchmark and model.

Figure 3: AUROC as a function of the temporal weighting parameter $\beta$ across datasets on Meta-Llama-3-8B-Instruct. $\beta>0$ up-weights later reasoning steps; $\beta=0$ recovers $\text{Con}_{\text{soft}}$ .

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Dataset	#Samples	#Choices
MMLU	1,531	4
CommonsenseQA	2,000	5
Belebele	900	4
ARC-Challenge	1,172	4

Appendix

Table 3: Dataset statistics used in our evaluation.

Confidence Estimation in Language Models

Confidence estimation in language models measures whether a model’s uncertainty signals predict answer correctness and support reliable decisions.

See all 249 papers

LLM Reasoning as Trajectories: Step-Specific Representation Geometry and Correctness Signals
Apr 7, 2026Lihao Sun, Hang Dong, Bo Qiao +3LLM Reasoning Representation Geometry in Language Models
Microsoft
Diagnosing Multi-step Reasoning Failures in Black-box LLMs via Stepwise Confidence Attribution
May 19, 2026Xiaoou Liu, Tiejin Chen, Dengjia Zhang +3Uncertainty Quantification Confidence Estimation in Language Models
1Arizona State University · 2Johns Hopkins University · 3Purdue +1
Confidence Geometry Reveals Trace-Level Correctness in Large Language Model Reasoning
May 16, 2026Shuo Liu, Ding Liu, Shi-Ju RanConfidence Estimation in Language Models LLM Reliability
School of Computer Science and Technology, Tiangong University, Tianjin, China. · Center for Quantum Physics and Intelligent Sciences, Department of Physics, Capital Normal University, Beijing, China.

Apr 7, 2026J/K move · Enter open · S save

LLM Reasoning as Trajectories: Step-Specific Representation Geometry and Correctness Signals

Lihao Sun, Hang Dong, Bo Qiao, Qingwei Lin+2

Microsoft

LLM Reasoning Representation Geometry in Language Models Language Model Steering

Open paper AskPDF arXiv

This work characterizes large language models' chain-of-thought generation as a structured trajectory through representation space. We show that mathematical reasoning traverses functionally ordered, step-specific subspaces that become increasingly separable with layer depth. This structure already exists in base models, while reasoning training primarily accelerates convergence toward termination-related subspaces rather than introducing new representational organization. While early reasoning steps follow similar trajectories, correct and incorrect solutions diverge systematically at late stages. This late-stage divergence enables mid-reasoning prediction of final-answer correctness with ROC-AUC up to 0.87. Furthermore, we introduce trajectory-based steering, an inference-time intervention framework that enables reasoning correction and length control based on derived ideal trajectories. Together, these results establish reasoning trajectories as a geometric lens for interpreting, predicting, and controlling LLM reasoning behavior.

Abstract

Confidence Estimation in Language Models

Explore similar work

LLM Reasoning as Trajectories: Step-Specific Representation Geometry and Correctness Signals

Diagnosing Multi-step Reasoning Failures in Black-box LLMs via Stepwise Confidence Attribution

Confidence Geometry Reveals Trace-Level Correctness in Large Language Model Reasoning