Large language models (LLMs) increasingly solve difficult problems by producing reasoning traces before emitting a final response. However, it remains unclear how accuracy and decision commitment evolve along a reasoning trajectory, and whether intermediate trace segments provide answer-relevant information beyond generic length or stylistic effects. Here, we propose a trajectory-probing protocol for evaluating how a reasoning model's predicted answer evolves along a partial reasoning trace. The protocol (1) generates a model's full reasoning trace, (2) truncates it at fixed token-percentiles, and (3) injects each partial trace back into the model, measuring the model's induced answer distribution. We apply the protocol to seven open-source reasoning models (Qwen3-4B/8B/14B, gpt-oss-20b/120b, DeepSeek-R1-Distill-Llama-8B and Gemma-4-31B-it) across four benchmarks (GPQA Diamond, MMLU-Pro, Omni-MATH-2 and CompMath). The protocol reveals that accuracy and decision commitment generally increase as the percentage of provided reasoning tokens grows, and length, form, and token-identity controls confirm that gains stem from instance-specific semantic content rather than context length or generic reasoning style effects. Furthermore, cross-model probing shows that even stronger models frequently repeat the source model's errors when given its partial reasoning trace, while allowing further reasoning substantially improves recovery. Together, these results provide a reproducible, model-agnostic measurement of reasoning trace dynamics that generalizes across answer formats and can reveal model-specific failure modes invisible to aggregate accuracy.
Figures & tables
Figure 1: Overview of the trajectory-probing protocol. (1) Generate full responses and extract reasoning traces. (2) Truncate each reasoning trace at fixed token deciles, including an empty-prefix baseline and the complete trace. (3) Reinject each prefix with the original question and an early-stopping suffix, then measure final answer probability and correctness. For exact-answer tasks, answer correctness is judged by GPT-5 mini and we average the answer’s token probabilities. Length-matched random, swapped, and shuffled prefixes provide control conditions. The main evaluation covers seven models, four benchmarks, and three runs; reasoning effort ablations are restricted to the gpt-oss models on GPQA Diamond and MMLU-Pro.
Figure 2: Accuracy and decision commitment across reasoning depth. a–d , Accuracy on GPQA Diamond ( n=198 ), MMLU-Pro ( n=12,032 ), CompMath ( n=267 ), and Omni-MATH-2 ( n=4,181 ), respectively. e–h , Corresponding decision-commitment. Curves show Qwen3-4B/8B/14B, gpt-oss-20b/120b, DeepSeek-R1-Distill-Llama-8B, and Gemma-4-31B-it, averaged over three runs. For multiple-choice tasks, accuracy uses the argmax over answer-letter probabilities and decision commitment is the unconditional probability assigned to the decile-100 predicted answer. For Omni-MATH-2, accuracy uses GPT-5 mini verdicts and decision commitment is the mean of generated-answer token probabilities. Shading shows pointwise 95% question-bootstrap confidence intervals with 2,000 resamples, retaining all observed runs together. Accuracy generally increases, with a pronounced Qwen3-8B endpoint decline, especially on CompMath ( Table A3 ). Companion non-choice and flip-rate panels for the multiple-choice settings are shown in Figure A1 .
Figure 3: Cross-model probing of incorrect partial traces. Select source-model prefixes yielding an incorrect answer at 20%, 40%, 60%, or 80% of the reasoning trace, then inject them into a target model. Compare an immediate answer with free continuation. Rescue denotes a correct target answer; anchoring denotes repetition of the source model’s incorrect answer. The target may also produce a different incorrect answer.
Figure 4: Original reasoning traces compared with length-matched controls. Columns show GPQA Diamond, MMLU-Pro, and CompMath; rows show original prefixes ( a–c ), random tokens ( d–f ), another question’s reasoning prefix ( g–i ), and a permutation of the original prefix’s tokens ( j–l ). Curves show the seven models in Figure 2 , averaged over three runs. Endpoint markers in the control panels indicate the original no-reasoning and full-trace accuracies. Shading shows pointwise 95% question-bootstrap confidence intervals with 2,000 resamples, retaining paired runs and conditions together. Original traces generally outperform the controls at full length; Qwen3-8B on CompMath is an exception, where its endpoint collapse puts original accuracy below all three controls. Paired endpoint comparisons and their statistical conventions are reported in Table A2 .
Figure 5: Rescue rates under immediate answering and free continuation. Each cell reports the percentage of selected transfers for which the target answers correctly, conditional on the source model answering incorrectly at that prefix. Columns of heatmaps correspond to 20%, 40%, 60%, and 80% prefixes; within each heatmap, rows identify source models and columns identify targets. a,b , GPQA Diamond, pooling measured transfers from runs 1–3. c,d , MMLU-Pro, run 1. Panels a,c use answer now; b,d allow free continuation. The figure includes ten forward pairs and two reverse pairs per benchmark (Qwen3-8B → Qwen3-4B and gpt-oss-20b → Qwen3-14B). These reverse directions account for accuracy rankings that do not consistently follow model size: Qwen3-4B exceeds Qwen3-8B at full trace on both benchmarks, while Qwen3-14B exceeds gpt-oss-20b on MMLU-Pro ( Figure 2 ). Blank cells denote unmeasured pairs. Two MMLU-Pro reverse-transfer measurements exceed the target context limit and are omitted. Free continuation improves recovery, with lower rescue rates at later prefixes for many pairs. For a given source model and prefix, gpt-oss targets show higher observed rescue rates than the evaluated Qwen3 targets under free continuation; this family ordering is less consistent when answering immediately. Table A4 summarizes the ten forward pairs only. In a separate continuation-trace pilot, explicit backtracking was more common than restarting among correct departures ( Table A5 ).
Appendix figures & tables25 assets
Supplementary material from the paper’s appendix.
Appendix
Figure A1: Non-choice probability and flip rate on multiple-choice tasks. Companion to Figure 2 , covering seven models and three runs. Columns show GPQA Diamond, MMLU-Pro, and CompMath. a–c , Probability mass outside the answer-letter set. d–f , Fraction of predictions whose argmax answer differs from the preceding decile. Qwen3-8B exhibits an endpoint increase in non-choice mass and switching, most prominently on CompMath; DeepSeek-R1-Distill-Llama-8B also assigns substantial mass to non-choice tokens. These choice-based metrics are not reported for Omni-MATH-2.
Figure A2: Early-stopping suffix ablation for the Qwen3 family. Per-decile accuracy on GPQA Diamond (left) and MMLU-Pro (right), averaged over 3 runs, with three suffix variants: default (Qwen documentation), minimal ( </think> only), and neutral (“Based on my reasoning so far, my answer is:”). Trajectories are nearly indistinguishable across the three suffixes, confirming the protocol’s measurements are not driven by suffix phrasing. Note that forcing reasoning to stop after a pre-specified number of tokens by appending a suffix is exactly how the Qwen3 documentation enforces this, and how it explicitly trains the model to handle these cases [ 3 ] . The gpt-oss family is excluded as it uses native API structural tokens with no linguistic suffix.
Figure A3: Endpoint trajectory outcomes on the multiple-choice benchmarks. Stacked bars compare correctness at decile 0 and decile 100 for seven models on GPQA Diamond ( a ), MMLU-Pro ( b ), and CompMath ( c ), pooling runs 1–3. Stable correct (dark blue) denotes correct answers at both endpoints; gained (light blue), incorrect to correct; lost (orange), correct to incorrect; and stable wrong (gray), incorrect at both endpoints. Gained outcomes exceed lost outcomes for every model–benchmark combination except Qwen3-8B on CompMath, which has 35.2% lost and 3.1% gained outcomes.
Figure A4: Endpoint trajectory outcomes on Omni-MATH-2 by difficulty. Stacked bars compare judge-scored correctness at decile 0 and decile 100 for all seven models, pooling runs 1–3 within each difficulty tier. Panels a–d show increasingly difficult tiers formed from quartile cut points of the dataset’s difficulty ratings. Colors denote stable correct, gained, lost, and stable wrong outcomes as in Figure A3 . The figure separates net accuracy gains from individual correct-to-incorrect reversals within each tier.
Figure A5: Prediction confidence for correct and incorrect decile predictions. Columns show GPQA Diamond, MMLU-Pro, and CompMath for all seven models. a–c , Mean unconditional probability assigned to the current argmax answer, separately for correct predictions (solid) and incorrect predictions (dashed). d–f , The correct-minus-incorrect prediction-confidence gap, in percentage points. Conditional means are computed within each run and then averaged over three runs. Here, prediction confidence is derived from answer-token probabilities; decision commitment in Figure 2 instead tracks the answer selected at the final decile. The gap measures discrimination between correct and incorrect predictions, not calibration. Overall, prediction confidence often rises for both correct and incorrect answers, while the gap does not consistently widen and narrows for most models on MMLU-Pro, indicating that additional reasoning does not consistently improve discrimination and can yield incorrect answers assigned high probability.
Figure A6: Reasoning trace length distributions and their relationship to accuracy. Analysis of full reasoning traces (decile 100) for the seven models on GPQA Diamond, MMLU-Pro, and CompMath, averaged over 3 runs. a,b , Violin plots showing trace length distributions (in tokens) by model on GPQA Diamond and MMLU-Pro. gpt-oss models produce substantially shorter traces than Qwen3 models, with gpt-oss-120b generating the most concise reasoning. c,d , Accuracy stratified by trace-length quintile (shortest to longest) reveals a consistent negative relationship on GQPA Diamond and MMLU-Pro: shorter traces are associated with higher accuracy across all models and benchmarks. This pattern suggests models produce extended reasoning when struggling with difficult questions, rather than extended reasoning leading to correct answers. On CompMath, six models remain near ceiling through the first four bins and have lower accuracy in the longest bin; Qwen3-8B instead shows a non-monotonic pattern.
Figure A7: Reasoning trace lengths and endpoint accuracy on Omni-MATH-2, stratified by difficulty. Columns show four increasingly difficult tiers for all seven models and three runs. a–d , Distributions of reasoning trace length in native tokens, on logarithmic axes. e–h , Judge-scored decile-100 accuracy across length quintiles formed within each model, run, and difficulty tier, averaged over runs. The overall negative association between trace length and accuracy persists within difficulty tiers for the six models without the Qwen3-8B endpoint anomaly; Qwen3-8B shows a non-monotonic relationship.
Figure A8: Accuracy on Omni-MATH-2 at each reasoning decile, stratified by the dataset’s predefined difficulty rating (1–9.5). The increase persists within every quartile and every model, ruling out question difficulty as a driver of the trajectory dynamics. Averaged over three runs.
Figure A9: Accuracy on MMLU-Pro at each reasoning decile, stratified across the 14 MMLU-Pro subject categories. The accuracy increase holds universally with the exception of Qwen3-8B (see Table A3 ); reasoning gains are steeper on subjects like Math and Physics than on subjects like Law and Psychology. Averaged over 3 runs.
Figure A10: Per-decile accuracy at three reasoning-effort settings (gpt-oss family). Accuracy at each reasoning decile for gpt-oss-20b and gpt-oss-120b on GPQA Diamond and MMLU-Pro, at low (light), medium (mid), and high (dark) reasoning effort. Low-effort traces plateau earlier; high-effort traces continue to gain in the second half of the trace. Averaged over 3 runs.
Figure A11: Trajectory outcomes by reasoning-effort setting. Stacked bar charts of decile-0 vs. decile-100 outcomes (stable correct / gained / lost / stable wrong) per reasoning effort level for gpt-oss-20b and gpt-oss-120b on GPQA Diamond and MMLU-Pro. The gained fraction is largest at high effort; the lost fraction is approximately stable across effort levels. Averaged over 3 runs.
Figure A12: Distribution of accuracy gains across deciles. Post-hoc analysis of where accuracy gains concentrate within the reasoning trajectory. a,c , Mean accuracy change per consecutive decile step on GPQA Diamond and MMLU-Pro. Gains are spread approximately uniformly across all decile intervals, with no single step dominating. b,d , Distribution of the per-question peak-change decile. Beyond the initial 0%→10% step, peak changes are spread approximately uniformly across deciles. This validates the fixed 10% decile slicing as a sensible default; semantic-boundary truncation [ 15 ] is complementary, providing finer model-specific resolution.
Figure A13: Highest-probability non-choice tokens by model and decile. For each of the seven models, tokens are ranked by their mean recorded probability across GPQA Diamond and MMLU-Pro question–run observations from runs 1–3. Five tokens are displayed at each decile. Tokens absent from a returned probability record contribute zero. Cell shading indicates mean probability, with a separate color scale for each model. The pooled means weight question–run observations equally, so the larger MMLU-Pro dataset contributes more observations.
Figure A14: Anchoring rates in cross-model probing. Each cell reports the percentage of selected transfers for which the target repeats the source model’s incorrect answer. Layout and cohorts match Figure 5 , with ten forward and two reverse model pairs: a,b , GPQA Diamond, pooling runs 1–3; c,d , MMLU-Pro, run 1; answer now in a,c and free continuation in b,d . Within each target-model block, columns show 20%, 40%, 60%, and 80% prefixes. Blank cells denote unmeasured pairs; two context-limited MMLU-Pro reverse measurements are omitted. Free continuation reduces anchoring, but substantial anchoring remains at later prefixes.
Figure A15: Accuracy with answer-deleted reasoning prefixes. Fixed-token baseline (solid blue) versus the answer-deleted reasoning condition (dashed orange) for seven models on GPQA Diamond ( a–g ) and MMLU-Pro ( h–n ). Curves average three runs; shading shows pointwise 95% question-bootstrap confidence intervals (2,000 resamples, retaining paired conditions and runs together). Broad accuracy gains remain, but the intervention changes some trajectories and endpoints, including Qwen3-8B’s endpoint behavior.
Figure A16: Sensitivity to truncation at paragraph boundaries. Fixed-token deciles (solid blue) versus the nearest-paragraph-boundary condition (dashed orange) for seven models on GPQA Diamond ( a–g ) and MMLU-Pro ( h–n ). Curves average three runs; shading shows pointwise 95% question-bootstrap confidence intervals (2,000 resamples, retaining paired conditions and runs together). The horizontal axis denotes the requested reasoning decile. Overall patterns are similar, with visible intermediate-decile differences, especially for the gpt-oss models on MMLU-Pro.
Figure A17: Example of a prompt formatted for the OpenAI gpt-oss family on question 1 of GPQA Diamond.
Figure A18: Example of a non-reasoning prompt formatted for the OpenAI gpt-oss family on question 1 of GPQA Diamond.
Figure A19: Example of a full response (from run 1) of OpenAI gpt-oss-20b on question 1 of GPQA Diamond.
Figure A20: Example of a 10% reasoning decile (from run 1) of OpenAI gpt-oss-20b on question 1 of GPQA Diamond.
Figure A21: Example of a prompt formatted for the Qwen3-4B model on question 1 of GPQA Diamond.
Figure A22: Example of a non-reasoning prompt formatted for the Qwen3-4B model on question 1 of GPQA Diamond.
Figure A23: Example of a full response (from run 1) of Qwen3-4B on question 1 of GPQA Diamond. The reasoning trace has been shortened with […] for display purposes (full trace: 7,639 tokens).
Figure A24: Example of a 10% reasoning decile (from run 1) of Qwen3-4B on question 1 of GPQA Diamond.
Figure A25: Example of a 10% reasoning decile with alternative early-stopping suffix.
Large language models (LLMs) are often post-trained on pre-collected reasoning trajectories to improve their reasoning capability. Such trajectories tend to be long due to complex, interwoven paths, which often include detours on the path toward the answer. However, it has been underexplored whether LLMs indeed benefit from learning complete trajectories in post-training, such as supervised fine-tuning (SFT). Starting from our pilot study, we find that full trajectories provide only limited benefit, while partial trajectories are effective even under heavy truncation. We analyze redundancy in reasoning trajectories through attention-based analyses and controlled token-removal studies, both of which show that intermediate tokens contribute minimally to final reasoning quality. This suggests that avoiding redundant information may allow LLMs to internally infer coherent alternatives by inferring missing steps from their internal knowledge, given known trajectory endpoints. Furthermore, we show that training LLMs using endpoints leads to consistent changes in reasoning behavior, and that it also benefits post-training methods based on reinforcement learning or on-policy distillation, highlighting the need to revisit complete reasoning traces. Code is available at https://github.com/naver-ai/revisiting-trace.
This work characterizes large language models' chain-of-thought generation as a structured trajectory through representation space. We show that mathematical reasoning traverses functionally ordered, step-specific subspaces that become increasingly separable with layer depth. This structure already exists in base models, while reasoning training primarily accelerates convergence toward termination-related subspaces rather than introducing new representational organization. While early reasoning steps follow similar trajectories, correct and incorrect solutions diverge systematically at late stages. This late-stage divergence enables mid-reasoning prediction of final-answer correctness with ROC-AUC up to 0.87. Furthermore, we introduce trajectory-based steering, an inference-time intervention framework that enables reasoning correction and length control based on derived ideal trajectories. Together, these results establish reasoning trajectories as a geometric lens for interpreting, predicting, and controlling LLM reasoning behavior.
Large language models (LLMs) generate not only reasoning text, but also token-level confidence trajectories that record how uncertainty evolves during inference. Whether these trajectories are relevant to reasoning correctness remains unclear. Here we show that confidence trajectories encode a content-agnostic confidence geometry associated with trace-level final-answer correctness. Using only token-level confidence values, without access to the input question, reasoning text, hidden states, or external verifiers, we find that low-dimensional representations of confidence trajectories separate correct from incorrect reasoning traces. Across GSM8K, MATH, and MMLU, this geometric separation is quantitatively linked to downstream predictability: stronger clustering of correct and incorrect traces, measured by the Davies--Bouldin index, consistently corresponds to higher correctness-discrimination AUC. We further show that correctness-related information is enriched in the tail of reasoning, suggesting that late-stage confidence dynamics carry key correctness signals. We propose NeuralConf, a lightweight estimator that learns from confidence trajectories for correctness evaluation. Under a fixed trace budget, NeuralConf-derived scores improve confidence-weighted answer aggregation over majority voting, tail confidence, and other static baselines. These results reveal that LLMs expose trace-intrinsic statistical signals of correctness through their own confidence dynamics, offering a route to improve inference using information already present within generation.
Shuo Liu, Ding Liu, Shi-Ju Ran
School of Computer Science and Technology, Tiangong University, Tianjin, China. · Center for Quantum Physics and Intelligent Sciences, Department of Physics, Capital Normal University, Beijing, China.