Modern reasoning models continue generating long after the answer is already determined. Across five model configurations, two families, and three benchmarks, we find that 52--88% of chain-of-thought tokens are produced after the answer is recoverable from a partial prefix. This post-commitment generation reveals a structural phenomenon: the detection-extraction gap. Free continuations from early prefixes recover the correct answer even at 10% of the trace, while forced extraction fails on 42% of these cases. The answer is recoverable from the model state, yet prompt-conditioned decoding fails to extract it. We formalize this mismatch via a total-variation bound between free and forced continuation distributions, yielding quantitative estimates of suffix-induced shift. Exploiting this asymmetry, we propose Black-box Adaptive Early Exit (BAEE), which uses free continuations for both detection and extraction, truncating 70--78% of serial generation while improving accuracy by 1--5pp across all models. For thinking-mode models, early exit prevents post-commitment overwriting, yielding gains of up to 5.8pp; a cost-optimized variant achieves 68--73% reduction at a median of 9 API calls. Code is available at https://github.com/EdWangLoDaSc/know2say.
Figures & tables
Figure 1: Two core findings. (a) Commitment map for 32B-Think: 69% of CoT tokens follow the commitment boundary (75% under PSC threshold; Table 2 ). (b) The detection–extraction gap: psc detects recoverability from 10% prefix, yet efa fails to extract on 42% of these problems. baee exploits this structure for 70–78% serial reduction with 1–5 pp accuracy gains (§ 4.5 ).
Figure 2: Method pipeline. At prefix fraction f=0.1 , psc (free continuation) recovers the correct answer 82% of the time, while efa (forced extraction) succeeds on only 34%. baee uses free continuations for both detection and extraction, sidestepping the gap entirely.
Category
Details
Models
Qwen3-32B-(Think/NoThink), Qwen3-8B-(Think/NoThink) Yang et al. (2025a) , and GPT-OSS-120B (medium reasoning) Agarwal et al. (2025) .
Full rollouts: 4 (temp. 1.0); psc samples: 8; entropy top- k : 20; efa max tokens: 64.
Grading
SymPy symbolic equivalence (MATH-500); Exact match (GPQA).
Re-grading
Fixed efa suffix-stripping bug (90 evaluations flipped). Verified against independent SymPy re-grade with zero discrepancies.
Table 1: Compact summary of the experimental setup.
Model
Acc.
Commitment profile
Latency reduction
Commit
95% CI
Post-commit
CoT len
Main-rollout red.
Avg calls
Qwen3-32B-Think
82%
25%
[23%, 26%]
75%
2878
72%
7.7
Qwen3-32B-NoThink
91%
39%
[35%, 42%]
61%
698
73%
8.0
Qwen3-8B-Think
76%
27%
[25%, 29%]
73%
2881
68%
7.3
Qwen3-8B-NoThink
90%
48%
[44%, 52%]
52%
696
73%
8.0
GPT-OSS-120B
96%
36%
[34%, 38%]
64%
823
85%
8.6
Table 2: Post-commitment generation metrics on MATH-500 (re-graded). “Acc.” = fraction of problems where the first of 4 sampled rollouts is correct (single-rollout accuracy, not best-of-4). Commitment fraction defined conditionally on solvable instances ( ≥ 1/4 correct); CIs from 10K bootstrap resamples. Latency reduction shown for psc -8 all (accuracy-optimal; see Table 5 for all strategies).
Figure 3: Main results. (a) efa accuracy by prefix length. (b) psc agreement ( > 70% from the first checkpoint). (c) Commitment distributions (Think median ∼ 25%, NoThink ∼ 40%). (d) Commitment increases monotonically with problem difficulty.
Problem
GT
PSC@10%
EFA@10%
Failure mode
1−2+3−4+⋯+99−100
−50
8/8
50
Sign dropped
f(x)=2x ; find f(f(f(f(1))))
256
8/8
4
Intermediate value
Table 3: Detection–extraction gap: two case studies (Qwen3-32B-Think, 10% prefix, psc 8/8). efa (shaded) extracts the wrong answer despite full recoverability via free continuation.
Figure 4: Commitment maps. (a) 32B-Think: 69% post-commitment. (b) 8B-NoThink: 35% post-commitment, later and more variable.
Model
Benchmark
Acc.
Commit
Post-commit
CoT len
PSC@10%
EFA@10%
Main-rollout red.
8B-Think
MATH-500
76%
27%
73%
2881
70%
25%
68%
GPQA-Diamond
77%
37%
63%
6884
64%
26%
49%
8B-NoThink
MATH-500
90%
48%
52%
696
74%
21%
73%
GPQA-Diamond
74%
38%
62%
1601
60%
34%
48%
32B-Think
MATH-500
82%
25%
75%
2878
75%
34%
72%
GPQA-Diamond
81%
33%
67%
6192
73%
34%
67%
Table 4: GPQA-Diamond vs MATH-500. The gap persists on a harder benchmark, but commitment occurs later and baee reduction is correspondingly lower.
Figure 5: (a) Failure taxonomy for 208 gap instances. (b) Gap magnitude across models.
Figure 6: MATH-500 vs GPQA-Diamond. (a,b) PSC vs EFA across prefixes. (c,d) Commitment distributions.
Figure 7: Entropy analysis. (a) Per-token entropy along the CoT. (b) Wrong problems (dashed) show higher entropy. (c) Post/pre-commit ratio separates Think ( > 1) from NoThink ( < 1).
Strategy
GT?
Calls
32B-Think
32B-NoThink
8B-Think
8B-NoThink
Full CoT
—
1
82% / —
91% / —
76% / —
90% / —
Naïve efa
No
∼ 6
34% / 90%
38% / 90%
35% / 90%
28% / 90%
EFA-oracle
Yes
∼ 6
82% / 67%
91% / 48%
76% / 67%
90% / 38%
psc -8 all
No
72
86% / 74%
92% / 78%
81% / 70%
91% / 77%
psc -8 adaptive
No
9 (med.)
82% / 72%
91% / 73%
76% / 68%
90% / 73%
Table 5: Accuracy / main-rollout reduction for each baee strategy. “Reduction” = serial main-rollout tokens avoided ( not total billed tokens; see § 4.5.3 ). psc -8 all probes every checkpoint; psc -8 adaptive stops at first trigger. Format: Acc. / Reduction .
MATH-500
GPQA-Diamond
Model
θ∗
Δ Acc
Red.
θ∗
Δ Acc
Red.
32B-Think
1.000
+ 1.8 pp
67%
0.875
+ 1.0 pp
54%
32B-NoThink
0.875
+ 0.8 pp
71%
1.000
± 0
36%
8B-Think
1.000
+ 1.2 pp
63%
0.750
+ 3.0 pp
47%
8B-NoThink
1.000
+ 1.4 pp
67%
0.750
+ 3.0 pp
47%
GPT-OSS-120B
1.000
+ 0.4 pp
81%
0.625
+ 4.0 pp
72%
Table 6: Calibrated threshold θ∗ and held-out test performance on MATH-500 (250-problem split) and GPQA-Diamond (99-problem split). θ∗=1.0 for 4/5 MATH models; GPQA selects lower thresholds, consistent with its harder problems relaxing the FP constraint.
MATH-500
Model
CoT
Ser. red.
Est. tot.
Ratio
32B-Think
2 879
63%
11 228
3.9 ×
32B-NoThink
699
55%
3 006
4.3 ×
8B-Think
2 881
57%
10 372
3.6 ×
8B-NoThink
697
55%
2 927
4.2 ×
GPT-OSS-120B
823
76%
4 115
5.0 ×
Table 7: Compute redistribution under baee (MATH-500). Estimated totals use measured continuation lengths (Appendix L ). GPQA-Diamond ratios are lower (3.1–5.0 × ); see Appendix P .
Appendix figures & tables23 assets
Supplementary material from the paper’s appendix.
Appendix
Model
Base acc.
BAEE acc.
Δ Acc
Main-rollout red.
Exit Rate
8B-Think
75.6%
81.4%
+ 5.8 pp
69.8%
80.6%
8B-NoThink
89.8%
90.6%
+ 0.8 pp
77.0%
89.4%
32B-Think
82.0%
86.4%
+ 4.4 pp
74.3%
85.6%
32B-NoThink
91.4%
92.0%
+ 0.6 pp
78.3%
90.6%
GPT-OSS-120B
95.6%
96.2%
+ 0.6 pp
85.6%
96.0%
Appendix
Table 8: psc -8 BAEE accuracy and main-rollout reduction under offline majority-vote simulation at θ=0.625 (5/8 trigger). Base is full-CoT accuracy; Δ Acc is the change from early exit. “Token reduction” refers to serial main-rollout tokens only. All models: 500 problems.
Figure 8: Offline majority-vote BAEE under aggressive thresholds. (a) Accuracy change vs full CoT across θ thresholds: Think models (solid lines) gain up to +5.8 pp; NoThink models (dashed) can degrade for low θ . (b) Pareto frontier of accuracy vs main-rollout reduction; large dots mark θ=0.625 . Dotted lines show full-CoT baseline accuracy for each model.
Figure 9: Outcome breakdown at θ=0.625 (offline majority-vote simulation). Green = overthinking corrected (wrong under full CoT, correct under BAEE); red = BAEE harmed (correct under full CoT, wrong under BAEE); blue = always correct; gray = always wrong.
Model
N wrong
Max psc
psc ≥50%
psc ≥75%
32B-Think
17
0.50
1 (at 25%)
0
32B-NoThink
43
0.75
3
3
8B-Think
20
0.31
0
0
8B-NoThink
51
0.88
6
2
GPT-OSS-120B
3
0.19
0
0
Appendix
Table 9: Maximum psc accuracy across all checkpoints for problems with 0/4 correct rollouts. NoThink models show nonzero proxy FP at θ=0.75 . “N wrong” uses 0/4 correct rollouts as a proxy for unsolvable instances.
Figure 10: Threshold-sweep frontier under PSC-8. Left: NoThink FP–savings trade-off on 500-problem runs. Right: mean savings as a function of θ across all models, with selected operating points highlighted.
Feature
TP mean
FP mean
Gap
PSC@10% (first checkpoint)
0.85
0.33
+ 0.52
Mean PSC across all checkpoints
0.86
0.44
+ 0.42
Max PSC
0.99
0.85
+ 0.14
PSC spread (max − min)
0.46
0.77
− 0.31
Number of PSC drops
1.3
3.2
− 1.9
Checkpoint of max PSC
22%
44%
− 22%
Appendix
Table 11: Trajectory features distinguishing true positives (correct, high psc ) from false positives (wrong, high psc ). All differences are large and statistically significant ( p<10−5 , permutation test).
Figure 11: Discriminative signals for filtering high- psc false positives. (a) PSC@10% distribution: TPs concentrate at high values; FPs spread across low values. (b) Number of psc drops: TPs are near-monotone; FPs oscillate. (c) FP rate decreases with stricter mean-PSC thresholds on all benchmarks.
Figure 12: PSC monotonicity: MATH-500 (non-monotone) vs GPQA-Diamond (monotone).
Perturbation
f=0.10
f=0.50
Mean PSC
Drops > 10 pp
Mean PSC
Drops > 10 pp
Intact (control)
97.5%
—
97.8%
—
Truncate last 20%
96.0%
5/50
97.5%
3/50
Shuffle last 30%
96.9%
6/50
97.0%
3/50
Random replace 30%
94.8%
14/50
88.5%
17/50
Appendix
Table 12: Prefix perturbation results. Even replacing 30% of tokens with random vocabulary items reduces psc by only 2.8 pp at f=0.10 .
32B-Think
8B-NoThink
GPT-OSS-120B
Suffix
EFA
PSC
Gap
EFA
PSC
Gap
EFA
PSC
Gap
original
35%
80%
45
27%
78%
51
26%
94%
68
natural
41%
80%
39
28%
78%
50
23%
94%
71
soft
35%
80%
45
29%
78%
49
18%
94%
76
plain
5%
80%
75
22%
78%
56
0%
94%
94
direct
17%
80%
63
16%
78%
62
11%
94%
83
Appendix
Table 13: efa accuracy at 10% prefix for five forcing templates across three models spanning two families, two scales, and two reasoning modes. The gap persists universally.
Prefix fraction f
Configuration
0.10
0.20
0.30
0.50
0.70
32B-Think
≥0.54
≥0.44
≥0.29
≥0.09
≥0.05
8B-Think
≥0.51
≥0.40
≥0.25
≥0.07
≥0.04
GPT-OSS
≥0.70
≥0.55
≥0.38
≥0.12
≥0.06
32B-NoThink
≥0.41
≥0.30
≥0.18
≥0.06
≥0.03
Appendix
Table 14: Lower bounds on dTV(Pfree,Pforced) derived from measured gaps via Proposition 2 . Each entry is gapk=\textscpsc(k)−\textscefa(k) .
Suffix
Mean gap (pp)
dTV lower bound
Expected shift
natural ( \boxed -like)
53.3
≥0.53
lowest
original ( \boxed{ )
54.7
≥0.55
low
soft (“answer is”)
56.7
≥0.57
moderate
direct (“Final answer:”)
69.3
≥0.69
high
plain (bare number)
75.0
≥0.75
highest
Appendix
Table 15: Suffix-ranking calibration. Mean gap ( ≥dTV lower bound) across 32B-Think, 8B-NoThink, and GPT-OSS vs. qualitative distributional-shift expectation. The ranking is perfectly monotone (Spearman ρ=1.0 ).
Fraction
PSC mean
EFA acc
Gap
2%
90.1%
32.4%
57.7 pp
4%
88.6%
32.4%
56.2 pp
5%
90.4%
29.4%
61.0 pp
8%
91.9%
38.2%
53.7 pp
10%
91.2%
35.3%
55.9 pp
15%
91.5%
47.1%
44.5 pp
Appendix
Table 16: Fine-grained checkpoint results (32B-Think, 50 problems). psc agreement is already high at 5%, confirming that the 10% grid does not inflate post-commitment estimates.
Model
Mean commitment fraction by difficulty
L1 (easy)
L2
L3
L4
L5 (hard)
32B-Think
13%
20%
23%
29%
32%
32B-NoThink
32%
28%
35%
45%
51%
8B-Think
18%
23%
26%
30%
33%
8B-NoThink
31%
39%
42%
58%
63%
GPT-OSS-120B
22%
26%
33%
40%
48%
Appendix
Table 17: Mean commitment fraction by problem difficulty level (MATH levels 1–5).
Figure 13: Post-commitment fraction across MATH-500 and GPQA-Diamond. Solid bars = MATH-500, hatched = GPQA-Diamond. The Think–NoThink gap is consistent across benchmarks.
Figure 14: Commitment map for GPQA-Diamond (same visualization as Figure 4 ). The commitment boundary appears later than on MATH-500, consistent with harder problems requiring more genuine reasoning, but substantial post-commitment generation remains (61–67%).
BAEE
SC-8-full
Bench
Model
Acc
Ratio
Acc
Ratio
Δ Acc
MATH-500
32B-Think
85.0%
4.1 ×
71.8%
8.0 ×
+13.2
32B-NoThink
91.4%
3.6 ×
81.4%
8.0 ×
+10.0
8B-Think
80.0%
3.9 ×
65.2%
8.0 ×
+14.8
8B-NoThink
90.2%
3.6 ×
77.2%
8.0 ×
+13.0
GPT-OSS-120B
96.2%
4.8 ×
92.4%
8.0 ×
+3.8
Appendix
Table 18: BAEE vs SC-8-full: accuracy and token cost on MATH-500 and GPQA-Diamond. “Ratio” = total tokens / single full-CoT tokens. BAEE achieves higher accuracy at lower total cost on both benchmarks.
Model
Acc.
Solvable
Commit
Post-commit
32B-Think
64.6%
138/164
14.7%
85.3%
32B-NoThink
82.9%
154/164
13.3%
86.7%
8B-Think
64.0%
130/164
14.1%
85.9%
8B-NoThink
74.4%
149/164
12.4%
87.6%
Appendix
Table 19: HumanEval results (4 models, 164 problems). Post-commitment fractions are the highest across all benchmarks (85–88%), and the Think–NoThink gap nearly vanishes ( < 2 pp).
Model
Full CoT
BAEE
Δ Acc
Reduction
32B-Think
64.6%
78.2%
+13.6
71.1%
32B-NoThink
82.9%
90.2%
+7.3
78.2%
8B-Think
64.0%
77.6%
+13.6
71.3%
8B-NoThink
74.4%
82.3%
+7.9
72.2%
Appendix
Table 20: BAEE on HumanEval ( θ=0.75 ). Think models gain +13.6 pp accuracy, the largest overthinking correction across all benchmarks.
Figure 15: HumanEval results. (a) psc agreement across prefix fractions. (b) efa accuracy (code extraction is harder than math extraction). (c) Commitment distributions: all models commit by 10–20%. (d) Detection–extraction gap persists on code generation.
Figure 16: Post-commitment fraction across three benchmarks (MATH-500, GPQA-Diamond, HumanEval) and five models. HumanEval shows the highest post-commitment fraction (85–88%) across all models, with a near-zero Think–NoThink gap. GPT-OSS-120B is not available for HumanEval (API limitation).
MATH-500
GPQA-D
Method
Access
Δ Acc
CR
Δ Acc
CR
Qwen3-14B (DEER) / Qwen3-8B-Think (ours)
Vanilla
—
—
100%
—
100%
NoThinking
black-box
− 5.6
27%
− 9.5
32%
TCC
black-box
− 0.6
61%
− 0.5
61%
Dynasor-CoT
black-box
− 0.2
72%
− 0.4
40%
Appendix
Table 21: Pareto comparison with concurrent methods on shared benchmarks. DEER numbers from Yang et al. [2025b] Table 1 (Qwen3-14B as closest size match to our 8B/32B); our numbers from Tables 2 – 4 . “CR” = compression rate (lower = more reduction). Δ Acc = accuracy change vs vanilla full CoT.
Large Language Models often achieve strong performance by generating long intermediate chain-of-thought reasoning. However, it remains unclear when a model's final answer is actually determined during generation. If the answer is already fixed at an intermediate stage, subsequent reasoning tokens may constitute post-decision explanation, increasing inference cost and latency without improving correctness. We study the evolution of predicted answers over reasoning steps using forced answer completion, which elicits the model's intermediate predictions at partial reasoning prefixes. Focusing on Qwen3-4B and averaging results across all datasets considered, we find that predicted answers change in only 32% of queries. Moreover, once the final answer switch occurs, the model generates an average of 760 additional reasoning tokens per query, accounting for a substantial fraction of the total reasoning budget. Motivated by these findings, we investigate early stopping strategies that halt generation once the answer has stabilized. We show that simple heuristics, including probe-based stopping, can reduce reasoning token usage by 500 tokens per query while incurring only a 2% drop in accuracy. Together, our results indicate that a large portion of chain-of-thought generation is redundant and can be reduced with minimal impact on performance.
Ayan Datta, Zhixue Zhao, Bhuvanesh Verma +3
IIIT Hyderabad · University of Sheffield, UK · Goethe University, Frankfurt am Main, Germany
We provide evidence of performative chain-of-thought (CoT) in reasoning models, where a model becomes strongly confident in its final answer, but continues generating tokens without revealing its internal belief. Our analysis compares activation probing, early forced answering, and a CoT monitor across two large models (DeepSeek-R1 671B & GPT-OSS 120B) and find task difficulty-specific differences: The model's final answer is decodable from activations far earlier in CoT than a monitor is able to say, especially for easy recall-based MMLU questions. We contrast this with genuine reasoning in difficult multihop GPQA-Diamond questions. Despite this, inflection points (e.g., backtracking, 'aha' moments) occur almost exclusively in responses where probes show large belief shifts, suggesting these behaviors track genuine uncertainty rather than learned "reasoning theater." Finally, probe-guided early exit reduces tokens by up to 80% on MMLU and 30% on GPQA-Diamond with similar accuracy, positioning attention probing as an efficient tool for detecting performative reasoning and enabling adaptive computation.
Chain-of-thought (CoT) reasoning improves large reasoning models (LRMs) on complex tasks but often produces long, redundant traces. Recent training-free early-exit methods shorten these traces by choosing an intermediate point to stop reasoning. We study one such strategy that injects an end-of-think token (EoT, </think>) at this point to trigger the reasoning-to-answering transition, and find that the injected EoT does not always induce a clean answering phase. Answering-phase generation can continue before the model regenerates another EoT, with the span preceding this regenerated EoT scaling with the reasoning tokens saved by early exit and exhibiting continued reasoning behavior. We call this spurious CoT termination, where reasoning-like generation continues into the answering phase. We hypothesize that insufficient attention to the injected EoT contributes to spurious CoT termination and probe this hypothesis with Exit-token Attention Biasing (EAB). Across four LRMs, five benchmarks, and two early-exit methods, increasing attention to the injected EoT reduces spurious CoT termination and answering-phase length. These results reveal a limitation of controlling LRMs by externally matching their explicit think-block format. Inserting the EoT token conforms to this format but does not by itself guarantee the intended reasoning-to-answering transition. Our code is available at https://github.com/Seunghee-Koh/Spurious-CoT-Termination.
Seunghee Koh, Sungjae Choi, Minchan Kwon +2
Korea Advanced Institute of Science and Technology, South Korea