cs.AIApr 12, 2026
SaveWhen More Thinking Hurts: Overthinking in LLM Test-Time Compute Scaling
Organizations: Nanjing University · Baidu · Nanjing University of Finance & Economics
Abstract
Scaling test-time compute through extended chains of thought has become a dominant paradigm for improving large language model reasoning. However, existing research implicitly assumes that longer thinking always yields better results. This assumption remains largely unexamined. We systematically investigate how the marginal utility of additional reasoning tokens changes as compute budgets increase. We find that marginal returns diminish substantially at higher budgets and that models exhibit ``overthinking'', where extended reasoning is associated with abandoning previously correct answers. Furthermore, we show that optimal thinking length varies across problem difficulty, suggesting that uniform compute allocation is suboptimal. Our cost-aware evaluation framework reveals that stopping at moderate budgets can reduce computation significantly while maintaining comparable accuracy.
Figures & tables
Figure 1: Marginal utility diminishes with compute budget. (a) By problem difficulty: easier problems (Level 1-2) reach negative marginal utility earlier than hard problems (Level 5). The shaded region indicates where additional thinking hurts performance. (b) Model comparison: R1-32B maintains positive marginal utility longer than s1-32B, showing better resistance to overthinking. Shaded bands show standard deviation across difficulty levels.
Figure 2: Overthinking can flip correct answers to incorrect ones. (a) Accuracy trajectories for individual problems, showing cases where extended thinking leads to answer changes. The red “overthinking zone” highlights where negative flips become dominant. (b) Frequency of “negative flips” (correct incorrect) versus “positive flips” (incorrect correct) across compute budgets. The crossover at 7K marks where extended thinking becomes harmful on average. (c) Flip ratio by problem difficulty, showing that easier problems cross the overthinking threshold earlier.
| (a) MU / 500 tokens | (b) Accuracy | |||||
|---|---|---|---|---|---|---|
| Range | R1 | s1 | Bud. | R1 | s1 | R1 |
| 0.5–2K | +3.2 | +2.8 | 2K | 37.8 | 33.2 | – |
| 2–4K | +1.8 | +1.5 | 4K | 46.5 | 41.8 | +8.7 |
| 4–6K | +0.9 | +0.7 | 6K | 50.2 | 44.5 | +3.7 |
| 6–8K | +0.9 | +0.6 | 8K | 53.8 | 47.1 | +3.6 |
| 8–12K | +0.1 | 0.2 | 12K | 55.8 | 47.6 | +2.0 |
Table 1: Marginal utility and accuracy (%) on AIME. (a) MU diminishes with budget, turning negative beyond 12K. (b) Peak accuracy at 12K; R1 shows R1 accuracy change from previous budget. Baseline accuracy at 500 tokens is 28.2% (R1) and 24.8% (s1).
Figure 3: GPQA Diamond: Model Comparison. (a) Accuracy curves showing R1-32B consistently outperforming s1-32B. (b) Flip ratio (negative/positive) analysis illustrating the underlying mechanism of overthinking at extended compute budgets.
| Budget | Pos. | Neg. | Ratio |
|---|---|---|---|
| 1000 | 142 | 31 | 0.22 |
| 2000 | 118 | 38 | 0.32 |
| 4000 | 87 | 52 | 0.60 |
| 5000 | 78 | 55 | 0.71 |
| 6000 | 67 | 58 | 0.87 |
| 7000 | 55 | 60 | 1.09 |
Table 2: Cumulative flip events from each budget threshold on AIME (R1-32B). For each budget , we count all flips occurring in transitions from through 16K tokens; a single problem may contribute multiple flips across different transitions. Flip ratio 1 indicates overthinking; the crossover occurs at 7K tokens.
| Budget | Flip Ratio | 95% CI | -value |
|---|---|---|---|
| 2,000 | 0.32 | [0.24, 0.41] | – |
| 4,000 | 0.60 | [0.48, 0.73] | – |
| 5,000 | 0.71 | [0.57, 0.87] | – |
| 6,000 | 0.87 | [0.71, 1.05] | – |
| 7,000 | 1.09 | [1.01, 1.18] | 0.014 |
| 8,000 | 1.42 | [1.21, 1.68] | 0.002 |
Table 3: Bootstrap confidence intervals for flip ratios. -values test whether the ratio significantly exceeds 1.0 (one-sided test). The crossover (ratio 1.0) occurs at 7K tokens.
Figure 4: Flip Event Analysis: R1 vs. s1. (a) Flip event counts showing s1-32B crosses the negative-dominated threshold earlier ( 5K tokens) than R1-32B ( 7K tokens). (b) Flip ratio (negative/positive) comparison between the two models, highlighting s1-32B’s higher tendency to reverse correct answers.
| Indicator | Correlation | [email protected] |
|---|---|---|
| Hesitation markers | 0.71 | 64.2% |
| Answer oscillation | 0.78 | 71.5% |
| Confidence drop | 0.63 | 58.7% |
| Combined | 0.82 | 76.3% |
Table 4: Overthinking indicator effectiveness on AIME (R1-32B). Correlation with negative flips and precision at 80% recall.
| Budget | R1-32B | Flip Ratio | MU/500 |
|---|---|---|---|
| 2,000 | 41.4% | 0.28 | – |
| 4,000 | 48.2% | 0.51 | +1.7% |
| 6,000 | 52.5% | 0.74 | +1.1% |
| 8,000 | 54.8% | 0.93 | +0.6% |
| 10,000 | 55.6% | 1.18 | +0.2% |
| 12,000 | 54.9% | 1.67 | 0.2% |
Table 5: GPQA Diamond results (R1-32B). Diminishing returns and overthinking generalize to scientific reasoning. Peak accuracy at 10K tokens.
| Token Range | Natural | Forced |
|---|---|---|
| 6–8K | 54.2% | 53.8% |
| 8–10K | 52.1% | 52.4% |
| 10–12K | 49.8% | 50.1% |
| 12–16K | 47.3% | 48.2% |
| Flip ratio ( 10K) | 1.31 | 1.42 |
Table 6: Natural vs. forced long reasoning (R1-32B). Natural samples show similar accuracy decline, confirming overthinking is not a Budget Forcing artifact.
Figure 5: MATH-500: Difficulty-Stratified Analysis. (a) Accuracy by difficulty level. (b) Optimal budget varies 7.5 across difficulty levels. (c) Marginal utility by difficulty level across budgets.
| Category | Count | Percentage |
|---|---|---|
| (A) Genuine overthinking | 54 | 67.5% |
| (B) Exploration divergence | 16 | 20.0% |
| (C) Degradation artifacts | 10 | 12.5% |
Table 7: Qualitative analysis of negative flips. Most negative flips (67.5%) involve genuine overthinking where models explicitly abandon correct answers.
Figure 6: Cost-aware evaluation reveals optimal stopping points. (a) The Pareto frontier shows the accuracy-compute trade-off. Markers indicate optimal budgets under different values: at (cost-agnostic), peak accuracy budget is optimal (not maximum, due to overthinking); at (cost-sensitive), early stopping achieves higher utility. (b) Utility curves shift as cost sensitivity increases, with optimal stopping points moving leftward.
Figure 7: Early Stopping Validation. (a) The compute-accuracy trade-off for different stopping constraints. (b) Strategy comparison showing our combined approach achieves strong accuracy with significant compute savings.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
| Natural Length | N | Accuracy |
|---|---|---|
| 4K tokens | 89 | 71.9% |
| 4–8K tokens | 159 | 58.5% |
| 8–12K tokens | 198 | 51.0% |
| 12K tokens | 114 | 44.7% |
Table 8: Accuracy by natural output length. Longer natural outputs correlate with lower accuracy, suggesting the model writes more when uncertain.