Early Stopping for Large Reasoning Models via Confidence Dynamics

Parsa Hosseini; Sumit Nawathe; Mahdi Salmani; Meisam Razaviyayn; Soheil Feizi

cs.CLApr 6, 2026

Early Stopping for Large Reasoning Models via Confidence Dynamics

Authors: Parsa Hosseini, Sumit Nawathe, Mahdi Salmani, Meisam Razaviyayn, Soheil Feizi

Organizations: University of Maryland · University of Southern California

Abstract

Large reasoning models rely on long chain-of-thought generation to solve complex problems, but extended reasoning often incurs substantial computational cost and can even degrade performance due to overthinking. A key challenge is determining when the model should stop reasoning and produce the final answer. In this work, we study the confidence of intermediate answers during reasoning and observe two characteristic behaviors: correct reasoning trajectories often reach high-confidence answers early, while incorrect rollouts tend to produce long, unproductive reasoning traces and exhibit less reliable confidence dynamics. Motivated by these observations, we propose CoDE-Stop (Confidence Dynamics Early Stop), an early stopping method that leverages the dynamics of intermediate answer confidence to decide when to terminate reasoning, requiring no additional training and easily integrating into existing models. We evaluate CoDE-Stop on diverse reasoning and science benchmarks across multiple models. Compared to prior early stopping methods, it achieves a more favorable accuracy-compute tradeoff and reduces total token usage by 25-50% compared to standard full-length reasoning. In addition, we provide analyses of confidence dynamics during reasoning, offering insights into how confidence changes in both correct and incorrect trajectories.

Figures & tables

Figure 1: Accuracy vs. compute cost on Qwen3-4B averaged over 4 reasoning and science benchmarks. Left: reasoning length only. Right: total token compute including intermediate answer-generation overhead. CoDE-Stop achieves a stronger accuracy–compute tradeoff among early stopping methods.

Figure 2: Confidence dynamics across reasoning trajectories. Correct trajectories reach high confidence early, while incorrect trajectories exhibit unstable and fluctuating confidence.

Figure 3: Incorrect trajectories are longer and exhibit a heavy-tailed distribution.

Panel of Figure 4 — Figure 4: Confidence and degeneration dynamics over reasoning steps. Left : Average confidence for correct and incorrect rollouts; confidence increases even for incorrect cases, while early steps provide better separation. Right : Average degeneration score $D_{K}$ ; incorrect rollouts consistently exhibit higher values, with the gap increasing over time.

Figure 5: Performance of CoDE-Stop against different baselines across multiple benchmarks. CoDE-Stop consistently reduces inference cost while maintaining comparable performance to the baselines, making it Pareto optimal.

Figure 6: Performance of CoDE-Stop with different prompting baselines. CoDE-Stop can be combined with various prompting strategies to further improve performance.

Figure 7: CoDE-Stop reduces unnecessary computation on incorrect rollouts compared to baselines at matched accuracy.

Panel of Figure 8 — Figure 8: Ablation of degeneration score functions: $v_{i}$ (left) and $w_{i}$ (right)

Figure 9: Threshold sensitivity: $\tau$ controls a smooth accuracy–compute tradeoff.

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Method	AIME24,25				MATH500				GSM8K				GPQA-Diamond				Overall
Model	Method	Acc $\uparrow$	Tok $\downarrow$	CR $\downarrow$	Cost $\downarrow$	Acc $\uparrow$	Tok $\downarrow$	CR $\downarrow$	Cost $\downarrow$	Acc $\uparrow$	Tok $\downarrow$	CR $\downarrow$	Cost $\downarrow$	Acc $\uparrow$	Tok $\downarrow$	CR $\downarrow$	Cost $\downarrow$	Acc $\uparrow$	Tok $\downarrow$	CR $\downarrow$	Cost $\downarrow$
Qwen3-4B	Vanilla	69.1	16326	100.0	16326	94.0	5210	100.0	5210	94.8	2306	100.0	2306	55.8	9536	100.0	9536	78.4	8344	100.0	8344
	Think or Not	56.7	14130	86.5	27343	80.8	3768	72.3	6497	87.4	1618	70.2	2462	54.1	6865	72.0	10371	69.8	6595	75.3	11668
	DEER	68.1	13115	80.3	13400	93.7	3803	73.0	3878	94.8	1374	59.6	1400	53.0	6412	67.2	6496	77.4	6176	70.0	6294
	EAT	61.7	13995	85.7	14489	89.0	3642	69.9	3789	94.6	2092	90.7	2161	49.0	6459	67.7	6670	73.6	6547	78.5	6777
	RCPD	68.7	16325	100.0	16960	91.9	4994	95.9	5211	94.7	2140	92.8	2264	55.8	9424	98.8	9825	77.8	8221	96.9	8565

Appendix

Table 1: Main results comparing CoDE-Stop with baseline methods across models and benchmarks (B=32768). Acc denotes accuracy, Tok denotes average reasoning tokens, CR denotes compression rate, and Cost denotes total token compute.

Model	Method	AIME24,25				MATH500				GSM8K				GPQA-Diamond				Overall
Model	Method	Acc $\uparrow$	Tok $\downarrow$	CR $\downarrow$	Cost $\downarrow$	Acc $\uparrow$	Tok $\downarrow$	CR $\downarrow$	Cost $\downarrow$	Acc $\uparrow$	Tok $\downarrow$	CR $\downarrow$	Cost $\downarrow$	Acc $\uparrow$	Tok $\downarrow$	CR $\downarrow$	Cost $\downarrow$	Acc $\uparrow$	Tok $\downarrow$	CR $\downarrow$	Cost $\downarrow$
Qwen3-4B	Vanilla	62.6	12509	100.0	12509	93.4	4921	100.0	4921	95.2	2236	100.0	2236	55.9	9204	100.0	9204	76.8	7218	100.0	7218
	Think or Not	62.6	12515	100.0	19721	80.3	3490	70.9	5702	88.2	1555	69.5	2374	53.0	6621	71.9	9271	71.0	6045	78.1	9267
	DEER	61.9	10666	85.3	10756	93.2	3666	74.5	3720	95.2	1379	61.7	1405	53.5	6406	69.6	6467	76.0	5529	72.8	5587
	EAT	59.0	11437	91.4	11726	89.8	3635	73.9	3767	94.8	2037	91.1	2103	49.9	6599	71.7	6785	73.4	5927	82.0	6095
	RCPD	58.9	12509	100.0	13011	91.0	4718	95.9	4927	95.2	2073	92.7	2193	55.9	9105	98.9	9503	75.2	7101	96.9	7408

Appendix

Table 2: Results with B=16384. Hyperparameters were tuned on AMC23 and transferred to the presented benchmarks, in the same manner as the main results.

Model	AMC23	GSM8K	MATH500	GPQA	AIME
Qwen3-4B	9.6	2.5	5.4	12.6	14.4
Qwen3-14B		2.0	5.3	12.5	15.7
DeepSeek-R1-Llama-8B	7.5	8.3	22.3	70.4	98.8
Llama-3.1-8B	8.1	2.2	4.7	11.9	15.8

Appendix

Table 3: Average number of reasoning steps across models and benchmarks.

	$\delta=10^{-1}$	$\delta=10^{-3}$	$\delta=10^{-5}$	$\delta=10^{-7}$
(Acc, Cost)	(20.0, 2037)	(55.0, 9425)	(70.0, 10441)	(70.0, 11381)

Appendix

Table 4: Performance of EAT on a subset of AIME for different values of

\delta

	AIME		MATH500		GSM8K		GPQAD
Method	Acc.	Cost	Acc.	Cost	Acc.	Cost	Acc.	Cost
vanilla	$68.70\%$	$16329$	$94.10\%$	$5210$	$94.80\%$	$2306$	$55.20\%$	$9638$
think_or_not ( $\alpha=0.2$ )	$68.70\%$	$31706$	$94.20\%$	$9581$	$94.80\%$	$3846$	$55.60\%$	$14924$
think_or_not ( $\alpha=0.4$ )	$56.70\%$	$27343$	$80.80\%$	$5211$	$87.40\%$	$2462$	$54.10\%$	$10371$

Appendix

Table 5: We compare the performance of the Think-or-Not ( Yong et al., 2025 ) baseline with different entropy thresholds (

\alpha

) with Qwen3-4B. The Vanilla performance values are provided as a reference. We observe that

\alpha=0.2

is a very conservative choice; we use

\alpha=0.4

in the main experiments.

		AIME		MATH500		GSM8K		GPQAD
Delimiter	Method	Acc.	Cost	Acc.	Cost	Acc.	Cost	Acc.	Cost
Wait	DEER	$68.10\%$	$13400$	$93.70\%$	$3878$	$94.80\%$	$1400$	$53.00\%$	$6496$
Wait	CoDE-Stop	$67.80\%$	$12759$	$92.40\%$	$3463$	$94.50\%$	$1300$	$52.50\%$	$6283$
Altern.	DEER	$68.20\%$	$13621$	$93.70\%$	$3344$	$94.70\%$	$1173$	$52.60\%$	$5883$
Altern.	CoDE-Stop	$67.70\%$	$13234$	$93.20\%$	$3289$	$94.50\%$	$1133$	$52.70\%$	$5623$

Appendix

Table 6: Comparison of DEER and CoDE-Stop Methods with different delimiters for reasoning steps. CoDE-Stop maintains similar accuracy and cost reductions compared to this baseline for both potential delimiters, providing evidence for its robustness.

Figure 12: Hyperparameter tuning CoDE-Stop on AMC23 for Deepseek-R1-Llama-8B. For CoDE-Stop, we set $r_{\text{min}}=0.9$ , the number of ramping steps to be 2, and vary the degeneration threshold $\tau$ . For this model, we choose $\tau=35$ as it is the smallest threshold providing the highest accuracy in our range. Similar plots and tuning decisions were made for the other models we evaluate.

Model	AMC23	AIME	MATH500	GSM8K	GPQA-D
Qwen3-4b	3.5	5.3	2	0.9	4.6
Deepseek-R1-Llama-8B	35	90.7	20.5	7.6	64.7
Qwen3-14b	3.3	6.9	2.3	0.9	5.5
Nemotron-8B	8	15.6	4.6	2.2	11.8

Appendix

Table 7:

\tau

values used in hyperparameter-transfer experiments. Grid search was performed to obtain optimal

\tau

-values for AMC23. These values were scaled according to the average number of reasoning steps for each benchmark to obtain their respective thresholds.

Model	Benchmark	Tuned Acc	Tuned Cost	Transf. Acc	Transf. Cost	$\Delta$ Acc	$\Delta$ Cost
Qwen3-4B	AIME	67.8	12759	67.0	12837	-0.8	+78
	MATH500	92.4	3463	93.5	3696	+1.1	+233
	GSM8K	94.5	1300	94.6	1268	+0.1	-32
	GPQA	52.5	6283	52.4	6038	-0.1	-245
DeepSeek-R1	AIME	34.7	11608	34.6	11621	-0.1	+13
	MATH500	84.0	2657	83.4	2491	-0.6	-166

Appendix

Table 8: Robustness check: comparison between per-benchmark tuned thresholds (Tuned) and the AMC23-based transferred thresholds used in our experiments (Transf.). The transferred thresholds achieve nearly identical accuracy–compute tradeoffs.

Figure 13: Analysis comparing high and low degeneration score trajectories.

Figure 14: Qualitative example of a high-degeneration trajectory from AIME. The model repeatedly cycles through nearly identical reasoning patterns and revisits the same candidate solution without making meaningful progress, resulting in prolonged repetitive reasoning behavior.

Panel of Figure 15 — Figure 15: Confidence trajectories for DeepSeek (correct: top/blue, incorrect: bottom/orange; red dashed = early-stop point).

Panel of Figure 16 — Figure 16: Confidence trajectories for Nemotron . Same trend as DeepSeek: correct trajectories stabilize early; incorrect ones stay long and volatile.

Panel of Figure 17 — Figure 17: Reasoning-step count distributions. Correct trajectories (blue) are consistently shorter than incorrect ones (orange) across all settings; GSM8K is the shortest, AIME the longest.

Panel of Figure 18 — Figure 18: Average degeneration score $D_{K}$ vs. reasoning step. Incorrect trajectories (orange) degenerate faster than correct ones (blue) in every model/benchmark combination.

Panel of Figure 19 — Figure 19: Average confidence vs. reasoning step. Correct trajectories (blue) are high and well-separated early; the gap from incorrect (orange) shrinks at late steps, confirming early confidence is more discriminative.

Explore similar work

Stop When Reasoning Converges: Semantic-Preserving Early Exit for Reasoning Models
May 17, 2026Dehai Min, Giovanni Vaccarino, Huiyi Chen +3Early Stopping CoT Reasoning
University of Illinois Chicago · Politecnico di Milano · University of Illinois Urbana-Champaign +1
Learning to Stop without Learning to Stop: Self-Supervised Confidence Training Improves Reasoning Efficiency
Sep 25, 2026Parsa Hosseini, Akasha Tigalappanavara, Sumit Nawathe +6Efficient Language Model Reasoning Metacognition in Language Models
University of Maryland · AI Foundations, Capital One
Stop When Further Reasoning Won't Help: Attention-State Adaptive Generation in Reasoning Models
Jun 13, 2026Jiakai Li, Ke Qin, Rongzheng Wang +4Efficient Language Model Inference LLM Reasoning
University of Electronic Science and Technology of China, Chengdu, China · Ubiquitous Intelligence and Trusted Services Key Laboratory of Sichuan Province, Chengdu, China

May 17, 2026J/K move · Enter open · S save

Stop When Reasoning Converges: Semantic-Preserving Early Exit for Reasoning Models

Dehai Min, Giovanni Vaccarino, Huiyi Chen, Yongliang Wu+2

University of Illinois Chicago · Politecnico di Milano · University of Illinois Urbana-Champaign+1

Early Stopping CoT Reasoning Efficient Language Model Reasoning Efficient Language Model Inference LLM Reasoning Large Reasoning Models

Open paper AskPDF arXiv

Large Reasoning Models (LRMs) achieve strong performance by generating long chains of thought (CoT), but often overthink, continuing to reason after a solution has already stabilized and thereby wasting tokens and increasing latency. Existing inference-time early-exit methods rely primarily on answer-level signals, such as confidence or trial-answer consistency, to decide when to stop. However, these signals mainly reflect answer readiness rather than reasoning convergence: they may trigger before the model has finished exploring or self-correcting, causing premature exits that can degrade final-answer accuracy and leave the retained reasoning chain semantically incomplete. We identify reasoning-level semantic redundancy as a complementary signal for semantic-preserving early exit: when successive steps no longer add novel progress and instead revisit established conclusions, the reasoning trajectory has likely converged. Building on this insight, we propose PUMA, a plug-and-play framework that combines a lightweight Redundancy Detector with answer-level verification. The detector flags semantically redundant candidate exits, while verification confirms whether stopping is safe, allowing PUMA to remove redundant continuation while preserving both answer accuracy and a coherent reasoning prefix. Across five LRMs and five challenging reasoning benchmarks, PUMA achieves 26.2% average token reduction while preserving accuracy and retained CoT quality. Additional experiments on code generation, zero-shot vision-language reasoning, and learned stopping-policy internalization further demonstrate that reasoning-level redundancy is a robust, transferable, and learnable signal for efficient reasoning. Our code is available at https://github.com/giovanni-vaccarino/PUMA.

Abstract

Explore similar work

Stop When Reasoning Converges: Semantic-Preserving Early Exit for Reasoning Models

Learning to Stop without Learning to Stop: Self-Supervised Confidence Training Improves Reasoning Efficiency

Stop When Further Reasoning Won't Help: Attention-State Adaptive Generation in Reasoning Models