CALM: Consistency-Aware Logit Masking in Decoding Space for Mitigating Overthinking of Large Reasoning Models

Shengjing Liu; Feiyang Li; Yuxuan Yang; Yile Wang; Hui Huang

cs.CLOct 19, 2026

CALM: Consistency-Aware Logit Masking in Decoding Space for Mitigating Overthinking of Large Reasoning Models

Authors: Shengjing Liu, Feiyang Li, Yuxuan Yang, Yile Wang, Hui Huang

Organizations: College of Computer Science and Software Engineering, Shenzhen University

Abstract

Large reasoning models (LRMs) have achieved strong performance on challenging mathematical and scientific tasks through long chain-of-thought reasoning, but they often suffer from overthinking with unnecessary reasoning steps. Existing truncation-based methods typically detect consecutive trial-answer agreement and force early exit, yet such local stopping signals do not necessarily indicate final-answer stability and can disrupt useful subsequent reflection or verification, potentially leading to performance degradation. We propose consistency-aware logit masking (CALM), a training-free decoding intervention for mitigating overthinking in LRMs. CALM first monitors trial answers at step boundaries and then applies logit masking to action transition point (ATP) tokens such as "Wait", "Alternatively", and "Hmm", suppressing redundant transitional reasoning while allowing the model to continue its thought process more naturally. Experiments across four advanced LRMs and five benchmarks show that CALM reduces token consumption by 18.6%-25.0% compared with full CoT reasoning while maintaining performance, and can even improve accuracy by 0.07%-1.48% by reducing redundant reasoning. LLM-as-a-Judge results also demonstrate that CALM provides more coherent and concise results.

Figures & tables

Figure 1: Comparison of (a) truncation-based method and (b) our method that allows continual generation with action transition point masking after answer agreement.

Figure 2: Pilot study on accuracy and tokens on continued reasoning trace after trial answer agreement for DeepSeek-R1-Distill-Qwen-7B on OlympiadBench.

Figure 3: The overall pipeline of CALM . A trial answer is probed at each step boundary. After the same trial answer remains identical for

k

consecutive steps, CALM lowers the logits of ATP tokens and continues reasoning.

Method	MATH-500		Olympiad		GPQA		AIME25		AMC23		Overall
Method	Acc $\uparrow$	Tok $\downarrow$	Acc $\uparrow$	Tok $\downarrow$	Acc $\uparrow$	Tok $\downarrow$	Acc $\uparrow$	Tok $\downarrow$	Acc $\uparrow$	Tok $\downarrow$	Acc $\uparrow$	Tok $\downarrow$	TE $\uparrow$
DeepSeek-R1-Distill-Qwen-7B
Full CoT	92.33	3,847	57.63	8,718	47.14	7,327	36.67	15,080	89.17	6,297	64.59	8,254	58.57
Ans. Conv.	63.60	1,006 (-73.8%)	31.46	1,490 (-82.9%)	26.77	348 (-95.3%)	14.44	3,020 (-80.0%)	44.17	1,293 (-79.5%)	36.09	1,432 (-82.7%)	34.75
Dynasor	85.13	2,655 (-31.0%)	51.41	5,820 (-33.2%)	34.18	1,802 (-75.4%)	34.44	13,202 (-12.5%)	81.67	4,348 (-31.0%)	57.37	5,565 (-32.6%)	52.95
TRACE	89.33	2,764 (-28.2%)	53.53	4,340 (-50.2%)	43.77	2,410 (-67.1%)	34.44	8,270 (-45.2%)	77.50	3,936 (-37.5%)	59.71	4,344 (-47.4%)	56.10

Table 2: Averaged results using three seeds across DeepSeek-R1 distillation models. Acc: accuracy. Tok: average token count. TE: token efficiency. Values in parentheses are token changes relative to Full CoT. The best and second-best accuracy and TE values among the efficient-reasoning methods are shown in bold and underlined , respectively; Full CoT is excluded from this ranking.

Figure 4: Comparison on right-to-wrong transition rates where a problem answered correctly by Full CoT is answered incorrectly by an efficient reasoning method.

Variant	MATH-500		Olympiad		GPQA		AIME25		AMC23		Overall
Variant	Acc $\uparrow$	Tok $\downarrow$	Acc $\uparrow$	Tok $\downarrow$	Acc $\uparrow$	Tok $\downarrow$	Acc $\uparrow$	Tok $\downarrow$	Acc $\uparrow$	Tok $\downarrow$	Acc $\uparrow$	Tok $\downarrow$	TE $\uparrow$
Full CoT	91.40	3,820	57.93	8,677	45.96	7,486	40.00	14,889	87.50	6,579	64.56	8,290	58.52
$\rho=0.80$	92.00	2,977 (-22.1%)	57.04	6,863 (-20.9%)	47.98	4,955 (-33.8%)	40.00	13,641 (-8.4%)	87.50	5,247 (-20.2%)	64.90	6,737 (-18.7%)	59.75
$\rho=0.90$	91.80	3,019 (-21.0%)	58.37	6,661 (-23.2%)	50.00	5,084 (-32.1%)	36.67	14,317 (-3.8%)	87.50	5,286 (-19.7%)	64.87	6,873 (-17.1%)	59.64
$\rho={\text{0}.95}$	91.00	3,165 (-17.1%)	56.89	7,067 (-18.6%)	43.43	5,527 (-26.2%)	33.33	14,149 (-5.0%)	85.00	5,165 (-21.5%)	61.93	7,015 (-15.4%)	56.61
$\rho=\textbf{0.98}$	92.40	3,197 (-16.3%)	59.70	7,061 (-18.6%)	48.99	5,270 (-29.6%)	46.67	11,976 (-19.6%)	87.50	4,972 (-24.4%)	67.05	6,495 (-21.7%)	62.05

Table 4: Comparison of different soft progressive penalty factors using DeepSeek-R1-Distill-Qwen-7B .

Figure 5: Accuracy (blue) and tokens (red) against window length

k

with CALM (Hard). The line is the mean over three seeds.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Method	MATH-500		Olympiad		GPQA		AIME25		AMC23		Overall
Method	Acc $\uparrow$	Tok $\downarrow$	Acc $\uparrow$	Tok $\downarrow$	Acc $\uparrow$	Tok $\downarrow$	Acc $\uparrow$	Tok $\downarrow$	Acc $\uparrow$	Tok $\downarrow$	Acc $\uparrow$	Tok $\downarrow$	TE $\uparrow$
Llama-3.1-Nemotron-Nano-8B
Full CoT	94.07	3,085	63.95	6,923	47.98	6,973	51.11	11,860	94.17	5,258	70.26	6,820	65.06
Ans. Conv.	57.20	804 (-73.9%)	25.28	1,056 (-84.7%)	16.67	336 (-95.2%)	4.44	1,183 (-90.0%)	28.33	807 (-84.7%)	26.39	837 (-87.7%)	25.59
Dynasor	93.00	3,288 (+6.6%)	63.95	7,081 (+2.3%)	46.97	7,664 (+9.9%)	45.56	12,651 (+6.7%)	93.33	5,449 (+3.6%)	68.56	7,227 (+6.0%)	63.12
TRACE	90.67	2,296 (-25.6%)	59.16	3,734 (-46.1%)	37.21	2,313 (-66.8%)	35.56	6,814 (-42.5%)	85.83	3,500 (-33.4%)	61.69	3,731 (-45.3%)	58.52

Appendix

Table 6: Averaged results over three seeds across additional reasoning models. Acc: accuracy, Tok: token count, TE: token efficiency. Values in parentheses are relative changes in tokens from Full CoT. The best and second-best accuracy and TE values among the efficient-reasoning methods are shown in bold and underlined , respectively; Full CoT is excluded from this ranking.

Model	Benchmark	CALM	Ans. Conv.	Dynasor	PUMA	DEER
DeepSeek-R1-Distill-Qwen-7B	MATH-500 (500)	421.7	458.3	352.0	307.0	397.3
	Olympiad (675)	634.0	649.3	520.7	464.0	420.3
	GPQA (198)	180.7	198.0	182.3	49.7	0.0
	AIME25 (30)	28.0	30.0	27.0	11.7	9.7
	AMC23 (40)	36.7	39.3	33.7	22.3	27.3
DeepSeek-R1-Distill-Qwen-14B	MATH-500 (500)	414.7	483.3	223.0	288.3	420.7

Appendix

Table 7: Questions stopped, or intervened on by CALM , as the mean over three seeds. The parenthetical value is the number of questions. CALM counts four consecutive identical trial answers. The other columns count questions on which that method’s stopping rule fires.

Figure 6: Right-to-wrong rates on MATH-500. Panels pool three seeds and show DeepSeek-R1-7B, DeepSeek-R1-14B, Qwen3-4B, and Qwen3-30B-A3B from left to right. A right-to-wrong transition occurs when full CoT answers a problem correctly and the method answers it incorrectly. CALM is the lowest of the four methods on each panel.

Panel of Figure 7 — Figure 7: Gray is the shared prefix through the first same-answer window, and CALM continues in green after that window. The blue box opens with </think> . Both close at 16.

Panel of Figure 8 — Figure 8: Gray is the shared prefix through the first same-answer window, and CALM continues in green after that window. The blue box opens with </think> . Both close at $-\sqrt{3}$ .

Panel of Figure 9 — Figure 9: CALM is gray before the window and green after it, and DEER stays gray. The blue box opens with </think> . CALM closes at $-2$ , and DEER closes at $(1+\sqrt{5})/2$ .

Panel of Figure 10 — Figure 10: CALM is gray before the window and green after it, and DEER stays gray. The blue box opens with </think> . CALM closes at 28, and DEER closes at 62.

Overthinking in Language Models

Overthinking in language models studies when extra reasoning becomes redundant or harmful, and how to control it without losing accuracy.

See all 68 papers

Thinking Past the Answer: Evaluating Harmful Overthinking in Large Reasoning Models
Jun 1, 2026Simone Caldarella, Davide Talon, Rahaf Aljundi +2LLM Evaluation Overthinking in Language Models
University of Trento · 3Fondazione Bruno Kessler · 2Toyota Motor Europe
Thinking by Subtraction: Confidence-Driven Contrastive Decoding for LLM Reasoning
Feb 20, 2026Lexiang Tang, Weihao Gao, Bingchen Zhao +4Efficient Language Model Reasoning LLM Reliability
Peking University, Beijing, China · University of Edinburgh, Edinburgh, UK.
Mitigating Overthinking in Large Reasoning Language Models via Reasoning Path Deviation Monitoring
Mar 15, 2026Weixin Guan, Liang Li, Jiapeng Liu +6Overthinking in Language Models Efficient Language Model Reasoning
Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · School of Cyberspace Security, University of Chinese Academy of Sciences, Beijing, China · School of Computer and Artificial Intelligence, Jiangxi University of Finance and Economics, Jiangxi, China +1

Jun 1, 2026J/K move · Enter open · S save

Thinking Past the Answer: Evaluating Harmful Overthinking in Large Reasoning Models

Simone Caldarella, Davide Talon, Rahaf Aljundi, Elisa Ricci+1

University of Trento · 3Fondazione Bruno Kessler · 2Toyota Motor Europe

LLM Evaluation Overthinking in Language Models Reasoning Evaluation LLM Reliability Large Reasoning Models Reasoning Trace Analysis

Open paper AskPDF arXiv

Large Reasoning Models (LRMs) improve performance by generating explicit intermediate reasoning traces through increased test-time compute, yet the assumption that longer reasoning is consistently beneficial remains under-examined. While recent evidence shows that additional reasoning can lead models to overthink, we ask: "Once a model has reached the correct answer, does further reasoning refine the solution, or deviate from it?" To study the dynamics after correctness, we introduce a prefix-level trajectory evaluation protocol grounded in reasoning sufficiency, defining the minimum reasoning budget required for a model to first generate the correct answer. This allows us to disentangle verbose overthinking, where additional reasoning is redundant but harmless, from harmful overthinking, where continued reasoning destabilizes an already-correct trajectory. Starting from multimodal benchmarks, we find that many instances considered reasoning-intensive require surprisingly little reasoning. Moreover, stopping at the first correct prefix improves accuracy over standard reasoning up to 21%, revealing that current models are limited not only by their ability to reason, but also by their inability to stop at the right time. Furthermore, while common efficiency strategies like early stopping substantially reduce verbose overthinking (up to 50%), they fail to mitigate harmful overthinking. Failure analysis reveals that correctness deviations are mainly driven by logical drift and visual reinterpretation. Finally, we show that our findings generalize to language-only reasoning benchmarks, highlighting harmful overthinking as a broader reliability risk. Code available at https://simonecaldarella.github.io/thinking-past-the-answer.

Abstract

Overthinking in Language Models

Explore similar work

Thinking Past the Answer: Evaluating Harmful Overthinking in Large Reasoning Models

Thinking by Subtraction: Confidence-Driven Contrastive Decoding for LLM Reasoning

Mitigating Overthinking in Large Reasoning Language Models via Reasoning Path Deviation Monitoring