DoubtLess: Training-Free Contrastive Decoding for Efficient Reasoning

Minghao Luo; Liang Chen

cs.CLOct 13, 2026

DoubtLess: Training-Free Contrastive Decoding for Efficient Reasoning

Authors: Minghao Luo, Liang Chen

Organizations: The Chinese University of Hong Kong · EPFL

Abstract

Reasoning models achieve strong performance through long chains of thought (CoT), but often spend more tokens than needed to solve a problem. Existing ways to shorten the CoT require additional training for each model, add a fixed steering vector to the model's hidden states, or use decoding heuristics such as stopping early or banning words like "wait", which can reduce accuracy. We propose DoubtLess, a training-free decoding method in which a much smaller reasoning model steers the large one towards brevity. The small model reads the large model's CoT so far twice, once prompted to be brief (the brief thinker) and once prompted to be verbose (the verbose thinker), and the difference between its two logit vectors, scaled by a strength, is added to the large model's logits at every step of the CoT. On seven benchmarks of competition mathematics, graduate-level science and competitive programming, DoubtLess shortens the CoT of Qwen3-32B by 31% with unchanged accuracy, outperforming five inference-time baselines. We also show that DoubtLess reduces thinking tokens per turn by 20% in a self-debugging coding agent with unchanged task performance. It also generalizes to two other model families with unchanged accuracy. When combined with training-based compression, it further shortens the CoT of a model already trained for brevity. Ablations and behavioral analysis show that DoubtLess reduces repeated checking, with most token savings coming after the correct answer first appears in the CoT. Our code is available at https://github.com/leoluolol/DoubtLess.

Figures & tables

Figure 1: Qwen3-32B: mean thinking tokens of a correct run before and after the correct answer first appears in the CoT, under plain decoding (the model as released) and under DoubtLess. Of the tokens DoubtLess removes, 65–89% come after the answer (Table 11 ).

Figure 2: DoubtLess. A small model that shares the tokenizer reads the large model’s CoT so far twice, once prompted to be brief (brief thinker) and once prompted to be verbose (verbose thinker); the difference between the two logit vectors, scaled by

\alpha

, is added to the large model’s logits while it writes the CoT.

Panel of Figure 3 — Figure 3: Before the answer, the direction leaves the CoT nearly untouched. After it, the words that open another check are pushed down. The CoT that plain decoding writes for AIME 2025 problem 1, read by Qwen3-0.6B under both instructions. Red : down-weighted; green : up-weighted (threshold in Appendix I ); [. . .] marks omitted text.

		Mathematics					Science	Code	Agent	Overall
		AIME 2024	AIME 2025	HMMT 2025	AMC 2023	Olympiad- Bench	GPQA- Diamond	LiveCode- Bench	HumanEval+ agent	All seven
	Acc.	81.1	72.2	53.3	96.7	70.7	68.7	66.3	88.4	70.8
Plain decoding	Tokens	12,152	15,606	16,817	6,130	12,246	7,354	11,511	2,718	10,254
	Acc. change	$+2.2$	$+3.3$	$-6.7$	$-0.8$	$-2.0$	$-1.0$	$-0.7$	$+1.4$	$-1.0$
Concise prompt	Token change	$-30\%$	$-28\%$	$-13\%$	$-31\%$	$-25\%$	$-40\%$	$-8\%$	$-13\%$	$-25\%$
	Acc. change	$-3.3$	$-14.4$	$-6.7$	$-2.5$	$-1.0$	$-0.3$	$-4.0$	$-0.8$	$-2.7$

Table 1: DoubtLess removes the most thinking tokens overall. Qwen3-32B, three seeds per problem. Plain decoding: accuracy (%) and mean thinking tokens; other rows: change against plain decoding, in points and in % (intervals in Table 7 ). Overall: the 528 problems of the seven benchmarks together. Bold and underlined: largest and second-largest removal among the six methods. Agent: pass rate and thinking tokens per turn on HumanEval+.

Figure 4: DoubtLess removes the most thinking tokens with unchanged accuracy. (a) Percentage of thinking tokens removed and accuracy change against plain decoding, for Qwen3-32B on the seven benchmarks. (b) When evaluated on other model families, DoubtLess removes far more than the concise prompt (AIME 2025 and AMC 2023, each large model steered by a small model of its own family). Three seeds per problem, 95% intervals (Tables 7 and 8 ).

Figure 5: DoubtLess removes mostly the thinking after the first answer, and the words of hesitation stay. AIME 2025. (a) Mean thinking tokens of a correct run before (light) and after (dark) the first occurrence of the correct answer, per method; number: tokens after it. (b) Selected words per thousand thinking tokens.

Panel of Figure 6 — Figure 6: After the answer, DoubtLess checks briefly and stops, while plain decoding gets stuck repeating the same check. AIME 2025 problem 1, first seed: the CoT under each method from the first statement of the answer (bold) to its end. Checks are in red, and [. . .] marks omitted text; full CoTs in Appendix L .

Figure 7: The two thinkers disagree after the answer. Correct plain-decoding runs on AIME 2025, read by the small model under both instructions. (a) Divergence of the two next-token distributions, aligned at the first automatic match to the correct answer. (b) Brief minus verbose log-probability of selected words, before and after the answer.

		Mathematics	Science	Agent
		Olympiad- Bench	GPQA- Diamond	HumanEval+ agent
	Acc.	64.7	65.5	91.9
LACONIC	Tokens	5,718	3,483	1,725
	Acc. change	$-1.3$	$-1.7$	$-1.2$
		$[-4.7,+1.7]$	$[-4.9,+1.5]$	$[-4.3,+1.6]$
+DoubtLess (0.6B)	Token change	$-42\%$	$-52\%$	$-29\%$

Table 2: DoubtLess further compresses a training-based compression model, Qwen3-32B fine-tuned with the adaptive length penalty of LACONIC (LACONIC-Qwen3-32B-3000). LACONIC: accuracy (%) and mean thinking tokens of this model; +DoubtLess: change in points (95% intervals) and in %. Three seeds per problem. Agent: pass rate and thinking tokens per turn.

Figure 8: Larger strengths remove more thinking tokens, and accuracy holds up to the default

\alpha{=}5

. AIME 2025, three seeds per problem, 95% intervals (Table 4 ).

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Benchmark	$n\times$ seeds	Budget	Answer	License
AIME 2024	$30\times 3$	32k	integer	MAA
AIME 2025	$30\times 3$	32k	integer	MAA
HMMT Feb. 2025	$30\times 3$	32k	expression	HMMT
AMC 2023	$40\times 3$	32k	integer	MAA
GPQA-Diamond	$198\times 3$	24k	letter	CC BY 4.0
OlympiadBench	$100\times 3$	32k	expression	Apache 2.0

Appendix

Table 3: Benchmarks: problems

\times

seeds per problem, thinking budget in tokens, answer type and license.

Setting	Acc.	Budget	Tokens	Token change	Acc. change
$\alpha{=}-5$	73.3	21.1	19,676	$+26\%$ $[+17,+39]$	$+1.1$
Plain decoding	72.2	13.3	15,606	–	–
$\alpha{=}2.5$	74.4	4.4	11,964	$-23\%$ $[-29,-17]$	$+2.2$
$\alpha{=}5$ (default)	72.2	1.1	11,092	$-29\%$ $[-37,-21]$	$0.0$
$\alpha{=}10$	64.4	3.3	9,999	$-36\%$ $[-44,-28]$	$-7.8$

Appendix

Table 4: AIME 2025, 30 problems, three seeds each: DoubtLess (0.6B) at three strengths and at a negative one (

\alpha{=}-5

adds the verbose minus the brief thinker). Accuracy, share of runs reaching the budget, mean thinking tokens, change in thinking tokens against plain decoding over all problems with its 95% interval, and accuracy change in points.

Small model	Acc.	Budget	Tokens	Token change	Acc. change
Qwen3-0.6B	72.2	1.1	11,092	$-29\%$ $[-37,-21]$	$0.0$
Qwen3-1.7B	50.0	0.0	6,584	$-58\%$ $[-64,-51]$	$-22.2$
Qwen3-4B	52.2	0.0	6,461	$-59\%$ $[-64,-53]$	$-20.0$
Qwen3-32B itself	73.3	4.4	11,590	$-26\%$ $[-32,-20]$	$+1.1$

Appendix

Table 5: AIME 2025, three seeds: DoubtLess with three small models of the same family at

\alpha{=}5

, and with Qwen3-32B itself as both thinkers at

\alpha{=}1

. Accuracy, share of runs reaching the budget, mean thinking tokens, change in thinking tokens over all problems, and accuracy change in points.

Setting	Acc.	Budget	Tokens	Token change	Acc. change
Plain decoding	68.9	8.9	13,543	–	–
$\alpha{=}2.5$	76.7	3.3	10,474	$-23\%$ $[-30,-16]$	$+7.8$
$\alpha{=}3.5$	67.8	5.6	10,662	$-21\%$ $[-30,-13]$	$-1.1$
$\alpha{=}5$ (default)	72.2	4.4	9,605	$-29\%$ $[-38,-20]$	$+3.3$
$\alpha{=}7$	68.9	3.3	9,275	$-32\%$ $[-40,-22]$	$0.0$
$\alpha{=}10$	64.4	2.2	9,135	$-33\%$ $[-42,-23]$	$-4.4$

Appendix

Table 6: AIME 2026 (30 problems, released after Qwen3), three seeds each: plain decoding, DoubtLess (0.6B) at five strengths and ASC at three. Accuracy, share of runs reaching the budget, mean thinking tokens, change in thinking tokens against plain decoding over all problems with its 95% interval, and accuracy change in points.

Figure 9: Thinking length under each method on each benchmark: box, quartiles; whiskers, 5th to 95th percentile; dot, mean; dotted line, the budget.

Setting	Acc. (%)	Acc. change	Budget (%)	Tokens	Token change, all	Token change, both solved
AIME 2024 (30 problems, 3 seeds each, budget 32,768)
Plain decoding	81.1	–	3.3	12,152	–	–
Concise prompt	83.3	$+2.2$ $[-5.6,+8.9]$	1.1	8,517	$-30\%$ $[-38,-21]$	$-28\%$ $[-37,-20]$ (20)
NoWait	77.8	$-3.3$ $[-12.2,+4.4]$	2.2	9,575	$-21\%$ $[-32,-10]$	$-26\%$ $[-40,-13]$ (18)
DoubtLess (0.6B)	83.3	$+2.2$ $[-6.7,+11.1]$	0.0	8,816	$-27\%$ $[-37,-18]$	$-34\%$ $[-41,-27]$ (18)
Random-target control	81.1	$0.0$ $[-7.8,+7.8]$	1.1	12,036	$-1\%$ $[-10,+9]$	$+8\%$ $[-1,+18]$ (18)

Appendix

Table 7: Every setting on every benchmark: accuracy and its change against plain decoding in points (paired over problems), share of runs that reach the budget (for DEER and ThinkBrake, in the answer after the thinking is closed), mean thinking tokens, change in thinking tokens against plain decoding over all problems, and the same change over the problems that both the setting and plain decoding solved in every run (their number in parentheses). 95% bootstrap intervals over problems. The last block pools the seven benchmarks.

Model	Acc.	Acc.	Tokens	Acc.	Tokens
	Plain	Concise		DoubtLess
Qwen3-32B	86.2	87.1	$-29\%$	86.7	$-31\%$
Qwen3-0.6B			$[-36,-22]$		$[-36,-25]$
EXAONE-4.0-32B	88.6	88.6	$-8\%$	90.5	$-25\%$
EXAONE-4.0-1.2B			$[-11,-4]$		$[-31,-19]$
Granite-4.2-30B	89.0	88.1	$-14\%$	91.4	$-40\%$

Appendix

Table 8: DoubtLess removes about three times as much thinking as the concise prompt on EXAONE-4.0 and Granite-4.2, with unchanged accuracy. Same instructions and strength for every family. AIME 2025 and AMC 2023, 70 problems, three seeds each; each large model is steered by the small model below it. Accuracy (%) and thinking-token change against plain decoding, the latter with 95% intervals.

Setting	Before	After	Share after	Runs
Plain decoding	7,624	3,553	32%	65
Concise prompt	6,096	2,551	30%	68
NoWait	5,478	1,334	20%	52
DoubtLess (0.6B)	6,023	1,710	22%	65
Random-target control	6,716	3,851	36%	68

Appendix

Table 9: AIME 2025: mean thinking tokens before and after the first occurrence of the correct answer, over the correct runs in which the answer can be located (last column).

Word	Plain	Concise	DoubtLess	Random
Wait	3.2	3.2	3.6	3.2
Hmm	0.1	0.1	0.1	0.1
But	2.7	2.7	3.5	2.8
Alternatively	1.1	1.2	1.3	1.0
However	1.5	1.4	1.3	1.6
verify	0.2	0.2	0.2	0.2

Appendix

Table 10: AIME 2025: occurrences per thousand thinking tokens of some words, pooled over all runs of a setting.

Benchmark	Runs	Before	After	Before	After	Before	After	after answer
		Plain decoding		DoubtLess (0.6B)		Change		Removed
AIME 2024	68	5,751	3,301	5,186	1,474	$-10\%$	$-55\%$	76%
AIME 2025	56	6,310	3,700	5,183	1,630	$-18\%$	$-56\%$	65%
HMMT Feb. 2025	19	4,000	4,973	3,573	1,382	$-11\%$	$-72\%$	89%
AMC 2023	115	1,880	3,974	1,666	2,330	$-11\%$	$-41\%$	89%
GPQA-Diamond	346	2,833	2,799	1,991	1,179	$-30\%$	$-58\%$	66%

Appendix

Table 11: Plain decoding and DoubtLess (0.6B) on the same runs, those that both settings solve and in which the answer can be located (integer answers as a standalone number, letters as the answer of a phrase). Mean thinking tokens before and after the first occurrence of the correct answer under each setting, the change of each part, and the share of the removed tokens that lie after the answer. HMMT and OlympiadBench cover their integer-answer problems only; on LiveCodeBench the program is written after the CoT, so no split is possible.

Figure 10: DoubtLess removes a similar percentage on short and on long problems, and the random-target control stays on the diagonal. AIME 2025: mean thinking tokens of each problem over its three runs, under DoubtLess (0.6B) and the control against plain decoding. The dotted line is the overall change.

Problem	Answer	Plain decoding	Concise prompt	NoWait	DoubtLess (0.6B)	Random-target
1	70	3,266 (3/3)	1,532 (3/3)	1,882 (3/3)	1,479 (3/3)	4,239 (3/3)
2	588	12,881 (3/3)	10,077 (3/3)	8,088 (3/3)	7,827 (3/3)	11,541 (3/3)
3	16	7,159 (3/3)	4,369 (3/3)	3,585 (3/3)	3,726 (3/3)	5,914 (3/3)
4	117	8,367 (3/3)	5,931 (3/3)	4,924 (2/3)	4,514 (3/3)	8,085 (3/3)
5	279	10,917 (3/3)	6,712 (3/3)	5,512 (2/3)	8,070 (3/3)	8,503 (3/3)
6	504	3,163 (3/3)	2,495 (3/3)	2,158 (3/3)	2,791 (3/3)	2,781 (3/3)

Appendix

Table 12: AIME 2025, per problem: mean thinking tokens over the three seeds and (correct runs / runs) under each setting.

Efficient Language Model Reasoning

Efficient language model reasoning studies how to improve problem-solving accuracy while reducing inference, memory, latency, and training costs.

See all 369 papers

Mitigating Query Shortcuts in Chain-of-Thought with Context-Aware Decoding
Oct 21, 2026Shin'ya YamaguchiCoT Reasoning Faithfulness of Language Model Reasoning
NTT
Thinking by Subtraction: Confidence-Driven Contrastive Decoding for LLM Reasoning
Feb 20, 2026Lexiang Tang, Weihao Gao, Bingchen Zhao +4Efficient Language Model Reasoning LLM Reliability
Peking University, Beijing, China · University of Edinburgh, Edinburgh, UK.
Quantized Reasoning Models Think They Need to Think Longer, but They Do Not
May 29, 2026Sanae Lotfi, Polina Kirichenko, Steven Li +1Logit Adjustment Overthinking in Language Models
1FAIR at Meta · 2Meta AI

Oct 21, 2026J/K move · Enter open · S save

Mitigating Query Shortcuts in Chain-of-Thought with Context-Aware Decoding

Shin'ya Yamaguchi

NTT

CoT Reasoning Faithfulness of Language Model Reasoning Contrastive Decoding

Open paper AskPDF arXiv

While Chain-of-Thought (CoT) prompting improves the reasoning performance of large language models (LLMs), recent studies have shown that final answers do not always depend on the generated rationales. In particular, LLMs may rely on query-specific shortcuts rather than the information provided by their reasoning steps. In this paper, we investigate whether context-aware decoding (CAD), an existing contrastive decoding method, can mitigate such shortcut behavior when applied to CoT reasoning. We study this application as Rationale Contrastive Decoding (RCD), which contrasts the next-token distributions conditioned on the query with and without its rationale, amplifying tokens whose likelihood increases due to the rationale. We provide a theoretical interpretation of this decoding rule via a KL-regularized reward-maximization objective. Experiments across five reasoning benchmarks demonstrate that RCD improves answer accuracy over standard CoT. Furthermore, intervention analyses using ground-truth and random rationales reveal that RCD increases the behavioral dependence of final answers on the rationale content. Our findings suggest that contrastive decoding offers a simple, training-free approach to mitigating query shortcuts.

Abstract

Efficient Language Model Reasoning

Explore similar work

Mitigating Query Shortcuts in Chain-of-Thought with Context-Aware Decoding

Thinking by Subtraction: Confidence-Driven Contrastive Decoding for LLM Reasoning

Quantized Reasoning Models Think They Need to Think Longer, but They Do Not