cs.CLOct 13, 2026

DoubtLess: Training-Free Contrastive Decoding for Efficient Reasoning

Authors: Minghao Luo, Liang Chen

Organizations: The Chinese University of Hong Kong · EPFL

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Reasoning models achieve strong performance through long chains of thought (CoT), but often spend more tokens than needed to solve a problem. Existing ways to shorten the CoT require additional training for each model, add a fixed steering vector to the model's hidden states, or use decoding heuristics such as stopping early or banning words like "wait", which can reduce accuracy. We propose DoubtLess, a training-free decoding method in which a much smaller reasoning model steers the large one towards brevity. The small model reads the large model's CoT so far twice, once prompted to be brief (the brief thinker) and once prompted to be verbose (the verbose thinker), and the difference between its two logit vectors, scaled by a strength, is added to the large model's logits at every step of the CoT. On seven benchmarks of competition mathematics, graduate-level science and competitive programming, DoubtLess shortens the CoT of Qwen3-32B by 31% with unchanged accuracy, outperforming five inference-time baselines. We also show that DoubtLess reduces thinking tokens per turn by 20% in a self-debugging coding agent with unchanged task performance. It also generalizes to two other model families with unchanged accuracy. When combined with training-based compression, it further shortens the CoT of a model already trained for brevity. Ablations and behavioral analysis show that DoubtLess reduces repeated checking, with most token savings coming after the correct answer first appears in the CoT. Our code is available at https://github.com/leoluolol/DoubtLess.

Figures & tables

Figure 1: Qwen3-32B: mean thinking tokens of a correct run before and after the correct answer first appears in the CoT, under plain decoding (the model as released) and under DoubtLess. Of the tokens DoubtLess removes, 65–89% come after the answer (Table 11 ).
Figure 2: DoubtLess. A small model that shares the tokenizer reads the large model’s CoT so far twice, once prompted to be brief (brief thinker) and once prompted to be verbose (verbose thinker); the difference between the two logit vectors, scaled by α\alpha , is added to the large model’s logits while it writes the CoT.
Figure 3: Before the answer, the direction leaves the CoT nearly untouched. After it, the words that open another check are pushed down. The CoT that plain decoding writes for AIME 2025 problem 1, read by Qwen3-0.6B under both instructions. Red : down-weighted; green : up-weighted (threshold in Appendix I ); [. . .] marks omitted text.
  MathematicsScienceCodeAgentOverall
  AIME 2024AIME 2025HMMT 2025AMC 2023Olympiad- BenchGPQA- DiamondLiveCode- BenchHumanEval+ agentAll seven
 Acc.81.172.253.396.770.768.766.388.470.8
Plain decodingTokens12,15215,60616,8176,13012,2467,35411,5112,71810,254
 Acc. change+2.2+2.2+3.3+3.3−6.7-6.7−0.8-0.8−2.0-2.0−1.0-1.0−0.7-0.7+1.4+1.4−1.0-1.0
Concise promptToken change−30%-30\%−28%-28\%−13%-13\%−31%-31\%−25%-25\%−40%-40\%−8%-8\%−13%-13\%−25%-25\%
 Acc. change−3.3-3.3−14.4-14.4−6.7-6.7−2.5-2.5−1.0-1.0−0.3-0.3−4.0-4.0−0.8-0.8−2.7-2.7
Table 1: DoubtLess removes the most thinking tokens overall. Qwen3-32B, three seeds per problem. Plain decoding: accuracy (%) and mean thinking tokens; other rows: change against plain decoding, in points and in % (intervals in Table 7 ). Overall: the 528 problems of the seven benchmarks together. Bold and underlined: largest and second-largest removal among the six methods. Agent: pass rate and thinking tokens per turn on HumanEval+.
Figure 4: DoubtLess removes the most thinking tokens with unchanged accuracy. (a) Percentage of thinking tokens removed and accuracy change against plain decoding, for Qwen3-32B on the seven benchmarks. (b) When evaluated on other model families, DoubtLess removes far more than the concise prompt (AIME 2025 and AMC 2023, each large model steered by a small model of its own family). Three seeds per problem, 95% intervals (Tables 7 and 8 ).
Figure 5: DoubtLess removes mostly the thinking after the first answer, and the words of hesitation stay. AIME 2025. (a) Mean thinking tokens of a correct run before (light) and after (dark) the first occurrence of the correct answer, per method; number: tokens after it. (b) Selected words per thousand thinking tokens.
Figure 6: After the answer, DoubtLess checks briefly and stops, while plain decoding gets stuck repeating the same check. AIME 2025 problem 1, first seed: the CoT under each method from the first statement of the answer (bold) to its end. Checks are in red, and [. . .] marks omitted text; full CoTs in Appendix L .
Figure 7: The two thinkers disagree after the answer. Correct plain-decoding runs on AIME 2025, read by the small model under both instructions. (a) Divergence of the two next-token distributions, aligned at the first automatic match to the correct answer. (b) Brief minus verbose log-probability of selected words, before and after the answer.
  MathematicsScienceAgent
  Olympiad- BenchGPQA- DiamondHumanEval+ agent
 Acc.64.765.591.9
LACONICTokens5,7183,4831,725
 Acc. change−1.3-1.3−1.7-1.7−1.2-1.2
  [−4.7,+1.7][-4.7,+1.7][−4.9,+1.5][-4.9,+1.5][−4.3,+1.6][-4.3,+1.6]
+DoubtLess (0.6B)Token change−42%-42\%−52%-52\%−29%-29\%
Table 2: DoubtLess further compresses a training-based compression model, Qwen3-32B fine-tuned with the adaptive length penalty of LACONIC (LACONIC-Qwen3-32B-3000). LACONIC: accuracy (%) and mean thinking tokens of this model; +DoubtLess: change in points (95% intervals) and in %. Three seeds per problem. Agent: pass rate and thinking tokens per turn.
Figure 8: Larger strengths remove more thinking tokens, and accuracy holds up to the default α=5\alpha{=}5 . AIME 2025, three seeds per problem, 95% intervals (Table 4 ).
Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Benchmarkn×n\times seedsBudgetAnswerLicense
AIME 202430×330\times 332kintegerMAA
AIME 202530×330\times 332kintegerMAA
HMMT Feb. 202530×330\times 332kexpressionHMMT
AMC 202340×340\times 332kintegerMAA
GPQA-Diamond198×3198\times 324kletterCC BY 4.0
OlympiadBench100×3100\times 332kexpressionApache 2.0
Appendix
Table 3: Benchmarks: problems ×\times seeds per problem, thinking budget in tokens, answer type and license.
SettingAcc.BudgetTokensToken changeAcc. change
α=−5\alpha{=}-573.321.119,676+26%+26\% [+17,+39][+17,+39]+1.1+1.1
Plain decoding72.213.315,606––
α=2.5\alpha{=}2.574.44.411,964−23%-23\% [−29,−17][-29,-17]+2.2+2.2
α=5\alpha{=}5 (default)72.21.111,092−29%-29\% [−37,−21][-37,-21]0.00.0
α=10\alpha{=}1064.43.39,999−36%-36\% [−44,−28][-44,-28]−7.8-7.8
Appendix
Table 4: AIME 2025, 30 problems, three seeds each: DoubtLess (0.6B) at three strengths and at a negative one ( α=−5\alpha{=}-5 adds the verbose minus the brief thinker). Accuracy, share of runs reaching the budget, mean thinking tokens, change in thinking tokens against plain decoding over all problems with its 95% interval, and accuracy change in points.
Small modelAcc.BudgetTokensToken changeAcc. change
Qwen3-0.6B72.21.111,092−29%-29\% [−37,−21][-37,-21]0.00.0
Qwen3-1.7B50.00.06,584−58%-58\% [−64,−51][-64,-51]−22.2-22.2
Qwen3-4B52.20.06,461−59%-59\% [−64,−53][-64,-53]−20.0-20.0
Qwen3-32B itself73.34.411,590−26%-26\% [−32,−20][-32,-20]+1.1+1.1
Appendix
Table 5: AIME 2025, three seeds: DoubtLess with three small models of the same family at α=5\alpha{=}5 , and with Qwen3-32B itself as both thinkers at α=1\alpha{=}1 . Accuracy, share of runs reaching the budget, mean thinking tokens, change in thinking tokens over all problems, and accuracy change in points.
SettingAcc.BudgetTokensToken changeAcc. change
Plain decoding68.98.913,543––
α=2.5\alpha{=}2.576.73.310,474−23%-23\% [−30,−16][-30,-16]+7.8+7.8
α=3.5\alpha{=}3.567.85.610,662−21%-21\% [−30,−13][-30,-13]−1.1-1.1
α=5\alpha{=}5 (default)72.24.49,605−29%-29\% [−38,−20][-38,-20]+3.3+3.3
α=7\alpha{=}768.93.39,275−32%-32\% [−40,−22][-40,-22]0.00.0
α=10\alpha{=}1064.42.29,135−33%-33\% [−42,−23][-42,-23]−4.4-4.4
Appendix
Table 6: AIME 2026 (30 problems, released after Qwen3), three seeds each: plain decoding, DoubtLess (0.6B) at five strengths and ASC at three. Accuracy, share of runs reaching the budget, mean thinking tokens, change in thinking tokens against plain decoding over all problems with its 95% interval, and accuracy change in points.
Figure 9: Thinking length under each method on each benchmark: box, quartiles; whiskers, 5th to 95th percentile; dot, mean; dotted line, the budget.
SettingAcc. (%)Acc. changeBudget (%)TokensToken change, allToken change, both solved
AIME 2024 (30 problems, 3 seeds each, budget 32,768)
Plain decoding81.1–3.312,152––
Concise prompt83.3+2.2+2.2 [−5.6,+8.9][-5.6,+8.9]1.18,517−30%-30\% [−38,−21][-38,-21]−28%-28\% [−37,−20][-37,-20] (20)
NoWait77.8−3.3-3.3 [−12.2,+4.4][-12.2,+4.4]2.29,575−21%-21\% [−32,−10][-32,-10]−26%-26\% [−40,−13][-40,-13] (18)
DoubtLess (0.6B)83.3+2.2+2.2 [−6.7,+11.1][-6.7,+11.1]0.08,816−27%-27\% [−37,−18][-37,-18]−34%-34\% [−41,−27][-41,-27] (18)
Random-target control81.10.00.0 [−7.8,+7.8][-7.8,+7.8]1.112,036−1%-1\% [−10,+9][-10,+9]+8%+8\% [−1,+18][-1,+18] (18)
Appendix
Table 7: Every setting on every benchmark: accuracy and its change against plain decoding in points (paired over problems), share of runs that reach the budget (for DEER and ThinkBrake, in the answer after the thinking is closed), mean thinking tokens, change in thinking tokens against plain decoding over all problems, and the same change over the problems that both the setting and plain decoding solved in every run (their number in parentheses). 95% bootstrap intervals over problems. The last block pools the seven benchmarks.
 PlainConciseDoubtLess
ModelAcc.Acc.TokensAcc.Tokens
Qwen3-32B86.287.1−29%-29\%86.7−31%-31\%
Qwen3-0.6B  [−36,−22][-36,-22] [−36,−25][-36,-25]
EXAONE-4.0-32B88.688.6−8%-8\%90.5−25%-25\%
EXAONE-4.0-1.2B  [−11,−4][-11,-4] [−31,−19][-31,-19]
Granite-4.2-30B89.088.1−14%-14\%91.4−40%-40\%
Appendix
Table 8: DoubtLess removes about three times as much thinking as the concise prompt on EXAONE-4.0 and Granite-4.2, with unchanged accuracy. Same instructions and strength for every family. AIME 2025 and AMC 2023, 70 problems, three seeds each; each large model is steered by the small model below it. Accuracy (%) and thinking-token change against plain decoding, the latter with 95% intervals.
SettingBeforeAfterShare afterRuns
Plain decoding7,6243,55332%65
Concise prompt6,0962,55130%68
NoWait5,4781,33420%52
DoubtLess (0.6B)6,0231,71022%65
Random-target control6,7163,85136%68
Appendix
Table 9: AIME 2025: mean thinking tokens before and after the first occurrence of the correct answer, over the correct runs in which the answer can be located (last column).
WordPlainConciseNoWaitDoubtLessRandom
Wait3.23.20.03.63.2
Hmm0.10.10.00.10.1
But2.72.70.03.52.8
Alternatively1.11.20.01.31.0
However1.51.40.01.31.6
verify0.20.20.00.20.2
Appendix
Table 10: AIME 2025: occurrences per thousand thinking tokens of some words, pooled over all runs of a setting.
  Plain decodingDoubtLess (0.6B)ChangeRemoved
BenchmarkRunsBeforeAfterBeforeAfterBeforeAfterafter answer
AIME 2024685,7513,3015,1861,474−10%-10\%−55%-55\%76%
AIME 2025566,3103,7005,1831,630−18%-18\%−56%-56\%65%
HMMT Feb. 2025194,0004,9733,5731,382−11%-11\%−72%-72\%89%
AMC 20231151,8803,9741,6662,330−11%-11\%−41%-41\%89%
GPQA-Diamond3462,8332,7991,9911,179−30%-30\%−58%-58\%66%
Appendix
Table 11: Plain decoding and DoubtLess (0.6B) on the same runs, those that both settings solve and in which the answer can be located (integer answers as a standalone number, letters as the answer of a phrase). Mean thinking tokens before and after the first occurrence of the correct answer under each setting, the change of each part, and the share of the removed tokens that lie after the answer. HMMT and OlympiadBench cover their integer-answer problems only; on LiveCodeBench the program is written after the CoT, so no split is possible.
Figure 10: DoubtLess removes a similar percentage on short and on long problems, and the random-target control stays on the diagonal. AIME 2025: mean thinking tokens of each problem over its three runs, under DoubtLess (0.6B) and the control against plain decoding. The dotted line is the overall change.
ProblemAnswerPlain decodingConcise promptNoWaitDoubtLess (0.6B)Random-target
1703,266 (3/3)1,532 (3/3)1,882 (3/3)1,479 (3/3)4,239 (3/3)
258812,881 (3/3)10,077 (3/3)8,088 (3/3)7,827 (3/3)11,541 (3/3)
3167,159 (3/3)4,369 (3/3)3,585 (3/3)3,726 (3/3)5,914 (3/3)
41178,367 (3/3)5,931 (3/3)4,924 (2/3)4,514 (3/3)8,085 (3/3)
527910,917 (3/3)6,712 (3/3)5,512 (2/3)8,070 (3/3)8,503 (3/3)
65043,163 (3/3)2,495 (3/3)2,158 (3/3)2,791 (3/3)2,781 (3/3)
Appendix
Table 12: AIME 2025, per problem: mean thinking tokens over the three seeds and (correct runs / runs) under each setting.
More in this topic

Efficient Language Model Reasoning

Efficient language model reasoning studies how to improve problem-solving accuracy while reducing inference, memory, latency, and training costs.

See all 369 papers

Explore similar work

CardsList
Oct 21, 2026

Mitigating Query Shortcuts in Chain-of-Thought with Context-Aware Decoding

While Chain-of-Thought (CoT) prompting improves the reasoning performance of large language models (LLMs), recent studies have shown that final answers do not always depend on the generated rationales. In particular, LLMs may rely on query-specific shortcuts rather than the information provided by their reasoning steps. In this paper, we investigate whether context-aware decoding (CAD), an existing contrastive decoding method, can mitigate such shortcut behavior when applied to CoT reasoning. We study this application as Rationale Contrastive Decoding (RCD), which contrasts the next-token distributions conditioned on the query with and without its rationale, amplifying tokens whose likelihood increases due to the rationale. We provide a theoretical interpretation of this decoding rule via a KL-regularized reward-maximization objective. Experiments across five reasoning benchmarks demonstrate that RCD improves answer accuracy over standard CoT. Furthermore, intervention analyses using ground-truth and random rationales reveal that RCD increases the behavioral dependence of final answers on the rationale content. Our findings suggest that contrastive decoding offers a simple, training-free approach to mitigating query shortcuts.

Shin'ya Yamaguchi

NTT

CoT ReasoningFaithfulness of Language Model ReasoningContrastive Decoding
AskPDF
Feb 20, 2026

Thinking by Subtraction: Confidence-Driven Contrastive Decoding for LLM Reasoning

Recent work on test-time scaling for large language model (LLM) reasoning typically assumes that allocating more inference-time computation uniformly improves correctness. However, prior studies show that reasoning uncertainty is highly localized: a small subset of low-confidence tokens disproportionately contributes to reasoning errors and unnecessary output expansion. Motivated by this observation, we propose Thinking by Subtraction, a confidence-driven contrastive decoding approach that improves reasoning reliability through targeted token-level intervention. Our method, Confidence-Driven Contrastive Decoding, detects low-confidence tokens during decoding and intervenes selectively at these positions. It constructs a contrastive reference by replacing high-confidence tokens with minimal placeholders, and refines predictions by subtracting this reference distribution at low-confidence locations. Experiments show that CCD significantly improves accuracy across mathematical reasoning benchmarks while substantially reducing output length, with minimal KV-cache overhead. As a training-free method, CCD enhances reasoning reliability through targeted low-confidence intervention without computational redundancy. Our code will be made available at: https://github.com/bolo-web/CCD.

Lexiang Tang, Weihao Gao, Bingchen Zhao +4

Peking University, Beijing, China · University of Edinburgh, Edinburgh, UK.

Efficient Language Model ReasoningLLM ReliabilityContrastive DecodingNumerical Reasoning in Language Models
AskPDF
May 29, 2026

Quantized Reasoning Models Think They Need to Think Longer, but They Do Not

Post-training quantization (PTQ) is widely used to deploy large language models efficiently, but its effect on reasoning models is not well understood. Across math, coding, and science QA, we find that aggressive PTQ reduces accuracy while increasing chain-of-thought (CoT) length. Surprisingly, we show that in up to 52% of the quantized models' failures, models reach the right answer in intermediate reasoning steps but do not output it as a final answer. To understand why quantization leads to this increase in overthinking errors, we measure the token-level KL divergence between quantized and full-precision output distributions. Positions with high KL divergence correlate strongly with high next-token entropy, and at these positions quantized models disproportionately sample overthinking markers such as "wait", "but", and "alternatively". We show that simply introducing a training-free logit penalty on a curated set of overthinking markers can reduce CoT length by 12--23% while preserving or improving accuracy across 5 models (1.5B-32B parameters), 3 quantization methods, and 5 benchmarks, yielding a favorable Pareto frontier of accuracy against reasoning cost compared to penalizing other token sets. Overthinking errors produced by quantized models are particularly reduced by up to 58%.

Sanae Lotfi, Polina Kirichenko, Steven Li +1

1FAIR at Meta · 2Meta AI

Logit AdjustmentOverthinking in Language ModelsCoT ReasoningEfficient Language Model ReasoningPost-Training QuantizationLLM Reasoning+1
AskPDF