Draft-Thinking: Learning Efficient Reasoning in Long Chain-of-Thought LLMs

Jie Cao; Tianwei Lin; Juekai Lin; Zhenxuan Fan; Yunqi Cao; Bo Yuan; Ziyuan Zhao; Rolan Yan

cs.AIFeb 28, 2026

Draft-Thinking: Learning Efficient Reasoning in Long Chain-of-Thought LLMs

Authors: Jie Cao, Tianwei Lin, Juekai Lin, Zhenxuan Fan, Yunqi Cao, Bo Yuan, Ziyuan Zhao, Rolan Yan, +2 more

Organizations: Zhejiang University · Tencent

Abstract

Long chain-of-thought~(CoT) reasoning improves the problem-solving capability of large reasoning models~(LRMs), but often incurs substantial inference cost and overthinking. Existing efficiency methods primarily rely on prompting, post-hoc compression, or length-regularized optimization; these approaches can be sensitive to model scale, depend on external estimators, or under-allocate reasoning to difficult instances. We propose Draft-Thinking, a lightweight adaptation framework that learns a model-adapted draft CoT: a compact, self-contained working derivation that directly supports the final answer. We identify a Cross-Scale Draft Mismatch: concise traces generated by a strong teacher can be too sparse for a smaller target model. To address this mismatch, Progressive Draft Adaptation first transfers a concise reasoning-form prior through supervised fine-tuning (SFT) and then applies two-stage reinforcement learning (RL) with progressively harder data and relaxed response budgets to calibrate the reasoning granularity to the target model. The method requires neither post-hoc trajectory compression nor an explicit length-penalty reward. At inference time, draft, adaptive, and step-by-step prompts provide efficiency-oriented, balanced, and detailed reasoning modes, respectively. The step-by-step mode largely preserves the original model's accuracy while using fewer tokens. With only 847 training problems and 120 RL optimization steps, Draft-Thinking substantially reduces reasoning cost across six mathematical and scientific reasoning benchmarks. On MATH500, its draft mode reduces the average response length from 5,668 to 986 tokens, an 82.6% reduction, while achieving 90.6% accuracy compared with 93.0% for the original long CoT model.

Figures & tables

Figure 1: Overview of Draft-Thinking. (A) Cross-Scale Draft Mismatch occurs when a strong teacher’s concise trace omits intermediate transitions required by a smaller target model. (B) Progressive Draft Adaptation initializes a concise reasoning form through SFT and then calibrates the draft’s reasoning granularity through two correctness-driven RL stages: tight-budget adaptation and hard-task adaptation. (C) The adapted model supports draft, adaptive, and step-by-step prompts as parallel inference modes that prioritize token efficiency, provide a balanced trade-off, and elicit detailed long CoT reasoning, respectively.

Method	MATH500			AIME2025			LiveMathBench			OlympiadBench
Method	Acc	Tokens	EFF	Acc	Tokens	EFF	Acc	Tokens	EFF	Acc	Tokens	EFF
Qwen3-8B
Original	93.00	5668	1.64	64.38	18034	0.36	32.17	17493	0.18	64.44	11887	0.54
SFT-only	91.00	4364	2.08	49.58	13129	0.37	26.17	13160	0.19	57.33	10424	0.55
GRPO-only	93.60	3074	3.04	53.96	11720	0.46	28.67	11043	0.26	65.48	6902	0.94
ThinkPrune	94.40	2615	3.61	48.13	9781	0.49	28.67	9333	0.31	65.63	5809	1.13

Table 1: Controlled comparison of draft-mode reasoning efficiency on four mathematical reasoning benchmarks using Qwen3-8B. The highest and second-highest EFF values are shown in bold and underlined , respectively. Draft-Thinking is evaluated under the draft prompt

\boldsymbol{p}_{\mathrm{draft}}

Method	Minerva			GPQA-D
Method	Acc	Tokens	EFF	Acc	Tokens	EFF
Qwen3-8B
Original	52.21	7640	0.68	55.05	11554	0.47
SFT-only	47.06	7349	0.64	37.37	16367	0.23
GRPO-only	53.68	3822	1.40	60.61	6322	0.96
ThinkPrune	54.04	2725	1.98	60.10	5166	1.16

Table 2: Controlled comparison on two transfer benchmarks using Qwen3-8B.

Method

Prompt

MATH500

AIME2025

LiveMathBench

OlympiadBench

Minerva

GPQA-D

Acc/Tokens

EFF

Acc/Tokens

EFF

Acc/Tokens

EFF

Acc/Tokens

EFF

Acc/Tokens

EFF

Acc/Tokens

EFF

Qwen3-8B

Original

step

93.00/5668

1.64

64.38 /18034

0.36

32.17 /17493

0.18

64.44/11887

0.54

52.21 /7640

0.68

55.05 /11554

0.47

Draft-Thinking

step

93.60 /4271

2.19

64.79 /14112

0.46

30.83/13980

0.22

66.37 /9519

0.69

53.68 /5180

1.03

63.13 /6620

0.95

adaptive

93.20 /2755

3.38

56.67/12155

0.46

31.83 /10341

0.31

69.19 /6849

1.01

48.53/2979

1.62

52.02/4800

1.08

draft

90.60/986

9.18

45.42/6527

0.69

22.33/6476

0.35

61.19/3433

1.78

46.32/960

4.82

54.04/2689

2.01

Table 3: Comparison of three prompt-controlled inference modes of the same adapted Qwen3-8B model. The draft, adaptive, and step-by-step rows share identical model parameters and differ only in the inference prompt.

Method

Prompt

MATH500

AIME2025

LiveMathBench

OlympiadBench

Minerva

GPQA-D

Acc/Tokens

EFF

Acc/Tokens

EFF

Acc/Tokens

EFF

Acc/Tokens

EFF

Acc/Tokens

EFF

Acc/Tokens

EFF

Qwen3-8B

Original

step

93.00/5668

1.64

64.38/18034

0.36

32.17/17493

0.18

64.44/11887

0.54

52.21/7640

0.68

55.05/11554

0.47

Original

draft

95.00/3714

2.55

65.62/16254

0.40

27.00/17990

0.15

65.33/9968

0.65

51.47/4889

1.05

57.07/8143

0.70

D-SFT

draft

69.80/1732

4.02

9.17/3928

0.23

12.50/3732

0.33

36.89/3535

1.04

37.87/549

6.88

41.92/1948

2.15

D-SFT->D-RL6k

draft

93.00/1453

6.39

48.75/7426

0.65

26.33/6949

0.37

63.41/4340

1.46

49.26/1677

2.93

57.07/3742

1.52

Table 4: Ablation study of Draft-Thinking on Qwen3-8B. Original (step) is the unadapted long CoT reference, whereas Original (draft) applies

\boldsymbol{p}_{\mathrm{draft}}

without training. D-SFT uses draft-form SFT only; D-SFT

\rightarrow

D-RL6k omits tight-budget adaptation and performs all 120 RL steps with

L_{\max}=6{,}000

; D-SFT

\rightarrow

D-RL3k (60 steps) is the intermediate checkpoint after tight-budget adaptation; and Draft-Thinking is the full D-SFT

\rightarrow

D-RL3k (60 steps)

\rightarrow

D-RL6k (60 steps) pipeline.

Figure 2: Reasoning behavior comparison between original Qwen3-8B and Draft-Thinking on MATH500. Each bar represents the cumulative number of reasoning steps within a phase category.

Metric	Original	Draft	Step	Adaptive
Avg Steps	111	22	88	50
Avg Tokens per Step	51	45	48	55

Table 5: Average number of reasoning steps and average tokens per step on MATH500. We use double newlines (" \n\n ") as a step delimiter.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Prompt Method	Content
Long CoT	Please reason step by step, and put your final answer within \boxed{}.
Draft	Let’s think step by step, but only keep a minimum draft for each thinking step, with as few words as possible, and output the final answer within \boxed{}.
Instance adaptive	First, you should decide the thinking mode based on the question’s difficulty. The first mode is the normal way, where you can think in detail. The second mode is to keep only a minimal draft with as few words as possible. Then reason step by step, and output the final answer within \boxed{}.

Appendix

Table 6: Detailed designs of three prompts:

\boldsymbol{p}_{\text{step}}

for long CoT reasoning,

\boldsymbol{p}_{\text{draft}}

for draft reasoning, and

\boldsymbol{p}_{\text{adaptive}}

for instance-adaptive reasoning.

Figure 3: Example resoning trace of original Qwen3-8B and Draft thinking on a MATH500 question.

Parameter	D-RL3k	D-RL6k
datasets	$D_{\text{rl}}$	AIME2024
advantage estimator	grpo
train batch size	96	30
max prompt length	240	500
max response length	3000	6000
actor optim lr	1e-6

Appendix

Table 7: Main RL training configurations of Draft-Thinking on Qwen3-8B using the Verl framework. D-RL3k and D-RL6k denote the two progressive RL stages with maximum response lengths of 3,000 and 6,000 tokens, respectively.

Parameter	D-RL3k	D-RL6k
datasets	$D_{\text{rl}}$	AIME2024
advantage estimator	grpo
train batch size	96	30
max prompt length	240	500
max response length	3000	6000
actor optim lr	1e-6

Appendix

Table 8: Main RL training configurations of Draft-Thinking on Qwen3-4B using the Verl framework. D-RL3k and D-RL6k denote the two progressive RL stages with maximum response lengths of 3,000 and 6,000 tokens, respectively.

Parameter	SFT-only	Draft-SFT (8B)	Draft-SFT (4B)
Base model	Qwen3-8B	Qwen3-8B	Qwen3-4B
Training data	LIMO	$D_{\text{rl}}$	$D_{\text{rl}}$
Training samples	817	342	342
Max sequence length	10,000	6,000	6,000
Global batch size	60	60	60
Micro batch size/GPU	1	1	2

Appendix

Table 9: SFT training configurations. Draft-SFT uses the filtered DeepSeek-V3-generated draft reasoning dataset

D_{\text{rl}}

, while SFT-only uses the original LIMO solutions. The maximum sequence length refers to the combined prompt and response sequence.

Method	Prompt	Minerva			AIME2025			LiveMathBench			MATH500
Method	Prompt	Acc	Tokens	EFF	Acc	Tokens	EFF	Acc	Tokens	EFF	Acc	Tokens	EFF
Qwen3-4B
Original	step	51.47	7253	0.71	63.12	17703	0.36	29	16813	0.17	92.2	5505	1.67
D-SFT->D-RL3k->D-RL6k	draft	46.69	1421	3.28	46.04	7097	0.65	22.67	6051	0.38	92	1269	7.24
	step	48.9	4815	1.01	60.21	12885	0.46	26.67	12863	0.21	92.8	4130	2.24
	adaptive	48.9	3620	1.35	53.33	10048	0.53	24.17	8353	0.29	92.8	2292	4.04

Appendix

Table 10: Comparison of different training strategies on Qwen3-4B.

Figure 4: Comparison of average accuracy and response length across different difficulty levels on MATH500.

Figure 5: Average lengths of correct vs. wrong responses on MATH500.

Figure 6: Accuracy, token count, and token efficiency trends for three reasoning modes (draft, long CoT, and adaptive) across the three training stages of Draft-Thinking on Qwen3-8B.

Figure 7: Accuracy, token count, and token efficiency trends for three reasoning modes (draft, long CoT, and adaptive) across the three training stages of Draft Thinking on Qwen3-4B.

Figure 8: Chunked Symbolism Prompt Aytes et al. (2025) .

Figure 9: Reasoning behavior analysis prompt Hou et al. (2025) .

Explore similar work

OS-Pruner: Pruning Chains-of-Thought of Reasoning Models via Optimal Stopping
Jul 13, 2026Mohammed Ehab, Aymane El Gadarri, Vivek F. Farias +2Overthinking in Language Models Sequential Decision Making
MIT · Columbia Business School
Selective Latent Thinking: Adaptive Compression of LLM Reasoning Chains
May 25, 2026Hui Xie, Jie Liu, Ziyue Qiao +1Efficient Language Model Reasoning LLM Inference Acceleration
Eindhoven University of Technology, Netherlands · School of Computing and Information Technology, Great Bay University, China
SmartThinker: Progressive Chain-of-Thought Length Calibration for Efficient Large Language Model Reasoning
Mar 9, 2026Chenzhi Hu, Qinzhe Hu, Yuhang Xu +6Efficient Language Model Reasoning Large Reasoning Models
Shanghai Jiao Tong University · Beihang University

Jul 13, 2026J/K move · Enter open · S save

OS-Pruner: Pruning Chains-of-Thought of Reasoning Models via Optimal Stopping

Mohammed Ehab, Aymane El Gadarri, Vivek F. Farias, Adam Jozefiak+1

MIT · Columbia Business School

Overthinking in Language Models Sequential Decision Making Optimal Stopping Efficient Language Model Reasoning Efficient Language Model Inference Large Reasoning Models

Open paper AskPDF arXiv

Large Language Models (LLMs) have achieved remarkable success in complex reasoning tasks through Chain-of-Thought (CoT) prompting. However, these models often exhibit "computational overthinking," generating redundant reasoning steps that increase latency and cost without improving accuracy. Recent studies suggest that CoT trajectories can be significantly pruned, yet existing methods often rely on forcing a static thinking budget, heuristic filtering, sub-optimal early exit via classification, or expensive re-training. In this paper, we introduce OS-Pruner, a lightweight plug-in framework that formulates chain-of-thought pruning as an optimal stopping problem. Given a reasoning prefix, OS-Pruner learns whether further reasoning is worth its token cost by optimizing an explicit utility that trades off final-answer accuracy against generated length. Our novel formulation enables the model to dynamically assess the sufficient point of termination for a reasoning chain. OS-Pruner is designed to be lightweight during both training and inference, and to provide users with fine-grained control over the reasoning-effort vs. accuracy trade-off. On diverse reasoning benchmarks and base models, OS-Pruner achieves 20-60% reduction in generation length with minimal accuracy sacrifice.

Abstract

Explore similar work

OS-Pruner: Pruning Chains-of-Thought of Reasoning Models via Optimal Stopping

Selective Latent Thinking: Adaptive Compression of LLM Reasoning Chains

SmartThinker: Progressive Chain-of-Thought Length Calibration for Efficient Large Language Model Reasoning