Reasoning distillation aims to transfer multi-step reasoning capabilities from large language models to smaller, more efficient ones. While recent methods have shown promising gains, they typically rely on static teacher-student hierarchies and frame distillation as trajectory imitation. This is misaligned with the structure of reasoning, where intermediate steps are often locally under-specified: global correctness constrains the final answer, but does not uniquely determine each intermediate move. We propose validity-calibrated reasoning distillation, a framework that treats reasoning distillation as a problem of local learning-signal allocation rather than path alignment. Instead of enforcing token-level imitation, we compare the student's and teacher's proposed next-step actions under the same prefix and use their relative local validity to modulate the strength of the distillation update. This yields a dynamic, context-dependent supervision mechanism that preserves the teacher's structural guidance while adapting update strength to local reasoning quality. Across mathematical reasoning, code generation, and instruction-following benchmarks, our method consistently outperforms strong distillation baselines. These results indicate that effective LLM reasoning distillation is governed not by rigid trajectory imitation, but by principled, locally calibrated allocation of learning signal.
Figures & tables
Figure 1: Distribution of the reward ratio rs/rt between the student Qwen2.5-Math-1.5B ( rs ) and the teacher Qwen2.5-Math-7B-Instruct ( rt ) policies. rs and rt are computed with Skywork-o1-OpenPRM-Qwen-2.5-1.5B. While the ratio is expected to concentrate below 1, a substantial fraction of probability mass (28.8%) lies in the region ≥1 , indicating frequent cases where the student attains higher reward than the teacher.
Figure 2: Overview of VCRD. Rather than enforcing uniform trajectory imitation, VCRD allocates token-level learning signal based on the relative local validity of teacher and student proposals under the same prefix. By modulating update strength rather than direction, the method preserves teacher guidance while adapting supervision to locally under-specified reasoning steps.
Table 1: Pass@1 results for distilling Qwen2-Math-7B-Instruct ( MT ) into Qwen2-Math-1.5B ( MS ) on eight math-reasoning benchmarks. AVG is the average over all datasets.
Table 2: Pass@1 results for distilling Qwen2.5-Math-7B-Instruct ( MT ) into Qwen2.5-Math-1.5B ( MS ) across seven math-reasoning benchmarks. AVG denotes the average score across all datasets.
Table 3: Pass@1 results on HumanEval and MBPP benchmarks for two Qwen2.5-Coder distillation settings. AVG denotes the average Pass@1 across the four tasks.
Method
Qwen2.5-7B-Inst ( MT ) → Qwen2.5-1.5B ( MS )
Qwen2-7B-Inst ( MT ) → Qwen2-1.5B ( MS )
AlpacaEval
Evol-Inst
UltraFeed
AVG.
AlpacaEval
Evol-Inst
UltraFeed
AVG.
MT
91.43
69.67
77.16
79.42
88.54
59.75
64.86
71.05
MS
60.09
30.73
42.94
44.59
56.11
19.49
38.20
37.93
KD
72.14
54.59
62.60
63.11
60.81
32.45
49.25
47.50
SeqKD
67.92
46.33
52.89
55.71
56.58
28.32
39.10
41.33
DistilLLM
75.11
55.38
65.92
65.47
64.13
35.66
51.26
50.35
Table 4: Win Rate (WR%) on instruction-following datasets for Qwen2.5-7B-Instruct → Qwen2.5-1.5B and Qwen2-7B-Instruct → Qwen2-1.5B. AVG is the average WR across all benchmarks.
Table 5: Contribution of validity calibration across mathematical reasoning and code generation. SRKL ( wS=1 ) denotes pure on-policy distillation with student rollouts, per-token reverse-KL supervision, and no validity modulation. SKL+SRKL ( wT=wS=1 ) is the constant-weight mixed-prefix baseline.
Method
HEval
HEval +
AVG
MT
77.40
70.70
74.05
MS
35.40
29.30
32.35
KD
42.10
37.20
39.65
SeqKD
41.50
36.00
38.75
GKD
40.09
36.00
38.05
DistilLLM
42.10
38.40
40.25
Table 6: Pass@1 on HumanEval for DS-Coder-6.9B → DS-Coder-1.3B.
Method
GSM8K
MATH
Gaokao
Olympiad
AVG
Distillm-2
85.70
73.20
61.60
34.10
63.65
VCRD-Prob
84.80
73.90
64.20
33.60
64.13
Table 7: Pass@1 results on four math-reasoning benchmarks. AVG is the mean across the four tasks. VCRD-Prob represents the PRM-free approach. Qwen-2.5-Math-7B-Instruct → Qwen-2.5-Math-1.5B
Shared prefix
Teacher continuation
Student continuation
rT
rS
Ratio
Better Local continuation
The cost of printing is 7 copies ×25 pages ×0.10/page=$17$
550
.50
0.085
0.589
6.97
Student . The correct amount is $17.50; the student completes the decimal.
Tim buys 3 goats for $400 each. Tim also buys twice as many llamas as goats, so he buys
2×2=6 llamas
3×2=6 llamas
0.152
0.847
5.57
Student . The multiplier should be the number of goats, 3, so 3×2=6 .
Cost per liter is 3andthepoolvolumeis25\times 60x.Thus,3\times 25\times 60x=90000.Simplify:3\times$
15×60x
25×60x
0.575
0.801
1.39
Student . The coefficient 25 should be preserved from 3×25×60x .
Find 1234+46÷44 . First, we simplify
16÷44
46÷44
0.601
0.877
1.46
Student . The expression being simplified is 46÷44 , not 16÷44 .
James drives at 30 mph for half an hour. Then he drives at twice the speed for twice as long, which means he drives at a speed of
3×30=6⋯
2×30=6⋯
0.360
0.827
2.30
Student . “Twice the speed” requires multiplying by 2, giving 2×30=60 mph.
Table 8: Qualitative examples where the student Qwen2.5-Math-1.5B continuation receives higher local validity than the teacher Qwen2.5-Math-7B-Instruct continuation under the same prefix. rT and rS denote the PRM (Skywork-o1-OpenPRM-Qwen-2.5-1.5B) rewards for the teacher and student first token continuations, respectively, and the ratio is rS/rT . The “Better Local Continuation” column indicates which continuation is locally correct or a valid better alternative under the shared prefix between teacher and student models.
Hyperparameter
Value
Finetuning method
LoRA (rank = 16, α = 128, dropout = 0.05)
LoRA target modules
all self-attention and MLP layers
Learning rate
5×10−5
Max sequence length
1024
Max prompt length
512
Effective batch size
128
Table 9: Hyperparameter values used in VCRD (distillation stage) experiments across all task families.
Large language models (LLMs) demonstrate remarkable reasoning capabilities in tasks such as algorithmic coding and mathematical problem-solving. Recent methods have improved reasoning through expanded corpus and multistage training combining reinforcement learning and supervised fine-tuning. Although some methods suggest that small but targeted dataset can incentivize reasoning via only distillation, a reasoning scaling laws is still taking shape, increasing computational costs. To address this, we propose a data-efficient distillation framework (DED) that optimizes the Pareto frontier of reasoning distillation. Inspired by the on-policy learning and diverse roll-out strategies of reinforcement learning, the key idea of our approach is threefold: (1) We identify that benchmark scores alone do not determine an effective teacher model. Through comprehensive comparisons of leading reasoning LLMs, we develop a method to select an optimal teacher model. (2) While scaling distillation can enhance reasoning, it often degrades out-of-domain performance. A carefully curated, smaller corpus achieves a balanced trade-off between in-domain and out-of-domain capabilities. (3) Diverse reasoning trajectories encourage the student model to develop robust reasoning skills. We validate our method through evaluations on mathematical reasoning (AIME 2024/2025, MATH-500) and code generation (LiveCodeBench), achieving state-of-the-art results with only 0.8k carefully curated examples, bypassing the need for extensive scaling. Our systematic analysis demonstrates that DED outperforms existing methods by considering factors beyond superficial hardness, token length, or teacher model capability. This work offers a practical and efficient pathway to advanced reasoning while preserving general capabilities.
Xiaojun Wu, Xiaoguang Jiang, Huiyang Li +11
1Zhongxing Telecom Equipment(ZTE), China · 2China Mobile Communications Group Co Ltd
Reasoning capabilities are critical for advancing Large Language Models, yet current approaches either require massive computational budgets or struggle to effectively distill reasoning to smaller models. Standard distillation methods rely on outcome-based rewards, failing to distinguish between sound reasoning and lucky guesses. We propose Collaborative Reasoning Distillation (CRD), a framework that enhances reasoning in compact models through three innovations: (1) interactive cross-feedback where teachers iteratively critique each other's reasoning, (2) fine-grained step-wise quality assessment capturing logical validity independent of final answers, and (3) coherence-aware step stitching that synthesizes complementary strengths. Students are trained via Reasoning Quality Optimization (RQO) with budget constraints. Our model, CRD-4B, achieves 97.3% on MATH-500 and 70.3% on AIME'25, surpassing baselines while using only 50K training examples, up to 12 times smaller than the datasets of comparable models.
Distilling reasoning traces from strong large language models into smaller ones is a promising route to improve intelligence in resource-constrained settings. Existing approaches face a fundamental trade-off: offline distillation from teacher-generated traces provides high-quality, sample-efficient supervision but suffers from distributional drift: during training, the student model conditions on teacher-generated prefixes, whereas during inference the student autoregresses on self-generated prefixes, leading to compounding errors over long reasoning trajectories. Meanwhile, on-policy or self-distillation methods better match the student's inference-time distribution, but require costly online sampling and often produce low-quality traces in early training. We propose a principled offline reasoning distillation framework that preserves the efficiency and supervision quality of offline teacher-generated data while correcting teacher-student distribution drift. It adaptively emphasizes teacher supervision that is better aligned with the student's on-policy distribution. Evaluations on mathematical reasoning benchmarks of GSM8K, MATH, MATH500, and harder held-out competition-style tasks, including AMC, AIME, and OlympiadBench, show that our method improves reasoning accuracy over prior offline distillation algorithms and yields more stable reasoning traces while preserving instruction-following capabilities. Our work shows that lightweight, distribution-correction-aware training can substantially strengthen offline reasoning distillation without online rollouts.