Distilling long-chain reasoning trajectories from large teacher models provides an effective way to improve the reasoning ability of small language models. However, existing methods typically select a fixed training set before distillation, overlooking that the usefulness of a reasoning example changes as the student evolves. This mismatch becomes especially costly when only a small number of teacher trajectories can be used for training. Besides, examples already mastered provide little new supervision, while examples far beyond the capability of student may be difficult to learn from. Training data should therefore adapt as model capability evolves, yet reliable data selection is difficult when the student is initially weak. To address these challenges, our key idea is to first establish the basic reasoning capability and then adapt the training data to the evolving capability boundary of the student. We propose BoundaryEvo, a two-stage data--model co-evolution framework. In Capability Priming, we directly distill a small set of teacher-generated long reasoning trajectories without using a separate model to judge and filter them, allowing the student to acquire an initial reasoning capability before capability-aware data selection. In Capability-Boundary Co-Evolution, we repeatedly select questions that the current student has not mastered but the expert can reliably solve, and update the training data as the student evolves. With only 1,000 training examples, BoundaryEvo consistently outperforms static and dynamic data selection baselines across 11 reasoning benchmarks, raising the original Qwen3.5-4B accuracy from 62.9 to 75.3 on IMO-AnswerBench and from 26.8 to 46.0 on AMO-Bench. The code and data will be made publicly available.
Figures & tables
Figure 1: BoundaryEvo evolves supervision along the capability boundary of the student model. (a-c) Comparison with static distillation and error-driven acquisition, BoundaryEvo dynamically adapts supervision according to the evolving capability boundary of the student model. (d) BoundaryEvo-4B consistently outperforms strong baselines across various reasoning benchmarks.
Figure 2: Overview of BoundaryEvo. Stage 1 performs judge-free teacher distillation to obtain the capability-primed checkpoint. In Stage 2, responses from the current student and a fixed expert are used to judge the candidate QAs, from which replay and novel records are selected for multiple teacher rollouts. The resulting trajectories are then independently verified against reference answers, and only those that pass verification supervise the next student.
Model
Params
Data
Mathematical Reasoning
General Reasoning
AIME
AIME
HMMT
HMMT
BruMO
IMO
AMO
Arxiv
Zebra
GPQA
Super
25
26
25
26
25
-Ans
-Bench
Math
Logic
-D
GPQA
Small reasoning models
SmolLM3-3B
3B
–
36.7
41.0
26.0
36.4
49.2
28.7
5.3
7.8
26.0
41.7
29.2
VibeThinker-3B
3B
–
91.4
94.3
89.3
80.3
93.8
76.4
41.0
27.2
74.7
70.2
34.8
DASD-4B-Thinking
4B
448K
83.3
88.3
80.0
72.7
85.0
61.6
26.5
9.7
69.0
68.4
42.9
Table 1: Comparison with Reasoning Models under Different Scales and Training Budgets. Scores are percentages. Data denotes the number of publicly disclosed reasoning post-training examples and – indicates unavailable information.
Method
Mathematical Reasoning
General Reasoning
Avg.
AIME
AIME
HMMT
HMMT
BruMO
IMO
AMO
Arxiv
Zebra
GPQA
Super
Math
General
25
26
25
26
25
-Ans
-Bench
Math
Logic
-D
GPQA
Qwen3.5-4B
+ Zero-shot
79.8
84.0
73.8
68.9
83.5
62.9
26.8
25.2
92.4
76.2
52.9
63.1
73.8
+ Stage 1
84.2 +4.4
88.3 +4.3
81.7 +7.9
69.7 +0.8
85.0 +1.5
66.3 +3.4
38.5 +11.7
28.2 +3.0
92.5 +0.1
79.8 +3.6
53.4 +0.5
67.7 +4.6
75.2 +1.4
Stage 2 with Random Selection
Table 2: Qwen3.5-4B results across three evolution turns. All Stage 2 strategies start from the same Stage 1 checkpoint. Scores are percentages, with Math and General averaged over eight and three benchmarks. Superscripts indicate absolute changes from zero-shot. Bold and underlining mark the highest and second-highest values in each column.
Method
Mathematical Reasoning
General Reasoning
Avg.
AIME
AIME
HMMT
HMMT
BruMO
IMO
Zebra
GPQA
Math
General
25
26
25
26
25
-Ans
Logic
-D
Qwen3.5-2B
+ Zero-shot
34.2
38.8
22.9
22.7
35.0
22.0
44.9
50.0
29.3
47.5
+ Stage 1
38.3 +4.1
50.0 +11.2
26.7 +3.8
24.5 +1.8
43.3 +8.3
35.2 +13.2
50.5 +5.6
50.5 +0.5
36.3 +7.0
50.5 +3.0
+ Turn 1
40.0 +5.8
46.7 +7.9
30.0 +7.1
33.3 +10.6
46.7 +11.7
35.7 +13.7
63.7 +18.8
52.5 +2.5
38.7 +9.4
58.1 +10.6
Table 3: Transfer across model size and architecture. Each backbone is primed before three sequential Stage 2 turns. Scores are percentages. Superscripts indicate absolute changes from zero-shot. Bold/underline denote the top two values, and blue shading marks the best checkpoint.
Figure 3: Training and selection dynamics across the Stage 2. (a) SFT loss for representative runs. (b) Mean student and expert scores on each turn. Annotations show the score gap.
Figure 4: Ablation study on IMO-AnswerBench performance. The curves compare BoundaryEvo with verified Stage 1 and no-replay variants, while reference lines show zero-shot and static SFT on 1K examples.
Method
Math AVG
General AVG
BoundaryEvo
76.4
77.4
⊢ Stage 1 (w/ verification)
73.1
73.9
⊢ w/o Stage 1
73.5
74.0
Table 4: Ablation study on effect of Stage 1 on final Qwen3.5-4B performance. Math and General average eight and three benchmarks.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Ablated factor
E
R
τg
Stage 2 data
Math / General
Reference
5
3
0.20
900
76.4 / 77.4
Epochs
1
3
0.20
900
72.6 / 75.9
Epochs
3
3
0.20
900
73.5 / 76.1
Epochs
10
3
0.20
900
74.3 / 77.0
Evolution turns
5
1
0.20
300
71.2 / 75.5
Evolution turns
5
2
0.20
600
73.3 / 76.9
Appendix
Table 5: Core hyperparameter ablations on Qwen3.5-4B. E is the number of SFT epochs per update, R is the number of Stage 2 turns, and τg is the minimum expert–student capability gap. Stage 2 data count is 300R and counts replayed examples whenever they are reused. Each non-reference row changes one factor only.
Setting
Qwen3.5-4B / 2B
Gemma-4-E4B-it
Training objective
Response-only SFT
Response-only SFT
Fine-tuning method
LoRA
LoRA
LoRA rank / scaling factor
64 / 128
64 / 128
LoRA target modules
All linear modules
q,k,v,o , gate, up, and down projections
Trainable parameters
129.9M / 67.3M
139.5M
Conversation template
qwen3
gemma4
Appendix
Table 6: Training configuration used in experiments. The effective batch size is the product of the per-device batch size, gradient accumulation steps, and the number of data-parallel workers.
Figure 5: Reasoning-length distributions on matched evaluation questions. Curves show smoothed densities of per-question reasoning-token counts, and vertical lines mark medians. The final panel reports the relative change in median length from Qwen3.5-4B to BoundaryEvo-4B. Repeated rollouts of the same question are summarized by their median before plotting.
Benchmark
n
Median
Δ
90th percentile
Base
Ours
Base
Ours
AIME25
30
42.8
20.5
−52.2%
99.4
52.1
AIME26
30
36.9
17.4
−52.9%
74.9
54.9
HMMT25
30
46.1
27.3
−40.8%
102.4
74.3
HMMT26
31
45.4
34.6
−23.9%
71.3
74.6
ArxivMath
103
56.4
52.2
−7.4%
82.1
71.2
Appendix
Table 7: Reasoning-length statistics for the matched questions in Figure 5 . Token counts are reported in thousands. Δ denotes the relative change in median length from the base model to BoundaryEvo-4B.
Figure 6: GPQA-D performance versus model scale. The horizontal axis is logarithmic, bubble area encodes parameter count, and color denotes the parameter range. BoundaryEvo-4B is highlighted with a white star. Only models with both a reported parameter count and GPQA-D score in Table 1 are included.
Figure 7: GPQA-D performance versus publicly reported reasoning post-training data. The horizontal axis is logarithmic and bubble area encodes model parameters. Right-pointing arrows mark reported lower bounds rather than exact dataset sizes. Only systems with disclosed data amounts and GPQA-D scores in Table 1 are shown.
Recent advances in large reasoning models (LRMs) have shown strong performance on complex problems through long chain-of-thought (Long CoT) reasoning. However, distilling such trajectories into smaller student models remains challenging: direct Long CoT supervision often provides limited gains and can be less effective than concise Short CoT rationales. In this work, we investigate this phenomenon from a gradient-centric perspective. Our analysis shows that Long CoT induces larger gradient magnitudes and more concentrated update directions than Short CoT, with this effect becoming more pronounced as student model capacity increases. These findings suggest that effective Long CoT distillation requires balancing the reasoning information density of reasoning trajectories with their distributional alignment to the student model. Motivated by this insight, we propose Model Interporlation Distillation (MI-Distillation), a framework that constructs a continuous Instruct-Reasoning data spectrum through model interpolation. To select suitable trajectories from this spectrum, we further introduce Sequential Learnable Surprisal Score (SeqLSS), which favors reasoning paths that are both informative and learnable for the student. Extensive experiments on reasoning benchmarks show that MI-Distillation consistently improves small model CoT distillation over strong Long CoT baselines.
Yangsong Lan, Renkai Hu, HongKai Zheng +4
College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics, Nanjing, China · 2The Key Laboratory of Brain-Machine Intelligence Technology, Ministry of Education, Nanjing, China
On-Policy Distillation (OPD) has emerged as a widely adopted post-training paradigm for enhancing large language models in reasoning domains. However, the data-centric mechanisms in OPD remain relatively underexplored. This paper presents an empirical study of data efficiency and data selection in OPD. We begin by investigating an extreme setting: training OPD on only one example, namely 1-shot OPD. Surprisingly, we find that 1-shot OPD is consistently effective across all sampled training examples and harder examples often yield superior performance gain. We next investigate what actually drives the student model's improvement in the training data. Our analysis reveals that the improvement is not driven by high token entropy, but the longer CoT paths which hard problem naturally generate. Training on longer CoT can help maintain closer alignment with the teacher over a long reasoning horizon, and learn critical thinking patterns usually missing in short CoTs, such as reflection (e.g., "Alternatively"). Based on these insights, we propose a simple data selection method that selects only hard examples for training, where even "unsolvable" examples that completely exceed the teacher's capability can be successfully used. Our experiments conducted on five models ranging from 1.5B to 8B show that training the student model on only 8 selected hard examples matches the performance of the 17K dataset baseline. Code and models will be publicly available.
Distilling large reasoning models is essential for making Long-CoT reasoning practical, as full-scale inference remains computationally prohibitive. Existing curation-based approaches select complete reasoning traces post-hoc, overlooking collaboration among heterogeneous teachers and lacking dynamic exploration, which leads to redundant sampling and missed complementary reasoning. We introduce CoRD, a collaborative multi-teacher decoding framework that performs step-wise reasoning synthesis guided by predictive perplexity-based scoring and beam search. This enables heterogeneous LRMs to jointly construct coherent reasoning trajectories while efficiently preserving diverse, high-potential hypotheses. Experiments show that CoRD produces higher-quality reasoning data and achieves near teacher-level student performance with fewer, structured supervision signals, without substantial efficiency overhead. CoRD further generalizes well to out-of-domain and open-ended settings. The dataset and model are available at https://github.com/DISL-Lab/CoRD.