In reasoning supervised fine-tuning, candidate responses for the same instruction can differ substantially in how well they match the student's current distribution. Recent likelihood-based response selection methods suggest that responses closer to the student distribution provide more effective supervision, motivating the hypothesis that high-likelihood responses may generally be preferable for fine-tuning. In this paper, we revisit this intuition and show that the value of likelihood-based data selection depends critically on model capacity and training duration. Through controlled experiments on mathematical reasoning, using students ranging from 1.5B to 8B parameters and supervision generated by stronger teacher models, we observe a clear capacity-dependent ``{\color{SMALLCOLOR}Fast-Fit} / {\color{LARGECOLOR}Slow-Gain}'' pattern. High-likelihood data provides faster and more stable early improvements, especially for smaller models, but low-likelihood data becomes increasingly beneficial for larger models when training is allowed to continue longer. To explain this phenomenon, we analyze learning dynamics, showing that small models often fail to absorb low-likelihood supervision and instead fall into shallow or repetitive behaviors, while larger models are better able to move toward the teacher distribution under such data. We further provide a capacity-constrained theoretical view of distillation that clarifies how data difficulty, data span, and student capacity jointly govern transfer. Overall, our findings show that effective data selection for reasoning should be aware of model capacity and computing budget rather than based on a single universal preference for high-likelihood supervision.
Figures & tables
Figure 1: Capacity-dependent learning dynamics under likelihood-based data selection. The illustration conceptualizes the optimization landscape faced by models with different capacities when learning reasoning tasks. A small model ( left ) can only climb to a nearby local peak, representing limited capability to learn from distant supervision signals. When trained on low-likelihood examples—responses far from its current policy—it struggles to escape local minima and therefore benefits from high-likelihood data, which allows it to exploit existing knowledge and achieve incremental improvements. In contrast, a large model ( right ) possesses sufficient representational capacity and optimization flexibility to traverse a longer path across the landscape. Although learning from low-likelihood data may require longer training, the model can ultimately reach a higher global peak, approaching the teacher distribution and acquiring stronger reasoning ability.
1.5B
3B
4B
7B
8B
Dataset
High
Low
High
Low
High
Low
High
Low
High
Low
AIME
3.33
0
10.00
3.33
20.00
10.00
13.33
13.33
36.67
26.67
AMC
27.50
17.50
47.50
32.50
62.00
50.00
62.50
47.50
87.50
80.00
CHMATH
6.67
3.33
10.00
3.33
26.67
13.33
30.00
20.00
56.67
50.00
GAOKAO
10.13
6.33
25.32
10.13
54.43
53.16
55.70
55.70
74.68
70.89
GPQA
28.79
16.67
34.85
18.18
36.36
26.26
39.39
27.78
78.79
75.25
Table 1: Best performance across checkpoints after ONE epoch for models of different sizes trained on high-likelihood and low-likelihood data . Highlighted cells mark the better performance between training on high and low-likelihood data.
1.5B
3B
4B
7B
8B
Dataset
High
Low
High
Low
High
Low
High
Low
High
Low
AIME
6.67
3.33
10.00
6.67
20.00
20.00
16.67
26.67
40.00
43.33
AMC
35.00
22.50
55.00
47.50
70.00
57.50
67.50
62.50
90.00
92.5
CHMATH
6.67
6.67
16.67
10.00
33.33
36.67
33.33
33.33
56.67
63.33
GAOKAO
11.39
8.86
25.32
24.05
56.96
58.23
55.70
62.03
74.68
67.09
GPQA
29.29
16.67
34.85
27.27
39.39
38.38
39.39
44.95
78.79
79.8
Table 2: Best performance across checkpoints after FIVE epochs for models of different sizes trained on high-likelihood and low-likelihood data . Highlighted cells mark the better performance between training on high and low-likelihood data.
Question: There is a sequence a1,a2,…,a2023 such that tan(2023x)=⋯ . What is a2023 ?
Qwen2.5-1.5B
Qwen2.5-3B-Instruct
Qwen2.5-Math-7B
There is a sequence a1,a2,…,a2023 such that tan(2023x)=⋯ . What is a2023 ? {{\color[rgb]{1,0,0}\boxed{2023}\ \boxed{2023}\ \boxed{2023}\ \boxed{2023}}\ \cdots}
Let P(x)=tan2023x . Then tan(2023x)=1−tan(2022x)tanxtan(2022x)+tanx{\color[rgb]{1,0,0}\tan(2023x)=\frac{\tan(2022x)+\tan x}{1-\tan(2022x)\tan x}}⋯
Let t=tanx . Then we have… We know that tan(A+B)=1−tanAtanBtanA+tanB⋯ Final answer: 1 .
Table 3: An illustrative example showing that larger models are less prone to getting lost under low-likelihood training. The 1.5B model repeats the question and then gets stuck repeating an incorrect final answer, the 3B model repeats intermediate reasoning, and the 7B model successfully completes the reasoning and returns the correct answer.
Figure 2: Loss dynamics under the teacher model. We measure the Gemma log-likelihood of answers on the test set (AMC) and training set (MATH12K) across checkpoints for the 1.5B, 3B, and 7B student models trained on low- and high-likelihood data.
Figure 3: Loss dynamics under the base model. Log-likelihood on AMC (test) and MATH12K (train) across checkpoints for 1.5B, 3B, and 7B students, evaluated under their own base models after training on low- and high-likelihood data.
Figure 4: Geometric intuition for distillation with a low-capacity student. The data space acts as a mirror that reveals teacher knowledge to the student, but limited capacity restricts learning to the portion that lies in the student’s feasible space, thereby shrinking the effective supervision. As a result, high-capacity students ∙ can absorb more of the exposed knowledge and move toward ∙ , whereas low-capacity students move only toward ∙ .
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Setting
Epoch
AIME24
AMC
CNM
GK
GPQA
GSM
KY
MATH
MNV
OB
W. Avg
Avg
High
0
0
0
0
0
3.03
0
0
1.4
1.47
0.44
0.88
0.58
0.25
0
22.5
6.67
2.53
27.27
16.67
15.58
52.6
19.49
20.59
26.85
21.26
0.5
0
27.5
6.67
10.13
26.26
22.38
19.1
52.2
22.06
19.7
27.9
23.27
0.75
3.33
25
6.67
7.59
28.79
21.43
16.58
54.6
18.01
21.78
28.52
23.62
1
3.33
27.5
6.67
10.13
22.73
20.48
17.59
53.4
24.63
19.26
27.99
24.16
1.25
6.67
25
0
11.39
23.74
22.86
18.09
54
22.43
18.07
27.6
22.93
Appendix
Table 4: Qwen-2.5-1.5B results across training checkpoints under high- and low-likelihood data selection. Cyan marks the best value within the early stage (epochs 0–1), yellow marks the best value across all checkpoints within each setting, and green marks cells that satisfy both. Dashed rules separate the earliest stage from later training.
Setting
Epoch
AIME24
AMC
CNM
GK
GPQA
GSM
KY
MATH
MNV
OB
W. Avg
Avg
High
0.25
10
47.5
10
25.32
27.27
19.52
24.12
64.6
29.78
26.37
34.87
30.59
0.5
10
32.5
0
12.66
27.78
17.62
23.12
63.2
32.35
25.04
33.46
27.11
0.75
6.67
25
0
13.92
28.79
14.29
16.58
66.4
28.68
26.52
33.46
26.44
1
3.33
40
10
12.66
34.85
16.67
24.12
62
29.78
26.52
34.25
29.08
1.25
6.67
45
16.67
17.72
30.81
15.71
20.6
65
30.51
29.63
35.79
31.67
1.5
3.33
37.5
6.67
12.66
30.3
11.9
19.1
63
29.41
28.74
33.9
28.24
Appendix
Table 5: Qwen-2.5-3B-Instruct results across training checkpoints under high- and low-likelihood data selection. Cyan marks the best value within the early stage (epochs 0.25–1), yellow marks the best value across epochs 0.25–5 within each setting, and green marks cells that satisfy both. Dashed rules separate the earliest stage from later training.
Setting
Epoch
AIME
AMC
CNM
GK
GPQA
GSM
KY
MATH
MNV
OB
W. Avg
Avg
High
0.25
10.00
65.00
26.67
49.37
37.37
45.71
46.23
77.80
43.01
39.70
49.80
44.09
0.5
13.33
62.50
30.00
55.70
33.84
48.10
35.68
78.00
42.65
39.70
49.04
43.95
0.75
13.33
57.50
13.33
51.90
39.39
42.86
39.70
75.80
42.65
39.11
48.28
41.56
1
10.00
62.50
20.00
49.37
33.84
47.14
38.69
78.80
43.01
42.37
49.84
42.57
1.25
16.67
62.50
23.33
49.37
32.83
45.24
39.70
77.80
41.54
41.93
49.26
43.09
1.5
13.33
52.50
16.67
50.63
31.31
47.62
39.20
75.20
40.07
40.15
47.74
40.67
Appendix
Table 6: Qwen-2.5-7B results across training checkpoints under high- and low-likelihood data selection. Cyan marks the best value within the early stage (epochs 0.25–1), yellow marks the best value across epochs 0.25–5 within each setting, and green marks cells that satisfy both. Dashed rules separate the earliest stage from later training.
Setting
Epoch
AIME24
AMC
CNM
GK
GPQA
GSM
KY
MATH
MNV
OB
W. Avg
Avg
High
0.25
36.67
80
50
64.56
78.79
68.10
71.86
91.6
58.09
61.93
70.98
66.16
0.5
26.67
87.5
46.67
68.35
72.73
66.67
67.34
90.0
58.46
63.56
70.17
64.79
0.75
33.33
85
46.67
69.62
75.25
68.10
67.84
90.0
55.51
64.30
70.53
65.56
1
26.67
82.5
56.67
74.68
76.77
67.62
62.31
91.0
56.99
62.67
70.22
65.79
1.25
36.67
85
43.33
73.42
72.73
68.57
63.82
92.0
55.88
62.52
70.09
65.39
1.5
33.33
87.5
50.00
73.42
76.77
70.48
63.32
91.2
57.72
62.67
70.76
66.64
Appendix
Table 7: Qwen3-8B results across training epochs under high- and low-likelihood data selection. Cyan marks the best value within the early stage (epochs 0.25–1), yellow marks the best value across epochs 0.25–5 within each setting, and green marks cells that satisfy both. Dashed rules separate the earliest stage from later training.
Setting
Epoch
AIME24
AMC
CNM
GK
GPQA
GSM
KY
MATH
MNV
OB
W. Avg
Avg
High
0.25
3.75
23.12
5.00
3.48
18.75
1.43
1.57
43.95
17.10
14.67
18.95
13.28
0.5
5.83
17.50
4.17
1.90
20.08
2.86
1.38
41.52
18.38
13.85
18.41
12.75
0.75
6.25
21.25
3.75
3.32
20.20
2.86
1.07
44.35
17.97
15.43
19.57
13.64
1
6.25
18.44
2.92
3.01
23.42
2.74
3.14
44.12
17.78
14.33
19.55
13.62
1.25
7.50
26.25
4.17
3.16
20.83
2.08
1.76
46.65
18.98
16.15
20.58
14.75
1.5
6.25
24.38
3.75
6.65
25.06
4.29
3.77
46.38
18.29
16.35
21.32
15.52
Appendix
Table 8: Llama-3.2-3B results across training epochs under high- and low-likelihood data selection. Cyan marks the best value within the early stage (epochs 0.25–1), yellow marks the best value across epochs 0.25–5 within each setting, and green marks cells that satisfy both. Dashed rules separate the earliest stage from later training.
Setting
Epoch
AIME24
AMC
CNM
GK
GPQA
GSM
KY
MATH
MNV
OB
W. Avg
Avg
High
0.25
4.58
19.38
2.92
3.48
28.85
6.61
1.38
43.80
23.81
14.74
21.04
14.95
0.5
2.92
17.19
1.25
5.38
29.10
10.54
4.65
43.85
21.97
13.74
21.19
15.06
0.75
2.92
17.19
3.75
3.32
27.65
5.60
3.08
43.08
21.28
13.46
20.07
14.13
1
2.92
20.00
1.67
5.54
33.40
7.38
3.64
43.15
22.15
14.65
21.38
15.45
1.25
5.00
21.56
4.17
9.18
36.24
12.20
5.09
46.50
23.58
17.04
24.08
18.05
1.5
4.58
18.12
2.92
6.33
36.81
11.19
5.28
48.65
25.00
15.63
24.10
17.45
Appendix
Table 9: Llama-3.1-8B results across training epochs under high- and low-likelihood data selection. Cyan marks the best value within the early stage (epochs 0.25–1), yellow marks the best value across epochs 0.25–5 within each setting, and green marks cells that satisfy both. Dashed rules separate the earliest stage from later training.
Supervised fine-tuning (SFT) on a small, high-quality set of long reasoning traces is an effective approach for eliciting strong reasoning capabilities in Large Language Models (LLMs). However, existing methods for curating high-quality SFT data rely heavily on strong reasoning models to filter examples based on diversity and difficulty, making the curation process costly while often yielding suboptimal data quality. In this work, we show that diverse and challenging reasoning examples can be identified using only the initial reasoning tokens. Specifically, we demonstrate that difficult problems can be reliably detected based on the loss of the first 100 reasoning tokens evaluated at a randomly perturbed checkpoint of the pretrained model. We further show that examples exhibiting similar loss patterns over their first 1k reasoning tokens across a small number of perturbed checkpoints extrapolating along the fine-tuning trajectory provably induce similar gradients. We validate our approach through extensive experiments on fine-tuning Qwen2.5-7B and Llama3.1-8B models on the M23K medical reasoning and OpenThoughts-Math datasets. Our method outperforms existing baselines by up to 1.7% while being 91% more token efficient.
Hongyi Henry Jin, Wenhan Yang, Meysam Ghaffari +2
Department of Computer Science, University of California Los Angeles (UCLA), California, USA · Optum AI, UnitedHealth Group, Minneapolis, Minnesota, USA
We study trajectory selection for reasoning distillation, where teacher-generated reasoning trajectories are selectively used as supervision for a student model. Existing methods rely on heuristics such as trajectory quality or model confidence, but they often overlook whether a trajectory is learnable by the student. In this paper, we present LARK, a learnability-grounded method for reasoning trajectory selection. LARK selects trajectories that the student can learn efficiently while preserving the generalization of the full training distribution. At the core of LARK is a learnability factor ρ, which characterizes the rate at which the student's training loss decreases. To estimate this rate efficiently and maintain generalization, we introduce a learnability proxy and a χ2-regularized selection policy that balances learnability and distributional coverage, both with strong theoretical guarantees on their estimation error. Empirically, LARK consistently outperforms data selection baselines across multiple base models and reasoning tasks. Diagnostic analyses show that the LARK score predicts downstream training utility and that LARK-selected trajectories induce faster supervised fine-tuning loss reduction. Our code is available at https://github.com/Tianrun-Yu/LARK.
Tianrun Yu, Kaixiang Zhao, Chih-Chun Chen +5
Brigham Young University · The Pennsylvania State University · University of North Carolina at Chapel Hill
Distilling long-chain reasoning trajectories from large teacher models provides an effective way to improve the reasoning ability of small language models. However, existing methods typically select a fixed training set before distillation, overlooking that the usefulness of a reasoning example changes as the student evolves. This mismatch becomes especially costly when only a small number of teacher trajectories can be used for training. Besides, examples already mastered provide little new supervision, while examples far beyond the capability of student may be difficult to learn from. Training data should therefore adapt as model capability evolves, yet reliable data selection is difficult when the student is initially weak. To address these challenges, our key idea is to first establish the basic reasoning capability and then adapt the training data to the evolving capability boundary of the student. We propose BoundaryEvo, a two-stage data--model co-evolution framework. In Capability Priming, we directly distill a small set of teacher-generated long reasoning trajectories without using a separate model to judge and filter them, allowing the student to acquire an initial reasoning capability before capability-aware data selection. In Capability-Boundary Co-Evolution, we repeatedly select questions that the current student has not mastered but the expert can reliably solve, and update the training data as the student evolves. With only 1,000 training examples, BoundaryEvo consistently outperforms static and dynamic data selection baselines across 11 reasoning benchmarks, raising the original Qwen3.5-4B accuracy from 62.9 to 75.3 on IMO-AnswerBench and from 26.8 to 46.0 on AMO-Bench. The code and data will be made publicly available.
Jiaming Su, Suorong Yang, Xudong Li +2
Shanghai Artificial Intelligence Laboratory · Nanjing University · Tongji University +1