Capacity-Dependent Effects of Data Selection for Reasoning

Cuong Dang; Hoang Anh Just; Ruoxi Jia

cs.LGAug 13, 2026

Capacity-Dependent Effects of Data Selection for Reasoning

Authors: Cuong Dang, Hoang Anh Just, Ruoxi Jia

Organizations: Department of Electrical and Computer Engineering Virginia Tech Blacksburg, VA, USA

Abstract

In reasoning supervised fine-tuning, candidate responses for the same instruction can differ substantially in how well they match the student's current distribution. Recent likelihood-based response selection methods suggest that responses closer to the student distribution provide more effective supervision, motivating the hypothesis that high-likelihood responses may generally be preferable for fine-tuning. In this paper, we revisit this intuition and show that the value of likelihood-based data selection depends critically on model capacity and training duration. Through controlled experiments on mathematical reasoning, using students ranging from 1.5B to 8B parameters and supervision generated by stronger teacher models, we observe a clear capacity-dependent ``{\color{SMALLCOLOR}Fast-Fit} / {\color{LARGECOLOR}Slow-Gain}'' pattern. High-likelihood data provides faster and more stable early improvements, especially for smaller models, but low-likelihood data becomes increasingly beneficial for larger models when training is allowed to continue longer. To explain this phenomenon, we analyze learning dynamics, showing that small models often fail to absorb low-likelihood supervision and instead fall into shallow or repetitive behaviors, while larger models are better able to move toward the teacher distribution under such data. We further provide a capacity-constrained theoretical view of distillation that clarifies how data difficulty, data span, and student capacity jointly govern transfer. Overall, our findings show that effective data selection for reasoning should be aware of model capacity and computing budget rather than based on a single universal preference for high-likelihood supervision.

Figures & tables

Figure 1: Capacity-dependent learning dynamics under likelihood-based data selection. The illustration conceptualizes the optimization landscape faced by models with different capacities when learning reasoning tasks. A small model ( left ) can only climb to a nearby local peak, representing limited capability to learn from distant supervision signals. When trained on low-likelihood examples—responses far from its current policy—it struggles to escape local minima and therefore benefits from high-likelihood data, which allows it to exploit existing knowledge and achieve incremental improvements. In contrast, a large model ( right ) possesses sufficient representational capacity and optimization flexibility to traverse a longer path across the landscape. Although learning from low-likelihood data may require longer training, the model can ultimately reach a higher global peak, approaching the teacher distribution and acquiring stronger reasoning ability.

	1.5B		3B		4B		7B		8B
Dataset	High	Low	High	Low	High	Low	High	Low	High	Low
AIME	3.33	0	10.00	3.33	20.00	10.00	13.33	13.33	36.67	26.67
AMC	27.50	17.50	47.50	32.50	62.00	50.00	62.50	47.50	87.50	80.00
CHMATH	6.67	3.33	10.00	3.33	26.67	13.33	30.00	20.00	56.67	50.00
GAOKAO	10.13	6.33	25.32	10.13	54.43	53.16	55.70	55.70	74.68	70.89
GPQA	28.79	16.67	34.85	18.18	36.36	26.26	39.39	27.78	78.79	75.25

Table 1: Best performance across checkpoints after ONE epoch for models of different sizes trained on high-likelihood and low-likelihood data . Highlighted cells mark the better performance between training on high and low-likelihood data.

	1.5B		3B		4B		7B		8B
Dataset	High	Low	High	Low	High	Low	High	Low	High	Low
AIME	6.67	3.33	10.00	6.67	20.00	20.00	16.67	26.67	40.00	43.33
AMC	35.00	22.50	55.00	47.50	70.00	57.50	67.50	62.50	90.00	92.5
CHMATH	6.67	6.67	16.67	10.00	33.33	36.67	33.33	33.33	56.67	63.33
GAOKAO	11.39	8.86	25.32	24.05	56.96	58.23	55.70	62.03	74.68	67.09
GPQA	29.29	16.67	34.85	27.27	39.39	38.38	39.39	44.95	78.79	79.8

Table 2: Best performance across checkpoints after FIVE epochs for models of different sizes trained on high-likelihood and low-likelihood data . Highlighted cells mark the better performance between training on high and low-likelihood data.

Question: There is a sequence $a_{1},a_{2},\ldots,a_{2023}$ such that $\tan(2023x)=\cdots$ . What is $a_{2023}$ ?
Qwen2.5-1.5B	Qwen2.5-3B-Instruct	Qwen2.5-Math-7B
There is a sequence $a_{1},a_{2},\ldots,a_{2023}$ such that $\tan(2023x)=\cdots$ . What is $a_{2023}$ ? ${{\color[rgb]{1,0,0}\boxed{2023}\ \boxed{2023}\ \boxed{2023}\ \boxed{2023}}\ \cdots}$	Let $P(x)=\tan 2023x$ . Then $\tan(2023x)=\frac{\tan(2022x)+\tan x}{1-\tan(2022x)\tan x}$ ${\color[rgb]{1,0,0}\tan(2023x)=\frac{\tan(2022x)+\tan x}{1-\tan(2022x)\tan x}}$ $\cdots$	Let $t=\tan x$ . Then we have… We know that $\tan(A+B)=\frac{\tan A+\tan B}{1-\tan A\tan B}$ $\cdots$ Final answer: $\boxed{1}$ .

Table 3: An illustrative example showing that larger models are less prone to getting lost under low-likelihood training. The 1.5B model repeats the question and then gets stuck repeating an incorrect final answer, the 3B model repeats intermediate reasoning, and the 7B model successfully completes the reasoning and returns the correct answer.

Figure 2: Loss dynamics under the teacher model. We measure the Gemma log-likelihood of answers on the test set (AMC) and training set (MATH12K) across checkpoints for the 1.5B, 3B, and 7B student models trained on low- and high-likelihood data.

Figure 3: Loss dynamics under the base model. Log-likelihood on AMC (test) and MATH12K (train) across checkpoints for 1.5B, 3B, and 7B students, evaluated under their own base models after training on low- and high-likelihood data.

Figure 4: Geometric intuition for distillation with a low-capacity student. The data space acts as a mirror that reveals teacher knowledge to the student, but limited capacity restricts learning to the portion that lies in the student’s feasible space, thereby shrinking the effective supervision. As a result, high-capacity students $\bullet$ can absorb more of the exposed knowledge and move toward $\bullet$ , whereas low-capacity students move only toward $\bullet$ .

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Setting	Epoch	AIME24	AMC	CNM	GK	GPQA	GSM	KY	MATH	MNV	OB	W. Avg	Avg
High	0	0	0	0	0	3.03	0	0	1.4	1.47	0.44	0.88	0.58
	0.25	0	22.5	6.67	2.53	27.27	16.67	15.58	52.6	19.49	20.59	26.85	21.26
	0.5	0	27.5	6.67	10.13	26.26	22.38	19.1	52.2	22.06	19.7	27.9	23.27
	0.75	3.33	25	6.67	7.59	28.79	21.43	16.58	54.6	18.01	21.78	28.52	23.62
	1	3.33	27.5	6.67	10.13	22.73	20.48	17.59	53.4	24.63	19.26	27.99	24.16
	1.25	6.67	25	0	11.39	23.74	22.86	18.09	54	22.43	18.07	27.6	22.93

Appendix

Table 4: Qwen-2.5-1.5B results across training checkpoints under high- and low-likelihood data selection. Cyan marks the best value within the early stage (epochs 0–1), yellow marks the best value across all checkpoints within each setting, and green marks cells that satisfy both. Dashed rules separate the earliest stage from later training.

Setting	Epoch	AIME24	AMC	CNM	GK	GPQA	GSM	KY	MATH	MNV	OB	W. Avg	Avg
High	0.25	10	47.5	10	25.32	27.27	19.52	24.12	64.6	29.78	26.37	34.87	30.59
	0.5	10	32.5	0	12.66	27.78	17.62	23.12	63.2	32.35	25.04	33.46	27.11
	0.75	6.67	25	0	13.92	28.79	14.29	16.58	66.4	28.68	26.52	33.46	26.44
	1	3.33	40	10	12.66	34.85	16.67	24.12	62	29.78	26.52	34.25	29.08
	1.25	6.67	45	16.67	17.72	30.81	15.71	20.6	65	30.51	29.63	35.79	31.67
	1.5	3.33	37.5	6.67	12.66	30.3	11.9	19.1	63	29.41	28.74	33.9	28.24

Appendix

Table 5: Qwen-2.5-3B-Instruct results across training checkpoints under high- and low-likelihood data selection. Cyan marks the best value within the early stage (epochs 0.25–1), yellow marks the best value across epochs 0.25–5 within each setting, and green marks cells that satisfy both. Dashed rules separate the earliest stage from later training.

Setting	Epoch	AIME	AMC	CNM	GK	GPQA	GSM	KY	MATH	MNV	OB	W. Avg	Avg
High	0.25	10.00	65.00	26.67	49.37	37.37	45.71	46.23	77.80	43.01	39.70	49.80	44.09
	0.5	13.33	62.50	30.00	55.70	33.84	48.10	35.68	78.00	42.65	39.70	49.04	43.95
	0.75	13.33	57.50	13.33	51.90	39.39	42.86	39.70	75.80	42.65	39.11	48.28	41.56
	1	10.00	62.50	20.00	49.37	33.84	47.14	38.69	78.80	43.01	42.37	49.84	42.57
	1.25	16.67	62.50	23.33	49.37	32.83	45.24	39.70	77.80	41.54	41.93	49.26	43.09
	1.5	13.33	52.50	16.67	50.63	31.31	47.62	39.20	75.20	40.07	40.15	47.74	40.67

Appendix

Table 6: Qwen-2.5-7B results across training checkpoints under high- and low-likelihood data selection. Cyan marks the best value within the early stage (epochs 0.25–1), yellow marks the best value across epochs 0.25–5 within each setting, and green marks cells that satisfy both. Dashed rules separate the earliest stage from later training.

Setting	Epoch	AIME24	AMC	CNM	GK	GPQA	GSM	KY	MATH	MNV	OB	W. Avg	Avg
High	0.25	36.67	80	50	64.56	78.79	68.10	71.86	91.6	58.09	61.93	70.98	66.16
	0.5	26.67	87.5	46.67	68.35	72.73	66.67	67.34	90.0	58.46	63.56	70.17	64.79
	0.75	33.33	85	46.67	69.62	75.25	68.10	67.84	90.0	55.51	64.30	70.53	65.56
	1	26.67	82.5	56.67	74.68	76.77	67.62	62.31	91.0	56.99	62.67	70.22	65.79
	1.25	36.67	85	43.33	73.42	72.73	68.57	63.82	92.0	55.88	62.52	70.09	65.39
	1.5	33.33	87.5	50.00	73.42	76.77	70.48	63.32	91.2	57.72	62.67	70.76	66.64

Appendix

Table 7: Qwen3-8B results across training epochs under high- and low-likelihood data selection. Cyan marks the best value within the early stage (epochs 0.25–1), yellow marks the best value across epochs 0.25–5 within each setting, and green marks cells that satisfy both. Dashed rules separate the earliest stage from later training.

Setting	Epoch	AIME24	AMC	CNM	GK	GPQA	GSM	KY	MATH	MNV	OB	W. Avg	Avg
High	0.25	3.75	23.12	5.00	3.48	18.75	1.43	1.57	43.95	17.10	14.67	18.95	13.28
	0.5	5.83	17.50	4.17	1.90	20.08	2.86	1.38	41.52	18.38	13.85	18.41	12.75
	0.75	6.25	21.25	3.75	3.32	20.20	2.86	1.07	44.35	17.97	15.43	19.57	13.64
	1	6.25	18.44	2.92	3.01	23.42	2.74	3.14	44.12	17.78	14.33	19.55	13.62
	1.25	7.50	26.25	4.17	3.16	20.83	2.08	1.76	46.65	18.98	16.15	20.58	14.75
	1.5	6.25	24.38	3.75	6.65	25.06	4.29	3.77	46.38	18.29	16.35	21.32	15.52

Appendix

Table 8: Llama-3.2-3B results across training epochs under high- and low-likelihood data selection. Cyan marks the best value within the early stage (epochs 0.25–1), yellow marks the best value across epochs 0.25–5 within each setting, and green marks cells that satisfy both. Dashed rules separate the earliest stage from later training.

Setting	Epoch	AIME24	AMC	CNM	GK	GPQA	GSM	KY	MATH	MNV	OB	W. Avg	Avg
High	0.25	4.58	19.38	2.92	3.48	28.85	6.61	1.38	43.80	23.81	14.74	21.04	14.95
	0.5	2.92	17.19	1.25	5.38	29.10	10.54	4.65	43.85	21.97	13.74	21.19	15.06
	0.75	2.92	17.19	3.75	3.32	27.65	5.60	3.08	43.08	21.28	13.46	20.07	14.13
	1	2.92	20.00	1.67	5.54	33.40	7.38	3.64	43.15	22.15	14.65	21.38	15.45
	1.25	5.00	21.56	4.17	9.18	36.24	12.20	5.09	46.50	23.58	17.04	24.08	18.05
	1.5	4.58	18.12	2.92	6.33	36.81	11.19	5.28	48.65	25.00	15.63	24.10	17.45

Appendix

Table 9: Llama-3.1-8B results across training epochs under high- and low-likelihood data selection. Cyan marks the best value within the early stage (epochs 0.25–1), yellow marks the best value across epochs 0.25–5 within each setting, and green marks cells that satisfy both. Dashed rules separate the earliest stage from later training.

Explore similar work

Reasoning Quality Emerges Early: Data Curation for Reasoning Models
Jun 25, 2026Hongyi Henry Jin, Wenhan Yang, Meysam Ghaffari +2LLM Fine-Tuning Training Data Selection
Department of Computer Science, University of California Los Angeles (UCLA), California, USA · Optum AI, UnitedHealth Group, Minneapolis, Minnesota, USA
LARK: Learnability-Grounded Trajectory Selection for Efficient Reasoning Distillation
May 28, 2026Tianrun Yu, Kaixiang Zhao, Chih-Chun Chen +5CoT Distillation Language Model Distillation
Brigham Young University · The Pennsylvania State University · University of North Carolina at Chapel Hill
BoundaryEvo: Data-Model Co-Evolution at the Capability Boundary for Long-Chain Reasoning Distillation
Oct 9, 2026Jiaming Su, Suorong Yang, Xudong Li +2Language Model Distillation Selective Knowledge Distillation
Shanghai Artificial Intelligence Laboratory · Nanjing University · Tongji University +1

Jun 25, 2026J/K move · Enter open · S save

Reasoning Quality Emerges Early: Data Curation for Reasoning Models

Hongyi Henry Jin, Wenhan Yang, Meysam Ghaffari, Carlos Morato+1

Department of Computer Science, University of California Los Angeles (UCLA), California, USA · Optum AI, UnitedHealth Group, Minneapolis, Minnesota, USA

LLM Fine-Tuning Training Data Selection LLM Reasoning Large Reasoning Models Reasoning Trace Analysis

Open paper AskPDF arXiv

Supervised fine-tuning (SFT) on a small, high-quality set of long reasoning traces is an effective approach for eliciting strong reasoning capabilities in Large Language Models (LLMs). However, existing methods for curating high-quality SFT data rely heavily on strong reasoning models to filter examples based on diversity and difficulty, making the curation process costly while often yielding suboptimal data quality. In this work, we show that diverse and challenging reasoning examples can be identified using only the initial reasoning tokens. Specifically, we demonstrate that difficult problems can be reliably detected based on the loss of the first 100 reasoning tokens evaluated at a randomly perturbed checkpoint of the pretrained model. We further show that examples exhibiting similar loss patterns over their first 1k reasoning tokens across a small number of perturbed checkpoints extrapolating along the fine-tuning trajectory provably induce similar gradients. We validate our approach through extensive experiments on fine-tuning Qwen2.5-7B and Llama3.1-8B models on the M23K medical reasoning and OpenThoughts-Math datasets. Our method outperforms existing baselines by up to 1.7% while being 91% more token efficient.

Abstract

Explore similar work

Reasoning Quality Emerges Early: Data Curation for Reasoning Models

LARK: Learnability-Grounded Trajectory Selection for Efficient Reasoning Distillation

BoundaryEvo: Data-Model Co-Evolution at the Capability Boundary for Long-Chain Reasoning Distillation