Understanding Curriculum Learning in Large Language Models via Cross-Difficulty Optimization Dynamics

Zhikai Ding; Ziyi Ye

cs.LGAug 18, 2026

Understanding Curriculum Learning in Large Language Models via Cross-Difficulty Optimization Dynamics

Authors: Zhikai Ding, Ziyi Ye

Organizations: Fudan University

Abstract

Curriculum learning has been widely adopted in the post-training of large language models by organizing training data from easy to hard. However, its effectiveness varies substantially across reasoning tasks, suggesting that no single curriculum is universally optimal and raising a fundamental question: what determines when curriculum learning works? In this paper, we answer this question by analyzing the optimization dynamics induced by different curriculum schedules. We show that the transfer relationship between different difficulty levels characterizes the optimization dynamics induced by curriculum learning, which in turn explains the effectiveness of different curriculum schedules, and formalize this relationship as Relative Transfer, a principled measure of cross-difficulty knowledge transfer. Based on this measurement, we derive Transfer-aware Dynamic Curriculum Sampling (TDCS), which dynamically adjusts the sampling distribution according to the estimated transfer relationship throughout training. Extensive experiments on multiple reasoning benchmarks demonstrate that TDCS consistently outperforms representative scheduling strategies across different tasks, model scales, and training paradigms. More importantly, our work provides a unified optimization-based explanation of curriculum learning through cross-difficulty transfer.

Figures & tables

Table 1

Panel of Figure 1 — Figure 1: Stepwise accuracy change of different schedules.

Figure 2: Transfer from the current difficulty to other difficulties. Sudoku exhibits consistently strong hard-to-easy transfer, while iGSM and Code show much weaker transfer.

Category	Method	Sudoku	iGSM	KodCode
Fixed Schedule	Random	0.164	0.353	0.607
	Curriculum	0.205	0.350	0.580
	Mix	0.190	0.390	0.582
Adaptive Schedule	Ours	0.231	0.428	0.619

Table 3: Main results on three reasoning benchmarks. The best result is shown in bold .

Panel of Figure 3 — Figure 3: Training dynamics of the proposed method and the strongest fixed scheduling baseline during supervised fine-tuning.

Panel of Figure 4 — Figure 4: Evolution of the sampling distribution produced by TDCS during supervised fine-tuning on the three reasoning benchmarks. Each stacked bar represents the sampling ratio assigned to different difficulty levels at a representative training stage, while dashed boxes indicate the current difficulty.

Model	Sudoku		iGSM
Model	Baseline	Ours	Baseline	Ours
Qwen2.5-3B	0.428	0.446 (+1.8%)	0.525	0.546 (+2.1%)
Qwen2.5-7B	0.438	0.497 (+5.9%)	0.766	0.819 (+5.3%)
Llama3.2-3B	0.405	0.433 (+2.8%)	0.673	0.875 (+20.8%)

Table 4: Generalization across different model scales. Numbers in parentheses denote the absolute accuracy improvement over the strongest baseline.

Task	Curriculum	Mix	Random	Ours
GSM8K	0.612	0.601	0.601	0.622
Code	0.347	0.338	0.335	0.352

Table 5: Self-improvement results under TDCS.

HA	EDA	Sudoku	iGSM	Code
$\times$	$\checkmark$	0.213	0.416	0.612
$\checkmark$	$\times$	0.220	0.390	0.589
$\checkmark$	$\checkmark$	0.231	0.428	0.619

Table 6: Ablation study of TDCS components.

Dataset	Train	Test	#Levels	Difficulty
Sudoku	800*4epochs	1000	4	Blank cells
iGSM	1000*4epochs	200	5	Reasoning steps
KodCode	800*4epochs	200	4	GPT pass rate

Table 7: Statistics of the datasets used in this work.

Hyperparameter	Value
Fine-tuning method	LoRA
LoRA rank	64
LoRA alpha	128
LoRA dropout	0.05
Target modules	all
Optimizer	AdamW

Table 8: Training hyperparameters used in all experiments.

Table 9: Sensitivity analysis of

\tau_{e}

and

\tau_{h}

. The middle row in each table corresponds to the configuration used in our final method.

Explore similar work

Anticipating the Consequences of Curriculum Decisions with Large Language Models
Oct 3, 2026Octavio Pappalardo, Nathan Herr, Tim RocktäschelCurriculum Learning Curriculum RL
University College London
Rethinking Easy-to-Hard: Limits of Curriculum Learning in Post-Training for Deductive Reasoning
Mar 28, 2026Maximilian Mordig, Andreas Opedal, Weiyang Liu +1Curriculum Learning Language Model Post-Training
Max Planck Institute for Intelligent Systems, Tübingen · ETH Zürich · The Chinese University of Hong Kong
Level Up: Defining and Exploiting Transitional Problems for Curriculum Learning
Mar 14, 2026Amogh Inamdar, Zhenwei Tang, Ashton Anderson +1Curriculum Learning Data-Efficient Learning
Department of Computer Science Columbia University · Department of Computer Science University of Toronto

Oct 3, 2026J/K move · Enter open · S save

Anticipating the Consequences of Curriculum Decisions with Large Language Models

Octavio Pappalardo, Nathan Herr, Tim Rocktäschel

University College London

Curriculum Learning Curriculum RL LLM-Guided RL

Open paper AskPDF arXiv

Automatic curriculum learning can improve the effectiveness of reinforcement learning by selecting the training experiences presented to the agent over time. Predicting the consequences of such decisions can, however, be difficult. We analyze automatic curriculum learning as a sequential decision-making problem, highlighting a gap between the quantities that determine the value of curriculum decisions and the information captured by local learning signals commonly used to guide them. We then investigate whether Large Language Models (LLMs) can exploit richer information about the learning problem to better anticipate the consequences of curriculum decisions. We introduce a method that combines online learning-progress estimates with LLM-informed estimates of (i) the potential downstream benefits of learning on each task and (ii) whether direct training on a task is currently likely to produce progress. We evaluate the approach on a custom benchmark of 256 textual goals in Craftax under different curriculum objectives. We observe the strongest gains when optimizing for individual target tasks. When optimizing across the full task set, the benefits vary across learners with different mechanisms for cross-task transfer, ranging from modest improvements in learning speed to larger gains that persist through the end of training.

Abstract

Explore similar work

Anticipating the Consequences of Curriculum Decisions with Large Language Models

Rethinking Easy-to-Hard: Limits of Curriculum Learning in Post-Training for Deductive Reasoning

Level Up: Defining and Exploiting Transitional Problems for Curriculum Learning