Curriculum learning has been widely adopted in the post-training of large language models by organizing training data from easy to hard. However, its effectiveness varies substantially across reasoning tasks, suggesting that no single curriculum is universally optimal and raising a fundamental question: what determines when curriculum learning works? In this paper, we answer this question by analyzing the optimization dynamics induced by different curriculum schedules. We show that the transfer relationship between different difficulty levels characterizes the optimization dynamics induced by curriculum learning, which in turn explains the effectiveness of different curriculum schedules, and formalize this relationship as Relative Transfer, a principled measure of cross-difficulty knowledge transfer. Based on this measurement, we derive Transfer-aware Dynamic Curriculum Sampling (TDCS), which dynamically adjusts the sampling distribution according to the estimated transfer relationship throughout training. Extensive experiments on multiple reasoning benchmarks demonstrate that TDCS consistently outperforms representative scheduling strategies across different tasks, model scales, and training paradigms. More importantly, our work provides a unified optimization-based explanation of curriculum learning through cross-difficulty transfer.
Figures & tables
Table 1
Figure 1: Stepwise accuracy change of different schedules.
Figure 2: Transfer from the current difficulty to other difficulties. Sudoku exhibits consistently strong hard-to-easy transfer, while iGSM and Code show much weaker transfer.
Category
Method
Sudoku
iGSM
KodCode
Fixed Schedule
Random
0.164
0.353
0.607
Curriculum
0.205
0.350
0.580
Mix
0.190
0.390
0.582
Adaptive Schedule
Ours
0.231
0.428
0.619
Table 3: Main results on three reasoning benchmarks. The best result is shown in bold .
Figure 3: Training dynamics of the proposed method and the strongest fixed scheduling baseline during supervised fine-tuning.
Figure 4: Evolution of the sampling distribution produced by TDCS during supervised fine-tuning on the three reasoning benchmarks. Each stacked bar represents the sampling ratio assigned to different difficulty levels at a representative training stage, while dashed boxes indicate the current difficulty.
Model
Sudoku
iGSM
Baseline
Ours
Baseline
Ours
Qwen2.5-3B
0.428
0.446 (+1.8%)
0.525
0.546 (+2.1%)
Qwen2.5-7B
0.438
0.497 (+5.9%)
0.766
0.819 (+5.3%)
Llama3.2-3B
0.405
0.433 (+2.8%)
0.673
0.875 (+20.8%)
Table 4: Generalization across different model scales. Numbers in parentheses denote the absolute accuracy improvement over the strongest baseline.
Task
Curriculum
Mix
Random
Ours
GSM8K
0.612
0.601
0.601
0.622
Code
0.347
0.338
0.335
0.352
Table 5: Self-improvement results under TDCS.
HA
EDA
Sudoku
iGSM
Code
×
✓
0.213
0.416
0.612
✓
×
0.220
0.390
0.589
✓
✓
0.231
0.428
0.619
Table 6: Ablation study of TDCS components.
Dataset
Train
Test
#Levels
Difficulty
Sudoku
800*4epochs
1000
4
Blank cells
iGSM
1000*4epochs
200
5
Reasoning steps
KodCode
800*4epochs
200
4
GPT pass rate
Table 7: Statistics of the datasets used in this work.
Hyperparameter
Value
Fine-tuning method
LoRA
LoRA rank
64
LoRA alpha
128
LoRA dropout
0.05
Target modules
all
Optimizer
AdamW
Table 8: Training hyperparameters used in all experiments.
Table 9: Sensitivity analysis of τe and τh . The middle row in each table corresponds to the configuration used in our final method.
Automatic curriculum learning can improve the effectiveness of reinforcement learning by selecting the training experiences presented to the agent over time. Predicting the consequences of such decisions can, however, be difficult. We analyze automatic curriculum learning as a sequential decision-making problem, highlighting a gap between the quantities that determine the value of curriculum decisions and the information captured by local learning signals commonly used to guide them. We then investigate whether Large Language Models (LLMs) can exploit richer information about the learning problem to better anticipate the consequences of curriculum decisions. We introduce a method that combines online learning-progress estimates with LLM-informed estimates of (i) the potential downstream benefits of learning on each task and (ii) whether direct training on a task is currently likely to produce progress. We evaluate the approach on a custom benchmark of 256 textual goals in Craftax under different curriculum objectives. We observe the strongest gains when optimizing for individual target tasks. When optimizing across the full task set, the benefits vary across learners with different mechanisms for cross-task transfer, ranging from modest improvements in learning speed to larger gains that persist through the end of training.
Curriculum learning (CL), motivated by the intuition that learning in increasing order of difficulty should ease generalization, is commonly adopted both in pre-training and post-training of large language models (LLMs). The intuition of CL is particularly compelling for compositional reasoning, where complex problems are built from elementary inference rules; however, the actual impact of CL on such tasks remains largely underexplored. We present a systematic empirical study of CL for post-training of LLMs, using synthetic arithmetic and logical benchmarks where difficulty is characterized by reasoning complexity rather than surface-level proxies. Surprisingly, across multiple model families and curriculum schedules, we find no robust advantage in difficulty-based sequencing over standard random sampling in either accuracy or response length. These findings persist across both supervised fine-tuning (SFT) and reinforcement learning (RL) methods. Our study suggests that, in the context of deductive reasoning, the specific ordering of training examples plays a negligible role in achieving compositional generalization, challenging the practical utility of curriculum-based post-training.
Maximilian Mordig, Andreas Opedal, Weiyang Liu +1
Max Planck Institute for Intelligent Systems, Tübingen · ETH Zürich · The Chinese University of Hong Kong
Curriculum learning--ordering training examples in a sequence to aid machine learning--takes inspiration from human learning, but has not gained widespread acceptance. Static strategies for scoring item difficulty rely on indirect proxy scores of varying quality and produce curricula that are not specific to the learner at hand. Dynamic approaches base difficulty estimates on gradient information, requiring considerable extra computation during training. We introduce a novel method for measuring the difficulty of individual problem instances that is calibrated to a series of models of increasing competence, and identify transitional problems that are consistently easier as model ability increases. Applying this method to diverse model series constructed from sets of models that are readily available on many tasks, we find that training on a curriculum that levels up from easier to harder transitional problems most efficiently improves a model to the next tier of competence. These problems induce a natural progression from easier to harder items, which outperforms other training strategies. By measuring difficulty directly relative to model competence, our method yields interpretable problems, learner-specific curricula, and a principled basis for step-by-step improvement.
Amogh Inamdar, Zhenwei Tang, Ashton Anderson +1
Department of Computer Science Columbia University · Department of Computer Science University of Toronto