Dual-Criterion Curriculum Learning: Application to Temporal Data

Gaspard Abel; Eloi Campagne; Mohamed Benloughmari; Argyris Kalogeratos

cs.LGMar 24, 2026

Dual-Criterion Curriculum Learning: Application to Temporal Data

Authors: Gaspard Abel, Eloi Campagne, Mohamed Benloughmari, Argyris Kalogeratos

Organizations: Université Paris Saclay, Université Paris Cité, ENS Paris Saclay, CNRS, SSA, INSERM, Centre Borelli, F-91190, Gif-sur-Yvette, France · Centre d’Analyse et de Mathématique Sociales, EHESS, CNRS, 75006 Paris, France · EDF R&D, 91120 Palaiseau, France

Abstract

Curriculum Learning (CL) is a meta-learning paradigm that trains a model by feeding the data instances incrementally according to a schedule, which is based on difficulty progression. Defining meaningful difficulty assessment measures is crucial and most usually the main bottleneck for effective learning, while also in many cases the employed heuristics are only application-specific. In this work, we propose the Dual-Criterion Curriculum Learning (DCCL) framework that combines two views of assessing instance-wise difficulty: a loss-based criterion is complemented by a density-based criterion learned in the data representation space. Essentially, DCCL calibrates training-based evidence (loss) under the consideration that data sparseness amplifies the learning difficulty. As a testbed, we choose the time-series forecasting task. We evaluate our framework on multivariate time-series benchmarks under standard One-Pass and Baby-Steps training schedules. Empirical results show the interest of density-based and hybrid dual-criterion curricula over loss-only baselines and standard non-CL training in this setting.

Figures & tables

Panel of Figure 1 — Figure 1: Schema of the proposed modular DCCL framework . A representation model $\phi_{\boldsymbol{\theta}}$ maps each data instance $\boldsymbol{x}_{n}\in\mathcal{D}$ to a vector $\phi_{\boldsymbol{\theta}}(\boldsymbol{x}_{n})\in\mathbb{R}^{d}$ . Based on these representations, a difficulty module $\delta$ (dashed block) assigns a score to each instance. These scores are then partitioned into $K$ ordered curriculum buckets $\mathcal{B}_{1}\preceq\cdots\preceq\mathcal{B}_{K}$ (easy to hard) via Adaptive Filtering . Next, the model $f_{\boldsymbol{\theta}}$ is fine-tuned sequentially on each training set $\mathcal{C}_{k}$ , given the Training Scheduler .

Figure 2: Scatter plot of loss vs. density scores for training instances . The complementarity of these two difficulty criteria motivates hybrid strategies that combine both views.

Figure 3: Effect of the convex-mixing parameter

\alpha

on curriculum buckets . As

\alpha

varies from

0

(pure density-based) to

1

(pure loss-based), the resulting difficulty ordering smoothly interpolates between the strategies.

Figure 4: Bivariate loss-density stratification. Instances are binned into a 2D grid according to their loss and density scores, then ordered cell-by-cell from easy (low loss, high density) to hard (high loss, low density). Each cell is taken as a bucket.

Strategy	View	Repr. model $\phi_{\boldsymbol{\theta}}$	Difficulty $\delta(\boldsymbol{x}_{n})$
STL [ 11 ]	STL residuals	$\phi_{\boldsymbol{\theta}}(\boldsymbol{x}_{n})=\mathbf{T}_{n}+\mathbf{S}_{n}+\mathbf{R}_{n}$	Magnitude of residuals $\|\|\mathbf{R}_{n}\|\|$
Loss-based	Loss	Forecasting $\phi_{\boldsymbol{\theta}}:\mathcal{X}\to\mathcal{Y}$	$\ell\!\left(\phi_{\boldsymbol{\theta}}(\boldsymbol{x}_{n}),\,\boldsymbol{y}_{n}\right)$
$k$ -NN	Density	Encoder $\phi_{\boldsymbol{\theta}}:\mathcal{X}\to\mathbb{R}^{d}$	$-\rho\!\left(\phi_{\boldsymbol{\theta}}(\boldsymbol{x}_{n})\right)$
KDE	Density	Encoder $\phi_{\boldsymbol{\theta}}:\mathcal{X}\to\mathbb{R}^{d}$	$-\hat{p}\!\left(\phi_{\boldsymbol{\theta}}(\boldsymbol{x}_{n})\right)$
Convex-value	Loss+Density	Encoder $+$ forecasting model	$\alpha\,\tilde{\delta}_{\mathrm{loss}}(\boldsymbol{x}_{n})+(1{-}\alpha)\,\tilde{\delta}_{\mathrm{dens}}(\boldsymbol{x}_{n}),\hskip 9.24994pt\alpha\in[0,1]$
Convex-ranks	Loss+Density	Encoder $+$ forecasting model	$\alpha\,r_{\mathrm{loss}}(\boldsymbol{x}_{n})+(1{-}\alpha)\,r_{\mathrm{dens}}(\boldsymbol{x}_{n}),\hskip 9.24994pt\alpha\in[0,1]$

Table 1: Summary of the CL strategies. Representation model

\phi_{\boldsymbol{\theta}}

and difficulty measurer

\delta

. High

\delta(\boldsymbol{x}_{n})

suggests a harder instance;

\rho

is the local neighbourhood density,

\hat{p}

is the kernel density estimate, and

\ell

is the training loss.

Dataset	$L$	$H$	$s$	Sample Rate	Subsample	Batch size (repr/curr)
Electricity	96	24	1	1 h	2000	32
ETT	96	24	1	1 h	2000	8 / 32
Weather	12	4	6	10 min	500	8 / 32
ILI	24	4	1	1 week	None	8 / 32
Solar AL	96	24	6	10 min	2000	8 / 32

Table 2: Dataset-specific forecasting and sampling settings used in experiments.

		Dataset					Mean rank
	Strategy	Electricity	ETT	ILI	Weather †	Solar AL	$\downarrow$
One-Pass	No-curriculum	0.486 $\pm$ 0.065	1.005 $\pm$ 0.085	0.865 $\pm$ 0.019	0.201 $\pm$ 0.036	0.080 $\pm$ 0.009	5.00
	Random	0.462 $\pm$ 0.059	1.004 $\pm$ 0.159	0.871 $\pm$ 0.043	0.196 $\pm$ 0.060	0.076 $\pm$ 0.007	4.40
	STL	0.604 $\pm$ 0.069	0.928 $\pm$ 0.137	0.932 $\pm$ 0.031	0.226 $\pm$ 0.071	0.082 $\pm$ 0.007	6.80
	Loss-based	0.399 $\pm$ 0.026	1.051 $\pm$ 0.627	0.946 $\pm$ 0.040	0.169 $\pm$ 0.025	0.103 $\pm$ 0.006	5.30
	$k$ -NN	0.532 $\pm$ 0.101	0.582 $\pm$ 0.159	1.092 $\pm$ 0.191	0.169 $\pm$ 0.053	0.186 $\pm$ 0.022	6.50

Table 3: Test losses (MSE

\downarrow

) across curriculum strategies and datasets . Values are mean

\pm

std over

10

runs. Bold and underlined entries denote best and second-best performance within each dataset and schedule (One-Pass or Baby-Steps). The last column reports the mean rank (

\downarrow

) per strategy over datasets (ties averaged). Weather † values are multiplied by

10^{2}

for readability.

Panel of Figure 5 — Figure 5: Percentage improvement over the No-curriculum baseline. Top: The results concern each strategy (rows) and dataset/schedule combination (columns). Green cells indicate improvement; red cells indicate degradation. Bottom: Radar plots of percentage improvement over No-curriculum per strategy, separately for the One-Pass (left) and Baby-Steps (right) schedules. Each axis corresponds to a dataset; polygons further from the black reference circle (0% improvement) indicate stronger gains.

Panel of Figure 6 — Figure 6: Training dynamics under different CL strategies. (a)(b): Training curves of different strategies across buckets under the Baby-Steps schedule. Distribution of instance train losses for two dataset-strategy pairs. (c)(d)(e): Positions corresponds to instance-wise train loss in the space of strategies and coloring to the bucketing of the $x$ -axis strategy. Instances located above the $y=x$ line are considered easier by the $x$ -axis strategy (and conversely)

Curriculum Learning

Curriculum learning organizes training examples, tasks, or environments to match how a model learns.

See all 173 papers

Confusion-Aware Transfer Teacher Curriculum Learning Framework: Disentangling Scoring and Pacing Effects
Jun 16, 2026Savini Kommalage, Sanka Mohottala, Asiri Gawesha +5Training Data Curation Curriculum Learning
Faculty of Computing, Sri Lanka Institute of Information Technology, Sri Lanka · Faculty of Engineering, University of Sri Jayewardenepura, Sri Lanka · Faculty of Engineering, Sri Lanka Institute of Information Technology, Sri Lanka +2
Curriculum Learning as Transport: Understanding Curricula with Wasserstein Geodesics
Sep 8, 2026Changho Shin, David Alvarez-MelisCurriculum Learning
Princeton University · Microsoft Research & Harvard University
Understanding Curriculum Learning in Large Language Models via Cross-Difficulty Optimization Dynamics
Aug 18, 2026Zhikai Ding, Ziyi YeCurriculum Learning LLM Training
Fudan University

Jun 16, 2026J/K move · Enter open · S save

Confusion-Aware Transfer Teacher Curriculum Learning Framework: Disentangling Scoring and Pacing Effects

Savini Kommalage, Sanka Mohottala, Asiri Gawesha, Dulara Madhusanka+4

Faculty of Computing, Sri Lanka Institute of Information Technology, Sri Lanka · Faculty of Engineering, University of Sri Jayewardenepura, Sri Lanka · Faculty of Engineering, Sri Lanka Institute of Information Technology, Sri Lanka+2

Training Data Curation Curriculum Learning Data-Efficient Learning

Open paper AskPDF arXiv

Curriculum learning couples two design choices, how samples are scored by difficulty and how harder samples are paced into training, making it difficult to attribute observed gains to either component. We disentangle these factors with two evaluation protocols: stage-wise test subsets that validate scoring functions independently of curriculum training, and a baseline that applies the same pacing schedule to randomly ordered data. Within the Transfer Teacher framework (TTF), we use these protocols to evaluate a confusion-aware difficulty score that considers both correct-class confidence and the probability distribution over incorrect classes. On CIFAR-10 with ResNet-18 and VGG-16, the proposed score produces model-interpretable difficulty rankings that align with human intuition. However, at full data, neither curriculum nor anti-curriculum ordering improves accuracy over standard training, indicating that improving the scoring function alone is insufficient to overcome the known failure modes of curriculum learning in TTF. In contrast, We find that confusion-aware curriculum ordering result in consistent data-efficiency benefits, outperforming random ordering by up to 8.7% points at the 20% data regime, suggesting the potential of TTF as a data-efficient training method.

Abstract

Curriculum Learning

Explore similar work

Confusion-Aware Transfer Teacher Curriculum Learning Framework: Disentangling Scoring and Pacing Effects

Curriculum Learning as Transport: Understanding Curricula with Wasserstein Geodesics

Understanding Curriculum Learning in Large Language Models via Cross-Difficulty Optimization Dynamics