What Is the Influence of Task Order in Continual Learning? An Empirical Study and Order-Aware Evaluation for Motion Forecasting

Zirui Li; Yunlong Lin; Haohan Yang; Xiaocong Zhao; Chen Lv

cs.LGNov 3, 2026

What Is the Influence of Task Order in Continual Learning? An Empirical Study and Order-Aware Evaluation for Motion Forecasting

Authors: Zirui Li, Yunlong Lin, Haohan Yang, Xiaocong Zhao, Chen Lv

Organizations: School of Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore 639798 · School of Mechanical Engineering, Beijing Institute of Technology, Beijing 100081, China, and also with the School of Computation, Information and Technology, Technical University of Munich, 85748 Garching, Germany · Department of Civil and Environmental Engineering, Samueli School of Engineering, University of California, Los Angeles, CA 90095 USA · Key Laboratory of Road and Traffic Engineering, Ministry of Education, Tongji University, Shanghai 201804, China

Abstract

Aggregate scores for continual learning algorithms rest on a distribution over task orders that is rarely stated. We formulate algorithm comparison as an expectation over a declared order distribution, which separates training variation from order variation, and apply it to motion forecasting with four road-layout tasks from the INTERACTION dataset, all 24 task orders, four algorithms, and three training replicates (288 runs). Task order accounts for 65% to 91% of the variation in current-task error and backward transfer, and 90% to 99% of this order effect comes from the identity of the starting task: the smallest task cannot be learned from a cold start. Beyond the first stage, the order effect is comparable to training variability. In most orders, the leading algorithm in current-task error and backward transfer is within one replicate standard deviation of the runner-up. Backward transfer depends on the order mainly through its reference term, which weights the first task most heavily, so relative improvement and final error identify different leading algorithms for displacement error. A transition-direction descriptor computed from task statistics tracks the position of the smallest task, and weighting orders by it leaves the algorithm ranking unchanged. These results call for benchmarks that declare their order distribution and report order-resolved, starting-task-resolved, warm-start, and final errors alongside aggregate scores. The code and data are available here.

Figures & tables

Fig. 1: Task order and algorithm comparison. (a) Current-task final displacement error (FDE-CT, m), averaged over three replicates, and algorithm ranking for two orders that differ only in their final two tasks. In both orders the leader lies within one pooled replicate standard deviation of the runner-up. (b) A common order policy $q(o)$ , uniform or weighted by the transition descriptor $D(o)$ , aggregates the order-resolved outcomes of every algorithm into a uniform mean or weighted score, shown with the range across orders.

Fig. 2: Order-aware evaluation workflow. (1) Tasks differ in size and speed distribution. (2) Each algorithm learns every order, starting cold in the first stage. (3) Testing all tasks after each stage gives the stage–task errors $R_{r,i}$ , summarised by CT, $\mathrm{CT}_{2:N}$ , $\mathrm{BWT}=P-B$ , and the final error. (4) W1 speed distances between consecutive tasks, $\Delta_{r}$ , define the descriptor $D(o)$ . (5) A common order policy $q(o)$ yields policy scores, and a sum-of-squares decomposition (FDE-CT) and a replicate bootstrap assess them. Networks, policy weights, and comparison marks are schematic.

Fig. 3: Task feature distributions. Dots, bands, and whiskers indicate medians, interquartile ranges, and 5th–95th percentiles. Acceleration is restricted to

[-5,5]

m/s 2 .

Fig. 4: UQnet backbone (adapted from [ 20 ] ): (a) encoder, (b) regularisation network, used during training and removed at inference, and (c) predictor, which decodes fused map and vehicle features with position features into an endpoint heatmap.

Algorithm	Stored examples	Output matching	Scenario partition	Gradient constraint	Gradient-based selection
Vanilla	$\times$	$\times$	$\times$	$\times$	$\times$
DER	$\checkmark$	$\checkmark$	$\times$	$\times$	$\times$
SyReM	$\checkmark$	$\times$	$\times$	$\checkmark$	$\checkmark$
GSM	$\checkmark$	$\times$	$\checkmark$	$\checkmark$	$\times$

TABLE I: Published mechanisms of the compared algorithms. Stored examples serve output matching, rehearsal, or gradient constraints, and scenario partition allocates memory by past task.

	All 24 orders (%)					Excl. A-first (%)
Metric	Start	Order $\mid$ start	Alg.	Order $\times$ alg.	Within	Order	Alg.	Within
FDE-CT	$69.6$	$1.7$	$1.3$	$11.3$	$16.0$	$30.6$	$2.3$	$50.8$
MR-CT	$58.3$	$6.3$	$1.7$	$10.3$	$23.4$	$24.5$	$2.4$	$51.2$
FDE-BWT	$75.7$	$0.5$	$0.9$	$9.7$	$13.1$	$19.3$	$17.4$	$45.9$
MR-BWT	$89.8$	$1.5$	$3.4$	$1.3$	$4.0$	$39.5$	$26.5$	$25.6$
FDE-CT 2:N	$4.9$	$20.7$	$2.3$	$20.2$	$52.0$	$24.6$	$1.8$	$53.1$

TABLE II: Percentage of the total centred sum of squares per component. The order term is split into starting-task identity (Start) and order within starting task, and the last three columns give the 18 orders that do not start with A.

Order	Vanilla		DER		SyReM		GSM
	FDE-CT	MR-CT	FDE-CT	MR-CT	FDE-CT	MR-CT	FDE-CT	MR-CT
A $\to$ B $\to$ C $\to$ D	$8.25_{\pm 0.46}$	$32.83_{\pm 2.59}$	$6.17_{\pm 2.13}$	$31.75_{\pm 1.97}$	$7.17_{\pm 3.71}$	$31.74_{\pm 1.65}$	$6.08_{\pm 1.36}$	$31.84_{\pm 0.56}$
A $\to$ B $\to$ D $\to$ C	$4.98_{\pm 2.49}$	$30.83_{\pm 4.43}$	$6.67_{\pm 2.39}$	$32.20_{\pm 1.14}$	$6.55_{\pm 2.48}$	$28.76_{\pm 2.27}$	$8.00_{\pm 3.49}$	$34.38_{\pm 0.52}$
A $\to$ C $\to$ B $\to$ D	$7.33_{\pm 3.01}$	$33.28_{\pm 4.02}$	$6.69_{\pm 2.57}$	$29.44_{\pm 1.99}$	$3.31_{\pm 0.31}$	$25.33_{\pm 0.67}$	$6.23_{\pm 1.09}$	$31.86_{\pm 2.69}$
A $\to$ C $\to$ D $\to$ B	$9.26_{\pm 1.95}$	$29.96_{\pm 1.81}$	$3.48_{\pm 1.02}$	$28.85_{\pm 3.15}$	$4.71_{\pm 1.20}$	$27.92_{\pm 4.05}$	$5.00_{\pm 1.84}$	$30.21_{\pm 1.17}$
A $\to$ D $\to$ B $\to$ C	$9.20_{\pm 1.98}$	$31.51_{\pm 2.50}$	$3.38_{\pm 0.70}$	$28.18_{\pm 4.41}$	$3.90_{\pm 1.97}$	$26.08_{\pm 6.67}$	$9.17_{\pm 2.11}$	$35.83_{\pm 0.44}$

TABLE III: Current-task errors by task order (mean

\pm

sample standard deviation over three replicates). FDE is in metres and MR in percent. Bold: per-order minimum separable from the runner-up (Section IV-D ).

Order	Vanilla		DER		SyReM		GSM
	FDE-BWT	MR-BWT	FDE-BWT	MR-BWT	FDE-BWT	MR-BWT	FDE-BWT	MR-BWT
A $\to$ B $\to$ C $\to$ D	$-16.39_{\pm 1.57}$	$-46.19_{\pm 3.15}$	$-10.99_{\pm 5.05}$	$-41.41_{\pm 0.95}$	$-14.77_{\pm 8.97}$	$-52.95_{\pm 2.18}$	$-12.00_{\pm 3.63}$	$-54.18_{\pm 2.86}$
A $\to$ B $\to$ D $\to$ C	$-8.41_{\pm 5.26}$	$-42.47_{\pm 2.39}$	$-12.28_{\pm 5.50}$	$-44.11_{\pm 2.93}$	$-13.55_{\pm 5.89}$	$-54.58_{\pm 3.96}$	$-15.61_{\pm 9.07}$	$-48.23_{\pm 11.43}$
A $\to$ C $\to$ B $\to$ D	$-13.97_{\pm 7.56}$	$-47.29_{\pm 3.65}$	$-13.49_{\pm 6.02}$	$-46.46_{\pm 2.77}$	$-5.96_{\pm 0.70}$	$-50.20_{\pm 1.11}$	$-12.35_{\pm 2.34}$	$-49.80_{\pm 3.44}$
A $\to$ C $\to$ D $\to$ B	$-19.84_{\pm 4.77}$	$-42.47_{\pm 11.19}$	$-5.36_{\pm 2.77}$	$-37.45_{\pm 12.64}$	$-9.18_{\pm 3.12}$	$-53.14_{\pm 5.23}$	$-8.97_{\pm 5.24}$	$-48.22_{\pm 7.97}$
A $\to$ D $\to$ B $\to$ C	$-19.42_{\pm 4.94}$	$-45.86_{\pm 2.94}$	$-5.44_{\pm 1.35}$	$-40.02_{\pm 8.01}$	$-7.15_{\pm 4.53}$	$-45.12_{\pm 8.76}$	$-18.75_{\pm 5.52}$	$-50.08_{\pm 2.73}$

TABLE IV: Backward transfer by task order (mean

\pm

sample standard deviation over three replicates). FDE is in metres and MR in percentage points. Bold as in Table III .

Start	$R_{1,1}$	CT	CT 2:N	$P$	$B$	BWT	Final
A	$21.52$	$6.25$	$1.16$	$1.46$	$13.69$	$-12.23$	$1.28$
B	$2.77$	$1.81$	$1.48$	$2.34$	$2.46$	$-0.12$	$1.52$
C	$1.04$	$1.12$	$1.15$	$1.64$	$1.13$	$0.52$	$1.50$
D	$1.32$	$1.45$	$1.50$	$2.04$	$1.50$	$0.54$	$1.48$

TABLE V: FDE (m) by starting task over 72 runs per row: initial error

R_{1,1}

, CT with and without (CT 2:N ) the first stage, subsequent error

P

, reference

B

, BWT

=P-B

, and final error.

Algorithm	Metric	W1 speed	KL speed	JS speed	MMD speed	W1 accel.
Vanilla	FDE-CT	$-0.75$	$-0.23$	$-0.20$	$-0.75$	$+0.69$
	MR-CT	$-0.75$	$-0.09$	$-0.12$	$-0.75$	$+0.69$
	FDE-BWT	$+0.66$	$+0.13$	$+0.10$	$+0.66$	$-0.55$
	MR-BWT	$+0.81$	$+0.26$	$+0.32$	$+0.81$	$-0.46$
DER	FDE-CT	$-0.57$	$-0.29$	$-0.36$	$-0.57$	$+0.46$
	MR-CT	$-0.54$	$-0.22$	$-0.27$	$-0.54$	$+0.48$

TABLE VI: Spearman correlations between candidate transition-direction descriptors and order-level performance. W1: Wasserstein-1. Bold:

|\rho|\geq 0.7

Fig. 5: Transition-direction descriptor versus performance for each algorithm. Points are replicate means for individual orders, coloured by the position of task A, and

\rho

is the Spearman correlation with

D

. Grey guides mark the six distinct descriptor values.

Metric	$\rho(D)$	Excl. A-first	Start-adj.	$\rho(p_{A})$	$p_{A}$ -adj.
FDE-CT	$-0.64$	$-0.47$	$-0.48$	$-0.77$	$-0.23$
MR-CT	$-0.63$	$-0.43$	$-0.49$	$-0.79$	$-0.11$
FDE-BWT	$+0.61$	$+0.40$	$+0.29$	$+0.73$	$+0.22$
MR-BWT	$+0.74$	$+0.64$	$+0.62$	$+0.77$	$+0.53$
FDE-CT 2:N	$-0.37$	$-0.49$	$-0.58$	$-0.20$	$-0.27$
MR-CT 2:N	$-0.27$	$-0.47$	$-0.50$	$-0.10$	$-0.20$

TABLE VII: Mean Spearman associations across algorithms.

\rho(D)

: descriptor and order means. Excl. A-first: 18 orders. Start-adj.: rank residuals after removing starting-task means.

\rho(p_{A})

: position of task A.

p_{A}

-adj.: rank residuals after removing position-of-A means.

Algorithm	FDE: mean [min, max]	Prob.	MR: mean [min, max]	Prob.
Vanilla	1.63 [1.18, 2.14]	$0.00$	17.22 [10.84, 25.67]	$0.00$
DER	1.63 [1.17, 2.34]	$0.00$	17.80 [11.47, 29.09]	$0.00$
SyReM	1.08 [0.87, 1.99]	$1.00$	9.23 [4.38, 24.69]	$1.00$
GSM	1.43 [1.03, 2.44]	$0.00$	15.86 [8.05, 36.25]	$0.00$

TABLE VIII: Final errors: mean [minimum, maximum] over order-level replicate means, and the bootstrap probability (Prob.) of the smallest uniform mean. FDE is in metres and MR in percent. Bold: smallest mean.

Fig. 6: Paths with the largest and smallest total transition distance

L

for each starting task. Order strips identify the paths, and paired points compare algorithm-averaged scores, standardised separately for each metric across task orders.

Metric	Policy	Vanilla	DER	SyReM	GSM
FDE-CT	Uniform	$2.94$ (0.00)	$2.42$ (0.29)	$2.33$ (0.71)	$2.94$ (0.00)
	$D$ -weighted	$3.76$ (0.00)	$3.06$ (0.43)	$3.01$ (0.57)	$3.76$ (0.00)
	Excl. A-first	$1.43$ (0.06)	$1.48$ (0.04)	$1.31$ (0.90)	$1.62$ (0.00)
MR-CT	Uniform	$18.10$ (0.00)	$17.30$ (0.05)	$16.04$ (0.94)	$19.58$ (0.00)
	$D$ -weighted	$21.87$ (0.00)	$20.12$ (0.21)	$19.34$ (0.79)	$22.98$ (0.00)
	Excl. A-first	$13.86$ (0.01)	$13.32$ (0.09)	$11.98$ (0.89)	$15.19$ (0.00)

TABLE IX: Policy scores (units as in Tables III and IV ) under the uniform policy, the descriptor-weighted policy, and the uniform policy over the 18 orders that do not start with A, with the bootstrap probability of the smallest score under that policy in parentheses. Bold: smallest score per row.

Forecasting

See all 9 papers

Apr 15, 2026

From Order to Distribution: An Exact Operator Framework for Forgetting in Continual Learning

A central challenge in continual learning is forgetting: the loss of performance on previously learned tasks after learning new ones. Prior theory has analyzed forgetting under random orderings of fixed task collections in overparameterized linear regression. We shift the focus from task order to task distribution, asking how its structure determines forgetting. In the linear setting with a shared solution, i.i.d. task sampling, and sequential exact fitting, we derive an exact operator identity expressing historical forgetting directly in terms of the task distribution. Building on this identity, we establish an exponential decay guarantee for expected historical forgetting under every fixed task distribution in finite dimensions, characterize its asymptotic behavior, and relate decay to the distribution's coverage of observable directions. For an individual learned task, we show that subsequent tasks can collectively support recovery without exact revisits. We derive a lower bound on recovery time and construct a task distribution attaining its inverse-coverage scaling.

Zonghuan Xu, Xingjun Ma

Institute of Trustworthy Embodied AI, Fudan University, Shanghai, China · Shanghai Key Laboratory of Multimodal Embodied AI, Shanghai, China

Continual Learning Catastrophic Forgetting

Ask PDF

Apr 23, 2026

Temporal Taskification in Streaming Continual Learning: A Source of Evaluation Instability

Streaming Continual Learning (CL) typically converts a continuous stream into a sequence of discrete tasks through temporal partitioning. We argue that this temporal taskification step is not a neutral preprocessing choice, but a structural component of evaluation: different valid splits of the same stream can induce different CL regimes and therefore different benchmark conclusions. To study this effect, we introduce a taskification-level framework based on plasticity and stability profiles, a profile distance between taskifications, and Boundary-Profile Sensitivity (BPS), which diagnoses how strongly small boundary perturbations alter the induced regime before any CL model is trained. We evaluate continual finetuning, Experience Replay, Elastic Weight Consolidation, and Learning without Forgetting on network traffic forecasting with CESNET-Timeseries24, keeping the stream, model, and training budget fixed while varying only the temporal taskification. Across 9-, 30-, and 44-day splits, we observe substantial changes in forecasting error, forgetting, and backward transfer, showing that taskification alone can materially affect CL evaluation. We further find that shorter taskifications induce noisier distribution-level patterns, larger structural distances, and higher BPS, indicating greater sensitivity to boundary perturbations. These results show that benchmark conclusions in streaming CL depend not only on the learner and the data stream, but also on how that stream is taskified, motivating temporal taskification as a first-class evaluation variable.

Nicolae Filat, Ahmed Hussain, Konstantinos Kalogiannis +1

Bitdefender, Romania · KTH Royal Institute of Technology · Stockholm, Sweden +2

Continual Learning

Ask PDF

Oct 30, 2026

Implicit Assumptions About Tasks in Continual Learning

Continual learning is the study of how a model accumulates knowledge from data received over time, rather than all at once. In practice, this is typically achieved by dividing a dataset into tasks and presenting them sequentially. Explanations of the resulting measurements take into account factors such as the similarity between segments, the rate at which the data changes and the order of presentation. Each of these factors can be defined in terms of the underlying data, the partition imposed on it, or the learner that encounters it. These levels do not necessarily coincide, and it is usually left implicit which level a reported quantity refers to. We separate these and identify which properties acquire a value at each level. In controlled teacher--student experiments, we demonstrate that partitioning the same underlying trajectory at different levels of granularity, changing the order of presentation, and varying the similarity structure of the underlying data all result in different measured outcomes. We review representative benchmarks to determine which of these properties are stated. Where the underlying data cannot be inspected, as in foundation models built on undisclosed pre-training corpora, its properties are inherited by the model's initial state, and the training applied on top of it can be described.

Kathrin Korte, Christian Metzner, Akshay Bedhotiya +3

IT University of Copenhagen Copenhagen, Denmark · Institute of Neuroinformatics University of Zurich and ETH Zurich · Peter Grünberg Institute Forschungszentrum Jülich & RWTH Aachen

Continual Learning

Ask PDF

Abstract

Forecasting

Explore similar work

From Order to Distribution: An Exact Operator Framework for Forgetting in Continual Learning

Temporal Taskification in Streaming Continual Learning: A Source of Evaluation Instability

Implicit Assumptions About Tasks in Continual Learning