TaPD: Temporal-adaptive Progressive Distillation for Observation-Adaptive Trajectory Forecasting in Autonomous Driving

Mingyu Fan; Yi Liu; Hao Zhou; Deheng Qian; Mohammad Haziq Khan; Matthias Raetsch

cs.CVMar 6, 2026

TaPD: Temporal-adaptive Progressive Distillation for Observation-Adaptive Trajectory Forecasting in Autonomous Driving

Authors: Mingyu Fan, Yi Liu, Hao Zhou, Deheng Qian, Mohammad Haziq Khan, Matthias Raetsch

Organizations: College of Information and Intelligent Science, Donghua University, Shanghai 201620 China. · School of Computing and Information Technology, Great Bay Institute for Advanced Study/Great Bay University, Dongguan, China, and also with the Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China. · Chongqing Chang’an Technology Co., Ltd, Chongqing 401120 China. · ViSiR, Reutlingen University, Alteburgstraße 150, 72762, Reutlingen, Germany.

Abstract

Trajectory prediction is essential for autonomous driving, enabling vehicles to anticipate the motion of surrounding agents to support safe planning. However, most existing predictors assume fixed-length histories and suffer substantial performance degradation when observations are variable or extremely short in real-world settings (e.g., due to occlusion or a limited sensing range). We propose TaPD (Temporal-adaptive Progressive Distillation), a unified plug-and-play framework for observation-adaptive trajectory forecasting under variable history lengths. TaPD comprises two cooperative modules: an Observation-Adaptive Forecaster (OAF) for future prediction and a Temporal Backfilling Module (TBM) for explicit reconstruction of the past. OAF is built on progressive knowledge distillation (PKD), which transfers motion pattern knowledge from long-horizon "teachers" to short-horizon "students" via hierarchical feature regression, enabling short observations to recover richer motion context. We further introduce a cosine-annealed distillation weighting scheme to balance forecasting supervision and feature alignment, improving optimization stability and cross-length consistency. For extremely short histories where implicit alignment is insufficient, TBM backfills missing historical segments conditioned on scene evolution, producing context-rich trajectories that strengthen PKD and thereby improve OAF. We employ a decoupled pretrain-reconstruct-finetune protocol to preserve real-motion priors while adapting to backfilled inputs. Extensive experiments on Argoverse 1 and Argoverse 2 show that TaPD consistently outperforms strong baselines across all observation lengths, delivers especially large gains under very short inputs, and improves other predictors (e.g., HiVT) in a plug-and-play manner. Code will be available at https://github.com/zhouhao94/TaPD.

Figures & tables

Fig. 1 : The OAF pipeline with parameter sharing and progressive knowledge distillation.

Fig. 2 : The TBM temporal backfilling module for unobserved past trajectories backfilling.

Fig. 3 : The overview of TaPD training strategy.

Method	Argoverse 2 ( $\textit{minADE}_{6}$ / $\textit{minFDE}_{6}$ )					Argoverse 1 ( $\textit{minADE}_{6}$ / $\textit{minFDE}_{6}$ )
Method	10Ts	20Ts	30Ts	40Ts	50Ts	5Ts	10Ts	15Ts	20Ts
DeMo_Ori	0.861/1.533	0.700/1.358	0.671/1.306	0.662/1.288	0.658/1.278	0.781/1.267	0.662/1.087	0.624/1.011	0.606/1.003
DeMo_IT	0.675/1.318	0.661/1.296	0.660/1.293	0.659/1.287	0.658/1.278	0.669/1.078	0.634/1.031	0.612/0.988	0.606/1.003
DeMo_DTO	0.672/1.307	0.658/1.291	0.650/1.279	0.647/1.268	0.645/1.265	0.662/1.064	0.628/1.025	0.605/0.991	0.599/1.010
DeMo_FLN	0.651/1.262	0.644/1.258	0.637/1.254	0.628/1.238	0.621/1.231	0.646/1.043	0.607/0.994	0.599/0.974	0.592/0.957
DeMo_LaKD	0.639 /1.262	0.627 /1.251	0.620 /1.243	0.617/1.236	0.617/1.232	0.631 /1.008	0.593/0.976	0.584/0.933	0.581/0.929

TABLE I : Variable-length trajectory prediction comparison on Argoverse 2 (left) and Argoverse 1 (right) validation sets. Best results are in bold and second best are underlined . Ts denotes the time steps of

T_{\tau},a.k.a.,\tau\Delta T

Method	$\textit{minFDE}_{6}$	$\textit{minADE}_{6}$	$\textit{MR}_{6}$
LGT [ 56 ]	1.93	1.02	0.30
SIMPL [ 57 ]	1.45	0.78	-
forecast-mae [ 32 ]	1.41	0.80	0.18
SeNeVA [ 58 ]	1.32	0.71	0.17
Realmotion [ 54 ]	1.31	0.66	0.15
QCNet [ 4 ]	1.27	0.73	0.16

TABLE II : Performance comparison on Argoverse 2 validation set . ”-” denotes that this result was not reported in their paper.

Method	$\textit{minADE}_{6}$	$\textit{minFDE}_{6}$	$\textit{MR}_{6}$
LTP [ 64 ]	0.78	1.07	-
LaneRCNN [ 17 ]	0.77	1.19	0.08
TPCN [ 65 ]	0.73	1.15	0.11
DenseTNT [ 27 ]	0.73	1.05	0.10
TNT [ 26 ]	0.73	1.29	0.09
mmTransformer [ 66 ]	0.71	1.15	0.11

TABLE III : Performance comparison on Argoverse 1 validation set . ”-” denotes that this result was not reported in their paper.

PS(OAF)	PKD(OAF)	TBM	$\textit{minADE}_{6}$ / $\textit{minFDE}_{6}$
PS(OAF)	PKD(OAF)	TBM	10Ts	20Ts	30Ts	40Ts	50Ts
			0.675/1.318	0.661/1.296	0.660/1.293	0.659/1.287	0.658/1.278
✓			0.650/1.289	0.637/1.256	0.629/1.238	0.624/1.226	0.616/1.211
✓	✓		0.621/1.216	0.610/1.200	0.608/1.196	0.604/1.185	0.601/1.172
✓	✓	✓	0.617/1.203	0.603/1.167	0.599/1.157	0.599/1.155	0.599/1.153

TABLE IV : Ablation study on the core components of TaPD on the Argoverse 2 validation set . PS denotes parameter-sharing mechanism.

Method	Combination	$\textit{minADE}_{6}$ / $\textit{minFDE}_{6}$
Method	Combination	10Ts	20Ts	30Ts	40Ts	50Ts
DeMo_IT	-	0.675/1.318	0.661/1.296	0.660/1.293	0.659/1.287	0.658/1.278
DeMo_TaPD-3	30/40/50	0.671/1.313	0.627/1.224	0.609/1.201	0.608/1.195	0.605/1.174
DeMo_TaPD-4	20/30/40/50	0.651/1.274	0.606/1.188	0.605/1.182	0.605/1.178	0.603/1.162
DeMo_TaPD-5	10/20/30/40/50	0.617/1.203	0.603/1.167	0.599/1.157	0.599/1.155	0.599/1.153

TABLE V : Ablation study of different length combinations of observation lengths on the Argoverse 2 single-agent validation set .

Method	$\textit{minADE}_{6}$ / $\textit{minFDE}_{6}$
Method	5Ts	10Ts	15Ts	20Ts
HiVT_Ori	0.943/1.540	0.768/1.201	0.732/1.124	0.702/1.070
HiVT_IT	0.781/1.224	0.738/1.121	0.711/1.083	0.702/1.070
HiVT_FLN	0.754/1.162	0.723/1.089	0.701/1.053	0.681/1.028
HiVT_TaPD(Ours)	0.704/1.083	0.662/1.012	0.674/1.003	0.660/0.964

TABLE VI : Performance Comparison of TaPD on HiVT on Argoverse 1 single-agent validation set .

Argoverse 2 ( $\textit{minADE}_{6}$ / $\textit{minFDE}_{6}$ )
Module	10Ts	20Ts	30Ts	40Ts
TBM	0.382/0.723	0.275/0.501	0.185/0.316	0.097/0.160

TABLE VII : Variable-length trajectory prediction performance of TBM on Argoverse 2 (upper) and Argoverse 1 (lower) validation sets.

Len	DeMo_IT			DeMo_TaPD
Len	Params $\downarrow$	FLOPs $\downarrow$	Lat. $\downarrow$	Params $\downarrow$	FLOPs $\downarrow$	Lat. $\downarrow$
10Ts	3.431	0.678	23.370	6.831	1.213	88.568
20Ts	3.431	0.680	23.578	6.815	1.140	80.626
30Ts	3.431	0.683	23.878	6.800	1.066	84.238
40Ts	3.431	0.685	23.760	6.784	0.992	86.447

TABLE VIII : Parameters (M), FLOPs (G), and latency (ms) under different input lengths.

Panel of Figure 12 — Fig. 4 : Qualitative comparison on the Argoverse 2 single-agent validation set under short observations (10 time steps). Each row corresponds to one scene, and the three columns show predictions from (a) DeMo_IT, (b) DeMo_OAF, and (c) our DeMo_TaPD, respectively, given the same input. The observed history is shown as a solid black segment. The ground-truth future trajectory is shown in solid red with an arrow. Green dashed arrows denote multiple predicted trajectories. For methods with backfilling, the reconstructed (unobserved) history is shown as a blue dashed segment, and the corresponding ground-truth unobserved history segment is shown in solid orange.

Trajectory Prediction

Trajectory prediction forecasts how people, vehicles, robots, and other dynamic systems will move from observed histories and context.

See all 186 papers

Recover to Predict: Progressive Retrospective Learning for Variable-Length Trajectory Prediction
Mar 11, 2026Hao Zhou, Lu Qi, Jason Li +5Trajectory Prediction Autonomous Driving
Great Bay University · Tsinghua SIGS · Wuhan University +3
MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
Apr 11, 2026Wenchang Duan, Zhenguo Gao, Jinguo Xian +1Model Compression Multi-Agent Trajectory Prediction
School of Mathematical Sciences, Shanghai Jiao Tong University, Shanghai 200240, China · Bio-X Institutes, Key Laboratory for the Genetics of Developmental and Neuropsychiatric Disorders, Shanghai Jiao Tong University, Shanghai 200030, China · Shanghai Key Laboratory of Psychotic Disorders, and Brain Science and Technology Research Center, Shanghai Jiao Tong University, Shanghai 200030, China
Towards Metric-Agnostic Trajectory Forecasting
Jul 1, 2026Markus Knoche, Daan de Geus, Bastian LeibeAutonomous Driving Benchmarks Multi-Agent Trajectory Prediction
RWTH Aachen University, Germany · Eindhoven University of Technology, Netherlands

Mar 11, 2026J/K move · Enter open · S save

Recover to Predict: Progressive Retrospective Learning for Variable-Length Trajectory Prediction

Hao Zhou, Lu Qi, Jason Li, Jie Zhang+4

Great Bay University · Tsinghua SIGS · Wuhan University+3

Trajectory Prediction Autonomous Driving

Open paper AskPDF arXiv

Trajectory prediction is critical for autonomous driving, enabling safe and efficient planning in dense, dynamic traffic. Most existing methods optimize prediction accuracy under fixed-length observations. However, real-world driving often yields variable-length, incomplete observations, posing a challenge to these methods. A common strategy is to directly map features from incomplete observations to those from complete ones. This one-shot mapping, however, struggles to learn accurate representations for short trajectories due to significant information gaps. To address this issue, we propose a Progressive Retrospective Framework (PRF), which gradually aligns features from incomplete observations with those from complete ones via a cascade of retrospective units. Each unit consists of a Retrospective Distillation Module (RDM) and a Retrospective Prediction Module (RPM), where RDM distills features and RPM recovers previous timesteps using the distilled features. Moreover, we propose a Rolling-Start Training Strategy (RSTS) that enhances data efficiency during PRF training. PRF is plug-and-play with existing methods. Extensive experiments on datasets Argoverse 2 and Argoverse 1 demonstrate the effectiveness of PRF. Code is available at https://github.com/zhouhao94/PRF.

Abstract

Trajectory Prediction

Explore similar work

Recover to Predict: Progressive Retrospective Learning for Variable-Length Trajectory Prediction

MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction

Towards Metric-Agnostic Trajectory Forecasting