TaPD: Temporal-adaptive Progressive Distillation for Observation-Adaptive Trajectory Forecasting in Autonomous Driving

Mingyu Fan; Yi Liu; Hao Zhou; Deheng Qian; Mohammad Haziq Khan; Matthias Raetsch

cs.CVMar 6, 2026

TaPD: Temporal-adaptive Progressive Distillation for Observation-Adaptive Trajectory Forecasting in Autonomous Driving

Authors: Mingyu Fan, Yi Liu, Hao Zhou, Deheng Qian, Mohammad Haziq Khan, Matthias Raetsch

Organizations: College of Information and Intelligent Science, Donghua University, Shanghai 201620 China. · School of Computing and Information Technology, Great Bay Institute for Advanced Study/Great Bay University, Dongguan, China, and also with the Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China. · Chongqing Chang’an Technology Co., Ltd, Chongqing 401120 China. · ViSiR, Reutlingen University, Alteburgstraße 150, 72762, Reutlingen, Germany.

Abstract

Trajectory prediction is essential for autonomous driving, enabling vehicles to anticipate the motion of surrounding agents to support safe planning. However, most existing predictors assume fixed-length histories and suffer substantial performance degradation when observations are variable or extremely short in real-world settings (e.g., due to occlusion or a limited sensing range). We propose TaPD (Temporal-adaptive Progressive Distillation), a unified plug-and-play framework for observation-adaptive trajectory forecasting under variable history lengths. TaPD comprises two cooperative modules: an Observation-Adaptive Forecaster (OAF) for future prediction and a Temporal Backfilling Module (TBM) for explicit reconstruction of the past. OAF is built on progressive knowledge distillation (PKD), which transfers motion pattern knowledge from long-horizon "teachers" to short-horizon "students" via hierarchical feature regression, enabling short observations to recover richer motion context. We further introduce a cosine-annealed distillation weighting scheme to balance forecasting supervision and feature alignment, improving optimization stability and cross-length consistency. For extremely short histories where implicit alignment is insufficient, TBM backfills missing historical segments conditioned on scene evolution, producing context-rich trajectories that strengthen PKD and thereby improve OAF. We employ a decoupled pretrain-reconstruct-finetune protocol to preserve real-motion priors while adapting to backfilled inputs. Extensive experiments on Argoverse 1 and Argoverse 2 show that TaPD consistently outperforms strong baselines across all observation lengths, delivers especially large gains under very short inputs, and improves other predictors (e.g., HiVT) in a plug-and-play manner. Code will be available at https://github.com/zhouhao94/TaPD.

Figures & tables

Fig. 1 : The OAF pipeline with parameter sharing and progressive knowledge distillation.

Fig. 2 : The TBM temporal backfilling module for unobserved past trajectories backfilling.

Fig. 3 : The overview of TaPD training strategy.

Method	Argoverse 2 ( $\textit{minADE}_{6}$ / $\textit{minFDE}_{6}$ )					Argoverse 1 ( $\textit{minADE}_{6}$ / $\textit{minFDE}_{6}$ )
Method	10Ts	20Ts	30Ts	40Ts	50Ts	5Ts	10Ts	15Ts	20Ts
DeMo_Ori	0.861/1.533	0.700/1.358	0.671/1.306	0.662/1.288	0.658/1.278	0.781/1.267	0.662/1.087	0.624/1.011	0.606/1.003
DeMo_IT	0.675/1.318	0.661/1.296	0.660/1.293	0.659/1.287	0.658/1.278	0.669/1.078	0.634/1.031	0.612/0.988	0.606/1.003
DeMo_DTO	0.672/1.307	0.658/1.291	0.650/1.279	0.647/1.268	0.645/1.265	0.662/1.064	0.628/1.025	0.605/0.991	0.599/1.010
DeMo_FLN	0.651/1.262	0.644/1.258	0.637/1.254	0.628/1.238	0.621/1.231	0.646/1.043	0.607/0.994	0.599/0.974	0.592/0.957
DeMo_LaKD	0.639 /1.262	0.627 /1.251	0.620 /1.243	0.617/1.236	0.617/1.232	0.631 /1.008	0.593/0.976	0.584/0.933	0.581/0.929

TABLE I : Variable-length trajectory prediction comparison on Argoverse 2 (left) and Argoverse 1 (right) validation sets. Best results are in bold and second best are underlined . Ts denotes the time steps of

T_{\tau},a.k.a.,\tau\Delta T

Method	$\textit{minFDE}_{6}$	$\textit{minADE}_{6}$	$\textit{MR}_{6}$
LGT [ 56 ]	1.93	1.02	0.30
SIMPL [ 57 ]	1.45	0.78	-
forecast-mae [ 32 ]	1.41	0.80	0.18
SeNeVA [ 58 ]	1.32	0.71	0.17
Realmotion [ 54 ]	1.31	0.66	0.15
QCNet [ 4 ]	1.27	0.73	0.16

TABLE II : Performance comparison on Argoverse 2 validation set . ”-” denotes that this result was not reported in their paper.

Method	$\textit{minADE}_{6}$	$\textit{minFDE}_{6}$	$\textit{MR}_{6}$
LTP [ 64 ]	0.78	1.07	-
LaneRCNN [ 17 ]	0.77	1.19	0.08
TPCN [ 65 ]	0.73	1.15	0.11
DenseTNT [ 27 ]	0.73	1.05	0.10
TNT [ 26 ]	0.73	1.29	0.09
mmTransformer [ 66 ]	0.71	1.15	0.11

TABLE III : Performance comparison on Argoverse 1 validation set . ”-” denotes that this result was not reported in their paper.

PS(OAF)	PKD(OAF)	TBM	$\textit{minADE}_{6}$ / $\textit{minFDE}_{6}$
PS(OAF)	PKD(OAF)	TBM	10Ts	20Ts	30Ts	40Ts	50Ts
			0.675/1.318	0.661/1.296	0.660/1.293	0.659/1.287	0.658/1.278
✓			0.650/1.289	0.637/1.256	0.629/1.238	0.624/1.226	0.616/1.211
✓	✓		0.621/1.216	0.610/1.200	0.608/1.196	0.604/1.185	0.601/1.172
✓	✓	✓	0.617/1.203	0.603/1.167	0.599/1.157	0.599/1.155	0.599/1.153

TABLE IV : Ablation study on the core components of TaPD on the Argoverse 2 validation set . PS denotes parameter-sharing mechanism.

Method	Combination	$\textit{minADE}_{6}$ / $\textit{minFDE}_{6}$
Method	Combination	10Ts	20Ts	30Ts	40Ts	50Ts
DeMo_IT	-	0.675/1.318	0.661/1.296	0.660/1.293	0.659/1.287	0.658/1.278
DeMo_TaPD-3	30/40/50	0.671/1.313	0.627/1.224	0.609/1.201	0.608/1.195	0.605/1.174
DeMo_TaPD-4	20/30/40/50	0.651/1.274	0.606/1.188	0.605/1.182	0.605/1.178	0.603/1.162
DeMo_TaPD-5	10/20/30/40/50	0.617/1.203	0.603/1.167	0.599/1.157	0.599/1.155	0.599/1.153

TABLE V : Ablation study of different length combinations of observation lengths on the Argoverse 2 single-agent validation set .

Method	$\textit{minADE}_{6}$ / $\textit{minFDE}_{6}$
Method	5Ts	10Ts	15Ts	20Ts
HiVT_Ori	0.943/1.540	0.768/1.201	0.732/1.124	0.702/1.070
HiVT_IT	0.781/1.224	0.738/1.121	0.711/1.083	0.702/1.070
HiVT_FLN	0.754/1.162	0.723/1.089	0.701/1.053	0.681/1.028
HiVT_TaPD(Ours)	0.704/1.083	0.662/1.012	0.674/1.003	0.660/0.964

TABLE VI : Performance Comparison of TaPD on HiVT on Argoverse 1 single-agent validation set .

Argoverse 2 ( $\textit{minADE}_{6}$ / $\textit{minFDE}_{6}$ )
Module	10Ts	20Ts	30Ts	40Ts
TBM	0.382/0.723	0.275/0.501	0.185/0.316	0.097/0.160

TABLE VII : Variable-length trajectory prediction performance of TBM on Argoverse 2 (upper) and Argoverse 1 (lower) validation sets.

Len	DeMo_IT			DeMo_TaPD
Len	Params $\downarrow$	FLOPs $\downarrow$	Lat. $\downarrow$	Params $\downarrow$	FLOPs $\downarrow$	Lat. $\downarrow$
10Ts	3.431	0.678	23.370	6.831	1.213	88.568
20Ts	3.431	0.680	23.578	6.815	1.140	80.626
30Ts	3.431	0.683	23.878	6.800	1.066	84.238
40Ts	3.431	0.685	23.760	6.784	0.992	86.447

TABLE VIII : Parameters (M), FLOPs (G), and latency (ms) under different input lengths.

Panel of Figure 12 — Fig. 4 : Qualitative comparison on the Argoverse 2 single-agent validation set under short observations (10 time steps). Each row corresponds to one scene, and the three columns show predictions from (a) DeMo_IT, (b) DeMo_OAF, and (c) our DeMo_TaPD, respectively, given the same input. The observed history is shown as a solid black segment. The ground-truth future trajectory is shown in solid red with an arrow. Green dashed arrows denote multiple predicted trajectories. For methods with backfilling, the reconstructed (unobserved) history is shown as a blue dashed segment, and the corresponding ground-truth unobserved history segment is shown in solid orange.

Trajectory Prediction

Trajectory prediction forecasts how people, vehicles, robots, and other dynamic systems will move from observed histories and context.

See all 186 papers

Mar 11, 2026

Recover to Predict: Progressive Retrospective Learning for Variable-Length Trajectory Prediction

Trajectory prediction is critical for autonomous driving, enabling safe and efficient planning in dense, dynamic traffic. Most existing methods optimize prediction accuracy under fixed-length observations. However, real-world driving often yields variable-length, incomplete observations, posing a challenge to these methods. A common strategy is to directly map features from incomplete observations to those from complete ones. This one-shot mapping, however, struggles to learn accurate representations for short trajectories due to significant information gaps. To address this issue, we propose a Progressive Retrospective Framework (PRF), which gradually aligns features from incomplete observations with those from complete ones via a cascade of retrospective units. Each unit consists of a Retrospective Distillation Module (RDM) and a Retrospective Prediction Module (RPM), where RDM distills features and RPM recovers previous timesteps using the distilled features. Moreover, we propose a Rolling-Start Training Strategy (RSTS) that enhances data efficiency during PRF training. PRF is plug-and-play with existing methods. Extensive experiments on datasets Argoverse 2 and Argoverse 1 demonstrate the effectiveness of PRF. Code is available at https://github.com/zhouhao94/PRF.

Hao Zhou, Lu Qi, Jason Li +5

Great Bay University · Tsinghua SIGS · Wuhan University +3

Trajectory Prediction Autonomous Driving

Ask PDF

Apr 11, 2026

MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction

Trajectory prediction is a key component of autonomous driving systems because future motions directly affect collision checking, behavior planning, and control. The task remains challenging under dense interactions, heterogeneous behaviors, multimodal futures, and limited on-board computation. Existing graph, attention, and generative predictors improve interaction reasoning or uncertainty modeling, but their high-capacity designs are often costly for real-time deployment. Lightweight predictors and conventional distillation reduce inference cost, yet usually rely on static imitation and do not explicitly correct safety-relevant teacher bias. This paper proposes MAVEN-T, a reinforced heterogeneous distillation framework for real-time multi-agent trajectory prediction. A high-capacity teacher models directed local interactions with a surround-aware graph encoder, combines efficient temporal filtering with shifted-window spatial attention, and decodes maneuver-specific futures through a sparse Mixture-of-Experts head. A compact GRU--Squeeze-and-Excitation student with a Low-Rank Adapted policy head is trained by feature-, attention-, and semantic-level distillation. To align prediction with downstream behavior, the student is further refined by Proximal Policy Optimization rewards for collision avoidance, comfort, and progress, while a complexity-aware curriculum and Elastic Weight Consolidation stabilize stage-wise training. Experiments on NGSIM, HighD, MoCAD, Argoverse~2, and the Waymo Open Motion Dataset evaluate accuracy, efficiency, generalization, robustness, and closed-loop safety. The student achieves 6.2

\times

parameter compression, 3.7

\times

inference acceleration, and 14.6,ms latency on an NVIDIA Jetson AGX Orin while maintaining competitive accuracy.

Wenchang Duan, Zhenguo Gao, Jinguo Xian +1

School of Mathematical Sciences, Shanghai Jiao Tong University, Shanghai 200240, China · Bio-X Institutes, Key Laboratory for the Genetics of Developmental and Neuropsychiatric Disorders, Shanghai Jiao Tong University, Shanghai 200030, China · Shanghai Key Laboratory of Psychotic Disorders, and Brain Science and Technology Research Center, Shanghai Jiao Tong University, Shanghai 200030, China

Model Compression Multi-Agent Trajectory Prediction Agent Trajectory Analysis Knowledge Distillation Multimodal Trajectory Prediction Trajectory Prediction

Ask PDF

Jul 1, 2026

Towards Metric-Agnostic Trajectory Forecasting

Accurate trajectory forecasting of surrounding traffic participants is a core capability for autonomous driving, enabling vehicles to anticipate behavior and plan safe maneuvers. We observe that current state-of-the-art forecasting models on Argoverse 2 and the Waymo Open Motion Dataset tailor their training objectives to the different benchmark metrics. Because these metrics encourage conflicting behavior, we propose a paradigm change for trajectory forecasting: training models with metric-agnostic probabilistic objectives and treating metric optimization as a downstream task applied to the predictive distribution. Concretely, we introduce Trajectory Distribution Evaluation (TraDiE) policies, metric-specific policies that map a predictive distribution to the set of

K

trajectories and confidences required by trajectory forecasting metrics. We evaluate this framework by introducing DONUT-NLL, which adapts the training objective of the state-of-the-art trajectory forecasting model DONUT to directly optimize the predictive distribution. Using our policies, DONUT-NLL achieves state-of-the-art results on all metrics of the Waymo motion prediction benchmark.

Markus Knoche, Daan de Geus, Bastian Leibe

RWTH Aachen University, Germany · Eindhoven University of Technology, Netherlands

Autonomous Driving Benchmarks Multi-Agent Trajectory Prediction Probabilistic Forecasting Trajectory Prediction Probabilistic Trajectory Prediction

Ask PDF

Abstract

Trajectory Prediction

Explore similar work

Recover to Predict: Progressive Retrospective Learning for Variable-Length Trajectory Prediction

MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction

Towards Metric-Agnostic Trajectory Forecasting