SAIL: Scene-aware Adaptive Iterative Learning for Long-Tail Trajectory Prediction in Autonomous Vehicles

Bin Rao; Haicheng Liao; Chengyue Wang; Keqiang Li; Zhenning Li; Hai Yang

cs.ETApr 6, 2026

SAIL: Scene-aware Adaptive Iterative Learning for Long-Tail Trajectory Prediction in Autonomous Vehicles

Authors: Bin Rao, Haicheng Liao, Chengyue Wang, Keqiang Li, Zhenning Li, Hai Yang

Organizations: State Key Laboratory of Internet of Things for Smart City and Department of Civil and Environmental Engineering, University of Macau, Macau SAR, China · State Key Laboratory of Internet of Things for Smart City and Department of Computer and Information Science, University of Macau, Macau SAR, China · Department of Automotive Engineering, Tsinghua University, Beijing, China · State Key Laboratory of Internet of Things for Smart City and Departments of Civil and Environmental Engineering and Computer and Information Science, University of Macau, Macau SAR, China ·

Abstract

Autonomous vehicles (AVs) rely on accurate trajectory prediction for safe navigation in diverse traffic environments, yet existing models struggle with long-tail scenarios-rare but safety-critical events characterized by abrupt maneuvers, high collision risks, and complex interactions. These challenges stem from data imbalance, inadequate definitions of long-tail trajectories, and suboptimal learning strategies that prioritize common behaviors over infrequent ones. To address this, we propose SAIL, a novel framework that systematically tackles the long-tail problem by first defining and modeling trajectories across three key attribute dimensions: prediction error, collision risk, and state complexity. Our approach then synergizes an attribute-guided augmentation and feature extraction process with a highly adaptive contrastive learning strategy. This strategy employs a continuous cosine momentum schedule, similarity-weighted hard-negative mining, and a dynamic pseudo-labeling mechanism based on evolving feature clustering. Furthermore, it incorporates a focusing mechanism to intensify learning on hard-positive samples within each identified class. This comprehensive design enables SAIL to excel at identifying and forecasting diverse and challenging long-tail events. Extensive evaluations on the nuScenes and ETH/UCY datasets demonstrate SAIL's superior performance, achieving up to 28.8% reduction in prediction error on the hardest 1% of long-tail samples compared to state-of-the-art baselines, while maintaining competitive accuracy across all scenarios. This framework advances reliable AV trajectory prediction in real-world, mixed-autonomy settings.

Figures & tables

Figure 1: Analyzing vehicle trajectories from the perspectives of Prediction Error, Risk (inverse time-to-collision, 1/TTC), and Vehicle State reveals their intrinsic long-tail nature. The top 5% of data in each distribution corresponds to distinct real-world scenarios, which are often critical to ensuring the safe operation of autonomous vehicles.

Figure 2: The overall architecture of our proposed SAIL framework. The framework takes historical trajectories and HD map data as input and processes them through a multi-stage pipeline, including the Scene Representation Learning module and the Attribute-aware Trajectory Generator, to output multiple future trajectories. Panels (b), (c), and (d) provide detailed views of our key components: the Multi-dimensional Long-Tail Attributes definition, the Attribute Disentanglement and Prediction module, and the Attribute-aware Trajectory Generator, respectively.

Symbol	Description	Symbol	Description
$A$	Total number of agents	$a$	Index of the target agent
$t_{o}$	Observation horizon	$t_{p}$	Prediction horizon
$H_{a}=\{s_{a}^{t}\}$	Full observed historical state sequence	$s_{a}^{t}$	Kinematic state of agent $a$ at time $t$
$M=\{m_{n}\}$	Set of map vectors	$N_{m}$	Number of map elements
$T=\{{c}_{t}\mid t\in(t_{o},t_{o}+t_{p}]\}$	Ground-truth future trajectory	$c_{t}=(x_{t},y_{t})$	Ground-truth coordinate at time $t$
$\hat{T}=\{\hat{T}^{(k)}\}_{k=1}^{K}$	Set of predicted future trajectories	$K$	Number of predicted modes

Table 1: Summary of main notations used throughout this paper.

Figure 3: Visualization of our Attribute-Guided Trajectory Augmentation strategies. Based on the identified long-tail attributes of a trajectory, AGTA applies a combination of targeted augmentations (Simplify, Shift, Mask, Subset) to create a diverse set of challenging positive samples for the subsequent contrastive learning stage.

Model	Top 1%	Top 2%	Top 3%	Top 4%	Top 5%	Rest	All
nuScenes Dataset
Traj++ EWTA [ 41 ]	1.73/4.43	1.36/3.54	1.17/3.03	1.04/2.68	0.95/2.41	0.16/0.26	0.22/0.39
Traj++ EWTA+contrastive [ 41 ]	1.28/2.85	0.97/2.15	0.83/1.83	0.76/1.64	0.70/1.48	0.15 /0.24	0.18/0.30
FEND [ 62 ]	1.21/2.50	0.92/1.88	0.79/1.61	0.72/1.43	0.66/1.31	0.14 / 0.20	0.17 / 0.26
TrACT [ 71 ]	1.23/2.65	0.98/2.11	0.85/1.82	0.78/1.64	0.72/1.49	-	0.19/0.31
CSD [ 13 ]	1.06 / 2.22	0.82 / 1.69	0.70 / 1.44	0.64 / 1.30	0.59 / 1.18	-	0.18/0.30

Table 2: Prediction results (minADE/minFDE) on nuScenes (2 s prediction) and ETH/UCY (4.8 s prediction) datasets. Samples are stratified based on their prediction error (FDE) to evaluate robustness, with the Top 1%-5% representing the highest error instances. Bold and underlined text indicate the best and second-best results, respectively. Cases marked with (’-’) indicate missing values. Improvement row shows the percentage improvement of our model over the second-best result.

Horizon (s)	Model	Top 1%	Top 2%	Top 3%	Top 4%	Top 5%	Rest	All
1	Q-EANet	0.18 /0.22	0.22/0.25	0.17 /0.21	0.15 /0.19	0.13 /0.17	0.08 / 0.10	0.13 /0.12
1	SAIL	0.19/ 0.21	0.20 / 0.23	0.18/ 0.20	0.16/ 0.18	0.14/ 0.16	0.13/0.11	0.14/ 0.12
2	Q-EANet	0.35/0.45	0.45/0.52	0.37/0.48	0.35/0.43	0.30/0.39	0.18/0.20	0.22/0.23
2	SAIL	0.32 / 0.40	0.42 / 0.48	0.36 / 0.45	0.33 / 0.40	0.28 / 0.37	0.16 / 0.19	0.18 / 0.23
3	Q-EANet	0.55/0.75	0.67/0.82	0.58/0.78	0.56/0.71	0.49/0.65	0.29 /0.33	0.34/0.38
3	SAIL	0.50 / 0.70	0.63 / 0.78	0.57 / 0.73	0.53 / 0.68	0.47 / 0.63	0.30/ 0.33	0.32 / 0.34

Table 3: Prediction results (minADE/minFDE) at different prediction horizons on the nuScenes dataset, based on Collision Risk. We compare our model with the baseline Q-EANet [ 6 ] . The Top 1%-5% intervals denote the most critical risk categories. Bold marks the best metric, while light gray background highlights cases where both metrics are simultaneously the best.

Figure 4: Heatmaps illustrating the performance improvements of the SAIL model relative to Q-EANet on the nuScenes dataset, categorized by collision risk levels and prediction horizons. Negative values indicate superior performance by SAIL. (a) Differences in minADE. (b) Differences in minFDE.

Horizon (s)	Model	Abrupt Acceleration	Abrupt Deceleration	Abrupt Lane Changes	High-Curvature Turns	Normal	All
1	Q-EANet	0.19 /0.17	0.22 /0.19	0.28/0.26	0.22/0.21	0.11 / 0.09	0.15/0.13
1	SAIL	0.20/ 0.16	0.23/ 0.18	0.26 / 0.24	0.21 / 0.19	0.12/0.10	0.14 / 0.12
2	Q-EANet	0.30 /0.29	0.36/0.34	0.42/0.44	0.34/0.37	0.18 / 0.18	0.23/0.23
2	SAIL	0.31/ 0.27	0.34 / 0.31	0.39 / 0.40	0.32 / 0.34	0.19/0.19	0.21 / 0.21
3	Q-EANet	0.42/0.45	0.49/0.50	0.56/0.62	0.47/0.56	0.29/0.32	0.35/0.38
3	SAIL	0.39 / 0.40	0.46 / 0.46	0.52 / 0.56	0.44 / 0.51	0.27 / 0.29	0.32 / 0.34

Table 4: Prediction results (minADE/minFDE) at different prediction horizons on the nuScenes dataset, based on vehicle state. We compare our model with the baseline Q-EANet [ 6 ] . Bold indicates the best result for each metric. Light gray background highlights cases where both metrics are simultaneously the best.

Model	Venue	Top 1%	Top 2%	Top 3%	Top 4%	Top 5%	All
PGP [ 10 ]	CoRL 2021	8.86/21.92	7.21/17.90	6.24/15.68	5.52/13.77	5.02/12.44	1.28/2.52
Q-EANet [ 6 ]	ITS 2024	7.55 / 18.78	6.15 / 15.58	5.44 / 13.76	4.94 / 12.49	4.55 / 11.49	1.20/2.45
LAformer [ 40 ]	CVPR 2024	8.19/19.03	6.73/15.81	5.89/13.90	5.33/12.60	4.90/11.61	1.19/ 2.42
UniTraj (MTR) [ 12 ]	ECCV 2024	7.84/21.69	6.44/18.06	5.69/15.95	5.18/14.49	4.78/13.37	1.15 /2.61
SAIL	-	6.84 / 17.21	5.80 / 14.57	5.05 / 12.88	4.58 / 11.68	4.22 / 10.72	1.18 / 2.42

Table 5: Model-agnostic worst-case performance comparison on the nuScenes dataset, reported in

\text{minADE}_{5}

\text{minFDE}_{5}

. The Top 1% to 5% subsets are dynamically defined for each model based on its own worst-performing samples, ranking by

\text{minFDE}_{5}

. The best results are in bold, and the second best are underlined.

Model	Venue	minADE 5	minADE 10	minFDE 1	minFDE 5	minFDE 10	MR 5
Trajectron++ [ 52 ]	ECCV 2020	1.88	1.51	9.52	5.63	-	0.70
LaPred [ 24 ]	CVPR 2021	1.47	1.12	8.12	3.37	2.39	0.53
MHA-JAM [ 44 ]	IV 2021	-	1.18	9.62	3.72	2.21	0.64
GoHome [ 16 ]	ICRA 2022	1.42	1.15	6.99	-	-	0.57
ContextVAE [ 64 ]	RAL 2023	1.59	-	8.24	3.28	-	-
AFormer-FLN [ 65 ]	CVPR 2024	1.83	1.32	-	3.78	2.86	-

Table 6: Quantitative evaluation of trajectory prediction performance on the full nuScenes dataset, with a prediction horizon of 6 s. Light gray background indicates the performance of our model.

Model	Venue	ETH	HOTEL	UNIV	ZARA1	ZARA2	AVG
PECNet [ 42 ]	ECCV 2020	0.54/0.87	0.18/0.24	0.22/0.39	0.17/0.30	0.35/0.60	0.29/0.48
AgentFormer [ 70 ]	ICCV 2021	0.45/0.75	0.14/0.22	0.25/0.45	0.18/0.30	0.14/0.24	0.23/0.39
Trajectron++ [ 52 ]	ECCV 2020	0.39/0.83	0.12/0.21	0.20/0.44	0.15 /0.33	0.11 /0.25	0.19/0.41
NPSN [ 2 ]	CVPR 2022	0.36/0.59	0.16/0.25	0.23/0.39	0.18/0.32	0.14/0.25	0.21/0.36
MID [ 18 ]	CVPR 2022	0.39/0.66	0.13/0.22	0.22/0.45	0.17/0.30	0.13/0.27	0.21/0.38
TUTR [ 54 ]	ICCV 2023	0.40/0.61	0.11/0.18	0.23/0.42	0.18/0.34	0.13/0.25	0.21/0.36

Table 7: Quantitative evaluation of trajectory prediction performance on the ETH/UCY dataset across all samples. Light gray background indicates the performance of our model.

Model

Components

Performance (minADE/minFDE)

AGTA

ADP

AMCL

EFC

FDCL

Top 1%

Top 2%

Top 3%

Top 4%

Top 5%

Rest

All

\times

\checkmark

\checkmark

\checkmark

\checkmark

1.30/1.88

1.05/1.55

0.88/1.35

0.79/1.24

0.73/1.16

0.19/0.23

0.22/0.27

\checkmark

\times

\checkmark

\checkmark

\checkmark

1.48/2.10

1.15/1.70

0.95/1.48

0.85/1.35

0.79/1.26

0.21/0.24

0.24/0.28

\checkmark

\checkmark

\times

\checkmark

\checkmark

1.25/1.83

1.01/1.51

0.84/1.31

0.76/1.21

0.70/1.13

0.18/0.22

0.21/0.26

\checkmark

\checkmark

\checkmark

\times

\checkmark

1.15/1.72

0.92/1.41

0.75/1.22

0.68/1.13

0.63/1.06

0.18/0.21

0.20/0.25

\checkmark

\checkmark

\checkmark

\checkmark

\times

1.22/1.80

0.98/1.49

0.81/1.29

0.73/1.19

0.68/1.11

0.19/0.22

0.21/0.26

Table 8: Ablation analysis of key components within our method on the nuScenes benchmark. Each row from A to E represents a variant of our full model with one key component removed. AGTA: Attribute-Guided Trajectory Augmentation, ADP: Attribute Disentanglement and Prediction, AMCL: Adaptive Momentum Contrastive Learning, EFC: Evolving Feature Clustering, FDCL: Focused Decoupled Contrastive Learning. Bold text represents the best results.

Attribute Pair	Jaccard Similarity
Attribute Pair	Top 20%	Top 15%	Top 10%	Top 5%
Prediction Error vs. Collision Risk	10.7%	8.3%	5.2%	1.4%
Prediction Error vs. State Complexity	13.0%	10.3%	6.8%	4.8%
Collision Risk vs. State Complexity	17.0%	14.1%	11.4%	5.7%

Table 9: Jaccard similarity between attribute-specific long-tail subsets on the nuScenes validation set. The overlap between subsets identified by Prediction Error, Collision Risk, and State Complexity is measured under different tail thresholds, from the Top 20% to the Top 5% subsets.

Figure 5: UpSet visualization of intersections among attribute-specific long-tail subsets on the nuScenes validation set. Subfigures (a) to (d) correspond to the Top 20%, Top 15%, Top 10%, and Top 5% tail thresholds, respectively. In each subfigure, the bars indicate the number of samples in each subset intersection, while the connected dots denote the corresponding combination of Prediction Error, Collision Risk, and State Complexity. The percentages above the bars represent the proportion of each intersection among all samples selected under the given threshold.

Model	Inference Time (ms)	$\text{minADE}_{5}$	$\text{minFDE}_{1}$	$\text{MR}_{5}$
MultiPath [ 5 ]	87	1.44	7.69	0.74
AgentFormer [ 70 ]	107	1.97	9.12	0.69
LAformer [ 40 ]	115	1.19	6.89	0.54
VisionTrap [ 46 ]	53	1.36	8.72	0.61
SAIL	18	1.18	6.45	0.50

Table 10: Comparative analysis of computational efficiency and predictive accuracy on the nuScenes dataset. We compare SAIL against SOTA baselines to demonstrate the trade-off between speed and performance. Bold and underlined text represent the best and second-best results, respectively.

Figure 6: Visualization of comparison results for inference time and error metrics on the nuScenes dataset. Panel (a) shows inference time, panel (b) depicts $\text{minADE}_{5}$ , panel (c) illustrates $\text{minFDE}_{1}$ , and panel (d) presents $\text{MR}_{5}$ .

$C$	Top 1%	Top 2%	Top 3%	Top 4%	Top 5%	Rest	All
2	1.35/1.95	1.08/1.60	0.90/1.40	0.82/1.30	0.76/1.22	0.20/0.24	0.23/0.27
3	1.21/1.80	0.98/1.49	0.82/1.29	0.74/1.20	0.68/1.12	0.18/0.22	0.21/0.25
4	1.10/1.70	0.88/1.38	0.76/1.20	0.67/1.12	0.62/1.05	0.17/0.20	0.19/0.24
5	1.02 / 1.58	0.81 / 1.30	0.65 / 1.12	0.60 / 1.05	0.56 / 0.98	0.16 / 0.19	0.18 / 0.23
6	1.05/1.62	0.84/1.34	0.69/1.16	0.62/1.08	0.58/1.01	0.16/0.20	0.18/0.23
7	1.12/1.73	0.90/1.42	0.75/1.24	0.68/1.15	0.63/1.08	0.17/0.21	0.20/0.25

Table 11: Performance (minADE/minFDE) on nuScenes for varying numbers of clusters (

C

) in our EFC module.

Figure 7: Stability analysis of the Evolving Feature Clustering (EFC) process. The Adjusted Rand Index (ARI) between consecutive clustering updates is plotted against training epochs on the nuScenes dataset.

Figure 8: t-SNE visualization of samples drawn from the Top 5% subsets under the three-dimensional long-tail attributes of prediction error, collision risk, and state complexity. The three groups exhibit relatively distinct yet partially overlapping manifold structures, indicating that they capture different but correlated aspects of long-tail driving scenarios.

Figure 9: Qualitative results of long-tail trajectory prediction across diverse high-curvature driving scenarios on the nuScenes dataset. Panels (a) and (b) depict left-turn maneuvers, while panels (c) and (d) illustrate right-turn maneuvers. The others represent baseline model [ 6 ] predictions. Model B and Model C represent the variants of our model. The red line depicts the highest-probability trajectory, whereas the light red lines illustrate the multimodal predictions.

Figure 11: Representative failure cases under extreme long-tail scenarios. (a) Failure cases caused by severe visual occlusion. (b) Failure cases at open intersections without explicit traffic signal information. The red line depicts the highest-probability trajectory, whereas the light red lines illustrate the multimodal predictions.

Trajectory Prediction

Trajectory prediction forecasts how people, vehicles, robots, and other dynamic systems will move from observed histories and context.

See all 187 papers

Pivot-Centric Trajectory Prediction: Bridging Long Horizons via Dynamical Guidance
Aug 4, 2026Xiucong Zhao, Jindong Tian, Hao MiaoMulti-Agent Trajectory Prediction Autonomous Driving
1Xi’an Jiaotong University · 2East China Normal University · 3Hong Kong Polytechnic University
Recover to Predict: Progressive Retrospective Learning for Variable-Length Trajectory Prediction
Mar 11, 2026Hao Zhou, Lu Qi, Jason Li +5Trajectory Prediction Autonomous Driving
Great Bay University · Tsinghua SIGS · Wuhan University +3
From Cues to Horizons: Dynamic Risk Horizon Profiling for Trajectory Prediction
May 30, 2026Xinyi Ning, Zilin Bian, Dachuan Zuo +2Autonomous Driving Risk-Aware Path Planning
Department of Civil and Urban Engineering, New York University. · Department of Civil Engineering Technology and Environmental Management Safety, Rochester Institute of Technology.

Aug 4, 2026J/K move · Enter open · S save

Pivot-Centric Trajectory Prediction: Bridging Long Horizons via Dynamical Guidance

Xiucong Zhao, Jindong Tian, Hao Miao

1Xi’an Jiaotong University · 2East China Normal University · 3Hong Kong Polytechnic University

Multi-Agent Trajectory Prediction Autonomous Driving Agent Trajectory Analysis Trajectory Prediction Vehicle Trajectory Prediction

Open paper AskPDF arXiv

Forecasting precise future motion of surrounding agents is essential for reliable autonomous vehicles. However, as the demand for longer prediction horizons increases, existing endpoint-completion or iterative-refine methods increasingly struggle with weak guidance and compounding errors. To tackle the long-horizon prediction challenge, we propose Pivot-Centric Trajectory Prediction (PCTP). By introducing

pivots'' and focusing on predicting pivot points along extended trajectories, we divide the long-term prediction task into short-term sub-tasks at various scales. Specifically, PCTP decouples the long-term trajectory predicting process into two processes: pivot prediction and pivot-based trajectory refinement. The pivot prediction process aims to utilize global map context and agent-to-agent interactions to identify these

pivot points'', while the pivot-based trajectory refinement process focuses on local map details and refines the short-term trajectory based on predicted ``pivot points''. Compared with existing methods, PCTP provides more intermediate guidance while reducing compounding errors. Moreover, PCTP is a flexible approach that can be integrated into most state-of-the-art trajectory prediction models. Experimental results show that PCTP improves the prediction accuracy of leading models on both Argoverse I and Argoverse II datasets with minimal impact on model size. Specifically, PCTP combined with QCNet outperforms all published ensemble-free methods on the Argoverse II leaderboard at submission.

Abstract

Trajectory Prediction

Explore similar work

Pivot-Centric Trajectory Prediction: Bridging Long Horizons via Dynamical Guidance

Recover to Predict: Progressive Retrospective Learning for Variable-Length Trajectory Prediction

From Cues to Horizons: Dynamic Risk Horizon Profiling for Trajectory Prediction