Forecasting the Past: Gradient-Based Distribution Shift Detection in Trajectory Prediction

Michele De Vita; Julian Wiederer; Vasileios Belagiannis

cs.LGApr 14, 2026

Forecasting the Past: Gradient-Based Distribution Shift Detection in Trajectory Prediction

Authors: Michele De Vita, Julian Wiederer, Vasileios Belagiannis

Organizations: Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany · Mercedes-Benz AG, Germany

Abstract

Trajectory prediction models often fail in real-world automated driving due to distributional shifts between training and test conditions. Such distributional shifts, whether behavioural or environmental, pose a critical risk by causing the model to make incorrect forecasts in unfamiliar situations. We propose a self-supervised method that trains a decoder in a post-hoc fashion on the self-supervised task of forecasting the second half of observed trajectories from the first half. The L2 norm of the gradient of this forecasting loss with respect to the decoder's final layer defines a score to identify distribution shifts. Our approach, first, does not affect the trajectory prediction model, ensuring no interference with original prediction performance and second, demonstrates substantial improvements on distribution shift detection for trajectory prediction on the Shifts and Argoverse datasets. Moreover, we show that this method can also be used to early detect collisions of a deep Q-Network motion planner in the Highway simulator. Source code is available at https://github.com/Michedev/forecasting-the-past.

Figures & tables

Panel of Figure 1 — Figure 1: Instead of (a) reconstructing the entire past trajectory $X_{past}$ and using the reconstruction loss $\mathcal{L}_{reconstruct}$ as a distribution shift score, we (b) forecast a portion of the observed trajectory $X_{past}$ and use the gradient of the forecasting loss, $\nabla\mathcal{L}_{forecast}$ , as the distribution shift score $S$ . Specifically, we divide $X_{past}$ into $X_{past,1}$ and $X_{past,2}$ and forecast $\hat{X}_{past,2}$ given $X_{past,1}$ . The gradient $\nabla\mathcal{L}_{forecast}$ effectively identifies unknown scenarios in trajectory prediction.

Figure 2: Overview of our post-hoc gradient-based distribution shift detection method. The upper part shows the pre-trained encoder-decoder for trajectory prediction. Given its encoder

e_{\gamma}

, we propose our forecast-the-past decoder

d_{past}

for distribution shift detection. The historical trajectory

X_{past}

is split into two segments,

X_{past,1}

and

X_{past,2}

, which are resized via linear interpolation. The frozen encoder processes

X_{past,1}

to produce the latent representation

z_{past,1}

, while our forecast-the-past decoder

d_{past}

predicts

X_{past,2}

. The gradient of the forecasting loss

\mathcal{L}_{past}

with respect to the input of the decoder’s last layer

h_{last}

serves as the distribution shift score

S=\nabla\mathcal{L}_{past}

Figure 3: Qualitative example of our gradient-based distribution shift detection method. The figure shows trajectory samples from in-distribution and distribution shifted scenarios. For each sample, we display the historical trajectory split into two segments ( $X_{past,1}$ with the blue line and $X_{past,2}$ with the green line), the forecasted second segment using our self-supervised decoder $\hat{X}_{past,2}$ with a dashed line, and the corresponding gradient-based anomaly score $S$ . Higher gradient magnitudes indicate distribution shift samples that deviate from learned motion dynamics.

Panel of Figure 4 — Figure 4: Highway simulator scenarios depicting a merge crash, a roundabout crash, and normal roundabout navigation. We mark the start and end points of trajectories, highlight the colliding vehicle in a different color, and mark the crash point. A horizontal red line in the OOD score plots indicates the crash timestep, and a thin line denotes the anomaly threshold. Collisions in Figures 4(a) and 4(b) produce high gradient values, while normal behavior in Figure 4(c) shows gradients below the threshold. In Figure 4(a) , the method predicts stopping (black line), and the score increases immediately after step 20, successfully detecting the collision. In Figure 4(b) , the gradient score reaches a peak at timestep 26, preceding a collision after 4 steps. Figure 4(c) shows normal driving behavior in a roundabout, with a stable gradient score over time.

Panel of Figure 5 — Figure 5: Distribution of rotation determinant and max velocity in Argoverse V1. Further details about their calculation are in Paragraphs 4.2 and 4.2 . We observe that trajectories are skewed towards turning right in Argoverse [ 6 ] .

Model	AUROC (%)
RIP-BC (K=1) [ 33 , 10 ]	52.8%
RIP-BC (K=5) [ 33 , 10 ]	52.1%
RIP-DIM (K=1) [ 33 , 29 ]	51.8%
RIP-DIM (K=5) [ 33 , 29 ]	50.9%
lGMM [ 38 ]	56.8%
Our Method	71.0 %

Table 1: Performance of Distribution shifts detection on Shifts [ 25 ]

	KDE	OC-SVM	IF	Our Method
Turn right	56.1%	56.4%	61.0%	71.3 %
Turn left	55.2%	52.8%	53.2%	70.3 %
Max Velocity	70.8%	61.6%	65.4%	79.2 %

Table 2: OOD detection performance in terms of AUROC on the Argoverse Dataset.

Panel of Figure 6 — Figure 6: Gradient Distribution for In-Distribution vs. Out-Of-Distribution samples for Shifts [ 25 ] and Argoverse [ 6 ] . We observe different distribution for ID (Blue area) and OOD (Red Area) in Shifts Dataset [ 25 ] .

Figure 7: Kernel density estimation (KDE) [ 8 ] of the last-layer gradients in the Highway environment [ 21 ] for the intersection driving task. We observed very distinct gradients between ID on OOD samples, leading to almost perfect collision detection.

Model	last	lat.	all	loss
AE	50.56%	56.26%	57.29%	50.00%
MAE	50.51%	56.08%	57.46%	48.95%
Our method	71.30%	60.20%	58.70%	51.00%

Table 3: Ablation study of model architectures and scoring strategies for distribution shift detection on the Shifts dataset [ 25 ] using the AUROC metric. We compare standard and masked autoencoders against our method using different scores: gradients w.r.t. the last layer ( last ), the latent space ( latent ), all model parameters ( all ), as well as the self-supervised loss function of the models ( loss ).

Method	Merge	Intersection	Roundabout
AE	99.9%	64.3%	85.1%
Masked AE	99.9%	65.5%	85.8%
IF	99.1%	67.9%	91.1%
Our method	99.9%	96.1%	95.8%

Table 4: Distribution Shifts detection performance (AUROC) across different scenarios in the Highway [ 21 ] Environment.

Figure 8: t-SNE visualization of the decoder’s last layer inputs for the Highway [ 21 ] Roundabout scenario. Safe (ID) and collision (OOD) trajectories form well-separated clusters, demonstrating the discriminative power of the learned representations.

Figure 9: Gradient vs loss-based OOD detection on the Shifts dataset [ 25 ] . While the loss function

\mathcal{L}_{past}

reaches 51.0 % AUROC, the gradient of the loss function

\nabla\mathcal{L}_{past}

reaches 70.9 % AUROC showcasing the effectiveness of gradient-based OOD detection (Sec. 3.2 ).

Explore similar work

ControlledShifts: Towards Standardizing Robustness Evaluation in Trajectory Prediction Under Distribution Shifts
Aug 18, 2026Ingrid Navarro, Pablo Ortega-Kral, Yutong Duan +2Distribution Shift Robustness Trajectory Prediction
Robotics Institute, School of Computer Science, Carnegie Mellon University · Work done as part of an internship at Lavoro AI. · Stack AV +1
Towards Metric-Agnostic Trajectory Forecasting
Jul 1, 2026Markus Knoche, Daan de Geus, Bastian LeibeAutonomous Driving Benchmarks Multi-Agent Trajectory Prediction
RWTH Aachen University, Germany · Eindhoven University of Technology, Netherlands
MetaDAT: Generalizable Trajectory Prediction via Meta Pre-training and Data-Adaptive Test-Time Updating
Mar 10, 2026Yuning Wang, Pu Zhang, Yuan He +2Trajectory Prediction Test-Time Adaptation
State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, IAIR, Xi’an Jiaotong University, China · KargoBot, China

Aug 18, 2026J/K move · Enter open · S save

ControlledShifts: Towards Standardizing Robustness Evaluation in Trajectory Prediction Under Distribution Shifts

Ingrid Navarro, Pablo Ortega-Kral, Yutong Duan, Jonathan Francis+1

Robotics Institute, School of Computer Science, Carnegie Mellon University · Work done as part of an internship at Lavoro AI. · Stack AV+1

Distribution Shift Robustness Trajectory Prediction Autonomous Driving Benchmarks

Open paper AskPDF arXiv

Trajectory prediction is central to safety in autonomous driving, yet learning-based predictors tend to degrade sharply when encountering scenarios poorly represented by their training data. Many methods attempt to mitigate distribution shift degradation through data-centric or test-time adaptation approaches; however, they are typically validated along fragmented axes of generalization, leaving the field without a standardized way to compare robustness across shifts a model may encounter. To address this, we introduce ControlledShifts, a framework and benchmark suite that systematically re-splits existing trajectory datasets into in-distribution (seen) and out-of-distribution (unseen) partitions, via a shared characterization-and-splitting formulation, in which a characterization function fixes the axis of variation a benchmark probes and a splitting function fixes how the tail of that axis is withheld. The suite comprises three benchmarks targeting key topological and behavioral distribution shifts. Furthermore, to aggregate multi-dimensional performance metrics across these benchmarks, we propose a unified robustness score that evaluates models along two complementary dimensions: prediction quality (relative performance gain) and prediction stability (performance preservation under shift). We showcase ControlledShifts by benchmarking prominent transformer-based architectures, exposing critical differences in how models of varying capacities handle latent relevance and environmental structure.

Abstract

Explore similar work

ControlledShifts: Towards Standardizing Robustness Evaluation in Trajectory Prediction Under Distribution Shifts

Towards Metric-Agnostic Trajectory Forecasting

MetaDAT: Generalizable Trajectory Prediction via Meta Pre-training and Data-Adaptive Test-Time Updating