Rectify, Don't Regret: On-Policy Closed-Loop Training for Multimodal Trajectory Prediction

Harsh Yadav; Christian Bohn; Tobias Meisen

cs.ROMar 24, 2026

Rectify, Don't Regret: On-Policy Closed-Loop Training for Multimodal Trajectory Prediction

Authors: Harsh Yadav, Christian Bohn, Tobias Meisen

Organizations: University of Wuppertal

Abstract

Current trajectory prediction models are primarily trained in an open-loop manner, which often leads to covariate shift and compounding errors when deployed in real-world, closed-loop settings. In this work, we propose a closed-loop training paradigm for receding-horizon ego prediction and identify two design choices that are critical for it to succeed. First, closed-loop supervision must be on-policy: each predicted mode is rolled out and supervised only from the states its own execution produced, preserving the prediction accuracy that cross-mode supervision would otherwise degrade. Second, gradients must not flow across the executed state transition, as backpropagating through these transitions allows the model to retroactively adjust an earlier prediction so that the resulting state is easier to predict from, mathematically regretting a mistake rather than learning to recover from it. We show empirically that this shortcut couples the learned policy to the specific replanning rate used during training and degrades at other rates, whereas detaching the gradient forces the model to rectify drift instead, allowing it to generalize across replanning rates unseen during training. Evaluated across three benchmarks (nuScenes, DeepScenario, and Argoverse 2) and two architectures, a decoder-only adaptation of LMFormer and an unmodified implementation of QCNet, our approach matches open-loop baselines under single-shot evaluation and improves with higher replanning rates, reducing collisions by up to 35% and off-road rates by up to 67%.

Figures & tables

Fig. 1 : On-policy closed-loop rollout for a single predicted mode. At $t{=}0$ the model produces a full $T_{\text{pred}}$ -horizon prediction from the logged state (open-loop sample). The first $T_{\text{sim}}$ seconds are executed, advancing the ego to a drifted state; the model then re-predicts from this self-induced state (closed-loop sample at $t{=}T_{\text{sim}}$ ). The process repeats at $t{=}2T_{\text{sim}}$ . Surrounding agents advance via log replay. Gradients are detached at each state transition (Sec. III-B ), so each planning cycle is supervised independently. Every resulting prediction is supervised against the static ground truth over its remaining horizon (Green).

Eval	Model	k=1		k=5		OffRoad% $\downarrow$
$T_{\text{sim}}$	Model	Col% $\downarrow$	minFDE $\downarrow$	minADE $\downarrow$	MR $\downarrow$	OffRoad% $\downarrow$
6.0(s)	OL	$2.97_{\pm.08}$	$\mathbf{7.17}_{\pm.12}$	$\mathbf{1.14}_{\pm.01}$	$\mathbf{0.50}_{\pm.01}$	$1.02_{\pm.05}$
6.0(s)	CL	$\mathbf{2.96}_{\pm.10}$	$\mathbf{7.17}_{\pm.14}$	$1.15_{\pm.02}$	$0.51_{\pm.01}$	$\mathbf{0.95}_{\pm.04}$
2.0(s)	OL	$2.49_{\pm.14}$	$7.18_{\pm.14}$	$1.13_{\pm.01}$	$0.50_{\pm.01}$	$1.13_{\pm.12}$
2.0(s)	CL	$\mathbf{2.38}_{\pm.10}$	$\mathbf{6.96}_{\pm.13}$	$\mathbf{1.11}_{\pm.02}$	$\mathbf{0.47}_{\pm.01}$	$\mathbf{0.86}_{\pm.07}$
0.5(s)	OL	$3.95_{\pm.19}$	$8.37_{\pm.21}$	$1.56_{\pm.03}$	$0.66_{\pm.02}$	$5.65_{\pm.69}$

TABLE I : OL vs. CL training with LMFormer-D on nuScenes . The CL advantage is negligible under single-shot evaluation and grows monotonically with the replanning rate.

Eval	Model	k=1		k=5		OffRoad% $\downarrow$
$T_{\text{sim}}$	Model	Col% $\downarrow$	minFDE $\downarrow$	minADE $\downarrow$	MR $\downarrow$	OffRoad% $\downarrow$
6.0(s)	OL	$2.23_{\pm.30}$	$7.50_{\pm.34}$	$\mathbf{1.10}_{\pm.05}$	$\mathbf{0.45}_{\pm.04}$	$7.26_{\pm.88}$
6.0(s)	CL	$\mathbf{1.90}_{\pm.28}$	$\mathbf{6.84}_{\pm.23}$	$1.19_{\pm.07}$	$0.49_{\pm.02}$	$\mathbf{6.03}_{\pm.90}$
2.0(s)	OL	$1.75_{\pm.05}$	$7.63_{\pm.28}$	$\mathbf{1.24}_{\pm.06}$	$0.48_{\pm.02}$	$5.64_{\pm.58}$
2.0(s)	CL	$\mathbf{1.33}_{\pm.24}$	$\mathbf{6.99}_{\pm.31}$	$\mathbf{1.24}_{\pm.14}$	$\mathbf{0.44}_{\pm.05}$	$\mathbf{4.72}_{\pm 1.45}$
0.5(s)	OL	$4.46_{\pm.20}$	$12.61_{\pm.83}$	$3.22_{\pm.07}$	$0.90_{\pm.02}$	$22.28_{\pm 4.04}$

TABLE II : OL vs. CL training on DeepScenario . Denser, more interactive scenarios amplify the gains of CL at higher replanning rates.

Eval	Model	k=1		k=6		OffRoad% $\downarrow$
$T_{\text{sim}}$	Model	Col% $\downarrow$	minFDE $\downarrow$	minADE $\downarrow$	MR $\downarrow$	OffRoad% $\downarrow$
6.0(s)	OL	$\mathbf{1.64}_{\pm.09}$	$\mathbf{4.44}_{\pm.04}$	$0.76_{\pm.01}$	$\mathbf{0.18}_{\pm.00}$	$\mathbf{11.76}_{\pm.29}$
6.0(s)	CL	$1.69_{\pm.06}$	$4.75_{\pm.05}$	$\mathbf{0.75}_{\pm.00}$	$0.19_{\pm.00}$	$11.99_{\pm.12}$
2.0(s)	OL	$1.98_{\pm.04}$	$5.47_{\pm.25}$	$0.93_{\pm.03}$	$0.29_{\pm.03}$	$13.66_{\pm.50}$
2.0(s)	CL	$\mathbf{1.87}_{\pm.06}$	$\mathbf{5.40}_{\pm.13}$	$\mathbf{0.89}_{\pm.01}$	$\mathbf{0.28}_{\pm.00}$	$\mathbf{13.13}_{\pm.52}$
0.5(s)	OL	$2.84_{\pm.14}$	$8.35_{\pm.47}$	$1.30_{\pm.01}$	$0.52_{\pm.03}$	$16.67_{\pm.67}$

TABLE III : OL vs. CL training with QCNet on Argoverse 2 . CL improves mean of reported metrics at

T_{\text{sim}}=

0.5\text{,}\mathrm{s}$$

Eval	Policy	k=1		k=5		OffRoad% $\downarrow$
$T_{\text{sim}}$	Policy	Col% $\downarrow$	minFDE $\downarrow$	minADE $\downarrow$	MR $\downarrow$	OffRoad% $\downarrow$
6.0(s)	Off-	$\mathbf{2.83}_{\pm.20}$	$\mathbf{7.02}_{\pm.12}$	$\mathbf{1.15}_{\pm.00}$	$\mathbf{0.50}_{\pm.01}$	$1.06_{\pm.04}$
6.0(s)	On-	$2.96_{\pm.10}$	$7.17_{\pm.14}$	$\mathbf{1.15}_{\pm.02}$	$0.51_{\pm.01}$	$\mathbf{0.95}_{\pm.04}$
2.0(s)	Off-	$\mathbf{1.92}_{\pm.04}$	$7.60_{\pm.01}$	$1.28_{\pm.02}$	$0.58_{\pm.02}$	$1.42_{\pm.02}$
2.0(s)	On-	$2.38_{\pm.10}$	$\mathbf{6.96}_{\pm.13}$	$\mathbf{1.11}_{\pm.02}$	$\mathbf{0.47}_{\pm.01}$	$\mathbf{0.86}_{\pm.07}$
0.5(s)	Off-	$3.50_{\pm.05}$	$12.51_{\pm.83}$	$2.35_{\pm.11}$	$0.77_{\pm.02}$	$2.47_{\pm 1.07}$

TABLE IV : Off- vs. on-policy closed-loop rollout during training (nuScenes, three seeds). On-policy rollout dominates every displacement and compliance metric at

T_{\text{sim}}\leq

2.0\text{,}\mathrm{s}$$ .

Panel of Figure 8 — Fig. 4 : Differentiable training produces a frequency-specific policy. Collision rate over a $6\text{\,}\mathrm{s}$ rollout versus evaluation replanning period for CL models trained at three different replanning periods. The Diff model’s minimum collision rate (red) tracks its training $T_{\text{sim}}$ in every panel. In contrast, Non-Diff models (blue) generalize better to higher replanning rates, with the $T_{\text{sim}}=$ 2.0\text{,}\mathrm{s}$$ variant achieving the lowest overall collision rates.

Eval	Sim.	k=1		k=5		OffRoad% $\downarrow$
$T_{\text{sim}}$	Sim.	Col% $\downarrow$	minFDE $\downarrow$	minADE $\downarrow$	MR $\downarrow$	OffRoad% $\downarrow$
6.0(s)	Scene	$\mathbf{2.89}_{\pm.11}$	$7.22_{\pm.21}$	$\mathbf{1.15}_{\pm.01}$	$\mathbf{0.51}_{\pm.01}$	$\mathbf{0.94}_{\pm.07}$
6.0(s)	Log	$2.96_{\pm.10}$	$\mathbf{7.17}_{\pm.14}$	$\mathbf{1.15}_{\pm.02}$	$\mathbf{0.51}_{\pm.01}$	$0.95_{\pm.04}$
2.0(s)	Scene	$\mathbf{2.24}_{\pm.10}$	$\mathbf{6.93}_{\pm.24}$	$\mathbf{1.11}_{\pm.01}$	$\mathbf{0.47}_{\pm.01}$	$\mathbf{0.78}_{\pm.08}$
2.0(s)	Log	$2.38_{\pm.10}$	$6.96_{\pm.13}$	$\mathbf{1.11}_{\pm.02}$	$\mathbf{0.47}_{\pm.01}$	$0.86_{\pm.07}$
0.5(s)	Scene	$3.35_{\pm.44}$	$\mathbf{7.85}_{\pm.12}$	$1.43_{\pm.05}$	$0.59_{\pm.02}$	$2.14_{\pm.37}$

TABLE V : Log-replay vs. goal-conditioned scene prediction during training on nuScenes. Differences are within or close to the seed spread.

Models	Prediction Metrics							Model Efficiency
	k=5			k=1			OffRoad	Parameters	FLOPS
	minFDE $\downarrow$	minADE $\downarrow$	MR $\downarrow$	minFDE $\downarrow$	minADE $\downarrow$	MR $\downarrow$	Rate $\downarrow$	(mil)	(GFLOPS)
LMFormer 1	2.145	1.155	0.500	7.114	3.136	0.832	0.010	2.234	3.365
LMFormer-D 1	2.141	1.141	0.492	7.069	3.152	0.829	0.011	1.207 (-45%)	1.249 (-62%)

TABLE VI : Effect of the decoder-only design on nuScenes. Prediction quality is preserved while parameters and FLOPs are roughly halved and thirded.

Panel of Figure 11 — Fig. 5 : Qualitative comparison of OL vs. CL rollouts with different replanning periods . While the OL model accumulates lateral drift and departs the lane, the CL model recovers lane alignment (bottom). Additionally, the CL ego decelerates for a leading vehicle during a right turn (top).

Explore similar work

Goal-Oriented Reactive Simulation for Closed-Loop Trajectory Prediction
Mar 25, 2026Harsh Yadav, Tobias MeisenTrajectory Prediction Autonomous Driving
University of Wuppertal, 42119, Germany
Guiding End-to-End Driving Models with Endpoint-Constrained Trajectory Optimization
Sep 25, 2026Brayden Zhang, Mahsa Golchoubian, Igor Gilitschenski +2End-to-End Autonomous Driving Trajectory Optimization
University of Toronto · Vector Institute · NVIDIA Research +2
REFINE: Trajectory Representation Learning via Closed-Loop Transcription -- Extended Version
Sep 7, 2026Sean Bin Yang, Ying Sun, Jilin Hu +5Contrastive Learning Self-Supervised Time Series Representation Learning
Aalborg University Aalborg, Denmark · Chongqing Universtiy of Posts and Telecommunications Chongqing, China · East China Normal University Shanghai, China +1

Mar 25, 2026J/K move · Enter open · S save

Goal-Oriented Reactive Simulation for Closed-Loop Trajectory Prediction

Harsh Yadav, Tobias Meisen

University of Wuppertal, 42119, Germany

Trajectory Prediction Autonomous Driving Autonomous Driving Simulation

Open paper AskPDF arXiv

Current trajectory prediction models are primarily trained in an open-loop manner, which often leads to covariate shift and compounding errors when deployed in real-world, closed-loop settings. Furthermore, relying on static datasets or non-reactive log-replay simulators severs the interactive loop, preventing the ego agent from learning to actively negotiate surrounding traffic. In this work, we propose an on-policy closed-loop training paradigm optimized for high-frequency, receding horizon ego prediction. To ground the ego prediction in a realistic representation of traffic interactions and to achieve reactive consistency, we introduce a goal-oriented, transformer-based scene decoder, resulting in an inherently reactive training simulation. By exposing the ego agent to a mixture of open-loop data and simulated, self-induced states, the model learns recovery behaviors to correct its own execution errors. Extensive evaluation demonstrates that closed-loop training significantly enhances collision avoidance capabilities at high replanning frequencies, yielding relative collision rate reductions of up to 27.0% on nuScenes and 79.5% in dense DeepScenario intersections compared to open-loop baselines. Additionally, we show that a hybrid simulation combining reactive with non-reactive surrounding agents achieves optimal balance between immediate interactivity and long-term behavioral stability.

Abstract

Explore similar work

Goal-Oriented Reactive Simulation for Closed-Loop Trajectory Prediction

Guiding End-to-End Driving Models with Endpoint-Constrained Trajectory Optimization

REFINE: Trajectory Representation Learning via Closed-Loop Transcription -- Extended Version