Rectify, Don't Regret: On-Policy Closed-Loop Training for Multimodal Trajectory Prediction

Harsh Yadav; Christian Bohn; Tobias Meisen

cs.ROMar 24, 2026

Rectify, Don't Regret: On-Policy Closed-Loop Training for Multimodal Trajectory Prediction

Authors: Harsh Yadav, Christian Bohn, Tobias Meisen

Organizations: University of Wuppertal

Abstract

Current trajectory prediction models are primarily trained in an open-loop manner, which often leads to covariate shift and compounding errors when deployed in real-world, closed-loop settings. In this work, we propose a closed-loop training paradigm for receding-horizon ego prediction and identify two design choices that are critical for it to succeed. First, closed-loop supervision must be on-policy: each predicted mode is rolled out and supervised only from the states its own execution produced, preserving the prediction accuracy that cross-mode supervision would otherwise degrade. Second, gradients must not flow across the executed state transition, as backpropagating through these transitions allows the model to retroactively adjust an earlier prediction so that the resulting state is easier to predict from, mathematically regretting a mistake rather than learning to recover from it. We show empirically that this shortcut couples the learned policy to the specific replanning rate used during training and degrades at other rates, whereas detaching the gradient forces the model to rectify drift instead, allowing it to generalize across replanning rates unseen during training. Evaluated across three benchmarks (nuScenes, DeepScenario, and Argoverse 2) and two architectures, a decoder-only adaptation of LMFormer and an unmodified implementation of QCNet, our approach matches open-loop baselines under single-shot evaluation and improves with higher replanning rates, reducing collisions by up to 35% and off-road rates by up to 67%.

Figures & tables

Fig. 1 : On-policy closed-loop rollout for a single predicted mode. At $t{=}0$ the model produces a full $T_{\text{pred}}$ -horizon prediction from the logged state (open-loop sample). The first $T_{\text{sim}}$ seconds are executed, advancing the ego to a drifted state; the model then re-predicts from this self-induced state (closed-loop sample at $t{=}T_{\text{sim}}$ ). The process repeats at $t{=}2T_{\text{sim}}$ . Surrounding agents advance via log replay. Gradients are detached at each state transition (Sec. III-B ), so each planning cycle is supervised independently. Every resulting prediction is supervised against the static ground truth over its remaining horizon (Green).

Eval	Model	k=1		k=5		OffRoad% $\downarrow$
$T_{\text{sim}}$	Model	Col% $\downarrow$	minFDE $\downarrow$	minADE $\downarrow$	MR $\downarrow$	OffRoad% $\downarrow$
6.0(s)	OL	$2.97_{\pm.08}$	$\mathbf{7.17}_{\pm.12}$	$\mathbf{1.14}_{\pm.01}$	$\mathbf{0.50}_{\pm.01}$	$1.02_{\pm.05}$
6.0(s)	CL	$\mathbf{2.96}_{\pm.10}$	$\mathbf{7.17}_{\pm.14}$	$1.15_{\pm.02}$	$0.51_{\pm.01}$	$\mathbf{0.95}_{\pm.04}$
2.0(s)	OL	$2.49_{\pm.14}$	$7.18_{\pm.14}$	$1.13_{\pm.01}$	$0.50_{\pm.01}$	$1.13_{\pm.12}$
2.0(s)	CL	$\mathbf{2.38}_{\pm.10}$	$\mathbf{6.96}_{\pm.13}$	$\mathbf{1.11}_{\pm.02}$	$\mathbf{0.47}_{\pm.01}$	$\mathbf{0.86}_{\pm.07}$
0.5(s)	OL	$3.95_{\pm.19}$	$8.37_{\pm.21}$	$1.56_{\pm.03}$	$0.66_{\pm.02}$	$5.65_{\pm.69}$

TABLE I : OL vs. CL training with LMFormer-D on nuScenes . The CL advantage is negligible under single-shot evaluation and grows monotonically with the replanning rate.

Eval	Model	k=1		k=5		OffRoad% $\downarrow$
$T_{\text{sim}}$	Model	Col% $\downarrow$	minFDE $\downarrow$	minADE $\downarrow$	MR $\downarrow$	OffRoad% $\downarrow$
6.0(s)	OL	$2.23_{\pm.30}$	$7.50_{\pm.34}$	$\mathbf{1.10}_{\pm.05}$	$\mathbf{0.45}_{\pm.04}$	$7.26_{\pm.88}$
6.0(s)	CL	$\mathbf{1.90}_{\pm.28}$	$\mathbf{6.84}_{\pm.23}$	$1.19_{\pm.07}$	$0.49_{\pm.02}$	$\mathbf{6.03}_{\pm.90}$
2.0(s)	OL	$1.75_{\pm.05}$	$7.63_{\pm.28}$	$\mathbf{1.24}_{\pm.06}$	$0.48_{\pm.02}$	$5.64_{\pm.58}$
2.0(s)	CL	$\mathbf{1.33}_{\pm.24}$	$\mathbf{6.99}_{\pm.31}$	$\mathbf{1.24}_{\pm.14}$	$\mathbf{0.44}_{\pm.05}$	$\mathbf{4.72}_{\pm 1.45}$
0.5(s)	OL	$4.46_{\pm.20}$	$12.61_{\pm.83}$	$3.22_{\pm.07}$	$0.90_{\pm.02}$	$22.28_{\pm 4.04}$

TABLE II : OL vs. CL training on DeepScenario . Denser, more interactive scenarios amplify the gains of CL at higher replanning rates.

Eval	Model	k=1		k=6		OffRoad% $\downarrow$
$T_{\text{sim}}$	Model	Col% $\downarrow$	minFDE $\downarrow$	minADE $\downarrow$	MR $\downarrow$	OffRoad% $\downarrow$
6.0(s)	OL	$\mathbf{1.64}_{\pm.09}$	$\mathbf{4.44}_{\pm.04}$	$0.76_{\pm.01}$	$\mathbf{0.18}_{\pm.00}$	$\mathbf{11.76}_{\pm.29}$
6.0(s)	CL	$1.69_{\pm.06}$	$4.75_{\pm.05}$	$\mathbf{0.75}_{\pm.00}$	$0.19_{\pm.00}$	$11.99_{\pm.12}$
2.0(s)	OL	$1.98_{\pm.04}$	$5.47_{\pm.25}$	$0.93_{\pm.03}$	$0.29_{\pm.03}$	$13.66_{\pm.50}$
2.0(s)	CL	$\mathbf{1.87}_{\pm.06}$	$\mathbf{5.40}_{\pm.13}$	$\mathbf{0.89}_{\pm.01}$	$\mathbf{0.28}_{\pm.00}$	$\mathbf{13.13}_{\pm.52}$
0.5(s)	OL	$2.84_{\pm.14}$	$8.35_{\pm.47}$	$1.30_{\pm.01}$	$0.52_{\pm.03}$	$16.67_{\pm.67}$

TABLE III : OL vs. CL training with QCNet on Argoverse 2 . CL improves mean of reported metrics at

T_{\text{sim}}=

0.5\text{,}\mathrm{s}$$

Eval	Policy	k=1		k=5		OffRoad% $\downarrow$
$T_{\text{sim}}$	Policy	Col% $\downarrow$	minFDE $\downarrow$	minADE $\downarrow$	MR $\downarrow$	OffRoad% $\downarrow$
6.0(s)	Off-	$\mathbf{2.83}_{\pm.20}$	$\mathbf{7.02}_{\pm.12}$	$\mathbf{1.15}_{\pm.00}$	$\mathbf{0.50}_{\pm.01}$	$1.06_{\pm.04}$
6.0(s)	On-	$2.96_{\pm.10}$	$7.17_{\pm.14}$	$\mathbf{1.15}_{\pm.02}$	$0.51_{\pm.01}$	$\mathbf{0.95}_{\pm.04}$
2.0(s)	Off-	$\mathbf{1.92}_{\pm.04}$	$7.60_{\pm.01}$	$1.28_{\pm.02}$	$0.58_{\pm.02}$	$1.42_{\pm.02}$
2.0(s)	On-	$2.38_{\pm.10}$	$\mathbf{6.96}_{\pm.13}$	$\mathbf{1.11}_{\pm.02}$	$\mathbf{0.47}_{\pm.01}$	$\mathbf{0.86}_{\pm.07}$
0.5(s)	Off-	$3.50_{\pm.05}$	$12.51_{\pm.83}$	$2.35_{\pm.11}$	$0.77_{\pm.02}$	$2.47_{\pm 1.07}$

TABLE IV : Off- vs. on-policy closed-loop rollout during training (nuScenes, three seeds). On-policy rollout dominates every displacement and compliance metric at

T_{\text{sim}}\leq

2.0\text{,}\mathrm{s}$$ .

Panel of Figure 8 — Fig. 4 : Differentiable training produces a frequency-specific policy. Collision rate over a $6\text{\,}\mathrm{s}$ rollout versus evaluation replanning period for CL models trained at three different replanning periods. The Diff model’s minimum collision rate (red) tracks its training $T_{\text{sim}}$ in every panel. In contrast, Non-Diff models (blue) generalize better to higher replanning rates, with the $T_{\text{sim}}=$ 2.0\text{,}\mathrm{s}$$ variant achieving the lowest overall collision rates.

Eval	Sim.	k=1		k=5		OffRoad% $\downarrow$
$T_{\text{sim}}$	Sim.	Col% $\downarrow$	minFDE $\downarrow$	minADE $\downarrow$	MR $\downarrow$	OffRoad% $\downarrow$
6.0(s)	Scene	$\mathbf{2.89}_{\pm.11}$	$7.22_{\pm.21}$	$\mathbf{1.15}_{\pm.01}$	$\mathbf{0.51}_{\pm.01}$	$\mathbf{0.94}_{\pm.07}$
6.0(s)	Log	$2.96_{\pm.10}$	$\mathbf{7.17}_{\pm.14}$	$\mathbf{1.15}_{\pm.02}$	$\mathbf{0.51}_{\pm.01}$	$0.95_{\pm.04}$
2.0(s)	Scene	$\mathbf{2.24}_{\pm.10}$	$\mathbf{6.93}_{\pm.24}$	$\mathbf{1.11}_{\pm.01}$	$\mathbf{0.47}_{\pm.01}$	$\mathbf{0.78}_{\pm.08}$
2.0(s)	Log	$2.38_{\pm.10}$	$6.96_{\pm.13}$	$\mathbf{1.11}_{\pm.02}$	$\mathbf{0.47}_{\pm.01}$	$0.86_{\pm.07}$
0.5(s)	Scene	$3.35_{\pm.44}$	$\mathbf{7.85}_{\pm.12}$	$1.43_{\pm.05}$	$0.59_{\pm.02}$	$2.14_{\pm.37}$

TABLE V : Log-replay vs. goal-conditioned scene prediction during training on nuScenes. Differences are within or close to the seed spread.

Models	Prediction Metrics							Model Efficiency
	k=5			k=1			OffRoad	Parameters	FLOPS
	minFDE $\downarrow$	minADE $\downarrow$	MR $\downarrow$	minFDE $\downarrow$	minADE $\downarrow$	MR $\downarrow$	Rate $\downarrow$	(mil)	(GFLOPS)
LMFormer 1	2.145	1.155	0.500	7.114	3.136	0.832	0.010	2.234	3.365
LMFormer-D 1	2.141	1.141	0.492	7.069	3.152	0.829	0.011	1.207 (-45%)	1.249 (-62%)

TABLE VI : Effect of the decoder-only design on nuScenes. Prediction quality is preserved while parameters and FLOPs are roughly halved and thirded.

Panel of Figure 11 — Fig. 5 : Qualitative comparison of OL vs. CL rollouts with different replanning periods . While the OL model accumulates lateral drift and departs the lane, the CL model recovers lane alignment (bottom). Additionally, the CL ego decelerates for a leading vehicle during a right turn (top).

Explore similar work

Mar 25, 2026

Goal-Oriented Reactive Simulation for Closed-Loop Trajectory Prediction

Current trajectory prediction models are primarily trained in an open-loop manner, which often leads to covariate shift and compounding errors when deployed in real-world, closed-loop settings. Furthermore, relying on static datasets or non-reactive log-replay simulators severs the interactive loop, preventing the ego agent from learning to actively negotiate surrounding traffic. In this work, we propose an on-policy closed-loop training paradigm optimized for high-frequency, receding horizon ego prediction. To ground the ego prediction in a realistic representation of traffic interactions and to achieve reactive consistency, we introduce a goal-oriented, transformer-based scene decoder, resulting in an inherently reactive training simulation. By exposing the ego agent to a mixture of open-loop data and simulated, self-induced states, the model learns recovery behaviors to correct its own execution errors. Extensive evaluation demonstrates that closed-loop training significantly enhances collision avoidance capabilities at high replanning frequencies, yielding relative collision rate reductions of up to 27.0% on nuScenes and 79.5% in dense DeepScenario intersections compared to open-loop baselines. Additionally, we show that a hybrid simulation combining reactive with non-reactive surrounding agents achieves optimal balance between immediate interactivity and long-term behavioral stability.

Harsh Yadav, Tobias Meisen

University of Wuppertal, 42119, Germany

Trajectory Prediction Autonomous Driving Autonomous Driving Simulation

Ask PDF

Sep 25, 2026

Guiding End-to-End Driving Models with Endpoint-Constrained Trajectory Optimization

End-to-end driving policies are commonly trained through open-loop behavior cloning, yet they must ultimately operate in closed-loop when deployed on a vehicle, creating a fundamental mismatch between training and execution. Beyond the commonly studied effects of covariate shift and causal confusion, we identify a complementary factor for this open-loop/closed-loop gap: waypoint-based supervision and displacement metrics do not ensure that the intermediate trajectory is physically coherent or easy for the controller to track. We observe that these inconsistencies concentrate primarily at intermediate waypoints, while the predicted endpoint remains comparatively reliable. Based on this observation, we introduce Endpoint-Constrained Optimization (ECO), a lightweight postprocessing layer that anchors the trajectory to the vehicle's executed history, preserves the policy's predicted endpoint, and reshapes the intermediate waypoints to improve feasibility. ECO requires no map, privileged simulator state, or additional training, and can be inserted between a broad range of waypoint-emitting policies and their controllers. Across two closed-loop simulators, it improves the aggregate closed-loop score of all six evaluated generative and regression-based driving policies, and the gains tend to increase with how often the base plans violate motion limits. On HUGSIM, ECO improves VaVAM from 18.1 to 31.0 HD-Score (+71%), achieving 1st place on the HUGSIM Closed-Loop Driving Challenge. Similarly, on AlpaSim, ECO increases the scene scores of VaVAM and DiffusionDrive by 123% and 22%, respectively. These results show that for a broad collection of end-to-end driving models, repairing the intermediate geometry of predicted trajectories without changing the policy's predicted endpoint can substantially improve closed-loop performance.

Brayden Zhang, Mahsa Golchoubian, Igor Gilitschenski +2

University of Toronto · Vector Institute · NVIDIA Research +2

End-to-End Autonomous Driving Trajectory Optimization Autonomous Driving Simulation

Ask PDF

Sep 7, 2026

REFINE: Trajectory Representation Learning via Closed-Loop Transcription -- Extended Version

Trajectory representation learning underpins a wide range of trajectory analytics tasks; however, most existing self-supervised approaches, whether discriminative or generative, adopt an open-loop paradigm, relying on fixed data augmentations or random masking without feedback, which limits their ability to generalize and scale. We propose REFINE, a simple yet effective Representation lEarning Framework vIa closed-loop traNscription rEfinement for trajectory data. Drawing upon feedback control theory, REFINE tightly couples road-network-aware generative reconstruction with feedback-driven contrastive learning, enabling the model to capture fine-grained local movement semantics and global spatio-temporal dependencies without manually designed augmentation views. We further provide a control-theoretic analysis that establishes convergence guarantees for the proposed closed-loop optimization. Extensive experiments on four real-world datasets demonstrate that REFINE consistently outperforms state-of-the-art methods across multiple downstream tasks while remaining computationally efficient and scalable. This paper is an extended version of REFINE: Trajectory Representation Learning via Closed-Loop Transcription, to appear in KDD 2026.

Sean Bin Yang, Ying Sun, Jilin Hu +5

Aalborg University Aalborg, Denmark · Chongqing Universtiy of Posts and Telecommunications Chongqing, China · East China Normal University Shanghai, China +1

Contrastive Learning Self-Supervised Time Series Representation Learning Trajectory Representation Learning

Ask PDF