In automated driving, world models provide future scene predictions for evaluating candidate trajectories, but fixed selection rules may not capture scene-dependent trade-offs. We propose a residual-guided reinforcement learning (RL) framework for lightweight trajectory selection with a frozen world model and a fixed candidate bank. The framework uses proximal policy optimization (PPO) to fine-tune a joint policy over candidate selection and bounded trajectory residuals. A context-conditioned ranking network learns candidate preferences from feedback obtained by evaluating local perturbations of the selected trajectories. At inference, the selector combines learned preference and predicted risk ranks using fixed weights and returns the selected candidate trajectory without modification. Fine-tuning updates 16,106 parameters across the actor and critic, while the deployed selector retains only the 11,725-parameter ranking network, with the residual branch and critic used exclusively during training. In open-loop evaluation on the nuScenes validation set, the method reduces average Euclidean trajectory error (L2) by 3.99% relative to the RiskWorld model baseline without RL-based selection. Under matched budgets for feedback accesses and actor updates, residual exploration reduces L2 by 0.83% relative to the categorical PPO variant trained without residuals. The method also maintains lower L2 than RiskWorld when risk features are removed or shuffled.
Figures & tables
Fig. 1: Overview of residual-guided reinforcement learning. (a) The frozen RiskWorld [ riskworld ] backbone fuses the RiskExpert and BEV features in GeoBEV [ geobev ] . (b) The ranking network predicts candidate scores. (c) Joint candidate–residual sampling provides feedback for PPO training. (d) Rank fusion selects an original candidate at inference. Training and inference share the ranking network in (b). MSE: mean squared error; KL: Kullback–Leibler divergence.
Method
Input
Aux. Sup.
Params. (M)
L2 (m) ↓
CR (%) ↓
1 s
2 s
3 s
Avg.
1 s
2 s
3 s
Avg.
UniAD [ uniad ]
C
M+B+Mo+Tr+O
125.00
0.44
0.67
0.96
0.69
0.04
0.08
0.23
0.12
VAD [ vad ]
C
M+B+Mo
58.36
0.41
0.70
1.05
0.72
0.07
0.17
0.41
0.22
SparseDrive-S [ sparsedrive ]
C
M+B+Mo+Tr
85.90
0.29
0.58
0.96
0.61
0.01
0.05
0.18
0.08
DiffusionDrive [ diffusiondrive ]
C
M+B+Mo+Tr
92.04
0.27
0.54
0.90
0.57
0.03
0.05
0.16
0.08
RiskWorld [ riskworld ]
C+M+GT-Tr
Risk/O+Coll.
91.099
0.182
0.374
0.660
0.405
0.068
0.083
0.147
0.099
TABLE I: Evaluation and comparison of Development-stage open-loop planning performance on nuScenes.
Risk features
RiskWorld [ riskworld ]
Ours
L2 (m) ↓
CR (%) ↓
L2 (m) ↓
CR (%) ↓
Original (Full)
0.4054
0.0993
0.3893
0.0979
Zeroed
0.6399
0.2816
0.5698
0.2491
Shuffled
0.5004
0.2675
0.4636
0.2344
TABLE II: Planning performance under inference-time perturbations of RiskExpert features.
Fig. 2: Qualitative comparison on nuScenes: (a) nighttime road, (b) left turn, and (c) intersection. Each row shows six camera views, BEV trajectories, and a close-up. Crosses mark collisions; trajectory markers indicate 1, 2, and 3 s. FL/F/FR and RL/R/RR denote the front-left/front/front-right and rear-left/rear/rear-right cameras. Nighttime views are gamma-corrected with an exponent of 0.5 for visualization.
Safe motion planning in automated driving requires anticipating evolving traffic risks and deciding when to revise the current planned trajectory. We introduce RiskWorld, a risk-aware world modeling framework for shared occupancy forecasting and selective trajectory replacement. Spatial risk fields and temporal actor context are fused with visual bird's-eye-view features. Flow-guided evolution transports occupancy and scene features, while signed residuals correct occupancy after transport. One forecast is generated per planning step and reused across candidates. Each candidate is compared with a current-state persistence reference, yielding a nonnegative collision-score correction. The trajectory selected by current-world evaluation serves as the planning anchor and is replaced only when additional predicted risk triggers intervention and an alternative satisfies component-wise constraints on predicted risk and trajectory error. Candidate geometries remain unchanged. We evaluate RiskWorld for open-loop planning on nuScenes using camera features, annotation-derived current and historical actor states, and dataset-provided map context. RiskWorld achieves the lowest collision rate at a long evaluation horizon of 3 s, and the second-best average L2 error among various state-of-the-art baselines, while running at 11.5 FPS on a single NVIDIA RTX 4090 with 90.81 M parameters. Within-setting ablations show that RiskWorld achieves lower collision rates than the current-state rescoring baseline, while forecast reuse enables additional candidates to be evaluated at low marginal computational cost.
Rongxiang Zeng, Linsen Cai, Jiafu Zhang +7
RWTH Aachen University, Aachen, Germany · Monash University, Melbourne, Australia · Technical University of Munich (TUM), Munich, Germany
Although existing candidate trajectory evaluators have substantially improved end-to-end planning accuracy, their underlying mechanism remains suboptimal. Model-based evaluators typically make scoring future-aware by explicitly predicting candidate-conditioned future states over the planning horizon, which we term rollout. However, rollout incurs high inference cost and accumulates prediction errors. We argue that planning does not require reconstructing a full future scene for each candidate, but only future-relevant information for reliable comparison. We therefore propose RFWorld, a Rollout-Free World model for end-to-end trajectory evaluation. RFWorld constructs a shared scene memory and shapes it into a future-readable representation via lightweight training-time temporal readout with semantic supervision, keeping training overhead low. At inference time, candidate trajectories directly query the shared memory via sparse spatial readout for scoring, reducing computation by roughly the planning horizon times the number of candidate trajectories, while avoiding recursive rollout errors. Experiments on NAVSIM and Bench2Drive show that RFWorld learns a more planning-aligned world representation and achieves state-of-the-art performance.
Junzhe Wang, Jinqing Zhang, Qingjie Liu
State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, China · Xiaomi EV
Driving world models learn rich predictive representations of the surrounding environment from visual observations, yet accurate visual prediction does not necessarily translate into effective trajectory planning. We argue that a key bottleneck lies in the mismatch between visual world states and raw geometric trajectories, which may limit the planner's ability to exploit action-relevant semantics encoded by the world model. To address this issue, we propose World-Model Alignment for Latent Trajectories (WALT), which learns a compact generative trajectory latent space by transferring information from a frozen pretrained driving world model without modifying the world model itself. Rather than directly generating raw waypoints, WALT maps them into compact representations through a dual-branch trajectory autoencoder and transfers semantic knowledge from the frozen visual world model into this trajectory space, encouraging the learned action representation to capture scene-level cues relevant to future motion and planning. Beyond our proposed formulation, we systematically study latent learning based on Joint-Embedding Predictive Architectures (JEPA) and feature alignment following Representation Alignment (REPA) to investigate how trajectory-only representation learning affects downstream planning. We evaluate WALT on the NAVSIM benchmarks. Relative to the raw-waypoint baseline, WALT improves PDMS from 89.4 to 89.8 on NAVSIMv1 and EPDMS from 87.3 to 87.9 on NAVSIMv2 while reducing trajectory planner FLOPs by 30.5%. These results suggest that preserving world representations while extracting action-relevant information provides an effective interface for world-model-based trajectory planning.
Mingkai Jia, Jiaxin Guo, Zhijian Shu +5
The Hong Kong University of Science and Technology. · Horizon Robotics. · The Chinese University of Hong Kong. +2