cs.AINov 17, 2026

Residual-Guided Lightweight Reinforcement Learning for Risk-Aware Trajectory Selection with a Frozen World Model

Authors: Jiaqi Wu, Yongqi Dong, Yijie Zhong, Maple Shao, Yide Tao, Linsen Cai, Weize Cai, Hai L. Vu, 

Organizations: RWTH Aachen University, Aachen, Germany. · Monash University, Melbourne, Australia.

Abstract

In automated driving, world models provide future scene predictions for evaluating candidate trajectories, but fixed selection rules may not capture scene-dependent trade-offs. We propose a residual-guided reinforcement learning (RL) framework for lightweight trajectory selection with a frozen world model and a fixed candidate bank. The framework uses proximal policy optimization (PPO) to fine-tune a joint policy over candidate selection and bounded trajectory residuals. A context-conditioned ranking network learns candidate preferences from feedback obtained by evaluating local perturbations of the selected trajectories. At inference, the selector combines learned preference and predicted risk ranks using fixed weights and returns the selected candidate trajectory without modification. Fine-tuning updates 16,106 parameters across the actor and critic, while the deployed selector retains only the 11,725-parameter ranking network, with the residual branch and critic used exclusively during training. In open-loop evaluation on the nuScenes validation set, the method reduces average Euclidean trajectory error (L2) by 3.99% relative to the RiskWorld model baseline without RL-based selection. Under matched budgets for feedback accesses and actor updates, residual exploration reduces L2 by 0.83% relative to the categorical PPO variant trained without residuals. The method also maintains lower L2 than RiskWorld when risk features are removed or shuffled.

Figures & tables

Explore similar work

CardsList
  1. Risk-Aware World Modeling with Flow-Guided Occupancy Evolution for Selective Trajectory Planning in Automated Driving

    Sep 16, 2026Rongxiang Zeng, Linsen Cai, Jiafu Zhang +7World Model-Based Planning3D Occupancy Prediction

  2. Trajectory Evaluation via Rollout-Free World Model for End-to-end Autonomous Driving

    Nov 3, 2026Junzhe Wang, Jinqing Zhang, Qingjie LiuAutonomous Driving PlanningEnd-to-End Autonomous Driving

  3. WALT: Learning World-Model-Aligned Latent Trajectories for Autonomous Driving

    Sep 24, 2026Mingkai Jia, Jiaxin Guo, Zhijian Shu +5World Model-Based PlanningAutonomous Driving Planning