Driving world action models (WAMs) learn from paired trajectories and future observations. Post-training with reinforcement learning allows the policy to explore alternative trajectories and improve planning using driving rewards. However, driving logs contain future videos only for the recorded trajectories, leaving these alternatives without corresponding visual supervision. Thus, we introduce SimVTA, a simulation-grounded post-training framework that combines trajectory optimization with video supervision. First, we sample candidate trajectories for each observation and use a real-world simulator (e.g., World Engine) to render future videos for high-reward candidates in real-world scenes reconstructed with 3D Gaussian Splatting (3DGS). This provides trajectory-matched supervision independent of the model's own predictions. Under a limited rendering budget, we prioritize video supervision for higher-error futures among these rendered candidates. Finally, we optimize trajectory planning with Group Relative Policy Optimization (GRPO) and use a flow-matching loss on the matched videos to update shared representations. At inference time, the model predicts trajectories directly, without simulator calls. On NAVSIM v1, applying SimVTA to a WAM initialization improves PDMS from 86.2 to 91.3 (+5.1). It achieves 88.2 EPDMS on NAVSIM v2 and an average HD-Score of 38.8 in closed-loop evaluation on HUGSIM. Ablations show that updating shared representations with video supervision improves planning, while prioritizing higher-error futures among high-reward trajectories further improves performance.
Figures & tables
Figure 1: From logged futures to candidate-matched supervision. (1) WAM pretraining uses the ground-truth (GT) trajectory and its logged future video. (2) Trajectory GRPO explores candidate trajectories using driving rewards, without matched video supervision. (3) World Engine renders reconstructed scenes along selected candidate paths, providing matched supervision independent of the model’s predictions.
Figure 2: Simulation-grounded video–trajectory alignment. (a) Training overview. (b) Trajectory learning combines GRPO with a behavior-cloning (BC) regularizer. (c) Video supervision selects high-reward, higher-error futures. (d) World Engine renders matched futures along executed paths.
Figure 3: Planning results and video-supervision comparisons. (a) NAVSIM v1. (b) NAVSIM v2 and HUGSIM. (c) Video-supervision configurations and rendering budget on NAVSIM v1.
Figure 4: Qualitative planning comparison. Left: planned trajectories of SimVTA (A, blue) and Epona (B, orange). Right: World Engine renderings along the corresponding paths at 1–4 s. Red boxes highlight Epona’s collision and road-departure regions.
Source
FID ↓
FVD ↓
3DGS renderings
World Engine
7.6
21.8
WAM predictions
Trajectory GRPO only (Exp0)
20.4
105.4
SimVTA (Exp2)
13.1
46.3
Table 4: Video quality.
Figure 5: Matched futures for alternative actions. Left: shared observation history and executed paths for going straight (A), changing lanes to the left (B), and changing lanes to the right (C). Right: corresponding future frames rendered by the simulator. Colors match the paths and image rows.
Table 7Table 8Table 9
Figure 6: Video–trajectory alignment before and after post-training. Left: observation history and trajectory in BEV. Right: World Engine renderings and predictions from SimVTA and Epona.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Parameter
Value
Sampled trajectories / rendering budget
G=32 / K=4
Batch size per GPU / default random seed
4 / 1234
Training hardware
16 NVIDIA H20 GPUs
Initial learning rate / AdamW betas
10−5 / (0.9,0.95)
Weight decay / LR decay factor
0.01 / 0.5
LR decay steps
300, 1500
Appendix
Table 11: Exp2 video-supervision settings and default optimization parameters.
Setting
K
Rendered pool
Video-training set
Video objective
PDMS
Exp0
0
None
None
None
89.1
Exp1
4
High reward
Q2
FM
89.1
Exp2
4
High reward
Q2
FM
91.3
Exp3
4
High reward
Q2, shuffled
FM
83.1
Exp4
4
2 high + 2 low
Q2 + Q3
FM
90.9
Exp5
4
High reward
Q1 + Q2
FM
91.0
Appendix
Table 12: Video-supervision configurations. K is the maximum number of rendered candidates per observation in each sampling round. For Exp9, this budget applies to offline cache construction. FM denotes flow matching.
Setting
NC
DAC
TTC
Comfort
EP
PDMS
Exp0
97.1
96.7
96.0
99.8
83.6
89.1
Exp1
97.6
96.7
96.4
99.7
83.3
89.1
Exp2
98.0
97.2
97.0
100.0
86.7
91.3
Exp3
94.1
91.3
91.8
99.6
81.2
83.1
Exp4
97.4
97.3
96.4
99.9
86.4
90.9
Exp5
98.0
97.1
97.0
99.9
86.1
91.0
Appendix
Table 13: NAVSIM v1 ablation results. Configurations are listed in Table 12 .
Setting
Random seed
Mean ± SD
1234
20260901
20260910
Exp0
89.1
89.2
89.0
89.1±0.1
Exp2
91.3
91.4
91.3
91.3±0.1
Exp5
91.0
90.9
91.0
91.0±0.1
Exp7
91.1
91.2
91.2
91.2±0.1
Appendix
Table 14: NAVSIM v1 PDMS across random seeds. The final column reports the mean and sample standard deviation over three runs.
Method
NAVSIM v1 PDMS
NAVSIM v2 EPDMS
SimWAM ( Zhao et al., 2026 )
91.5
90.2
Latent-WAM ( Wang et al., 2026b )
–
89.3
WA-JEPA ( Wang et al., 2026c )
91.8
91.7
SimVTA
91.3
88.2
Appendix
Table 15: Recent WAM results reported by their respective papers. Training data, initialization, and evaluators differ across methods.
Figure 7: Reported planning results across benchmarks. (a) NAVSIM v1 PDMS and v2 EPDMS, with NAVSIM v2 component-score differences ( SimVTA minus the NAVSIM-adapted Epona initialization). (b) HUGSIM RC and HD-Score using benchmark aggregation. (c) RC and (d) HD-Score for DrivoR and SimVTA by scenario difficulty. Values correspond to Tables 1 , 2 , and 3 .