World-action models (WAMs) jointly model future traffic states and driving actions for end-to-end autonomous driving in intelligent transportation systems. However, their action learning remains dominated by expert-trajectory imitation, leaving unsafe alternatives close to expert trajectories insufficiently constrained. To improve WAM robustness in safety-critical scenarios, we propose NegWAM, a safety post-training framework for unified WAMs that learns explicit safety boundaries from near-expert failure trajectories. NegWAM constructs negative trajectories that are geometrically close to the expert trajectory yet carry safety risks, uses them as repulsive anchors to steer predictions away from dangerous behavior, and encodes them as scene-conditioned rider tokens to learn scene--trajectory compatibility. During training, NegWAM jointly optimizes the flow-matching and near-expert negative objectives, while the auxiliary rider branch is discarded at inference, incurring no additional overhead. On NAVSIM v2, NegWAM achieves 40.4 EPDMS on navhard, a 4.1% relative improvement over the strongest compared scorer-free WAM baseline, and achieves 93.3 EPDMS on navtest. These results demonstrate that NegWAM achieves strong performance in safety-critical scenarios while preserving routine-driving performance. Zero-shot closed-loop evaluation on HUGSIM further demonstrates cross-benchmark generalization. The code is available at https://github.com/Zizizi-hao/NegWAM.
Figures & tables
Fig. 1: Sources of supervision for post-training in end-to-end autonomous driving. (a) Environment-derived rewards from closed-loop interaction with a simulator or world model. (b) Trajectory scores from an external rule-based or learned scorer. (c) NegWAM uses offline-mined negative trajectories as repulsive anchors and scene-conditioned rider tokens to introduce negative-aware losses into a unified world–action model.
Fig. 2: Architecture of our NegWAM. In Stage 1, we adapt a pretrained world-action DiT into a unified world–action model through flow matching. In Stage 2, scene-specific hard-negative trajectories are used as both repulsive anchors and trajectory riders for compatibility learning. The auxiliary rider branch is removed at inference, introducing no additional deployment cost.
Fig. 3: Scene-conditioned rider tokens for scene–trajectory compatibility learning. Expert and negative riders attend to observed scene tokens through shared DiT attention, and their compatibility scores define a ranking loss that updates the shared backbone.
Methods
Sensor
Stage
NC ↑
DAC ↑
DDC ↑
TLC ↑
EP ↑
TTC ↑
LK ↑
HC ↑
EC ↑
EPDMS ↑
S1
94.4
98.8
100
99.5
100
93.5
99.3
87.7
36.0
PDM-Closed
-
S2
90.5
90.6
95.4
98.4
100
86.6
74.2
91.9
29.7
56.6
E2E-based Methods
S1
96.0
79.7
97.4
99.5
81.3
93.1
90.8
96.8
73.8
DiffusionDrive [ 41 ]
MV
S2
82.1
72.2
88.5
98.7
85.1
78.8
49.2
89.3
71.2
24.2
S1
96.6
80.5
96.3
99.3
82.3
94.9
91.5
97.7
67.8
TABLE I: Performance comparison on the NAVSIM v2 navhard benchmark. S1 denotes evaluation on the initial real-world scenes, while S2 denotes evaluation on the pre-generated reactive follow-up scenes. All results are reported using the corrected official NAVSIM v2.2 evaluation protocol. MV and SV denote multi-view and single-view camera inputs, respectively. DriveFuture † denotes its variant without the GTRS-Dense scorer; the full system reports 55.5 with the GTRS-Dense scorer [ 40 ] .
Methods
Sensor
NC ↑
DAC ↑
EPDMS ↑
Human agent
-
100.0
100.0
97.9
E2E-based Methods
DiffusionDrive [ 41 ]
MV
98.2
95.9
84.5
DiffusionDriveV2 [ 49 ]
MV
97.7
96.6
87.5
VLA-based Methods
Recogdrive [ 50 ]
MV
98.3
95.2
83.6
TABLE II: Performance comparison on the NAVSIM v2 navtest benchmark. All results are reported using the corrected official NAVSIM v2.2 evaluation protocol. MV and SV denote multi-view and single-view camera inputs, respectively.
RC ↑
HD-Score ↑
Method
E
M
H
X
Overall
E
M
H
X
Overall
VAD [ 53 ]
38.7
27.0
25.5
23.0
27.9
24.3
9.9
10.4
8.2
12.3
LTF [ 19 ]
68.4
40.7
36.9
25.5
41.4
52.8
24.6
19.8
8.1
24.8
DriveWAM [ 7 ]
76.6
33.8
33.9
30.2
43.6
64.0
16.1
16.1
12.6
27.2
UniAD [ 1 ]
58.6
41.2
40.4
26.0
40.6
48.7
29.5
27.3
14.3
28.9
NegWAM (Ours)
90.3
32.8
31.2
26.1
45.1
81.7
15.8
14.2
10.8
30.6
TABLE III: Zero-shot performance on the HUGSIM benchmark. E/M/H/X denote easy, medium, hard, and extreme scenarios. Higher is better for both route completion (RC) and HD-Score.
Post-training
Repulsive anchors
Scene-conditioned rider tokens
EPDMS
Pretrained Baseline
37.7
✓
38.2
✓
✓
39.1
✓
✓
38.8
✓
✓
✓
40.4
TABLE IV: Ablation study of the proposed post-training components. A checkmark indicates that the corresponding component is enabled.
Fig. 4: Scene–trajectory compatibility scores for three example scenes. Rows correspond to the trajectories shown on the left, and columns correspond to the observed scenes shown above. Each cell reports the compatibility score of the corresponding scene–trajectory pair.
Fig. 5: Effect of Post-Training Steps for NAVSIM v2 navhard . NegWAM results are reported as mean ± one standard deviation over three random seeds, while the baseline is a single run.
Fig. 6: Effect of Post-Training Steps for NAVSIMv2 navtest . NegWAM results are reported as mean ± one standard deviation over three random seeds, while the baseline is a single run.
Learning rate
Repulsion
Compatibility
Learning rate
EPDMS
λrep
EPDMS
λcompat
EPDMS
5×10−7
39.5
50
39.0
0.05
39.3
1×10−6
40.4
100
40.4
0.1
40.4
2×10−6
39.1
150
39.8
0.2
39.1
TABLE V: Sensitivity to the learning rate, repulsion weight, and compatibility weight.
Fig. 7: Qualitative comparison between DriveWAM [ 7 ] and our NegWAM in safety-critical scenarios. DriveWAM (top) predicts trajectories that approach nearby traffic participants, whereas NegWAM (bottom) maintains safer clearance while better following the human-expert trajectory.
Fig. 8: Representative closed-loop rollout in HUGSIM. Rows are ordered chronologically from top to bottom, with three rendered camera views at each time point. Yellow dots mark the overlaid trajectory in the center view. An oncoming vehicle approaches and passes on the left as the ego vehicle proceeds along the road.
Method
Latency (ms) ↓
FPS ↑
Model size (MB) ↓
Peak GPU memory (MB) ↓
EPDMS ↑
DriveWAM [ 7 ]
871.35
1.15
6241.81
14029.08
37.7
NegWAM
877.86
1.14
6241.81
14029.08
40.4
TABLE VI: Planning score on navhard and inference efficiency. Higher EPDMS and FPS and lower latency, model size, and peak GPU memory are preferred.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Model Configuration
Sensors
Observed Input
Future Indices
Action Target
SNR Shift (V/A)
Denoising Steps (V/A)
Model Size
Peak GPU Memory
1 × Cam. (SV)
1×256×448 4 historical ego poses
{1,3,5,7}
8-step Δ -pose
5.0/1.0
3/10
6,241.81 MB
14,029.08 MB
Appendix
TABLE VII: Model and training configurations.
Methods
Sensor
NC ↑
DAC ↑
DDC ↑
TLC ↑
EP ↑
TTC ↑
LK ↑
HC ↑
EC ↑
EPDMS ↑
Human agent
-
100.0
100.0
99.8
100.0
97.8
100.0
100.0
98.1
90.1
97.9
E2E-based Methods
DiffusionDrive [ 41 ]
MV
98.2
95.9
99.4
99.8
87.5
97.3
96.8
98.3
87.7
84.5
DiffusionDriveV2 [ 49 ]
MV
97.7
96.6
99.2
99.8
88.9
97.2
96.0
97.8
91.0
87.5
VLA-based Methods
Recogdrive [ 50 ]
MV
98.3
95.2
98.3
99.8
87.1
97.5
96.6
99.5
86.5
83.6
Appendix
TABLE VIII: Performance comparison on the NAVSIM v2 navtest benchmark. All results are reported using the corrected official NAVSIM v2.2 evaluation protocol. MV and SV denote multi-view and single-view camera inputs, respectively. All evaluated methods use camera-only inputs without LiDAR.
Fig. 9: Additional closed-loop rollout near a junction in HUGSIM. The four rows are ordered chronologically from top to bottom; each row contains three rendered camera views. Yellow dots show the overlaid trajectory in the center view. A nearby white vehicle is visible in the early snapshots, followed by the ego vehicle’s continued motion along the road.
Fig. 10: Additional successful cases. Comparison between DriveWAM [ 7 ] (top) and NegWAM (bottom) in an intersection-straight scenario with a stopped roadside vehicle and an intersection left-turn scenario. NegWAM safely avoids the stopped vehicle and maintains its trajectory within the drivable area, while the baseline produces collision-prone or off-road trajectories.
Fig. 11: Representative failure cases. Comparison between DriveWAM (top) and NegWAM (bottom). In the first case, NegWAM fails to follow the expert’s leftward overtaking maneuver and collides with the stopped leading vehicle, whereas DriveWAM avoids the collision through conservative deceleration. In the second case, NegWAM maintains insufficient clearance from a vehicle in the target lane while turning, resulting in a collision.
In autonomous driving, World-Action Models (WAMs) have improved end-to-end planning by transferring video dynamics priors to action prediction, but many still couple planning with future-video generation at inference, incurring substantial computational overhead. We present SimWAM, a simple yet effective WAM that leverages future-video prediction solely as a training-time supervision signal. It co-trains a pretrained video expert and a lightweight action expert with joint flow matching. An isolated attention mask keeps action prediction independent of future frames, allowing trajectory prediction without future-frame generation at inference. This design supports multiple pretrained video backbones and independent action-expert scaling within a shared attention interface, while preserving the joint learning objective. Moreover, we apply reinforcement learning to optimize a compositional driving reward beyond trajectory imitation. Experiments show that SimWAM achieves 91.9 PDMS on NAVSIM with a favorable trade-off between accuracy and latency among world-model-based planners, while transferring zero-shot to nuScenes. It also achieves competitive planning accuracy on WOD-E2E and PhysicalAI-Autonomous-Vehicles. These results position SimWAM as a plain yet solid baseline for efficient autonomous driving. The code and model weights are available at https://github.com/H-EmbodVis/SimWAM/.
Zongchuang Zhao, Xin Zhou, Tianyang Xu +6
Huazhong University of Science & Technology · Dongfeng Research & Development Institute
We introduce Latent-WAM, an efficient end-to-end autonomous driving framework that achieves strong trajectory planning through spatially-aware and dynamics-informed latent world representations. Existing world-model-based planners suffer from inadequately compressed representations, limited spatial understanding, and underutilized temporal dynamics, resulting in sub-optimal planning under constrained data and compute budgets. Latent-WAM addresses these limitations with two core modules: a Spatial-Aware Compressive World Encoder (SCWE) that distills geometric knowledge from a foundation model and compresses multi-view images into compact scene tokens via learnable queries, and a Dynamic Latent World Model (DLWM) that employs a causal Transformer to autoregressively predict future world status conditioned on historical visual and ego status representations. Extensive experiments on NAVSIM v2 and HUGSIM demonstrate new state-of-the-art results: 89.3 EPDMS on NAVSIM v2 and 28.9 HD-Score on HUGSIM, surpassing the best prior perception-free method by 3.2 EPDMS with significantly less training data and a compact 104M-parameter model. The code and model weights have been open-sourced at https://github.com/yukki555/Latent-WAM.
Linbo Wang, Yupeng Zheng, Qiang Chen +13
Institute of Automation, Chinese Academy of Sciences · Chongqing Chang’an Technology Co., Ltd · Zhongguancun Academy +2
World models for autonomous driving should anticipate the consequences that matter for driving decisions rather than reconstruct every detail of future scenes. However, predictive representations alone are not guaranteed to distinguish candidate actions that lead to different planning outcomes.We propose DA-WAM, a decision-aligned world action model that couples a Joint-Embedding Predictive Architecture (JEPA) with candidate-level trajectory scoring. During joint training, the JEPA objective grounds future representations in observed scene dynamics, while the planning objectives encourage these representations to capture distinctions relevant to safety and utility. An action-conditioned predictor generates a distinct future latent for each candidate trajectory, and a multi-head trajectory scorer uses the corresponding latent to estimate planning factors and overall utility. In addition, safety-critical post-training with hard negatives further improves discrimination between geometrically similar trajectories with different safety outcomes. DA-WAM achieves state-of-the-art performance among learned camera-based planners on NAVSIM-v1, NAVSIM-v2, and Bench2Drive. Ablation studies demonstrate the effectiveness of each component in driving decision-making.
Ruiguo Zhong, Xiaolong Chen, Benshan Ma +5
The Hong Kong University of Science and Technology (Guangzhou) · Leapmotor · The Hong Kong University of Science and Technology