Organizations: Institute of Automation, Chinese Academy of Sciences · Chongqing Chang’an Technology Co., Ltd · Zhongguancun Academy · School of Artificial Intelligence, University of Chinese Academy of Sciences · College of AI, Tsinghua University
We introduce Latent-WAM, an efficient end-to-end autonomous driving framework that achieves strong trajectory planning through spatially-aware and dynamics-informed latent world representations. Existing world-model-based planners suffer from inadequately compressed representations, limited spatial understanding, and underutilized temporal dynamics, resulting in sub-optimal planning under constrained data and compute budgets. Latent-WAM addresses these limitations with two core modules: a Spatial-Aware Compressive World Encoder (SCWE) that distills geometric knowledge from a foundation model and compresses multi-view images into compact scene tokens via learnable queries, and a Dynamic Latent World Model (DLWM) that employs a causal Transformer to autoregressively predict future world status conditioned on historical visual and motion representations. Extensive experiments on NAVSIM v2 and HUGSIM demonstrate new state-of-the-art results: 89.3 EPDMS on NAVSIM v2 and 28.9 HD-Score on HUGSIM, surpassing the best prior perception-free method by 3.2 EPDMS with significantly less training data and a compact 104M-parameter model.
Figures & tables
Figure 1: Performance vs. training data scale on NAVSIM v2. Bubble size indicates model parameters. Our Latent-WAM achieves the highest EPDMS with significantly fewer training data and smaller model size, demonstrating superior data efficiency over existing world-model-based methods. World4Drive is marked separately as it employs an additional ViT-L depth estimator.
Figure 2: Overview of the Latent-WAM architecture. See Sec. 3.1 for details.
Figure 3: Teacher Forcing Attention Mask.
Method
NC ↑
DAC ↑
DDC ↑
TLC ↑
EP ↑
TTC ↑
LK ↑
HC ↑
EC ↑
EPDMS ↑
Perception-based Methods
TransFuser [ 30 ]
96.9
89.9
97.8
99.7
87.1
95.4
92.7
98.3
87.2
76.7
DriveSuprim [ 42 ]
97.5
96.5
99.4
99.6
88.4
96.6
95.5
98.3
77.0
83.1
ReCogDrive [ 21 ]
98.3
95.2
99.5
99.8
87.1
97.5
96.6
98.3
86.5
83.6
DiffusionDrive [ 23 ]
98.2
95.9
99.4
99.8
87.5
97.3
96.8
98.3
87.7
84.5
WorldRFT [ 40 ]
97.8
96.5
99.5
99.8
88.5
97.0
97.4
98.1
69.1
86.7
Table 1: Comparison with state-of-the-art methods on the NAVSIM v2 with extended metrics. We indicate the best and second best with bold and underlined respectively. †: The reported results are dependent on perception-based annotation.
Method
RC ↑
HD-Score ↑
E
M
H
X
Avg .
E
M
H
X
Avg .
UniAD
[ 13 ]
58.6
41.2
40.4
26.0
40.6
48.7
29.5
27.3
14.3
28.9
VAD
[ 15 ]
38.7
27.0
25.5
23.0
27.9
24.3
0 9.9
10.4
0 8.2
12.3
LTF
[ 4 ]
68.4
40.7
36.9
25.5
41.4
52.8
24.6
19.8
0 8.1
24.8
GTRS-Dense
[ 22 ]
64.2
50.0
20.7
22.3
38.0
55.5
39.0
11.7
14.3
28.6
Ours
84.2
42.5
30.6
35.5
45.9
72.5
24.0
12.2
18.1
28.9
Table 2: Photorealistic closed-loop evaluation on HUGSIM [ 48 ] . Zero-shot generalization using our model from the NAVSIM-v2 evaluation. Scores are per difficulty and overall average road completion (RC) and HD-Score, higher always better.
Table 3: Ablation study of each proposed component. + and - denote improvement/degradation relative to the baseline.
Table 7
Prediction Temporal Stride
EPDMS ↑
0→8
88.4
−3→0→4→8
89.3
−3→−2→−1→0→2→4→6→8
89.1
Table 6: Ablation study on world model prediction temporal stride
Figure 4: Visualization of planning trajectories, where the green line is the human trajectory, the yellow line is the predicted trajectory of corresponding method.
Figure 5: Visualization of attention maps between scene tokens and image patches. From top to bottom, the three groups correspond to going straight, turning right, and turning left respectively.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Module
Parameters
Memory Usage
Latency
World Encoder
86.6M
—
100ms
Trajectory Decoder
8.4M
—
6ms
All modules
104M
1.1GB
107ms
Appendix
Table 7: Per-module inference latency of Latent-WMA