Organizations: The Hong Kong University of Science and Technology. · Horizon Robotics. · The Chinese University of Hong Kong. · Nanjing University of Posts and Telecommunications. · Nankai University.
Driving world models learn rich predictive representations of the surrounding environment from visual observations, yet accurate visual prediction does not necessarily translate into effective trajectory planning. We argue that a key bottleneck lies in the mismatch between visual world states and raw geometric trajectories, which may limit the planner's ability to exploit action-relevant semantics encoded by the world model. To address this issue, we propose World-Model Alignment for Latent Trajectories (WALT), which learns a compact generative trajectory latent space by transferring information from a frozen pretrained driving world model without modifying the world model itself. Rather than directly generating raw waypoints, WALT maps them into compact representations through a dual-branch trajectory autoencoder and transfers semantic knowledge from the frozen visual world model into this trajectory space, encouraging the learned action representation to capture scene-level cues relevant to future motion and planning. Beyond our proposed formulation, we systematically study latent learning based on Joint-Embedding Predictive Architectures (JEPA) and feature alignment following Representation Alignment (REPA) to investigate how trajectory-only representation learning affects downstream planning. We evaluate WALT on the NAVSIM benchmarks. Relative to the raw-waypoint baseline, WALT improves PDMS from 89.4 to 89.8 on NAVSIMv1 and EPDMS from 87.3 to 87.9 on NAVSIMv2 while reducing trajectory planner FLOPs by 30.5%. These results suggest that preserving world representations while extracting action-relevant information provides an effective interface for world-model-based trajectory planning.
Figures & tables
Fig. 2 : WALT pipeline. In Stage 1, we train the dual-branch trajectory autoencoder with a frozen world model. The semantic branch feature is aligned with world state tokens with LWALT , while the geometric branch preserves reconstruction details and jointly trained with Lrecon . In Stage 2, we freeze the tokenizer and train the trajectory planner head of the frozen world model to generate latents, which are decoded into waypoints for future trajectory prediction.
Method
Venue
NC
DAC
EP
TTC
C
PDMS
Human
–
100
100
87.5
100
99.9
94.8
LAW [ 27 ]
ICLR’25
96.4
95.4
81.7
88.7
99.9
84.6
DrivingGPT [ 11 ]
ICCV’25
98.9
90.7
79.7
94.9
95.6
82.4
World4Drive [ 4 ]
ICCV’25
97.4
94.3
79.9
92.8
100
85.1
Epona [ 3 ]
ICCV’25
97.9
95.1
80.4
93.8
99.9
86.2
PWM [ 6 ]
NeurIPS’25
98.6
95.9
81.8
95.4
100
88.1
TABLE I : Reported results on NAVSIMv1 [ 23 ] . All metrics are on a 0–100 scale. Higher is better. Bold and underlined values indicate the best and second-best distinct scores among learned methods, respectively. These results include different model and training configurations.
Method
NC
DAC
DDC
TLC
EP
TTC
LK
HC
EC
EPDMS
Human
100
100
99.8
100
87.4
100
100
98.1
90.1
94.5
DriveVLA-W0 [ 5 ]
98.4
95.2
99.4
99.9
86.6
97.9
97.8
98.3
82.7
86.9
ARTEMIS ∗ [ 45 ]
98.3
95.1
98.6
99.8
81.5
97.4
96.5
98.3
–
83.1
PRIX ∗ [ 44 ]
98.0
85.6
99.5
99.8
87.4
97.2
97.1
98.3
87.6
84.2
DriveWorld-VLA [ 52 ]
98.6
99.1
99.6
99.8
87.4
97.9
97.0
97.8
78.6
86.8
EponaV2 [ 31 ] (w/o RL)
98.4
96.7
99.6
99.9
87.6
98.1
98.0
98.1
68.3
87.3
TABLE II : Comparison on NAVSIMv2 [ 24 ] . All metrics are on a 0–100 scale, where higher is better. ∗ denotes results reported using the original evaluator before the human-penalty aggregation update. Bold and underlined values indicate the best and second-best distinct scores among unstarred learned methods.
Fig. 3 : Qualitative trajectories and world-model alignment. Columns show cases for braking, straight driving, turn, and curve. From top to bottom: front-view images, world–trajectory correspondence maps, and predicted trajectories (orange) with ground truth (black). The maps visualize cosine similarity between projected semantic tokens from planner-generated latents and frozen visual world-model tokens, with warmer colors indicating higher similarity.
Fig. 4 : Trajectory representation analysis on NAVSIMv1. Columns compare raw waypoints, JEPA-Traj semantic latents, and WALT semantic latents on identical samples. From top to bottom: t-SNE embeddings, within-class and between-class cosine-similarity distributions, and class-pair mean-similarity matrices. Learned representations use only their semantic components.
Method
NC
DAC
EP
TTC
C
PDMS
Raw waypoints baseline
98.62
97.32
83.60
95.26
99.93
89.42
+ trajectory AE (w/o LWALT )
98.62
97.38
83.63
95.32
99.95
89.48
JEPA-Traj
98.55
97.37
82.85
95.91
100.00
89.46
REPA-Traj
98.48
97.46
83.79
95.11
99.98
89.49
WALT
99.11
96.99
83.63
96.34
100.00
89.83
TABLE III : Action-side representation comparison on NAVSIMv1 with the same frozen world-model backbone. Variants differ in generation target and representation supervision. All metrics are on a 0–100 scale. Higher is better.
Representation
Compression Ratio
GFLOPs ↓
Raw waypoint generation
–
297.47
WALT latent generation
4 ×
206.88
TABLE IV : Trajectory-generation cost for one denoising step under matched world-model conditioning. The latent-generation cost includes trajectory decoding, and the frozen world-model computation is excluded.
We introduce Latent-WAM, an efficient end-to-end autonomous driving framework that achieves strong trajectory planning through spatially-aware and dynamics-informed latent world representations. Existing world-model-based planners suffer from inadequately compressed representations, limited spatial understanding, and underutilized temporal dynamics, resulting in sub-optimal planning under constrained data and compute budgets. Latent-WAM addresses these limitations with two core modules: a Spatial-Aware Compressive World Encoder (SCWE) that distills geometric knowledge from a foundation model and compresses multi-view images into compact scene tokens via learnable queries, and a Dynamic Latent World Model (DLWM) that employs a causal Transformer to autoregressively predict future world status conditioned on historical visual and motion representations. Extensive experiments on NAVSIM v2 and HUGSIM demonstrate new state-of-the-art results: 89.3 EPDMS on NAVSIM v2 and 28.9 HD-Score on HUGSIM, surpassing the best prior perception-free method by 3.2 EPDMS with significantly less training data and a compact 104M-parameter model.
Linbo Wang, Yupeng Zheng, Qiang Chen +13
Institute of Automation, Chinese Academy of Sciences · Chongqing Chang’an Technology Co., Ltd · Zhongguancun Academy +2
World models in end-to-end autonomous driving predict future scene evolution to provide foresight for trajectory planning. Existing methods mainly study how to predict the future and how to use it, but less often ask which future representation is actually most useful for planning. To this end, we propose PlanWAM, a Planning-Shaped World Action Model. The key idea is to let the planning task shape the future-state representation, so that it retains the information most useful for planning. A latent world model then predicts this planning-shaped future latent representation from historical observations and uses it for planning, enabling foresighted planning. Specifically, we first use a Temporal Register Pyramid to compress multi-frame historical information in a recency-aware manner, learning a compact history representation oriented toward future reasoning and planning. We then introduce a privileged future posterior branch that observes ground-truth future frames, and shape its future latent representation with trajectory-planning objectives to obtain a planning-shaped future latent representation. Hindsight-to-Foresight Distillation trains a prior branch that depends only on history to predict this future latent representation. The predicted future latent representation serves as planning context and guides trajectory generation and selection. PlanWAM achieves 93.8 PDMS / 90.9 EPDMS on NAVSIM-v1/v2 navtest and reaches 38.7 HD-Score on closed-loop HUGSIM in a zero-shot setting, demonstrating leading planning performance across both open-loop and closed-loop evaluations. Extensive experiments further demonstrate that planning-shaped future representations provide an effective and deployable form of foresight for world-action models.
Jinchang Xu, Hongda Yu, Fengwei Dong +9
Southeast University · NIO · Nanyang Technological University +1
Existing latent world models are typically optimized for future predictability, yet the resulting representations are not necessarily useful for planning in autonomous driving. Predictions are commonly used for pretraining or auxiliary supervision rather than as direct conditioning signals for trajectory generation. We propose ForeDrive, which learns a planning-relevant latent representation and couples it asymmetrically to a Diffusion Transformer (DiT) planner. The planner consumes multi-horizon latent future representations learned with a JEPA-style world model; planning gradients update the shared online encoder, while stop-gradient routing trains the latent predictor with forecasting losses only. Because predicted futures have varying reliability across horizons and BEV trajectories are misaligned with image tokens, we use gated visual fusion, future-status injection, and Trajectory-Adaptive Bias (TAB) to inject future latents as guidance without overriding the current observation. Trained with pure imitation learning and using only the current front-view image as visual input at inference, ForeDrive attains 89.9 PDMS on NAVSIM v1 and 90.0 one-stage EPDMS on NAVSIM v2, without reinforcement learning or an external trajectory scorer.
Sinuo Wang, Zichong Gu, Yuhan Huang +10
Huazhong University of Science and Technology · Shanghai Zaofu Intelligent Technology Co., Ltd. · Tongji University