Organizations: The Hong Kong University of Science and Technology. · Horizon Robotics. · The Chinese University of Hong Kong. · Nanjing University of Posts and Telecommunications. · Nankai University.
Driving world models learn rich predictive representations of the surrounding environment from visual observations, yet accurate visual prediction does not necessarily translate into effective trajectory planning. We argue that a key bottleneck lies in the mismatch between visual world states and raw geometric trajectories, which may limit the planner's ability to exploit action-relevant semantics encoded by the world model. To address this issue, we propose World-Model Alignment for Latent Trajectories (WALT), which learns a compact generative trajectory latent space by transferring information from a frozen pretrained driving world model without modifying the world model itself. Rather than directly generating raw waypoints, WALT maps them into compact representations through a dual-branch trajectory autoencoder and transfers semantic knowledge from the frozen visual world model into this trajectory space, encouraging the learned action representation to capture scene-level cues relevant to future motion and planning. Beyond our proposed formulation, we systematically study latent learning based on Joint-Embedding Predictive Architectures (JEPA) and feature alignment following Representation Alignment (REPA) to investigate how trajectory-only representation learning affects downstream planning. We evaluate WALT on the NAVSIM benchmarks. Relative to the raw-waypoint baseline, WALT improves PDMS from 89.4 to 89.8 on NAVSIMv1 and EPDMS from 87.3 to 87.9 on NAVSIMv2 while reducing trajectory planner FLOPs by 30.5%. These results suggest that preserving world representations while extracting action-relevant information provides an effective interface for world-model-based trajectory planning.
Figures & tables
Fig. 2 : WALT pipeline. In Stage 1, we train the dual-branch trajectory autoencoder with a frozen world model. The semantic branch feature is aligned with world state tokens with LWALT , while the geometric branch preserves reconstruction details and jointly trained with Lrecon . In Stage 2, we freeze the tokenizer and train the trajectory planner head of the frozen world model to generate latents, which are decoded into waypoints for future trajectory prediction.
Method
Venue
NC
DAC
EP
TTC
C
PDMS
Human
–
100
100
87.5
100
99.9
94.8
LAW [ 27 ]
ICLR’25
96.4
95.4
81.7
88.7
99.9
84.6
DrivingGPT [ 11 ]
ICCV’25
98.9
90.7
79.7
94.9
95.6
82.4
World4Drive [ 4 ]
ICCV’25
97.4
94.3
79.9
92.8
100
85.1
Epona [ 3 ]
ICCV’25
97.9
95.1
80.4
93.8
99.9
86.2
PWM [ 6 ]
NeurIPS’25
98.6
95.9
81.8
95.4
100
88.1
TABLE I : Reported results on NAVSIMv1 [ 23 ] . All metrics are on a 0–100 scale. Higher is better. Bold and underlined values indicate the best and second-best distinct scores among learned methods, respectively. These results include different model and training configurations.
Method
NC
DAC
DDC
TLC
EP
TTC
LK
HC
EC
EPDMS
Human
100
100
99.8
100
87.4
100
100
98.1
90.1
94.5
DriveVLA-W0 [ 5 ]
98.4
95.2
99.4
99.9
86.6
97.9
97.8
98.3
82.7
86.9
ARTEMIS ∗ [ 45 ]
98.3
95.1
98.6
99.8
81.5
97.4
96.5
98.3
–
83.1
PRIX ∗ [ 44 ]
98.0
85.6
99.5
99.8
87.4
97.2
97.1
98.3
87.6
84.2
DriveWorld-VLA [ 52 ]
98.6
99.1
99.6
99.8
87.4
97.9
97.0
97.8
78.6
86.8
EponaV2 [ 31 ] (w/o RL)
98.4
96.7
99.6
99.9
87.6
98.1
98.0
98.1
68.3
87.3
TABLE II : Comparison on NAVSIMv2 [ 24 ] . All metrics are on a 0–100 scale, where higher is better. ∗ denotes results reported using the original evaluator before the human-penalty aggregation update. Bold and underlined values indicate the best and second-best distinct scores among unstarred learned methods.
Fig. 3 : Qualitative trajectories and world-model alignment. Columns show cases for braking, straight driving, turn, and curve. From top to bottom: front-view images, world–trajectory correspondence maps, and predicted trajectories (orange) with ground truth (black). The maps visualize cosine similarity between projected semantic tokens from planner-generated latents and frozen visual world-model tokens, with warmer colors indicating higher similarity.
Fig. 4 : Trajectory representation analysis on NAVSIMv1. Columns compare raw waypoints, JEPA-Traj semantic latents, and WALT semantic latents on identical samples. From top to bottom: t-SNE embeddings, within-class and between-class cosine-similarity distributions, and class-pair mean-similarity matrices. Learned representations use only their semantic components.
Method
NC
DAC
EP
TTC
C
PDMS
Raw waypoints baseline
98.62
97.32
83.60
95.26
99.93
89.42
+ trajectory AE (w/o LWALT )
98.62
97.38
83.63
95.32
99.95
89.48
JEPA-Traj
98.55
97.37
82.85
95.91
100.00
89.46
REPA-Traj
98.48
97.46
83.79
95.11
99.98
89.49
WALT
99.11
96.99
83.63
96.34
100.00
89.83
TABLE III : Action-side representation comparison on NAVSIMv1 with the same frozen world-model backbone. Variants differ in generation target and representation supervision. All metrics are on a 0–100 scale. Higher is better.
Representation
Compression Ratio
GFLOPs ↓
Raw waypoint generation
–
297.47
WALT latent generation
4 ×
206.88
TABLE IV : Trajectory-generation cost for one denoising step under matched world-model conditioning. The latent-generation cost includes trajectory decoding, and the frozen world-model computation is excluded.