EditWM: Event-Decomposed World Modeling with Incremental Correction for End-to-End Autonomous Driving
Authors: Junjie Yang, Qingwei Zeng, Youyou Li, Zicheng Ding, Ziyi Shi, Shuqi Shen, Hongliang Lu, Hai Yang
Organizations: The Hong Kong University of Science and Technology · Southern University of Science and Technology · The Chinese University of Hong Kong, Shenzhen
World models support autonomous driving by predicting the scene evolution associated with candidate trajectories. Driving dynamics differ in predictability, motivating a distinction between regular evolution and event-induced deviations that call for selective correction. We propose EditWM, a world model that decomposes future prediction into normal evolution and event-driven incremental correction in compact visual feature space. A trajectory-conditioned normal predictor provides the base forecast and is then frozen for correction learning. A correction decoder compares this forecast with observation history and planned actions, producing a bounded feature update whose contribution is regulated by a learned gate. The corrected future features condition trajectory scoring through candidate-specific cross-attention, linking world modeling to plan selection. At inference, EditWM uses only past and current observations, ego state, and candidate trajectories. Across all 12,146 NAVSIM navtest scenes, expert-trajectory-conditioned evaluation shows a 5.35% reduction in future-feature MSE over Normal, with improvements in 83.54% of scenes. The system achieves 91.05 EPDMS on a 100-point scale using the official EPDMS evaluator. These results demonstrate improved future-feature prediction and competitive trajectory selection when corrected future representations are integrated into planning.
Figures & tables
Fig. 1: The core idea of EditWM: predict normal evolution and correct what it misses. Left: a cut-in illustrates feature changes left unexplained by a normal forecast. Right: EditWM retains the normal forecast and selectively refines it with a gated feature correction. Orange highlights the unexplained change and its correction. The road scenes schematically illustrate latent feature prediction and correction.
Fig. 2: Overview of the EditWM world model and its integration with trajectory scoring. (a) Multi-view history is encoded with DINOv2, while a trajectory generator produces candidate plans. (b) For each candidate, a normal dynamics predictor produces future features, and a correction decoder predicts a gated residual from normal prediction context and history–action memory. The normal forecast and gated incremental correction together form the EditWM future representation. The frozen normal predictor supplies feature-residual supervision for correction and gating. (c) Each candidate and its corresponding predicted future features are fused for trajectory scoring and selection. World-model parameters are shared across candidates; colored token blocks depict latent features.
Method
Backbone
NC
DAC
DDC
TL
EP
TTC
LK
HC
EC
EPDMS
Hydra-MDP++ [ 24 ]
ResNet-34
98.4
98.0
99.4
99.8
87.5
97.7
95.3
98.3
77.4
85.1 ∗
DiffusionDrive [ 21 ]
ResNet-34
98.2
96.2
99.5
99.8
87.4
97.4
97.0
98.3
87.8
84.8 ∗
WoTE [ 9 ]
ResNet-34
98.5
96.8
98.8
99.8
86.1
97.9
95.5
98.3
82.9
84.2
iPad [ 25 ]
ResNet-34
98.7
97.8
99.1
99.8
84.0
98.0
96.0
98.0
68.2
84.1
DriveSuprim [ 26 ]
ResNet-34
97.8
97.9
99.5
99.9
90.6
97.1
96.6
98.3
77.9
86.0 ∗
DiffusionDriveV2 [ 27 ]
ResNet-34
97.7
96.6
99.2
99.8
88.9
97.2
96.0
97.8
91.0
87.5
TABLE I: EPDMS leaderboard and component scores (0–100; higher is better).
Method
NC
DAC
DDC
TL
EP
TTC
LK
HC
EC
EPDMS
w/o Future Features
97.55
98.67
99.59
99.70
87.34
96.80
94.75
98.21
96.36
90.84
w/o EditWM (Normal-only)
97.34
98.53
99.53
99.72
87.08
96.65
94.71
98.20
96.36
90.51
Ours
97.8
98.8
99.6
99.9
87.5
96.9
94.8
98.3
96.6
91.05
Oracle
99.79
99.69
99.79
99.97
95.56
99.77
99.16
98.35
97.99
97.73
TABLE II: Future-feature and trajectory-selection comparison on the same K=96 candidate pool (0–100; higher is better).
Fig. 3: Future DINO feature prediction on all 12,146 navtest scenes, conditioned on the expert trajectory. (a) Per-scene MSE reduction, defined as Normal minus EditWM; negative values indicate degradation and are retained. (b) Mean MSE for five equally sized groups ordered by Normal’s prediction error. (c) Mean MSE for each camera view. Percentage annotations in (b) and (c) give the relative reduction in group-mean MSE. EditWM improves 83.54% of scenes and reduces overall MSE by 5.35%.
Fig. 4: Selected qualitative planning comparisons using previously evaluated checkpoints. Each row presents one scene, with a camera view followed by Ours, Normal-only, and WoTE. The black dotted line is the human GT trajectory; colored lines show the actual selected trajectories. BEV panels share the same map, current objects, and spatial extent within each row. EPDMS and its displayed sub-scores use a 0–100 scale. GT ADE is the mean position error over eight corresponding future time steps. Future MSE measures expert-trajectory-conditioned future DINO feature prediction for EditWM and Normal.
Metric
WoTE (Normal)
EditWM
Validation MSE ↓
0.32490
0.29693
navtest MSE ↓
0.39352
0.37246
navtest cosine ↑
0.95078
0.95355
TABLE III: Future feature prediction under the expert trajectory.
World models in end-to-end autonomous driving predict future scene evolution to provide foresight for trajectory planning. Existing methods mainly study how to predict the future and how to use it, but less often ask which future representation is actually most useful for planning. To this end, we propose PlanWAM, a Planning-Shaped World Action Model. The key idea is to let the planning task shape the future-state representation, so that it retains the information most useful for planning. A latent world model then predicts this planning-shaped future latent representation from historical observations and uses it for planning, enabling foresighted planning. Specifically, we first use a Temporal Register Pyramid to compress multi-frame historical information in a recency-aware manner, learning a compact history representation oriented toward future reasoning and planning. We then introduce a privileged future posterior branch that observes ground-truth future frames, and shape its future latent representation with trajectory-planning objectives to obtain a planning-shaped future latent representation. Hindsight-to-Foresight Distillation trains a prior branch that depends only on history to predict this future latent representation. The predicted future latent representation serves as planning context and guides trajectory generation and selection. PlanWAM achieves 93.8 PDMS / 90.9 EPDMS on NAVSIM-v1/v2 navtest and reaches 38.7 HD-Score on closed-loop HUGSIM in a zero-shot setting, demonstrating leading planning performance across both open-loop and closed-loop evaluations. Extensive experiments further demonstrate that planning-shaped future representations provide an effective and deployable form of foresight for world-action models.
Jinchang Xu, Hongda Yu, Fengwei Dong +9
Southeast University · NIO · Nanyang Technological University +1
World models for autonomous driving should anticipate the consequences that matter for driving decisions rather than reconstruct every detail of future scenes. However, predictive representations alone are not guaranteed to distinguish candidate actions that lead to different planning outcomes.We propose DA-WAM, a decision-aligned world action model that couples a Joint-Embedding Predictive Architecture (JEPA) with candidate-level trajectory scoring. During joint training, the JEPA objective grounds future representations in observed scene dynamics, while the planning objectives encourage these representations to capture distinctions relevant to safety and utility. An action-conditioned predictor generates a distinct future latent for each candidate trajectory, and a multi-head trajectory scorer uses the corresponding latent to estimate planning factors and overall utility. In addition, safety-critical post-training with hard negatives further improves discrimination between geometrically similar trajectories with different safety outcomes. DA-WAM achieves state-of-the-art performance among learned camera-based planners on NAVSIM-v1, NAVSIM-v2, and Bench2Drive. Ablation studies demonstrate the effectiveness of each component in driving decision-making.
Ruiguo Zhong, Xiaolong Chen, Benshan Ma +5
The Hong Kong University of Science and Technology (Guangzhou) · Leapmotor · The Hong Kong University of Science and Technology
World models offer a promising paradigm for autonomous driving by predicting how traffic scenes may evolve and using such predictions to support action generation. However, existing approaches either separate future prediction from action generation or jointly predict them at the same temporal scale, making it difficult to simultaneously achieve long-horizon anticipation and responsive, observation-grounded decision making. We present Drive-HWM, a hierarchical slow--fast world modeling framework that organizes future representation prediction and action generation at complementary temporal scales. The slow world model predicts multi-step future representations to capture extended scene evolution. To explicitly model the abundant motion dynamics in driving environments, we introduce Dynamic-Aware Latents learned through optical-flow prediction. Guided by these future representations, the fast model uses a lightweight multimodal backbone and an autoregressive expert to jointly predict the next frame and the immediate action from the latest observation. Next-frame prediction encourages the fast model to capture imminent scene evolution, while one-step action generation allows decisions to be continuously updated as new observations arrive. Extensive experiments on NAVSIM v1 and v2 demonstrate the strong driving performance of Drive-HWM. Comprehensive ablation studies further validate the effectiveness of the hierarchical slow--fast design, dynamics-aware future representations, and joint next-frame and action prediction.
Zhaoxin Fan, Tianbao Zhang, Wenjun Wu +5
School of Artificial Intelligence, Beihang University, Beijing, China · Shanghai Jiao Tong University, Shanghai, China · Dim12 AI +3