Generative world-action models (WAMs) jointly generate future video and vehicle actions, while their action branches remain primarily optimized by expert imitation. Yet imitation provides no explicit closed-loop geometric verdict for generated trajectories, making verification important during both training and deployment. Closed-loop evaluators can check collision and drivable-area violations, but require privileged scene state unavailable at deployment. Existing approaches often close this gap by learning a verifier from sensor features. For these geometric checks, the rule itself is explicit. For example, collision is determined by whether the rolled-out ego footprint overlaps occupied vehicle space. What is unavailable at deployment is the scene state needed to apply the rule. We introduce DriveReferee, which uses a learned geometry readout to predict the scene representation from camera observations and executes the geometric safety rule directly rather than learning it. The resulting analytic referee evaluates collision and drivable-area safety from a scene state and candidate trajectory. During training, it scores self-sampled trajectories on ground-truth state and distills the resulting preferences into the WAM policy. At deployment, the same referee evaluates generated trajectories on this predicted state and selects a safer alternative when needed. The analytic referee requires no verdict-specific training, and its decisions follow an explicit geometric rule. Under matched candidates and inference budgets, it matches or outperforms all learned-verifier and heuristic baselines. Given the same predicted state and trajectory, learning the verdict provides no measurable downstream gain despite requiring tens of thousands of evaluator-labeled training examples. On the full NAVSIM navtest, DriveReferee reaches 92.02 PDMS with single-camera visual input and no external training data.
Figures & tables
Fig. 1 : DriveReferee recovers the state, instead of the rule. The geometric safety verdict is a computation whose input, not whose logic, is missing at deployment.
Fig. 2 : DriveReferee overview. The policy πθ (frozen reasoner, trained generator) generates future video V^k and candidate plans τk ; the readout gϕ predicts the BEV scene state M^=(M^drv,M^veh) , containing drivable-area and vehicle-occupancy maps, from frozen visual features and ego poses P−H:0 . The analytic referee R(τ,M) , with no learned parameters, is placed twice: at training (green) on Mgt to form preference pairs (τw,τl) distilled into the generator ( Eq. 3 ); at deployment (blue) on M^ to gate the default plan and, on alarm, rank K−1 further samples on the same map.
Method
Input
Video
NC ↑
DAC ↑
TTC ↑
Comf. ↑
EP ↑
PDMS ↑
EPDMS ↑
DrivingGPT [ 27 ]
1 × C
✓
98.9
90.7
94.9
95.6
79.7
82.4
—
WoTE [ 9 ]
3 × C+L
✗
98.5
96.8
94.9
99.9
81.9
88.3
—
DriveVLA-W0 [ 28 ]
1 × C
✓ †
98.7
99.1
95.3
99.3
83.3
90.2
86.1
PWM [ 29 ]
1 × C
✓
98.6
95.9
95.4
100.0
81.8
88.1
—
DriveLaW ‡ [ 30 ]
1 × C
✓ †
99.0
97.1
96.7
100.0
81.3
89.1
—
CoPhy § [ 10 ]
C+L
✗
99.0
98.2
96.8
100.0
85.3
91.4
86.1
TABLE I : Comparison with representative world-model-based planners on navtest (12,146 scenes). N× C denotes N cameras and +L denotes LiDAR. Video indicates whether video generation is part of the model. PDMS follows NAVSIM v1 and EPDMS follows v2. NC, DAC, TTC, Comf., and EP denote no at-fault collisions, drivable-area compliance, time-to-collision, comfort, and ego progress, respectively. Higher is better; “—” denotes unreported results.
Fig. 3 : Matched-budget verdict-source comparison. Per-scene paired Δ EPDMS relative to the imitation-only base on navtest ( n=12,146 ). All sources use the same pre-sampled K=2 candidates and are calibrated to an average sampling budget of about 1.35 × . Error bars are bootstrap 95% confidence intervals. Numbers at right report replaced plans.
Fig. 4 : Verdict-supervision scaling in the same-map setting. Only the amount of verdict-training data is varied; 100% corresponds to 66,385 training candidates. Points show the mean over two log-level draws, with bootstrap 95% confidence intervals. The analytic referee uses no verdict-training labels.
Fig. 5 : Deployment-time verification on self-predicted maps (base policy, K=2 ). (a) One intervention example. At the flagged state, the default plan crosses the predicted drivable boundary by 0.40 m, while the referee-selected plan keeps 1.26 m clearance. The signed-distance curve and projected boundaries show the same failure in BEV and video space; the referee’s clearance margin is clipped to zero once the boundary is crossed. (b) Four additional replacements found by automatic screening. In each case, the default plan leaves the ground-truth drivable area, while the selected plan remains inside both the predicted and ground-truth boundaries (local IoU ≥0.90 ).
Variant
Δ EPDMS
Preference objective
β=0
+0.56
full preference objective
+0.92
Winner construction
nearest to expert
+0.63
largest safety margin (ours)
+0.92
TABLE II : Training design ablations. Each block changes one design choice while keeping the remaining settings fixed. Values are paired Δ EPDMS over the imitation-only base on navtest ( n=12,146 ).
Method
FVD ↓
LPIPS ↓
PSNR ↑
Frames
Resolution
Base (imitation only)
23.90
0.4166
18.75
9 @ 2Hz
832 × 480
+ referee distillation
24.25
0.4126
18.92
9 @ 2Hz
832 × 480
DriveDreamer-Policy [ 4 ]
53.59
—
—
9
144 × 256
PWM [ 29 ]
85.95
—
—
10
128 × 224
ForgeDrive [ 35 ]
69.2
—
—
8 @ 2Hz
512 × 1024
DrivingGPT [ 27 ]
142.6
—
—
12
288 × 512
TABLE III : Video generation quality. Our rows use the same navtest scenes, protocol, and sampling seeds. Prior-work FVD values are for context only because evaluation settings differ.
World action models (WAMs) have recently gained increasing attention as a framework for jointly modeling scene evolution and ego actions in autonomous driving. Most existing WAMs learn scene dynamics in pixel space by combining a video-generation backbone for future-observation prediction with an action head for ego-trajectory prediction. Pixels, however, provide only an indirect representation of these dynamics: they entangle geometry and motion with appearance, texture, and illumination, forcing the model to infer three-dimensional transformations from two-dimensional observations. We argue that point-based geometry provides a more natural state space for driving. It explicitly captures spatial structure and both rigid and non-rigid scene dynamics while remaining aligned with the 3D space of driving actions. Building on this insight, we introduce GeoWAM, a visual geometry world action model for autonomous driving. Rather than predicting future images, GeoWAM is pretrained to forecast future scene geometry, yielding representations that jointly encode spatial structure and temporal evolution. A geometry-conditioned action head then leverages these learned geometric dynamics to predict future ego-trajectories. Extensive experiments show that GeoWAM outperforms image-based alternatives, achieving a combined EPDMS of 36.6 on navhard without PDMS supervision and strong zero-shot generalization to nuScenes, with a collision rate of 0.24%. Scaling geometry pretraining with unlabeled data further improves performance, increasing the navhard score by 8.2% to 39.6 and strengthening zero-shot transfer to nuScenes, where the collision rate is reduced by 50% to 0.12%. Together, these results establish geometry as an effective state representation for autonomous driving and geometry pretraining as a general, scalable strategy for downstream planning.
Recently, world-action models (WAM) have emerged to bridge vision-language-action (VLA) models and world models, unifying their reasoning and instruction-following capabilities and spatio-temporal world modeling. However, existing WAM approaches often focus on modeling 2D appearance or latent representations, with limited geometric grounding-an essential element for embodied systems operating in the physical world. We present DriveDreamer-Policy, a unified driving world-action model that integrates depth generation, future video generation, and motion planning within a single modular architecture. The model employs a large language model to process language instructions, multi-view images, and actions, followed by three lightweight generators that produce depth, future video, and actions. By learning a geometry-aware world representation and using it to guide both future prediction and planning within a unified framework, the proposed model produces more coherent imagined futures and more informed driving actions, while maintaining modularity and controllable latency. Experiments on the Navsim v1 and v2 benchmarks demonstrate that DriveDreamer-Policy achieves strong performance on both closed-loop planning and world generation tasks. In particular, our model reaches 89.2 PDMS on Navsim v1 and 88.7 EPDMS on Navsim v2, outperforming existing world-model-based approaches while producing higher-quality future video and depth predictions. Ablation studies further show that explicit depth learning provides complementary benefits to video imagination and improves planning robustness.
Generalization is a central challenge in autonomous driving, as real-world deployment requires robust performance under unseen scenarios, sensor domains, and environmental conditions. Recent world-model-based planning methods have shown strong capabilities in scene understanding and multi-modal future prediction, yet their generalization across datasets and sensor configurations remains limited. In addition, their loosely coupled planning paradigm often leads to poor video-trajectory consistency during visual imagination. To overcome these limitations, we propose DriveVA, a novel autonomous driving world model that jointly decodes future visual forecasts and action sequences in a shared latent generative process. DriveVA inherits rich priors on motion dynamics and physical plausibility from well-pretrained large-scale video generation models to capture continuous spatiotemporal evolution and causal interaction patterns. To this end, DriveVA employs a DiT-based decoder to jointly predict future action sequences (trajectories) and videos, enabling tighter alignment between planning and scene evolution. We also introduce a video continuation strategy to strengthen long-duration rollout consistency. DriveVA achieves an impressive PDM-based planning performance of 90.9 PDM score on the NAVSIM benchmark. Extensive experiments also demonstrate the zero-shot capability and cross-domain generalization of DriveVA, which reduces average L2 error and collision rate by 78.9% and 83.3% on nuScenes and 52.5% and 52.4% on the Bench2Drive built on CARLA v2 compared with the state-of-the-art world-model-based planner.
Mengmeng Liu, Diankun Zhang, Jiuming Liu +7
University of Twente, The Netherlands · Xiaomi EV, China · University of Cambridge, United Kingdom +1