World-action models have recently improved autonomous driving by jointly learning future scene prediction and trajectory generation. Most existing approaches model the future primarily through RGB appearance, and recent works have begun to incorporate geometric prediction to improve spatial understanding. However, dense geometry describes the spatial layout of the entire scene without indicating which parts are most relevant to the ego vehicle's action. For driving, the model must also identify and anticipate where it can safely move and which regions may pose collision risks. Jointly modeling action-relevant regions and future geometry can provide the policy with both driving-relevant cues and their corresponding spatial structure. We therefore propose AffordDrive3D, an affordance- and geometry-aware world-action model that jointly learns future action-relevant regions and spatial structure. In order to capture the scene semantics and driving context needed for driving affordance prediction, we build AffordDrive3D on a VLM backbone to forecast drivable areas and collision-critical regions that directly affect ego motion, while predicting future geometry from RGB world-model latents. On NAVSIM, AffordDrive3D achieves state-of-the-art performance with 91.3 PDMS and 89.9 EPDMS, demonstrating the effectiveness of jointly modeling future affordances and geometry for trajectory planning.
Figures & tables
Figure 1: AffordDrive3D vs. RGB-based world-action models. Existing RGB-based world-action models primarily capture future scene evolution through appearance prediction. AffordDrive3D additionally predicts future geometry to model how spatial structure evolves, alongside driving affordances that identify spatial regions critical to the ego vehicle’s motion. Together, these predictions align future world state modeling more closely with planning objectives and improve trajectory prediction.
Figure 2: Framework of AffordDrive3D. Multi-view images, the navigation command, and the ego state are encoded by a shared VLM backbone together with learnable affordance, RGB world, geometry, and action tokens. The auxiliary branches (dashed box) forecast driving affordances, future RGB, and RGB-conditioned depth to shape the shared representation during training, while only the action branch is used at inference.
Method
World Sup.
NC ↑
DAC ↑
TTC ↑
Comf. ↑
EP ↑
PDMS ↑
DRAMA
-
98.0
93.1
94.8
100.0
80.1
85.5
Hydra-MDP
-
98.3
96.0
94.6
100.0
78.7
86.5
DiffusionDrive
-
98.2
96.2
94.7
100.0
82.2
88.1
AutoVLA
-
98.4
95.6
98.0
99.9
81.9
89.1
ReCogDrive
-
97.9
97.3
94.9
100.0
87.3
90.8
WoTE
BEV
98.5
96.8
94.9
99.9
81.9
88.3
Table 1: Comparison with baselines on the NAVSIM v1 benchmark. Best results are bolded .
Model
NC ↑
DAC ↑
DDC ↑
TLC ↑
EP ↑
TTC ↑
LK ↑
HC ↑
EC ↑
EPDMS ↑
DiffusionDriveV2
97.7
96.6
99.2
99.8
88.9
97.2
96.0
97.8
91.0
85.5
DriveSuprim
98.4
98.6
99.6
99.8
90.5
97.8
97.0
98.3
78.6
87.1
DriveVLA-W0
98.5
99.1
98.0
99.7
86.4
98.1
93.2
97.9
58.9
86.1
DriveDreamer-Policy
98.4
97.1
99.5
99.9
87.9
97.7
97.6
98.3
79.4
88.7
ExploreVLA
98.8
96.2
99.6
99.8
87.1
98.2
97.8
98.3
86.8
88.8
Ours
98.4
98.2
99.7
99.9
87.9
97.8
97.4
98.3
80.4
89.9
Table 2: Comparison with baselines on NAVSIM v2 with extended metrics. Best results are bolded .
Setting
Aff.
RGB
Geo.
NC ↑
DAC ↑
TTC ↑
Comf. ↑
EP ↑
PDMS ↑
Action only
–
–
–
97.9
96.4
94.3
100.0
82.6
88.1
+ Affordance
✓
–
–
98.3
97.1
95.1
100.0
83.5
89.2
+ RGB
–
✓
–
98.3
96.8
95.3
100.0
83.0
88.9
+ Geometry
–
–
✓
98.2
97.1
95.2
100.0
83.2
89.1
Full model
✓
✓
✓
98.4
97.6
95.2
100.0
83.7
89.6
Table 3: Ablation study of affordance, RGB, and geometry future modeling. For the + Geometry setting, the RGB branch provides latent context for depth prediction, but its gradients are stopped from propagating to the shared backbone. Results are reported without RL training.
Figure 3: Qualitative results of AffordDrive3D in two challenging scenarios. (a) The narrow curvy-road scenario: AffordDrive3D forecasts the road structure and remains on-road, while both baselines deviate from the lane. (b) The leading-vehicle braking scenario: AffordDrive3D captures the potential collision risk from the leading vehicle together with RGB-consistent future geometry, maintaining a safe trajectory while both baselines collide. In all BEV visualizations, highlighted boxes show the ego footprint at the violation state obtained from the NAVSIM rollout, while other-agent boxes show their positions at t=0 .
Table 7
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Example of ground-truth affordance targets at t=0 – 4 s.
Figure 5: Additional world-modeling and planning visualizations. For each scenario, we show AffordDrive3D’s predicted future RGB, current and future depth, and current and future drivable-area and collision-critical masks overlaid on the corresponding RGB views. The left, front, and right camera views are spatially concatenated, and the right panels compare the planned trajectories of AffordDrive3D and the RGB-based DriveVLA-W0 ( Li et al., 2026b ) .
Figure 6: Additional planning comparisons. Each example shows the three input camera views and the BEV trajectories predicted by AffordDrive3D, the RGB-based DriveVLA-W0 ( Li et al., 2026b ) , and the action-only VLA baseline.
Autonomous driving requires both safe and efficient planning decisions in dynamic 3D environments. Although recent Vision/Video-Action models learn policies directly from visual observations and scale well with advances in vision transformers and large-scale training data, they often lack explicit geometric grounding and future-aware spatial guidance, limiting their ability to balance collision avoidance and driving progress. In this work, we propose GeoWorldAD, a geometry world action model that grounds trajectory planning in ego-aligned 3D space and anticipates short-horizon scene evolution with latent future geometry tokens. Present geometry provides essential spatial constraints for safe planning, while future geometry reveals how surrounding agents and ego-centric free space may evolve, reducing overly conservative decisions without sacrificing safety. To efficiently exploit these geometric cues, GeoWorldAD progressively aggregates multi-scale present geometry and latent future geometry through iterative trajectory refinement. Experiments on NAVSIM v1 and v2 demonstrate state-of-the-art performance, highlighting the effectiveness of explicit 3D geometry grounding and future geometry world modeling for safe and efficient autonomous driving.
Songyan Zhang, Jinyuan Tian, Hanbing Li +9
Nanyang Technological University · Xiaomi EV · Zhejiang University
Data scaling plays a pivotal role in the pursuit of general intelligence. However, the prevailing perception-planning paradigm in autonomous driving relies heavily on expensive manual annotations to supervise trajectory planning, which severely limits its scalability. Conversely, although existing perception-free driving world models achieve impressive driving performance, their real-world reasoning ability for planning is solely built on next frame image forecasting. Due to the lack of enough supervision, these models often struggle with comprehensive scene understanding, resulting in unsatisfactory trajectory planning. In this paper, we propose EponaV2, a novel paradigm of driving world models, which achieves high-quality planning with comprehensive future reasoning. Inspired by how human drivers anticipate 3D geometry and semantics, we train our model to forecast more comprehensive future representations, which can be additionally decoded to future geometry and semantic maps. Extracting the 3D and semantic modalities enables our model to deeply understand the surrounding environment, and the future prediction task significantly enhances the real-world reasoning capabilities of EponaV2, ultimately leading to improved trajectory planning. Moreover, inspired by the training recipe of Large Language Models (LLMs), we introduce a flow matching group relative policy optimization mechanism to further improve planning accuracy. The state-of-the-art (SOTA) performances of EponaV2 among perception-free models on three NAVSIM benchmarks (+1.3PDMS, +5.5EPDMS) demonstrate the effectiveness of our methods.
Jiawei Xu, Zhizhou Zhong, Zhijian Shu +8
1PCA Lab, VCIP, College of Computer Science, Nankai University · 2Horizon Robotics · 3HKUST +4
Driving world models serve as a pivotal technology for autonomous driving by simulating environmental dynamics. However, existing approaches predominantly focus on future scene generation, often overlooking comprehensive 3D scene understanding. Conversely, while Large Language Models (LLMs) demonstrate impressive reasoning capabilities, they lack the capacity to predict future geometric evolution, creating a significant disparity between semantic interpretation and physical simulation. To bridge this gap, we propose HERMES++, a unified driving world model that integrates 3D scene understanding and future geometry prediction within a single framework. Our approach addresses the distinct requirements of these tasks through synergistic designs. First, a BEV representation consolidates multi-view spatial information into a structure compatible with LLMs. Second, we introduce LLM-enhanced world queries to facilitate knowledge transfer from the understanding branch. Third, a Current-to-Future Link is designed to bridge the temporal gap, conditioning geometric evolution on semantic context. Finally, to enforce structural integrity, we employ a Joint Geometric Optimization strategy that integrates explicit geometric constraints with implicit latent regularization to align internal representations with geometry-aware priors. Extensive evaluations on multiple benchmarks validate the effectiveness of our method. HERMES++ achieves strong performance, outperforming specialist approaches in both future point cloud prediction and 3D scene understanding tasks. The model and code will be publicly released at https://github.com/H-EmbodVis/HERMESV2.
Xin Zhou, Dingkang Liang, Xiwu Chen +4
Huazhong University of Science and Technology · Mach Drive · University of Hong Kong