World Model Evaluation

Follow this topic Ask about this topic

World Model Evaluation studies whether a model captures how an environment changes, especially when an agent acts or observations are missing. Approaches include open-loop prediction metrics, action-conditioned rollouts, closed-loop policy tests, physics-grounded diagnostics, and human or vision-language model judging. Benchmarks span robotic manipulation such as LIBERO and RoboTwin, driving such as NuPlan, and interactive GUI, web, and game environments. A central challenge is that visual realism and one-step accuracy often fail to predict physical consistency, long-term reliability, or planning success, so evaluation remains fragmented across tasks and horizons.

Latest papers

212 papers49 papers in the last four weeks, up 75% on the four weeks before. 0.3% of all new papers.

OmniNWM: Omniscient Driving Navigation World Models
Oct 21, 2025Bohan Li, Zhuang Ma, Dalong Du +10Action-Conditioned Video Generation World Model Learning
Shanghai Jiao Tong University · Ningbo Institute of Digital Twin, Eastern Institute of Technology, Ningbo, China · PhiGent +3
A Comprehensive Survey on World Models for Embodied AI
Oct 19, 2025Xinqing Li, Xin He, Le Zhang +3World Model Learning World Models for Robotics
College of Computer Science and the Academy for Advanced Interdisciplinary Studies, Nankai University, Tianjin 300350, China

Oct 21, 2025J/K move · Enter open · S save

OmniNWM: Omniscient Driving Navigation World Models

Bohan Li, Zhuang Ma, Dalong Du, Baorui Peng+9

Shanghai Jiao Tong University · Ningbo Institute of Digital Twin, Eastern Institute of Technology, Ningbo, China · PhiGent+3

Action-Conditioned Video Generation World Model Learning World Model-Based Planning Autonomous Driving World Models for Autonomous Driving World Model Evaluation

Open paper AskPDF arXiv

Autonomous driving world models are expected to work effectively across three core dimensions: state, action, and reward. However, existing methods are typically restricted to fragmented modality modeling, short-horizon drift, and imprecise action control, while lacking intrinsic mechanisms for policy evaluation. In this paper, we introduce OmniNWM, an Omniscient panoramic Navigation World Model that addresses all three dimensions within a consistent probabilistic framework. For State, OmniNWM generates panoramic videos of RGB, semantics, metric depth, and 3D occupancy, ensuring pixel-level alignment across modalities with joint distribution modeling. To mitigate autoregressive exposure bias, we propose a structured panoramic forcing strategy to stabilize long-horizon generation via stochastic manifold thickening. For Action, we introduce canonical geometric action encoding with normalized panoramic Plücker ray-maps. This representation decouples motion dynamics from sensor intrinsics, enabling precise, zero-shot trajectory control across heterogeneous datasets and camera configurations. For Reward, we derive intrinsic occupancy-grounded dense rewards directly from generated 3D volumes, establishing a reliable closed-loop simulation cycle for evaluating diverse planning agents. Extensive experiments demonstrate that OmniNWM achieves SOTA performance in generation fidelity and control precision, with remarkable zero-shot robustness to novel scenes on NuPlan and in-house datasets with distinct camera rigs. Project page is available at https://arlo0o.github.io/OmniNWM/.

Latest papers

OmniNWM: Omniscient Driving Navigation World Models

A Comprehensive Survey on World Models for Embodied AI