World Model Evaluation

Follow this topic Ask about this topic

World Model Evaluation studies whether a model captures how an environment changes, especially when an agent acts or observations are missing. Approaches include open-loop prediction metrics, action-conditioned rollouts, closed-loop policy tests, physics-grounded diagnostics, and human or vision-language model judging. Benchmarks span robotic manipulation such as LIBERO and RoboTwin, driving such as NuPlan, and interactive GUI, web, and game environments. A central challenge is that visual realism and one-step accuracy often fail to predict physical consistency, long-term reliability, or planning success, so evaluation remains fragmented across tasks and horizons.

Latest papers

212 papers49 papers in the last four weeks, up 75% on the four weeks before. 0.3% of all new papers.

Understanding the limits of latent world models
Oct 27, 2026Rodrigo Gonzalez Laiz, Steffen SchneiderWorld Models Latent World Models
Helmholtz Munich Munich Center for Machine Learning (MCML)
MLE-World: Predicting Machine Learning Experiment Outcomes with a Learned World Model
Oct 27, 2026Brian Ko, Han Wang, Hejia Geng +2Test-Time Scaling World Model Evaluation
University of Washington · University of Illinois at Urbana-Champaign · University of Oxford +2
WMEval: Evaluating Interactive World Models Grounded on Physical Simulations
Oct 24, 2026Lanxiang Hu, Yixin Huang, Abhilash Shankarampeta +13World Model Evaluation Interactive World Models
UC San Diego · Unity
RoboWorM: Evaluating Movement Realism, Consistency and Acceleration Affects in Embodied Video World Models
Oct 24, 2026Doanh Le Thien, Viet-Thanh Nguyen, Pham Tri Quang +18World Model Evaluation Video World Models
Center for AI Research, VinUniversity · VinRobotics · MPI-INF +9
WorldFailBench: Do World Models Understand Policy Failures?
Oct 24, 2026Zheng Zhang, Xiaoyang Tang, Shuaiqi Cheng +11World Model Evaluation World Models for Robotics
Dual Exam: Controlled Evaluation of Multimodal Agents and World Models
Oct 22, 2026Zhefan Rao, Meng Chu, Xuan Zhang +80Multimodal Agent Evaluation Agent Evaluation
Does LeWorldModel Preserve Local Action Geometry?
Oct 22, 2026Vasil Vasilev, Siddharth NayakWorld Models World Model Evaluation
Technological School Electronic Systems (TUES) · Massachusetts Institute of Technology
Trace2World: Reconstructing App Worlds from Real Interaction Traces for Scalable GUI Agent Training
Oct 22, 2026Zezhou Wang, Hanzhang Zhou, Panrong Tong +5RL for GUI Agents World Model Evaluation
Nanjing University · Alibaba Token Hub, Alibaba Group · Beijing Institute of Technology
Events, Not Pixels: Where World Models Get the Return Wrong
Oct 21, 2026Haoyue Liu, Xiaoying TangWorld Model-Based Planning World Model Evaluation
School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen · Shenzhen Future Network of Intelligence Institute (FNii-Shenzhen) · Guangdong Provincial Key Laboratory of Future Networks of Intelligence, CUHK(SZ)
AnyGoal: On World Model Generalization Beyond Expert Rollouts
Oct 19, 2026Heejeong Nam, Divyat Mahajan, Quentin Le Lidec +2World Models World Model-Based Planning
Brown University · Cold Spring Harbor Laboratory · Mila, Université de Montréal +1
State World Models: Execute the Rules, Generate the Pixels
Oct 19, 2026Chan Yan Ki, Yanran Zhang, Wenzhao Zheng +5World Models Action-Conditioned World Models
Department of Automation, Tsinghua University, China · GigaAI
Operator-Mismatch Contrastive Learning for World-Relative Trajectory Coherence
Oct 18, 2026One Octadion, Firmansyah Sundana, Novanto YudistiraWorld Model Evaluation Contrastive Learning
Faculty of Computer Science, Universitas Brawijaya, Indonesia
Beyond Next-Screen Prediction: The Decision Value of GUI World Models
Oct 17, 2026Sizhe Tang, Hanhan Zhou, Jeffrey Svajlenko +1World Model Evaluation Computer-Use Agent Evaluation
George Washington University · Amazon Web Services
Execution Before Memory: Diagnosing Memory Failures in Video World Models
Oct 16, 2026Weiyang Jin, Jiwen Yu, Sihan Xu +2World Model Evaluation Video World Models
University of Hong Kong (hku.hk) · University of Michigan - Ann Arbor (umich.edu) · Peking University (pku.edu.cn)
GlobeBench: Evaluating Text-Based World Model Capabilities for Scalable Environment Simulations
Oct 15, 2026Nicholas Saban, Charvi Bannur, Mariya Vasileva +1World Model Evaluation LLM World Models
Patronus AI
OmniVerifier-C1: Decomposed Graded Verifier for Rule-Governed Generative World Modeling
Oct 15, 2026Bowei Liu, Xinchen Zhang, Xuhuan Li +10Multimodal Model Evaluation Vision-Language Model-as-a-Judge
Tsinghua University · Amap, Alibaba Group · Peking University +1
WorldCast: Exploring Intelligent Visual Generation in World State Prediction
Oct 14, 2026Wenhao Yan, Haodong Li, Xiaotong Li +9World Models Multimodal Generation
THU · CUHK · Meituan LongCat +4
Orbia: Do Generated Videos Form a Consistent 3D World?
Oct 14, 2026Dongyue Lu, Guian Fang, Hur Lim +20Geometric Consistency in Video Generation World Model Evaluation
National University of Singapore · LightSpeed Studios · HKU +5
One-Step Prediction Fidelity Is a Poor Proxy for Planning Performance in JEPA World Models
Oct 14, 2026Joshua Cheung, Zhang HeLatent World Models World Model Evaluation
WorldMemBench: A Comprehensive Benchmark for Memory of Video World Models
Oct 13, 2026Nan Zhou, Jianxiong Gao, Qiuyu Wang +10World Model Evaluation Video World Models
Zhejiang University · Ant Group · Fudan University +1
WorldOracle: Benchmarking Action-Conditioned World Models against Real-World Executions
Oct 13, 2026Haoyi Duan, Liliang Chen, Nan Wang +13World Models for Robotics Action-Conditioned World Models
AgiBot · Princeton University · NVIDIA +2
Grid or Nodes? Probing Object Memory in World Models Under Partial Observability
Oct 13, 2026Justin McMillen, Yasin YilmazWorld Model Learning Object-Centric World Models
University of South Florida Tampa, FL, USA
AffectWorld: Benchmarking Emotion-Driven Future World Prediction
Oct 13, 2026Changhao Song, Yacong Chen, Hui Gao +2World Model Evaluation Embodied World Models
School of Computer Science and Technology, Tianjin University, Tianjin, China · International Joint Institute of Tianjin University, Fuzhou, China
CivWorldBench: Evaluating Language World Models in Structured Multi-Agent Sandbox Worlds
Oct 12, 2026Ang Li, Haofei Yu, Zhengwei Tao +7LLM World Models World Model Evaluation
The Chinese University of Hong Kong · University of Illinois at Urbana-Champaign · UniPat AI +3
Same Goal, Different Words: Spatial Instruction Sensitivity in World-Action Models
Oct 12, 2026Ali-Adeeb Abbas, Esmaeil Seraj, Behrad Toghi +1World Model Evaluation Vision-Language-Action Models
RIVeR Laboratory, Northeastern University, Boston, MA, USA. · General Motors Robotics, Mountain View, CA, USA.
ProWorld: Multi-View World Model with Proprioceptive Feedback for Closed-Loop Policy Evaluation
Oct 11, 2026Zijun Wang, Tianyi Yan, Haokun Lin +12Action-Conditioned World Models Robotic Policy Evaluation
Sun Yat-sen University · University of Macau · Mohamed bin Zayed University of Artificial Intelligence +2
ExeWorldBench: Benchmarking Code-Based Executable 3D World Modeling
Oct 10, 2026Xinge Peng, Zhen Fang, Ruibing Wang +8World Models World Model Evaluation
University of Science and Technology of China · The Chinese University of Hong Kong
MultiWorldBench: Do Independently Controlled Views Describe One Shared World?
Oct 8, 2026Zhangbo Xu, Ruoxi Zhang, Rui Hu +3Multi-View Consistency World Model Evaluation
Zhiying Guangnian Technology Co., Ltd. · Imperial College London · HEC Paris
RoboJEPA: Scaling Robotic Latent World Models
Oct 7, 2026Artem Zholus, Nicolas Beltran-Velez, Jianhao Yuan +9World Models Latent World Models
FAIR at Meta · Chandar Research Lab · Mila - Quebec AI Institute +2
Kuration SDK: Addressing the Virtual2Real Gap via Data Curation
Oct 7, 2026Nirmit Desai, Eric Song, Mayank Sengupta +4Training Data Curation Action-Conditioned World Models
AIntropy AI

Oct 27, 2026J/K move · Enter open · S save

Understanding the limits of latent world models

Rodrigo Gonzalez Laiz, Steffen Schneider

Helmholtz Munich Munich Center for Machine Learning (MCML)

World Models Latent World Models Latent Dynamics Modeling World Model Evaluation Action-Conditioned World Models

Open paper AskPDF arXiv

World models are widely proposed as core components for enabling agents to reason over long temporal and spatial horizons. Yet current evaluations often emphasize task performance while providing limited insight into whether learned world models acquire predictive latent dynamics that reflect the underlying structure of the environment. Here we introduce the concept of faithful latent world models as an equivariance condition between the environment's transition map and the learned latent dynamics. We propose latent rollout consistency (LRC) as a metric to evaluate action-conditioned latent prediction. Strikingly, we find that across the studied state of the art latent world models, none solves the task by actually faithfully modeling the dynamics of the environment.