World models (WMs) are intended to serve as internal simulators of the real world that enable agents to understand, anticipate, and act upon complex environments. Existing WM benchmarks remain narrowly focused on next-state prediction and visual fidelity, overlooking the richer simulation capabilities required for intelligent behavior. To address this gap, we introduce WR-Arena, a comprehensive benchmark for evaluating WMs along three fundamental dimensions of next world simulation: (i) Action Simulation Fidelity, the ability to interpret and follow semantically meaningful, multi-step instructions and generate diverse counterfactual rollouts; (ii) Long-horizon Forecast, the ability to sustain accurate, coherent, and physically plausible simulations across extended interactions; and (iii) Simulative Reasoning and Planning, the ability to support goal-directed reasoning by simulating, comparing, and selecting among alternative futures in both structured and open-ended environments. We build a task taxonomy and curate diverse datasets designed to probe these capabilities, moving beyond single-turn and perceptual evaluations. Through extensive experiments with state-of-the-art WMs, our results expose a substantial gap between current models and human-level hypothetical reasoning, and establish WR-Arena as both a diagnostic tool and a guideline for advancing next-generation world models capable of robust understanding, forecasting, and purposeful action. The code is available at https://github.com/MBZUAI-IFM/WR-Arena.
Figures & tables
Figure 1: The different focus of existing world model and our evaluation benchmarks.
Figure 2: Taxonomy with detailed examples of our evaluation benchmark.
Action Simulation Fidelity
Long-horizon Forecast
Simulative Reasoning & Planning
Model
Agent Simulation
Env. Simulation
Transition Smoothness
Simulation Consistency
Step-Wise Sim.
Open-Ended Sim.
Struct. Sim.
KLING
51.0
43.3
40.5
59.2
MiniMax
72.3
51.7
30.7
57.2
Gen-3
45.3
47.3
47.8
45.1
Cosmos1-14B
37.7
37.7
31.6
54.0
7.8
+6.7%
-2.1%
Cosmos2-14B
58.0
44.0
17.3
59.2
31.1
+0%
+4.3%
Table 1: Model performance on Action Simulation Fidelity, Long-horizon Forecast and Simulative Reasoning & Planning tasks. Results in Action Simulation Fidelity are measured based on the criteria proposed in Li et al. (2025) and then normalized. Task success rates in Open-Ended Sim. and Struct. Sim. are measured in the trajectory level and represent improvements over the pure VLM agent without integrating WM. The best-performing model in each dimension is marked in bold , and the second best is underlined .
Figure 3: The generation consistency curve w.r.t. the action sequence length. Eight world models or video generators execute identical 9-round action sequences from the same initial observations, with consistency scores measured at each round. Most models achieve lower than 75% consistency after round 5 or 6, indicating fundamental limitations for long-horizon planning applications.
Figure 4: Qualitative comparison on Structured Simulation and Planning. Starting from the same tabletop observation ( S0 ) with the goal of arranging all blocks into a line, PAN, Cosmos 1, and Cosmos 2 each simulate multi-step action plans through an iterative VLM–world model planning loop.
Figure 5: Qualitative comparison on Simulation Fidelity. Given a shared initial driving scene ( S0 ), PAN, Cosmos 1, and Cosmos 2 simulate a three-step sequence of environmental interventions from a dry forest road to light rain to intensifying rainfall.
World Model Evaluation measures whether learned simulators predict consistent, physically plausible, action-responsive futures that help agents plan and act.
A world model predicts environment dynamics based on current observations and actions, serving as a core cognitive mechanism for reasoning and planning. In this work, we investigate how world modeling based on language models can further push the boundaries of general agents. (i) We first focus on building foundation models for agentic environment simulation. We introduce Qwen-AgentWorld-35B-A3B and Qwen-AgentWorld-397B-A17B, the first language world models capable of simulating agentic environments covering 7 domains via long chain-of-thought reasoning. Leveraging more than 10M environment interaction trajectories of 7 domains in real-world environments, we develop Qwen-AgentWorld through a three-stage training pipeline: CPT injects general-purpose world modeling capabilities from the state transition dynamics and augmented professional corpora, SFT activates next-state-prediction reasoning, and RL sharpens simulation fidelity through a tailored framework with hybrid rubric-and-rule rewards. To evaluate language world models, we present AgentWorldBench, a comprehensive benchmark constructed from real-world interactions of 5 frontier models on 9 established benchmarks. Empirical results demonstrate that Qwen-AgentWorld significantly outperforms existing frontier models. (ii) Beyond foundation models, we further investigate two complementary paradigms through which world modeling enhances general agents. First, as a decoupled environment simulator, Qwen-AgentWorld supports scalable and controllable simulation of thousands of real-world environments for agentic RL, yielding gains that surpass real-environment training alone. Second, as a unified agent foundation model, world-model training acts as a highly effective warm-up that improves downstream performance across 7 agentic benchmarks. Code: https://github.com/QwenLM/Qwen-AgentWorld
Text-based world models have recently shown potential to simulate complex RL environment state transitions for a diverse set of use cases. However, no standardized benchmark exists to compare, study and improve these models. Through this paper, we attempt to solve this problem by building GlobeBench, a novel benchmark with 1,632 tasks across five separate domains and nineteen simulation skills. The tasks check whether models can predict the environment's response to a specific action given the full agent trajectory by understanding earlier state changes, following environment rules, and predicting the effects of actions on both related and unrelated objects. We compare the predicted environment response with an execution-derived reference, using a hybrid architecture with both deterministic gates and a semantic judge. Among 20 general-purpose LLMs and specialized text world models, GPT-6 Astra achieves the highest task success rate at 49.57%. We find that model accuracy decreases as actions affect a larger portion of the environment. Specifically, when the agent's changes must propagate to dependent objects, task success drops to an average of 5.1% across all 20 models. Although model strengths differ across tasks, 631 tasks remain unsolved by any model, suggesting a significant gap in general LLM capabilities as a world model.
World models have emerged as a central paradigm for embodied intelligence, enabling agents to predict action-conditioned future and reason about environmental dynamics. However, existing embodied world model benchmarks are still largely confined to vision-only prediction, offline embodied applications, and simulator-based evaluation, making them insufficient for assessing increasingly comprehensive world models. In this work, we introduce WorldArena 2.0, an expanded benchmark that systematically broadens embodied world model evaluation along three dimensions: modality, functionality, and platform. Along the modality dimension, WorldArena 2.0 extends evaluation from vision-only to visuotactile modalities, enabling assessment of multimodal perception and prediction. Along the functionality dimension, it extends beyond policy evaluation and planning to assess world models as interactive RL environments for policy optimization. Along the platform dimension, it moves beyond simulator-only evaluation to a diverse suite of simulated and real-world robotic settings across multiple embodiments. Under a standardized protocol, WorldArena 2.0 comprehensively evaluates perceptual quality, interactive utility, and cross-platform performance, providing a comprehensive testbed for tracking progress toward embodied world models. The project page including codes and demos is available at: https://world-arena.ai/.
Zhuohang Li, Yiding Ma, Yu Shang +26
Tsinghua University · Shanghai Jiao Tong University · Zhejiang University +7