cs.AIOct 19, 2026

EmbodiedWorld-200K: Toward Embodied World Agents via Embodied-Planning Dataset and Interactive World Models

Authors: X. Feng, J. Tang, Z. Lin, L. Chen, S. Shao, Q. Lu, K. Huang

Organizations: Tencent Hunyuan

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Building embodied agents that can perceive, reason, and act across diverse 3D worlds is a central research direction toward artificial general intelligence. Yet most existing work remains confined to specific tasks in a single scene and transfers poorly to new environments, leaving general embodied agents for open-world settings under-explored. We refer to such agents as Embodied World Agents (EWAs), whose defining property is the ability to understand diverse visual environments and execute general actions in open-world conditions. To advance EWA research, we study open-world embodied navigation-planning and release EmbodiedWorld-200K, a large-scale dataset of over 200K samples spanning diverse scenes. Each example is equipped with multi-level annotations, ranging from low-level camera-motion trajectories to high-level semantic instructions. The dataset also provides unified evaluation splits and task-specific metrics, supporting offline pretraining and fair evaluation of EWAs. Beyond offline supervision, a defining characteristic of an EWA is its interaction with the environment; training solely on offline data is therefore insufficient, especially for generalizing to scenes not covered in the training distribution. Building dedicated real interactive environments, however, is often prohibitively expensive. To address this, we propose a novel training scheme that employs a video world model as a low-cost environment proxy and performs interactive reinforcement-learning (RL) post-training of the EWA within it. Empirically, pretraining on EmbodiedWorld-200K substantially improves the open-world embodied planning ability of the EWA, and further RL post-training inside a world model yields clear gains on out-of-distribution scenes. The dataset, baseline checkpoints, and training and evaluation code are available at https://xiaokunfeng.github.io/EmbodiedWorld-200K/.

Figures & tables

Figure 1: Overview of EmbodiedWorld-200K. A large-scale embodied-planning dataset designed to support the study of Embodied World Agents (EWAs)—general-purpose embodied agents operating across diverse open-world 3D environments. Each sample consists of an initial visual observation, a natural-language navigation instruction, and the corresponding action sequence.
Figure 2: Data annotation pipeline. Given (a) the raw video and its camera-motion trajectory, we first (b) leverage the trajectory to perform navigation-coherent segmentation; then on each resulting segment we carry out (c) discrete action quantization and (d) navigation task instruction annotation.
Figure 3: Statistical analysis of environment types. (a) Word cloud over the descriptive keywords of the observed environments. (b-c) Distribution of NCSs by game genre and by scene category.
Figure 4
Figure 6: Qualitative comparison examples before and after Stage-I pretraining , under (a) detailed movement instructions and (b) high-level goal-navigation instruction.
Figure 7: Qualitative comparison examples on out-of-distribution scenes before and after Stage-I pretraining , where the initial observations are synthesized by an image-generation model.
ModelTM ↑\uparrowDirAcc ↑\uparrownDTW ↑\uparrowNE ↓\downarrow
Uni-NaVid [ 68 ]47.961.430.9100.6
StreamVLN [ 69 ]47.261.525.9114.7
NitroGen [ 70 ]45.639.849.00 97.2
InternVL3-8B [ 71 ]45.357.463.40 59.7
Gemini-3.5 [ 3 ]52.872.460.00 54.7
Qwen3-VL-4B (Base) [ 4 ]54.970.167.10 51.7
Table 1: Comparison on the EmbodiedWorld-200K evaluation split . External baselines cover three families: VLN models adapted to our W/A/S/D action space, a game foundation model, and zero-shot frontier VLMs.
Figure 8: Overview of the Stage-II pipeline. The actor proposes a candidate action sequence, the video world model rolls it out, and a VQA reward turns the rollout into a preferred/rejected pair for DPO.
SettingTM ↑\uparrowDirAcc ↑\uparrownDTW ↑\uparrowNE ↓\downarrow
w/o DPO65.484.466.148.9
w/ DPO75.082.892.713.4
Table 2: Performance on GTA5 evaluation split, before (w/o DPO) and after (w/ DPO) Stage-II post-training.
Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Figure A1: Complementary statistical analysis of o0o_{0} in EmbodiedWorld-200K , along additional dimensions including camera viewpoint, art style, and other metadata axes.
SettingTM ↑\uparrowDirAcc ↑\uparrownDTW ↑\uparrowNE ↓\downarrow
Base54.970.167.151.7
+SFT ( 1/31/3 data)56.370.183.040.1
+SFT ( 2/32/3 data)59.771.884.436.7
+SFT ( 3/33/3 data)62.072.083.635.6
Appendix
Table A1: Performance on the EmbodiedWorld-200K evaluation split, before ( Base ) and after Stage-I pretraining with {1/3,2/3,3/3}\{1/3,2/3,3/3\} of the full training corpus.
More in this topic

Embodied Navigation

Embodied navigation studies how robots perceive, map, and move through environments to reach goals and follow instructions.

See all 138 papers

Explore similar work

CardsList
May 18, 2026

WorldArena 2.0: Extending Embodied World Model Benchmarking on Modality, Functionality and Platform

World models have emerged as a central paradigm for embodied intelligence, enabling agents to predict action-conditioned future and reason about environmental dynamics. However, existing embodied world model benchmarks are still largely confined to vision-only prediction, offline embodied applications, and simulator-based evaluation, making them insufficient for assessing increasingly comprehensive world models. In this work, we introduce WorldArena 2.0, an expanded benchmark that systematically broadens embodied world model evaluation along three dimensions: modality, functionality, and platform. Along the modality dimension, WorldArena 2.0 extends evaluation from vision-only to visuotactile modalities, enabling assessment of multimodal perception and prediction. Along the functionality dimension, it extends beyond policy evaluation and planning to assess world models as interactive RL environments for policy optimization. Along the platform dimension, it moves beyond simulator-only evaluation to a diverse suite of simulated and real-world robotic settings across multiple embodiments. Under a standardized protocol, WorldArena 2.0 comprehensively evaluates perceptual quality, interactive utility, and cross-platform performance, providing a comprehensive testbed for tracking progress toward embodied world models. The project page including codes and demos is available at: https://world-arena.ai/.

Zhuohang Li, Yiding Ma, Yu Shang +26

Tsinghua University · Shanghai Jiao Tong University · Zhejiang University +7

World Models for RoboticsWorld ModelsAction-Conditioned World ModelsWorld Model EvaluationInteractive World ModelsEmbodied World Models
AskPDF
May 5, 2026

iWorld-Bench: A Benchmark for Interactive World Models with a Unified Action Generation Framework

Achieving Artificial General Intelligence (AGI) requires agents that learn and interact adaptively, with interactive world models providing scalable environments for perception, reasoning, and action. Yet current research still lacks large-scale datasets and unified benchmarks to evaluate their physical interaction capabilities. To address this, we propose iWorld-Bench, a comprehensive benchmark for training and testing world models on interaction-related abilities such as distance perception and memory. We construct a diverse dataset with 330k video clips and select 2.1k high-quality samples covering varied perspectives, weather, and scenes. As existing world models differ in interaction modalities, we introduce an Action Generation Framework to unify evaluation and design six task types, generating 4.9k test samples. These tasks jointly assess model performance across visual generation, trajectory following, and memory. Evaluating 14 representative world models, we identify key limitations and provide insights for future research. The iWorld-Bench model leaderboard is publicly available at iWorld-Bench.com.

Jianjie Fang, Yingshan Lei, Qin Wan +8

Tsinghua University, China · Northeastern University, China · South China University of Technology, China

World Model EvaluationInteractive World ModelsVideo World Models
AskPDF
Oct 19, 2025

A Comprehensive Survey on World Models for Embodied AI

Embodied AI requires agents that perceive, act, and anticipate how actions reshape future world states. World models serve as internal simulators that capture environment dynamics, enabling forward and counterfactual rollouts to support perception, prediction, and decision making. This survey presents a unified framework for world models in embodied AI. Specifically, we formalize the problem setting and learning objectives, and propose a three-axis taxonomy encompassing: (1) Functionality, Decision-Coupled vs. General-Purpose; (2) Temporal Modeling, Sequential Simulation and Inference vs. Global Difference Prediction; (3) Spatial Representation, Global Latent Vector, Token Feature Sequence, Spatial Latent Grid, and Decomposed Rendering Representation. We systematize data resources and metrics across robotics, autonomous driving, and general video settings, covering pixel prediction quality, state-level understanding, and task performance. Furthermore, we offer a quantitative comparison of state-of-the-art models and distill key open challenges, including the scarcity of unified datasets and the need for evaluation metrics that assess physical consistency over pixel fidelity, the trade-off between model performance and the computational efficiency required for real-time control, and the core modeling difficulty of achieving long-horizon temporal consistency while mitigating error accumulation. Finally, we maintain a curated bibliography at https://github.com/Li-Zn-H/AwesomeWorldModels.

Xinqing Li, Xin He, Le Zhang +3

College of Computer Science and the Academy for Advanced Interdisciplinary Studies, Nankai University, Tianjin 300350, China

World Model LearningWorld Models for RoboticsWorld Model EvaluationEmbodied World Models
AskPDF