Action-conditioned video models offer a promising path to building general-purpose robot simulators that can improve directly from data. Yet, despite training on large-scale robot datasets, current state-of-the-art video models still struggle to predict physically consistent robot-object interactions that are crucial in robotic manipulation. To close this gap, we present PlayWorld, a simple, scalable, and fully autonomous pipeline for training high-fidelity video world simulators from interaction experience. In contrast to prior approaches that rely on success-biased human demonstrations, PlayWorld is the first system capable of learning entirely from unsupervised robot self-play, enabling naturally scalable data collection while capturing complex, long-tailed physical interactions essential for modeling realistic object dynamics. Experiments across diverse manipulation tasks show that PlayWorld generates high-quality, physically consistent predictions for contact-rich interactions that are not captured by world models trained on human-collected data. We further demonstrate the versatility of PlayWorld in enabling fine-grained failure prediction and policy evaluation, with up to 40% improvements over human-collected data. Finally, we demonstrate how PlayWorld enables reinforcement learning in the world model, improving policy performance by 65% in success rates when deployed in the real world.
Figures & tables
Figure 1: We introduce PlayWorld , a scalable framework for training high-fidelity video world models from autonomous robot play that enables fine-grained dynamics prediction for accurate policy evaluation, and online reinforcement learning policy fine-tuning that yields strong real-world success rate improvements.
Figure 2: PlayWorld System Diagram. Left: Autonomous data-collection pipeline in which the VLM and VLA iteratively propose and execute tasks. Right: Video world-model backbone and the setup for policy evaluation and fine-tuning.
Figure 3
Figure 4: t-SNE analysis of training samples . Robot play data exhibits markedly broader behavioral coverage than human-collected trajectories. Colors indicate coarse interaction modes assigned by a human annotator.
Figure 5: PlayWorld faithfully captures fine-grained dynamic behaviors. PlayWorld ’s predictions closely match fine-grained dynamic interaction modes in the real-world, such as missed grasps, collisions, deformations, and slips.
Figure 6: PlayWorld ’s predicted policy outcome distribution matches closely with real-world behavior.
Figure 7: Policy Evaluation Success Rate (SR) Correlation. Across diverse policies (architectures, training mixtures, and tasks), PlayWorld ’s predicted success rates are most strongly correlated with observed success rates and show the lowest variability.
Figure 8: Example Policy Interaction with different World Models. We provide closed-loop rollouts with two example policies to demonstrate how hallucinated success can impact prediction outcomes.
Figure 9: Fine-tuning Experiment Results. PlayWorld -finetuned policy demonstrates improved success rates and learns robust recovery behaviors beyond seen demonstrations.
Figure 10: Scaling behavior for world models trained with different data mixtures.
Figure 11: Object Generalization Results. We observe strong improvements in prediction quality of interaction with unseen objects as we include more diverse items in the training data.
Hyperparameter
Value
Learning Rate
5e-6
Warmup Steps
100
Scheduler
cosine
Gradient clipping
1.0
Global batch size
64
Observation Shape
320 × 192 × 3
Table 2: Hyperparameters used for video model training.
Figure 12: Example extracted centroids from the demonstration data that roughly corresponds to distinct nominal behavior episodes.
Figure 13: Example play data episodes selected from autonomous roll-outs for various tasks.
Figure 14: Training sample distribution for demonstration data, human-collected play data, and autonomously-collected play data at different scales. We observe that play data consistently provides the widest distribution over the state space, while other human-collected data exhibits limited expansion in coverage even at larger scale.
Table 3: Composition of the trajectory replay benchmark.
Table 4: Per-category perceptual similarity across different training data mixtures.
Hyperparameter
Value
Obs dim
8
Action dim
8
Image shape
[6, 192, 192]
Cond steps (state)
1
Cond steps (image)
1
Horizon
16
Table 5: Hyperparameters used for DSRL training and diffusion policy.
Figure 15: Task: remove the carrot from the bowl. Top: base policy behavior in the real world. Bottom: PlayWorld fine-tuned policy behavior in the world, where the robot learned to grasp a more stable position.
Figure 16: Task: fold the towel from left to right. Top: base policy behavior in the real world. Bottom: PlayWorld fine-tuned policy behavior in the world, where the grasping motion is more robust and accurate.
Action-conditioned video prediction models (often referred to as world models) have shown strong potential for robotics applications, but existing approaches are often slow and struggle to capture physically consistent interactions over long horizons, limiting their usefulness for scalable robot policy training and evaluation. We present Interactive World Simulator, a framework for building interactive world models from a moderate-sized robot interaction dataset. Our approach leverages consistency models for both image decoding and latent-space dynamics prediction, enabling fast and stable simulation of physical interactions. In our experiments, the learned world models produce interaction-consistent pixel-level predictions and support stable long-horizon interactions for more than 10 minutes at 15 FPS on a single RTX 4090 GPU. Our framework enables scalable demonstration collection solely within the world models to train state-of-the-art imitation policies. Through extensive real-world evaluation across diverse tasks involving rigid objects, deformable objects, object piles, and their interactions, we find that policies trained on world-model-generated data perform comparably to those trained on the same amount of real-world data. Additionally, we evaluate policies both within the world models and in the real world across diverse tasks, and observe a strong correlation between simulated and real-world performance. Together, these results establish the Interactive World Simulator as a stable and physically consistent surrogate for scalable robotic data generation and faithful, reproducible policy evaluation.
Yixuan Wang, Rhythm Syed, Fangyu Wu +7
Columbia University · Toyota Research Institute · Amazon +1
Real-world robot learning is constrained by the cost of collecting experience and evaluating candidate behaviors. Video generation models offer a scalable foundation for visual simulators that predict action outcomes before physical execution. Yet they often favor visual plausibility over accurate action following and coherent robot--object dynamics, while action-conditioned simulators depend on scarce, embodiment-specific data that are difficult to share across incompatible control spaces. We introduce WorldLine, an action-driven visual simulator that decouples transferable dynamics learning from heterogeneous action grounding. WorldLine learns manipulation dynamics from more than 10,000 hours of action-free robot videos and grounds them using over 2,000 hours of action trajectories across more than ten embodiments. An image-space action representation provides a shared control interface across embodiments, while multi-view and failure-enriched training with relational regularization improves interaction-sensitive prediction. Robot-focused few-step distillation enables efficient causal rollout while preserving action-critical motion. Across held-out and out-of-domain settings, WorldLine maintains strong visual quality and robot-motion agreement; on failed trajectories, it improves robot-mask IoU by 0.1626 over the strongest baseline. It predicts trajectory success with 74% mean accuracy across RoboTwin and AgiBot, one percentage point above the strongest baseline. Without RoboTwin training or adaptation, its rollouts improve task success by up to 21.4 percentage points over direct policy execution. Together, these capabilities make WorldLine a scalable and efficient visual simulator for policy evaluation and embodied planning. More results are available at project page.
Shenghe Zheng, Wenbo Li, Jiyao Zhang +4
The Hong Kong University of Science and Technology · Joy Future Academy · Peking University +1
Video world models are emerging as a scalable alternative for evaluating generalist robot policies, bypassing the physical constraints and engineering burdens of real-world deployment. However, evaluating policies with video world models remains challenging, as world-model errors can make generated rollouts unreliable and slow inference limits large-scale throughput. We introduce RoboWorld, an automated evaluation pipeline that pairs a fast autoregressive video world model with a task-progress-aware vision-language model scoring. To enable reliable long-horizon autoregressive world-model rollouts, we propose Step Forcing, which combines anchored and one-step self-forwarded contexts to reduce train-test mismatch while preserving action-observation dynamics. Together, these components enable RoboWorld to align strongly with real-world robot evaluation across tasks and environments, achieving Pearson's r = 0.989 and Spearman's ρ = 0.970.