Action-conditioned video models offer a promising path to building general-purpose robot simulators that can improve directly from data. Yet, despite training on large-scale robot datasets, current state-of-the-art video models still struggle to predict physically consistent robot-object interactions that are crucial in robotic manipulation. To close this gap, we present PlayWorld, a simple, scalable, and fully autonomous pipeline for training high-fidelity video world simulators from interaction experience. In contrast to prior approaches that rely on success-biased human demonstrations, PlayWorld is the first system capable of learning entirely from unsupervised robot self-play, enabling naturally scalable data collection while capturing complex, long-tailed physical interactions essential for modeling realistic object dynamics. Experiments across diverse manipulation tasks show that PlayWorld generates high-quality, physically consistent predictions for contact-rich interactions that are not captured by world models trained on human-collected data. We further demonstrate the versatility of PlayWorld in enabling fine-grained failure prediction and policy evaluation, with up to 40% improvements over human-collected data. Finally, we demonstrate how PlayWorld enables reinforcement learning in the world model, improving policy performance by 65% in success rates when deployed in the real world.
Figures & tables
Figure 1: We introduce PlayWorld , a scalable framework for training high-fidelity video world models from autonomous robot play that enables fine-grained dynamics prediction for accurate policy evaluation, and online reinforcement learning policy fine-tuning that yields strong real-world success rate improvements.
Figure 2: PlayWorld System Diagram. Left: Autonomous data-collection pipeline in which the VLM and VLA iteratively propose and execute tasks. Right: Video world-model backbone and the setup for policy evaluation and fine-tuning.
Figure 3
Figure 4: t-SNE analysis of training samples . Robot play data exhibits markedly broader behavioral coverage than human-collected trajectories. Colors indicate coarse interaction modes assigned by a human annotator.
Figure 5: PlayWorld faithfully captures fine-grained dynamic behaviors. PlayWorld ’s predictions closely match fine-grained dynamic interaction modes in the real-world, such as missed grasps, collisions, deformations, and slips.
Figure 6: PlayWorld ’s predicted policy outcome distribution matches closely with real-world behavior.
Figure 7: Policy Evaluation Success Rate (SR) Correlation. Across diverse policies (architectures, training mixtures, and tasks), PlayWorld ’s predicted success rates are most strongly correlated with observed success rates and show the lowest variability.
Figure 8: Example Policy Interaction with different World Models. We provide closed-loop rollouts with two example policies to demonstrate how hallucinated success can impact prediction outcomes.
Figure 9: Fine-tuning Experiment Results. PlayWorld -finetuned policy demonstrates improved success rates and learns robust recovery behaviors beyond seen demonstrations.
Figure 10: Scaling behavior for world models trained with different data mixtures.
Figure 11: Object Generalization Results. We observe strong improvements in prediction quality of interaction with unseen objects as we include more diverse items in the training data.
Hyperparameter
Value
Learning Rate
5e-6
Warmup Steps
100
Scheduler
cosine
Gradient clipping
1.0
Global batch size
64
Observation Shape
320 × 192 × 3
Table 2: Hyperparameters used for video model training.
Figure 12: Example extracted centroids from the demonstration data that roughly corresponds to distinct nominal behavior episodes.
Figure 13: Example play data episodes selected from autonomous roll-outs for various tasks.
Figure 14: Training sample distribution for demonstration data, human-collected play data, and autonomously-collected play data at different scales. We observe that play data consistently provides the widest distribution over the state space, while other human-collected data exhibits limited expansion in coverage even at larger scale.
Table 3: Composition of the trajectory replay benchmark.
Table 4: Per-category perceptual similarity across different training data mixtures.
Hyperparameter
Value
Obs dim
8
Action dim
8
Image shape
[6, 192, 192]
Cond steps (state)
1
Cond steps (image)
1
Horizon
16
Table 5: Hyperparameters used for DSRL training and diffusion policy.
Figure 15: Task: remove the carrot from the bowl. Top: base policy behavior in the real world. Bottom: PlayWorld fine-tuned policy behavior in the world, where the robot learned to grasp a more stable position.
Figure 16: Task: fold the towel from left to right. Top: base policy behavior in the real world. Bottom: PlayWorld fine-tuned policy behavior in the world, where the grasping motion is more robust and accurate.