How can robots autonomously collect the contact-rich experience needed to learn dexterous manipulation? We present PlayPolicy: Dexterity from supervised play (DSP), a framework in which a robot learns a play policy from minutes of human play through behavior cloning, then deploys it autonomously to collect hours of interaction data to train a world model. During inference, the same play policy then guides model-predictive control (MPC) by proposing candidate action sequences for evaluation under the learned world model. To accelerate inference, we also introduce a method for distilling a closed-loop, goal-conditioned policy through self-play within the learned world model. We evaluate this framework on underactuated pushing and any-to-any in-hand pose reorientation with a multifingered hand, in simulation and real world. In real world experiment, our method outperforms hindsight behavior cloning by 69.6% in success rate and achieves comparable success to reinforcement learning with 490 times less interaction data. These results demonstrate that supervised play can autonomously collect interaction data, guide model-based planning, and enable policy learning without any task-specific demonstrations or sim-to-real transfer. Project page: https://play-policy.github.io/
Figures & tables
Fig. 1: \dspfont DSP Pipeline Overview. (1) Human teleoperation trains a goal-agnostic play policy. (2) The policy collects interaction data through autonomous play. (3) These data train a world model to predict future states from past observations and proposed actions. (4) MPC uses the model to select actions for a requested goal. (5) Optionally, we can distill a goal-conditioned policy from self-play in the learned world model.
Fig. 2: Dexterity from Supervised Play. (a) A goal-agnostic play policy is trained by behavior cloning on human teleoperation data. (b) The learned policy interacts autonomously with the environment to collect observation–action transitions. (c) These transitions train an action-conditioned world model to predict future observations. (d) At inference time, the play policy proposes candidate actions, the world model predicts their outcomes, and model-predictive control selects the candidate with the highest goal-conditioned reward. The selected action is executed in the environment, and planning repeats from the new observation.
Fig. 3: Dexterous skills emerge through play-guided planning. MPC selects motions from a goal-agnostic play policy using their predicted outcomes, producing yaw rotations, edge and corner flips, fine adjustments, and composite actions without task-specific skill demonstrations. These examples illustrate how a shared repertoire of play motions can support both large reorientations and precise corrections toward a goal. Time progresses downward in each column; yellow arrows indicate object motion.
Fig. 4: Any-to-any pose in-hand object reorientation deployed on Sharpa hands. Two real-robot trials show the cube approaching the requested orientation despite substantial intermediate increases in angular error. Image sequences and the linked error curves show this evolution over time, while the SO(3) trajectories on the right show the overall paths from the starting orientations to finishes near the goals.
Fig. 5: Quantitative results of real-world object reorientation. (a) \dspfont DSP outperforms all baselines. Scoring predicted outcomes lets the controller direct the motions learned from play toward a requested goal. (b) Ours uses 701K counted frames; timesteps-matched PPO (PPO-TM) achieves 0% at this budget, while performance-matched PPO (PPO-PM) reaches 86.7% after 344M frames, about 490× the budget. PPO-unlimited serves as an upper bound of PPO with unlimited interaction frames.
Fig. 6: Experiment setup. Left: simulated environments for planar Push-T and in-hand cube reorientation. Right: the real-world setup uses a Sharpa hand mounted on a Tianji Marvin M6 arm. A Manus glove records human teleoperation, and a ZED 2i camera tracks the tagged cube.
Controller
SR@ 0.90 (%)
SR@ 0.95 (%)
Coverage
Behavior cloning [ 19 ]
64
28
0.751
Hindsight BC
74
36
0.764
Goal-conditioned BC [ 19 ]
60
34
0.733
RL, matched data [ 20 ]
0
0
0.155
Ours
98
74
0.953
Table I: Quantitative results on Push-T vs. baselines.
Fig. 7: Real vs. Sim. Our \dspfont DSP achieves matched performance with the simulation experiment (Hatched bars), and outperforms other baselines.
Controller
SR@ 0.4 rad
SR@ 0.6 rad
SR@ 0.8 rad
RGB-space DSP
34.4
53.1
76.6
State-space DSP
56.2
78.1
89.1
Table II: Quantitative results on visual reorientation from RGB. Our RGB-space method is able to achieve comparable results against state-space method without access to the object pose.
Transformer Policy
Avg. Rot. Speed
SO(3) Cov.
Trans. Cov.
(deg/s)
(%)
(%)
Deterministic
57.8
87.6
55.4
Diffusion
124.9
98.8
84.2
Table III: Coverage comparison of different play policy design.
Fig. 8: RGB world model. Snapshots of the pixel-space world model predictions across 3 seconds in Task D.
Stage
Median
p95
Visual Cube Pose Tracking
19.4
29.4
3 Play policy ( N=1,024 proposal chunks)
60.5
64.3
World model ( 1,024×25 -step rollout)
44.2
50.3
MPC scoring and selection
23.4
26.4
Replan, end to end
133.6
141.1
Table IV: Inference Time.
Fig. 9: Play-guided planning on Push-T. (a) The play policy proposes candidate action chunks (Orange) from the current state. (b) The world model predicts the resulting T-block trajectories (Blue), and MPC selects the best proposal (Yellow) that moves the block toward the target
Fig. 10: More autonomous play improves world model prediction.