Simulation Distillation: Pretraining World Models in Simulation for Rapid Real-World Adaptation
Authors: Jacob Levy, Tyler Westenbroek, Kevin Huang, Fernando Palafox, Patrick Yin, Shayegan Omidshafiei, Dong-Ki Kim, Abhishek Gupta, David Fridovich-Keil
Organizations: University of Texas at Austin · University of Washington · FieldAI
Robot learning requires adaptation methods that improve reliably from limited, mixed-quality interaction data. This is especially challenging in long-horizon, contact-rich tasks, where end-to-end policy finetuning remains inefficient and brittle. World models offer a compelling alternative: by predicting the outcomes of candidate action sequences, they enable online planning through counterfactual reasoning. However, training action-conditioned robotic world models directly in the real world requires diverse data at impractical scale. We introduce Simulation Distillation (SimDist), a framework that uses physics simulators as a scalable source of action-conditioned robot experience. During pretraining, SimDist distills structural priors from the simulator into a world model that enables planning from raw real-world observations. During real-world adaptation, SimDist transfers the encoder, reward model, and value function learned in simulation, and updates only the latent dynamics model using real-world prediction losses. This reduces adaptation to supervised system identification while preserving dense, long-horizon planning signals for online improvement. Across contact-rich manipulation and quadruped locomotion tasks, SimDist rapidly improves with experience, while prior adaptation methods struggle to make progress or degrade during online finetuning. Project website and code: https://sim-dist.github.io
Figures & tables
Fig. 2 : SimDist overview. 1) An expert policy, policy checkpoints, and a value function are trained in simulation using privileged state. 2) Large-scale training data are generated by combining expert and sub-optimal policies with contiguous action perturbations, yielding diverse trajectories with dense reward and value supervision. 3) A planning-oriented latent world model is pretrained on this data, learning representations, dynamics, rewards, and values from raw observations. 4a) At deployment, the learned representation and dense reward and value models are transferred to the real robot to enable planning with the latent dynamics. 4b) Real-world data is then used to finetune only the dynamics via supervised system identification, with representations, rewards, and values frozen. Deployment and finetuning are iterated, enabling rapid and stable real-world adaptation.
Fig. 3 : World model architecture. The most recent observation is encoded into a latent representation while a history encoder processes a history of observations and actions. These jointly condition a transformer-based latent dynamics model that predicts future latent trajectories under candidate action sequences. Transformer-based reward and value heads evaluate predicted trajectories to produce reward and value sequences, while a base policy head predicts action chunks used to warm-start sampling-based planning.
Fig. 4 : Real-world results. Success rate for two manipulation tasks, computed over 20 trials, and average forward progress for two quadruped locomotion tasks, averaged across all 15 trials (3 speeds, 5 trials each), as a function of real-world finetuning data. SimDist exhibits rapid and consistent improvement with limited data by finetuning only the latent dynamics model while planning with frozen reward and value models. In contrast, direct policy finetuning with the baselines shows limited or no improvement under the same data budgets.
Fig. 5 : Value predictions from SimDist along successful and failed real-world Peg trajectories starting from the same initial condition. The predicted values track task progress and clearly distinguish successful from failure.
Fig. 6 : Throughput of manipulation policies throughout training. SimDist reliably improves the velocity of successful task completions.
Fig. 7 : Scatter-Plot showing successful and failed attempts at solving the Peg Wide task for Diffusion Policy (Right) and the final trained policy for SimDist (Left). The broad coverage of pretraining data for SimDist enables efficiently learning policies which are far more robust than baselines.
Fig. 8 : (a) Finetuning drastically lowers dynamics prediction loss on a quadruped Slippery Slope trial. (b) Frames showing the front left foot slipping during the trial. (c) Foot-trajectory predictions from the world model at the same instant: the finetuned model correctly anticipates future slippage, while the pretrained model fails to do so. (d) Visualization of sampling-based planning. Candidate action sequences are evaluated with the world model. Sampled trajectories are shown from the finetuned dynamics model, along with the resulting optimal plans under the finetuned and pretrained models. The finetuned model produces plans that account for real world dynamics mismatch, while the pretrained model generates qualitatively different plans.
Fig. 9 : Real world camera observations and images reconstructed from the corresponding encoded latent state zt using an auxiliary encoder. We reiterate pixel reconstruction is not used as a training objective for the model.
Fig. 10 : Unfreezing world-model components during real-world adaptation.
Peg Insertion
Table Leg
Quadruped
(SR)
(SR)
(Reward)
SimDist
0.90
0.85
22.78
50% data
0.72
0.61
22.73
10% data
0.06
0.02
19.38
Expert Data Only
0.10
0.05
16.68
MLP Reward+Value Models
0.82
0.60
19.47
TABLE I : Ablation results in simulation, reporting success rates for manipulation tasks and average state-based reward per episode for the quadruped.
Parameter
Value
Embedding dimension
64
All transformers MLP hidden size
256
Dynamics transformer layers
3
Dynamics transformer heads
4
Reward transformer layers
1
Reward transformer heads
1
TABLE II : World model architectural parameters for manipulation.
Parameter
Value
Candidate actions batch size
250
Noised base policy actions batch size
100
Solver iterations
3
Initial action standard deviation
1.0
Minimum action standard deviation
0.05
Base policy action standard deviation
0.1
TABLE III : MPPI parameters for manipulation.
State Variable
Dim.
Domain Rand.
Base linear velocity
3
-
Base angular velocity
3
-
Projected gravity vector
3
-
Commanded base twist
3
-
Joint angles
12
-
Joint speeds
12
-
TABLE IV : Privileged simulator state space for the quadruped, along with domain randomization ranges, where applicable.
Term
Weight
Commanded x,y -velocity tracking reward
1.5
Commanded yaw rate tracking reward
0.75
Desired gait reward
0.05
Desired gait foot height reward
0.2
Base z -velocity penalty
−2.0
Base angular velocity penalty
−0.05
TABLE V : State-based reward terms used for quadruped expert policy training.
Observation
Dimension
Base linear velocity (local frame)
3
Base angular velocity (local frame)
3
Projected gravity vector
3
Joint angles
12
Joint speeds
12
Cosine and sine of phase
2
TABLE VI : Observation space for the quadruped.
Parameter
Value
Embedding dimension
64
Proprioceptive observations MLP hidden dims
128, 128
CNN kernel size
3
CNN strides
2, 2, 2
CNN features
8, 16, 32
All transformers MLP hidden size
256
TABLE VII : World model architectural parameters for the quadruped.
Parameter
Value
Candidate actions batch size
450
Noised base policy actions batch size
22
Solver iterations
8
Initial action standard deviation
2.0
Minimum action standard deviation
0.05
Base policy action standard deviation
0.05
TABLE VIII : MPPI parameters for the quadruped.
Fig. 11 : Detailed world model architecture for the quadruped.
Speed
Pretrained model
Single-step BC policy
SimDist (ours)
IQL
RLPD
ms−1
Fwd. Prog.
Success
Fwd. Prog.
Success
Fwd. Prog.
Success
Fwd. Prog.
Success
Fwd. Prog.
Success
Slippery
0.1
0.70 ± 0.56
0/5
1.49 ± 0.29
2/5
1.78 ± 0.08
4/5
0.00 ± 0.00
0/5
0.32 ± 0.01
0/5
Slope
0.3
0.43 ± 0.13
0/5
1.43 ± 0.25
1/5
1.82 ± 0.00
5/5
0.39 ± 0.56
0/5
0.34 ± 0.05
0/5
0.5
0.90 ± 0.40
0/5
0.62 ± 0.23
0/5
1.82 ± 0.00
5/5
0.46 ± 0.42
0/5
0.35 ± 0.01
0/5
Foam
0.2
2.98 ± 0.02
3/5
2.07 ± 1.01
1/5
3.00 ± 0.00
5/5
0.92 ± 0.48
1/5
–
–
0.7
2.39 ± 0.71
2/5
1.54 ± 0.81
1/5
3.00 ± 0.00
5/5
2.25 ± 0.87
2/5
–
–
TABLE IX : Real-world quadruped results for both tasks. Success is reported as successful trials out of five. Forward progress is reported as mean ± standard deviation (meters) across trials at each commanded speed. The Pretrained model corresponds to zero-shot deployment of the simulation-trained world model. The Single-step BC policy is the behavior cloning policy used to initialize IQL and RLPD prior to finetuning. SimDist , IQL, and RLPD results reflect performance after real-world finetuning using 35.7 minutes (Slippery Slope) and 32.1 minutes (Foam) of data. RLPD results on the Foam task are not reported, as the method destabilized the robot prior to evaluation.
Terrain
Speed
Friction
Terrain
Difficulty
0.2
0.2
Boxes
0.2
0.4
0.4
Rough
0.4
0.6
0.6
Stairs Up
0.6
0.8
0.8
Stairs Down
0.8
1.0
1
Slope Up
1
TABLE X : Parameter values used to construct quadruped ablation environments in simulation. Each environment is defined by a unique combination of commanded forward speed, ground friction coefficient, terrain type, and terrain difficulty. All combinations are evaluated for each model, yielding 1080 environments per model.
Action-conditioned video prediction models (often referred to as world models) have shown strong potential for robotics applications, but existing approaches are often slow and struggle to capture physically consistent interactions over long horizons, limiting their usefulness for scalable robot policy training and evaluation. We present Interactive World Simulator, a framework for building interactive world models from a moderate-sized robot interaction dataset. Our approach leverages consistency models for both image decoding and latent-space dynamics prediction, enabling fast and stable simulation of physical interactions. In our experiments, the learned world models produce interaction-consistent pixel-level predictions and support stable long-horizon interactions for more than 10 minutes at 15 FPS on a single RTX 4090 GPU. Our framework enables scalable demonstration collection solely within the world models to train state-of-the-art imitation policies. Through extensive real-world evaluation across diverse tasks involving rigid objects, deformable objects, object piles, and their interactions, we find that policies trained on world-model-generated data perform comparably to those trained on the same amount of real-world data. Additionally, we evaluate policies both within the world models and in the real world across diverse tasks, and observe a strong correlation between simulated and real-world performance. Together, these results establish the Interactive World Simulator as a stable and physically consistent surrogate for scalable robotic data generation and faithful, reproducible policy evaluation.
Yixuan Wang, Rhythm Syed, Fangyu Wu +7
Columbia University · Toyota Research Institute · Amazon +1
Adapting robots to new objects and tasks requires interaction experience that can be costly to obtain. We present WorldContact, a contact-centric world model for deformable-object manipulation, constructed from a limited set of high-quality trajectories to generate additional training data efficiently. It predicts object dynamics using larger time steps than the source numerical simulator, which requires small integration steps to resolve rapid motion and prevent interpenetration. We evaluate WorldContact across 16 shopping-bag manipulation tasks. State-rollout measurements on a single H100 GPU show a 10× speedup over the source simulator, excluding rendering and disk I/O. We use the generated data to fine-tune an existing vision-language-action policy and deploy it directly on a real robot. In bag lifting, the same policy achieves 65% single-attempt success when fine-tuned on source simulation data alone, compared with 95% when fine-tuned on the dataset expanded with WorldContact. These results support efficient data generation with WorldContact for robot policy adaptation.
Caoliwen Wang, Mengdi Wang, Heng Zhang +7
University of British Columbia · Westlake University · Style3D Research
Action-conditioned video models offer a promising path to building general-purpose robot simulators that can improve directly from data. Yet, despite training on large-scale robot datasets, current state-of-the-art video models still struggle to predict physically consistent robot-object interactions that are crucial in robotic manipulation. To close this gap, we present PlayWorld, a simple, scalable, and fully autonomous pipeline for training high-fidelity video world simulators from interaction experience. In contrast to prior approaches that rely on success-biased human demonstrations, PlayWorld is the first system capable of learning entirely from unsupervised robot self-play, enabling naturally scalable data collection while capturing complex, long-tailed physical interactions essential for modeling realistic object dynamics. Experiments across diverse manipulation tasks show that PlayWorld generates high-quality, physically consistent predictions for contact-rich interactions that are not captured by world models trained on human-collected data. We further demonstrate the versatility of PlayWorld in enabling fine-grained failure prediction and policy evaluation, with up to 40% improvements over human-collected data. Finally, we demonstrate how PlayWorld enables reinforcement learning in the world model, improving policy performance by 65% in success rates when deployed in the real world.