Simulation Distillation: Pretraining World Models in Simulation for Rapid Real-World Adaptation

Jacob Levy; Tyler Westenbroek; Kevin Huang; Fernando Palafox; Patrick Yin; Shayegan Omidshafiei; Dong-Ki Kim; Abhishek Gupta

cs.ROMar 16, 2026

Simulation Distillation: Pretraining World Models in Simulation for Rapid Real-World Adaptation

Authors: Jacob Levy, Tyler Westenbroek, Kevin Huang, Fernando Palafox, Patrick Yin, Shayegan Omidshafiei, Dong-Ki Kim, Abhishek Gupta,

Organizations: University of Texas at Austin · University of Washington · FieldAI

Abstract

Robot learning requires adaptation methods that improve reliably from limited, mixed-quality interaction data. This is especially challenging in long-horizon, contact-rich tasks, where end-to-end policy finetuning remains inefficient and brittle. World models offer a compelling alternative: by predicting the outcomes of candidate action sequences, they enable online planning through counterfactual reasoning. However, training action-conditioned robotic world models directly in the real world requires diverse data at impractical scale. We introduce Simulation Distillation (SimDist), a framework that uses physics simulators as a scalable source of action-conditioned robot experience. During pretraining, SimDist distills structural priors from the simulator into a world model that enables planning from raw real-world observations. During real-world adaptation, SimDist transfers the encoder, reward model, and value function learned in simulation, and updates only the latent dynamics model using real-world prediction losses. This reduces adaptation to supervised system identification while preserving dense, long-horizon planning signals for online improvement. Across contact-rich manipulation and quadruped locomotion tasks, SimDist rapidly improves with experience, while prior adaptation methods struggle to make progress or degrade during online finetuning. Project website and code: https://sim-dist.github.io

Figures & tables

Fig. 2 : SimDist overview. 1) An expert policy, policy checkpoints, and a value function are trained in simulation using privileged state. 2) Large-scale training data are generated by combining expert and sub-optimal policies with contiguous action perturbations, yielding diverse trajectories with dense reward and value supervision. 3) A planning-oriented latent world model is pretrained on this data, learning representations, dynamics, rewards, and values from raw observations. 4a) At deployment, the learned representation and dense reward and value models are transferred to the real robot to enable planning with the latent dynamics. 4b) Real-world data is then used to finetune only the dynamics via supervised system identification, with representations, rewards, and values frozen. Deployment and finetuning are iterated, enabling rapid and stable real-world adaptation.

Fig. 3 : World model architecture. The most recent observation is encoded into a latent representation while a history encoder processes a history of observations and actions. These jointly condition a transformer-based latent dynamics model that predicts future latent trajectories under candidate action sequences. Transformer-based reward and value heads evaluate predicted trajectories to produce reward and value sequences, while a base policy head predicts action chunks used to warm-start sampling-based planning.

Panel of Figure 3 — Fig. 4 : Real-world results. Success rate for two manipulation tasks, computed over 20 trials, and average forward progress for two quadruped locomotion tasks, averaged across all $15$ trials (3 speeds, 5 trials each), as a function of real-world finetuning data. SimDist exhibits rapid and consistent improvement with limited data by finetuning only the latent dynamics model while planning with frozen reward and value models. In contrast, direct policy finetuning with the baselines shows limited or no improvement under the same data budgets.

Fig. 5 : Value predictions from SimDist along successful and failed real-world Peg trajectories starting from the same initial condition. The predicted values track task progress and clearly distinguish successful from failure.

Fig. 6 : Throughput of manipulation policies throughout training. SimDist reliably improves the velocity of successful task completions.

Fig. 7 : Scatter-Plot showing successful and failed attempts at solving the Peg Wide task for Diffusion Policy (Right) and the final trained policy for SimDist (Left). The broad coverage of pretraining data for SimDist enables efficiently learning policies which are far more robust than baselines.

Panel of Figure 7 — Fig. 8 : (a) Finetuning drastically lowers dynamics prediction loss on a quadruped Slippery Slope trial. (b) Frames showing the front left foot slipping during the trial. (c) Foot-trajectory predictions from the world model at the same instant: the finetuned model correctly anticipates future slippage, while the pretrained model fails to do so. (d) Visualization of sampling-based planning. Candidate action sequences are evaluated with the world model. Sampled trajectories are shown from the finetuned dynamics model, along with the resulting optimal plans under the finetuned and pretrained models. The finetuned model produces plans that account for real world dynamics mismatch, while the pretrained model generates qualitatively different plans.

Fig. 9 : Real world camera observations and images reconstructed from the corresponding encoded latent state $z_{t}$ using an auxiliary encoder. We reiterate pixel reconstruction is not used as a training objective for the model.

Fig. 10 : Unfreezing world-model components during real-world adaptation.

	Peg Insertion	Table Leg	Quadruped
	(SR)	(SR)	(Reward)
SimDist	0.90	0.85	22.78
50% data	0.72	0.61	22.73
10% data	0.06	0.02	19.38
Expert Data Only	0.10	0.05	16.68
MLP Reward+Value Models	0.82	0.60	19.47

TABLE I : Ablation results in simulation, reporting success rates for manipulation tasks and average state-based reward per episode for the quadruped.

Parameter	Value
Embedding dimension	64
All transformers MLP hidden size	256
Dynamics transformer layers	3
Dynamics transformer heads	4
Reward transformer layers	1
Reward transformer heads	1

TABLE II : World model architectural parameters for manipulation.

Parameter	Value
Candidate actions batch size	250
Noised base policy actions batch size	100
Solver iterations	3
Initial action standard deviation	1.0
Minimum action standard deviation	0.05
Base policy action standard deviation	0.1

TABLE III : MPPI parameters for manipulation.

State Variable	Dim.	Domain Rand.
Base linear velocity	$3$	-
Base angular velocity	$3$	-
Projected gravity vector	$3$	-
Commanded base twist	3	-
Joint angles	$12$	-
Joint speeds	$12$	-

TABLE IV : Privileged simulator state space for the quadruped, along with domain randomization ranges, where applicable.

Term	Weight
Commanded $x,y$ -velocity tracking reward	$1.5$
Commanded yaw rate tracking reward	$0.75$
Desired gait reward	$0.05$
Desired gait foot height reward	$0.2$
Base $z$ -velocity penalty	$-2.0$
Base angular velocity penalty	$-0.05$

TABLE V : State-based reward terms used for quadruped expert policy training.

Observation	Dimension
Base linear velocity (local frame)	3
Base angular velocity (local frame)	3
Projected gravity vector	3
Joint angles	12
Joint speeds	12
Cosine and sine of phase	2

TABLE VI : Observation space for the quadruped.

Parameter	Value
Embedding dimension	64
Proprioceptive observations MLP hidden dims	128, 128
CNN kernel size	3
CNN strides	2, 2, 2
CNN features	8, 16, 32
All transformers MLP hidden size	256

TABLE VII : World model architectural parameters for the quadruped.

Parameter	Value
Candidate actions batch size	450
Noised base policy actions batch size	22
Solver iterations	8
Initial action standard deviation	2.0
Minimum action standard deviation	0.05
Base policy action standard deviation	0.05

TABLE VIII : MPPI parameters for the quadruped.

Fig. 11 : Detailed world model architecture for the quadruped.

	Speed	Pretrained model		Single-step BC policy		SimDist (ours)		IQL		RLPD
	$\mathrm{m}\text{\,}{\mathrm{s}}^{-1}$	Fwd. Prog.	Success	Fwd. Prog.	Success	Fwd. Prog.	Success	Fwd. Prog.	Success	Fwd. Prog.	Success
Slippery	0.1	0.70 $\pm$ 0.56	0/5	1.49 $\pm$ 0.29	2/5	1.78 $\pm$ 0.08	4/5	0.00 $\pm$ 0.00	0/5	0.32 $\pm$ 0.01	0/5
Slope	0.3	0.43 $\pm$ 0.13	0/5	1.43 $\pm$ 0.25	1/5	1.82 $\pm$ 0.00	5/5	0.39 $\pm$ 0.56	0/5	0.34 $\pm$ 0.05	0/5
	0.5	0.90 $\pm$ 0.40	0/5	0.62 $\pm$ 0.23	0/5	1.82 $\pm$ 0.00	5/5	0.46 $\pm$ 0.42	0/5	0.35 $\pm$ 0.01	0/5
Foam	0.2	2.98 $\pm$ 0.02	3/5	2.07 $\pm$ 1.01	1/5	3.00 $\pm$ 0.00	5/5	0.92 $\pm$ 0.48	1/5	–	–
	0.7	2.39 $\pm$ 0.71	2/5	1.54 $\pm$ 0.81	1/5	3.00 $\pm$ 0.00	5/5	2.25 $\pm$ 0.87	2/5	–	–

TABLE IX : Real-world quadruped results for both tasks. Success is reported as successful trials out of five. Forward progress is reported as mean

\pm

standard deviation (meters) across trials at each commanded speed. The Pretrained model corresponds to zero-shot deployment of the simulation-trained world model. The Single-step BC policy is the behavior cloning policy used to initialize IQL and RLPD prior to finetuning. SimDist , IQL, and RLPD results reflect performance after real-world finetuning using 35.7 minutes (Slippery Slope) and 32.1 minutes (Foam) of data. RLPD results on the Foam task are not reported, as the method destabilized the robot prior to evaluation.

			Terrain
Speed	Friction	Terrain	Difficulty
0.2	0.2	Boxes	0.2
0.4	0.4	Rough	0.4
0.6	0.6	Stairs Up	0.6
0.8	0.8	Stairs Down	0.8
1.0	1	Slope Up	1

TABLE X : Parameter values used to construct quadruped ablation environments in simulation. Each environment is defined by a unique combination of commanded forward speed, ground friction coefficient, terrain type, and terrain difficulty. All combinations are evaluated for each model, yielding 1080 environments per model.

World Models for Robotics

World models for robotics predict how scenes and robot states change under actions to support planning, control, and learning.

See all 384 papers

Interactive World Simulator for Robot Policy Training and Evaluation
Mar 9, 2026Yixuan Wang, Rhythm Syed, Fangyu Wu +7Interactive World Models World Model Learning
Columbia University · Toyota Research Institute · Amazon +1
WorldContact: A Contact-Centric World Model for Scalable Robot Learning
Sep 17, 2026Caoliwen Wang, Mengdi Wang, Heng Zhang +7Robot Policy Adaptation World Models for Robotics
University of British Columbia · Westlake University · Style3D Research
PlayWorld: Learning Robot World Models from Autonomous Play
Mar 9, 2026Tenny Yin, Zhiting Mei, Zhonghe Zheng +8World Models for Robotics Video World Models
Princeton University

Mar 9, 2026J/K move · Enter open · S save

Interactive World Simulator for Robot Policy Training and Evaluation

Yixuan Wang, Rhythm Syed, Fangyu Wu, Mengchao Zhang+6

Columbia University · Toyota Research Institute · Amazon+1

Interactive World Models World Model Learning Action-Conditioned World Models Synthetic Data Generation for Robotics Robotic Policy Evaluation

Open paper AskPDF arXiv

Action-conditioned video prediction models (often referred to as world models) have shown strong potential for robotics applications, but existing approaches are often slow and struggle to capture physically consistent interactions over long horizons, limiting their usefulness for scalable robot policy training and evaluation. We present Interactive World Simulator, a framework for building interactive world models from a moderate-sized robot interaction dataset. Our approach leverages consistency models for both image decoding and latent-space dynamics prediction, enabling fast and stable simulation of physical interactions. In our experiments, the learned world models produce interaction-consistent pixel-level predictions and support stable long-horizon interactions for more than 10 minutes at 15 FPS on a single RTX 4090 GPU. Our framework enables scalable demonstration collection solely within the world models to train state-of-the-art imitation policies. Through extensive real-world evaluation across diverse tasks involving rigid objects, deformable objects, object piles, and their interactions, we find that policies trained on world-model-generated data perform comparably to those trained on the same amount of real-world data. Additionally, we evaluate policies both within the world models and in the real world across diverse tasks, and observe a strong correlation between simulated and real-world performance. Together, these results establish the Interactive World Simulator as a stable and physically consistent surrogate for scalable robotic data generation and faithful, reproducible policy evaluation.

Abstract

World Models for Robotics

Explore similar work

Interactive World Simulator for Robot Policy Training and Evaluation

WorldContact: A Contact-Centric World Model for Scalable Robot Learning

PlayWorld: Learning Robot World Models from Autonomous Play