Simulation Priors for Rapid Adaptation in Model-Based RL for Autonomous Racing

Adrian Remonda; Jiajun Xi; Nicklas Hansen; Marcus Greiff; Thomas Lew; John Talbot; John Subosits; Xiaolong Wang

cs.LGOct 19, 2026

Simulation Priors for Rapid Adaptation in Model-Based RL for Autonomous Racing

Authors: Adrian Remonda, Jiajun Xi, Nicklas Hansen, Marcus Greiff, Thomas Lew, John Talbot, John Subosits, Xiaolong Wang

Organizations: Toyota Research Institute, Los Altos, CA, USA. · University of California San Diego (UCSD), La Jolla, CA, USA.

Abstract

Reinforcement learning enables learning high-performance control policies, but its reliance on extensive interaction limits deployment on physical systems where data collection is costly, and failures are unsafe. We introduce Prior-Informed Adaptation (PIA), an offline-to-online model-based RL framework that uses structured simulation priors for rapid adaptation. PIA pretrains a world model on heterogeneous low- and high-fidelity simulated trajectories, then adapts it with limited target-domain interaction using uncertainty-regularized planning. Rather than requiring an accurate simulator for policy transfer, PIA treats simulated trajectories as biased priors whose residual mismatch is corrected during online adaptation. We evaluate PIA in autonomous racing using high-fidelity simulation and on a real full-size autonomous race car. In simulation, PIA adapts faster across track and surface shifts than strong baselines, achieving lower lap times and fewer failures. On hardware, PIA achieves reliable few-shot adaptation with less than one hour of on-vehicle interaction.

Figures & tables

Panel of Figure 1 — Fig. 2 : Sequences from real-world deployments. After adaptation (bottom), PIA completes the oval racing task.

Fig. 3 : Tracks. Left : The F317 car and Assetto Corsa. Right : Tracks considered in the ACgym experiments.

Table 3

Fig. 4 : Adaptation performance. Best valid lap time by episode. Mean and

\pm 1

standard deviation over three seeds.

Run	Real data	Real %	Ep. to 1st success	Succ. %	Best time (s) OOT (m)	Mean max OOT (m)	Mean speed (m/s)
s1	00:11	0.28	5	37.5 (3/8)	FAIL (–)	1.53 $\pm$ 0.09	10.99 $\pm$ 3.68
s2	00:18	0.46	2	62.5 (5/8)	51.43 (0.13)	0.30 $\pm$ 0.29	14.63 $\pm$ 0.98
s3	00:39	1.02	1	75.0 (6/8)	50.18 (0.17)	0.26 $\boldsymbol{\pm}$ 0.11	13.57 $\pm$ 5.63
s4	00:44	1.13	1	100 (8/8)	49.55 (0.30)	0.32 $\pm$ 0.29	15.38 $\boldsymbol{\pm}$ 0.94
MPC	–	–	1	100.0	52.47 (0.87)	0.87	14.30

TABLE III : Progressive real-car adaptation. Each deployment stage is initialized from simulation priors plus accumulated real-car data collected before that stage. Real data is the amount of real-car data (HH:MM) available before that stage; Real % is its percentage of the pretraining buffer; Succ. % is the episode index of the first episode classified as successful; Best time is the fastest valid sector time with max

OOT<1m

and no Did-not-finish (DNF).

Fig. 5 : Episode outcomes. Distribution of successful episodes and failures during real-car adaptation.

Panel of Figure 7 — Fig. 6 : Vehicle XY traces. All episode trajectories for deployment sessions s1 – s4 .

Model	ODE (M)	AC (M)	Best lap (s)	# Ep. to 1st success	# Failures	Miles per Failure
SCRATCH	0.0	0.0	Fail	–	50	0.03
ODE ONLY	39.0	0.0	115.90	36	46	3.98
AC ONLY	0.0	36.0	114.16	8	18	34.62
LOW AC	39.0	1.0	112.90	6	19	33.46
LOW ODE	3.0	36.0	112.17	5	11	68.80

TABLE IV : Synthetic-only pretraining. Pretraining on synthetic rollouts alone enables competitive adaptation. Adding a small amount of AC data improves over ODE-only, while the best lap time and safety are achieved by LOW ODE, where a small amount of ODE data complements a larger AC buffer.

Experiment	Best Lap (s)	# Failures	Miles per Incident
No ODE	114.16	18	34.62
ODE G only	113.39	14	49.72
ODE D only	114.43	27	17.46
ODE G+D	112.17	11	68.80
W/o context	112.98	29	19.12
W/ context	112.17	11	68.80

TABLE V : Ablations. ODE data (partitioning G eometry and D ynamics), sub-task ID, and uncertainty penalization.

Offline data	Best lap (s) $\downarrow$	Miles per Failure $\uparrow$
$\mathrm{ODE}_{\mathrm{DR,SIL}}+\mathrm{AC}_{\mathrm{BRN}}$	113.37	13.31
$\mathrm{ODE}_{\mathrm{SIL}}+\mathrm{AC}_{\mathrm{BRN}}$	113.37	11.26
$\mathrm{AC}_{\mathrm{BRN}}$	115.46	5.20
$\mathrm{ODE}_{\mathrm{DR,SIL}}$	115.54	3.43
$\mathrm{ODE}_{\mathrm{DR,BRN}}$	117.01	2.62

TABLE VI : Domain-randomized ODE priors. All methods are fine-tuned and evaluated on the Silverstone track.

RL for Autonomous Driving

Reinforcement learning for autonomous driving trains vehicle policies through interaction to plan, control, and coordinate safely in traffic.

See all 120 papers

Simulation Distillation: Pretraining World Models in Simulation for Rapid Real-World Adaptation
Mar 16, 2026Jacob Levy, Tyler Westenbroek, Kevin Huang +6World Models for Robotics Robot Policy Adaptation
University of Texas at Austin · University of Washington · FieldAI
From Static Policies to Adaptive Priors in Offline Reinforcement Learning
Sep 26, 2026Tianwei Ni, Vineet Jain, Akash Karthikeyan +1Offline RL
Mila - Quebec AI Institute · Université de Montréal · McGill University
PEARS: Physical-Prior-Guided Efficient Adaptation via Failure Reasoning and Diffusion Steering for Tactile Manipulation
Oct 6, 2026Kun Song, Yiming Wang, Yilin Chen +5Robot Policy Adaptation Robotic RL
The University of Hong Kong · Shanghai Jiao Tong University · Xense Robotics +1

Mar 16, 2026J/K move · Enter open · S save

Simulation Distillation: Pretraining World Models in Simulation for Rapid Real-World Adaptation

Jacob Levy, Tyler Westenbroek, Kevin Huang, Fernando Palafox+5

University of Texas at Austin · University of Washington · FieldAI

World Models for Robotics Robot Policy Adaptation Sim-to-Real Transfer Model-Based RL Robotics Simulation

Open paper AskPDF arXiv

Robot learning requires adaptation methods that improve reliably from limited, mixed-quality interaction data. This is especially challenging in long-horizon, contact-rich tasks, where end-to-end policy finetuning remains inefficient and brittle. World models offer a compelling alternative: by predicting the outcomes of candidate action sequences, they enable online planning through counterfactual reasoning. However, training action-conditioned robotic world models directly in the real world requires diverse data at impractical scale. We introduce Simulation Distillation (SimDist), a framework that uses physics simulators as a scalable source of action-conditioned robot experience. During pretraining, SimDist distills structural priors from the simulator into a world model that enables planning from raw real-world observations. During real-world adaptation, SimDist transfers the encoder, reward model, and value function learned in simulation, and updates only the latent dynamics model using real-world prediction losses. This reduces adaptation to supervised system identification while preserving dense, long-horizon planning signals for online improvement. Across contact-rich manipulation and quadruped locomotion tasks, SimDist rapidly improves with experience, while prior adaptation methods struggle to make progress or degrade during online finetuning. Project website and code: https://sim-dist.github.io

Abstract

RL for Autonomous Driving

Explore similar work

Simulation Distillation: Pretraining World Models in Simulation for Rapid Real-World Adaptation

From Static Policies to Adaptive Priors in Offline Reinforcement Learning

PEARS: Physical-Prior-Guided Efficient Adaptation via Failure Reasoning and Diffusion Steering for Tactile Manipulation