Simulation Priors for Rapid Adaptation
in Model-Based RL for Autonomous Racing

Prior-Informed Adaptation (PIA) pretrains a world model on heterogeneous simulation priors, then performs few-shot online adaptation in the target domain. Across racing tasks, PIA adapts faster than baselines and deploys on a vehicle despite sim-to-real mismatch.

Video

Abstract

Reinforcement learning enables learning high-performance control policies, but its reliance on extensive interaction limits deployment on physical systems where data collection is costly, and failures are unsafe. We introduce Prior-Informed Adaptation (PIA), an offline-to-online model-based RL framework that uses structured simulation priors for rapid adaptation. PIA pretrains a world model on heterogeneous low- and high-fidelity simulated trajectories, then adapts it with limited target-domain interaction using uncertainty-regularized planning. Rather than requiring an accurate simulator for policy transfer, PIA treats simulated trajectories as biased priors whose residual mismatch is corrected during online adaptation. We evaluate PIA in autonomous racing using high-fidelity simulation and on a real full-size autonomous race car. In simulation, PIA adapts faster across track and surface shifts than strong baselines, achieving lower lap times and fewer failures. On hardware, PIA achieves reliable few-shot adaptation with less than one hour of on-vehicle interaction.

Real-Car Deployment

Before
adaptation After
adaptation

Sequences from real-world deployments. After adaptation (bottom), PIA completes the oval racing task.

We deploy the same offline-to-online pipeline on a full-size autonomous race car on an oval track. The goal is to drive at the handling limit while respecting a strict out-of-track (OOT) constraint. Each deployment stage is initialized from synthetic physics rollouts, Assetto Corsa data, and the real-car data collected in earlier stages. Starting from 11 minutes of seed data, PIA reaches 100% success by the fourth stage with 44 minutes of real-car data in total, and beats the MPC reference on best valid time and mean speed. Trained on real-car data alone, without simulation priors, the controller was erratic and the run was stopped for safety.

Progressive real-car adaptation. Each stage is initialized from simulation priors plus the real-car data collected before it. Best time is the fastest valid sector time with max OOT < 1 m and no did-not-finish.

Method

Simulation data is cheap to generate, whereas real-world data is costly and potentially dangerous to collect. PIA treats simulation priors as useful but mismatched data for initializing a world model: simulated trajectories supply broad structured coverage, limited target-domain interaction corrects their residual bias, and uncertainty-regularized planning keeps control conservative while that correction is still incomplete. PIA has two components:

• Mixed-replay, context-aware, offline-to-online training. Low-fidelity ODE rollouts, high-fidelity Assetto Corsa trajectories and real-car data are mapped to a common transition format and labeled with their source and task. The world model is updated from a mixture of the offline (simulation-prior) and online (target-domain) buffers.
• Uncertainty-regularized latent planning. The planner is discouraged from exploiting predictions in regions poorly supported by target-domain data.

Simulation prior vs. real car. The world model is pretrained on simulated driving (top) and adapted with less than one hour of real-car data (bottom).

Experiments in Simulation

Tracks. Left: the F317 car in Assetto Corsa. Right: tracks considered in the AssettoCorsaGym experiments.

We pretrain on 6 Assetto Corsa training tracks and adapt on three held-out tracks: Indianapolis (IND), Red Bull Ring (RBR) and Silverstone (SIL). PIA-AC is pretrained only on Assetto Corsa trajectories from the training tracks and tests transfer to unseen track geometry. PIA-ODE also receives low-fidelity ODE rollouts on the target track, testing cross-fidelity adaptation with known geometry and mismatched dynamics. Surface experiments keep the SIL geometry fixed and shift grip between optimal, varying and low.

Adaptation performance. Best valid lap time by episode. Mean and ±1 standard deviation over three seeds.

IQL, SAC (from scratch), FOWM and PIA: 3 seeds. Averages are reported only when all tasks are completed.