cs.CVJun 5, 2026

Dash2Sim: Closed-Loop Driving Simulation from in-the-wild Dashcam Videos

Authors: Anurag GhoshFrancesco PittalugaKhiem VuongAngela ChenJuan Alvarez-PadillaManmohan ChandrakerSrinivasa Narasimhan

Organizations: 1Carnegie Mellon University · 2NEC Labs America · 3MIT∗ · 4UC San Diego

Abstract

Self-driving simulations typically rely on data collected in a small number of cities or on hand-authored synthetic scenarios. Dashcam videos cover a far broader range of locations and situations, including rare or long-tailed scenarios. They are considered less usable for simulation because it is difficult to recover accurate 4D scenes from monocular in-the-wild videos. Work zones are one such class of long-tailed situations that dashcams capture. We present Dash2Sim, a framework that turns in-the-wild monocular dashcam videos into metric, geo-referenced 4D driving logs compatible with existing simulators, and verifies eachone against an independently maintained map without annotations. We apply Dash2Sim to a large video corpus to create the ROADWork4D benchmark dataset, which spans 4,244 scenes with 2.7M 3D objects across 17 cities. On a verified subset ROADWork4D-CL (2,201 scenes), we study privileged closed-loop planners and find that work zone scenarios are difficult: while rule-based and hybrid planners generalize better than learning-based ones, all fall short, failing to make the lane changes that temporary work zone channels require. Beyond planning, dense depth recovered by Dash2Sim improves novel-view synthesis quality by up to 19% on perceptual metrics, suggesting its potential to provide rich conditioning for closed-loop sensor simulation from monocular videos.

Explore similar work

CardsList
  1. From Driving Videos to Simulatable Scenarios

    Jun 20, 2026Alexandre Levy, Ernest Valveny Llobet, Antonio Manuel LópezAutonomous Driving SimulationScenario Generation