The generation of safety-critical traffic scenarios is essential for training and evaluating autonomous vehicles. Prior approaches typically perturb the trajectories of existing agents in a traffic scenario using simplified adversarial objectives to induce safety-critical interactions, which can limit the plausibility and diversity of the generated scenarios. Although inserting new adversarial vehicles can alleviate this limitation, determining when and where to introduce them in a scenario-specific manner remains challenging. In this work, we introduce COllision Snapshot guided TimE-Reversed safety-critical scenario generation (COSTER), a framework that leverages learned traffic priors to determine plausible collision times and locations. COSTER first constructs a collision snapshot by inserting a new vehicle in contact with the target vehicle at the identified collision state within a traffic scenario. Starting from this collision snapshot, a conditional variational autoencoder is used to perform a time-reversed rollout, reconstructing the trajectory of the inserted vehicle backward toward earlier timesteps. Experiments show that COSTER outperforms existing methods in plausibility, diversity, and data efficiency. Moreover, agents trained on COSTER-generated scenarios reduce collision rates by 31% on safety-critical scenarios from the Waymo Open Motion Dataset while also improving ego task completion. The project website is available at https://anonym-121.github.io/COSTER/.
Figures & tables
Fig. 1: The overview of COSTER. COSTER predicts surrounding vehicle states at each timestep (Sec. III-B ), identifies the highest-risk timestep to construct a collision snapshot by inserting a contextually plausible adversarial vehicle in contact with the target vehicle (Sec. III-C ), and generates physically feasible safety-critical scenarios via time-reversed trajectory rollout with clustering-based selection and refinement (Sec. III-D ).
Fig. 2: Visualization of a sample scenario with overlaid local regions. (a) Region-wise representation. (b) Adversarial vehicle placement. At the given collision time, the peak-probability region within the elliptical area (red dashed line) is selected. The heading is then estimated at the red region, and the adversarial vehicle is translated into contact with the target vehicle to be placed at its final high-probability position.
minADE (m) ↓
minFDE (m) ↓
avgADE (m) ↓
avgFDE (m) ↓
Inference Time (ms) ↓
Constant Velocity
1.695
4.961
1.695
4.961
-
Constant Lane
14.860
17.442
14.860
17.442
-
BMT [ 6 ]
0.417
1.022
0.838
2.190
536.2
COSTER (w/o DSA)
0.557
1.545
1.212
3.460
5.7
COSTER (w/ DSA)
0.259
0.828
0.570
1.966
39.4
TABLE I: Quantitative Comparison on Past Trajectory Prediction Accuracy
Position ↓
Heading ↓
Speed ↓
Size ↓
Mean Value
0.2249
0.4237
0.1564
0.5451
LCTGen [ 21 ]
0.1466
0.1555
0.2286
0.1052
TrafficGen [ 20 ]
0.1367
0.1277
0.1704
0.0937
COSTER
0.1242
0.1116
0.1399
0.0764
TABLE II: MMD Comparison on Vehicle State Inference.
Data Efficiency
Adversariality
Plausibility
Diversity
SUR (%) ↑
CR (%) ↑
ORR (%) ↓
VDD ↓
ADD ↓
CSD (m/s) ↑
CPD ( \SIUnitSymbolDegree ) ↑
CAT [ 14 ]
8.9
53.2
12.3
0.2775
0.2370
3.14
80.1
SEAL [ 17 ]
8.7
60.8
19.1
0.2828
0.2419
3.31
70.8
Adv-BMT [ 6 ]
61.2
100
62.3
0.2394
0.3019
3.37
29.9
COSTER (Ours)
74.1
100
10.2
0.2485
0.0983
4.75
86.2
Random + CVAE
63.5
100
51.8
0.2451
0.1147
3.42
31.6
TABLE III: Quantitative Comparison of Generated Safety-Critical Scenarios.
Fig. 3: Qualitative comparison of generated safety-critical scenarios.
CSD (m/s) ↑
CPD ( \SIUnitSymbolDegree ) ↑
Random timestep
4.12
70.2
Last timestep
4.29
73.3
Velocity entropy timestep
4.75
86.2
TABLE IV: Comparison of Collision Time Determination.
WOMD-Normal
WOMD-Hard [ 32 ]
Training Data
RCR ↑
CR ↓
ORR ↓
RCR ↑
CR ↓
ORR ↓
WOMD-Only
0.68 ±0.03
0.17 ±0.01
0.24 ±0.02
0.30 ±0.02
0.32 ±0.03
0.32 ±0.04
CAT [ 14 ]
0.67 ±0.05
0.18 ±0.01
0.26 ±0.02
0.29 ±0.04
0.33 ±0.03
0.34 ±0.01
SEAL [ 17 ]
0.69 ±0.03
0.16 ±0.01
0.26 ±0.02
0.35 ±0.01
0.27 ±0.02
0.30 ±0.03
Adv-BMT [ 6 ]
0.64 ±0.07
0.20 ±0.02
0.29 ±0.02
0.26 ±0.02
0.38 ±0.04
0.36 ±0.01
COSTER (w/o Aug)
0.71 ±0.06
0.15 ±0.03
0.23 ±0.04
0.38 ±0.04
0.24 ±0.01
0.31 ±0.02
TABLE V: Downstream Evaluation of Policy Safety under Normal and Hard Scenarios
Safety-critical scenarios are essential for the development of autonomous vehicles (AVs) but are rare in real-world driving data. While simulation offers a way to generate such scenarios, manually designed test cases lack scalability, and adversarial optimization often produces unrealistic behaviors. In this work, we introduce a conditional latent flow matching approach for scalable and realistic safety-critical scenario generation. Our method uses distribution matching to transform nominal scenes into safety-critical rollouts. Furthermore, we demonstrate that incorporating both simulation and real-world data enables our framework to efficiently generate diverse, data-driven scenarios. Experimental results highlight that our approach is able to more consistently and realistically generate novel safety-critical scenarios, making it a valuable tool for training and benchmarking AV systems.
Zimu Gong, Brian Zhaoning Zhang, Chris Zhang +2
University of Michigan-Ann Arbor · University of Waterloo · 1Waabi Innovation Inc +1
Generating safety-critical scenarios is essential for validating the robustness of autonomous driving systems, yet existing methods often struggle to produce collisions that are both realistic and diverse while ensuring explicit interaction logic among traffic participants. This paper presents a novel framework for traffic-flow level safety-critical scenario generation via multi-objective Monte Carlo Tree Search (MCTS). We reframe trajectory feasibility and naturalistic behavior as optimization objectives within a unified evaluation function, enabling the discovery of diverse collision events without compromising realism. A hybrid Upper Confidence Bound (UCB) and Lower Confidence Bound (LCB) search strategy is introduced to balance exploratory efficiency with risk-averse decision-making. Furthermore, our method is map-agnostic and supports interactive scenario generation with each vehicle individually powered by SUMO's microscopic traffic models, enabling realistic agent behaviors in arbitrary geographic locations imported from OpenStreetMap. We validate our approach across four high-risk accident zones in Hong Kong's complex urban environments. Experimental results demonstrate that our framework achieves an 85% collision failure rate while generating trajectories with superior feasibility and comfort metrics. The resulting scenarios exhibit greater complexity, as evidenced by increased vehicle mileage and CO2 emissions. Our work provides a principled solution for stress testing autonomous vehicles through the generation of realistic yet infrequent corner cases at traffic-flow level.
Wenyun Li, Zejian Deng, Chen Sun
Department of Mathematics, The University of Hong Kong (HKU) · Department of Data and Systems Engineering, HKU
Safety-critical traffic scenario generation is essential for evaluating autonomous driving systems under rare but high-risk interactions. Existing diffusion-based methods offer strong controllability in closed-loop generation, but their iterative denoising process is computationally expensive and may accumulate sampling and guidance errors over long rollouts, causing unrealistic motion artifacts such as jitter, abnormal acceleration, and off-road behavior. To address these issues, we propose RiskFlow, a closed-loop safety-critical multi-agent traffic generation framework that formulates future trajectory generation as transport in the action space. Instead of relying on iterative denoising, RiskFlow learns an average velocity field over a finite interval to transform Gaussian action sequences into future acceleration and yaw-rate commands with a single forward pass, using a JVP-based objective for efficient and stable training. At test time, RiskFlow applies output-space guidance to the generated actions, steering selected critical agents toward risky interactions while regularizing off-road behavior, and reconstructs physically feasible trajectories through vehicle dynamics. Experiments on nuScenes with tbsim closed-loop evaluation show that RiskFlow achieves a strong adversariality-realism trade-off across multi-agent and long-horizon settings. Compared with representative baselines, RiskFlow consistently improves realism while maintaining competitive safety-critical generation capability, and substantially reduces inference time for evaluation.
Qi Lan, Yining Tang, Yu Shen +4
College of Mechanical and Vehicle Engineering Chongqing University, Chongqing, China