SafeDrive: Fine-Grained Safety Reasoning for End-to-End Driving in a Sparse World

Jungho Kim; Jiyong Oh; Seunghoon Yu; Hongjae Shin; Donghyuk Kwak; Jun Won Choi

cs.CVFeb 21, 2026

SafeDrive: Fine-Grained Safety Reasoning for End-to-End Driving in a Sparse World

Authors: Jungho Kim, Jiyong Oh, Seunghoon Yu, Hongjae Shin, Donghyuk Kwak, Jun Won Choi

Organizations: Seoul National University, Republic of Korea

Abstract

The end-to-end (E2E) paradigm, which maps sensor inputs directly to driving decisions, has recently attracted significant attention due to its unified modeling capability and scalability. However, ensuring safety in this unified framework remains one of the most critical challenges. In this work, we propose SafeDrive, an E2E planning framework designed to perform explicit and interpretable safety reasoning through a trajectory-conditioned Sparse World Model. SafeDrive comprises two complementary networks: the Sparse World Network (SWNet) and the Fine-grained Reasoning Network (FRNet). SWNet constructs trajectory-conditioned sparse worlds that simulate the future behaviors of critical dynamic agents and road entities, providing interaction-centric representations for downstream reasoning. FRNet then evaluates agent-specific collision risks and temporal adherence to drivable regions, enabling precise identification of safety-critical events across future timesteps. SafeDrive achieves state-of-the-art performance on both open-loop and closed-loop benchmarks. On NAVSIM, it records a PDMS of 91.6 and an EPDMS of 87.5, with only 61 collisions out of 12,146 scenarios (0.5%). On Bench2Drive, SafeDrive attains a 66.8% driving score.

Figures & tables

Figure 2: Overall architecture of SafeDrive. ProposalNet evaluates the scene-level safety of anchor trajectories using BEV features and selects safety-aware candidates. SWNet constructs trajectory-conditioned Sparse Worlds by simulating the future behaviors of dynamic agents and road entities. FRNet performs fine-grained safety reasoning by estimating pair-wise No Collision scores and evaluating Time-wise Drivable Area Compliance scores over time, enabling interpretable and temporally grounded safety assessment.

Method	Venue	Input	NC	DAC	TTC	EP	Comfort	PDMS
Human	-	-	100.0	100.0	100.0	87.5	99.9	94.8
UniAD [ 9 ]	CVPR 2023	C	97.8	91.9	92.9	78.8	100	83.4
Transfuser [ 29 ]	CVPR 2021	C&L	97.7	92.8	92.8	79.2	100	84.0
PARA-Drive [ 38 ]	CVPR 2024	C	97.9	92.4	93.0	79.3	99.8	84.0
LAW [ 21 ]	ICLR 2025	C	96.4	95.4	88.7	81.7	99.9	84.6
DistillDrive [ 45 ]	ICCV 2025	C&L	98.1	94.6	93.6	81.0	100	86.2

Table 1: Evaluation of PDMS performance on the NAVSIM dataset. " C " and " L " denote Camera and LiDAR inputs, respectively. The best and second-best results are highlighted in bold and underline, respectively. Unless noted, models use ResNet-34 as the image backbone, and "

\dagger

" marks the one using V2-99. The " - " denotes that the associated results are not available.

Method	Venue	Input	NC	DAC	DDC	TLC	EP	TTC	LK	HC	EC	EPDMS
Hydra - MDP++ [ 17 ]	arXiv 2025	C	97.2	97.5	99.4	99.6	83.1	96.5	94.4	98.2	70.9	81.4
DriveSuprim [ 44 ]	arXiv 2025	C	97.5	96.5	99.4	99.6	88.4	96.6	95.5	98.3	77.0	83.1
WoTE * [ 22 ]	ICCV 2025	C&L	98.5	96.8	98.8	99.8	86.0	97.9	95.4	98.3	83.0	84.1
DiffusionDrive * [ 26 ]	CVPR 2025	C&L	98.2	96.2	99.5	99.8	87.4	97.4	97.0	98.3	87.8	84.8
GaussianFusion [ 27 ]	NeurIPS 2025	C&L	98.3	97.3	99.0	97.4	87.5	97.4	97.4	98.3	-	85.0
SafeDrive (Ours)	-	C&L	99.5	99.0	99.7	99.9	88.6	98.9	97.5	98.2	81.9	87.5

Table 2: Evaluation of EPDMS performance on the NAVSIM dataset. " C " and " L " denote Camera and LiDAR inputs, respectively. All methods use a ResNet-34 backbone for a fair comparison. Methods marked with " * " are reproduced from released checkpoints.

Method	Venue	DS	SR(%)
TCP [ 40 ]	NeurIPS 2022	40.70	15.00
VAD [ 13 ]	ICCV 2023	42.35	15.00
UniAD-Base [ 9 ]	CVPR 2023	45.81	16.36
BridgeAD [ 46 ]	CVPR 2025	50.06	22.73
Hydra-MDP [ 23 ]	arXiv 2024	59.95	29.82
WoTE [ 22 ]	ICCV 2025	61.71	31.36

Table 3: Closed-loop Results on Bench2Drive Benchmark.

Figure 3: Qualitative comparison with other SOTA models on the NAVSIM test set. The expert trajectory (GT) is shown in purple, the predicted trajectories of each model are shown in blue, and the log-replayed future trajectories of surrounding agents are shown in black.

Figure 4: Visualization of the PwNC-based reasoning process. (a) Current scene images. (b) Predicted PwNC values overlaid on future positions of surrounding vehicles (green: safe, red: risky). (c) Candidate trajectory scoring by ProposalNet (NC), FRNet (PwNC), and GT (NC).

Diff	Prop	SL Eval	SWNet	FRNet	NC	DAC	TTC	EP	PDMS
✓					98.8	96.3	95.8	82.3	88.6
	✓				98.9	97.1	95.1	83.2	89.1
	✓	✓			99.0	97.7	96.3	83.5	90.1
	✓	✓	✓		99.2	98.2	96.7	84.4	90.9
	✓	✓		✓	99.2	98.5	96.0	84.6	90.9
	✓	✓	✓	✓	99.5	99.0	97.2	84.3	91.6

Table 4: Ablation study for evaluating the main components. " Diff " denotes the DiffusionDrive [ 26 ] head, " Prop " indicates the ProposalNet, and " SL Eval " refers to the scene-level safety evaluation.

World Representation	Safety	NC	DAC	TTC	PDMS
No	-	99.0	97.7	96.3	90.1
BEV	-	98.8	98.0	95.5	90.3
BEV	S	99.2	98.3	96.5	90.9
Sparse	S	99.2	98.2	96.7	90.9
Sparse	S+F	99.5	99.0	97.2	91.6

Table 5: Ablation study on world representation. " S " and " F " denote scene-level and fine-grained safety, respectively.

PwNC	TwDAC	NC	DAC	TTC	PDMS
		99.2	98.5	96.0	90.9
✓		99.5	98.7	97.3	91.5
	✓	99.2	99.0	96.7	91.4
✓	✓	99.5	99.0	97.2	91.6

Table 6: Ablation study on the impact of PwNC and TwDAC.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Metric	EPDMS	Weight	Range
Metric	only	Weight	Range
No at-fault Collisions (NC)		multiplier	{0, 1/2, 1}
Drivable Area Compliance (DAC)		multiplier	{0, 1}
Driving Direction Compliance (DDC)	$\checkmark$	multiplier	{0, 1/2, 1}
Traffic Light Compliance (TLC)	$\checkmark$	multiplier	{0, 1}
Ego Progress (EP)		5	[0, 1]

Appendix

Table 7: Metrics used in PDMS and EPDMS. " * " indicates metrics used only in PDMS.

Infraction	Penalty	Note
Pedestrian Collision	0.50	Each occurrence
Vehicle Collision	0.60	Each occurrence
Other Collision	0.65	Each occurrence
Running Red Light	0.70	Each occurrence
Scenario Timeout	0.70	Timeout (4 min)
Too Slow	0.70	Low speed

Appendix

Table 8: Infraction types and penalty factors used in the DS.

Backbone	Head	mAP	mIoU	NC	DAC	PDMS	Latency (ms)
TF	Diff	53.2	45.6	98.4	96.2	88.3	40
TF	Ours	51.8	44.7	99.0	97.3	89.6	55
BF	Diff	91.4	56.4	98.8	96.3	88.6	50
BF	Ours	86.6	54.7	99.5	99.0	91.6	67

Appendix

Table 9: Ablation study on BEV backbone and planning head. " TF " , " BF " , and " Diff " denote TransFuser, BEVFormer, and DiffusionDrive, respectively.

ProposalNet	SWNet Refinement		NC	DAC	TTC	PDMS
Refinement	Planning	Motion	NC	DAC	TTC	PDMS
			98.4	96.9	93.6	88.1
✓			99.1	98.0	96.5	90.6
✓	✓		99.1	98.3	96.7	90.9
✓	✓	✓	99.5	99.0	97.2	91.6

Appendix

Table 10: Ablation study on the impact of trajectory refinement

Method	NC	DAC	TTC	PDMS
Scene-Level NC	99.2	98.2	96.7	90.9
Conflict Filtering	99.3	98.2	96.8	91.0
Pair-wise NC	99.5	98.7	97.3	91.5

Appendix

Table 11: Comparison of collision-avoidance methods. Conflict Filtering removes trajectories overlapping with predicted agent motions.

Temporal	Interpolated	NC	DAC	TTC	PDMS
		99.2	98.2	96.7	90.9
✓		99.2	98.7	96.7	91.2
	✓	99.2	98.5	96.6	91.1
✓	✓	99.2	99.0	96.7	91.4

Appendix

Table 12: Ablation of TwDAC Components. Temporal predicts and evaluates drivable-area compliance at each time step. Interpolated enhances compliance checking using bilinear interpolation over the predicted BEV segmentation.

Figure 5: Visualization of reasoning process. The figure compares two forms of fine-grained safety reasoning, PwNC in (a) and TwDAC in (b). The bottom-left panel visualizes predicted fine-grained safety scores across Sparse Worlds using red–green shading, with (a) visualizing PwNC scores for the future boxes of surrounding agents and (b) visualizing TwDAC scores for the future ego boxes. The bottom-right panel presents the corresponding trajectory-level scores, showing NC scores for (a) and DAC scores for (b), each generated by ProposalNet, FRNet, and the ground truth.

Panel of Figure 6 — Figure 6: Additional Comparison with SOTA methods. Qualitative comparisons of DiffusionDrive, WoTE, and SafeDrive in complex traffic scenarios and turning scenarios.

Panel of Figure 7 — Figure 7: Additional Comparison with SOTA methods. Qualitative comparisons of DiffusionDrive, WoTE, and SafeDrive in long-tail scenarios and boundary-sensitive narrow road scenarios.

Figure 8: Visualization of the Bench2Drive benchmark. The predicted trajectories are visualized as blue curves in each scene. SafeDrive demonstrates robust performance across diverse scenarios in closed-loop environments, consistently generating stable and safe motions.

Figure 9: Failure Case: Inability to Anticipate Pedestrian Exit from an Opening Door. At time $t$ , SafeDrive generates a trajectory that continues in its lane without colliding with the stopped vehicle, but the predicted ego path eventually collides with a pedestrian who steps out of the vehicle as the door is opening. This occurs because SafeDrive fails to incorporate the contextual cue that a slightly open door may indicate an imminent pedestrian exit. This case highlights the need for a more comprehensive understanding of contextual cues in the scene, beyond simply recognizing the current and past positions of vehicles and pedestrians.

Autonomous Driving

Autonomous driving develops vehicles that perceive road scenes, predict interactions, and plan and control safe movement with limited or no human input.

See all 536 papers

SparseWorld: Enhancing End-to-End Autonomous Driving via World Models with Sparse Scene Representation
May 23, 2026Ruoyu Wang, Jingke Wang, Yukai Ma +5World Model Learning End-to-End Autonomous Driving
Institute of Cyber-Systems and Control, Zhejiang University, Hangzhou, China · Labs, Huawei
GraphWorld: Long-Horizon Planning with World Models for End-to-End Autonomous Driving
Jun 15, 2026Ziying Song, Caiyan Jia, Lin Liu +8World Model Learning End-to-End Autonomous Driving
Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence, School of Computer Science and Technology, Beijing Jiaotong University, Beijing, China
DriveSafer: End-to-End Autonomous Driving with Safety Guidance
May 16, 2026Shounak Sural, Raj RajkumarEnd-to-End Autonomous Driving Diffusion-Based Planning
Carnegie Mellon University Pittsburgh, PA

May 23, 2026J/K move · Enter open · S save

SparseWorld: Enhancing End-to-End Autonomous Driving via World Models with Sparse Scene Representation

Ruoyu Wang, Jingke Wang, Yukai Ma, Yuehao Huang+4

Institute of Cyber-Systems and Control, Zhejiang University, Hangzhou, China · Labs, Huawei

World Model Learning End-to-End Autonomous Driving 3D Scene Representation World Models for Autonomous Driving Autonomous Driving Planning

Open paper AskPDF arXiv

Recently, world models have made significant progress in enhancing end-to-end driving systems through both future situation forecasting and improved scene understanding. However, existing driving world models are typically built upon dense scene representations, causing high computational costs and redundant information. In this paper, we present SparseWorld, a lightweight world model that focuses on predicting only the critical layout of the scene, enabling efficient future forecasting for end-to-end driving systems. SparseWorld first performs autoregressive rollout to forecast future map elements and surrounding agents, enabling the model to learn how driving scenarios evolve over time. It then leverages these predicted futures to refine downstream motion prediction and trajectory planning. Specifically, we propose a Sparse Dreamer that anticipates future instances in the latent space through joint temporal and spatial attention. By interacting with predicted future instances, the motion planner captures more accurate motion patterns and generates more informed and safety-aware trajectories. Extensive experiments demonstrate that SparseWorld significantly reduces collision risk and achieves state-of-the-art performance on the open-loop planning metrics of the nuScenes dataset with a collision rate of 0.05%. Moreover, it substantially outperforms the baseline method in closed-loop planning metrics on the Bench2Drive benchmark. Supplementary material is available at the project page: https://wryzju.github.io/SparseWorld/.

Abstract

Autonomous Driving

Explore similar work

SparseWorld: Enhancing End-to-End Autonomous Driving via World Models with Sparse Scene Representation

GraphWorld: Long-Horizon Planning with World Models for End-to-End Autonomous Driving

DriveSafer: End-to-End Autonomous Driving with Safety Guidance