Conventional end-to-end driving systems model the environment with sparse objects and lane elements. While efficient, this paradigm discards planning-critical information in crowded and occluded scenarios, particularly for unstructured obstacles, ambiguous free space, and complex interactions. We propose risk-aware occupancy, a dense BEV representation that explicitly fuses geometric occupancy, map-derived traffic constraints, and future dynamic-agent occupancy as complementary risk signals. Built upon this representation, we develop ROIDrive, an instance-centric end-to-end framework with a dedicated risk-aware occupancy branch. The predicted occupancy is tokenized via sliding-window sampling and injected into planning queries via cross-attention, while temporal query consistency mitigates unreliable flickering queries. We also contribute RiskOcc4D-nuScenes, a benchmark derived from nuScenes and Occ3D-nuScenes with four automated annotation pipelines for multi-dimensional risk supervision. Experiments on representative occupancy architectures verify the learnability and transferability of our representation. Integrated with GenAD, it reduces collision rates by 35.0% (UniAD metric) and 52.9% (ST-P3 metric), confirming the efficacy of the proposed representation modality.
Figures & tables
Figure 1: Motivation and design of risk-aware occupancy. Left: Limitations of sparse agent/map detection and naive 3D occupancy, and human-inspired requirements for dense representations covering geometry, drivability, traffic rules and vulnerable road users. Middle: Comparison between conventional decoupled E2E pipelines and our unified prediction-planning framework, which constructs global and dynamic risk-aware occupancy maps by encoding collision and traffic-regulation risks. Right: Collision-rate benchmarks on UniAD and ST-P3 metrics, verifying the safety improvement of the proposed risk-aware occupancy representation.
Figure 2: ROIDrive architecture. A BEVFormer backbone yields the BEV feature B ; the risk-aware occupancy (RiskOcc) branch regresses a normalized risk-aware occupancy via a Bi-Mamba2 block and a class-to-risk lookup table (optionally fused with an SDMap+ prior), tokenizes it by sliding-window sampling, and injects it into the queries via cross-attention. Temporal Query Consistency (TQC) then gates the queries by cross-frame reliability before a VAE generative planner decodes multimodal trajectories.
Figure 3: The four automated annotation pipelines building RiskOcc4D-nuScenes, fusing static, regulatory, drivable-area, and dynamic-future cues into the global and dynamic risk-aware occupancy maps.
Baselines
Release
Metrics
Global
Dynamic
Infer. Speed
mIoU ↑
MAE ↓
MAE@15 ↓
MAE@30 ↓
SSIM ↑
Mask-MAE ↓
FPS ↑
PanoOcc-S ( Wang et al. 2024 )
CVPR 24
14.54
0.121
0.022
0.079
0.131
0.732
6.0
PanoOcc ( Wang et al. 2024 )
CVPR 24
14.71
0.121
0.022
0.079
0.131
0.731
1.9
OccProphet ( Chen et al. 2025 )
ICLR 25
17.39
0.099
0.017
0.067
0.355
0.524
0.7
ViewFormerOcc ( Li et al. 2024 )
ECCV 24
21.38
0.093
0.015
0.060
0.379
0.734
19.0
Table 1: Feasibility benchmark for predicting risk-aware occupancy with representative 3D occupancy backbones. Best values are shown in bold .
Figure 4: Architecture of the RiskOcc-GenAD fusion experiment: the predicted risk-aware occupancy map is sampled by a sliding window and fused with GenAD’s instance tokens via cross-attention, entering as an additional input while GenAD’s protocols stay unchanged.
Methods
ST-P3 Metrics ↓
UniAD Metrics ↓
1 s
2 s
3 s
Avg.
1 s
2 s
3 s
Avg.
ST-P3
0.23
0.62
1.27
0.71
-
-
-
-
OccNet
0.21
0.59
1.37
0.72
-
-
-
-
VAD-Tiny
0.21
0.35
0.58
0.38
-
-
-
-
VAD-Base
0.07
0.17
0.41
0.22
0.01
0.30
0.95
0.42
FusionAD
0.25
0.13
0.25
0.21
-
-
-
-
Table 2: Collision rate (%) comparison among E2E methods. Ours = GenAD ( Zheng et al. 2024 ) + RiskOcc Map; P. = progressive training. Best values are shown in bold .
Inputs
ST-P3 Metrics ↓
UniAD Metrics ↓
1 s
2 s
3 s
Avg.
1 s
2 s
3 s
Avg.
RiskOcc Map
0.02
0.04
0.19
0.08
0.00
0.14
0.64
0.26
3D-Occ
0.09
0.18
0.33
0.20
0.08
0.29
0.74
0.37
Binary Map
0.11
0.21
0.39
0.24
0.14
0.39
0.90
0.48
Gaussian Map
0.21
0.32
0.37
0.33
0.27
0.51
0.86
0.55
Table 3: Ablation of intermediate representations on collision rate (%). Best in bold .
Methods
UniAD Metrics
ST-P3 Metrics
M1
M2
M3
L2 ( m ) ↓
Collision Rate (%) ↓
L2 ( m ) ↓
Collision Rate (%) ↓
1 s
2 s
3 s
Avg.
1 s
2 s
3 s
Avg.
1 s
2 s
3 s
Avg.
1 s
2 s
3 s
Avg.
✓
0.38
0.85
1.52
0.92
0.07
0.07
0.64
0.26
0.29
0.51
0.78
0.52
0.08
0.07
0.21
0.12
✓
✓
0.21
0.58
1.22
0.67
0.08
0.16
0.84
0.36
0.16
0.31
0.56
0.34
0.08
0.10
0.27
0.15
✓
✓
✓
0.21
0.59
1.26
0.69
0.00
0.14
0.86
0.33
0.16
0.32
0.57
0.35
0.00
0.04
0.21
0.08
Table 4: ROIDrive framework ablation. M1: risk-aware occupancy perception; M2: ego-token prior initialization; M3: Temporal Query Consistency (TQC). Best values are shown in bold .
Components
UniAD Collision Rate ↓
ST-P3 Collision Rate ↓
C1
C2
1 s
2 s
3 s
Avg.
1 s
2 s
3 s
Avg.
✓
0.08
0.18
0.84
0.37
0.08
0.11
0.30
0.16
✓
0.06
0.27
0.54
0.29
0.05
0.12
0.22
0.13
✓
✓
0.07
0.07
0.64
0.26
0.08
0.07
0.21
0.12
Table 5: RiskOcc-branch component ablation without ego-token initialization or TQC. C1: Bi-Mamba2; C2: SDMap+ fusion. Best values are shown in bold .
Multimodal trajectory prediction improves behavioral coverage in end-to-end autonomous driving, but existing methods remain limited by sparse scene representations. Incomplete evidence leads to low-quality candidate generation and unreliable ranking among geometrically similar trajectories. On a register-based baseline, bad and poor candidates constitute 19.74% of the candidate set, while the oracle-best candidate ranks only 33.9th on average. We propose RRDrive, which introduces risk-aware occupancy as a dense, temporally aligned, and trajectory-queryable representation. Its global structure guides high-quality multimodal generation, while candidate-conditioned risk queries support fine-grained selection. We further construct RiskOcc4D-NAVSIM with automatic risk annotations. RRDrive achieves a selected-trajectory PDMS of 0.951, representing a 1.5% relative improvement over the baseline (0.937), and improves the average candidate PDMS by 7.7%. In challenging scenes, it improves candidate PDMS by 30.2% and increases the Spearman correlation among good candidates by 0.41, from 0.26 to 0.67. To move beyond this oracle setting, we further develop an external RiskOcc predictor, a perception module that estimates risk-aware occupancy directly from sensor inputs. The competitive performance validates the representation's feasibility.
Jiaxing Chen, Hengduo Zou, YuKai Qin +3
School of Vehicle and Mobility, Tsinghua University · The Hong Kong University of Science and Technology (Guangzhou) · Neolix, Beijing, China
Safety is a fundamental requirement for autonomous driving, yet existing end-to-end driving models still lack explicit risk-aware learning capacities. Existing rule-based risk models provide interpretable safety priors, yet their absolute risk scores depend on handcrafted functions, coefficients, and thresholds. Learning-based risk representations reduce part of this manual design, but their supervision often relies on occupancy-derived labels or heuristic cost values, which may not capture ego-conditioned planning risk. In this paper, we propose DRiF, a data-driven risk-field framework for safer end-to-end autonomous driving. DRiF learns a shared BEV feature with static map segmentation, dynamic risk prediction, and vehicle planning. For dynamic risk learning, DRiF converts rule-based safety priors into pairwise risk labels, and trains the risk field to preserve relative risk ordering instead of regressing handcrafted absolute scores. Experiments on Bench2Drive show that DRiF achieves competitive overall performance, with consistent improvements in driving score, success rate, and collision-related metrics. These results establish relative risk supervision as an effective way to connect explicit safety structure with end-to-end planning. The data and code will be publicly available.
Occlusion-aware prediction remains a critical challenge in autonomous driving due to the inherent uncertainty of unobserved regions. Existing approaches either overestimate risk based on reachable states or struggle to predict accurate trajectories under high occlusion uncertainty. To address these limitations, we propose a unified risk map modeling and learning framework for partially observable environments. Our method integrates traffic flow risk and collision risk through spatiotemporal modeling, enabling fine-grained assessment of occlusion-induced hazards. To address the scarcity of scenarios involving occluded interactions, we introduce a diffusion-based scenario generation framework that produces realistic yet adversarial scenarios. We integrate the modeling and learning of a unified risk map into a framework that supports risk-aware planning under partial observability. Experiments on the Waymo Open Motion Dataset show that our method significantly outperforms the state-of-the-art occlusion-aware baseline, improving minimum time-to-collision by 0.78 times and average time-to-collision by 1.67 times. The proposed framework offers a comprehensive and practical solution for risk-aware planning in partially observable environments.
Jie Jia, Yaofeng Su, Zeyu Bao +4
Fudan University, China · Tongji University, China