Organizations: Zhongguancun Laboratory, Beijing, China · Tianjin University, Beijing, China · Hong Kong Polytechnic University, Hong Kong, China · Nanyang Technological University, Singapore · Beihang University, Beijing, China
Autonomous driving risk identification aims to determine which observed object is likely to become safety-critical to the ego vehicle. Existing approaches typically predict scene-level accidents, infer risk objects indirectly from ego behavior, or apply geometric checks after trajectory forecasting, without directly using predicted ego--object relations for risk-source localization. We propose RiskWorld, an object-centric latent world model that identifies risk from the imagined evolution of each candidate relative to the ego vehicle. RiskWorld combines pretrained predictive video representations with structured ego--object histories, contextualizes observed interactions, and rolls relation-aware object states into the future using RSSM-style latent dynamics. It decodes the rollout into object-level risk scores, supported by auxiliary future-relation and temporal-risk predictions. Inference uses only observations up to the current time, while logged futures provide training supervision. On RiskBench, RiskWorld achieves the best overall F1 of 63.0% and the lowest false-alarm rate of 2.1%. Further analyses show that the learned rollout captures the evolution of object-level risk before critical events, while RiskWorld's selections preserve planning-critical information under filtered observation.
Figures & tables
Figure 1: Existing approaches predict scene-level risk, infer risk objects from ego behavior, or apply post-hoc geometric checks. RiskWorld instead predicts object-conditioned future ego–object relations and uses them to identify the risk source.
Figure 2: Overview of RiskWorld. Observed RGB frames, object tracks, and ego motion are encoded into relation-aware object states, rolled forward with object-wise RSSM-style latent dynamics, and decoded into risk-source scores with auxiliary future-relation and temporal-risk predictions. Future annotations supervise training only; inference uses observed history.
Interactive
Collision
Obstacle
Non-inter.
All
Method
P(%) ↑
R(%) ↑
PIC ↓
P(%) ↑
R(%) ↑
PIC ↓
P(%) ↑
R(%) ↑
PIC ↓
FA(%) ↓
F1 ↑
Random
9.5
50.0
16.7
19.2
49.8
15.7
7.2
50.3
11.4
80.8
15.1
Range (5m)
38.7
10.7
20.3
85.6
27.7
4.1
45.0
18.3
10.2
3.3
30.0
Range (10m)
43.6
69.1
7.1
69.3
62.4
3.8
33.3
88.5
0.8
15.2
53.6
Kalman filter
37.0
54.0
15.4
59.9
58.1
4.2
35.3
61.2
10.2
18.8
46.7
Social-GAN
39.4
57.7
14.1
60.8
45.4
7.8
36.9
69.3
4.0
16.3
46.0
Table 1: Risk localization and anticipation results. P, R, PIC, and FA denote precision, recall, Progressive Increasing Cost, and false-alarm rate. F1 is micro-averaged over all scenario types. ∗ denotes our object-level adaptations of BADAS and FLaRA.
Figure 3: Risk-localization behavior and operating point. (a–b) Test-set recall and false-positive rate across ego–object distances for RiskWorld and fixed-range rules. (c) Overall, interactive, and collision F1 together with non-interactive false-alarm rate across validation thresholds; the marker denotes the selected η=0.70 used for all test categories.
Figure 4: Future-risk prediction and representation. (a) Horizon-wise Brier score on risk-source objects, comparing the learned rollout with current-state persistence. Lower is better. (b) t-SNE of 1,500 balanced decision representations from risk sources and hard non-risk objects.
Figure 5: Temporal risk separation. Mean scores of the annotated risk source and the strongest non-risk object from 3 s before the critical frame; shaded regions denote 95% confidence intervals.
Variant
I ↑
C ↑
O ↑
N ↓
All ↑
w/o world features
47.6
60.0
56.7
21.3
50.6
w/o object visual token
55.7
56.6
51.9
3.3
54.6
w/o relation attention
63.8
45.9
60.7
1.2
54.3
Deterministic GRU
65.3
63.9
68.5
6.0
62.0
w/o future rollout
62.4
59.9
64.4
5.9
60.2
w/o future supervision
68.5
57.5
65.4
4.4
61.6
Table 2: Ablation of the main RiskWorld components. I, C, and O report F1 on interactive, collision, and obstacle scenarios; N reports the false-alarm rate on non-interactive scenes; All is overall micro-F1. All values are percentages.
Interactive
Obstacle
Method
IR ↓
CR(%) ↓
IR ↓
CR(%) ↓
Auto-pilot
0.45
33.6
0.52
58.7
Full observation
0.00
0.0
0.00
0.0
Ground-truth risk
0.02
0.4
0.04
5.3
Random
0.11
7.7
0.44
48.1
Range (10m)
0.01
6.2
0.24
26.4
Table 3: Planning-aware evaluation under the filtered-observation protocol. Lower values indicate that the selected object better preserves planning-critical information.
Figure 6: Risk identification and temporal evolution. The upper panels mark the predicted risk source in red. The lower panels show the annotated source score (orange) and maximum candidate score (cyan) from −3 s to +1 s relative to the critical frame. Black and red vertical lines indicate the displayed and critical frames, respectively.