Maintaining situational awareness in complex driving scenarios is challenging. It requires continuously prioritizing attention among extensive scene entities and understanding how prominent hazards might affect the ego vehicle. While existing studies excel at detecting specific semantic categories and visually salient regions, they lack the ability to assess safety-relevance. Meanwhile, the generic spatial predicates either for foreground objects only or for all scene entities modeled by existing scene graphs are inadequate for driving scenarios. To bridge this gap, we introduce a novel task, Traffic Scene Graph Generation, which captures traffic-specific relations between prominent hazards and the ego vehicle. We propose a novel framework that explicitly uses traffic accident data and depth cues to supplement visual features and semantic information for reasoning. The output traffic scene graphs provide intuitive guidelines that stress prominent hazards by color-coding their severity and notating their effect mechanism and relative location to the ego vehicle. We create relational annotations on Cityscapes dataset and evaluate our model on 10 tasks from 5 perspectives. The results in comparative experiments and ablation studies demonstrate our capacity in ego-centric reasoning for hazard-aware traffic scene understanding.
TABLE I: Core component explanations and statistics of the Traffic Accident Knowledge Graph (side-by-side node/edge view)
Fig. 2: Inference performance vs. training set size (5%–80% of total training set). For each size, five models were trained with five-fold splits, with 20% of training images held out for validation per fold.
Object Prediction
Subject Prediction
Triplet Prediction
Method
MRR
H@1
H@10
MRR
H@1
H@10
MRR
H@1
H@10
StareE [ 27 ]
38.66
31.50
52.63
77.21
67.97
93.36
57.94
49.73
73.00
OUR-1HOP
88.04
79.81
99.77
96.76
94.38
99.74
92.40
87.09
99.75
OUR-3HOP
88.31
80.54
99.77
95.96
92.77
99.80
92.14
86.66
99.79
TABLE II: Comparison on Three Subtasks of KGE (%)
MAP@K
MRR@K
NDCG@K
Method
K=3
K=5
K=10
K=3
K=5
K=10
K=3
K=5
K=10
CFHP [ 7 ]
31.66
45.64
60.97
58.41
73.90
77.78
37.94
53.40
56.95
OUR
54.87
68.04
82.90
78.33
88.34
96.60
64.19
77.48
89.47
TABLE III: Comparison on Hazards Prioritization (%)
Method
R@20 / mR@20
R@30 / mR@30
R@50 / mR@50
MOTIFS [ 33 ]
25.99 / 7.39
29.65 / 9.37
29.68 / 9.38
VCTREE [ 34 ]
26.95 / 4.12
29.60 / 5.23
29.60 / 5.23
PSGTR [ 15 ]
17.95 / 5.53
23.71 / 12.20
23.72 / 12.21
OUR
30.47 / 53.75
50.00 / 67.63
62.79 / 76.13
TABLE IV: Comparison on SGDet Task (%)
Task
Method
P (%)
R (%)
F1 (%)
AUC (%)
MAE ↓
Entity Relevance
MOTIFS-predcls [ 33 ]
82.96
43.46
53.42
81.14
0.323
MOTIFS-sgdet [ 33 ]
85.73
50.90
61.61
77.11
0.270
VCTREE-predcls [ 34 ]
83.59
43.40
53.05
81.63
0.315
VCTREE-sgdet [ 34 ]
88.15
36.87
49.84
76.19
0.319
PSGTR [ 15 ]
87.62
31.74
44.24
62.88
0.327
OUR
96.96
94.07
95.03
97.20
0.047
TABLE V: Comparison on Two Classification Tasks (%)
Relation
Method
R@1/mR@1
R@2/mR@2
R@3/mR@3
Severity
HATS w/o ERES
31.27 / 35.32
44.42 / 50.05
48.42 / 54.89
HATS w/o KGE
42.03 / 44.99
60.00 / 63.02
65.39 / 68.89
HATS w/o Depth
41.21 / 43.15
60.47 / 61.69
65.02 / 66.96
HATS w/o KGE+Depth
39.46 / 43.25
58.39 / 61.09
63.61 / 66.43
HATS w/o ERES+KGE+Depth
26.29 / 31.68
42.01 / 47.91
47.56 / 53.39
HATS
80.10 / 80.72
92.19 / 92.76
93.93 / 94.99
TABLE VI: Ablation Study of HATS components
Fig. 3: Our ego-centric hazard-aware TSGs of traffic images
Technical University of Munich, Chair of Robotics, Artificial Intelligence (AI) and Embedded Systems, Boltzmannstraße 3, 85748 Garching bei München, Germany
Institute for Intelligent Systems, Esslingen University of Applied Sciences, Germany. · Department of Computer Science, University of Freiburg, Germany.