We present a risk-aware formalism for evaluating system trajectories in the presence of uncertain interactions between the system and its environment. The proposed formalism supports reasoning under uncertainty and systematically handles complex relationships among requirements and objectives, including hierarchical priorities and non-comparability. Rather than treating the environment as exogenous noise, we explicitly model how each system trajectory influences the environment and evaluate trajectories under the resulting distribution of environment responses. We prove that the formalism induces a preorder on the set of system trajectories, ensuring consistency and preventing cyclic preferences. Finally, we illustrate the approach with an autonomous driving example that demonstrates how the formalism enhances explainability by clarifying the rationale behind trajectory selection.
Figures & tables
Fig. 1: Illustrative probability density of a random cost variable f , highlighting key risk measures for α=0.9 .
Fig. 2: AV–pedestrian interaction with their candidate trajectories.
Environment scenario
ω1
ω2
ω3
ω4
Pr
0.98
0.001
0.009
0.01
Eτ1
ξ1
ξ2
ξ1
ξ1
Eτ2
ξ1
ξ2
ξ2
ξ1
Eτ3
ξ1
ξ2
ξ2
ξ2
Eτ4
ξ1
ξ2
ξ1
ξ1
Fig. 3: The interaction Eτi(ωj) and probability measure Pr(ωj) for each i,j∈{1,…,4} .
r1(τi,ξj)
r2(τi,ξj)
r3(τi,ξj)
r4(τi,ξj)
ξ1
ξ2
ξ1
ξ2
ξ1
ξ2
ξ1
ξ2
τ1
0
225
0
0
0
0
0
0
τ2
0
175
0
0
1.77
1.77
0
0
τ3
0
0
0
0
15
15
12.25
12.25
τ4
0
0
1
1
0
0
0
0
Fig. 4: The degree of rule violations for each combination of AV and pedestrian trajectories.
r2
r3
r4
τ1
0
0
0
τ2
0
1.77
0
τ3
0
15
12.25
τ4
1
0
0
Fig. 5: The degree of rule violations for different AV trajectories. [Left] Probability mass function of rule r1 values for different AV trajectory τi∈T under environmental uncertainty. The x-axis represents the realized value r1(τi) and the y-axis represents the probability of observing that value according to the probability measure Pr over the environment scenarios Ω defined in Figure 3 . [Right] The violations for r2 , r3 , and r4 , where the degree of violation is independent of the environment scenarios.
Collision rate remains a dominant metric for evaluating autonomous driving trajectory safety, but its binary nature obscures substantial safety differences among collision-free trajectories. Existing surrogate-safety metrics provide finer-grained assessments, yet often trade off physical interpretability against adaptivity to interaction context. We introduce FutureRisk, a continuous metric that quantifies the latent future risk of a trajectory by how closely and persistently its realized interactions approach unsafe states. At its core, our Context-Adaptive Risk Model (CARM) combines a context-dependent safety-time distribution with an independently supervised critical-event-time distribution. Its safety criterion is bounded around a closed-form physical prior, while a bounded residual provides additional adaptation without abandoning this physical anchor. Experiments on DeepAccident and Bench2Drive show that FutureRisk achieves stronger agreement with native collision labels than existing surrogate-safety measures, discriminates risk among collision-free trajectories, and transfers zero-shot across datasets. Comparisons across four planners further show that FutureRisk reveals safety differences hidden by binary collision outcomes. Together, these results establish FutureRisk as a fine-grained, interpretable, and context-adaptive metric for trajectory safety evaluation. Code is available at https://github.com/Xuewen2025/safety-evaluation.
Xuewen Luo, Linlian Jiang, Bo Yu +3
Xuewen Luo and Bo Yu are with the University of Utah. · Linlian Jiang, Xinxin Zuo, and Yang Wang are with Concordia University. · Fan Ding is with Monash University.
Autonomous vehicles must plan trajectories that satisfy a multitude of requirements on safety, passenger comfort, and compliance with traffic rules. However, in safety-critical scenarios, it is not always possible to satisfy all requirements simultaneously, necessitating their prioritization based on importance. At the same time, in these safety-critical scenarios, the uncertainty in trajectory predictions of the surrounding traffic, such as other vehicles and pedestrians, should be explicitly accounted for. In this work, we propose an uncertainty-aware trajectory planning framework that incorporates a predefined lexicographic ordering over Signal Temporal Logic (STL) specifications that stays valid under uncertainty. We implement this formulation with Model Predictive Path Integral (MPPI) control and we demonstrate the effectiveness of our method on simulation scenarios, showing that our framework efficiently handles conflicting objectives under realistic multi-modal uncertainty.
Taha Bouzid, Shuhao Qi, Mircea Lazar +1
Control Systems Group, Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, The Netherlands
Developing autonomous driving systems for complex traffic environments requires balancing multiple objectives, such as avoiding collisions, obeying traffic rules, and making efficient progress. In many situations, these objectives cannot be satisfied simultaneously, and explicit priority relations naturally arise. Also, driving rules require context, so it is important to formally model the environment scenarios within which such rules apply. Existing benchmarks for evaluating autonomous vehicles lack such combinations of multi-objective prioritized rules and formal environment models. In this work, we introduce ScenicRules, a benchmark for evaluating autonomous driving systems in stochastic environments under prioritized multi-objective specifications. We first formalize a diverse set of objectives to serve as quantitative evaluation metrics. Next, we design a Hierarchical Rulebook framework that encodes multiple objectives and their priority relations in an interpretable and adaptable manner. We then construct a compact yet representative collection of scenarios spanning diverse driving contexts and near-accident situations, formally modeled in the Scenic language. Experimental results show that our formalized objectives and Hierarchical Rulebooks align well with human driving judgments and that our benchmark effectively exposes agent failures with respect to the prioritized objectives. Our benchmark can be accessed at https://github.com/BerkeleyLearnVerify/ScenicRules/.
Kevin Kai-Chun Chang, Ekin Beyazit, Alberto Sangiovanni-Vincentelli +2
University of California, Berkeley · Iowa State University