Organizations: Beihang University · Horizon Robotics · University of Macau · BAAI · AIR, Tsinghua University · OpenDriveLab, The University of Hong Kong
Real-world driving requires context-aware decision-making. Existing end-to-end driving benchmarks primarily assess trajectory quality through safety, compliance, comfort, and driving progress, leaving context-dependent decision-making insufficiently characterized. To fill this gap, we present PlanContext, a benchmark for evaluating in-context driving decision-making. PlanContext constructs paired evaluations by varying high-level driving intentions while keeping the observed scene fixed, enabling a natural and controllable assessment of context-dependent decision-making. We further define six representative driving contexts and jointly evaluate contextual decision correctness, road compliance, and dynamic safety with class-balanced metrics. Experiments on 9,561 scene-condition instances from NAVSIM and 23 public end-to-end driving models show that high conventional driving-quality scores do not consistently correspond to strong route-level in-context driving decision-making. Further analysis shows that model behavior varies with decision timing, road structure, and traffic interaction depending on the decision required by the current context. Our results highlight in-context driving decision-making as an important complementary dimension for evaluating end-to-end driving systems.
Figures & tables
Figure 1: PlanContext. Conventional driving metrics primarily evaluate trajectory quality in terms of safety, compliance, comfort, and driving progress. However, models with high PDMS can still produce context-inappropriate trajectories when navigation intentions change. PlanContext uses a natural and controllable navigation intervention to evaluate in-context driving decision-making.
Figure 2: Representative examples of the six PlanContext categories. Parentheses indicate the change in navigation intention.
Category
Context condition
Requirement
A
Target route reachable within 2 s
Follow
B1
Target route reachable in 4.5–8 s
Wait
B2
No locally feasible target
Reject
C
Ego beyond route commitment point
Reject
L
Adjacent target lane geometrically and topologically reachable
Follow
LC
Lateral maneuver already in progress
Preserve
Table 1: PlanContext scenario categories and contextual requirements.
Model
Venue
PDMS
mPDMS
NAS
CPS
NCS
LAS
MCS
VA Models
DrivoR ( Kirby et al., 2026 )
CVPR 2026
93.96
83.56
19.76
61.23
34.78
0.00
92.62
GoalFlow ( Xing et al., 2025 )
CVPR 2025
88.31
79.82
15.57
63.89
31.54
0.07
87.82
TransFuser ( Chitta et al., 2022 )
IEEE TPAMI
83.39
74.16
13.77
52.98
27.01
0.15
73.59
SparseDriveV2 ( Sun et al., 2026 )
ECCV 2026
92.22
81.22
12.57
56.25
26.60
0.15
93.64
DiffusionDrive ( Liao et al., 2025 )
CVPR 2025
88.05
79.80
3.29
65.44
14.68
0.00
92.11
Table 2: Overall comparison of 23 end-to-end driving models on PlanContext.
Figure 3: Driving quality and in-context performance across 23 end-to-end driving models.
Category
Instances
Success (%)
A
334
10.01
B1
251
62.91
B2
5,285
73.28
C
580
34.98
L
2,716
0.45
LC
395
85.51
Table 3: Contextual success across PlanContext categories.
Decision Urgency
Geometric Salience
Topological Evidence
Category
Low
Medium
High
Low
Medium
High
Δq [95% CI]
A
15.59
11.28
3.22
6.66
12.34
11.02
+13.15∗ [10.62, 15.72]
B1
64.22
61.90
62.63
55.74
77.43
55.49
+35.67∗ [21.22, 49.46]
B2
73.50
75.06
71.29
73.10
75.36
71.39
−13.45∗ [-15.16, -11.78]
C
23.74
43.39
37.79
29.96
36.09
38.86
−20.77∗ [-28.05, -13.39]
L
0.49
0.43
0.42
0.46
0.43
0.46
+0.08 [-0.07, 0.22]
Table 4: Associations between scene context factors and PlanContext performance.
Category
Lead Vehicle
TTC Risk
Target Occupancy
Crossing Conflict
Traffic Density
A
−4.88
−9.08∗
−9.94∗
−0.18
+5.31∗
B1
+17.47
−5.15
−6.00
+12.39
−0.88
B2
+9.05∗
+6.29∗
+5.54∗
−4.45∗
+12.10∗
C
+3.58
−10.36
−10.02∗
−11.51∗
−9.75∗
L
−0.50∗
−0.26∗
−0.39∗
+0.20
−0.13∗
LC
−7.62∗
−8.62∗
−6.88∗
−9.56∗
−7.82∗
Table 5: Associations between traffic interaction factors and PlanContext performance.
Appendix figures & tables20 assets
Supplementary material from the paper’s appendix.
Appendix
Stage
Instances
Description
Initial candidates
12,840
Original scene–condition candidates
After motion filtering
12,242
Remove near-stationary instances
Final benchmark
9,561
Apply final context-aware filtering
Appendix
Table 6: Scenario filtering statistics for PlanContext.
Category
Original condition
Alternative condition
A
Original branch
Target branch
B1
Original approach branch
Original approach branch
B2
Local valid roads and adjacent roads
Local valid roads and adjacent roads
C
Committed original branch
Committed original branch
L
Current lane
Current and target adjacent lanes
LC
Maneuver-related local roads
Maneuver-related local roads
Appendix
Table 7: Permitted road regions for the six PlanContext categories.
Category
Requirement
Instances
A
Follow
334
B1
Wait
251
B2
Reject
5,285
C
Reject
580
L
Follow
2,716
LC
Preserve
395
Appendix
Table 8: Composition of the PlanContext benchmark.
Model
NCS
95% CI
OneVL
39.30
[34.73, 43.41]
ChainFlow-VLA
35.13
[31.19, 38.83]
DrivoR
34.78
[30.76, 38.58]
ReCogDrive-IL
34.31
[30.38, 38.01]
SUV
32.03
[27.10, 36.55]
ReCogDrive-RL
31.64
[27.82, 35.18]
Appendix
Table 9: NCS and 95% scene-level bootstrap confidence intervals for the 23 evaluated models.
Setting
Minimum Spearman correlation
Maximum rank change
w∈[0.40,0.60]
0.9960
1
w∈[0.30,0.70]
0.9703
5
Omit B1 from CPS
0.9960
2
Omit B2 from CPS
0.9970
1
Omit C from CPS
0.9970
1
Appendix
Table 10: Robustness of the PlanContext ranking under metric variations.
Category
Alt. Correct.
RoadOK
Dynamic Safety
Overall Safety
Alt. Success
Contextual Success
A
44.94
19.49
66.12
11.95
11.44
10.01
B1
94.37
68.63
85.43
63.59
62.91
62.91
B2
92.50
79.86
84.19
73.95
73.28
73.28
C
88.12
46.87
67.93
35.85
35.01
34.98
L
3.51
97.57
94.16
93.55
1.58
0.45
LC
97.51
94.32
87.05
85.85
85.64
85.64
Appendix
Table 11: Atomic metric decomposition across the six PlanContext categories.
Figure 4: Category A: an immediately actionable alternative route. The navigation command changes from straight to left at an intersection where the alternative left-turn route is reachable within the evaluation horizon. The trajectories illustrate differences in how models respond to the alternative navigation intention.
Figure 5: Category B1: an alternative route beyond the current decision window. The navigation command changes from straight to right before the right-turn target becomes actionable. Continuing along the current road remains appropriate within the evaluation horizon. The trajectories illustrate differences between maintaining the current path and deviating prematurely toward the requested direction.
Figure 6: Category B2: a locally unsupported navigation intention. The navigation command changes from straight to right where the local road topology provides no feasible right-turn route. The trajectories illustrate differences between following the existing road geometry and deviating toward the unsupported direction.
Figure 7: Category C: a navigation intervention after route commitment. The navigation command changes from left to straight after the ego vehicle has passed the route commitment point. The trajectories illustrate differences between continuing along the committed route and deviating in response to the changed command. Collision annotations identify the marked failures in the displayed predictions.
Figure 8: Qualitative examples for OneVL. The first two cases show collisions following navigation interventions in categories C and A. The third case shows a directionally correct response in category C.
Figure 9: Qualitative examples for ChainFlow-VLA. The cases show an incorrect route choice after route commitment in category C, a collision in category B2, and a correct alternative route choice in category A.
Figure 10: Qualitative examples for DrivoR. The first case shows a collision in category B2. The two category A cases illustrate different outcomes when responding to an immediately actionable alternative navigation intention, including an incorrect route choice and a correct route choice.
Figure 11: Qualitative examples for StateMLP. The first two cases show collisions in categories B2 and L. The third case shows a correct alternative route choice in category A.
Figure 12: Qualitative examples for Curious-VLA. The first two cases show collisions in categories A and C. The third case shows continued travel along the committed route after the navigation command changes in category C.
Figure 13: Qualitative examples for CLOVER. The cases show an incorrect alternative route choice in category A, a collision in category B2, and a correct alternative route choice in another category A scene.
Figure 14: Qualitative examples for WAM-Flow. The two category C cases show a collision and travel against the traffic direction following navigation interventions after route commitment. The category A case shows a correct alternative route choice.
Figure 15: Qualitative examples for SimWAM. The first case shows an incorrect route choice in category C. The two category A cases show travel against the traffic direction and a correct alternative route choice, respectively.
Figure 16: Qualitative examples for Drive-JEPA. The cases show a collision in category L, little visible change between the paired predictions in category A, and travel against the traffic direction in category B2.
Figure 17: Qualitative examples for DynVLA Stage 2. The cases show little visible response to a lane-level navigation intervention in category L, an incorrect route choice in category A, and continued travel along the current road in category B2.
Autonomous driving develops vehicles that perceive road scenes, predict interactions, and plan and control safe movement with limited or no human input.