Multi-modal planning is promising for autonomous driving by representing multiple plausible behaviors in ambiguous and long-tail scenarios. Existing methods mainly focus on improving trajectory multi-modality, enhancing trajectory representations, or reshaping the candidate distribution. Nevertheless, we identify a pronounced generation-evaluation asymmetry in multi-modal planning: despite strong oracle performance, existing planners often fail to reliably select the best available candidate, leaving substantial planning potential unrealized. To address this challenge, we propose iDriveVLA, a multi-modal planning framework that improves the candidate trajectory space while enabling more reliable and context-aware trajectory evaluation. Specifically, iDriveVLA introduces a unified trajectory evaluator comprising a Safety-aware Scorer for quality and risk estimation, together with a VLM-guided Modulator for scene-adaptive criterion weighting. We further develop an oracle-aligned progressive training strategy consisting of candidate imitation pretraining, candidate space refinement, and semantic ranking alignment. On the public NAVSIM v1 leaderboard, iDriveVLA achieves a new state-of-the-art performance of 94.95 PDMS, surpassing the human-expert reference.
Figures & tables
Figure 1: Core idea. (a) Multiple plausible trajectories may coexist in challenging scenes, making reliable trajectory selection critical. (b) Increasing trajectory diversity improves oracle performance but also introduces more competing behaviors for the evaluator, revealing a generation–evaluation asymmetry in multi-modal planning. (c) This asymmetry leaves substantial oracle potential untapped in the strong multi-modal planner DrivoR ( Kirby et al., 2026 ) . In contrast, iDriveVLA improves trajectory evaluation, substantially narrows the oracle gap, and surpasses the human-expert reference on NAVSIM v1 ( Dauner et al., 2024 ) .
Figure 2: PDMS distributions of the selected and oracle candidates from the base planner. The oracle distribution is strongly concentrated near high PDMS, whereas the selected candidates exhibit a substantially heavier low-score tail.
Figure 3: Overview of the iDriveVLA framework. The candidate generator produces multiple trajectory hypotheses from multi-view observations and the ego state. Our evaluator consists of a safety-aware Scorer and a VLM-guided Modulator. The Scorer predicts criterion-wise planning scores and safety risks, while the Modulator generates scene-adaptive criterion weights. Unsafe candidates are filtered before score aggregation, and the highest-scoring trajectory is selected. The framework is progressively optimized through three training stages. Flame and snowflake icons denote trainable and frozen modules, respectively.
Table 2: Closed-loop evaluation on NAVSIM v2. All results are computed using the original NAVSIM v2 evaluation code without the human-behavior filtering fix.
Components
Planning Metrics
Δ PDMS
Safety Gate
Refinement
Modulator
NC ↑
DAC ↑
TTC ↑
Comf. ↑
EP ↑
PDMS ↑
Base Planner
99.0
98.9
96.7
100.0
90.0
93.7
–
✓
99.1
98.9
96.7
100.0
91.1
93.9
+0.2
✓
✓
99.0
99.0
96.6
100.0
91.3
94.2
+0.5
✓
✓
✓
99.1
99.1
96.6
100.0
91.6
94.4
+0.7
Table 3: Ablation study of the proposed components on NAVSIM v1. Safety Gate denotes the Safety-aware Scorer, Refinement denotes candidate space refinement fine-tuning, and Modulator denotes the VLM-guided Modulator. All variants are trained exclusively on NAVSIM data.
Figure 4: Comparison between DrivoR and iDriveVLA. Red and green curves denote the DrivoR and iDriveVLA trajectories, respectively. The paired camera and bird’s-eye-view panels show how the predicted paths differ under varied road geometry and surrounding traffic.
Figure 5: Qualitative comparison of trajectory selection without and with the VLM-guided Modulator. Red and green curves denote the selected trajectories without and with the Modulator, respectively. The Modulator changes the ranking of existing candidates according to the surrounding road structure and traffic context.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Configuration
Value
Candidate number K
64
Future planning steps T
8
Planning time interval
0.5 s
Visual registers per view
16
Trajectory decoder layers
4
Scorer layers
4
Appendix
Table 4: Model and training configurations.
Module
Time (ms)
Parameter memory (MiB)
Vision Encoder
97.73
90.53
Action Decoder
1.25
41.74
Scorer
3.40
7.15
Safety Head
0.20
2.02
Modulator
194.62
3999.51
Total
297.20
4140.95
Appendix
Table 5: Computational and memory overhead of the iDriveVLA inference pipeline.
Figure 6: Representative catastrophic selection failures of DrivoR. Yellow curves denote the candidate trajectories, while red and green curves denote the selected and oracle trajectories, respectively. Despite the presence of a safe oracle trajectory, the evaluator selects an unsafe candidate with zero PDMS.
Figure 7: Qualitative comparison between the trajectory selected by iDriveVLA and the oracle trajectory. The selected trajectories closely follow or overlap with the oracle trajectories, demonstrating reliable identification of high-quality candidates.
Institute for Intelligent Systems, Esslingen University of Applied Sciences, Germany. · Department of Computer Science, University of Freiburg, Germany.