Organizations: College of Transportation, Tongji University · Department of Computer Science, UNC Chapel Hill · Department of Control Science and Engineering, Tongji University
End-to-end autonomous driving increasingly depends on scaling training demonstrations, yet data growth does not guarantee proportional gains. However, additional data do not guarantee better policies: a demonstration that benefits one planner may be redundant or harmful to another. Existing curation methods typically infer data value from intrinsic sample properties or training-dependent proxies that do not measure downstream policy utility. To address this, we introduce Policy-Aware Data Valuation (PAD), a framework that assigns each candidate demonstration a utility specific to the target driving policy. PAD combines scene-level calibration feedback with projected policy gradients and a regularized approximation of local training geometry, producing scalable influence-based values for the policy. A controlled group-removal study establishes the directional fidelity of these values, while cross-planner analysis shows that the same candidate pool induces markedly different rankings across planner families. On DiffusionDrive, PAD improves EPDMS over real-only training by 3.5 points on NavTest and 5.0 points on NavHard. Policy-specific curation consistently outperforms real-only and full-pool training across diffusion-, regression-, and flow-matching-based planners while substantially reducing scene-level regressions. These results establish policy-aware data valuation as a basis for replacing indiscriminate data accumulation with targeted policy improvement, enabling more efficient and reliable scaling of autonomous-driving systems.
Figures & tables
Figure 1: Overview of PAD, which combines scene-level advantages from an independent calibration split with projected planning gradients and local training geometry to estimate the planner-relative influence of each demonstration, enabling budgeted selection of beneficial data for retraining.
Figure 2: Removal-based validation of ψ . Each point measures the EPDMS change after removing one score-ranked group and retraining the planner.
Table 1: Performance on NavTest Leaderboard. PAD retains top-ranked 25% of the candidate pool.
Method
Data Source
Stage
NC ↑
DAC ↑
DDC ↑
TLC ↑
EP ↑
TTC ↑
LK ↑
HC ↑
EC ↑
EPDMS ↑
Vanilla
Real
S 1
96.6
85.6
99.0
99.3
84.4
95.3
95.1
97.6
81.8
27.9
S 2
80.5
72.2
84.8
98.7
86.7
76.7
45.9
95.3
71.8
SimScale
Real+Syn@PDM
S 1
97.9
89.6
99.6
99.3
83.8
96.9
98.2
97.6
76.4
31.7
S 2
84.0
68.7
92.7
98.0
93.3
77.3
63.5
95.2
40.1
SimScale
Real+Syn@Full
S 1
96.8
87.1
99.3
99.8
84.4
96.7
98.4
97.6
78.2
31.3
S 2
84.2
70.8
90.8
98.6
91.2
77.3
61.4
96.0
39.2
Table 2: Performance on NavHard Leaderboard. Per-stage metrics are reported for both stages, and EPDMS denotes the combined score. † AutoScale results are taken from the original paper, as its code and model weights are not publicly available.
Figure 3: Policy-specific PAD values for the same synthetic candidate pool across DiffusionDrive, LTF, and MeanFuser. Values are standardized within each policy.
Figure 4: Scene-level regression exposure on NavTest.
Table 3: NavTest performance across policy families and selection budgets. PAD independently estimates ψ for each policy and retains the top-ranked 25% , 50% , or 75% of the common candidate pool.
Figure 5: Representative NavTest trajectories across policy families. Full-pool synthetic training produces zero-score failures, whereas PAD retains feasible, higher-scoring predictions.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Policy
Trajectory Generation
Attribution Objective
Randomness
Attributed Parameters
Excluded Auxiliary Objectives
DiffusionDrive
Noisy-anchor multimodal refinement
Et,ϵ[Lfocal+L1]
t,ϵ
Trajectory head
Agent detection; BEV semantics
LTF
Direct trajectory regression
wtrajL1
None
Trajectory head
Agent detection; BEV semantics
MeanFuser
MeanFlow proposals followed by ARM reconstruction
Ee,r,t[7LMF+2LARM]
e,r,t
MeanFlow and ARM heads
BEV semantics
Appendix
Table 4: Policy-specific instantiations of the shared PAD valuation interface. Auxiliary losses are excluded only during attribution; final curated retraining retains each policy’s complete native objective.
Configuration
NC ↑
DAC ↑
DDC ↑
TLC ↑
EP ↑
TTC ↑
LK ↑
HC ↑
EC ↑
EPDMS ↑
95% CI
Full pool
97.65
95.01
99.47
99.78
96.86
96.98
97.08
98.20
69.03
84.31
–
Remove G0
+0.15
+0.26
+0.03
+0.07
−0.24
+0.04
+0.43
+0.03
+4.84
+0.70
[+0.24,+1.16]
Remove G1
+0.78
+0.72
+0.09
+0.02
−0.82
+0.58
−0.12
+0.12
+4.47
+1.41
[+0.97,+1.85]
Remove G2
+0.02
+0.01
+0.06
+0.00
+0.04
−0.41
−0.43
+0.01
+0.53
−0.05
[−0.52,+0.42]
Remove G3
+0.42
+0.17
−0.30
+0.05
−1.03
+0.68
+0.33
+0.12
−5.72
−0.40
[−0.86,+0.07]
Remove G4
−0.15
+0.35
+0.14
+0.02
+0.00
−0.40
−0.15
+0.05
−14.95
−1.30
[−1.76,−0.86]
Appendix
Table 5: Component-wise NavTest effects of value-ranked group removal. The full-pool row reports absolute scores; removal rows report changes relative to that reference and paired bootstrap confidence intervals for Δ EPDMS.
Table 6: NavHard performance across policy families and selection budgets. PAD recomputes demonstration values for each policy. Stage-specific metrics are reported for both evaluation stages; EPDMS is the combined score.
Setting
Diffusion- Drive
LTF
MeanFuser
Policy configuration
Cameras
3
3
3
Resolution
2048×512
2048×512
2048×512
Backbone
R34
R34
R34
Horizon
4 s
4 s
4 s
Frequency
2 Hz
2 Hz
2 Hz
Appendix
Table 7: Policy and valuation configurations. Nh is the geometry-set size, and K is the Monte Carlo sample count per demonstration.
Setting
Diffusion- Drive
LTF
MeanFuser
Optimizer
AdamW
Adam
AdamW
Initial learning rate
6×10−4
10−4
10−4
Schedule
Warmup cosine
Constant
Warmup cosine
Warmup epochs
3
–
3
Minimum learning rate
10−6
–
10−7
Weight decay
10−4
0
0
Appendix
Table 8: Full-policy training configurations. Batch size is reported per GPU.
Figure 6: Cross-policy improvements on two representative NavTest scenes. Within each scene, rows show DiffusionDrive, LTF, and MeanFuser under real-only training (Vanilla, black) and PAD curation (blue).
Figure 7: Representative failures induced by full-pool synthetic training. Rows show two scenes per policy. Black, orange, and blue denote the real-only, full-pool SimScale, and PAD trajectories, respectively.
Figure 8: Component-wise effects of PAD. Black and blue denote the real-only and PAD trajectories. Metric arrows report the change from real-only training to PAD. Across policies, PAD improves safety, drivable-area compliance, lane keeping, or progress without degrading the paired component shown in each row.
Figure 9: Representative unresolved failures for DiffusionDrive, LTF, and MeanFuser. Black and blue denote the real-only and PAD trajectories. Both variants fail the indicated time-to-collision, traffic-rule, or drivable-area criterion.
Data selection chooses informative, representative, or reliable examples to improve model training and evaluation while reducing data, compute, or labeling costs.
Large-scale deep learning models for physical AI applications depend on diverse training data collection efforts. These models and correspondingly, the training data, must address different evaluation criteria necessary for the models to be deployable in real-world environments. Data selection policies can guide the development of the training set, but current frameworks do not account for the ambiguity in how data points affect different metrics. In this work, we propose Mixture Optimization via Scaling-Aware Iterative Collection (MOSAIC), a general data selection framework that operates by: (i) partitioning the dataset into domains; (ii) fitting neural scaling laws from each data domain to the evaluation metrics; and (iii) optimizing a data mixture by iteratively adding data from domains that maximize the change in metrics. We apply MOSAIC to autonomous driving (AD), where an End-to-End (E2E) planner model is evaluated on the Extended Predictive Driver Model Score (EPDMS), an aggregate of driving rule compliance metrics. Here, MOSAIC outperforms a diverse set of baselines on EPDMS with up to 80% less data.
Tolga Dimlioglu, Nadine Chang, Maying Shen +2
New York University · NVIDIA · University of Ottawa
Autonomous driving has shifted towards end-to-end policy learning, where reliable, interpretable policy evaluation is a fundamental challenge as driving quality is highly context-dependent. Commonly used rule-based driving metrics like EPDMS are interpretable but lack context-awareness, while recent VLMbased evaluations are context-aware but limited by ambiguous VLM outputs and weak physical grounding. To evaluate driving in a manner that is both interpretable and context-aware, we introduce DriveJudge. DriveJudge is a driving evaluation agent that combines rule-grounded evaluation with Vision-Language Model (VLM) reasoning and selectively invokes physically-grounded deterministic rule functions after interpreting the environmental context. To train and evaluate DriveJudge, we curate a large-scale dataset of 33,577 challenging driving samples with human annotations on whether the driving behavior is reasonable in the given scenario. With this dataset, we address the underexplored problem of driving metric evaluation, and introduce two human-aligned benchmark tasks: Driving Quality Classification and Trajectory Preference Selection. DriveJudge outperforms EPDMS for driving quality classification by 21.23 AUC, and the recent VLM-based DriveCritic for trajectory preference selection by 6.5%, setting a new standard for interpretable and precise driving evaluation.
End-to-end autonomous driving models are typically trained on real-world driving logs, which record only the trajectory actually executed in each scene. This provides limited supervision for learning alternative valid behaviors. Multi-target supervision can mitigate this limitation, but broader trajectory coverage makes safe candidate selection more challenging. We propose FPD-Drive, an end-to-end planner that distills privileged future knowledge into a visual driving policy to improve both multi-trajectory supervision and safety selection. Specifically, we introduce Future-Privileged Multi-Target Distillation to transfer diverse, high-quality trajectory targets from a teacher with access to recorded future observations to the student. These targets extend supervision beyond the single recorded human trajectory. For candidate selection, we further introduce Safety-Aware Score Correction to correct ranking errors between nearby safe and unsafe candidates and raise underestimated safety scores of safe candidates. At deployment, the student requires neither future observations nor teacher inference. Our base model achieves 94.9 PDMS on NAVSIM v1 navtest and 55.9 EPDMS on NAVSIM v2 navhard. The scaled FPD-Drive-Pro improves these scores to 95.5 PDMS and 58.1 EPDMS, respectively, achieving SOTA performance among the compared methods on both benchmarks. Zero-shot closed-loop evaluation on HUGSIM further yields an HD-Score of 45.1 and RC of 55.3.
Li-Heng Chen, Qifeng Chen, Chengwei Wu +4
NIO · The Hong Kong University of Science and Technology · University of Science and Technology of China