FPD-Drive: Future Policy Distillation for End-to-End Autonomous Driving

Li-Heng Chen; Qifeng Chen; Chengwei Wu; Yingjie Zhang; Sheng Yang; Hongbo Fu; Shaoqing Ren

cs.CVOct 9, 2026

FPD-Drive: Future Policy Distillation for End-to-End Autonomous Driving

Authors: Li-Heng Chen, Qifeng Chen, Chengwei Wu, Yingjie Zhang, Sheng Yang, Hongbo Fu, Shaoqing Ren

Organizations: NIO · The Hong Kong University of Science and Technology · University of Science and Technology of China

Abstract

End-to-end autonomous driving models are typically trained on real-world driving logs, which record only the trajectory actually executed in each scene. This provides limited supervision for learning alternative valid behaviors. Multi-target supervision can mitigate this limitation, but broader trajectory coverage makes safe candidate selection more challenging. We propose FPD-Drive, an end-to-end planner that distills privileged future knowledge into a visual driving policy to improve both multi-trajectory supervision and safety selection. Specifically, we introduce Future-Privileged Multi-Target Distillation to transfer diverse, high-quality trajectory targets from a teacher with access to recorded future observations to the student. These targets extend supervision beyond the single recorded human trajectory. For candidate selection, we further introduce Safety-Aware Score Correction to correct ranking errors between nearby safe and unsafe candidates and raise underestimated safety scores of safe candidates. At deployment, the student requires neither future observations nor teacher inference. Our base model achieves 94.9 PDMS on NAVSIM v1 navtest and 55.9 EPDMS on NAVSIM v2 navhard. The scaled FPD-Drive-Pro improves these scores to 95.5 PDMS and 58.1 EPDMS, respectively, achieving SOTA performance among the compared methods on both benchmarks. Zero-shot closed-loop evaluation on HUGSIM further yields an HD-Score of 45.1 and RC of 55.3.

Figures & tables

Figure 1: Overview of FPD-Drive. The teacher–student framework employs a common coarse-to-fine architecture comprising a visual encoder, a proposal decoder, a diffusion refiner, and a scorer (Sec. 3.1 ). Recorded future observations additionally guide the teacher’s refinement and scoring (Sec. 3.2 ). Future-Privileged Multi-Target Distillation transfers high-quality, geometrically diverse trajectories selected from evaluated teacher candidates to supervise the student’s proposal decoder and refiner (Sec. 3.3 ). Safety-Aware Score Correction uses evaluated student candidates to penalize highly ranked unsafe trajectories relative to nearby safe alternatives and increase confidence in underestimated safe candidates (Sec. 3.4 ). Only the student is used at deployment.

Figure 2: Qualitative comparison on NAVSIM v1 navtest . We show the metrics for each scene, with PDMS and inference latency as mean/p99 in milliseconds.

Entry	Venue	NAVSIM v1					PDMS $\uparrow$
Entry	Venue	NC $\uparrow$	DAC $\uparrow$	EP $\uparrow$	TTC $\uparrow$	C $\uparrow$	PDMS $\uparrow$
Human Driver	–	100	100	87.5	100	99.9	94.8
MeanFuser	CVPR 26	98.6	97.0	82.8	95.0	100.0	89.0
ReCogDrive	ICLR 26	97.9	97.3	87.3	94.9	100.0	90.8
WA-JEPA	arXiv 26.08	99.4	98.2	85.3	97.2	100.0	91.6
DriveFine	ECCV 26	98.8	99.2	86.9	96.2	100.0	91.8

Table 1: Comparison on the NAVSIM v1 navtest . Best overall PDMS is in bold , second-best is underlined .

Entry	Venue	Stage	NAVSIM v2									EPDMS $\uparrow$
Entry	Venue	Stage	NC $\uparrow$	DAC $\uparrow$	DDC $\uparrow$	TLC $\uparrow$	EP $\uparrow$	TTC $\uparrow$	LK $\uparrow$	HC $\uparrow$	EC $\uparrow$	EPDMS $\uparrow$
RAP	ICLR 26	1	97.1	94.4	98.8	99.8	83.9	96.9	94.7	96.4	66.2	39.6
RAP	ICLR 26	2	83.2	83.9	87.4	98.0	86.9	80.4	52.3	95.2	52.4	39.6
ZTRS	ECCV 26	1	98.9	97.6	100.0	100.0	66.7	98.9	96.2	96.7	44.0	48.1
ZTRS	ECCV 26	2	91.1	90.4	95.8	99.0	63.6	89.8	60.4	97.6	66.1	48.1
GuideFlow	CVPR 26	1	99.6	98.0	99.4	99.3	79.7	99.3	94.9	97.1	58.2	51.5

Table 2: Comparison on the NAVSIM v2 navhard . Best combined EPDMS is in bold , second-best is underlined .

Method	Venue	RC $\uparrow$					HD-Score $\uparrow$
Method	Venue	E	M	H	X	Avg.	E	M	H	X	Avg.
VAD	ICCV 23	51.3	31.1	25.3	26.5	31.4	36.3	9.5	8.0	11.5	13.4
LTF	TPAMI 22	67.8	35.1	26.2	40.5	38.9	58.9	18.0	9.8	25.9	23.7
UniAD	CVPR 23	78.4	60.5	33.6	17.8	45.9	64.9	45.8	20.6	6.6	32.7
DrivoR	CVPR 26	80.9	50.5	33.8	47.1	49.8	73.3	34.6	18.8	32.5	35.7
GigaPixel	arXiv 26.06	78.6	60.7	32.7	35.4	50.1	67.4	51.9	19.1	21.6	38.5

Table 3: Zero-shot closed-loop evaluation on HUGSIM. Best average scores are in bold , second-best are underlined . E/M/H/X denote Easy/Medium/Hard/Extreme. FPD-Drive reports preliminary results on 345 scenarios. Visualizations are shown in the supplementary video.

Model	Traj. Dec.	Ref. DiT	Global Scene Enc.	Future Token	Scorer	Distillation		PDMS
Model	Traj. Dec.	Ref. DiT	Global Scene Enc.	Future Token	Scorer	FPMT	SASC	PDMS
Teacher	✓	✗	✗	✗	✓	–	–	94.0 /	96.7
	✓	✗	✗	✓	✓	–	–	94.8 /	98.8
	✓	✓	✗	✓	✓	–	–	95.1 /	99.2
	✓	✓	✓	✓	✗	–	–	– /	98.8
	✓	✓	✓	✓	✓	–	–	95.6 /	99.6

Table 4: Ablations of FPD-Drive’s architecture and distillation components on NAVSIM v1. Teacher PDMS is reported as learned-scorer selection / PDM-Closed evaluator selection.

(a) Multi-target distillation
Target $K$	$\bar{J}$	PDMS $\uparrow$
1	1.0	94.6
2	1.9	94.7
4 (default)	3.1	94.9
6	3.3	94.9

Table 5: Ablations of FPD-Drive’s teacher-target supervision on NAVSIM v1.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Component	Student configuration
Visual token dim	$4\times 16\times 256$
Visual backbone	DINOv2 (ViT-S/14 vision transformer), frozen
Visual backbone adaptation	LoRA, rank 32
Cameras	Front, left, right, and rear
Input resolution	$1148\times 672$
Input history	Current frame only

Appendix

Table 6: Model configuration.

Navigation command group			Speed group
Condition	Original data (%)	Generated data (K)	Condition	Original data (%)	Generated data (K)
Left	25.36	60.6	2–5 m/s	32.40	55.9
Right	11.46	96.4	5–10 m/s	35.89	46.4
Straight	63.18	1.8	Other speeds	31.71	56.5

Appendix

Table 7: Data allocation used for the 158.8K generated scenes.

Figure 3: Training-data scaling of FPD-Drive.

Figure 4: Overview of the data generation pipeline.

Figure 5: Additional NAVSIM navtest scenarios.

Setting	Traj. Dec.	Ref. DiT	Scorer	PDMS $\uparrow$
Causal baseline (No Future Token)	$\times$	$\times$	$\times$	94.13
Decoder only	$\checkmark$	$\times$	$\times$	94.26
DiT only	$\times$	$\checkmark$	$\times$	95.29
Scorer only	$\times$	$\times$	$\checkmark$	94.79
Decoder + Scorer	$\checkmark$	$\times$	$\checkmark$	94.89
Decoder + DiT	$\checkmark$	$\checkmark$	$\times$	95.37

Appendix

Table 8: Ablation study on future-token injection in the privileged teacher.

Distillation strategy	PDMS $\uparrow$
EvoDrive-VLA	85.3
Ours w/ Token-level KL	94.5
FPD-Drive	94.9

Appendix

Table 9: Token-level versus multi-target distillation on NAVSIM v1.

Candidate source	Candidates	Stu. PDMS $\uparrow$
Predefined trajectory vocabularies	64	94.2
Future-privileged teacher (ours)	64	94.9

Appendix

Table 10: Ablation of distillation target sources on NAVSIM v1. Both variants supervise our student using up to

K=4

targets selected from 64 candidate trajectories per scene.

$\mathcal{L}_{\mathrm{neg}}$	$\mathcal{L}_{\mathrm{pos}}$	Unsafe cases (%) $\downarrow$	PDMS $\uparrow$
✗	✗	1.91	94.7
✓	✗	1.77	94.8
✗	✓	1.89	94.7
✓	✓	1.35	94.9

Appendix

Table 11: Ablation of the Components of SASC.

Denoising steps	2	4	6	8	12
PDMS $\uparrow$	94.7	94.7	94.8	94.9	94.9

Appendix

Table 12: Ablation of DiT denoising steps on NAVSIM v1.

End-to-End Autonomous Driving

End-to-end autonomous driving learns to turn sensor observations and driving context into vehicle actions or trajectories.

See all 227 papers

Drive-JEPA: Video JEPA Meets Multimodal Trajectory Distillation for End-to-End Driving
Jan 29, 2026Linhan Wang, Zichong Yang, Chen Bai +6End-to-End Autonomous Driving NavSim
Virginia Tech · Purdue University · XPENG Motors +1
Post-Training in End-to-End Autonomous Driving
Jul 9, 2026Ruining Yang, Muxing Wang, Yixiao Chen +8End-to-End Autonomous Driving Autonomous Driving
Northeastern University, Boston, USA · Purdue University, West Lafayette, USA
DriveZero: End-to-End Driving Beyond Human Demonstrations
Sep 5, 2026Hao He, Chengcheng Hu, Zirun Su +17End-to-End Autonomous Driving RL for Autonomous Driving
Xiaomi EV AD & Robotics, L3 Team

Jan 29, 2026J/K move · Enter open · S save

Drive-JEPA: Video JEPA Meets Multimodal Trajectory Distillation for End-to-End Driving

Linhan Wang, Zichong Yang, Chen Bai, Guoxiang Zhang+5

Virginia Tech · Purdue University · XPENG Motors+1

End-to-End Autonomous Driving NavSim Joint-Embedding Predictive Architecture Video Representation Learning Autonomous Driving Planning

Open paper AskPDF arXiv

End-to-end autonomous driving increasingly leverages self-supervised video pretraining to learn transferable planning representations. However, pretraining video world models for scene understanding has so far brought only limited improvements. This limitation is compounded by the inherent ambiguity of driving: each scene typically provides only a single human trajectory, making it difficult to learn multimodal behaviors. In this work, we propose Drive-JEPA, a framework that integrates Video Joint-Embedding Predictive Architecture (V-JEPA) with multimodal trajectory distillation for end-to-end driving. First, we adapt V-JEPA for end-to-end driving, pretraining a ViT encoder on large-scale driving videos to produce predictive representations aligned with trajectory planning. Second, we introduce a proposal-centric planner that distills diverse simulator-generated trajectories alongside human trajectories, with a momentum-aware selection mechanism to promote stable and safe behavior. When evaluated on NAVSIM, the V-JEPA representation combined with a simple transformer-based decoder outperforms prior methods by 3 PDMS in the perception-free setting. The complete Drive-JEPA framework achieves 93.3 PDMS on v1 and 87.8 EPDMS on v2, setting a new state-of-the-art.

Abstract

End-to-End Autonomous Driving

Explore similar work

Drive-JEPA: Video JEPA Meets Multimodal Trajectory Distillation for End-to-End Driving

Post-Training in End-to-End Autonomous Driving

DriveZero: End-to-End Driving Beyond Human Demonstrations