EvoDriveVLA: Evolving Driving VLA Models via Collaborative Perception-Planning Distillation

Jiajun Cao; Xiaoan Zhang; Xiaobao Wei; Liyuqiu Huang; Zijian Wang; Hanzhen Zhang; Zhengyu Jia; Wei Mao

cs.CVMar 10, 2026

EvoDriveVLA: Evolving Driving VLA Models via Collaborative Perception-Planning Distillation

Authors: Jiajun Cao, Xiaoan Zhang, Xiaobao Wei, Liyuqiu Huang, Zijian Wang, Hanzhen Zhang, Zhengyu Jia, Wei Mao,

Organizations: State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · XPeng Motors

Abstract

Vision-Language-Action models have shown great promise for autonomous driving, yet they suffer from degraded perception after unfreezing the visual encoder and struggle with accumulated instability in long-term planning. To address these challenges, we propose EvoDriveVLA-a novel collaborative perception-planning distillation framework that integrates self-anchored perceptual constraints and future-informed trajectory optimization. Specifically, self-anchored visual distillation leverages self-anchor teacher to deliver visual anchoring constraints, regularizing student representations via trajectory-guided key-region awareness. In parallel, future-informed trajectory distillation employs a future-aware oracle teacher with coarse-to-fine trajectory refinement and Monte Carlo dropout sampling to synthesize reasoning trajectories that model future evolutions, enabling the student model to internalize the future-aware insights of the teacher. EvoDriveVLA achieves SOTA performance in nuScenes open-loop evaluation and significantly enhances performance in NAVSIM closed-loop evaluation. Our code is available at: https://github.com/hey-cjj/EvoDriveVLA.

Figures & tables

Figure 1: Comparison of distillation methods for autonomous driving. (a) Single-Trajectory Distillation; (b) Multi-Trajectory Distillation; (c) Collaborative Perception-Planning Distillation (Ours).

Figure 2: Overview of the EvoDriveVLA framework. (Left) Self-anchored visual distillation imposes token-leve visual anchoring constraints across the scene; (Right) Future-informed trajectory distillation leverages future ground-truth information for trajectory refinement and diversity sampling; (Middle) Collaborative perception-planning distillation enhances autonomous driving VLA model capabilities in both perception and planning to achieve superior driving performance.

Method	ST-P3								UniAD
	L2 (m) $\downarrow$				Collision (%) $\downarrow$				L2 (m) $\downarrow$				Collision (%) $\downarrow$
	1s	2s	3s	Avg.	1s	2s	3s	Avg.	1s	2s	3s	Avg.	1s	2s	3s	Avg.
Traditional
ST-P3 [ECCV 22]	1.44	2.11	2.90	2.11	0.23	0.62	1.27	0.71	-	-	-	-	-	-	-	-
VAD [ICCV 23]	0.17	0.34	0.60	0.37	0.04	0.27	0.67	0.33	-	-	-	-	-	-	-	-
BEV-Planner [CVPR 24]	0.16	0.32	0.57	0.35	0.00	0.29	0.73	0.34	-	-	-	-	-	-	-	-

Table 1: Open-loop evaluation on nuScenes. We conduct open-loop evaluation on the nuScenes, comparing our method against traditional, LLM-based, and distillation-based baselines.

Method	NC $\uparrow$	DAC $\uparrow$	TTC $\uparrow$	Comf. $\uparrow$	EP $\uparrow$	PDMS $\uparrow$
Constant Velocity	68.0	57.8	50.0	100	19.4	20.6
Ego Status MLP	93.0	77.3	83.6	100	62.8	65.6
VADv2- $\mathcal{V}_{\text{8192}}$ [ICCV 23]	97.2	89.1	91.6	100	76.0	80.9
UniAD [CVPR 23]	97.8	91.9	92.9	100	78.8	83.4
TransFuser [TPAMI 23]	97.7	92.8	92.8	100	79.2	84.0
PARA-Drive [CVPR 24]	97.9	92.4	93.0	99.8	79.3	84.0

Table 2: Performance comparison on NAVSIM navtest using closed-loop metrics.

Ablation					L2 (m) $\downarrow$				Collision (%) $\downarrow$
Traj KD	Refine	MC-Drop	Frozen Enc	Visual KD	1s	2s	3s	Avg.	1s	2s	3s	Avg.
${\times}$	${\times}$	${\times}$	${\times}$	${\times}$	0.17	0.47	1.02	0.55	0.03	0.08	0.45	0.19
✓	${\times}$	${\times}$	${\times}$	${\times}$	0.17	0.46	1.00	0.54	0.05	0.07	0.38	0.17
✓	✓	${\times}$	${\times}$	${\times}$	0.16	0.46	0.99	0.53	0.05	0.03	0.40	0.16
✓	✓	✓	${\times}$	${\times}$	0.16	0.45	0.98	0.53	0.03	0.05	0.35	0.14
✓	✓	✓	✓	${\times}$	0.17	0.47	1.04	0.56	0.03	0.10	0.42	0.18

Table 3: Ablation study on algorithmic components.

Figure 3: (Left) : Kernel density estimation of trajectory loss distributions for pre-refine and post-refine trajectories. The overlaid boxplots summarize the median, interquartile range, and extreme values. (Right) : Comparison of trajectory loss distributions before and after MC-Dropout sampling.

Metric	L2 (m) $\downarrow$				Collision (%) $\downarrow$
Metric	1s	2s	3s	Avg.	1s	2s	3s	Avg.
ST-P3	0.10	0.14	0.18	0.14	0.02	0.03	0.05	0.04
UniAD	0.13	0.20	0.27	0.20	0.02	0.05	0.05	0.04

Table 4: Oracle teacher performance on nuScenes.

Figure 4: Qualitative comparison on nuScenes.

Methods	L2 (m) $\downarrow$				Collision (%) $\downarrow$
Methods	1s	2s	3s	Avg.	1s	2s	3s	Avg.
w/o	0.16	0.46	0.99	0.53	0.05	0.03	0.40	0.16
Parameter	0.17	0.46	1.01	0.54	0.07	0.07	0.38	0.17
Hidden State	0.16	0.44	0.97	0.52	0.03	0.03	0.30	0.12

Table 5: Ablation on MC-Dropout trajectory sampling.

Figure 5: Visualization of the visual anchors.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Weight Factors	L2 (m) $\downarrow$				Collision (%) $\downarrow$
Weight Factors	1s	2s	3s	Avg.	1s	2s	3s	Avg.
Traj KD Weights(Temp/Weight)
6/0.1	0.16	0.47	1.00	0.56	0.05	0.07	0.38	0.17
5/0.1	0.16	0.45	0.98	0.53	0.02	0.05	0.38	0.15
5/0.2	0.17	0.48	1.01	0.55	0.03	0.06	0.42	0.17
4/0.1	0.16	0.46	1.00	0.54	0.03	0.03	0.38	0.15

Appendix

Table 6: Weight factor sensitivity analysis.

Methods	ST-P3 L2 (m) $\downarrow$				UniAD L2 (m) $\downarrow$
Methods	1s	2s	3s	Avg.	1s	2s	3s	Avg.
Rainy/Night	0.12	0.23	0.38	0.24	0.16	0.41	0.76	0.43
Sunny	0.15	0.24	0.40	0.27	0.17	0.36	0.80	0.44
Rainy/Night+KD	0.10	0.21	0.35	0.22	0.13	0.39	0.68	0.40
Sunny+KD	0.15	0.22	0.37	0.25	0.17	0.36	0.78	0.44

Appendix

Table 7: Verification of perceptual robustness

End-to-End Autonomous Driving

End-to-end autonomous driving learns to turn sensor observations and driving context into vehicle actions or trajectories.

See all 225 papers

Teaching Vision-Language-Action Models What to See and Where to Look
Jul 2, 2026Yuguang Yang, Canyu Chen, Zhewen Tan +10End-to-End Autonomous Driving VLM Distillation
School of Electronic Information Engineering, Beihang University · Institute for AI Industry Research (AIR), Tsinghua University · National College for Excellent Engineers, Beihang University +5
RAF-VLA: Representation Alignment with the Future for End-to-End Autonomous Driving
Sep 15, 2026Dogun Kim, Yongjae Lee, Joonhee Lim +4End-to-End Autonomous Driving Vision-Language-Action Models
Graduate School of Mobility, Korea Advanced Institute of Science & Technology (KAIST), Daejeon, Korea · Robotics Program, Korea Advanced Institute of Science & Technology (KAIST), Daejeon, Korea
UniDriveVLA: Unifying Understanding, Perception, and Action Planning for Autonomous Driving
Apr 2, 2026Yongkang Li, Lijun Zhou, Sixu Yan +11Mixture-of-Transformers Vision-Language-Action Models
Huazhong University of Science and Technology · Xiaomi EV · SKL-IOTSC, University of Macau

Jul 2, 2026J/K move · Enter open · S save

Teaching Vision-Language-Action Models What to See and Where to Look

Yuguang Yang, Canyu Chen, Zhewen Tan, Yizhi Wang+9

School of Electronic Information Engineering, Beihang University · Institute for AI Industry Research (AIR), Tsinghua University · National College for Excellent Engineers, Beihang University+5

End-to-End Autonomous Driving VLM Distillation Vision-Language-Action Models Autonomous Driving Planning Vehicle Trajectory Prediction

Open paper AskPDF arXiv

Vision-Language-Action (VLA) models have emerged as a promising paradigm for end-to-end autonomous driving. However, existing VLAs' training relies heavily on text-centric visual question answering and chain-of-thought reasoning data, which emphasizes linguistic reasoning rather than action-grounded planning. As a result, the learned representations capture semantic knowledge but lack spatial dependencies crucial for reliable trajectory prediction. We propose DriveTeach-VLA, a framework that explicitly teaches VLAs what to see and where to look. Driving-aware Vision Distillation (DVD) injects driving-specific perceptual priors into the vision encoder, while 2D Trajectory-Guided Prompts (2D-TGP) provide spatial conditioning aligned with feasible driving trajectories. Together, they form a vision-guided learning pipeline: what to see (DVD pretraining) - where to look (TGP-guided SFT) - how to act (TGP-guided GRPO). DriveTeach-VLA achieves the state-of-the-art performance on NAVSIM and nuScenes. Our code is available at: https://github.com/ShivaTeam/DriveTeach-VLA.

Abstract

End-to-End Autonomous Driving

Explore similar work

Teaching Vision-Language-Action Models What to See and Where to Look

RAF-VLA: Representation Alignment with the Future for End-to-End Autonomous Driving

UniDriveVLA: Unifying Understanding, Perception, and Action Planning for Autonomous Driving