DVGT-2: Vision-Geometry-Action Model for Autonomous Driving at Scale

Sicheng Zuo; Zixun Xie; Wenzhao Zheng; Shaoqing Xu; Fang Li; Hanbing Li; Long Chen; Zhi-Xin Yang

cs.CVApr 1, 2026

DVGT-2: Vision-Geometry-Action Model for Autonomous Driving at Scale

Authors: Sicheng Zuo, Zixun Xie, Wenzhao Zheng, Shaoqing Xu, Fang Li, Hanbing Li, Long Chen, Zhi-Xin Yang, +1 more

Organizations: Tsinghua University · Xiaomi EV · Peking University · University of Macau

Abstract

End-to-end autonomous driving has evolved from the conventional paradigm based on sparse perception into vision-language-action (VLA) models, which focus on learning language descriptions as an auxiliary task to facilitate planning. In this paper, we propose an alternative Vision-Geometry-Action (VGA) paradigm that advocates dense 3D geometry as the critical cue for autonomous driving. As vehicles operate in a 3D world, we think dense 3D geometry provides the most comprehensive information for decision-making. However, most existing geometry reconstruction methods (e.g., DVGT) rely on computationally expensive batch processing of multi-frame inputs and cannot be applied to online planning. To address this, we introduce a streaming Driving Visual Geometry Transformer (DVGT-2), which processes inputs in an online manner and jointly outputs dense geometry and trajectory planning for the current frame. We employ temporal causal attention and cache historical features to support on-the-fly inference. To further enhance efficiency, we propose a sliding-window streaming strategy and use historical caches within a certain interval to avoid repetitive computations. Despite the faster speed, DVGT-2 achieves superior geometry reconstruction performance on various datasets. The same trained DVGT-2 can be directly applied to planning across diverse camera configurations without fine-tuning, including closed-loop NAVSIM and open-loop nuScenes benchmarks.

Figures & tables

Figure 1: DVGT-2 is a streaming visual geometry transformer specifically designed for autonomous driving . It inputs multi-view images and jointly predicts the 3D pointmaps, the ego poses, and future trajectory planning in an online manner.

Figure 2: Comparison of different paradigms for end-to-end autonomous driving . Conventional end-to-end models rely on sparse perception representations for scene understanding. VLA models predict language descriptions to interpret driving scenarios. Our VGA model reconstructs dense 3D geometry to facilitate safe planning.

Figure 3: Comparison of different paradigms for geometry reconstruction . Batch-processing models like DVGT [ 93 ] compute pair-wise relations across all frames, incurring an overall $\mathcal{O}(T^{2})$ complexity. Full-history streaming models like StreamVGGT [ 92 ] extract temporal cues from the entire history, leading to an $\mathcal{O}(T)$ per-frame complexity. In contrast, our sliding-window streaming strategy attends to a fixed-size cache of length $W$ , achieving a constant $\mathcal{O}(W)$ per-frame complexity.

Figure 4: Overall archetecture of DVGT-2 . Our model consists of an image encoder, a geometry transformer with temporal causal attention, and a set of prediction heads to jointly output geometry reconstruction and trajectory planning.

Figure 5: Efficient inference of DVGT-2 . Given the current frame multi-view input and the cache of past $W$ frames, our model performs efficient geometry reconstruction and trajectory planning in an online manner, avoiding recomputing historical frames.

Method	Paradigm	Acc $\downarrow$	Comp $\downarrow$	Abs Rel $\downarrow$	$\delta$ < $1.25\uparrow$	AUC $\uparrow$	Time
VGGT* [ 66 ]	Full-Seq.	1.705	1.711	0.280	0.669	74.4	$\sim$ 5.31s
MapAnything [ 29 ]	Full-Seq.	3.269	4.214	0.476	0.253	69.5	$\sim$ 2.28s
DVGT [ 93 ]	Full-Seq.	0.412	0.491	0.048	0.971	76.6	$\sim$ 1.88s
CUT3R* [ 67 ]	Streaming	1.858	2.245	0.275	0.596	36.9	$\sim$ 0.35 s
StreamVGGT* [ 92 ]	Streaming	2.209	2.060	0.303	0.620	74.1	$\sim$ 1.94s
Driv3R* [ 7 ]	Streaming	0.884	1.693	0.188	0.740	-	$\sim$ 0.56s

Table 1: Quantitative 3D geometry reconstruction results on OpenScene [ 5 ] . * denotes performing post-alignment with sparse LiDAR to recover the metric scale.

Method	Paradigm	Acc $\downarrow$	Comp $\downarrow$	Abs Rel $\downarrow$	$\delta$ < $1.25\uparrow$	AUC $\uparrow$
VGGT* [ 66 ]	Full-Seq.	1.944	2.071	0.348	0.573	83.9
MapAnything [ 29 ]	Full-Seq.	4.447	4.860	0.562	0.271	84.9
DVGT [ 93 ]	Full-Seq.	0.469	0.508	0.067	0.955	86.4
CUT3R* [ 67 ]	Streaming	2.055	2.611	0.330	0.547	44.5
StreamVGGT* [ 92 ]	Streaming	2.414	2.284	0.375	0.563	86.0
Driv3R* [ 7 ]	Streaming	0.742	1.345	0.189	0.721	–

Table 2: Quantitative 3D geometry reconstruction results on nuScenes [ 1 ] .

Method	Paradigm	Acc $\downarrow$	Comp $\downarrow$	Abs Rel $\downarrow$	$\delta$ < $1.25\uparrow$	AUC $\uparrow$
VGGT* [ 66 ]	Full-Seq.	3.202	3.390	0.333	0.567	84.3
MapAnything [ 29 ]	Full-Seq.	9.903	8.365	0.492	0.217	82.3
DVGT [ 93 ]	Full-Seq.	1.752	2.263	0.102	0.922	86.0
CUT3R* [ 67 ]	Streaming	3.369	4.192	0.290	0.563	50.9
StreamVGGT* [ 92 ]	Streaming	3.207	2.960	0.302	0.604	85.6
Driv3R* [ 7 ]	Streaming	0.800	1.311	0.168	0.770	–

Table 3: Quantitative 3D geometry reconstruction results on Waymo [ 62 ] .

Method	Paradigm	Acc $\downarrow$	Comp $\downarrow$	Abs Rel $\downarrow$	$\delta$ < $1.25\uparrow$	AUC $\uparrow$
VGGT* [ 66 ]	Full-Seq.	2.322	2.879	0.798	0.395	86.3
MapAnything [ 29 ]	Full-Seq.	7.442	7.928	1.836	0.207	87.4
DVGT [ 93 ]	Full-Seq.	0.751	1.017	0.145	0.848	95.3
CUT3R* [ 67 ]	Streaming	2.827	4.724	0.875	0.317	48.6
StreamVGGT* [ 92 ]	Streaming	2.655	2.731	0.810	0.419	91.9
Driv3R* [ 7 ]	Streaming	0.950	1.259	0.185	0.740	–

Table 4: Quantitative 3D geometry reconstruction results on DDAD [ 11 ] .

Method	Input	Aux. Sup.	NC $\uparrow$	DAC $\uparrow$	TTC $\uparrow$	Comf. $\uparrow$	EP $\uparrow$	PDMS $\uparrow$
PARA-Drive [ 74 ]	C	Map & Mot. & Occ	97.9	92.4	93.0	99.8	79.3	84.0
VADv2 [ 3 ]	C	Map & Mot. & Traffic	97.2	89.1	91.6	100	76.0	80.9
UniAD [ 16 ]	C	Map & Box & Mot. & Occ	97.8	91.9	92.9	100	78.8	83.4
Transfuser [ 4 ]	C & L	Map & Box	97.7	92.8	92.8	100	79.2	84.0
Hydra-MDP [ 38 ]	C & L	Map & Box	98.3	96.0	94.6	100	78.7	86.5
GoalFlow [ 76 ]	C & L	Map & Box	98.3	93.8	94.3	100	79.8	85.7

Table 5: Closed-loop planning results on NAVSIM v1 navtest split .

\dagger

denotes using reinforcement learning to boost planning scores. Future states represent world-modeling-based methods. C and L denote camera and LiDAR, respectively.

Method	NC $\uparrow$	DAC $\uparrow$	DDC $\uparrow$	TL $\uparrow$	EP $\uparrow$	TTC $\uparrow$	LK $\uparrow$	HC $\uparrow$	EC $\uparrow$	EPDMS $\uparrow$
Ego Status MLP	93.1	77.9	92.7	99.6	86.0	91.5	89.4	98.3	85.4	64.0
Transfuser [ 4 ]	96.9	89.9	97.8	99.7	87.1	95.4	92.7	98.3	87.2	76.7
Hydra-MDP++ [ 33 ]	97.2	97.5	99.4	99.6	83.1	96.5	94.4	98.2	70.9	81.4
DriveSuprim [ 82 ]	97.5	96.5	99.4	99.6	88.4	96.6	95.5	98.3	77.0	83.1
ARTEMIS [ 8 ]	98.3	95.1	98.6	99.8	81.5	97.4	96.5	98.3	-	83.1
DiffusionDrive [ 41 ]	98.2	95.9	99.4	99.8	87.5	97.3	96.8	98.3	87.7	84.5

Table 6: Closed-loop planning results on NAVSIM v2 navtest split .

Figure 6: Qualitative visualizations . These results demonstrate that DVGT-2 can predict high-fidelity dense scene geometry and perform robust trajectory planning.

Method	Input	Aux. Sup.	L2 (m) $\downarrow$				Collision Rate (%) $\downarrow$
Method	Input	Aux. Sup.	1s	2s	3s	Avg.	1s	2s	3s	Avg.
IL [ 57 ]	LiDAR	None	0.44	1.15	2.47	1.35	0.08	0.27	1.95	0.77
NMP [ 83 ]	LiDAR	Box & Motion	0.53	1.25	2.67	1.48	0.04	0.12	0.87	0.34
FF [ 14 ]	LiDAR	Freespace	0.55	1.20	2.54	1.43	0.06	0.17	1.07	0.43
EO [ 30 ]	LiDAR	Freespace	0.67	1.36	2.78	1.60	0.04	0.09	0.88	0.33
ST-P3 [ 15 ]	Camera	Map & Box & Depth	1.33	2.11	2.90	2.11	0.23	0.62	1.27	0.71

Table 7: Open-looped planning results on nuScenes [ 1 ] .

\dagger

denotes the results computed with an average of previous frames as adopted in VAD [ 26 ] . Aux. Sup. represents auxiliary supervision. Avg. computes the average result of 1s, 2s, and 3s.

Panel of Figure 7 — Figure 7: Efficiency comparison of online inference . We report the per-frame latency and memory cost of different methods on multi-frame, 8-view sequences.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Table 15

Method	Paradigm	Acc $\downarrow$	Comp $\downarrow$	Abs Rel $\downarrow$	$\delta$ < $1.25\uparrow$	AUC@30 $\uparrow$
VGGT* [ 66 ]	Full-Seq.	1.154	1.294	0.158	0.801	96.91
MapAnything [ 29 ]	Full-Seq.	1.807	1.006	0.184	0.727	90.51
DVGT [ 93 ]	Full-Seq.	0.846	1.468	0.136	0.849	87.63
CUT3R* [ 67 ]	Streaming	0.973	2.054	0.217	0.660	51.82
StreamVGGT* [ 92 ]	Streaming	3.393	2.181	0.365	0.467	95.57
Driv3R* [ 7 ]	Streaming	0.864	1.083	0.164	0.784	-

Appendix

Table A.3: Quantitative 3D geometry reconstruction results on KITTI [ 10 ] . * denotes performing post-alignment with sparse LiDAR to recover the metric scale.

Dataset	Train Scenes	Test Scenes	Min Frames	Max Frames	Avg Frames	Avg Aspect Ratio	Num of Views
nuScenes	700	150	32	41	40	1.77	6
OpenScene	19376	2026	1	41	38	1.77	8
Waymo	798	202	34	40	40	1.77	5
KITTI	138	13	2	1033	62	3.31	2
DDAD	150	50	10	20	17	1.59	6

Appendix

Table A.4 : Detailed statistics of the datasets used in our experiments. All temporal statistics are reported at a 2Hz sampling rate.

Explore similar work

Explicit Geometric Chain-of-Thought for Vision-Language-Action in Autonomous Driving
Oct 7, 2026Xingtai Gui, Yucheng Zhou, Dongqian Guo +3Autonomous Driving Planning Reasoning for VLA Models
SKL-IOTSC, Department of AI, University of Macau · Afari Intelligent Drive
VLGA: Vision-Language-Geometry-Action Models for Autonomous Driving
Jun 10, 2026Jin Yao, Dhruva Dixith Kurra, Tom Lampo +3VLMs for Autonomous Driving End-to-End Autonomous Driving
Uber AV Labs · University of Virginia
Geo-VLA: Geometry-Aware Vision-Language-Action Planning via Internalization of Map Semantics
Aug 18, 2026Ran Chen, Jiaxing Ren, Zhikun Zhang +3Vision-Language-Action Models Autonomous Driving Planning
University of Science and Technology Beijing Beijing, China

Oct 7, 2026J/K move · Enter open · S save

Explicit Geometric Chain-of-Thought for Vision-Language-Action in Autonomous Driving

Xingtai Gui, Yucheng Zhou, Dongqian Guo, Jiahao Gong+2

SKL-IOTSC, Department of AI, University of Macau · Afari Intelligent Drive

Autonomous Driving Planning Reasoning for VLA Models

Open paper AskPDF arXiv

Vision-language-action~(VLA) models have emerged as a promising paradigm for autonomous driving. However, existing VLA models still suffer from a fundamental mismatch: driving actions require precise 3D geometric cues, while visual-language understanding and reasoning are largely conducted in a 2D semantic space. In this paper, we propose GeoCoTDrive, an explicit geometric chain-of-thought framework that grounds geometry in a planning-oriented manner. GeoCoTDrive follows a think with 2D first, drive with dedicated 3D priors paradigm. It first grounds 2D regions corresponding to decision-critical cues, and then retrieves localized 3D priors by sampling features from a geometric foundation model within the grounded regions. These localized geometric features are interleaved into the autoregressive context to support the trajectory generation. To supervise this process, we introduce planning-relevant grounding, a new region-level grounding task that focuses on local spatial cues directly affecting ego planning decisions, and construct the PlanningGrounding dataset to endow VLAs with planning-oriented grounding capability. Experiments across multiple end-to-end autonomous driving benchmarks show that GeoCoTDrive consistently improves safety-critical planning performance, demonstrating the effectiveness of the explicit geometric chain-of-thought process for VLA-based planning.

Abstract

Explore similar work

Explicit Geometric Chain-of-Thought for Vision-Language-Action in Autonomous Driving

VLGA: Vision-Language-Geometry-Action Models for Autonomous Driving

Geo-VLA: Geometry-Aware Vision-Language-Action Planning via Internalization of Map Semantics