Organizations: AutoLab, School of Artificial Intelligence, Shanghai Jiao Tong University · Voyager Research, Didi Chuxing · The Chinese University of Hong Kong, Shenzhen
High-fidelity closed-loop simulation for autonomous driving increasingly relies on dense multi-view 3D reconstruction. While recent feed-forward geometric foundation models have revolutionized per-scene optimization, applying them to continuous, long-horizon driving sequences is computationally intractable due to GPU memory limits and the quadratic complexity of global attention. Existing scaling strategies are fundamentally limited, as causal streaming suffers from compounding scale drift and trajectory deviation, while rigid "chunk-then-align" heuristics fragment global context and produce trajectory discontinuities. To address these limitations, we introduce Drive3R, a block-wise streaming geometric foundation model. Its block-wise causal attention resolves the scalability bottleneck of fully bidirectional attention while avoiding the loss of local future context in purely causal streaming. Its confidence-aware re-anchoring further mitigates long-range drift by detecting unreliable streaming states, re-initializing the KV cache, and stitching refreshed segments through SE(3) alignment. Extensive experiments demonstrate that Drive3R establishes a new state-of-the-art on nuScenes, KITTI, and Waymo. Specifically, on nuScenes, it improves over the recent DVGT-2 baseline from 0.1806 to 0.1081 in AbsRel, from 0.6750 to 0.5344 in point-cloud Accuracy, and from 1.5812 to 0.9092 in ATE.
Figures & tables
Figure 1: Teaser of Drive3R. Compared with the per-frame streaming baseline DVGT-2 [ 34 ] , Drive3R reconstructs cleaner and more coherent long-horizon driving scenes with better preserved road geometry. The radar chart further shows state-of-the-art performance across depth, point-cloud, and ego-motion metrics on nuScenes [ 1 ] , Waymo [ 15 ] , and KITTI [ 5 ] .
Figure 2: Paradigm comparison between previous sequence processing strategies (offline, streaming, chunk-then-align) and our proposed block-wise streaming approach, Drive3R.
Figure 3: The pipeline of our method. Multi-view image streams are processed in a block-wise manner. Each image block is first encoded with intrinsic priors and then processed by a spatio-temporal decoder. By maintaining a streamed KV memory across consecutive blocks, the model continuously extracts consistent geometry features for multi-task predictions, including ego-centric motion and dense metric 3D reconstruction.
Model
Depth
Point Cloud
Ego-Motion
AbsRel ↓
δ<1.25↑
Acc. ↓
Comp. ↓
AUC@30 ↑
ATE ↓
Offline
VGGT* [ 19 ]
0.2386
0.7657
0.9033
1.4851
86.78
3.2108
π3x [ 22 ]
0.5005
0.3273
0.7513
1.5304
77.04
6.9732
DA3-Nested [ 10 ]
0.3970
0.3629
1.0020
2.0973
79.20
3.3885
DVGT [ 33 ]
0.4698
0.4183
0.9653
1.6218
85.58
3.6566
Table 1: Quantitative comparison on Waymo [ 15 ] under the 15-frame, full-view setting. Best results are bolded , and second best are underlined . * indicates models predicting relative scale that are aligned to recover the metric scale.
Figure 4: Qualitative comparison with per-frame streaming baselines. Red boxes highlight representative local regions where our method recovers cleaner geometry and sharper object structures.
Methods
KITTI [ 5 ] Stereo 2Hz (ATE ↓ )
Avg. ↓
00
01
02
03
04
05
06
07
08
09
10
T=909V=2 3.7km
T=221V=2 2.5km
T=933V=2 5.1km
T=161V=2 0.6km
T=55V=2 0.4km
T=553V=2 2.2km
T=221V=2 1.2km
T=221V=2 0.7km
T=815V=2 3.2km
T=319V=2 1.7km
T=241V=2 0.9km
Per-frame Streaming
CUT3R [ 20 ]
181.0
692.4
271.6
51.3
16.2
147.5
133.3
84.8
256.1
155.6
63.0
186.6
TTT3R [ 2 ]
181.1
68.6
243.2
16.2
6.3
138.8
127.5
64.9
126.3
110.4
43.3
102.4
DVGT-2 [ 34 ]
148.4
29.6
252.9
15.6
2.8
59.2
12.1
5.2
181.2
105.5
18.2
75.5
Table 3: Quantitative comparison on KITTI [ 5 ] Stereo 2Hz. The metric is ATE after Sim(3) alignment. † Average over completed sequences only; missing StreamVGGT [ 32 ] entries failed with OOM.
Figure 5: Qualitative trajectory comparison on KITTI [ 5 ] sequences 09 and 10. The dashed gray curves denote ground-truth poses, and the solid teal curves denote estimated poses. Our method produces trajectories that are more consistent with the ground truth and obtains lower ATE.
Setting
Depth
Point Cloud
Ego-Motion
AbsRel
δ<1.25
Acc.
Comp.
AUC@30
ATE
↓
↑
↓
↓
↑
↓
w/o Conf.
0.1730
0.8274
0.8320
0.7918
70.66
18.1769
w/ Conf.
0.1063
0.9039
0.7129
0.6137
88.16
4.1422
Table 4: Re-anchoring ablation on nuScenes [ 1 ] .
End-to-end autonomous driving has evolved from the conventional paradigm based on sparse perception into vision-language-action (VLA) models, which focus on learning language descriptions as an auxiliary task to facilitate planning. In this paper, we propose an alternative Vision-Geometry-Action (VGA) paradigm that advocates dense 3D geometry as the critical cue for autonomous driving. As vehicles operate in a 3D world, we think dense 3D geometry provides the most comprehensive information for decision-making. However, most existing geometry reconstruction methods (e.g., DVGT) rely on computationally expensive batch processing of multi-frame inputs and cannot be applied to online planning. To address this, we introduce a streaming Driving Visual Geometry Transformer (DVGT-2), which processes inputs in an online manner and jointly outputs dense geometry and trajectory planning for the current frame. We employ temporal causal attention and cache historical features to support on-the-fly inference. To further enhance efficiency, we propose a sliding-window streaming strategy and use historical caches within a certain interval to avoid repetitive computations. Despite the faster speed, DVGT-2 achieves superior geometry reconstruction performance on various datasets. The same trained DVGT-2 can be directly applied to planning across diverse camera configurations without fine-tuning, including closed-loop NAVSIM and open-loop nuScenes benchmarks.
Sicheng Zuo, Zixun Xie, Wenzhao Zheng +6
Tsinghua University · Xiaomi EV · Peking University +1
High-fidelity reconstruction of driving scenes is crucial for autonomous driving. While recent feedforward 3D Gaussian Splatting (3DGS) methods enable fast reconstruction, their per-pixel Gaussian prediction paradigm often suffers from multi-view inconsistency and layering artifacts. Moreover, existing methods often model dynamic instances via dense flow prediction, which lacks explicit cross-view correspondence and instance-level consistency. In this paper, we propose PointForward, a feedforward driving reconstruction framework through point-aligned representations. Unlike pixel-aligned methods, we initialize sparse 3D queries in world space and aggregate multi-view image information via spatial-temporal fusion onto these queries, enforcing explicit cross-view consistency in a single feedforward pass. To handle scene dynamics, we introduce scene graphs that explicitly organize moving instances during reconstruction. By leveraging 3D bounding boxes, our method enables instance-level motion propagation and temporally consistent dynamic representations. Extensive experiments demonstrate that PointForward achieves state-of-the-art performance on large-scale driving benchmarks. The code will be available upon the publication of the paper.
Cheng Chi, Xianqi Wang, Hongcheng Luo +9
Xiaomi EV · Huazhong University of Science and Technology · Zhejiang University
We present Qwen-Drive-1.0, an initial step towards a vision-language foundation model for autonomous driving. Qwen-Drive-1.0 retains the architecture of the pretrained vision-language model (VLM) and integrates 3D perception, visual question answering, and motion planning within a unified framework. An external bird's-eye-view (BEV) perception head jointly performs 3D object detection, semantic occupancy prediction, and BEV map segmentation. It serves as a probe of the 3D information accessible from the shared representations and provides an explicit, inspectable interface to 3D scene structure. A Planning Expert conditions on shared VLM representations to generate future ego trajectories. A staged training recipe combines driving supervision with general-purpose vision-language data to acquire driving-specific competence while helping preserve broad visual understanding and instruction-following capabilities. Experiments demonstrate strong 3D perception and driving scene understanding while largely preserving general vision-language capability. Comprehensive evaluations across open-loop, pseudo-closed-loop, and closed-loop settings further show highly competitive motion-planning performance.
Xin Zhou, Zongchuang Zhao, Zhibo Yang +13
Qwen Team, Huazhong University of Science and Technology