PointForward V2: Point-Aligned Interaction for Feedforward Driving Reconstruction
Authors: Xianqi Wang, Cheng Chi, Yuechuan Pu, Hongcheng Luo, Junda Cheng, Guang Chen, Hangjun Ye, Bing Wang, Xin Yang
Organizations: Huazhong University of Science and Technology · Xiaomi EV · Beijing Key Laboratory of Intelligent Driving Foundation Models and Simulation · HKUST
Feedforward reconstruction of dynamic driving scenes requires integrating observations across multiple views and time steps into a coherent 3D representation. PointForward established state-of-the-art feedforward driving reconstruction by lifting image observations into sparse 3D queries, demonstrating the effectiveness of point-aligned scene representations. However, these queries are largely processed independently, limiting information exchange across space and time, while direct global attention among tens of thousands of point queries is computationally prohibitive. In this work, we present PointForward V2, which advances point-aligned representation to point-aligned interaction. Our key idea is to mediate query interaction through compact, scene-structured registers. For the static background, frame-wise Static Scene Registers compress point information, communicate across frames, and propagate the aggregated context back to the queries. For each dynamic instance, queries from different frames are aligned in an object-centric canonical space and interact through instance-specific Dynamic Instance Registers. We further employ 4D positional encoding that combines aligned spatial coordinates with original timestamps, enabling spatially aligned yet temporally distinguishable interactions. In this way, PointForward V2 enables scalable spatio-temporal reasoning directly among lifted 3D queries without dense global attention. Experiments on Waymo, nuScenes, and Argoverse 2 demonstrate state-of-the-art performance across large-scale driving benchmarks.
Figures & tables
Figure 1: From point-aligned representation to interaction. While PointForward processes 3D queries independently, PointForward V2 enables scene-structured interaction: static queries communicate across frames, while dynamic queries interact within each instance.
Figure 2: Overview of PointForward V2. It first unprojects images into a 3D representation and decomposes it into static and dynamic point queries. Inter-Frame Scene Interaction refines static queries, while Intra-Instance Interaction refines dynamic queries. Both branches use a compress–communicate–broadcast register mechanism repeated for L blocks. Updated queries are finally decoded into 3D Gaussians.
Method
Dynamic Only
Full Image
PSNR↑
SSIM↑
D-RMSE↓
PSNR↑
SSIM↑
D-RMSE↓
Per-Scene Optimization Methods
EmerNeRF [ 35 ]
17.79
0.255
40.88
24.51
0.738
33.99
3DGS [ 16 ]
17.13
0.267
13.88
25.13
0.741
19.68
PVG [ 5 ]
15.51
0.128
15.91
22.38
0.661
13.01
DeformableGS [ 38 ]
17.10
0.266
12.14
25.29
0.761
14.79
Table 1: Quantitative results on Waymo. We report PSNR, SSIM, and D-RMSE over both dynamic regions and full images. PointForward V2 achieves the best performance across all metrics, with particularly great improvements on dynamic regions.
Figure 3: Qualitative comparison on Waymo. Compared with STORM, DGGT, and PointForward, PointForward V2 produces more complete scene geometry and cleaner reconstructions of both static and dynamic content, with improved object shapes and scene structures.
Table 5
Figure 4: Zero-shot qualitative comparison on nuScenes. Models are evaluated without nuScenes-specific fine-tuning. PointForward V2 better preserves scene structures and dynamic objects with cleaner geometry and more complete reconstructions.
Figure 5: Qualitative comparison under extrapolated viewpoints. PointForward renders feature maps followed by convolutional RGB refinement, which can fill newly disoccluded regions but may also introduce incorrect image-space completion. PointForward V2 directly renders RGB Gaussians, avoiding such erroneous completion and preserving the reconstructed 3D content more faithfully.
Capacity
PSNR↑
SSIM↑
D-RMSE↓
Rs/Rd
128 / 8
29.31
0.856
2.96
256 / 16
29.88
0.865
2.89
512 / 32
29.68
0.861
2.90
Ns/Nd
16,384 / 1,024
28.61
0.843
2.94
Table 5: Effect of register and query capacity. We vary register sizes Rs/Rd and query numbers Ns/Nd independently while keeping the remaining configuration fixed.
High-fidelity reconstruction of driving scenes is crucial for autonomous driving. While recent feedforward 3D Gaussian Splatting (3DGS) methods enable fast reconstruction, their per-pixel Gaussian prediction paradigm often suffers from multi-view inconsistency and layering artifacts. Moreover, existing methods often model dynamic instances via dense flow prediction, which lacks explicit cross-view correspondence and instance-level consistency. In this paper, we propose PointForward, a feedforward driving reconstruction framework through point-aligned representations. Unlike pixel-aligned methods, we initialize sparse 3D queries in world space and aggregate multi-view image information via spatial-temporal fusion onto these queries, enforcing explicit cross-view consistency in a single feedforward pass. To handle scene dynamics, we introduce scene graphs that explicitly organize moving instances during reconstruction. By leveraging 3D bounding boxes, our method enables instance-level motion propagation and temporally consistent dynamic representations. Extensive experiments demonstrate that PointForward achieves state-of-the-art performance on large-scale driving benchmarks. The code will be available upon the publication of the paper.
Cheng Chi, Xianqi Wang, Hongcheng Luo +9
Xiaomi EV · Huazhong University of Science and Technology · Zhejiang University
Feedforward reconstruction is crucial for autonomous driving applications, where rapid scene reconstruction enables efficient utilization of large-scale driving datasets in closed-loop simulation and other downstream tasks, eliminating the need for time-consuming per-scene optimization. We present StreetForward, a pose-free and tracker-free feedforward framework for dynamic street reconstruction. Building upon the alternating attention mechanism from Visual Geometry Grounded Transformer (VGGT), we propose a simple yet effective temporal mask attention module that captures dynamic motion information from image sequences and produces motion-aware latent representations. Static content and dynamic instances are represented uniformly with 3D Gaussian Splatting, and are optimized jointly by cross-frame rendering with spatio-temporal consistency, allowing the model to infer per-pixel velocities and produce high-fidelity novel views at new poses and times. We train and evaluate our model on the Waymo Open Dataset, demonstrating superior performance on novel view synthesis and depth estimation compared to existing methods. Furthermore, zero-shot inference on CARLA and other datasets validates the generalization capability of our approach. More visualizations are available on our project page: https://streetforward.github.io.
Dynamic scene reconstruction in autonomous driving remains a fundamental challenge due to significant temporal variations, moving objects, and complex scene dynamics. Existing feed-forward 3D models have demonstrated strong performance in static reconstruction but still struggle to capture dynamic motion. To address these limitations, we propose DynamicVGGT, a unified feed-forward framework that extends VGGT from static 3D perception to dynamic 4D reconstruction. Our goal is to model point motion within feed-forward 3D models in a dynamic and temporally coherent manner. To this end, we jointly predict the current and future point maps within a shared reference coordinate system, allowing the model to implicitly learn dynamic point representations through temporal correspondence. To efficiently capture temporal dependencies, we introduce a Motion-aware Temporal Attention (MTA) module that learns motion continuity. Furthermore, we design a Dynamic 3D Gaussian Splatting Head that explicitly models point motion by predicting Gaussian velocities using learnable motion tokens under scene flow supervision. It refines dynamic geometry through continuous 3D Gaussian optimization. Extensive experiments on autonomous driving datasets demonstrate that DynamicVGGT significantly outperforms existing methods in reconstruction accuracy, achieving robust feed-forward 4D dynamic scene reconstruction under complex driving scenarios.
Zhuolin He, Jing Li, Guanghao Li +9
Fudan University · Yinwang Intelligent Technology · Huawei +2