PointForward V2: Point-Aligned Interaction for Feedforward Driving Reconstruction

Xianqi Wang; Cheng Chi; Yuechuan Pu; Hongcheng Luo; Junda Cheng; Guang Chen; Hangjun Ye; Bing Wang

cs.CVOct 20, 2026

PointForward V2: Point-Aligned Interaction for Feedforward Driving Reconstruction

Authors: Xianqi Wang, Cheng Chi, Yuechuan Pu, Hongcheng Luo, Junda Cheng, Guang Chen, Hangjun Ye, Bing Wang,

Organizations: Huazhong University of Science and Technology · Xiaomi EV · Beijing Key Laboratory of Intelligent Driving Foundation Models and Simulation · HKUST

Abstract

Feedforward reconstruction of dynamic driving scenes requires integrating observations across multiple views and time steps into a coherent 3D representation. PointForward established state-of-the-art feedforward driving reconstruction by lifting image observations into sparse 3D queries, demonstrating the effectiveness of point-aligned scene representations. However, these queries are largely processed independently, limiting information exchange across space and time, while direct global attention among tens of thousands of point queries is computationally prohibitive. In this work, we present PointForward V2, which advances point-aligned representation to point-aligned interaction. Our key idea is to mediate query interaction through compact, scene-structured registers. For the static background, frame-wise Static Scene Registers compress point information, communicate across frames, and propagate the aggregated context back to the queries. For each dynamic instance, queries from different frames are aligned in an object-centric canonical space and interact through instance-specific Dynamic Instance Registers. We further employ 4D positional encoding that combines aligned spatial coordinates with original timestamps, enabling spatially aligned yet temporally distinguishable interactions. In this way, PointForward V2 enables scalable spatio-temporal reasoning directly among lifted 3D queries without dense global attention. Experiments on Waymo, nuScenes, and Argoverse 2 demonstrate state-of-the-art performance across large-scale driving benchmarks.

Figures & tables

Figure 1: From point-aligned representation to interaction. While PointForward processes 3D queries independently, PointForward V2 enables scene-structured interaction: static queries communicate across frames, while dynamic queries interact within each instance.

Figure 2: Overview of PointForward V2. It first unprojects images into a 3D representation and decomposes it into static and dynamic point queries. Inter-Frame Scene Interaction refines static queries, while Intra-Instance Interaction refines dynamic queries. Both branches use a compress–communicate–broadcast register mechanism repeated for $L$ blocks. Updated queries are finally decoded into 3D Gaussians.

Per-Scene Optimization Methods
Method	Dynamic Only			Full Image
Method	PSNR↑	SSIM↑	D-RMSE↓	PSNR↑	SSIM↑	D-RMSE↓
EmerNeRF [ 35 ]	17.79	0.255	40.88	24.51	0.738	33.99
3DGS [ 16 ]	17.13	0.267	13.88	25.13	0.741	19.68
PVG [ 5 ]	15.51	0.128	15.91	22.38	0.661	13.01
DeformableGS [ 38 ]	17.10	0.266	12.14	25.29	0.761	14.79

Table 1: Quantitative results on Waymo. We report PSNR, SSIM, and D-RMSE over both dynamic regions and full images. PointForward V2 achieves the best performance across all metrics, with particularly great improvements on dynamic regions.

Figure 3: Qualitative comparison on Waymo. Compared with STORM, DGGT, and PointForward, PointForward V2 produces more complete scene geometry and cleaner reconstructions of both static and dynamic content, with improved object shapes and scene structures.

Table 5

Figure 4: Zero-shot qualitative comparison on nuScenes. Models are evaluated without nuScenes-specific fine-tuning. PointForward V2 better preserves scene structures and dynamic objects with cleaner geometry and more complete reconstructions.

Figure 5: Qualitative comparison under extrapolated viewpoints. PointForward renders feature maps followed by convolutional RGB refinement, which can fill newly disoccluded regions but may also introduce incorrect image-space completion. PointForward V2 directly renders RGB Gaussians, avoiding such erroneous completion and preserving the reconstructed 3D content more faithfully.

$R_{s}/R_{d}$
Capacity	PSNR↑	SSIM↑	D-RMSE↓
128 / 8	29.31	0.856	2.96
256 / 16	29.88	0.865	2.89
512 / 32	29.68	0.861	2.90
$N_{s}/N_{d}$
16,384 / 1,024	28.61	0.843	2.94

Table 5: Effect of register and query capacity. We vary register sizes

R_{s}/R_{d}

and query numbers

N_{s}/N_{d}

independently while keeping the remaining configuration fixed.

Dynamic Scene Reconstruction

Dynamic scene reconstruction builds time-varying 3D models of moving objects and environments from video or multi-view images.

See all 290 papers

PointForward: Feedforward Driving Reconstruction through Point-Aligned Representations
May 12, 2026Cheng Chi, Xianqi Wang, Hongcheng Luo +93D Reconstruction Multi-View 3D Reconstruction
Xiaomi EV · Huazhong University of Science and Technology · Zhejiang University
StreetForward: Perceiving Dynamic Street with Feedforward Causal Attention
Mar 20, 2026Zhongrui Yu, Zhao Wang, Yijia Xie +4Dynamic Scene Reconstruction Feed-Forward 3D Reconstruction
Li Auto Inc. · Zhejiang University
DynamicVGGT: Learning Dynamic Point Maps for 4D Scene Reconstruction in Autonomous Driving
Mar 9, 2026Zhuolin He, Jing Li, Guanghao Li +9Dynamic Scene Reconstruction Autonomous Driving Perception
Fudan University · Yinwang Intelligent Technology · Huawei +2

May 12, 2026J/K move · Enter open · S save

PointForward: Feedforward Driving Reconstruction through Point-Aligned Representations

Cheng Chi, Xianqi Wang, Hongcheng Luo, Mingfei Tu+8

Xiaomi EV · Huazhong University of Science and Technology · Zhejiang University

3D Reconstruction Multi-View 3D Reconstruction Dynamic Scene Reconstruction 3D Scene Representation Feed-Forward 3D Reconstruction

Open paper AskPDF arXiv

High-fidelity reconstruction of driving scenes is crucial for autonomous driving. While recent feedforward 3D Gaussian Splatting (3DGS) methods enable fast reconstruction, their per-pixel Gaussian prediction paradigm often suffers from multi-view inconsistency and layering artifacts. Moreover, existing methods often model dynamic instances via dense flow prediction, which lacks explicit cross-view correspondence and instance-level consistency. In this paper, we propose PointForward, a feedforward driving reconstruction framework through point-aligned representations. Unlike pixel-aligned methods, we initialize sparse 3D queries in world space and aggregate multi-view image information via spatial-temporal fusion onto these queries, enforcing explicit cross-view consistency in a single feedforward pass. To handle scene dynamics, we introduce scene graphs that explicitly organize moving instances during reconstruction. By leveraging 3D bounding boxes, our method enables instance-level motion propagation and temporally consistent dynamic representations. Extensive experiments demonstrate that PointForward achieves state-of-the-art performance on large-scale driving benchmarks. The code will be available upon the publication of the paper.

Abstract

Dynamic Scene Reconstruction

Explore similar work

PointForward: Feedforward Driving Reconstruction through Point-Aligned Representations

StreetForward: Perceiving Dynamic Street with Feedforward Causal Attention

DynamicVGGT: Learning Dynamic Point Maps for 4D Scene Reconstruction in Autonomous Driving