StreetForward: Perceiving Dynamic Street with Feedforward Causal Attention

Zhongrui Yu; Zhao Wang; Yijia Xie; Yida Wang; Xueyang Zhang; Yifei Zhan; Kun Zhan

cs.CVMar 20, 2026

StreetForward: Perceiving Dynamic Street with Feedforward Causal Attention

Authors: Zhongrui Yu, Zhao Wang, Yijia Xie, Yida Wang, Xueyang Zhang, Yifei Zhan, Kun Zhan

Organizations: Li Auto Inc. · Zhejiang University

Abstract

Feedforward reconstruction is crucial for autonomous driving applications, where rapid scene reconstruction enables efficient utilization of large-scale driving datasets in closed-loop simulation and other downstream tasks, eliminating the need for time-consuming per-scene optimization. We present StreetForward, a pose-free and tracker-free feedforward framework for dynamic street reconstruction. Building upon the alternating attention mechanism from Visual Geometry Grounded Transformer (VGGT), we propose a simple yet effective temporal mask attention module that captures dynamic motion information from image sequences and produces motion-aware latent representations. Static content and dynamic instances are represented uniformly with 3D Gaussian Splatting, and are optimized jointly by cross-frame rendering with spatio-temporal consistency, allowing the model to infer per-pixel velocities and produce high-fidelity novel views at new poses and times. We train and evaluate our model on the Waymo Open Dataset, demonstrating superior performance on novel view synthesis and depth estimation compared to existing methods. Furthermore, zero-shot inference on CARLA and other datasets validates the generalization capability of our approach. More visualizations are available on our project page: https://streetforward.github.io.

Figures & tables

Figure 1. S t r e e t F o r w a r d : High-fidelity spatio-temporal extrapolated view synthesis via feedforward 3DGS dynamic street reconstruction. The right pane illustrates our 3DGS representation of a dynamic street with precise velocities, enabling motion-aware rendering at novel poses and times independent of segmentation or tracking.

	SfM		Feedforward
Capability	StreetGS ( Yan et al., 2024 )	OmniRe ( Chen et al., )	VGGT ( Wang et al., 2025a )	DVGT ( Zuo et al., 2025 )	DGGT ( Chen et al., 2025a )	STORM ( Yang et al., 2024a )	EVolSplat ${}^{\text{4D}}$ ( Miao et al., 2026 )	Flux4D ( Wang et al., )	Ours
Novel View Synthesis (3DGS)
No camera poses needed
No per-scene optimization
No LiDAR needed
Models objects motions

Table 1. Dynamic 3D street reconstruction approaches. Approaches are grouped into Structure from Motion (SfM) and Feedforward. indicates the capability is supported while is not; “No tracker needed” means no instance-level tracker is required to synthesize views at novel timestamps. Our method is highlighted in the last column, which satisfies all listed capabilities.

Figure 2. The proposed S t r e e t F o r w a r d pipeline. We illustrate two common types of dynamic scenes with rigid objects(vehicles) and deformable objects(pedestrians) in and . The input video is first encoded into per-frame patchified features and then processed by $L$ times alternating global- and frame-attention to aggregate information across frames. These aggregated features are directly decoded by a camera head, a depth head and a Gaussian Head to obtain poses, depth and Gaussian attributes. Then causal masked attention is introduce to form motion-aware features, which are used to estimate both forward and backward motion as well as dynamic mask for separating static and dynamic Gaussians. The final 4D scene is obtained by combining static Gaussians with dynamic Gaussians propagated across time using the predicted motion.

Figure 3. Robustness to false-positive motion-mask prompts. Our method correctly assigns near-zero dynamic probability to parked or slowly moving vehicles, even if they are labeled as dynamic in annotations. This accurate understanding of motion (c) eliminates the motion ambiguity seen in DGGT (a), resulting in stable and clear rendering of the target vehicle over time (d).

Figure 4. Enforcing rigid regularization ( $\mathcal{L}_{\text{rigid}}$ ) removes the structural floaters seen around rigid objects in (a).

Figure 5. Temporal interpolation. With observation at time t-1 and t+1, we manage to correctly synthesize at time t. The figure compares the results of our full model against an ablation without backward fusion by predicting forward velocity only, and DGGT ( Chen et al., 2025a ) . The left three columns show pedestrian is with correct poses, and the right three columns display vehicle geometry reconstruction, where our model delivers complete geometries.

Figure 6. Qualitative comparision on Waymo Open Datasets.

Table 8

	Dynamic Only		Full Image
Method / Variant	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$
Ours	24.30	0.827	27.01	0.818
w/o causal attention	22.36	0.714	24.07	0.781
fixed causal horizon $k{=}1$	22.96	0.722	24.66	0.760
w/o $\mathcal{L}_{\text{rigid}}$	22.64	0.743	26.21	0.786
w/o $v^{-}$ (forward-only)	23.26	0.786	26.88	0.809

Table 6. Ablation on causal modeling and regularization (Waymo). Variants: w/o causal attention; fixed horizon (

k{=}1

); w/o

\mathcal{L}_{\text{rigid}}

; w/o backward velocity (

v^{-}

); w/o

\mathcal{L}_{\text{fb}}

; w/o

\mathcal{L}_{\alpha}

Explore similar work

PointForward: Feedforward Driving Reconstruction through Point-Aligned Representations
May 12, 2026Cheng Chi, Xianqi Wang, Hongcheng Luo +93D Reconstruction Multi-View 3D Reconstruction
Xiaomi EV · Huazhong University of Science and Technology · Zhejiang University
ReconDrive: Fast Feed-Forward 4D Gaussian Splatting for Autonomous Driving Scene Reconstruction
Mar 8, 2026Haibao Yu, Kuntao Xiao, Jiahang Wang +7Dynamic Scene Reconstruction 3D Gaussian Splatting
The University of Hong Kong · Tuojing Intelligence · King’s College London +2
DynamicVGGT: Learning Dynamic Point Maps for 4D Scene Reconstruction in Autonomous Driving
Mar 9, 2026Zhuolin He, Jing Li, Guanghao Li +9Dynamic Scene Reconstruction Autonomous Driving Perception
Fudan University · Yinwang Intelligent Technology · Huawei +2

May 12, 2026J/K move · Enter open · S save

PointForward: Feedforward Driving Reconstruction through Point-Aligned Representations

Cheng Chi, Xianqi Wang, Hongcheng Luo, Mingfei Tu+8

Xiaomi EV · Huazhong University of Science and Technology · Zhejiang University

3D Reconstruction Multi-View 3D Reconstruction Dynamic Scene Reconstruction 3D Scene Representation Feed-Forward 3D Reconstruction

Open paper AskPDF arXiv

High-fidelity reconstruction of driving scenes is crucial for autonomous driving. While recent feedforward 3D Gaussian Splatting (3DGS) methods enable fast reconstruction, their per-pixel Gaussian prediction paradigm often suffers from multi-view inconsistency and layering artifacts. Moreover, existing methods often model dynamic instances via dense flow prediction, which lacks explicit cross-view correspondence and instance-level consistency. In this paper, we propose PointForward, a feedforward driving reconstruction framework through point-aligned representations. Unlike pixel-aligned methods, we initialize sparse 3D queries in world space and aggregate multi-view image information via spatial-temporal fusion onto these queries, enforcing explicit cross-view consistency in a single feedforward pass. To handle scene dynamics, we introduce scene graphs that explicitly organize moving instances during reconstruction. By leveraging 3D bounding boxes, our method enables instance-level motion propagation and temporally consistent dynamic representations. Extensive experiments demonstrate that PointForward achieves state-of-the-art performance on large-scale driving benchmarks. The code will be available upon the publication of the paper.

Abstract

Explore similar work

PointForward: Feedforward Driving Reconstruction through Point-Aligned Representations

ReconDrive: Fast Feed-Forward 4D Gaussian Splatting for Autonomous Driving Scene Reconstruction

DynamicVGGT: Learning Dynamic Point Maps for 4D Scene Reconstruction in Autonomous Driving