Organizations: School of Computer Science, Northwestern Polytechnical University, Xi’an, China · vivo BlueImage Lab, vivo Mobile Communication Co., Ltd., Hangzhou, China · Northwestern Polytechnical University, Xi’an, China
4D scene reconstruction requires robust handling of dynamic objects, yet existing streaming methods commonly assume static scenes, leading to pose drift and ghosting artifacts over long sequences. We introduce Stream4D, a training-free, motion-decoupled streaming reconstruction framework that can be directly integrated into causal Transformer backbones (e.g., StreamVGGT, Stream3R). Our key observation is that causal attention implicitly encodes rich temporal motion cues within the KV cache. We adaptively aggregate these cues from multiple attention layers and across temporal windows, and fuse them with depth priors to handle challenging cases such as occlusions and textureless regions, producing precise dynamic masks that decouple static and dynamic regions. We then apply a soft key masking scheme that leverages these dynamic masks to suppress interference from dynamic regions in causal attention, thereby improving camera pose estimation accuracy. Stream4D inherits the efficiency of causal attention and remains compatible with existing streaming architectures. On diverse dynamic-scene benchmarks, Stream4D significantly improves camera pose estimation and static-scene reconstruction quality, yields reliable dynamic masks, and effectively mitigates long-sequence ghosting.
Figures & tables
Figure 1. We present Stream4D, a training-free framework that infers dynamic masks from causal-attention cues to decouple motion in streaming reconstruction. By masking dynamic regions during historical aggregation, Stream4D stabilizes online pose estimation and reduces long-sequence ghosting, yielding cleaner dynamic reconstructions.
Figure 2. Streaming reconstruction overview. Each incoming frame is encoded into tokens and decoded with temporal causal attention that attends only to past frames via a KV Cache. The cache is incrementally updated with (Kt,Vt) , enabling efficient online inference.
Figure 3. Temporal saliency aggregation and fusion visualization. We compare attention statistics across frames and layers: per-token mean and variance heatmaps revealing low-mean/high-variance patterns on dynamic objects; cross-frame attention maps highlighting temporal inconsistency; multi-scale attention fusion; and final fused saliency and binary masks.
Figure 4. Streaming dynamic segmentation architecture. Left: standard causal streaming reconstruction with a KV Cache, where each frame attends only to past tokens. Right: our Stream4D augments the causal pipeline with motion-decoupled masking. Queries load cached keys/values to compute attention scores, aggregate temporal saliency across frames, and produce dynamic masks. These masks are then applied to modify the cached keys, effectively suppressing interference from dynamic regions during historical aggregation.
Method
Type
DAVIS-2016
DAVIS-2017
DAVIS-all
JM
JR
JM
JR
JM
JR
DUSt3R ( Wang et al., 2024a )
Offline
42.10
45.70
35.20
35.30
35.90
34.00
Easi3R_dust3r ( Chen et al., 2025 )
Offline
50.10
55.77
46.86
50.54
44.10
50.85
Easi3R_monst3r ( Chen et al., 2025 )
Offline
54.93
68.00
54.75
66.16
51.64
63.06
MonST3R ( Zhang et al., 2024a )
Offline
40.42
40.39
38.07
36.05
36.98
34.52
DAS3R ( Xu et al., 2024 )
Offline
41.13
38.67
44.51
43.95
43.33
38.93
Table 1. Dynamic object segmentation on DAVIS benchmarks. We report mean IoU ( JM ) and IoU recall ( JR ). “Type” indicates Offline vs. Streaming (single-pass, causal). ∗ indicates our adapted implementation with segmentation capability, as the original methods do not support segmentation.
Figure 5. Qualitative results on DAVIS dataset. Each panel shows the input frame, the Easi3R mask, and our Stream4D mask. Easi3R exhibits temporal jitter and boundary breaks under occlusions and non-rigid motions. Our streaming multi-level temporal aggregation with consistency constraints produces temporally smooth and spatially complete masks across slow motions (camel, goat, etc.) and fast motions (skateboarder, motorcyclist), demonstrating robust segmentation under causal KV-Cache attention.
Method
Type
ATE ↓
RTE ↓
RRE ↓
Easi3R_monst3r ( Chen et al., 2025 )
Offline
0.032
0.008
1.075
Easi3R_dust3r ( Chen et al., 2025 )
Offline
0.022
0.009
0.806
MonST3R ( Zhang et al., 2024a )
Offline
0.038
0.010
1.172
POMATO ( Zhang et al., 2025a )
Offline
0.128
0.027
3.648
SpatialTrackerV2 ( Xiao et al., 2025a )
Offline
0.011
0.006
0.347
DAS3R ( Xu et al., 2024 )
Offline
0.052
0.012
1.560
Table 2. 4D reconstruction on DyCheck. We report pose metrics (ATE, RTE, RRE); lower is better. “Type” indicates Offline vs. Streaming (single-pass, causal).
Figure 6. Ghosting suppression in streaming 4D reconstruction. Naïve causal streaming methods integrate geometry from all past frames, causing moving objects to be repeatedly fused into the scene and leaving stacked replicas along their trajectories (ghosting artifacts), which grow with sequence length and can eventually corrupt nearby static surfaces. In contrast, Stream4D derives training-free dynamic masks from causal attention cues and applies soft key masking to suppress dynamic regions in the KV cache, so that dynamic objects appear only in the current frame while the background remains clean and sharp even in long sequences.
Figure 7. Qualitative comparison of estimated camera trajectories on three DyCheck sequences ( creeper , handwavy , mochi-high-five ). We plot the predicted trajectory (solid blue) against the ground truth (dashed gray) in the x - z plane. Left: Stream3R exhibits noticeable drift and erratic deviations, particularly in sequences with large dynamic motion ( handwavy , mochi-high-five ), where moving objects corrupt the pose estimation. Right: our Stream4D produces trajectories that more closely follow the ground truth, demonstrating that motion-decoupled masking effectively mitigates the interference of dynamic regions on camera pose estimation.
Figure 8. Inference time comparison across different sequence lengths. While Easi3R exhibits exponential time growth and suffers from Out-of-Memory (OOM) errors between 100 and 200 frames, Stream4D maintains linear scalability, performing comparably to the efficient StreamVGGT baseline.
Reconstructing dynamic 4D scenes remains challenging due to the presence of moving objects that corrupt camera pose estimation. Existing optimization methods alleviate this issue with additional supervision, but they are mostly computationally expensive and impractical in real-time applications. To address these limitations, we propose MoRe, a feedforward 4D reconstruction network that efficiently recovers dynamic 3D scenes from monocular videos. Built upon a strong static reconstruction backbone, MoRe employs an attention-forcing strategy to disentangle dynamic motion from static structure. To further enhance robustness, we fine-tune the model on large-scale, diverse datasets encompassing both dynamic and static scenes. Moreover, our grouped causal attention captures temporal dependencies and adapts to varying token lengths across frames, ensuring temporally coherent geometry reconstruction. Extensive experiments on multiple benchmarks demonstrate that MoRe achieves high-quality dynamic reconstructions with exceptional efficiency.
Juntong Fang, Zequn Chen, Weiqi Zhang +4
School of Software, Tsinghua University, Beijing, China · Li Auto
Reconstructing dynamic 4D scenes from monocular videos is a fundamental yet challenging task. While recent 3D foundation models provide strong geometric priors, their performance significantly degrades in dynamic environments. This degradation stems from a fundamental tension: the inherent coupling of camera ego-motion and object motion within global attention mechanisms. In this paper, we propose a novel, training-free progressive decoupling framework that disentangles dynamics from statics in a principled, coarse-to-fine manner. Our core insight is to resolve the tension by first stabilizing the camera pose, followed by geometric refinement. Specifically, our approach consists of three synergistic components: (1) a Dynamic-Mask-Guided Pose Decoupling module that isolates pose estimation from dynamic interference, yielding a stable motion-free reference frame; (2) a Topological Subspace Surgery mechanism that orthogonally decomposes the depth manifold, safely preserving dynamic objects while injecting refined, mask-aware geometry into static regions; and (3) an Information-Theoretic Confidence-Aware Fusion strategy that formulates depth integration as a heteroscedastic Bayesian inference problem, adaptively blending multi-pass predictions via inverse-variance weighting. Extensive experiments on standard 4D reconstruction benchmarks demonstrate that our method achieves consistent and substantial improvements across principal point-cloud metrics. Notably, our approach shows competitive performance in robust 4D scene reconstruction without requiring fine-tuning, suggesting the potential of mathematically grounded dynamic-static disentanglement.
Ying Zang, Xuanyi Liu, Yidong Han +9
4Huzhou University · 2Peking University · 1KOKONI 3D, Moxin Technology +4
Reconstructing dynamic 4D scenes requires jointly estimating correspondence, geometry, object motion, and camera motion. Existing feed-forward methods typically predict dense task-specific maps or independently process source-target pairs, leading to unnecessary computation for sparse queries and limited feature reuse across different frame pairs. We present UniQuery4R, a query-conditioned framework that encodes a multi-frame clip once and selects the source view, target view, and continuous source-image coordinate only at decoding time via source-to-target cross-attention. Each query jointly predicts target correspondence, target-time 3D position, and scene flow, along with source depth, while camera parameters are estimated per view. This design allows the encoded clip to be reused across arbitrary source-target selections and supports both sparse inference and dense reconstruction through batched queries, without learned temporal embeddings tied to a fixed clip length. We further introduce a direction-magnitude parameterization of scene flow with separate supervision for moving and static points. Among the evaluated methods, UniQuery4R achieves the best macro-average results on WorldTrack for both scene-flow estimation and dynamic-point reconstruction.
Tiancheng Chen, Sheng Tang, Wenhua Jin +4
Kosmo Research · Automotive Engineering Department, Jilin University · School of Software, Tsinghua University