Online reconstruction of dynamic scenes aims to learn from streaming multi-view inputs under low-latency constraints. The fast training and real-time rendering capabilities of 3D Gaussian Splatting have made on-the-fly reconstruction practically feasible, enabling online 4D reconstruction. However, existing online approaches, despite their efficiency and visual quality, fail to learn per-Gaussian motion that reflects true scene dynamics. Without explicit motion cues, appearance and motion are optimized solely under photometric loss, causing per-Gaussian motion to chase pixel residuals rather than true 3D motion. To address this, we propose MoRGS, an efficient online per-Gaussian motion reasoning framework that explicitly models per-Gaussian motion to improve 4D reconstruction quality. Specifically, we leverage optical flow on a sparse set of key views as lightweight motion cues that regularize per-Gaussian motion beyond photometric supervision. To compensate for the sparsity of flow supervision, we learn a per-Gaussian motion offset field that reconciles discrepancies between projected 3D motion and observed flow across views and time. In addition, we introduce a per-Gaussian motion confidence that separates dynamic from static Gaussians and weights Gaussian attribute residual updates, thereby suppressing redundant motion in static regions for better temporal consistency and accelerating the modeling of large motions. Extensive experiments demonstrate that MoRGS achieves state-of-the-art reconstruction quality and motion fidelity among online methods, while maintaining streamable performance.
Figures & tables
Figure 2: Illustration of the MoRGS framework. (a) We incrementally update Gaussian attributes at each time step while jointly modeling per-Gaussian motion between frames. (b) Per-Gaussian motion is guided by sparse motion cues and refined by a per-Gaussian motion offset field to compensate for discrepancies in the sparse motion cues. (c) To identify dynamic Gaussians, we obtain motion masks by thresholding the motion cues and then apply a segmentation model for view consistency. The per-Gaussian motion confidence is learned from these masks to suppress redundant background motion, improve temporal consistency, and concentrate learning on large motions.
Category
Method
PSNR (dB) ↑
SSIM ↑
LPIPS ↓
Storage (MB) ↓
Training (sec) ↓
Render (FPS) ↑
Offline
Real-Time 4DGS [ 29 ]
32.01
0.972
0.055
3.30
108
114
4DGS [ 29 ]
31.02
0.940
0.150
0.30
10
30
STG [ 13 ]
32.05
0.972
0.044
0.67
120
110
Swift4D [ 30 ]
32.23
0.972
0.043
0.40
5.0
125
Online
StreamRF [ 11 ]
30.68
-
-
31.5
15
8.3
3DGStream [ 24 ]
31.67
-
-
7.80
13
215
Table 1: Quantitative comparison on the N3DV dataset. For each scene, all metrics are averaged over 300 frames. Storage and training time both include the initial frame size and time. Red and orange highlight the best and second-best results in each category, respectively.
Figure 3: Per-Gaussian Motion Visualization. We visualize per-Gaussian motion under (a) no flow guidance, (b) sparse flow guidance only, (c) sparse flow guidance with motion offset, and (d) sparse flow guidance with both motion offset and motion confidence, while (e) and (f) show the learned motion confidence map and motion offset map, respectively.
PSNR
SSIM
LPIPS
Storage
Training
Render
(dB) ↑
↑
↓
(MB) ↓
(sec) ↓
(FPS) ↑
3DGStream [ 24 ]
30.79
-
-
4.10
7.2
288
QUEEN-l † [ 7 ]
29.47
0.946
0.185
0.38
1.5
317
HiCoM [ 6 ]
26.73
-
-
0.60
9.8
258
4DGC [ 8 ]
28.08
0.922
-
0.42
33.8
213
MoRGS (ours)
31.79
0.957
0.152
2.01
2.3
308
Table 2: Quantitative comparison on Meet Room dataset. QUEEN-l † refers to our re-implementation result through official code in the same experimental environment.
Figure 4: Qualitative Results. A visualization of various scenes in N3DV and Meet Room dataset. We include additional video results in the supplementary material.
Figure 5: Gaussian Motion Visualization. We observe that motion updates are confined to genuinely dynamic regions, and the recovered per-Gaussian motion closely follows the true scene dynamics.
Figure 6: Temporal Consistency Comparison. A visualization of spatiotemporal images from a fixed vertical scanline over time.
Scene
Method
PSNR (dB) ↑
SSIM ↑
mTV×100↓
COFFEE MARTINI
3DGStream
27.75
-
0.892
QUEEN-l
28.38
0.915
1.51
Ours
29.01
0.923
0.671
FLAME STEAK
3DGStream
28.39
-
0.668
QUEEN-l
29.25
0.923
1.30
Ours
30.11
0.931
0.587
Table 3: Analysis of rendering quality and temporal consistency on two N3DV scenes. mTV is computed only in static regions defined by predefined masks.
Modules
N3DV
Meet Room
ML
MO
MC
PSNR ↑
SSIM ↑
Train (s) ↓
PSNR ↑
SSIM ↑
Train (s) ↓
✗
✗
✗
31.33
0.946
3.3
29.40
0.952
1.7
✓
✗
✗
31.85
0.948
3.7
30.55
0.954
2.1
✓
✓
✗
32.21
0.949
3.8
31.21
0.955
2.2
✓
✓
✓
32.53
0.950
4.0
31.79
0.957
2.3
Table 4: Ablation on main components of our MoRGS framework.
# of Motion Supervision Views
Method
2
4
8
12
w/o Motion Offset
31.60
31.95
32.07
32.12
w/ Motion Offset
31.82
32.21
32.22
32.22
Per-frame time (s)
3.57
3.95
4.58
6.32
Table 5: Ablation on number of motion supervision and offset in N3DV dataset.
Online reconstruction of dynamic 4D scenes from long, unposed streaming videos requires both continuous processing and photorealistic rendering, which existing methods struggle to achieve simultaneously. Existing feed-forward Gaussian methods are restricted to offline processing, whereas online point-cloud approaches struggle to maintain dense geometry and high-fidelity rendering. We present DynStream, a framework for streaming 4D Gaussian reconstruction from long, unposed videos. Given a continuous video stream, DynStream reconstructs the scene within local temporal windows and incrementally aligns and fuses these local reconstructions into a globally consistent scene, enabling online 4D reconstruction without per-scene optimization. By jointly enforcing cross-window geometric consistency and modeling time-varying scene content, DynStream supports efficient reconstruction and photorealistic rendering over extended video streams. Experiments demonstrate that DynStream enables high-fidelity online dynamic reconstruction and rendering from long video streams, achieving state-of-the-art performance across diverse dynamic indoor and outdoor scenes.
Dingwei Xian, Xiaoyu Zhou, Yajiao Xiong +2
Wangxuan Institute of Computer Technology, Peking University · VGI Labs Co., Ltd. · University of California, Merced
Streaming 3D reconstruction demands both speed and temporal fidelity, goals that existing methods undermine by updating every Gaussian every frame, even in static regions. We present DecoGS, a method for efficient online training of 3D Gaussians from streaming videos. Unlike prior methods that update the entire scene indiscriminately, DecoGS introduces an adaptive mechanism that selectively focuses optimization on spatiotemporal regions exhibiting motion or photometric changes. This targeted training strategy eliminates redundant updates that cause flickering and drift in nominally static regions, while enabling fast, high-fidelity scene updates. The pipeline further integrates region-aware Gaussian management through gradient gating and efficient visibility filtering to maintain temporal coherence and a compact memory footprint. On N3DV and MeetRoom, DecoGS achieves 34.55 and 31.60 dB PSNR respectively, outperforming all streaming and offline baselines, while rendering at 261 FPS with 70× lower temporal flicker than the best prior method, requiring no large-scale pretraining.
Reconstructing dynamic 3D scenes from monocular videos is a fundamental yet highly challenging task, as real-world motions often involve both long-term smooth transformations and short-term complex deformations. Existing methods either struggle to maintain temporal consistency or fail to capture high-frequency dynamics due to limited motion modeling capacity. In this work, we present Rigid-aware 4D Gaussian Splatting (RiGS), which simultaneously captures motions across multiple temporal scales. Specifically, RiGS introduces three types of Gaussian primitives: static, rigid, and transient, which represent static backgrounds, long-term low-frequency motions, and short-term high-frequency dynamics, respectively. An object-wise dynamic mask is proposed to aggregate long-range spatiotemporal motion information and guide the decomposition of static and dynamic regions. To jointly model motion across scales, rigid Gaussians are allowed to transition into transient Gaussians based on their temporal duration, and both are optimized under scene flow guidance, providing dense 3D motion supervision. Extensive experiments demonstrate that RiGS achieves state-of-the-art performance on novel view synthesis benchmarks. Code is available at \hyperlink{https://github.com/ladvu/RiGS}{https://github.com/ladvu/RiGS}.
Chenyu Wu, Wanhua Li, Zhu-Tian Chen +1
Harvard University · Zhejiang University · Nanyang Technological University +1