Online reconstruction of dynamic scenes aims to learn from streaming multi-view inputs under low-latency constraints. The fast training and real-time rendering capabilities of 3D Gaussian Splatting have made on-the-fly reconstruction practically feasible, enabling online 4D reconstruction. However, existing online approaches, despite their efficiency and visual quality, fail to learn per-Gaussian motion that reflects true scene dynamics. Without explicit motion cues, appearance and motion are optimized solely under photometric loss, causing per-Gaussian motion to chase pixel residuals rather than true 3D motion. To address this, we propose MoRGS, an efficient online per-Gaussian motion reasoning framework that explicitly models per-Gaussian motion to improve 4D reconstruction quality. Specifically, we leverage optical flow on a sparse set of key views as lightweight motion cues that regularize per-Gaussian motion beyond photometric supervision. To compensate for the sparsity of flow supervision, we learn a per-Gaussian motion offset field that reconciles discrepancies between projected 3D motion and observed flow across views and time. In addition, we introduce a per-Gaussian motion confidence that separates dynamic from static Gaussians and weights Gaussian attribute residual updates, thereby suppressing redundant motion in static regions for better temporal consistency and accelerating the modeling of large motions. Extensive experiments demonstrate that MoRGS achieves state-of-the-art reconstruction quality and motion fidelity among online methods, while maintaining streamable performance.
Figures & tables
Figure 2: Illustration of the MoRGS framework. (a) We incrementally update Gaussian attributes at each time step while jointly modeling per-Gaussian motion between frames. (b) Per-Gaussian motion is guided by sparse motion cues and refined by a per-Gaussian motion offset field to compensate for discrepancies in the sparse motion cues. (c) To identify dynamic Gaussians, we obtain motion masks by thresholding the motion cues and then apply a segmentation model for view consistency. The per-Gaussian motion confidence is learned from these masks to suppress redundant background motion, improve temporal consistency, and concentrate learning on large motions.
Category
Method
PSNR (dB) ↑
SSIM ↑
LPIPS ↓
Storage (MB) ↓
Training (sec) ↓
Render (FPS) ↑
Offline
Real-Time 4DGS [ 29 ]
32.01
0.972
0.055
3.30
108
114
4DGS [ 29 ]
31.02
0.940
0.150
0.30
10
30
STG [ 13 ]
32.05
0.972
0.044
0.67
120
110
Swift4D [ 30 ]
32.23
0.972
0.043
0.40
5.0
125
Online
StreamRF [ 11 ]
30.68
-
-
31.5
15
8.3
3DGStream [ 24 ]
31.67
-
-
7.80
13
215
Table 1: Quantitative comparison on the N3DV dataset. For each scene, all metrics are averaged over 300 frames. Storage and training time both include the initial frame size and time. Red and orange highlight the best and second-best results in each category, respectively.
Figure 3: Per-Gaussian Motion Visualization. We visualize per-Gaussian motion under (a) no flow guidance, (b) sparse flow guidance only, (c) sparse flow guidance with motion offset, and (d) sparse flow guidance with both motion offset and motion confidence, while (e) and (f) show the learned motion confidence map and motion offset map, respectively.
PSNR
SSIM
LPIPS
Storage
Training
Render
(dB) ↑
↑
↓
(MB) ↓
(sec) ↓
(FPS) ↑
3DGStream [ 24 ]
30.79
-
-
4.10
7.2
288
QUEEN-l † [ 7 ]
29.47
0.946
0.185
0.38
1.5
317
HiCoM [ 6 ]
26.73
-
-
0.60
9.8
258
4DGC [ 8 ]
28.08
0.922
-
0.42
33.8
213
MoRGS (ours)
31.79
0.957
0.152
2.01
2.3
308
Table 2: Quantitative comparison on Meet Room dataset. QUEEN-l † refers to our re-implementation result through official code in the same experimental environment.
Figure 4: Qualitative Results. A visualization of various scenes in N3DV and Meet Room dataset. We include additional video results in the supplementary material.
Figure 5: Gaussian Motion Visualization. We observe that motion updates are confined to genuinely dynamic regions, and the recovered per-Gaussian motion closely follows the true scene dynamics.
Figure 6: Temporal Consistency Comparison. A visualization of spatiotemporal images from a fixed vertical scanline over time.
Scene
Method
PSNR (dB) ↑
SSIM ↑
mTV×100↓
COFFEE MARTINI
3DGStream
27.75
-
0.892
QUEEN-l
28.38
0.915
1.51
Ours
29.01
0.923
0.671
FLAME STEAK
3DGStream
28.39
-
0.668
QUEEN-l
29.25
0.923
1.30
Ours
30.11
0.931
0.587
Table 3: Analysis of rendering quality and temporal consistency on two N3DV scenes. mTV is computed only in static regions defined by predefined masks.
Modules
N3DV
Meet Room
ML
MO
MC
PSNR ↑
SSIM ↑
Train (s) ↓
PSNR ↑
SSIM ↑
Train (s) ↓
✗
✗
✗
31.33
0.946
3.3
29.40
0.952
1.7
✓
✗
✗
31.85
0.948
3.7
30.55
0.954
2.1
✓
✓
✗
32.21
0.949
3.8
31.21
0.955
2.2
✓
✓
✓
32.53
0.950
4.0
31.79
0.957
2.3
Table 4: Ablation on main components of our MoRGS framework.
# of Motion Supervision Views
Method
2
4
8
12
w/o Motion Offset
31.60
31.95
32.07
32.12
w/ Motion Offset
31.82
32.21
32.22
32.22
Per-frame time (s)
3.57
3.95
4.58
6.32
Table 5: Ablation on number of motion supervision and offset in N3DV dataset.