Streaming 3D Gaussian Splatting (3DGS) allows for online reconstruction of dynamic scenes from multi-view videos for Free Viewpoint Video (FVV). Current time and storage efficient methods rely on selecting a subset of anchor Gaussians and neighborhood interpolation for fast streamable 3DGS. However, these methods can overlook object-specific motion and introduce inconsistencies near object boundaries. We propose OCLDA-GS, an object-centric streaming framework that dynamically reallocates object and motion-aware Gaussian anchors to allocate anchor Gaussian support for objects undergoing large motion while at the same time, restricting the gaussian neighborhood within object groups for more precise object boundaries. Furthermore, it estimates object-wise rigid transformations (e.g. a hand moving) and leverages these estimates to learn per-anchor local deformation correction for small deformations (e.g. wrinkles on a shirt). Then, we conduct experiments on three dynamic multi-view video datasets and demonstrate improvements over the previous baselines alongside improved quality between refinement timesteps. Under matched refinement budgets, our method improves PSNR over Instant Gaussian Stream (IGS) by 0.169 dB on N3DV, 0.202 dB on Meeting Room and +0.199 on Google Immersive, while reducing the mean PSNR drop between refinement timesteps by 9.2% and 13.8% in N3DV and Meeting Room, respectively.
Figures & tables
Figure 4: Overview of our method. (a) Flow-guided anchor selection within object groups. (b) Object-wise rigid motion estimation from AGM-Net predictions. (c) Learned local deformation corrections propagated to Gaussians within each object. (d) Improved rendering quality compared to Farthest Point Sampling (FPS) -based IGS.
Method
# Params
# Online refinements
N3DV
Meeting Room
Google Immersive
Average
PSNR ↑
DSSIM ↓
s/frm ↓
PSNR ↑
DSSIM ↓
s/frm ↓
PSNR ↑
DSSIM ↓
s/frm ↓
PSNR ↑
DSSIM ↓
s/frm ↓
Offline methods
RT 4DGS [ 36 ]
0
32.010
0.014
30.470
0.025
26.760
0.030
29.747
0.023
4D-GS [ 33 ]
2.46M
31.150
0.016
31.275
0.024
23.990
0.079
28.805
0.040
STG [ 18 ]
90
32.050
0.026
29.510
0.034
29.200
0.042
30.253
0.034
Swift4D [ 34 ]
31.92M
32.230
0.014
31.306
0.022
23.482
0.060
29.006
0.032
Table 1: Quantitative comparison on N3DV, Meeting Room, and Google Immersive. Average is the equal-weight mean of the three displayed dataset results. Gray rows are offline methods; the remaining rows are online methods. Darker blue indicates the best, and second-best online results. Offline quality includes published reference results.
Figure 5: Qualitative comparison of online methods across three datasets. Each row uses the same frame, viewpoint, and crop for all methods. Orange boxes mark regions enlarged in the insets, highlighting motion alignment and object boundaries.
Figure 6: Flow-guided anchor selection and temporal propagation. Flow guides anchor reselection within a fixed budget. Colors denote SAGA groups; white points mark selected anchors. Markers and trails visualize anchor and Gaussian motion. Ours shows the refined target keyframe from an input-camera view. Reselection and refinement occur only at keyframes.
Object
Flow
LD
N3DV
Meeting Room
Average
PSNR ↑
DSSIM ↓
s/frm ↓
PSNR ↑
DSSIM ↓
s/frm ↓
PSNR ↑
DSSIM ↓
s/frm ↓
×
×
×
32.877
0.020
2.00
30.088
0.028
1.96
31.482
0.024
1.98
✓
×
×
32.882
0.020
2.20
30.116
0.028
2.14
31.499
0.024
2.17
×
✓
×
32.913
0.020
2.37
30.263
0.027
2.31
31.588
0.023
2.34
×
×
✓
33.059
0.020
2.28
30.109
0.028
2.24
31.584
0.024
2.26
✓
✓
×
32.927
0.020
2.33
30.267
0.027
2.27
31.597
0.023
2.30
Table 2: Component ablation on N3DV and Meeting Room. Checkmarks indicate enabled components. The first row corresponds to IGS (FPS), and the last row to our full method. Average equally weights the two datasets. LD: Local Deformation
Configuration
PSNR ↑
Average
N3DV
MR
PSNR ↑
DSSIM ↓
Local deformation configuration
No correction
32.927
30.267
31.597
0.023
Rigid replacement
32.988
30.204
31.596
0.024
LD MLP w/o Rigid
32.991
30.280
31.636
0.023
Anchor Reallocation
Table 3: Ablation of local deformation and anchor allocation on N3DV and Meeting Room (MR). Average metrics equally weight the two datasets. LD: Local Deformation
Figure 7: Temporal reconstruction quality with selected online baselines. Faint traces show raw values; solid curves show a 45-frame EMA. Full comparisons appear in Tab. 1 .
Figure 8: Mean PSNR drop relative to keyframes on N3DV and Meeting Room; lower is better.
Table 4: Motion-anchor budget ablation across the datasets. PSNR ( ↑ ) averages evaluated targets per scene, followed by an equal-scene average within each dataset. Shading indicates the highest PSNR per dataset.
Streaming 3D reconstruction demands both speed and temporal fidelity, goals that existing methods undermine by updating every Gaussian every frame, even in static regions. We present DecoGS, a method for efficient online training of 3D Gaussians from streaming videos. Unlike prior methods that update the entire scene indiscriminately, DecoGS introduces an adaptive mechanism that selectively focuses optimization on spatiotemporal regions exhibiting motion or photometric changes. This targeted training strategy eliminates redundant updates that cause flickering and drift in nominally static regions, while enabling fast, high-fidelity scene updates. The pipeline further integrates region-aware Gaussian management through gradient gating and efficient visibility filtering to maintain temporal coherence and a compact memory footprint. On N3DV and MeetRoom, DecoGS achieves 34.55 and 31.60 dB PSNR respectively, outperforming all streaming and offline baselines, while rendering at 261 FPS with 70× lower temporal flicker than the best prior method, requiring no large-scale pretraining.
Feed-forward 3D Gaussian Splatting (3DGS) allows efficient and high-fidelity novel view synthesis (NVS) from an offline recorded image sequence. However, achieving online NVS from streaming and unposed image inputs remains challenging. Although online feed-forward geometric estimation methods have been proposed for streaming depth and point cloud recovery, they cannot be adapted to NVS due to severe rendering artifacts. This is because NVS demands stricter multi-view consistency in Gaussian scales and pose-geometry alignment; even minor deviations would accumulate over time and visibly degrade rendering quality. To this end, we propose FreeStreamGS, a robust online feed-forward framework for efficient and high-quality NVS. We introduce two key mechanisms: a Decoupled Intrinsic Recovery Head that removes cumulative camera intrinsic bias and prevents scene scale jitter during long-term streaming, and a Dynamic Point Refinement Offset strategy that relaxes rigid unprojection to correct coupled pose-depth drift. Extensive experiments show that FreeStreamGS achieves rendering quality competitive with state-of-the-art offline feed-forward 3DGS methods, despite operating without access to future frames.
Ruiyang Chen, Feiran Li, Chu Zhou +3
Beijing University of Posts and Telecommunications, China · Independent Researcher · National Institute of Informatics, Japan
Online reconstruction of dynamic scenes aims to learn from streaming multi-view inputs under low-latency constraints. The fast training and real-time rendering capabilities of 3D Gaussian Splatting have made on-the-fly reconstruction practically feasible, enabling online 4D reconstruction. However, existing online approaches, despite their efficiency and visual quality, fail to learn per-Gaussian motion that reflects true scene dynamics. Without explicit motion cues, appearance and motion are optimized solely under photometric loss, causing per-Gaussian motion to chase pixel residuals rather than true 3D motion. To address this, we propose MoRGS, an efficient online per-Gaussian motion reasoning framework that explicitly models per-Gaussian motion to improve 4D reconstruction quality. Specifically, we leverage optical flow on a sparse set of key views as lightweight motion cues that regularize per-Gaussian motion beyond photometric supervision. To compensate for the sparsity of flow supervision, we learn a per-Gaussian motion offset field that reconciles discrepancies between projected 3D motion and observed flow across views and time. In addition, we introduce a per-Gaussian motion confidence that separates dynamic from static Gaussians and weights Gaussian attribute residual updates, thereby suppressing redundant motion in static regions for better temporal consistency and accelerating the modeling of large motions. Extensive experiments demonstrate that MoRGS achieves state-of-the-art reconstruction quality and motion fidelity among online methods, while maintaining streamable performance.
Wonjoon Lee, Sungmin Woo, Donghyeong Kim +3
Yonsei University · Electronics and Telecommunications Research Institute (ETRI)