Organizations: Guangdong Provincial Key Laboratory of Ultra High Definition Immersive Media Technology, Shenzhen Graduate School, Peking University · Pengcheng Laboratory · Peking University · MIGU Video Co., Ltd.
Dynamic 3D scene reconstruction is essential for immersive media such as VR, MR, and XR, yet remains challenging for long multi-view sequences with large-scale motion. Existing dynamic Gaussian approaches are either Frame-Stream, offering scalability but poor temporal stability, or Clip, achieving local consistency at the cost of high memory and limited sequence length. We propose ClipGStream, a hybrid reconstruction framework that performs stream optimization at the clip level rather than the frame level. The sequence is divided into short clips, where dynamic motion is modeled using clip-independent spatio-temporal fields and residual anchor compensation to capture local variations efficiently, while inter-clip inherited anchors and decoders maintain structural consistency across clips. This Clip-Stream design enables scalable, flicker-free reconstruction of long dynamic videos with high temporal coherence and reduced memory overhead. Extensive experiments demonstrate that ClipGStream achieves state-of-the-art reconstruction quality and efficiency. The project page is available at: https://liangjie1999.github.io/ClipGStreamWeb/
Figures & tables
Figure 2: Our training process is divided into two stages: the Reference Clip ( Clip0 ) Training Stage and Source Clip ( Clipn∈[1,N−1] ) Training Stage. In Reference Clip Training Stage, anchors A0 are initialized by the fused point cloud from all frames of Clip0 . The anchors are fed into a spatio-temporal field ( STF0 ) to extract dynamic features, which are then concatenated with static features. They are decoded to Temporal Gaussian attributes and rendered via rasterization. In Source Clip Training Stage, we inherit the anchors, decoder and the static features from Reference Clip. Residual anchors are computed between the current and Reference Clip A0 . Finally, the same training pipeline as in the Reference Clip stage is applied using the An conpensated with residual anchors and the newly initialized STF .
Figure 3: (a)Static features fs learns all background information, therefore sharing fs across clips ensures temporal consistency. (b) Dynamic features learns residual information that controls the visibility of dynamic content, and thus are clip-independent.
Figure 4: Geometry-aware deduplication. (a) The points from the Reference Clip is converted into a spherical coverage field : for each point p , a sphere s is centered at p with radius set to the mean distance to its three nearest neighbors (computed via KNN). (b) Given the field and the Source Clip candidate anchors, residual anchors are selected based on their signed distance to the field surface. q1 (red box), SDF(q1)<0 , is filtered out; while q2 (green box), SDF(q2)>0 , is retained as a residual anchor.
Figure 5: The ablation study on the Residual Anchors Compensation Module (RAC) and the Anchors Inheritance Module (AI) . As seen from the residual heatmaps between adjacent clips, removing either module leads to strong responses in static regions as shown in (a)(b)(c), while enabling them, as illustrated in (d), significantly suppresses flicker and preserves smooth clip transitions, which demonstrates that both components play essential roles in maintaining inter clip stability.
Figure 6: Qualitative results on Long 360 (1,400 frames; extreme motion amplitudes and long-sequence challenges) and VRU GZ [ 31 ] (complex dynamic interactions). Compared to 4DGaussian [ 29 ] and LocalDyGS [ 31 ] , our method produces sharper renderings in both dynamic regions (e.g., athletes) and static areas (e.g., court floor), with more stable and temporally coherent reconstructions. Additional results are provided in the supplementary video and materials .
Method
PSNR ↑
DSSIM 1 ↓
DSSIM 2 ↓
LPIPS ↓
Static methods
2DGS [ 23 ]
23.80
0.085
0.042
0.181
3DGS [ 54 ]
24.13
0.087
0.040
0.159
ScaffoldGS [ 56 ]
23.53
0.096
0.048
0.203
Frame-Stream methods
3DGStream [ 27 ]
21.94
0.105
0.053
0.200
Table 1: Quantitative comparison on Long 360 , which contains 1400 frames. Static methods are tested on frame 0.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
GOF [ 21 ]
30.39
0.949
0.141
2DGS [ 23 ]
30.78
0.949
0.187
3DGS [ 54 ]
30.50
0.949
0.171
4DGaussian [ 29 ]
28.32
0.930
0.186
SpaceTimeGS [ 30 ]
27.42
0.926
0.193
LocalDyGS [ 31 ]
30.58
0.944
0.173
Table 2: Quantitative comparison on VRU ( GZ ) dataset [ 39 ] . Static methods are evaluated on frame 0 only, serving as an upper-bound reference for dynamic reconstruction.
Figure 7: Qualitative results of flame salmon from the N3DV dataset [ 14 ] , a fine-scale motion sequence . Our method outperforms SOTA approaches (Grid4D [ 42 ] , LocalDyGS [ 31 ] ) by better preserving details in dynamic regions such as the dog’s face and flames.
Method
PSNR ↑
DSSIM 1 ↓
DSSIM 2 ↓
FPS ↑
Time ↓
Size ↓
Frame-Stream methods
StremRF [ 40 ]
28.26
-
-
10.9
-
5310MB
3DGStream [ 27 ]
31.67
-
-
215
1.0h
1230MB
4DGC [ 64 ]
31.58
-
-
-
-
-
HiCoM [ 45 ]
31.17
-
-
-
-
-
Clip methods
Table 3: Quantitative comparisons are conducted on the Neural 3D Video Dataset [ 63 ] , comprising five 300-frame scenes.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
4DGaussian [ 29 ]
28.89
0.952
0.196
4K4D [ 65 ]
21.29
0.826
0.196
ENeRF [ 66 ]
23.48
0.894
0.259
3DGS [ 54 ]
28.61
0.949
0.210
Dy3DGS [ 26 ]
25.91
0.880
0.255
LocalDyGS [ 31 ]
28.15
0.912
0.153
Table 4: Quantitative comparison on flame salmon (1200f).
Method
PSNR ↑
DSSIM1↓
DSSIM2↓
LPIPS ↓
w/o DI
24.34
0.081
0.038
0.152
w/o RAC
23.62
0.083
0.044
0.160
ours
24.54
0.079
0.036
0.146
Table 5: Ablation on Decoder Inheritance Module (DI) and Residual Anchors Compensation Module (RAC) in Long 360 .
Figure 8: The Ablation study on decoder inheritance. (a) Without inheriting the decoder, the rendered image exhibits noticeable blurriness. In contrast, employing decoder inheritance yields clear details, as visualized in (b).
Method
PSNR ↑
DSSIM 1 ↓
DSSIM 2 ↓
LPIPS ↓
Independent Training
21.85
0.142
0.068
0.316
Shared STF
23.11
0.093
0.046
0.178
ours
24.54
0.079
0.036
0.146
Table 6: Experiments on two clip training strategies. Our method achieves superior objective quality. Experiments are carried out on the Long 360 dataset.
Figure 9: (a) Clip-based method LocalDyGS exhibits temporal instability with small M and fail to train with larger M . In (b, c), our PSNR outperform LocalDyGS on both short and long sequences.