Organizations: Guangdong Provincial Key Laboratory of Ultra High Definition Immersive Media Technology, Shenzhen Graduate School, Peking University · Pengcheng Laboratory · Peking University · MIGU Video Co., Ltd.
Dynamic 3D scene reconstruction is essential for immersive media such as VR, MR, and XR, yet remains challenging for long multi-view sequences with large-scale motion. Existing dynamic Gaussian approaches are either Frame-Stream, offering scalability but poor temporal stability, or Clip, achieving local consistency at the cost of high memory and limited sequence length. We propose ClipGStream, a hybrid reconstruction framework that performs stream optimization at the clip level rather than the frame level. The sequence is divided into short clips, where dynamic motion is modeled using clip-independent spatio-temporal fields and residual anchor compensation to capture local variations efficiently, while inter-clip inherited anchors and decoders maintain structural consistency across clips. This Clip-Stream design enables scalable, flicker-free reconstruction of long dynamic videos with high temporal coherence and reduced memory overhead. Extensive experiments demonstrate that ClipGStream achieves state-of-the-art reconstruction quality and efficiency. The project page is available at: https://liangjie1999.github.io/ClipGStreamWeb/
Figures & tables
Figure 2: Our training process is divided into two stages: the Reference Clip ( Clip0 ) Training Stage and Source Clip ( Clipn∈[1,N−1] ) Training Stage. In Reference Clip Training Stage, anchors A0 are initialized by the fused point cloud from all frames of Clip0 . The anchors are fed into a spatio-temporal field ( STF0 ) to extract dynamic features, which are then concatenated with static features. They are decoded to Temporal Gaussian attributes and rendered via rasterization. In Source Clip Training Stage, we inherit the anchors, decoder and the static features from Reference Clip. Residual anchors are computed between the current and Reference Clip A0 . Finally, the same training pipeline as in the Reference Clip stage is applied using the An conpensated with residual anchors and the newly initialized STF .
Figure 3: (a)Static features fs learns all background information, therefore sharing fs across clips ensures temporal consistency. (b) Dynamic features learns residual information that controls the visibility of dynamic content, and thus are clip-independent.
Figure 4: Geometry-aware deduplication. (a) The points from the Reference Clip is converted into a spherical coverage field : for each point p , a sphere s is centered at p with radius set to the mean distance to its three nearest neighbors (computed via KNN). (b) Given the field and the Source Clip candidate anchors, residual anchors are selected based on their signed distance to the field surface. q1 (red box), SDF(q1)<0 , is filtered out; while q2 (green box), SDF(q2)>0 , is retained as a residual anchor.
Figure 5: The ablation study on the Residual Anchors Compensation Module (RAC) and the Anchors Inheritance Module (AI) . As seen from the residual heatmaps between adjacent clips, removing either module leads to strong responses in static regions as shown in (a)(b)(c), while enabling them, as illustrated in (d), significantly suppresses flicker and preserves smooth clip transitions, which demonstrates that both components play essential roles in maintaining inter clip stability.
Figure 6: Qualitative results on Long 360 (1,400 frames; extreme motion amplitudes and long-sequence challenges) and VRU GZ [ 31 ] (complex dynamic interactions). Compared to 4DGaussian [ 29 ] and LocalDyGS [ 31 ] , our method produces sharper renderings in both dynamic regions (e.g., athletes) and static areas (e.g., court floor), with more stable and temporally coherent reconstructions. Additional results are provided in the supplementary video and materials .
Method
PSNR ↑
DSSIM 1 ↓
DSSIM 2 ↓
LPIPS ↓
Static methods
2DGS [ 23 ]
23.80
0.085
0.042
0.181
3DGS [ 54 ]
24.13
0.087
0.040
0.159
ScaffoldGS [ 56 ]
23.53
0.096
0.048
0.203
Frame-Stream methods
3DGStream [ 27 ]
21.94
0.105
0.053
0.200
Table 1: Quantitative comparison on Long 360 , which contains 1400 frames. Static methods are tested on frame 0.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
GOF [ 21 ]
30.39
0.949
0.141
2DGS [ 23 ]
30.78
0.949
0.187
3DGS [ 54 ]
30.50
0.949
0.171
4DGaussian [ 29 ]
28.32
0.930
0.186
SpaceTimeGS [ 30 ]
27.42
0.926
0.193
LocalDyGS [ 31 ]
30.58
0.944
0.173
Table 2: Quantitative comparison on VRU ( GZ ) dataset [ 39 ] . Static methods are evaluated on frame 0 only, serving as an upper-bound reference for dynamic reconstruction.
Figure 7: Qualitative results of flame salmon from the N3DV dataset [ 14 ] , a fine-scale motion sequence . Our method outperforms SOTA approaches (Grid4D [ 42 ] , LocalDyGS [ 31 ] ) by better preserving details in dynamic regions such as the dog’s face and flames.
Method
PSNR ↑
DSSIM 1 ↓
DSSIM 2 ↓
FPS ↑
Time ↓
Size ↓
Frame-Stream methods
StremRF [ 40 ]
28.26
-
-
10.9
-
5310MB
3DGStream [ 27 ]
31.67
-
-
215
1.0h
1230MB
4DGC [ 64 ]
31.58
-
-
-
-
-
HiCoM [ 45 ]
31.17
-
-
-
-
-
Clip methods
Table 3: Quantitative comparisons are conducted on the Neural 3D Video Dataset [ 63 ] , comprising five 300-frame scenes.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
4DGaussian [ 29 ]
28.89
0.952
0.196
4K4D [ 65 ]
21.29
0.826
0.196
ENeRF [ 66 ]
23.48
0.894
0.259
3DGS [ 54 ]
28.61
0.949
0.210
Dy3DGS [ 26 ]
25.91
0.880
0.255
LocalDyGS [ 31 ]
28.15
0.912
0.153
Table 4: Quantitative comparison on flame salmon (1200f).
Method
PSNR ↑
DSSIM1↓
DSSIM2↓
LPIPS ↓
w/o DI
24.34
0.081
0.038
0.152
w/o RAC
23.62
0.083
0.044
0.160
ours
24.54
0.079
0.036
0.146
Table 5: Ablation on Decoder Inheritance Module (DI) and Residual Anchors Compensation Module (RAC) in Long 360 .
Figure 8: The Ablation study on decoder inheritance. (a) Without inheriting the decoder, the rendered image exhibits noticeable blurriness. In contrast, employing decoder inheritance yields clear details, as visualized in (b).
Method
PSNR ↑
DSSIM 1 ↓
DSSIM 2 ↓
LPIPS ↓
Independent Training
21.85
0.142
0.068
0.316
Shared STF
23.11
0.093
0.046
0.178
ours
24.54
0.079
0.036
0.146
Table 6: Experiments on two clip training strategies. Our method achieves superior objective quality. Experiments are carried out on the Long 360 dataset.
Figure 9: (a) Clip-based method LocalDyGS exhibits temporal instability with small M and fail to train with larger M . In (b, c), our PSNR outperform LocalDyGS on both short and long sequences.
Online reconstruction of dynamic 4D scenes from long, unposed streaming videos requires both continuous processing and photorealistic rendering, which existing methods struggle to achieve simultaneously. Existing feed-forward Gaussian methods are restricted to offline processing, whereas online point-cloud approaches struggle to maintain dense geometry and high-fidelity rendering. We present DynStream, a framework for streaming 4D Gaussian reconstruction from long, unposed videos. Given a continuous video stream, DynStream reconstructs the scene within local temporal windows and incrementally aligns and fuses these local reconstructions into a globally consistent scene, enabling online 4D reconstruction without per-scene optimization. By jointly enforcing cross-window geometric consistency and modeling time-varying scene content, DynStream supports efficient reconstruction and photorealistic rendering over extended video streams. Experiments demonstrate that DynStream enables high-fidelity online dynamic reconstruction and rendering from long video streams, achieving state-of-the-art performance across diverse dynamic indoor and outdoor scenes.
Dingwei Xian, Xiaoyu Zhou, Yajiao Xiong +2
Wangxuan Institute of Computer Technology, Peking University · VGI Labs Co., Ltd. · University of California, Merced
3D scene representation techniques such as neural radiance fields (NeRFs) and Gaussian splatting have made substantial progress in novel view synthesis, achieving high-quality renderings from arbitrary view angles. More recently, such techniques have been extended to dynamic 3D scenes; however, achieving sustainable online free-viewpoint video (FVV) streaming remains challenging, especially for longer videos, due to significant storage demands of detailed scene representations and high reconstruction/rendering speed needs. To address these challenges, we propose Quantized Anchored Residual Coding Gaussian Streaming (QuARC-GS), a quantization-aware 4D scene optimization framework for online dynamic scene reconstruction that achieves ultra-high compression while maintaining reconstruction speed and quality. QuARC-GS represents a scene using a single canonical frame and highly compressed per-frame residuals. Specifically, we compress each residual through two complementary strategies targeting motion, appearance, and densification. We introduce quantization-aware anchor deformation, which suppresses insignificant motion updates while preserving meaningful deformations, maintaining reconstruction quality under low-storage streaming. Furthermore, we design a change-gated densification strategy that allocates new Gaussians only in regions exhibiting genuine temporal changes, effectively eliminating redundant appearance updates and reducing storage overhead. Extensive experiments on widely used datasets demonstrate that QuARC-GS enables competitive reconstruction quality and training speed while cutting per-frame storage by up to 11× compared to the state-of-the-art.
Vu Trung Nghia Nguyen, Yuchen Wang, Kyung Chul Lee +1
Dynamic scene reconstruction and novel view synthesis are fundamental to next-generation visual intelligence applications such as virtual reality, robotics, and digital twins. However, high-fidelity reconstruction of complex, time-varying scenes from arbitrary viewpoints remains a significant challenge. Existing dynamic 3DGS methods suffer from computational inefficiency, since they model all Gaussians as dynamic components. While recent decomposition-based approaches address this issue, they still struggle with degraded reconstruction quality and prolonged training time. To mitigate these limitations, we propose a novel dynamic reconstruction framework built upon an efficient static-dynamic decomposition strategy using a Feed-Forward Gaussian Splatting encoder and an optical flow model. By eliminating redundant computations on static regions, our method achieves state-of-the-art performance, outperforming existing baselines across rendering quality, training and rendering speed, and storage efficiency. Notably, on the Neural 3D dataset, our framework requires only 10 minutes for training and achieves a rendering speed of over 700 FPS on a single NVIDIA RTX 5090 GPU at resolution of 1352x1014. Furthermore, our decomposition strategy eliminates the need for COLMAP preprocessing and enables deterministic initialization, thereby enhancing both efficiency and reproducibility.
Youngtae Han, Sung-hwan Han, Youngmin Yi
Department of Artificial Intelligence Engineering, Sogang University