3D scene representation techniques such as neural radiance fields (NeRFs) and Gaussian splatting have made substantial progress in novel view synthesis, achieving high-quality renderings from arbitrary view angles. More recently, such techniques have been extended to dynamic 3D scenes; however, achieving sustainable online free-viewpoint video (FVV) streaming remains challenging, especially for longer videos, due to significant storage demands of detailed scene representations and high reconstruction/rendering speed needs. To address these challenges, we propose Quantized Anchored Residual Coding Gaussian Streaming (QuARC-GS), a quantization-aware 4D scene optimization framework for online dynamic scene reconstruction that achieves ultra-high compression while maintaining reconstruction speed and quality. QuARC-GS represents a scene using a single canonical frame and highly compressed per-frame residuals. Specifically, we compress each residual through two complementary strategies targeting motion, appearance, and densification. We introduce quantization-aware anchor deformation, which suppresses insignificant motion updates while preserving meaningful deformations, maintaining reconstruction quality under low-storage streaming. Furthermore, we design a change-gated densification strategy that allocates new Gaussians only in regions exhibiting genuine temporal changes, effectively eliminating redundant appearance updates and reducing storage overhead. Extensive experiments on widely used datasets demonstrate that QuARC-GS enables competitive reconstruction quality and training speed while cutting per-frame storage by up to 11× compared to the state-of-the-art.
Figures & tables
Figure 1: Rate-distortion comparison on N3DV ( Li et al. 2022b ) . QuARC-GS streams dynamic scenes at a fraction of the per-frame storage of recent online methods (ReCon-GS † ( Fu et al. 2025 ) , QUEEN ( Girish et al. 2024 ) , ComGS ( Chen et al. 2025 ) , 4DGC ( Hu et al. 2025 ) , 3DGStream † ( Sun et al. 2024 ) ) and the offline 4DGS ( Wu et al. 2024 ) , while matching their rendering quality. Methods marked with † are reproduced by us in the same environment.
Figure 2: Overview of QuARC-GS for online FVV: (a) QuARC-GS starts with a base 3DGS from multi-view images at t=0 optimized with noise injection ( Gao et al. 2024 ) . This canonical 3DGS will be causally transformed to represent the scene at subsequent time points ( t>0 ). (b) Following ( Fu et al. 2025 ) , QuARC-GS takes advantage of a hierarchical anchor-based representation of the deformation of the Gaussian 6D poses ( δr,δt ). Our insight was to quantize these deformations, resulting in a large number of static anchors thereby delivering substantial storage savings. (c) Complementarily, QuARC-GS includes a novel module that stems growth in number of Gaussians based on view-space gradients, resulting in storage savings. (d) The quantized anchor residuals and incremental Gaussian attributes are used to warm-start the next frame, or entropy-coded and transmitted to enable FVV.
Category
Method
PSNR (dB) ↑
SSIM ↑
LPIPS ↓
Storage (KB) ↓
Train (sec) ↓
Render (FPS) ↑
Offline
STG
32.05
0.948
–
670
20
140
SaRO-GS
32.15
–
–
1000
–
40
Swift4D
32.23
–
–
400
5.0
125
SplineGS
32.60
–
–
–
11
76
Online
Dynamic 3DGS
30.67
–
–
–/9200
560
–
StreamRF
30.68
0.930
–
17700/31500
15
12
Table 1: Quantitative comparison on the N3DV dataset. The storage metric is reported without and with the initial frame, separated by “/”; the training-time metric includes first-frame training. Methods marked with † are reproduced by us with official code in the same environment as a mean of 3 runs. The top-3 results are highlighted in each column.
Method
PSNR (dB) ↑
SSIM ↑
Storage (KB) ↓
Train (sec) ↓
FPS ↑
3DGStream †
29.30
0.948
4000/4100
4.77
260
HiCoM
26.73
–
600
9.8
258
ReCon-GS †
30.68
0.949
300/390
3.2
303
QuARC-GS
30.67
0.949
18/26
3.3
271
Table 2: Quantitative comparison on the MeetRoom dataset. The storage metric is reported without and with the initial frame, separated by “/”; the training-time metric includes first-frame training. Methods marked with † are reproduced by us with official code in the same environment.
Figure 3: Qualitative comparison of held-out-view renders from QuARC-GS (ours), ReCon-GS ( Fu et al. 2025 ) , and 3DGStream ( Sun et al. 2024 ) on the Sear Steak scene from the N3DV dataset ( Li et al. 2022b ) and the trimming scene from the Meeting Room dataset ( Li et al. 2022a ) . QuARC-GS maintains quality while substantially reducing storage.
Variant
PSNR (dB) ↑
SSIM ↑
LPIPS ↓
Storage (KB/f) ↓
New GS/f ↓
Full (Ours)
32.16
0.951
0.129
36/56
153
w/o quantization
32.16
0.951
0.129
402/441
154
w/o change gate
32.09
0.951
0.129
40/60
468
Table 3: Ablation of the QuARC-GS components on N3DV. “New GS/f” is the incremental Gaussians added per frame (excluding pruning). The storage metric is reported without and with the initial frame, separated by “/”.
Figure 4: Rate-distortion frontier (PSNR versus per-frame storage on N3DV) as the anchor quantization step ( Δt ) varies. The shipped configuration lies at the knee of the frontier.
Figure 5: Net total Gaussians added per frame on N3DV, with and without change-gated densification. The change gate confines new Gaussians to genuinely changing regions, keeping the scene size nearly constant over the sequence.
Streaming 3D reconstruction demands both speed and temporal fidelity, goals that existing methods undermine by updating every Gaussian every frame, even in static regions. We present DecoGS, a method for efficient online training of 3D Gaussians from streaming videos. Unlike prior methods that update the entire scene indiscriminately, DecoGS introduces an adaptive mechanism that selectively focuses optimization on spatiotemporal regions exhibiting motion or photometric changes. This targeted training strategy eliminates redundant updates that cause flickering and drift in nominally static regions, while enabling fast, high-fidelity scene updates. The pipeline further integrates region-aware Gaussian management through gradient gating and efficient visibility filtering to maintain temporal coherence and a compact memory footprint. On N3DV and MeetRoom, DecoGS achieves 34.55 and 31.60 dB PSNR respectively, outperforming all streaming and offline baselines, while rendering at 261 FPS with 70× lower temporal flicker than the best prior method, requiring no large-scale pretraining.
Dynamic 3D Gaussian Splatting (3DGS) holds great promise as a 3D video streaming technology since it can represent complex 3D scenes with high fidelity. In this approach, every frame in a 3D video represents the environment as a collection of Gaussians with position and other attributes such as scale, rotation, opacity, and color. Frames capture fine details, permit views from any arbitrary perspective, but are an order of magnitude, or more, larger than 2D video frames. A line of recent work has explored how to compress dynamic 3DGS frames, but these approaches are often slow, in part because their compression techniques are not amenable to efficient acceleration. GS-NFS accelerates dynamic 3DGS compression and decompression on a GPU, to the point where it can encode and decode at full frame rate. It achieves this by developing novel GPU-based parallelizations of existing algorithms for encoding both positions and attributes of Gaussians. As a result, it is 1-2 orders of magnitude faster than the state-of-the-art in encoding and decoding a frame, while offering competitive compression performance and rendering quality.
Constructing photorealistic Free-Viewpoint Videos (FVVs) of dynamic scenes from a set of posed 2D images has been an intriguing yet challenging task in computer vision. Methods based on neural rendering achieve high-fidelity image quality in FVV construction. However, most of these methods are unable to achieve real-time rendering and often require complete video sequences to train. Despite the existence of some online training methods capable of rendering FVVs in real time, they struggle to meet the requirements for storage and training time for downstream applications. To overcome this problem, we propose Struct-GStream, which can achieve efficient FVV streaming using structured 3D Gaussians (3DGs). Specifically, we introduce dynamic anchor points to generate structured 3DGs to construct basic scenes and model approximate scene movements based on the assumption of local rigidity in object motion. Besides, we introduce a global free 3DGs patching strategy involving free 3DGs' generation, pruning, and optimization to patch and model deficient areas and emerging objects. Our method achieves fast training at low bitrates while maintaining high rendering quality. Extensive experiments demonstrate that Struct-GStream significantly outperforms existing online training methods for FVV construction in terms of training time, storage, and rendering quality while maintaining competitive rendering speed.
Han Jiao, Jiakai Sun, Lei Zhao +4
College of Computer Science and Technology, Zhejiang University, China · Chinese Academy of Sciences, Beijing, China · University of Science and Technology Beijing, China