GSStream: 3D Gaussian Splatting based Volumetric Scene Streaming System
Authors: Zhiye Tang, Qiudan Zhang, Lei Zhang, Junhui Hou, You Yang, Xu Wang
Organizations: College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, 518060, China. · Department of Computer Science, City University of Hong Kong, Kowloon, Hong Kong SAR, China. · School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan 430074, China.
Recently, the 3D Gaussian splatting (3DGS) technique for real-time radiance field rendering has revolutionized the field of volumetric scene representation, providing users with an immersive experience. But in return, it also poses a large amount of data volume, which is extremely bandwidth-intensive. Cutting-edge researchers have tried to introduce different approaches and construct multiple variants for 3DGS to obtain a more compact scene representation, but it is still challenging for real-time distribution. In this paper, we propose GSStream, a novel volumetric scene streaming system to support 3DGS data format. Specifically, GSStream integrates a collaborative viewport prediction module to better predict users' future behaviors by learning collaborative priors and historical priors from multiple users and users' viewport sequences and a deep reinforcement learning (DRL)-based bitrate adaptation module to tackle the state and action space variability challenge of the bitrate adaptation problem, achieving efficient volumetric scene delivery. Besides, we first build a user viewport trajectory dataset for volumetric scenes to support the training and streaming simulation. Extensive experiments prove that our proposed GSStream system outperforms existing representative volumetric scene streaming systems in visual quality and network usage. Demo video: https://youtu.be/3WEe8PN8yvA.
Figures & tables
Fig. 1 : Visualization effect of different volumetric scene streaming systems. Our proposed GSStream achieves the best visual performance among the state-of-the-art (SOTA) systems with the help of the collaborative viewport prediction module and the DRL-based bitrate adaptation module. Note that these examples are rendered on the stump scene under a network constraint of 120Mbps, at the 10th second (300th frame) of the viewing process.
Fig. 2 : Overview of the proposed GSStream system. ① In the pre-processing phase, the volumetric scene is tiled into K tiles, and each is downsampled into representations in L quality levels. ② Representations are selectively transmitted to the client side to reconstruct a 3DGS scene for display. ③ User’s viewports captured by HMD are utilized to predict future viewports. ④ The predicted future viewports are then utilized to generate the representation selection strategy for the next time slot.
Fig. 3 : Viewport trajectories captured from four of the subjects. Different subjects have different behavioral characteristics in viewing volumetric scenes. For example, subjects like 07 and 16 prefer to surround the scenes randomly, while subjects like 05 and 22 tend to stand in fixed positions with fewer movements.
Fig. 4 : Illustration of the CVP module. The network utilizes both historical information and collaborative information.
Fig. 5 : The illustration of the whole architecture of the proposed DRL-based bitrate adaptation algorithm. The proposed algorithm is based on DDPG, which is under the Actor-Critic framework [ 34 ] , with the actor head π(st) outputting an action at , and the critic head outputting the corresponding action value Q(st,at) .
Scene
bicycle
bonsai
chtf
counter
drjohnson
#Tiles
2,130
529
2,736
180
59
Scene
flowers
garden
kitchen
playroom
room
#Tiles
1,340
693
480
124
197
Scene
sculpture
stump
train
treehill
truck
#Tiles
1,471
11,601
1,267
2,326
1,741
TABLE I : The numbers of tiles of each scene. The 3 scenes marked red construct the testing set while the remaining 12 scenes form the training set.
Fig. 6 : Thumbnails of the volumetric scenes, with scenes marked red forming the testing set, and the others forming the training set.
Fig. 7 : The temporal viewport SSIM of the four volumetric scene streaming systems at each time slot t , under three typical network conditions.
Fig. 8 : The accumulative transmitted data of the four volumetric scene streaming systems at each time slot t , under three typical network conditions.
Scene
Average Viewport SSIM ↑
System
40Mbps
80Mbps
120Mbps
chtf 187MB
ViVo
0.10±0.08
0.15±0.12
0.18±0.13
CaV3
0.56±0.43
0.77±0.38
0.84±0.34
GS3D
0.61±0.31
0.77±0.31
0.85±0.28
Ours
0.64±0.37
0.81±0.32
0.84±0.31
garden 224MB
ViVo
0.49±0.33
0.54±0.32
0.58±0.31
TABLE II : The average viewport SSIM of different methods.
Fig. 9 : Results of the comparison experiments on viewport prediction task. Note that the compared baselines have different lengths in the prediction time window; therefore, the experimental results have limited significance.
Fig. 10 : Results of the ablation experiments.
Fig. 11 : The collected viewport trajectories from 32 subjects.
Streaming 3D reconstruction incrementally builds geometrically consistent scene maps from incoming images or video under tight memory and latency limits.
Real-time mobile volumetric video enables immersive free-viewpoint experiences, but delivering high-quality 3D content under constrained bandwidth and computation remains challenging. Recent 3D Gaussian Splatting (3DGS) systems, such as V3, enable efficient mobile streaming by encoding time-varying Gaussian attributes with commodity video codecs and hardware decoding. However, existing pipelines treat all Gaussians as frame-varying signals, which is poorly suited to full-scene content where large portions remain temporally stable. Our empirical study shows that frame-wise adaptation and video coding of all Gaussians cause quality degradation in complex full-scene content, particularly in temporally stable regions. To address this issue, we propose ResStreamGS, a residual-sensitive dual-stream framework that processes Gaussians according to their temporal behavior. ResStreamGS identifies residual-sensitive primitives through lightweight probe adaptation and reuses stable Gaussians within each Group of Pictures (GOP). Stable Gaussians are transmitted once per GOP, while only residual-sensitive Gaussians remain in the frame-wise hardware-decodable video stream. Experiments on complex full-scene volumetric videos show that ResStreamGS improves post-codec PSNR by 7.51--8.35dB over a controlled V3-style all-Gaussians baseline, reduces amortized payload by up to 56.5%, and achieves 30.883FPS on a Qualcomm Snapdragon 778G platform.
Streaming 3D Gaussian Splatting requires highly scalable, progressive representations. Existing progressive methods rely on discrete layering, accumulating separate splat sets for each level of detail. This structural independence between layers inherently leads to error accumulation, severe splat redundancy, and uncontrolled quality transitions. We propose EvoGS, the first continuous-layering representation. Organized as an Evolution Tree, EvoGS generates finer details via an explicit, wavelet-inspired parent-child refinement. This empowers child nodes to structurally correct ancestral errors, yield inherently sparse and highly compressible inter-layer signals. Extensive experiments show EvoGS eliminates splat redundancy from over 65% to under 25%. Compared to state-of-the-art baselines, it reduces transmission payload and GPU VRAM footprint by up to 2.4× and 5.5×, respectively, and achieves smooth quality transitions optimal for real-time adaptive streaming. Project page: https://yuang-ian.github.io/evogs/
Yuang Shi, Simone Gasparini, Géraldine Morin +1
National University of Singapore, Singapore · IRIT - University of Toulouse, France · IPAL, IRL2955, Singapore
3D scene representation techniques such as neural radiance fields (NeRFs) and Gaussian splatting have made substantial progress in novel view synthesis, achieving high-quality renderings from arbitrary view angles. More recently, such techniques have been extended to dynamic 3D scenes; however, achieving sustainable online free-viewpoint video (FVV) streaming remains challenging, especially for longer videos, due to significant storage demands of detailed scene representations and high reconstruction/rendering speed needs. To address these challenges, we propose Quantized Anchored Residual Coding Gaussian Streaming (QuARC-GS), a quantization-aware 4D scene optimization framework for online dynamic scene reconstruction that achieves ultra-high compression while maintaining reconstruction speed and quality. QuARC-GS represents a scene using a single canonical frame and highly compressed per-frame residuals. Specifically, we compress each residual through two complementary strategies targeting motion, appearance, and densification. We introduce quantization-aware anchor deformation, which suppresses insignificant motion updates while preserving meaningful deformations, maintaining reconstruction quality under low-storage streaming. Furthermore, we design a change-gated densification strategy that allocates new Gaussians only in regions exhibiting genuine temporal changes, effectively eliminating redundant appearance updates and reducing storage overhead. Extensive experiments on widely used datasets demonstrate that QuARC-GS enables competitive reconstruction quality and training speed while cutting per-frame storage by up to 11× compared to the state-of-the-art.
Vu Trung Nghia Nguyen, Yuchen Wang, Kyung Chul Lee +1