Unfolding 3D Gaussian Splatting via Iterative Gaussian Synopsis
Authors: Yuqin Lu, Yihua Dai, Yang Zhou, Guiqing Li, Shengfeng He
Organizations: School of Computer Science and Engineering, South China University of Technology, Guangzhou, Guangdong 510006, China · School of Computing and Information Systems, Singapore Management University, Singapore 188065
3D Gaussian Splatting (3DGS) has become a state-of-the-art framework for real-time, high-fidelity novel view synthesis. However, its substantial storage requirements and inherently unstructured representation pose challenges for deployment in streaming and resource-constrained environments. Existing Level-of-Detail (LOD) strategies, particularly those based on bottom-up construction, often introduce redundancy or lead to fidelity degradation. To overcome these limitations, we propose Iterative Gaussian Synopsis, a novel framework for compact and progressive rendering through a top-down "unfolding" scheme. Our approach begins with a full-resolution 3DGS model and iteratively derives coarser LODs using an adaptive, learnable mask-based pruning mechanism. This process constructs a multi-level hierarchy that preserves visual quality while improving efficiency. We integrate hierarchical spatial grids, which capture the global scene structure, with a shared Anchor Codebook that models localized details. This combination produces a compact yet expressive feature representation, designed to minimize redundancy and support efficient, level-specific adaptation. The unfolding mechanism promotes inter-layer reusability and requires only minimal data overhead for progressive refinement. Experiments show that our method maintains high rendering quality across all LODs while achieving substantial storage reduction. These results demonstrate the practicality and scalability of our approach for real-time 3DGS rendering in bandwidth- and memory-constrained scenarios.
Figures & tables
Fig. 1: Iterative Gaussian Synopsis enables hierarchical “unfolding” of 3D Gaussian Splatting. The top rows illustrate the anchor structure alongside the derived 3D Gaussians at both full resolution and progressively condensed levels. The bottom rows compare the Level-of-Detail progression from a compact synopsis (starting at 6.5MB) to the highest fidelity (50.7MB) for our method, against a standard 3DGS baseline with a total size of 601.1MB. Our approach achieves clearer intermediate representations and significantly improved efficiency, delivering comparable visual quality at a fraction of the storage cost.
Fig. 2: Comparison of LOD Construction Strategies. Left (Bottom-Up Increment): A conventional bottom-up approach that builds from a coarse base layer by sequentially adding enhancement layers, typically requiring updates to the opacity or parameters of previous layers. Right (Top-Down Unfolding): Our proposed top-down unfolding strategy, where lower levels of detail are systematically derived from a full-fidelity model, ensuring coherent simplification and improved inter-layer consistency.
Fig. 3: Overview of the Iterative Gaussian Synopsis Framework. Top: The full-resolution 3DGS model is progressively “unfolded” into coarser LODs using a learnable, mask-guided pruning strategy. Bottom: The resulting hierarchical LODs support efficient progressive rendering, where visual quality improves smoothly as more data is streamed, transitioning seamlessly from a coarse structural outline to a high-fidelity reconstruction.
Fig. 4: Our hierarchical architecture. The hierarchical grids capture multi-scale contextual information while the learnable codebook models localized details, formimg the foundation for unfolding into LODs.
Level
Metric
Mip-NeRF360 [ 43 ]
BungeeNeRF [ 40 ]
DeepBlending [ 45 ]
Tanks&Temples [ 44 ]
3DGS
Oct-GS
LapisGS
Ours
3DGS
Oct-GS
LapisGS
Ours
3DGS
Oct-GS
LapisGS
Ours
3DGS
Oct-GS
LapisGS
Ours
L0
PSNR ↑
18.39
17.44
28.10
28.02
17.05
19.20
26.23
27.23
20.55
22.56
19.99
27.95
15.10
12.55
23.41
23.68
SSIM ↑
0.465
0.447
0.869
0.860
0.473
0.594
0.886
0.912
0.614
0.720
0.569
0.861
0.492
0.342
0.864
0.865
LPIPS ↓
0.380
0.446
0.117
0.134
0.384
0.293
0.108
0.106
0.324
0.282
0.374
0.142
0.347
0.512
0.119
0.123
FPS ↑
189
180
151
183
157
132
158
159
191
182
125
188
202
229
178
185
Size ↓
99.41
12.40
304.24
9.79
204.59
60.76
253.82
27.36
87.97
12.80
554.02
5.95
52.74
5.44
181.72
6.09
TABLE I: Quantitative comparisons of our method against 3DGS [ 1 ] , Octree-GS (Oct-GS) [ 15 ] , and LapisGS [ 18 ] across different datasets and LOD levels. Best and second-best results are highlighted in red and yellow , respectively. Size is measured in MB. It should be noted that the reported sizes for 3DGS and LapisGS represent their default, uncompressed explicit Gaussian representations. Following LapisGS, metrics at each LOD are computed at the proportionally scaled target resolution of that level for every method. Coarser levels therefore face easier targets and are comparable only within a level.
Fig. 5: Qualitative comparison of LOD rendering across different methods. Our method yields superior visual quality (higher PSNR, and clearer details as shown in zoomed-in views) at each level, while requiring substantially less storage compared to all baseline methods.
Fig. 6: Quantitative results of the ablation study for our adaptation modules. Across all metrics (PSNR, SSIM, LPIPS) and at every level, the full integration of both modules yields the highest fidelity, validating the effectiveness of our framework.
Level
Scaffold-GS (Independent)
Ours (Unified Progressive)
PSNR
SSIM
Tr.(min)
Cum. Size
PSNR
SSIM
Tr.(min)
Cum. Size
Mip.
L0
28.75
0.874
12.2
89.80 MB
28.02
0.860
4.5
9.79 MB
L1
28.15
0.850
17.3
223.23 MB
27.64
0.842
9.0
27.65 MB
L2
27.23
0.813
23.1
388.47 MB
26.95
0.796
12.4
62.79 MB
L3
27.09
0.793
39.0
560.27 MB
27.17
0.790
32.2
90.11 MB
DB
L0
29.13
0.885
7.4
39.10 MB
27.95
0.861
5.0
5.95 MB
TABLE II: Architectural Analysis across multiple datasets. Our unified, progressive framework is compared against Scaffold-GS models trained independently at corresponding scaled resolutions.
3D Gaussian Splatting (3DGS) has recently enabled real-time rendering of unbounded 3D scenes for novel view synthesis. However, this technique requires dense training views to accurately reconstruct 3D geometry. A limited number of input views will significantly degrade reconstruction quality, resulting in artifacts such as "floaters" and "background collapse" at unseen viewpoints. In this work, we introduce SparseGS, an efficient training pipeline designed to address the limitations of 3DGS in scenarios with sparse training views. SparseGS incorporates depth priors, novel depth rendering techniques, and a pruning heuristic to mitigate floater artifacts, alongside an Unseen Viewpoint Regularization module to alleviate background collapses. Our extensive evaluations on the Mip-NeRF360, LLFF, and DTU datasets demonstrate that SparseGS achieves high-quality reconstruction in both unbounded and forward-facing scenarios, with as few as 12 and 3 input images, respectively, while maintaining fast training and real-time rendering capabilities.
While 3D Gaussian Splatting (3DGS) has revolutionized 3D reconstruction, it suffers from significant overhead due to massive redundant primitives. Existing compression methods typically rely on local sampling or fixed pruning thresholds, which often struggle to balance redundancy reduction with high-fidelity rendering. To address this, we propose a novel framework that formulates Gaussian optimization as a global geometric distribution matching problem. Specifically, our approach integrates three components: (1) we introduce a multi-view 3D Gaussian contribution ranking mechanism that filters primitives using geometric consistency instead of local heuristics; (2) we propose a global Optimal Transport (OT)-based aggregation algorithm that merges redundant primitives while preserving the underlying geometry; and (3) we design an OT-based densification operator that maintains the Gaussian's distributional properties for stable optimization. Our approach achieves state-of-the-art rendering quality with only \textbf{10%} primitives and \textbf{10×} accelerated training speeds compared to vanilla 3DGS.
Beizhen Zhao, Sicheng Yu, Ziran Yin +2
The Hong Kong University of Science and Technology (Guangzhou)
While 3D Gaussian Splatting (3DGS) has emerged as a powerful representation for real-time novel view synthesis, rendering high-fidelity scenes often relies on a massive number of Gaussian primitives, incurring substantial storage and computational overhead. Existing simplification techniques are largely intrusive, requiring training-time pruning, architectural modifications, or computationally expensive per-scene fine-tuning. These drawbacks limit their deployment on off-the-shelf pretrained models. In this paper, we propose CVT-GS, a novel optimization-free post-hoc simplification framework that directly compresses trained 3DGS scenes without sacrificing visual fidelity. Our approach first constructs spatially coherent cells over Gaussian centers via a geometry-aware Centroidal Voronoi Tessellation (CVT). Subsequently, a lightweight neural cell merger predicts the geometry and appearance of a single, highly representative Gaussian primitive for each cell under differentiable rendering supervision. By formulating simplification as a rendering-aware many-to-one merging process rather than naive primitive pruning, CVT-GS outputs a standard 3DGS scene that is seamlessly compatible with existing renderers. Experiments on various datasets demonstrate the superiority of our method. Notably, when achieving a 100-fold reduction in Gaussian points, our method operates 12 times faster than state-of-the-art methods while improving the PSNR by 1.3 dB.
Bingxian Li, Yilong Li, Jingliang Peng +6
Beijing Institute of Technology, Beijing, China · Peking University, Beijing, China