The ability to render scenes at adjustable fidelity from a single model, known as level of detail (LoD), is crucial for practical deployment of 3D Gaussian Splatting (3DGS). Existing discrete LoD methods expose only a limited set of operating points, while concurrent continuous LoD approaches enable smoother scaling but often suffer noticeable quality degradation at full capacity, making LoD a costly design decision. We introduce Matryoshka Gaussian Splatting (MGS), a training framework that enables continuous LoD for standard 3DGS pipelines without sacrificing full-capacity rendering quality. MGS learns a single ordered set of Gaussians such that rendering any prefix, the first k splats, produces a coherent reconstruction whose fidelity improves smoothly with increasing budget. Our key idea is stochastic budget training: each iteration samples a random splat budget and optimises both the corresponding prefix and the full set. This strategy requires only two forward passes and introduces no architectural modifications. Experiments across four benchmarks and six baselines show that MGS matches the full-capacity performance of its backbone while enabling a continuous speed-quality trade-off from a single model. Extensive ablations on ordering strategies, training objectives, and model capacity further validate the designs.
Figures & tables
Figure 1: Continuous LoD need not sacrifice full-capacity quality to enable budget trade-off. Our method, MGS (top), learns an ordered set of Gaussian primitives whose prefixes yield coherent reconstructions at any splat budget. Compared to CLoD-3DGS [ 26 ] (mid) and CLoD-GS [ 4 ] (bot), MGS achieves the highest fidelity at every operating point with quality degrading gracefully under budget reduction. Scene: Garden [ 1 ] .
Figure 2: Framework of Matryoshka Gaussian Splatting.
Method
MipNeRF 360
Tanks & Temples
PSNR ↑
SSIM ↑
LPIPS ↓
AUC fps↑
AUC splats↑
PSNR ↑
SSIM ↑
LPIPS ↓
AUC fps↑
AUC splats↑
3DGS-MCMC [ 17 ]
28.40
0.843
0.133
–
–
24.76
0.877
0.083
–
–
Disc.
H3DGS [ 16 ]
25.41
0.780
0.230
11.62
60.51
20.41
0.808
0.186
8.23
58.84
FlexGaussian [ 36 ]
26.86
0.793
0.237
30.04
62.05
22.89
0.823
0.198
52.38
64.93
MaskGaussian [ 22 ]
27.42
0.815
0.218
34.11
67.10
23.66
0.846
0.181
45.15
68.89
Octree-GS [ 31 ]
27.62
0.813
0.221
21.61
68.43
24.59
0.865
0.157
23.80
67.81
Table 1: Quantitative comparison on four benchmarks. PSNR, SSIM [ 38 ] , LPIPS [ 44 ] reported at highest splat budget per baseline. AUC fps summarises quality–FPS trade-off frontier, and AUC splats encapsulates quality/splat efficiency. The best and second best results are highlighted. ↑ higher is better, ↓ lower is better.
Figure 3: Quality-budget trade-off on Mip-NeRF 360 [ 1 ] , averaged across all nine scenes. Left: Quality ( Qˉ , Eq. 9 ) vs. FPS. Right: Quality vs. number of Gaussian splats. Curves trace continuous LoD models across prefix ratios 1%–100%, and trace discrete LoD models at their recommended operating points respectively. MGS (ours, dark blue) achieves the highest quality at every speed and splat budget, while spanning a much wider FPS range than any baseline.
Figure 4: Qualitative comparison of continuous LoD methods across four benchmarks. Renderings are shown at 5%, 10%, 30%, 60%, and 100% of the full splat budget. We compare MGS with CLoD-3DGS [ 26 ] and CLoD-GS [ 4 ] . Under highly constrained budgets (5–10%), MGS maintains coherent reconstructions with PSNR of 21–28 dB, while both baselines suffer from severe artifacts and quality collapse (11–17 dB).
Figure 5: Additional qualitative results including a failure case. On stump, train, and rome, MGS maintains higher fidelity across all budget levels. On DrJohnson (bottom), CLoD-3DGS [ 26 ] achieves higher peak PSNR at 100% budget (29.1 vs. 27.7 dB); however, MGS still degrades more gracefully at reduced budgets (5–30%).
Figure 6: Ablations on the importance score. The opacity-descending dominates other importance scores across the full budget range.
PSNR ↑
SSIM ↑
LPIPS ↓
AUC fps↑
AUC splats↑
Importance Score
Opacity ↓ (ours)
25.47
0.776
0.174
68.62
61.05
Opacity ↑
24.91
0.743
0.219
30.53
33.48
Color variance ↓
25.25
0.770
0.183
51.94
46.26
SH energy ↓
25.17
0.766
0.187
59.39
47.12
Volume ↓
25.19
0.764
0.183
52.15
44.73
Fixed append
25.51
0.773
0.175
46.41
43.09
Table 2: Ablation study on importance score and budget training strategy. All metrics reported at split=1.0 (full budget). PSNR, SSIM and LPIPS represent raw quality; AUC fps and AUC splats summarise quality–efficiency frontiers. The best and second best results per column are highlighted. ↑ higher is better, ↓ lower is better.
Figure 7: Ablations on Prefix/Full Loss Weight distribution. While prefix-heavy or full-heavy weighting can marginally improve their respective endpoints, a 6:4 ratio provides the best area under the curve. For the sake of simplicity and balanced performance, we utilise a 5:5 ratio for our MGS approach and for all other ablation experiments.
3D Gaussian Splatting (3DGS) enables real-time novel view synthesis, but existing general-purpose acceleration methods suffer severe rendering quality degradation when extended to more complex, large-scale scenes. To address this issue, we propose EffGS, a more general acceleration framework that improves training and rendering efficiency while maintaining reconstruction quality comparable to or better than vanilla 3DGS across bounded and large-scale scenes. EffGS combines frequency-aware guidance, localized density control, and adaptive primitive scale modulation. First, an importance scoring mechanism combines pixel-wise reconstruction errors with a difference-of-Gaussians mask scheduled over training to provide stage-dependent spatial guidance. Second, localized densification and pruning restricts density modifications to Gaussians with valid projected footprints in the sampled views. Third, learnable per-Gaussian scale modulation adjusts effective primitive extent during optimization while retaining the Compact Box rasterization rule. Extensive experiments on bounded and large-scale scene datasets demonstrate a favorable balance between reconstruction quality, training time, and primitive count. Component ablations and matched-primitive-budget comparisons further support the effectiveness of the framework.
Changbai Li, Shuo Yang, Yichen Yang +2
Beihang University · Nanyang Technological University
3D Gaussian Splatting has revolutionized neural rendering with real-time performance. However, scaling this approach to large scenes using Level-of-Detail methods faces critical challenges: inefficient serial traversal consuming over 60% of rendering time, and redundant Gaussian-tile pairs that incur unnecessary processing overhead. To address these limitations, we introduce FilterGS, featuring a parallel filtering mechanism with two complementary filters that select Gaussian elements efficiently without tree traversal. Additionally, we propose a novel GTC metric that quantifies the redundancy of Gaussian-tile key-value pairs. Based on this metric, we introduce a scene-adaptive Gaussian shrinking strategy that effectively reduces redundant pairs. Extensive experiments demonstrate that FilterGS achieves state-of-the-art rendering speeds while maintaining competitive visual quality across multiple large-scale datasets. Project page: https://github.com/xenon-w/FilterGS
3D Gaussian Splatting achieves excellent visual quality with real-time rendering, but at the scale of entire cities it does not fit: a trained model carries millions of primitives and gigabytes of memory, and real-time rendering at high quality on a consumer GPU remains out of reach. We introduce Budgeted-GS, a post-hoc method that turns any trained 3DGS model into a factoring tree, a multi-resolution hierarchy of moment-matched aggregates. After a construction pass of a few seconds, a single quality parameter selects, for each view, the level of detail that fits the memory of the target device, so the same city-scale model serves GPUs with widely different memory capacities. When a new scene is to be trained, the same theory applies: instead of growing a full-sized model and compressing it afterwards, budget-centered training first measures how many primitives the scene needs and then trains the model directly at that size, avoiding the wasted effort of optimizing primitives that are later discarded. Both methods are grounded in a measurable capacity floor, a budget-error law derived from optimal transport in phase space; selection rules certified by recent covering theorems decide which primitives are redundant. The floor answers how many primitives a scene actually needs and how many can safely be given up. We validate the floor on 13 public scenes under a preregistered protocol, and exercise both methods from object scenes to an official city capture, rendering it at native 1920x1080, full SH, in real time on one consumer GPU.