QuantileGS: Quantile-Based Conformal Bounds on RGB Rendering Errors for Novel View Synthesis with 3D Gaussian Splatting
Authors: Haiyang Xu, Ziye Zhang, Bin Pan, Xia Xu, Zhenwei Shi
Organizations: School of Statistics and Data Science, LEBPS, KLMDASR, LPMC, AAIS, NITFID, Nankai University · School of Computer Science and Technology, Tiangong University · Department of Aerospace Intelligent Science and Technology, School of Astronautics, Beihang University
Uncertainty quantification for novel view synthesis with 3D Gaussian Splatting (3DGS) assesses rendering reliability at unseen viewpoints. This reliability depends on the discrepancy between rendered and true colors, which cannot be measured without ground-truth images. We therefore quantify uncertainty in this unknown discrepancy through upper bounds on per-pixel RGB mean absolute error. However, learned bounds may underestimate these errors and overstate rendering reliability, motivating calibration to a prescribed coverage level. In this paper, we propose QuantileGS, a post-hoc framework that combines learned rendering-error quantiles with scale-normalized conformal calibration to produce spatially adaptive error bounds. The framework comprises three phases. Phase 1, Rendering-Error Computation, measures RGB mean absolute error between observed images and predictions from a frozen 3DGS model. Phase 2, Ordered Gaussian Quantile Fields, learns two scalar attributes per Gaussian to predict lower and upper rendering-error quantiles. Phase 3, Scale-Normalized Conformal Calibration, calibrates the upper prediction on held-out images, using the quantile gap to scale the correction. Under the stated assumptions, the resulting bounds provide finite-sample marginal coverage. At a nominal coverage level of 90%, calibration raises mean empirical test coverage from 69.6% to 89.6% on LLFF and from 80.1% to 92.1% on Mip-NeRF 360. On LF, coverage remains conservative at 96.3% before and 96.4% after calibration. Applications to active view selection, Gaussian pruning, and RGB-D simultaneous localization and mapping illustrate how these bounds inform downstream decisions.
Figures & tables
Figure 1: QuantileGS estimates rendering-error uncertainty and uses conformal calibration to produce pixel-wise RGB-MAE upper bounds. The reference error map compares the original 3DGS rendering with ground-truth RGB. Calibrated UQ guides model optimization to produce the refined rendering. We apply the resulting bounds to RGB-D SLAM, as well as active view selection and Gaussian pruning.
Figure 2: QuantileGS overview. Phase 1 computes RGB-MAE on Vfit using a frozen pretrained 3DGS model. Phase 2 learns ordered Gaussian error quantiles with fixed blending weights and background contributions. Phase 3 calibrates the upper prediction on Vcal , using the interquantile gap with a positive floor as the local scale. The prediction procedure renders the learned quantiles for a new view and applies a locally scaled conformal correction to produce an RGB-MAE upper-bound map.
Dataset
Method
AUSE ↓
Pears. ↑
Coverage (%)
PSNR ↑
SSIM ↑
LPIPS ↓
LLFF
Variational-3DGS
0.381
0.269
—
26.179
0.847
0.143
FisherRF
0.420
-0.059
—
24.880
0.824
0.156
PRIMU
0.289
0.364
—
24.508
0.820
0.163
Manifold
0.436
0.186
—
25.311
0.826
0.151
QuantileGS-U (ours)
0.250
0.428
69.6
24.407
0.796
0.173
QuantileGS-C (ours)
0.249
0.421
89.6
Table 1: Uncertainty quantification, empirical coverage, and novel-view synthesis results on LLFF, Mip-NeRF 360, and LF. The best , second-best , and third-best results for AUSE and Pearson correlation are highlighted, with the best also in bold. Coverage is the mean empirical coverage across scenes, with a nominal coverage level of 90% ( α=0.1 ). It is reported for both QuantileGS variants; — denotes baselines without conformal calibration, for which coverage is not reported.
Figure 3: Qualitative comparison of Variational-3DGS [ 28 ] , FisherRF [ 23 ] , PRIMU [ 17 ] , Manifold [ 30 ] , and QuantileGS-C (Ours). Each scene shows uncertainty maps. Scenes from top to bottom: Mip-NeRF 360 bicycle and counter, LLFF fern and horns, and LF basket.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
Random
19.361
0.584
0.364
FisherRF
20.316
0.597
0.355
Manifold
20.543
0.609
0.338
PRIMU
20.846
0.636
0.328
QuantileGS (Ours)
21.090
0.635
0.328
Table 2: AVS on Mip-NeRF 360: nine-scene means with 20 reconstruction views. The best , second-best , and third-best results are highlighted, with the best also in bold. Ties at the displayed precision share a rank.
Method
Kept
PSNR ↑
SSIM ↑
LPIPS ↓
Unpruned 3DGS
100%
27.170
0.810
0.215
LightGaussian
20%
26.814
0.794
0.257
PUP 3D-GS
20%
27.037
0.801
0.242
QuantileGS (Ours)
20%
27.090
0.784
0.274
LightGaussian
10%
26.328
0.770
0.300
PUP 3D-GS
10%
26.579
0.785
0.274
Table 3: Pruning on Mip-NeRF 360: nine-scene means, seed 0. Within each retention level, the best and second-best results are highlighted, with the best also in bold. Rendering metrics are computed from floating-point images.
Method
ATE ↓
RPE ↓
FPS ↑
Gaussian-SLAM
0.329
0.0878
0.423
VarSplat
0.237
0.0517
0.256
SplaTAM
0.393
0.0689
0.135
QuantileGS (Ours)
0.290
0.0046
1.058
Table 4: Replica tracking results averaged over eight scenes, with seed 0. ATE and RPE are in cm. The best and second-best results are highlighted, with the best also in bold.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Method
LLFF
Mip-NeRF 360
LF
Flower
Room
Bonsai
Stump
Torch
AUSE ↓
Pears. ↑
AUSE ↓
Pears. ↑
AUSE ↓
Pears. ↑
AUSE ↓
Pears. ↑
AUSE ↓
Pears. ↑
Variational-3DGS
0.379
0.226
0.357
0.279
0.473
0.072
0.526
0.123
0.223
0.434
FisherRF
0.328
-0.094
0.382
-0.070
0.503
-0.071
0.356
-0.020
0.700
-0.051
PRIMU
0.257
0.406
0.375
0.207
0.364
0.264
0.435
0.134
0.345
0.358
Manifold
0.462
0.182
0.468
0.179
0.350
0.302
0.483
0.144
0.514
0.126
Appendix
Table 5: UQ results on five additional selected scenes. The best , second-best , and third-best AUSE and Pearson results are highlighted, with the best also in bold.
Figure 4: Additional qualitative comparisons of Variational-3DGS [ 28 ] , FisherRF [ 23 ] , PRIMU [ 17 ] , Manifold [ 30 ] , and QuantileGS-C (Ours). Each scene shows uncertainty maps. Scenes from top to bottom: Mip-NeRF 360 bonsai and stump, LLFF flower and room, and LF torch.
Dataset
Scene
Coverage (%)
Mean bound
QuantileGS-U
QuantileGS-C
QuantileGS-U
QuantileGS-C
LLFF
fern
57.29
89.57
0.0290
0.0953
flower
73.49
93.48
0.0337
0.0713
fortress
66.36
89.53
0.0254
0.0503
horns
82.40
89.39
0.0408
0.0529
leaves
65.55
89.96
0.0907
0.2319
Appendix
Table 6: Coverage and bound size before and after calibration. The nominal coverage is 90% . Mean bounds are in RGB-MAE units for RGB values in [0,1] . Shaded rows are equal-scene means.
Scene
Gaussian count
Coverage (%)
Mean bound
Original
20%
10%
Original
20%
10%
Original
20%
10%
bicycle
4393644
878729
439365
94.95
97.80
97.94
0.1350
0.1597
0.1630
bonsai
1102270
220454
110227
91.31
95.81
95.99
0.0425
0.0523
0.0545
counter
1015704
203141
101571
91.67
95.91
96.08
0.0554
0.0664
0.0690
flowers
2712997
542600
271300
92.79
96.38
96.79
0.1495
0.1755
0.1828
garden
3309450
661890
330945
90.42
95.39
95.67
0.0642
0.0834
0.0880
Appendix
Table 7: Compression, coverage, and bound size after pruning. The 20% and 10% columns denote Gaussian retention levels. Counts are identical across the three compared methods. QuantileGS bounds for pruned models are transported from the original calibrated model. Coverage (%) and mean RGB-MAE bounds pool test pixels within each scene. Average weights scenes equally. All evaluated coverage-implication violation counts are zero.
Figure 5: Pruning comparisons on Mip-NeRF 360 kitchen (top) and bicycle (bottom). Columns show ground truth, unpruned 3DGS, LightGaussian, PUP 3D-GS, and QuantileGS. Within each scene, the upper and lower row pairs show RGB renderings and RGB-MAE maps at 20% and 10% retention, respectively. All error maps share the [0,0.20] color scale, with errors above 0.20 shown at the maximum color.
Table 10: Fixed and online SLAM calibration on Replica. Both variants use seed 0; Online uses a step size of 0.01 . ATE and translational RPE are RMSE values in centimeters. Bold marks the better value in each Fixed/Online pair. Darker shades indicate better ranks in the equal-scene Average row.
Figure 6: Uncertainty and calibration on eight Replica sequences. (a,b) Mean pre-update RGB coverage and RGB-MAE bounds. The dashed line marks 90%. (c,d) Per-scene ATE and translational RPE in centimeters. Online reuses the previously evaluated runs.
3D Gaussian Splatting (3DGS) renders novel views in real time, but an uncertainty heatmap does not certify that a rendered view meets a certain prediction coverage. We treat novel-view synthesis as structured regression and ask that, with probability at least 1−α, RGB prediction boxes cover at least a 1−β fraction of pixels in a new view. We propose View-Structured Conformal Prediction (VSCP). It splits the pre-calibration scale into a spatial shape from the renderer and a transferable view-difficulty factor, which predicts the smallest view-wise multiplier that shape needs. A held-out quantile over views (View-CP) then gives finite-sample validity even when transferring to new scenes. The same factorization makes the analysis exact: a conformity score is the ratio of oracle to predicted view difficulty, and excess width separates into a test-side and a calibration-side term. Across 13 real scenes, pixel-pooled calibration reaches 89.9% marginal pixel coverage but only 61.4% view-event coverage at a 90% target, while View-CP reaches 91.7--92.0%. At matched coverage VSCP cuts width by 22.1% against a constant scale, and matches a ten-model ensemble's 21.0% reduction using only one model per scene and four rather than ten rasterization passes per query. VSCP also improves on the closest single-model baseline, the 3DGS-U field, by 4.7 points (p=0.0225). The view predictor transfers from bounded source families to all nine unbounded Mip-NeRF360 scenes. There the full scale beats the constant scale with 20.7% width saving on all nine scenes. It also keeps an 18.3% saving under a different densification backbone and runs at 216--280 FPS on an RTX4090.
Junzheng Chu, Bin Pan, Zhenwei Shi
School of Statistics and Data Science, LEBPS, KLMDASR, LPMC, AAIS, NITFID, Nankai University · Department of Aerospace Intelligent Science and Technology, School of Astronautics, Beihang University
3D Gaussian splatting (3DGS) is a strong representation for real-time novel-view synthesis, but its standard training pipeline relies on point estimates and hand-tuned heuristics, providing no native uncertainty or principled complexity control. This is most limiting under sparse views or fixed acquisition budgets, where a model must identify weakly supported geometry and select informative views. We introduce a rendering-aware Bayesian 3DGS framework that tracks Gaussian geometry with a Normal-Inverse-Wishart posterior over means and covariances using renderer-derived surrogate summaries. An optional Dirichlet-process extension adds a probabilistic component-usage signal, and the training schedule makes the closed-form versus approximate inference boundary explicit. Re-rendering posterior geometry samples yields native predictive uncertainty for interval calibration and active view selection. In a fixed-budget 16-to-32 active-view task, native NIW acquisition improves PSNR by +0.453 dB and LPIPS by -0.0146 over a scoring-only 3-member standard-ensemble baseline, winning 29/39 scene-seed pairs and 10/13 scene means; it also improves over PPU-style (+0.355 dB) and NIW-proxy (+0.401 dB) acquisition. NIW native intervals reduce 95% coverage error by about 17x relative to a shared proxy (0.046 vs. 0.796) and are about 10x closer to nominal coverage than a 3-member deep ensemble (0.047 vs. 0.454) at roughly one-third the training cost. As a reconstruction compatibility check, paired NIW-vs-standard analysis over 39 scene-seed runs yields +0.030 dB PSNR with 1.6% additional training time. These results position Bayesian 3DGS as a practical probabilistic scene representation for decision-facing tasks such as active view selection.
Gaoxiang Jia, Vikram Appia, Junzhou Huang +1
Advanced Micro Devices, Inc. · Dept. of Computer Science and Engineering University of Texas at Arlington · Dept. of Mathematics & Division of Data Science University of Texas at Arlington
3D Gaussian Splatting (3DGS) enables real-time novel view synthesis by representing scenes as collections of anisotropic Gaussians optimized via differentiable rasterization. However, standard pixel-space losses (L1, SSIM) constrain only aggregate reconstruction error, permitting the optimization to redistribute error across frequency scales. This leads to oversmoothing and structural artifacts, particularly in sparse-view settings where supervision is limited. We propose KC-3DGS, which augments 3DGS training with wavelet-domain supervision based on natural image statistics. Our method combines three components: (1) a multi-scale wavelet coefficient alignment loss that explicitly penalizes missing high-frequency detail, (2) a supervised kurtosis concentration loss that encourages rendered images to match the heavy-tailed frequency statistics of ground-truth images, and (3) a cross-band covariance penalty that promotes frequency specialization. We provide theoretical analysis showing that pixel-space losses admit a family of indistinguishable perturbations under wavelet redistribution, and that our joint objective excludes degenerate solutions. Experiments across MipNeRF360, Tanks&Temples, MVImgNet, DeepBlending, and WRIVA-ULTRRA demonstrate consistent improvements in perceptual quality. On the challenging WRIVA-ULTRRA outdoor dataset, KC-3DGS achieves a 9.48% improvement in DreamSim while also improving PSNR, SSIM, and LPIPS. In sparse-view settings with only 12 training images, our method improves PSNR by up to 0.5 dB on MipNeRF360 while maintaining perceptual quality. The approach integrates seamlessly into existing 3DGS pipelines as a plug-and-play regularization strategy.