VCURF: Virtual Camera-based Uncertainty of Radiance Fields

Liyan Chen; Nathaniel Burgdorfer; Philippos Mordohai

cs.CVOct 2, 2026

VCURF: Virtual Camera-based Uncertainty of Radiance Fields

Authors: Liyan Chen, Nathaniel Burgdorfer, Philippos Mordohai

Organizations: Stevens Institute of Technology

Abstract

Radiance fields, implemented with either implicit (NeRF) or explicit (Gaussian Splatting) representations, are advancing the state of the art in novel view synthesis at a rapid pace. Even though the rendered views they generate are often compelling, they are not free of errors. In this paper, we propose a new approach for pixel-wise uncertainty quantification based on measuring the inconsistencies among renderings by the radiance field model in virtual cameras sampled near the target viewpoint. We named our approach VCURF for Virtual Camera-based Uncertainty of Radiance Fields. VCURF treats the radiance field model as a black box, only assuming that it is capable of rendering color and depth on demand. This property makes our approach applicable to both NeRF and GS models without any modification. Our experiments on a combination of datasets, radiance field models and baselines demonstrate VCURF's effectiveness in pixel-wise uncertainty estimation. We conclude the paper with findings that question the way view selection is tackled by the majority of the current literature.

Figures & tables

Figure 1: Overview of our approach. Top Left: virtual cameras are sampled around the reference camera. Top Right: uncertainty predicted by VCURF. Bottom: (a) real image, (b-c) depth and color rendered by the RF model, and (d) rendered color image warped from a virtual camera to the reference.

Figure 2: Pixel $p_{i}^{(r)}$ is projected to virtual view $v$ via the reconstructed 3D point $x_{i}^{(r)}$ . Color and depth are obtained by bilinear interpolation at the projected point $p_{i}^{(r,v)}$ , and then associated back to the reference pixel $p_{i}^{(r)}$ .

Figure 3: Illustration of how variance can fail as a measure of inconsistency. $C_{1}$ is the reference, while $C_{2}$ and $C_{3}$ are the virtual cameras. Due to occlusion, when warping the rendered images from $C_{2}$ or $C_{3}$ to $C_{1}$ , the prediction for $A_{1}$ will be associated with $A_{2}$ and $A_{3}$ . However, since $A_{2}$ and $A_{3}$ have similar depths with respect to the reference camera and similar RGB values, the depth and color variance among the three observations will be low. On the other hand, if we use differences with the reference depth and color as the criterion, the inconsistency of $A_{1}$ will be high, which is the correct behavior for a heavily occluded point.

Dataset	Real / Synthetic	Scene / Capture Setup
Blender [ 32 ]	Synthetic	Object-centric
Local Light Field Fusion(LLFF) [ 31 ]	Real	Indoor/Outdoor, forward-facing
Light Field(LF) [ 53 ]	Real	Object-centric
Tanks and Temples(TNT) [ 24 ]	Real	Indoor/Outdoor, large-scale
Deep Blending(DB) [ 15 ]	Real	Indoor/Outdoor, hand-held
Mip-NeRF 360(M360) [ 2 ]	Real	Indoor/Outdoor, object-centric (orbit)

Table 1: Summary of datasets

Panel of Figure 4 — Figure 4: Rendered images, error maps, and corresponding uncertainty maps for one representative scene from each dataset. Within each row, results share the same backbone (indicated in the first column). In these scenes, the RF models can generate reasonably accurate novel views but still exhibits visible errors. We use the color map of Fig. 1 where yellow represents high and blue represents low values. The uncertainty maps of VCURF have better alignment with the error maps compared to the baselines, especially in the red-highlighted regions. The datasets from top to bottom are as follows: [M360, Blender, LF, LLFF]

LLFF	room	t-rex	fortress	horns	avg.
Opt.	0.0423	0.0622	0.0287	0.0967	0.0575
MCD-depth	0.1897	0.3898	0.2455	1.2493	0.5186
MCD-rgb	0.3533	0.3094	0.0822	0.7804	0.3813
Entropy	0.5001	0.8343	0.3667	2.0371	0.9346
VCURF	0.2050	0.4116	0.1783	0.6776	0.3681
LF	africa	basket	statue	torch	avg.

Table 2: Comparison of AUC scores on the LLFF and LF datasets using InstantNGP [ 33 ] as the backbone RF.

M360	kitchen	garden	bicycle	counter	avg.
Opt.	0.0837	0.1040	0.2158	0.0902	0.1234
FisherRF	0.8291	1.1571	2.0789	1.2301	1.3238
WarpRF	0.4902	0.5268	1.3243	0.7166	0.7645
VCURF	0.4837	0.4816	1.2062	0.4945	0.6665
Ensemble Opt	0.0497	0.1057	0.1682	0.0780	0.1004
Ensemble - rgb	0.1422	1.1441	1.0737	0.3951	0.6888

Table 3: Comparison of AUC scores on mip-NeRF 360, Blender, Tanks and Temples, and Deep Blending datasets using 3DGS [ 20 ] as the backbone RF.

Figure 5: Floater clean-up for Train (T&T) and Bicycle (M360) . For each group, the top images are filtered by VCURF while the bottom ones by FisherRF. The plots show PSNR after progressive noise removal.

	garden	bicycle	room	stump	avg.
PSNR
Oracle	21.72	16.46	23.02	19.51	20.18
VCURF	21.40	17.89	22.39	19.66	20.38
SSIM
Oracle	0.634	0.353	0.788	0.435	0.553
VCURF	0.652	0.425	0.793	0.487	0.589

Table 4: Diagnostic active-view-selection experiment comparing candidate ranking by VCURF uncertainty with ranking by true average rendering error. Despite having perfect access to the current rendering errors, the error-based oracle produces worse average NVS performance, showing that rendering error is not equivalent to view utility.

Sample radius	ship	kitchen	playroom	avg.
0.05 $d_{med}$	0.3606	0.5407	1.5623	0.8212
0.1 $d_{med}$	0.3565	0.4837	1.5269	0.7890
0.3 $d_{med}$	0.3565	0.6031	1.7759	0.9118
0.5 $d_{med}$	0.3543	0.6387	1.9711	0.9880
std.	0.0026	0.0685	0.2064	0.0902

Table S.1 : AUC values for different sampling radii, expresssed as fractions of the median depth

d_{med}

, and standard deviation across all runs

Figure S.1 : Illustration of fixed camera configuration. Each virtual camera (gray) is rotated around the scene center $O$ by a small angle $\theta$ along either $X$ or $Y$ axis relative to the reference camera (blue). Consequently, the baselines and distances from the virtual cameras to the scene center remain identical.

$N$ vcam	ship	kitchen	playroom	avg.
4 (fixed)	0.3760	0.5744	1.5779	0.8428
2	0.3594	0.6699	1.9731	1.0008
4	0.3566	0.5762	1.5654	0.8327
6	0.3565	0.4837	1.5269	0.7890
8	0.3607	0.5737	1.6784	0.8709
10	0.3627	0.5221	1.5251	0.8033

Table S.2 : AUC scores for different number of virtual cameras

Panel of Figure 13 — Figure S.2 : Rendered RGB, depth, error maps using 3DGS and uncertainty maps; and densification curves for ship from Blender. (Yellow represents high and blue represents low values.) We only set 4 virtual cameras in each configuration. VCURF with random configuration is better than the fixed version and FisherRF. As shown in the uncertainty map, it can successfully highlight the areas of the ship that cause more errors due to the complexity of the structure.

Components	ship	kitchen	playroom	avg.
depth-only	0.5162	0.8291	3.7496	1.6983
rgb-only	0.3033	0.8366	3.5968	1.5789
depth+rgb	0.3565	0.4837	1.5269	0.7890

Table S.3 : AUC scores for different components of VCURF.

Panel of Figure 15 — Figure S.3 : Rendered images, error maps and corresponding uncertainty maps for one scene from each dataset. From top to bottom, they are: (1) chair from Blender, (2) train from Tank and Temples, (3) bicycle from Mip-Nerf 360 ∘ , (4) africa from the Light Field, (5) t-rex from Local Light Field Fusion. Following the colorbar of the main paper, yellow represents high values and blue represents low values. The uncertainty maps of VCURF have better alignment with the error maps compared to the baselines.

Uncertainty Quantification

Uncertainty quantification estimates how reliable predictions and scientific inferences are, and how uncertainty propagates into decisions.

See all 636 papers

Evidential Neural Radiance Fields
Feb 27, 2026Ruxiao Duan, Alex WongUQ for 3D Reconstruction Uncertainty Quantification
Yale University
PCM-NeRF: Probabilistic Camera Modeling for Neural Radiance Fields under Pose Uncertainty
Apr 20, 2026Shravan Venkatraman, Rakesh Raj Madavan, Pavan Kumar Sathya Venkatesh3D Reconstruction Camera Pose Estimation
Mohamed bin Zayed University of AI · University of Amsterdam · University of Massachusetts Amherst
MU-GeNeRF: Multi-view Uncertainty-guided Generalizable Neural Radiance Fields for Distractor-aware Scene
Apr 20, 2026Wenjie Mu, Zhan Li, Chuanzhou Su +83D Reconstruction Neural Rendering
School of Computer Science and Technology, Tongji University, Shanghai, China · MOE Key Lab of Embedded System and Service Computing, Tongji University, Shanghai, China · College of Surveying and Geographic Informatics, Tongji University, Shanghai, China +1

Feb 27, 2026J/K move · Enter open · S save

Evidential Neural Radiance Fields

Ruxiao Duan, Alex Wong

Yale University

UQ for 3D Reconstruction Uncertainty Quantification Neural Radiance Fields Uncertainty Decomposition

Open paper AskPDF arXiv

Understanding sources of uncertainty is fundamental to trustworthy three-dimensional scene modeling. While recent advances in neural radiance fields (NeRFs) achieve impressive accuracy in scene reconstruction and novel view synthesis, the lack of uncertainty estimation significantly limits their deployment in safety-critical settings. Existing uncertainty quantification methods for NeRFs fail to separately capture both aleatoric and epistemic uncertainties. Among those that do quantify one or the other, many of them either compromise rendering quality or incur significant computational overhead to obtain uncertainty estimates. To address these issues, we introduce Evidential Neural Radiance Fields, a probabilistic approach that seamlessly integrates with the NeRF rendering process, enabling direct quantification of both aleatoric and epistemic uncertainties from a single forward pass. We compare multiple uncertainty quantification methods on three standardized benchmarks, where our approach demonstrates state-of-the-art scene reconstruction fidelity and uncertainty estimation quality. Code is available at https://github.com/KerryDRX/EvidentialNeRF.

Abstract

Uncertainty Quantification

Explore similar work

Evidential Neural Radiance Fields

PCM-NeRF: Probabilistic Camera Modeling for Neural Radiance Fields under Pose Uncertainty

MU-GeNeRF: Multi-view Uncertainty-guided Generalizable Neural Radiance Fields for Distractor-aware Scene