VCURF: Virtual Camera-based Uncertainty of Radiance Fields

Liyan Chen; Nathaniel Burgdorfer; Philippos Mordohai

cs.CVOct 2, 2026

VCURF: Virtual Camera-based Uncertainty of Radiance Fields

Authors: Liyan Chen, Nathaniel Burgdorfer, Philippos Mordohai

Organizations: Stevens Institute of Technology

Abstract

Radiance fields, implemented with either implicit (NeRF) or explicit (Gaussian Splatting) representations, are advancing the state of the art in novel view synthesis at a rapid pace. Even though the rendered views they generate are often compelling, they are not free of errors. In this paper, we propose a new approach for pixel-wise uncertainty quantification based on measuring the inconsistencies among renderings by the radiance field model in virtual cameras sampled near the target viewpoint. We named our approach VCURF for Virtual Camera-based Uncertainty of Radiance Fields. VCURF treats the radiance field model as a black box, only assuming that it is capable of rendering color and depth on demand. This property makes our approach applicable to both NeRF and GS models without any modification. Our experiments on a combination of datasets, radiance field models and baselines demonstrate VCURF's effectiveness in pixel-wise uncertainty estimation. We conclude the paper with findings that question the way view selection is tackled by the majority of the current literature.

Figures & tables

Figure 1: Overview of our approach. Top Left: virtual cameras are sampled around the reference camera. Top Right: uncertainty predicted by VCURF. Bottom: (a) real image, (b-c) depth and color rendered by the RF model, and (d) rendered color image warped from a virtual camera to the reference.

Figure 2: Pixel $p_{i}^{(r)}$ is projected to virtual view $v$ via the reconstructed 3D point $x_{i}^{(r)}$ . Color and depth are obtained by bilinear interpolation at the projected point $p_{i}^{(r,v)}$ , and then associated back to the reference pixel $p_{i}^{(r)}$ .

Figure 3: Illustration of how variance can fail as a measure of inconsistency. $C_{1}$ is the reference, while $C_{2}$ and $C_{3}$ are the virtual cameras. Due to occlusion, when warping the rendered images from $C_{2}$ or $C_{3}$ to $C_{1}$ , the prediction for $A_{1}$ will be associated with $A_{2}$ and $A_{3}$ . However, since $A_{2}$ and $A_{3}$ have similar depths with respect to the reference camera and similar RGB values, the depth and color variance among the three observations will be low. On the other hand, if we use differences with the reference depth and color as the criterion, the inconsistency of $A_{1}$ will be high, which is the correct behavior for a heavily occluded point.

Dataset	Real / Synthetic	Scene / Capture Setup
Blender [ 32 ]	Synthetic	Object-centric
Local Light Field Fusion(LLFF) [ 31 ]	Real	Indoor/Outdoor, forward-facing
Light Field(LF) [ 53 ]	Real	Object-centric
Tanks and Temples(TNT) [ 24 ]	Real	Indoor/Outdoor, large-scale
Deep Blending(DB) [ 15 ]	Real	Indoor/Outdoor, hand-held
Mip-NeRF 360(M360) [ 2 ]	Real	Indoor/Outdoor, object-centric (orbit)

Table 1: Summary of datasets

Panel of Figure 4 — Figure 4: Rendered images, error maps, and corresponding uncertainty maps for one representative scene from each dataset. Within each row, results share the same backbone (indicated in the first column). In these scenes, the RF models can generate reasonably accurate novel views but still exhibits visible errors. We use the color map of Fig. 1 where yellow represents high and blue represents low values. The uncertainty maps of VCURF have better alignment with the error maps compared to the baselines, especially in the red-highlighted regions. The datasets from top to bottom are as follows: [M360, Blender, LF, LLFF]

LLFF	room	t-rex	fortress	horns	avg.
Opt.	0.0423	0.0622	0.0287	0.0967	0.0575
MCD-depth	0.1897	0.3898	0.2455	1.2493	0.5186
MCD-rgb	0.3533	0.3094	0.0822	0.7804	0.3813
Entropy	0.5001	0.8343	0.3667	2.0371	0.9346
VCURF	0.2050	0.4116	0.1783	0.6776	0.3681
LF	africa	basket	statue	torch	avg.

Table 2: Comparison of AUC scores on the LLFF and LF datasets using InstantNGP [ 33 ] as the backbone RF.

M360	kitchen	garden	bicycle	counter	avg.
Opt.	0.0837	0.1040	0.2158	0.0902	0.1234
FisherRF	0.8291	1.1571	2.0789	1.2301	1.3238
WarpRF	0.4902	0.5268	1.3243	0.7166	0.7645
VCURF	0.4837	0.4816	1.2062	0.4945	0.6665
Ensemble Opt	0.0497	0.1057	0.1682	0.0780	0.1004
Ensemble - rgb	0.1422	1.1441	1.0737	0.3951	0.6888

Table 3: Comparison of AUC scores on mip-NeRF 360, Blender, Tanks and Temples, and Deep Blending datasets using 3DGS [ 20 ] as the backbone RF.

Figure 5: Floater clean-up for Train (T&T) and Bicycle (M360) . For each group, the top images are filtered by VCURF while the bottom ones by FisherRF. The plots show PSNR after progressive noise removal.

	garden	bicycle	room	stump	avg.
PSNR
Oracle	21.72	16.46	23.02	19.51	20.18
VCURF	21.40	17.89	22.39	19.66	20.38
SSIM
Oracle	0.634	0.353	0.788	0.435	0.553
VCURF	0.652	0.425	0.793	0.487	0.589

Table 4: Diagnostic active-view-selection experiment comparing candidate ranking by VCURF uncertainty with ranking by true average rendering error. Despite having perfect access to the current rendering errors, the error-based oracle produces worse average NVS performance, showing that rendering error is not equivalent to view utility.

Sample radius	ship	kitchen	playroom	avg.
0.05 $d_{med}$	0.3606	0.5407	1.5623	0.8212
0.1 $d_{med}$	0.3565	0.4837	1.5269	0.7890
0.3 $d_{med}$	0.3565	0.6031	1.7759	0.9118
0.5 $d_{med}$	0.3543	0.6387	1.9711	0.9880
std.	0.0026	0.0685	0.2064	0.0902

Table S.1 : AUC values for different sampling radii, expresssed as fractions of the median depth

d_{med}

, and standard deviation across all runs

Figure S.1 : Illustration of fixed camera configuration. Each virtual camera (gray) is rotated around the scene center $O$ by a small angle $\theta$ along either $X$ or $Y$ axis relative to the reference camera (blue). Consequently, the baselines and distances from the virtual cameras to the scene center remain identical.

$N$ vcam	ship	kitchen	playroom	avg.
4 (fixed)	0.3760	0.5744	1.5779	0.8428
2	0.3594	0.6699	1.9731	1.0008
4	0.3566	0.5762	1.5654	0.8327
6	0.3565	0.4837	1.5269	0.7890
8	0.3607	0.5737	1.6784	0.8709
10	0.3627	0.5221	1.5251	0.8033

Table S.2 : AUC scores for different number of virtual cameras

Panel of Figure 13 — Figure S.2 : Rendered RGB, depth, error maps using 3DGS and uncertainty maps; and densification curves for ship from Blender. (Yellow represents high and blue represents low values.) We only set 4 virtual cameras in each configuration. VCURF with random configuration is better than the fixed version and FisherRF. As shown in the uncertainty map, it can successfully highlight the areas of the ship that cause more errors due to the complexity of the structure.

Components	ship	kitchen	playroom	avg.
depth-only	0.5162	0.8291	3.7496	1.6983
rgb-only	0.3033	0.8366	3.5968	1.5789
depth+rgb	0.3565	0.4837	1.5269	0.7890

Table S.3 : AUC scores for different components of VCURF.

Panel of Figure 15 — Figure S.3 : Rendered images, error maps and corresponding uncertainty maps for one scene from each dataset. From top to bottom, they are: (1) chair from Blender, (2) train from Tank and Temples, (3) bicycle from Mip-Nerf 360 ∘ , (4) africa from the Light Field, (5) t-rex from Local Light Field Fusion. Following the colorbar of the main paper, yellow represents high values and blue represents low values. The uncertainty maps of VCURF have better alignment with the error maps compared to the baselines.

Uncertainty Quantification

Uncertainty quantification estimates how reliable predictions and scientific inferences are, and how uncertainty propagates into decisions.

See all 636 papers

Feb 27, 2026

Evidential Neural Radiance Fields

Understanding sources of uncertainty is fundamental to trustworthy three-dimensional scene modeling. While recent advances in neural radiance fields (NeRFs) achieve impressive accuracy in scene reconstruction and novel view synthesis, the lack of uncertainty estimation significantly limits their deployment in safety-critical settings. Existing uncertainty quantification methods for NeRFs fail to separately capture both aleatoric and epistemic uncertainties. Among those that do quantify one or the other, many of them either compromise rendering quality or incur significant computational overhead to obtain uncertainty estimates. To address these issues, we introduce Evidential Neural Radiance Fields, a probabilistic approach that seamlessly integrates with the NeRF rendering process, enabling direct quantification of both aleatoric and epistemic uncertainties from a single forward pass. We compare multiple uncertainty quantification methods on three standardized benchmarks, where our approach demonstrates state-of-the-art scene reconstruction fidelity and uncertainty estimation quality. Code is available at https://github.com/KerryDRX/EvidentialNeRF.

Ruxiao Duan, Alex Wong

Yale University

UQ for 3D Reconstruction Uncertainty Quantification Neural Radiance Fields Uncertainty Decomposition

Ask PDF

Apr 20, 2026

PCM-NeRF: Probabilistic Camera Modeling for Neural Radiance Fields under Pose Uncertainty

Neural surface reconstruction methods typically treat camera poses as fixed values, assuming perfect accuracy from Structure-from-Motion (SfM) systems. This assumption breaks down with imperfect pose estimates, leading to distorted or incomplete reconstructions. We present PCM-NeRF, a probabilistic framework that augments neural surface reconstruction with per-camera learnable uncertainty, built on top of SG-NeRF. Rather than treating all cameras equally throughout optimization, we represent each pose as a distribution with a learnable mean and variance, initialized from SfM correspondence quality. An uncertainty regularization loss couples the learned variance to view confidence, and the resulting uncertainty directly modulates the effective pose learning rate: uncertain cameras receive damped gradient updates, preventing poorly initialized views from corrupting the reconstruction. This lightweight mechanism requires no changes to the rendering pipeline and adds negligible overhead. Experiments on challenging scenes with severe pose outliers demonstrate that PCM-NeRF consistently outperforms state-of-the-art methods in both Chamfer Distance and F-Score, particularly for geometrically complex structures, without requiring foreground masks.

Shravan Venkatraman, Rakesh Raj Madavan, Pavan Kumar Sathya Venkatesh

Mohamed bin Zayed University of AI · University of Amsterdam · University of Massachusetts Amherst

3D Reconstruction Camera Pose Estimation Neural Radiance Fields

Ask PDF

Apr 20, 2026

MU-GeNeRF: Multi-view Uncertainty-guided Generalizable Neural Radiance Fields for Distractor-aware Scene

Generalizable Neural Radiance Fields (GeNeRFs) enable high-quality scene reconstruction from sparse views and can generalize to unseen scenes. However, in real-world settings, transient distractors break cross-view structural consistency, corrupting supervision and degrading reconstruction quality. Existing distractor-free NeRF methods rely on per-scene optimization and estimate uncertainty from per-view reconstruction errors, which are not reliable for GeNeRFs and often misjudge inconsistent static structures as distractors. To this end, we propose MU-GeNeRF, a Multi-view Uncertainty-guided distractor-aware GeNeRF framework designed to alleviate GeNeRF's robust modeling challenges in the presence of transient distractions. We decompose distractor awareness into two complementary uncertainty components: Source-view Uncertainty, which captures structural discrepancies across source views caused by viewpoint changes or dynamic factors; and Target-view Uncertainty, which detects observation anomalies in the target image induced by transient distractors.These two uncertainties address distinct error sources and are combined through a heteroscedastic reconstruction loss, which guides the model to adaptively modulate supervision, enabling more robust distractor suppression and geometric modeling.Extensive experiments show that our method not only surpasses existing GeNeRFs but also achieves performance comparable to scene-specific distractor-free NeRFs.

Wenjie Mu, Zhan Li, Chuanzhou Su +8

School of Computer Science and Technology, Tongji University, Shanghai, China · MOE Key Lab of Embedded System and Service Computing, Tongji University, Shanghai, China · College of Surveying and Geographic Informatics, Tongji University, Shanghai, China +1

3D Reconstruction Neural Rendering Sparse-View 3D Reconstruction Neural Radiance Fields

Ask PDF

Abstract

Uncertainty Quantification

Explore similar work

Evidential Neural Radiance Fields

PCM-NeRF: Probabilistic Camera Modeling for Neural Radiance Fields under Pose Uncertainty

MU-GeNeRF: Multi-view Uncertainty-guided Generalizable Neural Radiance Fields for Distractor-aware Scene