Geometry Regularization for Vanilla NeRF via Self-Supervised Cross-Ray Consistency

Zongji Wang

cs.CVOct 20, 2026

Geometry Regularization for Vanilla NeRF via Self-Supervised Cross-Ray Consistency

Authors: Zongji Wang

Abstract

Neural Radiance Fields (NeRF) achieve high-quality novel-view synthesis by representing a scene as a continuous volumetric radiance field. However, the geometry encoded by the density field of vanilla NeRF can be noisy and inaccurate, since different combinations of shape and radiance may explain the same input images. To reduce this ambiguity, we augment vanilla NeRF with a geometry-optimization branch based on self-supervised cross-ray consistency. Specifically, this branch samples candidate surface points from the density field, generates virtual rays that intersect at each point from different directions, and renders their appearance using the NeRF network. Based on the visibility-weighted color consistency across these rays, we constrain the implicit geometry. We further regularize the implicit geometry by penalizing opacity at candidates with inconsistent appearance predictions and encouraging local normal smoothness. These geometric constraints are derived from the NeRF predictions and require no ground-truth depth or surface-normal supervision. Experiments on eight Replica scenes show that the proposed method reduces the mean Chamfer distance by 30.8% compared with vanilla NeRF.

Figures & tables

Figure 1. Schematic illustration of geometry–appearance ambiguity. (a) Two captured rays observe the same physical surface point

P

on a coherent 3D object. (b) An inflated, distorted surface intercepts the same rays at different points

Q_{1}

and

Q_{2}

. Based on view-dependent radiance, the incorrect geometry can nevertheless reproduce similar rendered RGB values along each captured ray, as indicated by the paired color swatches. Thus, image agreement alone does not uniquely determine the geometry. The dashed contour denotes the reference geometry, and the dashed ray segments extend toward

P

behind the incorrect surface. An original three-dimensional schematic showing two vase-shaped surface hypotheses under the same two camera viewpoints. The correct surface is smooth, and both rays meet at a shared point P. The incorrect surface is inflated and distorted, with a visible triangular mesh; the rays intersect it at distinct points Q1 and Q2. Paired color swatches indicate similar rendered RGB values for corresponding rays despite the geometry difference. A dashed reference contour and dashed ray extensions show the underlying reference shape and point.

Figure 2. Overview of the proposed framework. The shared NeRF renders training views for RGB reconstruction and supplies the density field for candidate surface sampling. The geometry-optimization branch generates virtual cross-rays through each candidate, renders their appearances, and estimates visibility weights to reduce the influence of occluded rays. Cross-ray consistency, occupancy suppression, and local normal smoothness jointly regularize the field. A shared NeRF with an RGB reconstruction path and a geometry-optimization branch. The branch samples candidate points, generates rays that intersect at a common candidate, renders their colors with the shared network, and applies visibility-aware consistency, occupancy suppression, and normal smoothness. Both paths update the same NeRF.

Figure 3. Details of candidate sampling and visibility-aware cross-ray construction. (a) Uniform proposals and rendered-depth anchors are filtered by an intermediate opacity band to obtain candidate points. The curved patch schematically represents the current density boundary. (b) For a candidate

\mathbf{x}_{i}

, the auxiliary sphere is centered at

\mathbf{x}_{i}+l_{0}\mathbf{n}_{i}

with radius

r

. Its sampled locations

\widetilde{\mathbf{o}}_{ij}

determine ray directions, while the actual origins

\mathbf{o}_{ij}

are placed independently at distances

l_{ij}

. All generated rays pass through the same candidate. (c) Opacity samples along each origin-to-candidate segment determine a continuous visibility weight: an intervening occupied region increases the opacity sum and reduces the ray’s influence. This segment is used for visibility estimation; appearance is obtained by standard volume rendering along the generated ray. Three original schematic panels. Candidate sampling combines orange uniform samples near a curved density boundary with blue rendered-depth anchors; gray proposals outside the opacity band are discarded. Cross-ray generation uses a sphere shifted along a green outward normal to determine directions; blue camera symbols show actual origins at independently sampled distances. Visibility estimation compares two rays through the same candidate, with sample points on their paths. An intervening occupied region produces a smaller soft weight, while the clearer path receives a larger weight.

Scene	Baseline (Vanilla NeRF)	Ours	Relative change
Room 0	0.49	0.29	$\boldsymbol{\downarrow}$ 40.0%
Room 1	0.54	0.65	$\uparrow$ 21.2%
Room 2	0.67	0.22	$\boldsymbol{\downarrow}$ 67.4%
Office 0	0.41	0.48	$\uparrow$ 16.2%
Office 1	2.78	2.43	$\boldsymbol{\downarrow}$ 12.5%
Office 2	1.33	0.48	$\boldsymbol{\downarrow}$ 64.2%

Table 1. Chamfer distance on Replica scenes after 300,000 cumulative training iterations for both methods, multiplied by 100 and rounded to two decimal places (lower is better). Relative changes are computed from the unrounded distances with respect to vanilla NeRF; downward arrows indicate reductions. The lower distance in each scene and the entire mean row are highlighted in bold.

Figure 4. Qualitative geometry comparison on Replica scenes. From left to right, each row shows the ground-truth geometry, vanilla NeRF, and the reconstruction after optimization with the proposed geometry-refinement objective. Geometry reconstructions for five office scenes and three room scenes, comparing ground truth, vanilla NeRF, and refinement with the proposed objective.

Neural Radiance Fields

Neural radiance fields represent 3D scenes as continuous functions of position and viewing direction to synthesize images from new camera views.

See all 82 papers

DriftNeRF: Cross-Ray Density Drifting for Sparse-View Neural Radiance Fields
Oct 21, 2026Juntang Wang, Yihan Wang, Sheng'en Li +3Novel View Synthesis Neural Radiance Fields
Harvard University, Cambridge, USA · Duke University, Durham, USA · Duke Kunshan University, Kunshan, China
NaCR: Visual Localization via NeRF-aided Camera Ray Regression
Sep 22, 2026Yesheng Zhang, Xiang Dai, Xu Zhao +1Camera Pose Estimation
Shanghai Jiao Tong University
MU-GeNeRF: Multi-view Uncertainty-guided Generalizable Neural Radiance Fields for Distractor-aware Scene
Apr 20, 2026Wenjie Mu, Zhan Li, Chuanzhou Su +83D Reconstruction Neural Rendering
School of Computer Science and Technology, Tongji University, Shanghai, China · MOE Key Lab of Embedded System and Service Computing, Tongji University, Shanghai, China · College of Surveying and Geographic Informatics, Tongji University, Shanghai, China +1

Oct 21, 2026J/K move · Enter open · S save

DriftNeRF: Cross-Ray Density Drifting for Sparse-View Neural Radiance Fields

Juntang Wang, Yihan Wang, Sheng'en Li, Hao Wu+2

Harvard University, Cambridge, USA · Duke University, Durham, USA · Duke Kunshan University, Kunshan, China

Novel View Synthesis Neural Radiance Fields Sparse-View 3D Reconstruction

Open paper AskPDF arXiv

Neural Radiance Fields (NeRF) synthesize photorealistic novel views from dense image collections, but degrade sharply when few views are available. We attribute a specific, measurable defect to the objective itself: the rendering loss is a sum over independent rays and contains no term comparing densities that different rays assign to the same region of space, so nearby 3D points can settle on contradictory densities while every training pixel is reproduced correctly. We make this quantity explicit by measuring cross-ray density disagreement directly during training, and show that it stays high in vanilla NeRF, falls only incidentally under frequency-based regularization, and moves together with held-out quality in the sparse regime. We then close the gap with DriftNeRF, a drop-in regularizer that instantiates the drifting-field template on density samples: a masked exponential kernel aggregates densities from nearby points on other rays into a stop-gradient consensus target, and the resulting drifting field penalizes unsupported disagreement while leaving agreed-upon boundaries intact. The regularizer reads only sampled positions, ray identifiers, and predicted densities, so it requires no pretrained model, external geometry, or additional images, and transfers across field parameterizations. On NeRF-Synthetic it reduces residual disagreement by an order of magnitude, and across NeRF-Synthetic, DTU, and LLFF it improves 3-view PSNR over the strongest MLP-NeRF baseline by 0.42-0.65 dB and remains competitive with sparse-view Gaussian pipelines under a matched protocol without their depth, flow, or covisibility preprocessing.

Abstract

Neural Radiance Fields

Explore similar work

DriftNeRF: Cross-Ray Density Drifting for Sparse-View Neural Radiance Fields

NaCR: Visual Localization via NeRF-aided Camera Ray Regression

MU-GeNeRF: Multi-view Uncertainty-guided Generalizable Neural Radiance Fields for Distractor-aware Scene