DriftNeRF: Cross-Ray Density Drifting for Sparse-View Neural Radiance Fields

Juntang Wang; Yihan Wang; Sheng'en Li; Hao Wu; Dongmian Zou; Shixin Xu

cs.CVOct 21, 2026

DriftNeRF: Cross-Ray Density Drifting for Sparse-View Neural Radiance Fields

Authors: Juntang Wang, Yihan Wang, Sheng'en Li, Hao Wu, Dongmian Zou, Shixin Xu

Organizations: Harvard University, Cambridge, USA · Duke University, Durham, USA · Duke Kunshan University, Kunshan, China

Abstract

Neural Radiance Fields (NeRF) synthesize photorealistic novel views from dense image collections, but degrade sharply when few views are available. We attribute a specific, measurable defect to the objective itself: the rendering loss is a sum over independent rays and contains no term comparing densities that different rays assign to the same region of space, so nearby 3D points can settle on contradictory densities while every training pixel is reproduced correctly. We make this quantity explicit by measuring cross-ray density disagreement directly during training, and show that it stays high in vanilla NeRF, falls only incidentally under frequency-based regularization, and moves together with held-out quality in the sparse regime. We then close the gap with DriftNeRF, a drop-in regularizer that instantiates the drifting-field template on density samples: a masked exponential kernel aggregates densities from nearby points on other rays into a stop-gradient consensus target, and the resulting drifting field penalizes unsupported disagreement while leaving agreed-upon boundaries intact. The regularizer reads only sampled positions, ray identifiers, and predicted densities, so it requires no pretrained model, external geometry, or additional images, and transfers across field parameterizations. On NeRF-Synthetic it reduces residual disagreement by an order of magnitude, and across NeRF-Synthetic, DTU, and LLFF it improves 3-view PSNR over the strongest MLP-NeRF baseline by 0.42-0.65 dB and remains competitive with sparse-view Gaussian pipelines under a matched protocol without their depth, flow, or covisibility preprocessing.

Figures & tables

Panel of Figure 1 — Figure 1. Novel-view synthesis from 3 training views (DTU Scan 34). NeRF’s per-ray photometric loss leaves nearby densities from different rays free to contradict each other, producing floaters and haze. Sparse-view methods (DNGaussian, FreeNeRF) reduce these artifacts but do not eliminate them. DriftNeRF (ours) enforces local field agreement across rays, suppressing floaters where cross-ray support is absent. A five-column row of renderings of DTU Scan 34 from three training views. Columns left to right are ground truth, DriftNeRF (ours), DNGaussian, FreeNeRF, and NeRF. The rightmost NeRF column shows heavy floaters and haze around the object, the two sparse-view baselines in the middle show fewer, and the DriftNeRF column second from the left is closest to the ground truth.

Panel of Figure 2 — Figure 2. Overview of DriftNeRF. Scene (left): rays from a novel view $a$ and input views sample nearby points ${\bm{x}}_{a,i}$ , ${\bm{x}}_{b,j}$ inside a shared volume. Density field (bottom-left): the anchor density $\sigma_{a,i}$ is compared with cross-ray neighbors $\operatorname{sg}(\sigma_{b,j})$ , whose stop-gradiented values are pooled into a consensus target $\bar{\sigma}_{\mathrm{cross}}$ . Pipeline (right): the MLP $f_{\bm{\theta}}$ produces color and density; the $\sigma$ portal feeds two paths. Volume rendering yields pixel colors $C({\bm{r}})$ compared against ground truth $C^{*}({\bm{r}})$ via $\mathcal{L}_{\mathrm{render}}$ . Cross-ray consensus defines the drifting field $V=\bar{\sigma}_{\mathrm{cross}}-\sigma_{a,i}$ , minimized by $\mathcal{L}_{\mathrm{drift}}$ . The combined $\mathcal{L}_{\mathrm{total}}$ trains $f_{\bm{\theta}}$ end-to-end. A schematic in two parts. On the left, camera rays from several views cross inside a shared volume, and an anchor sample point on one ray is surrounded by nearby sample points belonging to other rays. On the right, a block diagram: the MLP outputs color and density; density feeds both the volume renderer, which produces a pixel color compared against ground truth by the rendering loss, and the cross-ray consensus branch, which produces the drifting field and the drifting loss. The two losses are summed into the total training objective.

Dataset	Sparse split, evaluation, and coordinates
NeRF-Syn. ( $800^{2}$ )	Evenly spaced official training frames; all official test frames. Blender camera-to-world coordinates, white background, bounds $[2,6]$ .
DTU ( $400{\times}300$ )	Every eighth camera is test; sparse train cameras are evenly selected. OpenCV poses are converted to OpenGL; bounds come from MVSNet camera files.
LLFF ( $504{\times}378$ )	Every eighth camera is test; sparse train cameras are evenly selected. Poses are recentered, no NDC transform is used, and bounds come from poses_bounds.npy .

Table 1. Code-aligned dataset protocol. Kernel distances use the loader coordinates without additional scene normalization.

Panel of Figure 3 — Figure 3. Training dynamics on NeRF-Synthetic with 3 views, averaged across eight scenes. Top: the cross-ray drift diagnostic measured for every method. Bottom: held-out-view PSNR. Green (solid) : DriftNeRF explicitly reduces disagreement while reaching the highest final PSNR; the other objectives reduce the same diagnostic only indirectly. Bands show inter-scene variation. Two vertically stacked training curves. DriftNeRF's measured cross-ray loss falls much faster and lower than the baselines, while its held-out PSNR rises to the highest final value.

Panel of Figure 4 — Figure 4. Relative contribution profiles along representative 3-view test rays. Top: Lego. Bottom: DTU Scan 34. The dashed line is the reference surface depth $t^{*}$ . Green (solid) : DriftNeRF concentrates its dominant mass closest to $t^{*}$ , whereas NeRF and the frequency-annealed baseline retain broader shoulders or isolated residual peaks. The displayed peak scale is normalized to 1; the area need not integrate to 1. Two vertically stacked ray profiles. For Lego and DTU Scan 34, DriftNeRF has a narrow peak near the ground-truth surface, while NeRF and frequency annealing show wider or extra peaks.

	DTU			NeRF-Synthetic			LLFF
Method	3	6	9	3	6	9	3	6	9
NeRF	13.42	18.67	22.14	15.83	22.41	26.18	14.52	19.52	22.56
InfoNeRF	15.21	19.84	22.87	17.14	23.42	27.08	15.83	20.41	23.12
DietNeRF	15.89	20.52	23.41	17.62	23.87	27.35	16.28	20.73	23.34
DS-NeRF	16.73	21.38	24.57	18.21	24.56	27.89	17.46	22.14	24.06
RegNeRF	17.24	21.95	24.44	19.47	25.13	28.41	18.63	22.87	24.67

Table 2. Sparse-view reconstruction quality. (a) PSNR across 3, 6, and 9 input views. (b) SSIM/LPIPS in the most data-starved 3-view setting. Ours+Freq adds only the progressive frequency mask implemented in our code, not the complete FreeNeRF objective. Our rows are three-seed means. Bold marks the best value among these rows only; Table 3 adds sparse-view Gaussian baselines, where NexusGS reaches

20.38

dB on LLFF at 3 views under the same protocol.

	DTU			NeRF-Synthetic			LLFF
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
3DGS ( Kerbl et al., 2023 )	12.07	.461	.478	14.97	.589	.461	15.66	.437	.512
FSGS ( Zhu et al., 2024 )	15.83	.703	.308	18.34	.698	.348	20.04	.669	.338
DNGaussian ( Li et al., 2024 )	17.96	.694	.311	20.31	.761	.281	19.32	.615	.374
SparseGS ( Xiong et al., 2025 )	16.94	.667	.341	–	–	–	18.71	.592	.397
DropoutGS ( Xu et al., 2025 )	18.41	.708	.302	20.05	.743	.292	19.26	.627	.366

Table 3. Matched 3-view comparison with sparse-view 3D Gaussian Splatting. (a) Quality under the exact camera splits and evaluator in Table 1 . (b) Same-hardware quality–compute snapshot on DTU, separating one-time preprocessing from optimization. – marks a dataset the official release does not support.

Setting	Method	PSNR $\uparrow$
NeRF-Syn., 3 views	Instant-NGP ( Müller et al., 2022 )	17.42
NeRF-Syn., 3 views	Instant-NGP $+$ drifting loss	17.81
NeRF-Syn., 6 views	ZeroRF ( Shi et al., 2024 )	26.69
NeRF-Syn., 6 views	Ours (MLP backbone)	26.08
DTU, 3 views	ZeroRF ( Shi et al., 2024 )	18.05
DTU, 3 views	Ours (MLP backbone)	18.31

Table 4. Backbone study. Top: the drifting loss added to an Instant-NGP backbone without other changes. Bottom: ZeroRF, a per-scene sparse-view method built on a tensorial backbone, on the two settings it emphasizes. All rows use the protocol in Table 1 .

Panel of Figure 5 — Figure 5. Qualitative comparison under the 3-view protocol. Columns left to right: ground truth, DriftNeRF (ours), DNGaussian, FreeNeRF, NeRF. DriftNeRF produces the cleanest silhouettes and surfaces; NeRF exhibits severe floaters; DNGaussian and FreeNeRF fall in between. Top: Lego (NeRF-Synthetic). Bottom: Scan 24 (DTU). Figure 1 shows DTU Scan 34 under the same protocol, and further scenes are in Appendix F . A two-row by five-column grid of renderings for Lego and DTU Scan 24. Columns are ground truth, DriftNeRF, DNGaussian, FreeNeRF, and NeRF. The NeRF column has the most floaters and haze, DNGaussian and FreeNeRF have fewer, and the DriftNeRF column has the cleanest silhouettes and surfaces.

Variant	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Ours (full)	20.83	.768	.274
$-$ warmup	20.41	.758	.282
$-$ stop-gradient	20.58	.759	.278
$-$ drifting loss (NeRF)	15.83	.612	.421

Table 5. Subtractive ablation on NeRF-Synthetic, 3 views (eight-scene mean). The full system is shaded as the reference row; each row below removes one component from it. The no-stop-gradient control differentiates through both anchor and candidate densities.

Param	Values	PSNR $\uparrow$
$\lambda_{\mathrm{drift}}$	0.01 / 0.05 / 0.1 / 0.5 / 1.0	19.24	19.53	19.71	19.78	19.41
$\tau$	0.01 / 0.05 / 0.1 / 0.2 / 0.5	18.94	19.52	19.71	19.43	18.67
$d_{\max}$	0.1 / 0.2 / 0.5 / 1.0 / 2.0	18.81	19.38	19.71	19.56	19.12
$P$	256 / 512 / 2048 / 4096 / 8192	19.14	19.42	19.71	19.74	19.73
$Q$	64 / 128 / 256 / 512 / 1024	19.31	19.58	19.71	19.69	19.67

Table 6. Hyperparameter sensitivity on Lego (3 views, 50k iterations). Default values are shaded . Extended SSIM/LPIPS and 200k-iteration results are in Appendix C .

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Method	8	21	30	31	34	38	40	41	45	55	63	82	103	110	114	Avg
DNGaussian	19.07	17.56	17.84	19.62	18.34	16.73	17.98	15.81	17.82	20.08	20.81	18.49	16.32	15.67	17.26	17.96
FreeNeRF	18.71	17.24	17.58	19.03	18.42	16.89	18.17	16.23	17.94	19.37	20.12	18.63	16.78	15.94	17.30	17.89
Ours	19.14	17.68	18.03	19.47	18.89	17.21	18.62	16.08	18.37	20.34	21.07	19.06	16.62	16.42	17.65	18.31

Appendix

Table 7. Per-scene PSNR

\uparrow

on DTU under the 3-view protocol. DriftNeRF improves over FreeNeRF on 13 of 15 scans; FreeNeRF retains an edge on Scans 41 and 103, and DNGaussian achieves the top score on Scan 31.

SSIM $\uparrow$
Method	8	21	30	31	34	38	40	41	45	55	63	82	103	110	114	Avg
DNGaussian	.731	.673	.687	.744	.710	.656	.700	.623	.695	.749	.765	.712	.638	.614	.713	.694
FreeNeRF	.724	.667	.681	.738	.703	.652	.694	.631	.689	.742	.758	.706	.645	.617	.688	.689
Ours	.732	.675	.689	.746	.712	.659	.703	.626	.697	.751	.768	.714	.641	.628	.714	.697
LPIPS $\downarrow$
Method	8	21	30	31	34	38	40	41	45	55	63	82	103	110	114	Avg

Appendix

Table 8. Per-scene SSIM

\uparrow

/ LPIPS

\downarrow

on DTU under the 3-view protocol.

Method	Room	Fern	Leaves	Fortress	Orchids	Flower	T-Rex	Horns	Avg
DNGaussian	22.17	18.78	16.89	22.84	16.21	17.94	20.63	19.10	19.32
FreeNeRF	21.83	19.12	17.34	22.56	16.74	18.47	21.08	18.14	19.41
Ours	22.31	19.58	17.82	22.94	16.53	18.24	21.57	19.65	19.83

Appendix

Table 9. Per-scene PSNR

\uparrow

on LLFF under the 3-view protocol. DriftNeRF improves on 6 of 8 scenes; FreeNeRF retains an edge on Orchids and Flower. DNGaussian struggles on thin organic structures (Orchids, Flower).

SSIM $\uparrow$
Method	Room	Fern	Leaves	Fortress	Orchids	Flower	T-Rex	Horns	Avg
DNGaussian	.714	.604	.531	.735	.509	.574	.664	.589	.615
FreeNeRF	.704	.612	.543	.726	.524	.589	.672	.598	.621
Ours	.724	.631	.558	.743	.517	.581	.691	.659	.638
LPIPS $\downarrow$
Method	Room	Fern	Leaves	Fortress	Orchids	Flower	T-Rex	Horns	Avg

Appendix

Table 10. Per-scene SSIM

\uparrow

/ LPIPS

\downarrow

on LLFF under the 3-view protocol.

Method	Chair	Drums	Ficus	Hotdog	Lego	Matrl.	Mic	Ship	Avg
DNGaussian	25.44	17.21	20.01	22.96	22.34	18.38	22.52	13.62	20.31
FreeNeRF	24.31	16.83	20.42	23.17	21.64	18.92	22.08	14.07	20.18
Ours	25.38	17.59	21.24	24.06	22.78	18.76	22.85	13.98	20.83

Appendix

Table 11. Per-scene PSNR

\uparrow

on NeRF-Synthetic under the 3-view protocol. These eight scene means are the input to the paired Wilcoxon test reported in Section 4.3 . DriftNeRF improves over FreeNeRF on 6 of 8 scenes and holds the top score among the three methods on 5 of 8; FreeNeRF retains an edge on Materials and Ship.

SSIM $\uparrow$
Method	Chair	Drums	Ficus	Hotdog	Lego	Matrl.	Mic	Ship	Avg
DNGaussian	.871	.656	.770	.829	.794	.704	.812	.652	.761
FreeNeRF	.842	.641	.774	.828	.771	.718	.796	.678	.756
Ours	.864	.662	.791	.843	.801	.704	.822	.657	.768
LPIPS $\downarrow$
Method	Chair	Drums	Ficus	Hotdog	Lego	Matrl.	Mic	Ship	Avg

Appendix

Table 12. Per-scene SSIM

\uparrow

/ LPIPS

\downarrow

on NeRF-Synthetic under the 3-view protocol.

	DTU			NeRF-Synthetic
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
NeRF	28.43	.917	.189	31.02	.947	.058
Instant-NGP	29.81	.931	.162	33.18	.962	.038
Ours	28.57	.918	.186	31.09	.948	.057

Appendix

Table 13. Dense-view results on DTU and NeRF-Synthetic under the conventional splits.

		SSIM $\uparrow$					LPIPS $\downarrow$
$\lambda_{\mathrm{drift}}$	0.01 / 0.05 / 0.1 / 0.5 / 1.0	.742	.754	.761	.764	.748	.298	.286	.278	.275	.293
$\tau$	0.01 / 0.05 / 0.1 / 0.2 / 0.5	.731	.753	.761	.749	.718	.312	.288	.278	.291	.324
$d_{\max}$	0.1 / 0.2 / 0.5 / 1.0 / 2.0	.724	.746	.761	.756	.738	.318	.294	.278	.284	.302
$P$	256 / 512 / 2048 / 4096 / 8192	.739	.751	.761	.762	.761	.302	.289	.278	.277	.277
$Q$	64 / 128 / 256 / 512 / 1024	.747	.756	.761	.760	.758	.293	.284	.278	.279	.281

Appendix

Table 14. Extended hyperparameter sensitivity on Lego (3 views, 50k iters) with SSIM

\uparrow

and LPIPS

\downarrow

. Default values are shaded .

Param	Values	PSNR $\uparrow$
$\lambda_{\mathrm{drift}}$	0.01 / 0.05 / 0.1 / 0.5 / 1.0	21.72	22.34	22.78	22.91	21.83
$\tau$	0.01 / 0.05 / 0.1 / 0.2 / 0.5	20.86	22.28	22.78	22.17	20.24
$d_{\max}$	0.1 / 0.2 / 0.5 / 1.0 / 2.0	20.63	22.08	22.78	22.39	21.46
$P$	256 / 512 / 2048 / 4096 / 8192	21.62	22.24	22.78	22.83	22.81
$Q$	64 / 128 / 256 / 512 / 1024	22.01	22.46	22.78	22.76	22.73

Appendix

Table 15. Hyperparameter sensitivity on Lego (3 views, 200k iters) with PSNR

\uparrow

. Default values are shaded . Compare with Table 6 (50k).

Shared across datasets
Parameter	NeRF-Syn.	DTU	LLFF
Resolution	$800^{2}$	$400{\times}300$	$504{\times}378$
Near / far	2.0 / 6.0	scan cameras	poses_bounds
Coarse / fine samples	64 / 64
Batch size (rays)	1024
Iterations	200k

Appendix

Table 16. Training configuration per dataset. All experiments use the Adam optimizer with

\beta_{1}=0.9

\beta_{2}=0.999

. No NDC transform is applied on any dataset; kernel distances are evaluated in loader coordinates.

Component	NeRF (ms)	DriftNeRF (ms)
Forward pass (MLP)	42.3	42.3
Volume rendering	8.7	8.7
Rendering loss + backward	78.4	78.4
Drifting loss (forward)	–	5.8
Drifting loss (backward)	–	3.6
Total	129.4	138.8

Appendix

Table 17. Per-iteration timing breakdown (ms/iter, A100 GPU). The drifting loss adds

{\sim}7\%

wall-clock overhead.

Method	Release	Steps	Priors
3DGS	graphdeco/gaussian-splatting	30k	none (random init on NeRF-Syn.)
FSGS	VITA-Group/FSGS	10k	DPT monocular depth
DNGaussian	Fictionarry/DNGaussian	6k	monocular depth
SparseGS	ForMyCat/SparseGS	30k	monocular depth, floater pruning
DropoutGS	official release	30k	none beyond SfM
NexusGS	USMizuki/NexusGS	30k	optical flow, epipolar depth

Appendix

Table 18. Reproduction sheet for the matched 3-view 3DGS comparison. “Priors” lists auxiliary estimators applied to the three training images only.

Panel of Figure 6 — Figure 6. Additional qualitative comparisons on four NeRF-Synthetic scenes under the 3-view protocol. Columns left to right: ground truth, DriftNeRF (ours), DNGaussian, FreeNeRF, NeRF. A four-row by five-column grid of renderings for Ship, Materials, Hotdog, and Drums. Columns are ground truth, DriftNeRF, DNGaussian, FreeNeRF, and NeRF. The NeRF column is the haziest with the most floaters; DriftNeRF is closest to the ground truth on Hotdog and Drums, while Ship and Materials remain difficult for all methods.

Novel View Synthesis

Novel View Synthesis creates images of a scene from camera viewpoints not present in the input, using learned geometry and appearance models.

See all 289 papers

Geometry Regularization for Vanilla NeRF via Self-Supervised Cross-Ray Consistency
Oct 20, 2026Zongji WangNeural Radiance Fields 3D Reconstruction
Low-Cost Neural Radiance Fields
May 10, 2026Alice Huang, Prathamesh Sonawane, Yashdeep Thorat +1Neural Rendering Sparse-View 3D Reconstruction
University of Illinois Urbana Champaign
Remote Sensing Sparse-View 3D Gaussian Splatting via Depth Image-Based Rendering
Sep 28, 2026Jiaming Kang, Zhengxia Zou, Zhenwei Shi3D Gaussian Splatting 3D Reconstruction
Beihang University

Oct 20, 2026J/K move · Enter open · S save

Geometry Regularization for Vanilla NeRF via Self-Supervised Cross-Ray Consistency

Zongji Wang

Neural Radiance Fields 3D Reconstruction Neural Representation Geometry

Open paper AskPDF arXiv

Neural Radiance Fields (NeRF) achieve high-quality novel-view synthesis by representing a scene as a continuous volumetric radiance field. However, the geometry encoded by the density field of vanilla NeRF can be noisy and inaccurate, since different combinations of shape and radiance may explain the same input images. To reduce this ambiguity, we augment vanilla NeRF with a geometry-optimization branch based on self-supervised cross-ray consistency. Specifically, this branch samples candidate surface points from the density field, generates virtual rays that intersect at each point from different directions, and renders their appearance using the NeRF network. Based on the visibility-weighted color consistency across these rays, we constrain the implicit geometry. We further regularize the implicit geometry by penalizing opacity at candidates with inconsistent appearance predictions and encouraging local normal smoothness. These geometric constraints are derived from the NeRF predictions and require no ground-truth depth or surface-normal supervision. Experiments on eight Replica scenes show that the proposed method reduces the mean Chamfer distance by 30.8% compared with vanilla NeRF.

Abstract

Novel View Synthesis

Explore similar work

Geometry Regularization for Vanilla NeRF via Self-Supervised Cross-Ray Consistency

Low-Cost Neural Radiance Fields

Remote Sensing Sparse-View 3D Gaussian Splatting via Depth Image-Based Rendering