Neural Radiance Fields (NeRF) synthesize photorealistic novel views from dense image collections, but degrade sharply when few views are available. We attribute a specific, measurable defect to the objective itself: the rendering loss is a sum over independent rays and contains no term comparing densities that different rays assign to the same region of space, so nearby 3D points can settle on contradictory densities while every training pixel is reproduced correctly. We make this quantity explicit by measuring cross-ray density disagreement directly during training, and show that it stays high in vanilla NeRF, falls only incidentally under frequency-based regularization, and moves together with held-out quality in the sparse regime. We then close the gap with DriftNeRF, a drop-in regularizer that instantiates the drifting-field template on density samples: a masked exponential kernel aggregates densities from nearby points on other rays into a stop-gradient consensus target, and the resulting drifting field penalizes unsupported disagreement while leaving agreed-upon boundaries intact. The regularizer reads only sampled positions, ray identifiers, and predicted densities, so it requires no pretrained model, external geometry, or additional images, and transfers across field parameterizations. On NeRF-Synthetic it reduces residual disagreement by an order of magnitude, and across NeRF-Synthetic, DTU, and LLFF it improves 3-view PSNR over the strongest MLP-NeRF baseline by 0.42-0.65 dB and remains competitive with sparse-view Gaussian pipelines under a matched protocol without their depth, flow, or covisibility preprocessing.
Figures & tables
Figure 1. Novel-view synthesis from 3 training views (DTU Scan 34). NeRF’s per-ray photometric loss leaves nearby densities from different rays free to contradict each other, producing floaters and haze. Sparse-view methods (DNGaussian, FreeNeRF) reduce these artifacts but do not eliminate them. DriftNeRF (ours) enforces local field agreement across rays, suppressing floaters where cross-ray support is absent. A five-column row of renderings of DTU Scan 34 from three training views. Columns left to right are ground truth, DriftNeRF (ours), DNGaussian, FreeNeRF, and NeRF. The rightmost NeRF column shows heavy floaters and haze around the object, the two sparse-view baselines in the middle show fewer, and the DriftNeRF column second from the left is closest to the ground truth.
Figure 2. Overview of DriftNeRF. Scene (left): rays from a novel view a and input views sample nearby points xa,i , xb,j inside a shared volume. Density field (bottom-left): the anchor density σa,i is compared with cross-ray neighbors sg(σb,j) , whose stop-gradiented values are pooled into a consensus target σˉcross . Pipeline (right): the MLP fθ produces color and density; the σ portal feeds two paths. Volume rendering yields pixel colors C(r) compared against ground truth C∗(r) via Lrender . Cross-ray consensus defines the drifting field V=σˉcross−σa,i , minimized by Ldrift . The combined Ltotal trains fθ end-to-end. A schematic in two parts. On the left, camera rays from several views cross inside a shared volume, and an anchor sample point on one ray is surrounded by nearby sample points belonging to other rays. On the right, a block diagram: the MLP outputs color and density; density feeds both the volume renderer, which produces a pixel color compared against ground truth by the rendering loss, and the cross-ray consensus branch, which produces the drifting field and the drifting loss. The two losses are summed into the total training objective.
Dataset
Sparse split, evaluation, and coordinates
NeRF-Syn. ( 8002 )
Evenly spaced official training frames; all official test frames. Blender camera-to-world coordinates, white background, bounds [2,6] .
DTU ( 400×300 )
Every eighth camera is test; sparse train cameras are evenly selected. OpenCV poses are converted to OpenGL; bounds come from MVSNet camera files.
LLFF ( 504×378 )
Every eighth camera is test; sparse train cameras are evenly selected. Poses are recentered, no NDC transform is used, and bounds come from poses_bounds.npy .
Table 1. Code-aligned dataset protocol. Kernel distances use the loader coordinates without additional scene normalization.
Figure 3. Training dynamics on NeRF-Synthetic with 3 views, averaged across eight scenes. Top: the cross-ray drift diagnostic measured for every method. Bottom: held-out-view PSNR. Green (solid) : DriftNeRF explicitly reduces disagreement while reaching the highest final PSNR; the other objectives reduce the same diagnostic only indirectly. Bands show inter-scene variation. Two vertically stacked training curves. DriftNeRF's measured cross-ray loss falls much faster and lower than the baselines, while its held-out PSNR rises to the highest final value.
Figure 4. Relative contribution profiles along representative 3-view test rays. Top: Lego. Bottom: DTU Scan 34. The dashed line is the reference surface depth t∗ . Green (solid) : DriftNeRF concentrates its dominant mass closest to t∗ , whereas NeRF and the frequency-annealed baseline retain broader shoulders or isolated residual peaks. The displayed peak scale is normalized to 1; the area need not integrate to 1. Two vertically stacked ray profiles. For Lego and DTU Scan 34, DriftNeRF has a narrow peak near the ground-truth surface, while NeRF and frequency annealing show wider or extra peaks.
DTU
NeRF-Synthetic
LLFF
Method
3
6
9
3
6
9
3
6
9
NeRF
13.42
18.67
22.14
15.83
22.41
26.18
14.52
19.52
22.56
InfoNeRF
15.21
19.84
22.87
17.14
23.42
27.08
15.83
20.41
23.12
DietNeRF
15.89
20.52
23.41
17.62
23.87
27.35
16.28
20.73
23.34
DS-NeRF
16.73
21.38
24.57
18.21
24.56
27.89
17.46
22.14
24.06
RegNeRF
17.24
21.95
24.44
19.47
25.13
28.41
18.63
22.87
24.67
Table 2. Sparse-view reconstruction quality. (a) PSNR across 3, 6, and 9 input views. (b) SSIM/LPIPS in the most data-starved 3-view setting. Ours+Freq adds only the progressive frequency mask implemented in our code, not the complete FreeNeRF objective. Our rows are three-seed means. Bold marks the best value among these rows only; Table 3 adds sparse-view Gaussian baselines, where NexusGS reaches 20.38 dB on LLFF at 3 views under the same protocol.
DTU
NeRF-Synthetic
LLFF
Method
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
3DGS ( Kerbl et al., 2023 )
12.07
.461
.478
14.97
.589
.461
15.66
.437
.512
FSGS ( Zhu et al., 2024 )
15.83
.703
.308
18.34
.698
.348
20.04
.669
.338
DNGaussian ( Li et al., 2024 )
17.96
.694
.311
20.31
.761
.281
19.32
.615
.374
SparseGS ( Xiong et al., 2025 )
16.94
.667
.341
–
–
–
18.71
.592
.397
DropoutGS ( Xu et al., 2025 )
18.41
.708
.302
20.05
.743
.292
19.26
.627
.366
Table 3. Matched 3-view comparison with sparse-view 3D Gaussian Splatting. (a) Quality under the exact camera splits and evaluator in Table 1 . (b) Same-hardware quality–compute snapshot on DTU, separating one-time preprocessing from optimization. – marks a dataset the official release does not support.
Setting
Method
PSNR ↑
NeRF-Syn., 3 views
Instant-NGP ( Müller et al., 2022 )
17.42
NeRF-Syn., 3 views
Instant-NGP + drifting loss
17.81
NeRF-Syn., 6 views
ZeroRF ( Shi et al., 2024 )
26.69
NeRF-Syn., 6 views
Ours (MLP backbone)
26.08
DTU, 3 views
ZeroRF ( Shi et al., 2024 )
18.05
DTU, 3 views
Ours (MLP backbone)
18.31
Table 4. Backbone study. Top: the drifting loss added to an Instant-NGP backbone without other changes. Bottom: ZeroRF, a per-scene sparse-view method built on a tensorial backbone, on the two settings it emphasizes. All rows use the protocol in Table 1 .
Figure 5. Qualitative comparison under the 3-view protocol. Columns left to right: ground truth, DriftNeRF (ours), DNGaussian, FreeNeRF, NeRF. DriftNeRF produces the cleanest silhouettes and surfaces; NeRF exhibits severe floaters; DNGaussian and FreeNeRF fall in between. Top: Lego (NeRF-Synthetic). Bottom: Scan 24 (DTU). Figure 1 shows DTU Scan 34 under the same protocol, and further scenes are in Appendix F . A two-row by five-column grid of renderings for Lego and DTU Scan 24. Columns are ground truth, DriftNeRF, DNGaussian, FreeNeRF, and NeRF. The NeRF column has the most floaters and haze, DNGaussian and FreeNeRF have fewer, and the DriftNeRF column has the cleanest silhouettes and surfaces.
Variant
PSNR ↑
SSIM ↑
LPIPS ↓
Ours (full)
20.83
.768
.274
− warmup
20.41
.758
.282
− stop-gradient
20.58
.759
.278
− drifting loss (NeRF)
15.83
.612
.421
Table 5. Subtractive ablation on NeRF-Synthetic, 3 views (eight-scene mean). The full system is shaded as the reference row; each row below removes one component from it. The no-stop-gradient control differentiates through both anchor and candidate densities.
Param
Values
PSNR ↑
λdrift
0.01 / 0.05 / 0.1 / 0.5 / 1.0
19.24
19.53
19.71
19.78
19.41
τ
0.01 / 0.05 / 0.1 / 0.2 / 0.5
18.94
19.52
19.71
19.43
18.67
dmax
0.1 / 0.2 / 0.5 / 1.0 / 2.0
18.81
19.38
19.71
19.56
19.12
P
256 / 512 / 2048 / 4096 / 8192
19.14
19.42
19.71
19.74
19.73
Q
64 / 128 / 256 / 512 / 1024
19.31
19.58
19.71
19.69
19.67
Table 6. Hyperparameter sensitivity on Lego (3 views, 50k iterations). Default values are shaded . Extended SSIM/LPIPS and 200k-iteration results are in Appendix C .
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
Method
8
21
30
31
34
38
40
41
45
55
63
82
103
110
114
Avg
DNGaussian
19.07
17.56
17.84
19.62
18.34
16.73
17.98
15.81
17.82
20.08
20.81
18.49
16.32
15.67
17.26
17.96
FreeNeRF
18.71
17.24
17.58
19.03
18.42
16.89
18.17
16.23
17.94
19.37
20.12
18.63
16.78
15.94
17.30
17.89
Ours
19.14
17.68
18.03
19.47
18.89
17.21
18.62
16.08
18.37
20.34
21.07
19.06
16.62
16.42
17.65
18.31
Appendix
Table 7. Per-scene PSNR ↑ on DTU under the 3-view protocol. DriftNeRF improves over FreeNeRF on 13 of 15 scans; FreeNeRF retains an edge on Scans 41 and 103, and DNGaussian achieves the top score on Scan 31.
SSIM ↑
Method
8
21
30
31
34
38
40
41
45
55
63
82
103
110
114
Avg
DNGaussian
.731
.673
.687
.744
.710
.656
.700
.623
.695
.749
.765
.712
.638
.614
.713
.694
FreeNeRF
.724
.667
.681
.738
.703
.652
.694
.631
.689
.742
.758
.706
.645
.617
.688
.689
Ours
.732
.675
.689
.746
.712
.659
.703
.626
.697
.751
.768
.714
.641
.628
.714
.697
LPIPS ↓
Method
8
21
30
31
34
38
40
41
45
55
63
82
103
110
114
Avg
Appendix
Table 8. Per-scene SSIM ↑ / LPIPS ↓ on DTU under the 3-view protocol.
Method
Room
Fern
Leaves
Fortress
Orchids
Flower
T-Rex
Horns
Avg
DNGaussian
22.17
18.78
16.89
22.84
16.21
17.94
20.63
19.10
19.32
FreeNeRF
21.83
19.12
17.34
22.56
16.74
18.47
21.08
18.14
19.41
Ours
22.31
19.58
17.82
22.94
16.53
18.24
21.57
19.65
19.83
Appendix
Table 9. Per-scene PSNR ↑ on LLFF under the 3-view protocol. DriftNeRF improves on 6 of 8 scenes; FreeNeRF retains an edge on Orchids and Flower. DNGaussian struggles on thin organic structures (Orchids, Flower).
SSIM ↑
Method
Room
Fern
Leaves
Fortress
Orchids
Flower
T-Rex
Horns
Avg
DNGaussian
.714
.604
.531
.735
.509
.574
.664
.589
.615
FreeNeRF
.704
.612
.543
.726
.524
.589
.672
.598
.621
Ours
.724
.631
.558
.743
.517
.581
.691
.659
.638
LPIPS ↓
Method
Room
Fern
Leaves
Fortress
Orchids
Flower
T-Rex
Horns
Avg
Appendix
Table 10. Per-scene SSIM ↑ / LPIPS ↓ on LLFF under the 3-view protocol.
Method
Chair
Drums
Ficus
Hotdog
Lego
Matrl.
Mic
Ship
Avg
DNGaussian
25.44
17.21
20.01
22.96
22.34
18.38
22.52
13.62
20.31
FreeNeRF
24.31
16.83
20.42
23.17
21.64
18.92
22.08
14.07
20.18
Ours
25.38
17.59
21.24
24.06
22.78
18.76
22.85
13.98
20.83
Appendix
Table 11. Per-scene PSNR ↑ on NeRF-Synthetic under the 3-view protocol. These eight scene means are the input to the paired Wilcoxon test reported in Section 4.3 . DriftNeRF improves over FreeNeRF on 6 of 8 scenes and holds the top score among the three methods on 5 of 8; FreeNeRF retains an edge on Materials and Ship.
SSIM ↑
Method
Chair
Drums
Ficus
Hotdog
Lego
Matrl.
Mic
Ship
Avg
DNGaussian
.871
.656
.770
.829
.794
.704
.812
.652
.761
FreeNeRF
.842
.641
.774
.828
.771
.718
.796
.678
.756
Ours
.864
.662
.791
.843
.801
.704
.822
.657
.768
LPIPS ↓
Method
Chair
Drums
Ficus
Hotdog
Lego
Matrl.
Mic
Ship
Avg
Appendix
Table 12. Per-scene SSIM ↑ / LPIPS ↓ on NeRF-Synthetic under the 3-view protocol.
DTU
NeRF-Synthetic
Method
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
NeRF
28.43
.917
.189
31.02
.947
.058
Instant-NGP
29.81
.931
.162
33.18
.962
.038
Ours
28.57
.918
.186
31.09
.948
.057
Appendix
Table 13. Dense-view results on DTU and NeRF-Synthetic under the conventional splits.
SSIM ↑
LPIPS ↓
λdrift
0.01 / 0.05 / 0.1 / 0.5 / 1.0
.742
.754
.761
.764
.748
.298
.286
.278
.275
.293
τ
0.01 / 0.05 / 0.1 / 0.2 / 0.5
.731
.753
.761
.749
.718
.312
.288
.278
.291
.324
dmax
0.1 / 0.2 / 0.5 / 1.0 / 2.0
.724
.746
.761
.756
.738
.318
.294
.278
.284
.302
P
256 / 512 / 2048 / 4096 / 8192
.739
.751
.761
.762
.761
.302
.289
.278
.277
.277
Q
64 / 128 / 256 / 512 / 1024
.747
.756
.761
.760
.758
.293
.284
.278
.279
.281
Appendix
Table 14. Extended hyperparameter sensitivity on Lego (3 views, 50k iters) with SSIM ↑ and LPIPS ↓ . Default values are shaded .
Param
Values
PSNR ↑
λdrift
0.01 / 0.05 / 0.1 / 0.5 / 1.0
21.72
22.34
22.78
22.91
21.83
τ
0.01 / 0.05 / 0.1 / 0.2 / 0.5
20.86
22.28
22.78
22.17
20.24
dmax
0.1 / 0.2 / 0.5 / 1.0 / 2.0
20.63
22.08
22.78
22.39
21.46
P
256 / 512 / 2048 / 4096 / 8192
21.62
22.24
22.78
22.83
22.81
Q
64 / 128 / 256 / 512 / 1024
22.01
22.46
22.78
22.76
22.73
Appendix
Table 15. Hyperparameter sensitivity on Lego (3 views, 200k iters) with PSNR ↑ . Default values are shaded . Compare with Table 6 (50k).
Parameter
NeRF-Syn.
DTU
LLFF
Resolution
8002
400×300
504×378
Near / far
2.0 / 6.0
scan cameras
poses_bounds
Shared across datasets
Coarse / fine samples
64 / 64
Batch size (rays)
1024
Iterations
200k
Appendix
Table 16. Training configuration per dataset. All experiments use the Adam optimizer with β1=0.9 , β2=0.999 . No NDC transform is applied on any dataset; kernel distances are evaluated in loader coordinates.
Component
NeRF (ms)
DriftNeRF (ms)
Forward pass (MLP)
42.3
42.3
Volume rendering
8.7
8.7
Rendering loss + backward
78.4
78.4
Drifting loss (forward)
–
5.8
Drifting loss (backward)
–
3.6
Total
129.4
138.8
Appendix
Table 17. Per-iteration timing breakdown (ms/iter, A100 GPU). The drifting loss adds ∼7% wall-clock overhead.
Method
Release
Steps
Priors
3DGS
graphdeco/gaussian-splatting
30k
none (random init on NeRF-Syn.)
FSGS
VITA-Group/FSGS
10k
DPT monocular depth
DNGaussian
Fictionarry/DNGaussian
6k
monocular depth
SparseGS
ForMyCat/SparseGS
30k
monocular depth, floater pruning
DropoutGS
official release
30k
none beyond SfM
NexusGS
USMizuki/NexusGS
30k
optical flow, epipolar depth
Appendix
Table 18. Reproduction sheet for the matched 3-view 3DGS comparison. “Priors” lists auxiliary estimators applied to the three training images only.
Figure 6. Additional qualitative comparisons on four NeRF-Synthetic scenes under the 3-view protocol. Columns left to right: ground truth, DriftNeRF (ours), DNGaussian, FreeNeRF, NeRF. A four-row by five-column grid of renderings for Ship, Materials, Hotdog, and Drums. Columns are ground truth, DriftNeRF, DNGaussian, FreeNeRF, and NeRF. The NeRF column is the haziest with the most floaters; DriftNeRF is closest to the ground truth on Hotdog and Drums, while Ship and Materials remain difficult for all methods.
Neural Radiance Fields (NeRF) achieve high-quality novel-view synthesis by representing a scene as a continuous volumetric radiance field. However, the geometry encoded by the density field of vanilla NeRF can be noisy and inaccurate, since different combinations of shape and radiance may explain the same input images. To reduce this ambiguity, we augment vanilla NeRF with a geometry-optimization branch based on self-supervised cross-ray consistency. Specifically, this branch samples candidate surface points from the density field, generates virtual rays that intersect at each point from different directions, and renders their appearance using the NeRF network. Based on the visibility-weighted color consistency across these rays, we constrain the implicit geometry. We further regularize the implicit geometry by penalizing opacity at candidates with inconsistent appearance predictions and encouraging local normal smoothness. These geometric constraints are derived from the NeRF predictions and require no ground-truth depth or surface-normal supervision. Experiments on eight Replica scenes show that the proposed method reduces the mean Chamfer distance by 30.8% compared with vanilla NeRF.
Neural Radiance Fields (NeRF) achieve high-quality novel-view synthesis, but their long training times and reliance on dense input views limit accessibility. We present a comparative study of three accelerated NeRF variants - DS-NeRF, TensoRF, and HashNeRF and explore extensions targeted at the low-compute, low-data regime. First, we add a depth-supervision loss derived from COLMAP keypoints to TensoRF (TensoRF-DS) and evaluate it on the LLFF dataset under reduced view counts. Second, we ablate the feature-decoding MLP of TensoRF and study the effect of input downsampling on PSNR and runtime on the synthetic Lego scene. Third, we propose four architectural variants of the HashNeRF color and density networks, including residual and convolutional designs, and report PSNR/training-time tradeoffs under matched iteration budgets. Under iso-time evaluation, none of our extensions conclusively outperform the published baselines, but the experiments characterize which extensions transfer to constrained settings and surface design questions for future work.
Alice Huang, Prathamesh Sonawane, Yashdeep Thorat +1
Remote sensing novel view synthesis under sparse observations remains challenging due to insufficient geometric constraints and limited cross-view supervision. Existing Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) methods are prone to overfitting and face challenges of depth ambiguities, missing cross-view information, and insufficient constraints in under-observed regions. To address these challenges, we propose DIBR-GS, a neural Gaussian Splatting framework that exploits Depth Image-Based Rendering (DIBR) to generate pseudo views for cross-view consistency supervision. Specifically, reliable geometric initialization is constructed by aligning monocular depth priors with sparse SfM reconstruction, and cross-view appearance priors are incorporated into neural Gaussian representations to enhance appearance modeling under sparse observations. Furthermore, we introduce a progressive DIBR-based pseudo-view supervision strategy to provide additional geometric and appearance constraints, enabling more complete reconstruction of weakly observed regions. In addition, a height-constrained anchor growth strategy is designed to suppress unreasonable Gaussian expansion. Experiments demonstrate that the proposed method achieves superior performance over existing approaches when training with only 3 input views. Compared with the previous best-performing method, it improves PSNR by 6.83 dB, with relative gains of 14% in SSIM and 60% in LPIPS, while maintaining competitive computational efficiency. Our code is available at https://github.com/kanehub/DIBR-GS