3D Gaussian Splatting (3DGS) has recently emerged as a promising approach in novel view synthesis, combining photorealistic rendering with real-time efficiency. However, its success heavily relies on dense camera coverage; under sparse-view conditions, insufficient supervision leads to irregular Gaussian distributions, characterized by globally sparse coverage, blurred background, and distorted high-frequency areas. To address this, we propose HeroGS, Hierarchical Guidance for Robust 3D Gaussian Splatting, a unified framework that establishes hierarchical guidance across the image, feature, and parameter levels. At the image level, sparse supervision is converted into pseudo-dense guidance, globally regularizing the Gaussian distributions and forming a consistent foundation for subsequent optimization. Building upon this, Feature-Adaptive Densification and Pruning (FADP) at the feature level leverages low-level features to refine high-frequency details and adaptively densifies Gaussians in background regions. The optimized distributions then support Co-Pruned Geometry Consistency (CPG) at parameter level, which guides geometric consistency through parameter freezing and co-pruning, effectively removing inconsistent splats. The hierarchical guidance strategy effectively constrains and optimizes the overall Gaussian distributions, thereby enhancing both structural fidelity and rendering quality. Extensive experiments demonstrate that HeroGS achieves high-fidelity reconstructions and consistently surpasses state-of-the-art baselines under sparse-view conditions.
Figures & tables
Figure 1: Motivation of HeroGS. (Left) Image-level pseudo-labels bridge the gap between sparse and dense supervision, yielding a more complete Gaussian distributions. (Right) FADP and CPG refine inaccurate Gaussians by enhancing distributions closer to ground-truth geometry and pruning inconsistencies.
Figure 2: HeroGS Overview. Initialized from SfM, our framework operates across three levels. (1) Image-level Guidance: A set of intermediate RGB frames is synthesized from sparse inputs, offering pseudo-dense guidance that globally regularizes the Gaussian distributions and, at the feature level, manifests as patch-based Gaussian numbers C . (2) Feature-level Refinement: The Feature-Adaptive Densification and Pruning (FADP) leverages edge- and patch-aware features from training views to enhance high-frequency and background regions, while suppressing redundant Gaussians and dilivering finer details for next level. (3) Parameter-level Consistency: The Co-Pruned Geometry Consistency (CPG) employs auxiliary Gaussian fields with partially frozen parameters to perform co-pruning, eliminating geometrically inconsistent splats. These levels form a hierarchical guidance with interconnections (dashed lines) that jointly constrains and optimizes the Gaussian field for improved structural fidelity.
Figure 3: Removing inconsistent Gaussians. Columns 3 and 4: original training views and frame-interpolated pseudo-labels, respectively. Although pseudo-labels provide overall supervisory signals, they may lack accuracy in fine details and fail to offer precise guidance for high-frequency geometric structures. Columns 1 and 2: novel-view renderings produced by full pipeline versus those without CPG. It suppresses drifts, yielding sharper edges, coherent surfaces and markedly improved spatial fidelity.
Methods
2 views
3 views
6 views
PSNR↑
SSIM↑
LPIPS↓
PSNR↑
SSIM↑
LPIPS↓
PSNR↑
SSIM↑
LPIPS↓
Mip-NeRF [ 2 ]
-
-
-
16.11
0.401
0.460
22.91
0.756
0.213
SparseNeRF [ 28 ]
17.80
0.532
0.372
19.86
0.624
0.328
23.26
0.741
0.235
FrugalNeRF [ 17 ]
18.29
0.557
0.342
19.92
0.634
0.297
-
-
-
3DGS [ 13 ]
16.19
0.437
0.388
19.24
0.649
0.237
23.63
0.809
0.129
DRGS [ 7 ]
17.04
0.513
0.318
16.73
0.487
0.310
18.60
0.560
0.239
Table 1: Quantitative results on LLFF with 2, 3, 6 training views. The best and second-best entries are denoted using bold and underline respectively. 3DGS-based methods require multi-view stereo estimation from COLMAP, which fails on 3 scenes for 2 training views. We therefore report the averaged metrics of the remaining scenes.
Methods
2 views
3 views
6 views
PSNR↑
SSIM↑
LPIPS↓
PSNR↑
SSIM↑
LPIPS↓
PSNR↑
SSIM↑
LPIPS↓
3DGS
16.19
0.437
0.388
19.24
0.649
0.237
23.63
0.809
0.129
FSGS
15.65
0.460
0.412
20.43
0.682
0.248
24.15
0.823
0.128
+VFI
16.91
0.489
0.384
20.68
0.704
0.204
24.18
0.822
0.135
+FADP
17.28
0.503
0.370
20.99
0.720
0.192
24.25
0.824
0.136
+GSField (freeze scale )
17.93
0.564
0.345
21.08
0.732
0.191
24.40
0.832
0.136
Table 2: Ablation study on LLFF dataset with 2, 3, 6 training views. The best and second-best entries are denoted using bold and underline respectively. Under the 2-view setting, the baseline method (FSGS [ 36 ] ) performs poorly, while our approach achieves a significant performance improvement.
Methods
3 Views
6 Views
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
3DGS
16.39
0.442
0.432
22.49
0.725
0.279
FSGS
16.88
0.474
0.434
23.63
0.742
0.281
CoR-GS
17.06
0.491
0.443
23.59
0.742
0.275
DropGaussian
16.81
0.480
0.438
24.15
0.772
0.215
HeroGS
17.51
0.512
0.422
24.70
0.781
0.272
Table 3: Quantitative results on Tanks&Temples with 3, 6 training views without downsampling.
Figure 4: Qualitative Comparison on LLFF (3 training views). Under extreme view sparsity, 3DGS [ 13 ] collapses into severe artifacts and blurred geometry. DRGS and FSGS recover coarse structure yet still exhibit over-smoothed textures and noisy backgrounds. In contrast, HeroGS guides the model from complementary levels to deliver markedly sharper object boundaries, richer high-frequency textures, and distinctly clearer distant regions—demonstrating the efficacy of our full pipeline.
Figure 5: Comparison with and without image and feature level guidance. Columns 1 and 2: novel-view renderings produced by baseline versus baseline with pseudo-labels and FADP. Columns 3 and 4: their corresponding Gaussians.
Settings
PSNR
SSIM
LPIPS
w/o Post-Freeze
21.16
0.735
0.190
All
21.30
0.739
0.189
Table 4: Ablation study on Post-Freeze Behavior on LLFF dataset for 3 training views.
Settings
PSNR
SSIM
LPIPS
PerVFI [ 29 ]
20.78
0.711
0.204
BIMVFI
20.99
0.720
0.192
Ground Truth
21.09
0.720
0.195
Ours
21.30
0.739
0.189
Table 5: Comparison between different Interpolation Model on LLFF dataset for 3 training views.
Figure 6: Analysis of training dynamics. Comparison of PSNR (left) and the number of Gaussians (right) between our HeroGS and baseline during training.
PL
FADP
CPG
PSNR ↑
SSIM ↑
LPIPS ↓
One Level
✓
16.91
0.489
0.384
✓
16.64
0.483
0.390
✓
16.78
0.535
0.405
average
16.78
0.502
0.393
Two Levels
✓
✓
17.28
0.503
0.370
✓
✓
18.33
0.577
0.336
Table 6: Ablation study on level interdependencies on LLFF dataset for 2 training views.
Figure 7: Qualitative Comparison on Tanks for 3 training views. In large-scale dataset, 3DGS and DropGaussian struggles to maintain geometry and texture fidelity, exhibiting significant artifacts. FSGS and CoR-GS recover coarse structure yet still exhibit over-smoothed geometry and artifacts. In contrast, HeroGS reconstruct fine structures and high-frequency textures.
Methods
12 views
24 views
PSNR↑
SSIM↑
LPIPS↓
PSNR↑
SSIM↑
LPIPS↓
3DGS
18.44
0.521
0.385
23.22
0.730
0.234
FSGS
18.93
0.539
0.380
23.46
0.738
0.237
CoR-GS
19.59
0.578
0.374
23.39
0.727
0.272
DropGaussian
19.49
0.573
0.366
24.03
0.762
0.225
HeroGS
19.99
0.591
0.373
24.18
0.766
0.229
Table 7: Quantitative results on Mip-NeRF360 [ 3 ] with 12, 24 training views.
Figure 8: Ablation study on LLFF dataset showing the impact of varying interpolation factor S on PSNR, SSIM, and LPIPS.
Settings
PSNR
SSIM
LPIPS
w/o Selection
21.19
0.736
0.190
All
21.30
0.739
0.189
Table 8: Ablation study on Selection module on LLFF dataset for 3 training views.
Figure 9: Qualitative comparison of depth rendering quality. We visualize the depth maps rendered from the reconstructed 3D Gaussian fields of different methods and our proposed HeroGS framework. The visualizations show that our method produces more consistent and artifact-free depth results with finer geometry structure preservation.