HeroGS: Hierarchical Guidance for Robust 3D Gaussian Splatting under Sparse Views

Jiashu Li; Xumeng Han; Zhaoyang Wei; Zipeng Wang; Kuiran Wang; Guorong Li; Zhenjun Han; Jianbin Jiao

cs.CVMar 1, 2026

HeroGS: Hierarchical Guidance for Robust 3D Gaussian Splatting under Sparse Views

Authors: Jiashu Li, Xumeng Han, Zhaoyang Wei, Zipeng Wang, Kuiran Wang, Guorong Li, Zhenjun Han, Jianbin Jiao

Organizations: University of the Chinese Academy of Sciences

Abstract

3D Gaussian Splatting (3DGS) has recently emerged as a promising approach in novel view synthesis, combining photorealistic rendering with real-time efficiency. However, its success heavily relies on dense camera coverage; under sparse-view conditions, insufficient supervision leads to irregular Gaussian distributions, characterized by globally sparse coverage, blurred background, and distorted high-frequency areas. To address this, we propose HeroGS, Hierarchical Guidance for Robust 3D Gaussian Splatting, a unified framework that establishes hierarchical guidance across the image, feature, and parameter levels. At the image level, sparse supervision is converted into pseudo-dense guidance, globally regularizing the Gaussian distributions and forming a consistent foundation for subsequent optimization. Building upon this, Feature-Adaptive Densification and Pruning (FADP) at the feature level leverages low-level features to refine high-frequency details and adaptively densifies Gaussians in background regions. The optimized distributions then support Co-Pruned Geometry Consistency (CPG) at parameter level, which guides geometric consistency through parameter freezing and co-pruning, effectively removing inconsistent splats. The hierarchical guidance strategy effectively constrains and optimizes the overall Gaussian distributions, thereby enhancing both structural fidelity and rendering quality. Extensive experiments demonstrate that HeroGS achieves high-fidelity reconstructions and consistently surpasses state-of-the-art baselines under sparse-view conditions.

Figures & tables

Figure 1: Motivation of HeroGS. (Left) Image-level pseudo-labels bridge the gap between sparse and dense supervision, yielding a more complete Gaussian distributions. (Right) FADP and CPG refine inaccurate Gaussians by enhancing distributions closer to ground-truth geometry and pruning inconsistencies.

Figure 2: HeroGS Overview. Initialized from SfM, our framework operates across three levels. (1) Image-level Guidance: A set of intermediate RGB frames is synthesized from sparse inputs, offering pseudo-dense guidance that globally regularizes the Gaussian distributions and, at the feature level, manifests as patch-based Gaussian numbers $\mathcal{C}$ . (2) Feature-level Refinement: The Feature-Adaptive Densification and Pruning (FADP) leverages edge- and patch-aware features from training views to enhance high-frequency and background regions, while suppressing redundant Gaussians and dilivering finer details for next level. (3) Parameter-level Consistency: The Co-Pruned Geometry Consistency (CPG) employs auxiliary Gaussian fields with partially frozen parameters to perform co-pruning, eliminating geometrically inconsistent splats. These levels form a hierarchical guidance with interconnections (dashed lines) that jointly constrains and optimizes the Gaussian field for improved structural fidelity.

Figure 3: Removing inconsistent Gaussians. Columns 3 and 4: original training views and frame-interpolated pseudo-labels, respectively. Although pseudo-labels provide overall supervisory signals, they may lack accuracy in fine details and fail to offer precise guidance for high-frequency geometric structures. Columns 1 and 2: novel-view renderings produced by full pipeline versus those without CPG. It suppresses drifts, yielding sharper edges, coherent surfaces and markedly improved spatial fidelity.

Methods	2 views			3 views			6 views
Methods	PSNR↑	SSIM↑	LPIPS↓	PSNR↑	SSIM↑	LPIPS↓	PSNR↑	SSIM↑	LPIPS↓
Mip-NeRF [ 2 ]	-	-	-	16.11	0.401	0.460	22.91	0.756	0.213
SparseNeRF [ 28 ]	17.80	0.532	0.372	19.86	0.624	0.328	23.26	0.741	0.235
FrugalNeRF [ 17 ]	18.29	0.557	0.342	19.92	0.634	0.297	-	-	-
3DGS [ 13 ]	16.19	0.437	0.388	19.24	0.649	0.237	23.63	0.809	0.129
DRGS [ 7 ]	17.04	0.513	0.318	16.73	0.487	0.310	18.60	0.560	0.239

Table 1: Quantitative results on LLFF with 2, 3, 6 training views. The best and second-best entries are denoted using bold and underline respectively. 3DGS-based methods require multi-view stereo estimation from COLMAP, which fails on 3 scenes for 2 training views. We therefore report the averaged metrics of the remaining scenes.

Methods	2 views			3 views			6 views
Methods	PSNR↑	SSIM↑	LPIPS↓	PSNR↑	SSIM↑	LPIPS↓	PSNR↑	SSIM↑	LPIPS↓
3DGS	16.19	0.437	0.388	19.24	0.649	0.237	23.63	0.809	0.129
FSGS	15.65	0.460	0.412	20.43	0.682	0.248	24.15	0.823	0.128
+VFI	16.91	0.489	0.384	20.68	0.704	0.204	24.18	0.822	0.135
+FADP	17.28	0.503	0.370	20.99	0.720	0.192	24.25	0.824	0.136
+GSField (freeze scale )	17.93	0.564	0.345	21.08	0.732	0.191	24.40	0.832	0.136

Table 2: Ablation study on LLFF dataset with 2, 3, 6 training views. The best and second-best entries are denoted using bold and underline respectively. Under the 2-view setting, the baseline method (FSGS [ 36 ] ) performs poorly, while our approach achieves a significant performance improvement.

Methods	3 Views			6 Views
Methods	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
3DGS	16.39	0.442	0.432	22.49	0.725	0.279
FSGS	16.88	0.474	0.434	23.63	0.742	0.281
CoR-GS	17.06	0.491	0.443	23.59	0.742	0.275
DropGaussian	16.81	0.480	0.438	24.15	0.772	0.215
HeroGS	17.51	0.512	0.422	24.70	0.781	0.272

Table 3: Quantitative results on Tanks&Temples with 3, 6 training views without downsampling.

Figure 4: Qualitative Comparison on LLFF (3 training views). Under extreme view sparsity, 3DGS [ 13 ] collapses into severe artifacts and blurred geometry. DRGS and FSGS recover coarse structure yet still exhibit over-smoothed textures and noisy backgrounds. In contrast, HeroGS guides the model from complementary levels to deliver markedly sharper object boundaries, richer high-frequency textures, and distinctly clearer distant regions—demonstrating the efficacy of our full pipeline.

Figure 5: Comparison with and without image and feature level guidance. Columns 1 and 2: novel-view renderings produced by baseline versus baseline with pseudo-labels and FADP. Columns 3 and 4: their corresponding Gaussians.

Settings	PSNR	SSIM	LPIPS
w/o Post-Freeze	21.16	0.735	0.190
All	21.30	0.739	0.189

Table 4: Ablation study on Post-Freeze Behavior on LLFF dataset for 3 training views.

Settings	PSNR	SSIM	LPIPS
PerVFI [ 29 ]	20.78	0.711	0.204
BIMVFI	20.99	0.720	0.192
Ground Truth	21.09	0.720	0.195
Ours	21.30	0.739	0.189

Table 5: Comparison between different Interpolation Model on LLFF dataset for 3 training views.

Figure 6: Analysis of training dynamics. Comparison of PSNR (left) and the number of Gaussians (right) between our HeroGS and baseline during training.

	PL	FADP	CPG	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
One Level	✓			16.91	0.489	0.384
		✓		16.64	0.483	0.390
			✓	16.78	0.535	0.405
	average			16.78	0.502	0.393
Two Levels	✓	✓		17.28	0.503	0.370
	✓		✓	18.33	0.577	0.336

Table 6: Ablation study on level interdependencies on LLFF dataset for 2 training views.

Figure 7: Qualitative Comparison on Tanks for 3 training views. In large-scale dataset, 3DGS and DropGaussian struggles to maintain geometry and texture fidelity, exhibiting significant artifacts. FSGS and CoR-GS recover coarse structure yet still exhibit over-smoothed geometry and artifacts. In contrast, HeroGS reconstruct fine structures and high-frequency textures.

Methods	12 views			24 views
Methods	PSNR↑	SSIM↑	LPIPS↓	PSNR↑	SSIM↑	LPIPS↓
3DGS	18.44	0.521	0.385	23.22	0.730	0.234
FSGS	18.93	0.539	0.380	23.46	0.738	0.237
CoR-GS	19.59	0.578	0.374	23.39	0.727	0.272
DropGaussian	19.49	0.573	0.366	24.03	0.762	0.225
HeroGS	19.99	0.591	0.373	24.18	0.766	0.229

Table 7: Quantitative results on Mip-NeRF360 [ 3 ] with 12, 24 training views.

Figure 8: Ablation study on LLFF dataset showing the impact of varying interpolation factor $S$ on PSNR, SSIM, and LPIPS.

Settings	PSNR	SSIM	LPIPS
w/o Selection	21.19	0.736	0.190
All	21.30	0.739	0.189

Table 8: Ablation study on Selection module on LLFF dataset for 3 training views.

Figure 9: Qualitative comparison of depth rendering quality. We visualize the depth maps rendered from the reconstructed 3D Gaussian fields of different methods and our proposed HeroGS framework. The visualizations show that our method produces more consistent and artifact-free depth results with finer geometry structure preservation.

View-Dependent 3DGS

See all 8 papers

Nov 30, 2023

SparseGS: Sparse View Synthesis using 3D Gaussian Splatting

3D Gaussian Splatting (3DGS) has recently enabled real-time rendering of unbounded 3D scenes for novel view synthesis. However, this technique requires dense training views to accurately reconstruct 3D geometry. A limited number of input views will significantly degrade reconstruction quality, resulting in artifacts such as "floaters" and "background collapse" at unseen viewpoints. In this work, we introduce SparseGS, an efficient training pipeline designed to address the limitations of 3DGS in scenarios with sparse training views. SparseGS incorporates depth priors, novel depth rendering techniques, and a pruning heuristic to mitigate floater artifacts, alongside an Unseen Viewpoint Regularization module to alleviate background collapses. Our extensive evaluations on the Mip-NeRF360, LLFF, and DTU datasets demonstrate that SparseGS achieves high-quality reconstruction in both unbounded and forward-facing scenarios, with as few as 12 and 3 input images, respectively, while maintaining fast training and real-time rendering capabilities.

Haolin Xiong, Sairisheek Muttukuru, Hanyuan Xiao +4

University of California, Los Angeles · USC Institute for Creative Technologies · University of Southern California

3D Gaussian Splatting Novel View Synthesis Sparse-View 3D Reconstruction

Ask PDF

Sep 19, 2026

D3GS: Depth, DINO, and RGB Diffusion Co-Guided 3D Gaussian Splatting for Sparse-View Reconstruction

Novel view synthesis from sparse inputs remains challenging for 3D Gaussian Splatting (3DGS) due to ambiguous geometry, cross-view inconsistency, and missing details in under-constrained regions, resulting in degraded reconstruction and unstable rendering. To tackle these issues, we propose D

^{3}

GS, a Depth-DINO-Diffusion guided sparse-view Gaussian reconstruction framework that jointly enhances geometry and appearance. D

^{3}

GS first recovers a high-resolution, metric depth map via diffusion-based completion and DPT (Dense Prediction Transformer) refinement, providing robust Gaussian initialization and geometric constraints. Then, a DINO-guided view-consistent learning is introduced to augment Gaussian attributes with structural features, improving multi-view consistency. Finally, a diffusion-based Gaussian refinement module injects generative priors into an iterative optimization strategy, enhancing high-frequency geometric and appearance details within the Gaussian representation. Experiments on DTU, LLFF, and Mip-NeRF 360 show that D

^{3}

GS achieves consistent and substantial improvements over strong baselines, with ablation studies validating the effectiveness and complementary roles of each component.

Yunqi Gao, Zhanfeng Liao, Hanzhang Tu +7

Central China Normal University · Tsinghua University · Fuzhou University +4

Sparse-View 3D Reconstruction 3D Gaussian Splatting 3D Reconstruction Multi-View Consistency

Ask PDF

Jun 29, 2026

StereoGS: Sparse-View 3D Gaussian Splatting via Stereo Priors

3D Gaussian Splatting (3DGS) has achieved remarkable success in real-time novel view synthesis, yet it suffers from severe overfitting under sparse-view settings due to insufficient geometric constraints. While recent methods introduce monocular depth priors to mitigate this, they inherently struggle with scale ambiguity and cross-view inconsistency, leading to defective geometry. In this paper, we propose StereoGS, a novel sparse-view 3DGS framework that integrates stereo priors to establish reliable binocular consistency. Unlike scale-agnostic monocular constraints, StereoGS introduces a Stereo Depth Regularization by constructing virtual stereo pairs during optimization and leveraging a foundation stereo model to enforce absolute scale and binocular-consistent structures. To further suppress overfitting and eliminate redundant primitives, we design a Gradient-Aware Opacity Decay strategy that dynamically penalizes Gaussians based on their relative opacity gradient magnitudes. Combined with a Consistency-Aware Dense Initialization using zero-shot multi-view depth estimation, StereoGS effectively anchors primitives to accurate scene surfaces. Extensive experiments on LLFF, DTU, Mip-NeRF360, and Blender datasets demonstrate that StereoGS achieves state-of-the-art performance in sparse-view settings without incurring any additional inference overhead. Project Page: https://stringerywh00.github.io/StereoGS_project_page/

Wenhao Yuan, Yiyuan Ge, Deli Cai

South China University of Technology, China

Multi-View Consistency 3D Gaussian Splatting Sparse-View 3D Reconstruction

Ask PDF

Abstract

View-Dependent 3DGS

Explore similar work

SparseGS: Sparse View Synthesis using 3D Gaussian Splatting

D3GS: Depth, DINO, and RGB Diffusion Co-Guided 3D Gaussian Splatting for Sparse-View Reconstruction

StereoGS: Sparse-View 3D Gaussian Splatting via Stereo Priors