Multimodal-Prior-Guided Importance Sampling for Hierarchical Gaussian Splatting in Sparse-View Novel View Synthesis

Kaiqiang Xiong; Zhanke Wang; Ronggang Wang

cs.CVMar 3, 2026

Multimodal-Prior-Guided Importance Sampling for Hierarchical Gaussian Splatting in Sparse-View Novel View Synthesis

Authors: Kaiqiang Xiong, Zhanke Wang, Ronggang Wang

Organizations: Guangdong Provincial Key Laboratory of Ultra High Definition Immersive Media Technology, Shenzhen Graduate School, Peking University, China · Peng Cheng Laboratory, China · Migu Culture Technology Co., Ltd., China

Abstract

We present multimodal-prior-guided importance sampling as the central mechanism for hierarchical 3D Gaussian Splatting (3DGS) in sparse-view novel view synthesis. Our sampler fuses complementary cues { -- } photometric rendering residuals, semantic priors, and geometric priors { -- } to produce a robust, local recoverability estimate that directly drives where to inject fine Gaussians. Built around this sampling core, our framework comprises (1) a coarse-to-fine Gaussian representation that encodes global shape with a stable coarse layer and selectively adds fine primitives where the multimodal metric indicates recoverable detail; and (2) a geometric-aware sampling and retention policy that concentrates refinement on geometrically critical and complex regions while protecting newly added primitives in underconstrained areas from premature pruning. By prioritizing regions supported by consistent multimodal evidence rather than raw residuals alone, our method alleviates overfitting texture-induced errors and suppresses noise from pose/appearance inconsistencies. Experiments on diverse sparse-view benchmarks demonstrate state-of-the-art reconstructions, with up to +0.3 dB PSNR on DTU.

Figures & tables

Figure 2: Framework. Our hierarchical Gaussian Splatting framework with Multimodal-Prior-Guided importance sampling for sparse-view novel view synthesis. It consists of three main components: (a) a Hierarchical Gaussian representation with coarse and fine levels (in Sec. 2.2 ), (b) a Multi-Modal Importance Assessment module (in Sec. 2.3 ), and (c) a Geometric-aware Sampling and pruning strategy (in Sec. 2.4 ).

Figure 3: Qualitative results on LLFF [ 11 ] and DTU [ 3 ] Datasets.

Method	LLFF (3 Views)			DTU (3 Views)			MipNeRF-360 (24 Views)
Method	PSNR↑	SSIM↑	LPIPS↓	PSNR↑	SSIM↑	LPIPS↓	PSNR↑	SSIM↑	LPIPS↓
DietNeRF [ 13 ]	14.94	0.370	0.496	11.85	0.633	0.314	20.21	0.557	0.387
FreeNeRF [ 14 ]	19.63	0.612	0.308	19.92	0.787	0.182	22.78	0.689	0.323
SparseNeRF [ 15 ]	19.86	0.624	0.328	19.55	0.769	0.201	22.85	0.693	0.315
3DGS [ 4 ]	18.54	0.588	0.272	17.65	0.816	0.146	21.71	0.672	0.248
DNGaussian [ 6 ]	19.12	0.591	0.294	18.91	0.790	0.176	18.06	0.423	0.584

Table 1: Quantitative comparison on LLFF [ 11 ] , DTU [ 3 ] , and MipNeRF-360 [ 12 ] datasets. We color each cell as best , second best .

$Hier$	$S_{rend}$	$S_{sem}$	$S_{geo}$	$RA$	$AGP$	$PM$	PSNR↑	SSIM↑	LPIPS↓
$\times$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	20.35	0.870	0.103
$\checkmark$	$\times$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	20.32	0.870	0.104
$\checkmark$	$\checkmark$	$\times$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	20.41	0.871	0.101
$\checkmark$	$\checkmark$	$\checkmark$	$\times$	$\checkmark$	$\checkmark$	$\checkmark$	20.43	0.871	0.104
$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\times$	$\checkmark$	$\checkmark$	20.30	0.869	0.103
$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\times$	$\checkmark$	20.36	0.872	0.104

Table 2: Ablation study on DTU [ 3 ] with 3 training views.

Novel View Synthesis

Novel View Synthesis creates images of a scene from camera viewpoints not present in the input, using learned geometry and appearance models.

See all 289 papers

D3GS: Depth, DINO, and RGB Diffusion Co-Guided 3D Gaussian Splatting for Sparse-View Reconstruction
Sep 19, 2026Yunqi Gao, Zhanfeng Liao, Hanzhang Tu +7Sparse-View 3D Reconstruction 3D Gaussian Splatting
Central China Normal University · Tsinghua University · Fuzhou University +4
SplatGuide: Geometric Priors from 3D Gaussians for Pose-Free Novel View Synthesis
Aug 17, 2026Yejun Zhang, Zihan Wang, Xu Ji +8Multi-View Diffusion Models Novel View Synthesis
Aalto University, Finland · Deep Render, UK · ELLIS Institute Finland +3
HeroGS: Hierarchical Guidance for Robust 3D Gaussian Splatting under Sparse Views
Mar 1, 2026Jiashu Li, Xumeng Han, Zhaoyang Wei +5View-Dependent 3DGS 3D Gaussian Splatting
University of the Chinese Academy of Sciences

Sep 19, 2026J/K move · Enter open · S save

D3GS: Depth, DINO, and RGB Diffusion Co-Guided 3D Gaussian Splatting for Sparse-View Reconstruction

Yunqi Gao, Zhanfeng Liao, Hanzhang Tu, Zhaoqi Su+6

Central China Normal University · Tsinghua University · Fuzhou University+4

Sparse-View 3D Reconstruction 3D Gaussian Splatting 3D Reconstruction Multi-View Consistency

Open paper AskPDF arXiv

Novel view synthesis from sparse inputs remains challenging for 3D Gaussian Splatting (3DGS) due to ambiguous geometry, cross-view inconsistency, and missing details in under-constrained regions, resulting in degraded reconstruction and unstable rendering. To tackle these issues, we propose D

^{3}

GS, a Depth-DINO-Diffusion guided sparse-view Gaussian reconstruction framework that jointly enhances geometry and appearance. D

^{3}

GS first recovers a high-resolution, metric depth map via diffusion-based completion and DPT (Dense Prediction Transformer) refinement, providing robust Gaussian initialization and geometric constraints. Then, a DINO-guided view-consistent learning is introduced to augment Gaussian attributes with structural features, improving multi-view consistency. Finally, a diffusion-based Gaussian refinement module injects generative priors into an iterative optimization strategy, enhancing high-frequency geometric and appearance details within the Gaussian representation. Experiments on DTU, LLFF, and Mip-NeRF 360 show that D

^{3}

GS achieves consistent and substantial improvements over strong baselines, with ablation studies validating the effectiveness and complementary roles of each component.

Abstract

Novel View Synthesis

Explore similar work

D3GS: Depth, DINO, and RGB Diffusion Co-Guided 3D Gaussian Splatting for Sparse-View Reconstruction

SplatGuide: Geometric Priors from 3D Gaussians for Pose-Free Novel View Synthesis

HeroGS: Hierarchical Guidance for Robust 3D Gaussian Splatting under Sparse Views