Remote Sensing Sparse-View 3D Gaussian Splatting via Depth Image-Based Rendering

Jiaming Kang; Zhengxia Zou; Zhenwei Shi

cs.CVSep 28, 2026

Remote Sensing Sparse-View 3D Gaussian Splatting via Depth Image-Based Rendering

Authors: Jiaming Kang, Zhengxia Zou, Zhenwei Shi

Organizations: Beihang University

Abstract

Remote sensing novel view synthesis under sparse observations remains challenging due to insufficient geometric constraints and limited cross-view supervision. Existing Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) methods are prone to overfitting and face challenges of depth ambiguities, missing cross-view information, and insufficient constraints in under-observed regions. To address these challenges, we propose DIBR-GS, a neural Gaussian Splatting framework that exploits Depth Image-Based Rendering (DIBR) to generate pseudo views for cross-view consistency supervision. Specifically, reliable geometric initialization is constructed by aligning monocular depth priors with sparse SfM reconstruction, and cross-view appearance priors are incorporated into neural Gaussian representations to enhance appearance modeling under sparse observations. Furthermore, we introduce a progressive DIBR-based pseudo-view supervision strategy to provide additional geometric and appearance constraints, enabling more complete reconstruction of weakly observed regions. In addition, a height-constrained anchor growth strategy is designed to suppress unreasonable Gaussian expansion. Experiments demonstrate that the proposed method achieves superior performance over existing approaches when training with only 3 input views. Compared with the previous best-performing method, it improves PSNR by 6.83 dB, with relative gains of 14% in SSIM and 60% in LPIPS, while maintaining competitive computational efficiency. Our code is available at https://github.com/kanehub/DIBR-GS

Figures & tables

Fig. 1: Visual and quantitative comparison on the LEVIR-NVS dataset with 3 input views. The proposed method produces more complete structures and finer details compared with existing approaches, especially in weakly observed regions. The radar chart summarizes the performance comparison in terms of reconstruction quality and rendering efficiency. For visualization, we report the negative LPIPS, AVGE, and $1+\log(\mathrm{FPS})$ values to enable unified comparison.

Fig. 2: Overview of the proposed DIBR-GS framework. The framework consists of a DIBR branch and a neural Gaussian branch. In the DIBR branch, monocular depth priors are aligned with sparse SfM reconstruction through affine transformation and multi-view fusion to construct a reliable geometric initialization. Cross-view appearance features are extracted following the image-based rendering paradigm, and progressive DIBR is further employed to generate pseudo views that provide additional supervision. In the neural Gaussian branch, Gaussian anchors are initialized from the dense point cloud, while the anchor features and cross-view appearance features are adaptively integrated to predict Gaussian attributes. The rendered training views and pseudo views are jointly optimized, enabling more complete and consistent reconstruction under sparse-view remote sensing scenes.

Fig. 3: Illustration of anchor growth. (a) Anchor distribution after growth. The original anchors contain numerous floating artifacts and deviate significantly from the initial point cloud, while the Gaussians become more compact and precise with the constraint. (b) The proposed height-constrained anchor growth strategy, where only neural Gaussians whose heights fall within the predefined bounds are considered as candidate new anchors.

Fig. 4: Visual comparison on LEVIR-NVS Dataset.

Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	AVGE $\downarrow$	FPS $\uparrow$
RegNeRF [ 20 ]	19.83	0.695	0.389	0.131	0.19
FreeNeRF [ 21 ]	19.04	0.524	0.373	0.148	0.19
MPNeRF [ 4 ]	21.72	0.800	0.190	0.083	0.18
TriDF [ 52 ]	24.07	0.820	0.213	0.071	0.20
3DGS [ 16 ]	18.29	0.593	0.313	0.144	280
FSGS [ 27 ]	21.18	0.772	0.230	0.094	343

TABLE I: Quantitative comparison of rendering quality and efficiency between different methods. The best, second-best, and third-best entries are marked in , , and , respectively.

Fig. 5: Visualization of rendering quality when removing different model components.

Setting	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	AVGE $\downarrow$
w/o Dense Init.	29.18	0.917	0.097	3.232
w/o IBR Feature	30.56	0.933	0.082	2.652
w/o Height Map	30.68	0.935	0.080	2.593
w/o Pseudo Views	21.06	0.782	0.186	8.795
Ours	30.90	0.938	0.076	2.488

TABLE III: Ablation study of different components. The AVGE values are multiplied by 1e2.

Fig. 6: Visualization of depth map and depth error with different initialization strategies.

Map	Source	Refine	MAE $\downarrow$	Abs Rel $\downarrow$	AVGE $\downarrow$
Neg.	SfM	–	3.72	0.0322	3.853
Inv.	SfM	–	3.16	0.0273	3.522
Inv.	Fused	–	3.31	0.0285	3.485
Inv.	SfM	Reproject	2.18	0.0187	3.406
Inv.	SfM	No ovlp.	1.91	0.0165	3.302

TABLE IV: Quantitative comparison under different mapping, rescaling, and refinement strategies. The AVGE values are multiplied by 1e2.

Fig. 7: Performance under different size of voxel grid down-sampling.

Fig. 8: Performance under different size of anchor voxel.

Size	Pts(k)	GS(k)	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	T(min) $\downarrow$
-	574.4	525.5	28.43	0.954	0.052	43.9
0.2	463.4	421.1	28.22	0.952	0.052	38.5
0.3	282.8	263.2	28.04	0.950	0.056	28.7
0.5	130.3	139.0	27.72	0.943	0.064	19.9
1.0	42.2	92.7	27.36	0.935	0.074	17.8
$\infty$	5.5	69.9	25.59	0.906	0.105	17.3

TABLE V: Performance comparison for different sizes of voxel grid downsampling.

Voxel Size	Pts(k)	GS(k)	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
0.01	42.2	150.4	28.32	0.951	0.057
0.03	42.2	92.7	27.36	0.935	0.074
0.05	42.2	64.6	26.54	0.921	0.089
0.10	42.2	54.3	25.95	0.909	0.100
0.54	38.1	43.7	25.50	0.896	0.115

TABLE VI: Performance comparison for different sizes of anchor voxel.

$C_{f}$	$C_{r}$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
32	32	30.90	0.938	0.076
32	64	30.72	0.935	0.080
32	128	30.71	0.935	0.080
32	256	30.84	0.936	0.078
16	64	30.39	0.930	0.088
32	64	30.72	0.935	0.080

TABLE VII: Effect of different anchor feature and IBR feature channels.

Setting	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
w/o MLP $\mathcal{E}_{\mathrm{mod}}$	30.82	0.937	0.077
w/o ft. $\mathcal{P}$	30.79	0.936	0.079
w/o ResNet Feature	30.85	0.937	0.078
w/o ViT Feature	30.81	0.936	0.078
Ours	30.90	0.938	0.076

TABLE VIII: Ablation study on different network settings.

Fig. 9: Illustration and analysis of the proposed pseudo view synthesis. (a) Pipeline of the pseudo view synthesis through progressive DIBR.(b) Example of the progressive synthesis process.

Explore similar work

SparseGS: Sparse View Synthesis using 3D Gaussian Splatting
Nov 30, 2023Haolin Xiong, Sairisheek Muttukuru, Hanyuan Xiao +4Novel View Synthesis Synthesis
University of California, Los Angeles · USC Institute for Creative Technologies · University of Southern California
AugSplat: Radiance Field-Informed Gaussian Splatting for Sparse-View Settings
Jun 30, 2026Lorenzo Lazzaroni, Riccardo Bollati, Daniel Barath +2Sparse-View Gaussian Splatting
ETH Zurich · 1Google · 2ETH Zurich
StereoGS: Sparse-View 3D Gaussian Splatting via Stereo Priors
Jun 29, 2026Wenhao Yuan, Yiyuan Ge, Deli Cai3D Gaussian Stereo Vision
South China University of Technology, China

Nov 30, 2023, cs.CVJ/K move · Enter open · S save

SparseGS: Sparse View Synthesis using 3D Gaussian Splatting

Haolin Xiong, Sairisheek Muttukuru, Hanyuan Xiao, Rishi Upadhyay+3

University of California, Los Angeles · USC Institute for Creative Technologies · University of Southern California

Novel View Synthesis Synthesis Artifacts

Open paper AskPDF arXiv

3D Gaussian Splatting (3DGS) has recently enabled real-time rendering of unbounded 3D scenes for novel view synthesis. However, this technique requires dense training views to accurately reconstruct 3D geometry. A limited number of input views will significantly degrade reconstruction quality, resulting in artifacts such as "floaters" and "background collapse" at unseen viewpoints. In this work, we introduce SparseGS, an efficient training pipeline designed to address the limitations of 3DGS in scenarios with sparse training views. SparseGS incorporates depth priors, novel depth rendering techniques, and a pruning heuristic to mitigate floater artifacts, alongside an Unseen Viewpoint Regularization module to alleviate background collapses. Our extensive evaluations on the Mip-NeRF360, LLFF, and DTU datasets demonstrate that SparseGS achieves high-quality reconstruction in both unbounded and forward-facing scenarios, with as few as 12 and 3 input images, respectively, while maintaining fast training and real-time rendering capabilities.

Abstract

Explore similar work

SparseGS: Sparse View Synthesis using 3D Gaussian Splatting

AugSplat: Radiance Field-Informed Gaussian Splatting for Sparse-View Settings

StereoGS: Sparse-View 3D Gaussian Splatting via Stereo Priors