S2D: Sparse to Dense Lifting for 3D Reconstruction with Minimal Inputs

Yuzhou Ji; Qijian Tian; He Zhu; Xiaoqi Jiang; Guangzhi Cao; Lizhuang Ma; Yuan Xie; Xin Tan

cs.CVMar 11, 2026

S2D: Sparse to Dense Lifting for 3D Reconstruction with Minimal Inputs

Authors: Yuzhou Ji, Qijian Tian, He Zhu, Xiaoqi Jiang, Guangzhi Cao, Lizhuang Ma, Yuan Xie, Xin Tan

Organizations: Shanghai Jiao Tong University · Chery Automobile · East China Normal University · Shanghai Artificial Intelligence Laboratory

Abstract

Explicit 3D representations have already become an essential medium for 3D simulation and understanding. However, the most commonly used point cloud and 3D Gaussian Splatting (3DGS) each suffer from non-photorealistic rendering and significant degradation under sparse inputs. In this paper, we introduce Sparse to Dense lifting (S2D), a novel pipeline that bridges the two representations and achieves high-quality 3DGS reconstruction with minimal inputs. Specifically, the S2D lifting is two-fold. We first present an efficient one-step diffusion model that lifts sparse point cloud for high-fidelity image artifact fixing. Meanwhile, to reconstruct 3D consistent scenes, we also design a corresponding reconstruction strategy with random sample drop and weighted gradient for robust model fitting from sparse input views to dense novel views. Extensive experiments show that S2D achieves the best consistency in generating novel view guidance and first-tier sparse view reconstruction quality under different input sparsity. By reconstructing stable scenes with the least possible captures among existing methods, S2D enables minimal input requirements for 3DGS applications.

Figures & tables

Figure 2: S2D reconstruction pipeline and model architecture of artifact fixer.

Figure 3: Evaluation on parameter

\alpha

and

\beta

Figure 4: Qualitative results in different situations.

Datasets	3DOVS @ 1 Image				RE10K @ 2 Images				MIP360 @ 6 Images				DL3DV @ 6 Images
Datasets	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$
3DGS [ 19 ]	10.12	0.30	0.69	278.9	21.14	0.61	0.36	64.3	17.14	0.48	0.41	71.9	15.26	0.51	0.46	97.7
Mip-Splatting [ 72 ]	11.31	0.29	0.61	249.6	21.35	0.65	0.31	63.2	16.33	0.44	0.48	91.3	18.50	0.59	0.41	86.3
ScaffoldGS [ 25 ]	10.50	0.23	0.62	230.4	20.96	0.60	0.32	67.6	16.02	0.35	0.46	162.2	17.41	0.60	0.39	89.7
NoPoSplat [ 70 ]	-	-	-	-	24.30	0.70	0.36	41.0	15.48	0.37	0.46	91.4	18.11	0.50	0.41	86.4
AnySplat [ 17 ]	17.91	0.48	0.42	67.0	24.32	0.69	0.35	40.6	18.12	0.60	0.44	96.4	17.39	0.59	0.43	85.3

Table 1: Quantitative results in in-the-wild scenes.

Methods	Interpolation		Lane Shift @ FID $\downarrow$
Methods	PSNR $\uparrow$	LPIPS $\downarrow$	side 2m	side 3m	up 1.5m
EmerNeRF [ 68 ]	25.32	0.20	96.4	101.3	90.2
StreetGaussians [ 64 ]	30.14	0.14	75.0	96.7	70.3
StreetCrafter [ 65 ]	29.31	0.10	57.4	66.4	59.0
DIFIX [ 58 ]	30.26	0.11	60.2	71.6	60.3
S2D (Ours)	31.44	0.07	46.1	53.9	41.3

Table 2: Quantitative results on Waymo Open Dataset.

Figure 5: Comparison of artifact removal with ablations on guidance mixing.

#	Ablations	PSNR $\uparrow$	LPIPS $\downarrow$
①	Generation from point cloud	12.7	0.71
②	Mixing latents	15.6	0.49
③	No mixing	19.0	0.38
④	Dual guidance w/o DINO	22.1	0.27
⑤	Dual guidance w DINO	23.0	0.26

Table 3: Ablation on dual guidance in artifact fixer.

Figure 6: Ablations on reconstruction strategy. WG indicates weighted gradient, and RSD for random sample drop.

Ablations	PSNR $\uparrow$	LPIPS $\downarrow$
$\mathcal{L}_{LPIPS}+\mathcal{L}_{2}+\mathcal{L}_{GAN}+\mathcal{L}_{CLIP}$	20.1	0.35
$\mathcal{L}_{LPIPS}+\mathcal{L}_{2}+\mathcal{L}_{GAN}$	20.1	0.35
$\mathcal{L}_{LPIPS}+\mathcal{L}_{2}+\mathcal{L}_{GAN}+\mathcal{L}_{DINO}$	20.5	0.33
$\mathcal{L}_{LPIPS}+\mathcal{L}_{2}+\mathcal{L}_{GAN}+\mathcal{L}_{SSIM}$	21.8	0.28
$\mathcal{L}_{LPIPS}+\mathcal{L}_{2}+\mathcal{L}_{GAN}+\mathcal{L}_{SSIM}+\mathcal{L}_{DINO}$	22.0	0.27

Table 4: Ablation on loss terms.

Figure 7: Training data with different levels of artifacts (line 1) and example of random perturbations (line 2).

Figure 8: Reconstruction quality versus input density.

Figure 9: Extra results of DiffusionGS and driving scene comparison.

Figure 10: Point cloud rendering with single image input.

Sparse-View 3D Reconstruction

Sparse-view 3D reconstruction recovers scene geometry and appearance from a small number of images for novel-view rendering and 3D use.

See all 188 papers

Sparse-to-Complete: From Sparse Image Captures to Complete 3D Scenes
May 7, 2026Yiyang Shen, Yin Yang, Kun Zhou +13D Gaussian Splatting Diffusion Models for Image Restoration
State Key Lab of CAD&CG, Zhejiang University, China · University of Utah, USA · State Key Lab of CAD&CG, Zhejiang University, China and Hangzhou Research Institute of Holographic and AI Technology, China +1
SparseGS: Sparse View Synthesis using 3D Gaussian Splatting
Nov 30, 2023Haolin Xiong, Sairisheek Muttukuru, Hanyuan Xiao +43D Gaussian Splatting Novel View Synthesis
University of California, Los Angeles · USC Institute for Creative Technologies · University of Southern California
D3GS: Depth, DINO, and RGB Diffusion Co-Guided 3D Gaussian Splatting for Sparse-View Reconstruction
Sep 19, 2026Yunqi Gao, Zhanfeng Liao, Hanzhang Tu +7Sparse-View 3D Reconstruction 3D Gaussian Splatting
Central China Normal University · Tsinghua University · Fuzhou University +4

May 7, 2026J/K move · Enter open · S save

Sparse-to-Complete: From Sparse Image Captures to Complete 3D Scenes

Yiyang Shen, Yin Yang, Kun Zhou, Tianjia Shao

State Key Lab of CAD&CG, Zhejiang University, China · University of Utah, USA · State Key Lab of CAD&CG, Zhejiang University, China and Hangzhou Research Institute of Holographic and AI Technology, China+1

3D Gaussian Splatting Diffusion Models for Image Restoration Sparse-View 3D Reconstruction

Open paper AskPDF arXiv

We introduce S2C-3D, a novel sparse-view 3D reconstruction framework for high-fidelity and complete scene reconstruction from as few as six to eight images. Our framework features three components: a specialized diffusion model for scene-specific image restoration, a training-free view-consistency conditioned sampling process in the diffusion model for refined Gaussian optimization, and a camera trajectory planning scheme to ensure comprehensive scene coverage. The specialized diffusion model is developed by finetuning a pretrained architecture on the input views and their corresponding degraded counterparts. The adaptation to the scene distribution allows the model to repair Gaussian renderings while effectively eliminating domain gaps. Meanwhile, the trajectory planning scheme optimizes scene coverage by connecting each newly sampled camera to its two nearest neighbors. By iteratively constructing paths and retaining only those that significantly enhance visibility, the scheme establishes a trajectory that covers the entire scene. To address multi-view conflicts, the view-consistency conditioned sampling process quantifies the consistency between neighboring repaired images. This information is injected as a condition into the sampling process of the frozen diffusion model, facilitating the generation of view-consistent images without additional training. Consequently, our approach produces high-fidelity 3D Gaussians that are robust to artifacts. Experimental results demonstrate that S2C-3D outperforms state-of-the-art methods, constructing high-quality scenes that are free from missing regions, blurring, or other artifacts with very sparse inputs. The source code and data are available at https://gapszju.github.io/S2C-3D.

Abstract

Sparse-View 3D Reconstruction

Explore similar work

Sparse-to-Complete: From Sparse Image Captures to Complete 3D Scenes

SparseGS: Sparse View Synthesis using 3D Gaussian Splatting

D3GS: Depth, DINO, and RGB Diffusion Co-Guided 3D Gaussian Splatting for Sparse-View Reconstruction