Spackle: Completing Large View Single Image NVS with Adaptive Gaussians

Xuanzhi Liu; Yuhe Zhou; Xinyi Wu; Zhenyao Wu; Jinghao Chen; Ruize Han; Song Wang

cs.CVSep 25, 2026

Spackle: Completing Large View Single Image NVS with Adaptive Gaussians

Authors: Xuanzhi Liu, Yuhe Zhou, Xinyi Wu, Zhenyao Wu, Jinghao Chen, Ruize Han, Song Wang

Organizations: Shenzhen University of Advanced Technology · HONOR

Abstract

Single-image novel view synthesis (NVS) enables photorealistic rendering of un- observed viewpoints from a single input. Practical NVS systems require two key capabilities: robust reconstruction of occluded regions and high inference effi- ciency. While hybrid decoupled frameworks combining feedforward 3D Gaussian Splatting (3DGS) and diffusion models show promise for large-view-deviation NVS, they suffer from capacity competition: a fixed number of Gaussians forces resource shifts from visible to newly disoccluded areas, degrading original scene fidelity when the target view deviates significantly from the input. To address this, we propose Spackle, a lightweight residual learning framework that mit- igates capacity competition without sacrificing efficiency. Spackle operates in three stages: predicting base 3DGS attributes from given views, automatically identifying poorly reconstructed regions, and learning a residual 3DGS optimized exclusively for these areas. At inference, we combine the baseline and aug- mented Gaussians for NVS. We conduct comprehensive experiments and show that Spackle achieves state-of-the-art performance on large-view-deviation cases.

Figures & tables

Figure 1: Comparison of single-image NVS paradigms. (a) Feed-forward 3DGS is efficient and preserves visible regions but leaves disoccluded regions unfilled. (b) Decoupled framework inpaints disoccluded regions but suffers from capacity competition. (c) Our residual-learning framework mitigates capacity competition, effectively fills disoccluded regions, and preserves visible regions while maintaining computational efficiency.

Figure 2: Overview of the Spackle module. The frozen baseline 3DGS model renders RGB, depth, opacity, and a binary mask for the target novel view. During training, we use ProPainter Zhou et al. (2023) to generate video inpainting pseudo targets for regions indicated by the binary mask. Spackle trains a residual 3DGS on these rendered cues to predict augmented Gaussian attributes on a fixed grid. At inference, only predictions inside the binary mask are converted into augmented Gaussians and combined with the baseline Gaussians for final NVS. This improves unseen regions that are not well reconstructed while preserving visible-region fidelity.

Figure 3: Qualitative comparison under large viewpoint changes. From left to right, each row shows the input image, the SHARP novel-view rendering with representative artifacts highlighted, and our Spackle novel-view rendering. The baseline 3DGS model uses a fixed total number of 1,179,648 Gaussians. The tag at the top-right corner of each Spackle result reports the percentage of augmented Gaussians introduced by Spackle relative to this fixed baseline Gaussian count. When the target viewpoint deviates significantly from the input camera pose, Spackle better reconstructs unseen regions that are not well reconstructed by the baseline model, while preserving the fidelity of visible regions.

Dataset	Method	S1: Disparity 0.3			S2: Disparity 0.4			S3: Disparity 0.5
Dataset	Method	CLIPIQA $\uparrow$	NIQE $\downarrow$	MUSIQ $\uparrow$	CLIPIQA $\uparrow$	NIQE $\downarrow$	MUSIQ $\uparrow$	CLIPIQA $\uparrow$	NIQE $\downarrow$	MUSIQ $\uparrow$
Middlebury	SHARP	0.3904	4.3102	58.2782	0.3913	4.2697	57.7797	0.3926	4.1992	56.8124
Middlebury	Spackle	0.3921	4.3127	58.1583	0.3969	4.2825	57.6487	0.4032	4.2268	56.9233
Booster	SHARP	0.4221	5.1467	55.7906	0.4314	5.1630	55.0354	0.4397	5.1894	54.6178
Booster	Spackle	0.4224	5.1466	55.7929	0.4325	5.1625	54.9989	0.4443	5.1868	54.5166
WildRGBD	SHARP	0.2782	4.3972	45.2073	0.2595	4.4207	42.3572	0.2451	4.4680	39.9257

Table 1: Quantitative Evaluation under Controlled Large-View Rendering. Three increasingly challenging novel view settings are generated by increasing disparity. Higher CLIPIQA and MUSIQ scores indicate better perceptual quality, while lower NIQE scores indicate better perceptual quality.

Dataset	S1: Disparity 0.3			S2: Disparity 0.4			S3: Disparity 0.5
Dataset	SHARP HAR $\downarrow$	Spackle HAR $\downarrow$	Reduction $\uparrow$	SHARP HAR $\downarrow$	Spackle HAR $\downarrow$	Reduction $\uparrow$	SHARP HAR $\downarrow$	Spackle HAR $\downarrow$	Reduction $\uparrow$
Middlebury	0.0121	0.0011	90.47%	0.0305	0.0035	88.52%	0.0566	0.0084	85.13%
Booster	0.0036	0.0003	91.84%	0.0103	0.0006	94.51%	0.0242	0.0011	95.40%
WildRGBD	0.0503	0.0043	91.37%	0.0885	0.0166	81.29%	0.1286	0.0338	73.75%
Tanks	0.1464	0.0369	74.77%	0.2177	0.0569	73.83%	0.2794	0.0620	77.81%
ETH3D	0.0680	0.0120	82.33%	0.1181	0.0261	77.87%	0.1720	0.0406	76.41%

Table 2: Large view hole coverage evaluation on benchmark datasets. Controlled camera changes with increasing disparity expose newly disoccluded regions. We report Hole Area Ratio (HAR), defined as

1-\mathrm{Coverage}

from the rendered opacity map. Lower HAR indicates fewer remaining holes, and Reduction measures the relative HAR decrease of Spackle over SHARP.

Dataset	S1: Disparity 0.3		S2: Disparity 0.4		S3: Disparity 0.5
Dataset	Added GS	Added GS (%)	Added GS	Added GS (%)	Added GS	Added GS (%)
Middlebury	3,387	0.29	8,501	0.72	15,782	1.34
Booster	1,074	0.09	3,077	0.26	7,226	0.61
WildRGBD	3,823	0.32	6,755	0.57	9,826	0.83
Tanks	33,817	2.87	50,254	4.26	64,466	5.46
ETH3D	18,663	1.58	32,382	2.75	47,105	3.99

Table 3: Efficiency and Gaussian overhead under controlled large view settings. SHARP uses a fixed base set of 1,179,648 Gaussians. Spackle adaptively introduces additional Gaussians according to the disocclusion regions. Runtime is reported as the median over datasets for each setting to reduce the influence of measurement outliers.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

	Middlebury		Booster		WildRGBD		Tanks		ETH3D		LLFF
Method	DISTS	LPIPS	DISTS	LPIPS	DISTS	LPIPS	DISTS	LPIPS	DISTS	LPIPS	DISTS	LPIPS
SHARP	0.0875	0.3675	0.0872	0.3076	0.0451	0.2002	0.5238	0.7392	0.3644	0.6627	0.4837	0.7154
Direct FT	0.1506	0.4447	0.1390	0.3740	0.0662	0.2414	0.5698	0.7602	0.4501	0.7261	0.6042	0.7325
Spackle	0.0874	0.3674	0.0871	0.3075	0.0451	0.2004	0.4839	0.7313	0.3318	0.6425	0.4589	0.7180

Appendix

Table 4: Quantitative comparison between SHARP, the naive decoupled framework, and Spackle. Lower is better. The naive decoupled framework directly optimizes the baseline 3DGS model with pseudo supervision, while Spackle preserves the baseline 3DGS model and trains augmented Gaussians for newly disoccluded regions.

Dataset	Method	Base GS	Aux. GS $\downarrow$	Reduction $\uparrow$	DISTS $\downarrow$	LPIPS $\downarrow$
Middlebury	Dense grid init	1,179,648	414,018.78	–	0.0874	0.3674
	Mask guided init		330.65	99.92%	0.0874	0.3674
Booster	Dense grid init		431,616.00	–	0.0871	0.3075
	Mask guided init		144.17	99.97%	0.0871	0.3075
WildRGBD	Dense grid init		76,800.00	–	0.0450	0.2002
	Mask guided init		91.11	99.88%	0.0451	0.2004

Appendix

Table 5: Ablation on disocclusion guided sparse initialization across evaluation datasets. Compared with dense grid initialization, disocclusion guided initialization reduces the number of initialized auxiliary Gaussians while maintaining rendering quality. Base GS denotes the fixed number of Gaussians predicted by SHARP, and Aux. GS denotes the average number of initialized auxiliary Gaussians per novel view.

Dataset	Setting	Added GS	Added GS (%)	Merged GS	SHARP Time	Spackle Time	Time Overhead	SHARP FPS	Spackle FPS	$n$
Middlebury	S1	3,387	0.29	1,183,035	6410.3	1449.4	-3.3%	0.671	0.691	23
	S2	8,501	0.72	1,188,149	1448.8	1452.7	0.8%	0.695	0.689	23
	S3	15,782	1.34	1,195,430	1448.4	1476.4	2.5%	0.695	0.679	23
Booster	S1	1,074	0.09	1,180,722	1427.8	1451.4	1.9%	0.703	0.691	228
	S2	3,077	0.26	1,182,725	1427.6	1456.1	2.3%	0.703	0.689	228
	S3	7,226	0.61	1,186,874	1427.9	1471.6	3.3%	0.703	0.682	228

Appendix

Table 6: Expanded efficiency and Gaussian overhead analysis under controlled large view settings. SHARP uses a fixed base set of 1,179,648 Gaussians. Spackle adds auxiliary Gaussians according to the disocclusion regions. The parameter overhead is constant across settings: SHARP has 702.31M parameters, while Spackle has 710.32M parameters, introducing 8.02M additional parameters, corresponding to 1.14%. Time is measured in milliseconds, and FPS denotes rendering throughput. A negative time overhead indicates that Spackle is faster than SHARP in the corresponding aggregated entry, which can occur due to runtime measurement outliers.

Figure 4: Additional qualitative analysis of directly applying generated supervision to the baseline 3DGS model under large viewpoint changes. From left to right, each row shows the input image, the SHARP rendering, our Spackle rendering, the rendering from the naive decoupled framework, and the detected occluded regions. The naive decoupled framework can reduce some missing regions, but it often introduces noticeable appearance degradation in already reliable areas. In contrast, Spackle keeps the baseline 3DGS model frozen and introduces augmented Gaussians for newly disoccluded regions, preserving the main scene appearance while improving hole completion.

Novel View Synthesis

Novel View Synthesis creates images of a scene from camera viewpoints not present in the input, using learned geometry and appearance models.

See all 289 papers

Rethinking Sparse-View 3D Gaussian Splatting: A Few-Shot Learning Perspective on Overfitting Prevention
Oct 19, 2026Lejia Ye, Bin Wu, Jingjing Gao +2Novel View Synthesis Sparse-View 3D Reconstruction
School of Aeronautics and Astronautics, Sun Yat-sen University, China
SplatGuide: Geometric Priors from 3D Gaussians for Pose-Free Novel View Synthesis
Aug 17, 2026Yejun Zhang, Zihan Wang, Xu Ji +8Multi-View Diffusion Models Novel View Synthesis
Aalto University, Finland · Deep Render, UK · ELLIS Institute Finland +3
ViewSplat: View-Adaptive 3D Gaussian Splatting for Feed-Forward Synthesis
Mar 26, 2026Moonyeon Jeong, Seunggi Min, Suhyeon Lee +13D Gaussian Splatting Novel View Synthesis
University of Seoul · Korea Electronics Technology Institute

Oct 19, 2026J/K move · Enter open · S save

Rethinking Sparse-View 3D Gaussian Splatting: A Few-Shot Learning Perspective on Overfitting Prevention

Lejia Ye, Bin Wu, Jingjing Gao, Yiyang Wu+1

School of Aeronautics and Astronautics, Sun Yat-sen University, China

Novel View Synthesis Sparse-View 3D Reconstruction 3DGS Training Few-Shot Learning

Open paper AskPDF arXiv

Sparse-view novel view synthesis (NVS) is essential for AR/VR and real-time communication applications, yet remains challenging when reconstructing 3D scenes from limited viewpoints. While 3D Gaussian Splatting (3DGS) excels in dense-view scenarios, it suffers from overfitting and geometry degradation under sparse inputs. Current approaches often tackle these problems from a local perspective, addressing either data quality or training regularization individually, lacking a unified and interpretable framework for systematic improvement. We introduce a new perspective that reframes sparse-view 3DGS from a few-shot learning viewpoint, treating each Gaussian primitive as a neuron and leveraging deep learning regularization techniques. Under this unified framework, our approach generates more effective supervision from scarce observations through two strategies: (1) data-related supervision enrichment via optical flow-guided pseudo-view synthesis and joint camera-scene optimization; and (2) training-time regularization via motion-aware dropout and early stopping, which suppress overfitting while preserving dynamic-scene stability. Extensive experiments on the SIGGRAPH Asia 2025 volumetric video challenge sparse-view dataset demonstrate significant improvements over state-of-the-art optimization-based and feed-forward baselines. Our system won first place in the sparse-view track of the SIGGRAPH Asia 2025 challenge.

Abstract

Novel View Synthesis

Explore similar work

Rethinking Sparse-View 3D Gaussian Splatting: A Few-Shot Learning Perspective on Overfitting Prevention

SplatGuide: Geometric Priors from 3D Gaussians for Pose-Free Novel View Synthesis

ViewSplat: View-Adaptive 3D Gaussian Splatting for Feed-Forward Synthesis