GaussFusion: Improving 3D Reconstruction in the Wild with A Geometry-Informed Video Generator

Liyuan Zhu; Manjunath Narayana; Michal Stary; Will Hutchcroft; Gordon Wetzstein; Iro Armeni

cs.CVMar 26, 2026

GaussFusion: Improving 3D Reconstruction in the Wild with A Geometry-Informed Video Generator

Authors: Liyuan Zhu, Manjunath Narayana, Michal Stary, Will Hutchcroft, Gordon Wetzstein, Iro Armeni

Organizations: Stanford University · Zillow Group

Abstract

We present GaussFusion, a novel approach for improving 3D Gaussian splatting (3DGS) reconstructions in the wild through geometry-informed video generation. GaussFusion mitigates common 3DGS artifacts, including floaters, flickering, and blur caused by camera pose errors, incomplete coverage, and noisy geometry initialization. Unlike prior RGB-based approaches limited to a single reconstruction pipeline, our method introduces a geometry-informed video-to-video generator that refines 3DGS renderings across both optimization-based and feed-forward methods. Given an existing reconstruction, we render a Gaussian primitive video buffer encoding depth, normals, opacity, and covariance, which the generator refines to produce temporally coherent, artifact-free frames. We further introduce an artifact synthesis pipeline that simulates diverse degradation patterns, ensuring robustness and generalization. GaussFusion achieves state-of-the-art performance on novel-view synthesis benchmarks, and an efficient variant runs in real time at 15 FPS while maintaining similar performance, enabling interactive 3D applications.

Figures & tables

		DL3DV				RE10K
Method	Venue	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$	Infer. Speed
Splatfacto [ 63 ]	SIGG. 2023	17.417	0.605	0.412	6.488	19.234	0.708	0.457	12.623	(118.3 FPS)
GenFusion [ 59 ]	CVPR 2025	18.363	0.690	0.391	9.981	20.618	0.796	0.344	14.747	1.1 FPS
Difix3D+ [ 57 ]	CVPR 2025	20.095	0.765	0.302	4.217	24.347	0.868	0.275	13.994	12.8 FPS
ExploreGS [ 27 ]	ICCV 2025	20.689	0.760	0.345	6.269	24.025	0.874	0.272	15.629	1.2 FPS
Ours (Single)	CVPR 2026	22.266	0.827	0.282	4.171	24.754	0.897	0.215	7.496	4.3 FPS

Table 1: Rendering Refinement Performance on DL3DV and RE10K Datasets. We compare our method against state-of-the-art 3DGS refinement approaches. The joint training variant achieves the best overall fidelity and perceptual quality (highest PSNR/SSIM, lowest LPIPS/FID), while the distilled model attains comparable performance with significantly improved runtime efficiency, reaching 15 FPS.

Feed-forward Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$
DepthSplat [ 60 ]	21.773	0.842	0.278	28.903
w/ Difix3D+ [ 57 ]	21.415	0.837	0.303	17.968
w/ ExploreGS [ 27 ]	21.195	0.816	0.317	35.475
w/ GaussFusion (Ours)	22.802	0.848	0.258	15.128
MVSplat [ 7 ]	18.744	0.706	0.375	44.479
w/ Difix3D+ [ 57 ]	18.261	0.702	0.385	23.954

Table 2: Feed-forward View Synthesis Results on RE10K [ 74 ] . Our method consistently improves feed-forward 3DGS reconstruction on different backbones while baseline methods Difix3D and ExploreGS decrease the consistency with the original scene. MVSplat360 is on par to our method in terms of consistency within its specialized domain, but still lacks in visual quality.

	DL3DV			RE10K
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Splatfacto [ 63 ]	17.42	0.605	0.412	19.23	0.708	0.457
GenFusion [ 59 ]	18.08	0.615	0.421	19.52	0.719	0.329
Difix3D+ [ 57 ]	18.87	0.626	0.265	21.98	0.767	0.187
ExploreGS [ 27 ]	18.71	0.611	0.377	22.06	0.742	0.257
GaussFusion (Ours)	19.36	0.628	0.279	24.17	0.817	0.175

Table 3: Performance on Improving 3D Reconstruction . GaussFusion demonstrates superior performance on most metrics, showcasing the multi-view consistency in our enhanced frames.


Splatfacto [ 63 ]	GenFusion [ 59 ]	Difix3D+ [ 57 ]	ExploreGS [ 27 ]	GaussFusion (Ours)	Ground Truth

DepthSplat [ 60 ]	MVSplat360 [ 8 ]	Difix3D+ [ 57 ]	ExploreGS [ 27 ]	GaussFusion (Ours)	Ground Truth

Figure 3: Qualitative Comparison on Novel-View Refinement. We compare GaussFusion with baseline methods on diverse scenes from DL3DV [ 33 ] and RE10K [ 74 ] . GaussFusion effectively removes rendering artifacts such as blur, floaters, ghosting, and texture distortions, producing sharper geometry, cleaner reconstruction than Splatfacto [ 63 ] , GenFusion [ 59 ] , DiFiX3D+ [ 57 ] , and ExploreGS [ 27 ] , and achieving visual fidelity closest to the ground truth. From top to bottom, our model demonstrates strong generalization across various refinement scenarios: ( 1 ) inpainting missing regions, ( 2 ) outpainting beyond the original view, ( 3–4 ) correcting blur, ghosting, and geometric errors, and ( 5–6 ) handling artifacts from feed-forward 3DGS models, including transparent Gaussian primitives and spiky geometry.

Input Modality					Performance
RGB	Depth	Normal	Alpha	Cov.	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$
✓					19.148	0.719	0.385	15.451
✓	✓				19.289	0.726	0.361	10.542
✓	✓	✓			19.744	0.738	0.355	10.291
✓	✓	✓	✓		19.958	0.748	0.344	8.612
✓	✓	✓	✓	✓	20.753	0.773	0.329	6.724

Table 4: Ablation on Input Modalities on DL3DV Dataset. Inclusion of more geometric cues (depth, normal, alpha, covariance) improves reconstruction fidelity and perceptual quality.

Training Config.		Performance
Artifact Sim.	Architecture	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$
Baseline	w/. GA	21.087	0.789	0.303	5.717
Ours	w/o. GA	20.904	0.794	0.308	5.296
Ours	w/. GA	22.548	0.832	0.278	3.933

Table 5: Ablation on Artifact Simulation and Architecture. Evaluating the impact of artifact simulation and geometry adapter block on reconstruction fidelity and perceptual quality.

Modality	# Channel	MAE $\downarrow$	MSE $\downarrow$	RelErr $\downarrow$	SqRel $\downarrow$
RGB	3	0.0323	0.00362	0.0199	0.0217
Depth	1	0.0253	0.00252	0.0044	0.0182
Alpha	1	0.0138	0.00150	0.0075	0.0053
Covar.	3	0.0758	0.01048	0.0147	0.0847
Normal	3	0.1890	0.06684	0.1333	0.3507

Table E.1 : Per-Modality VAE Reconstruction Error. Comparison of mean absolute error (MAE), mean squared error (MSE), relative error (RelErr), and squared relative error (SqRel).

Feature	Optimization-Based 3DGS	Feed-Forward 3DGS
Primary visual flaw	High-frequency noise (Floaters)	Low-frequency blur (Oversmoothing)
Geometry artifacts	”Needles” (over-stretched covariance)	Global depth warping / Scale ambiguity
Unseen areas	Empty / Holes	Plausible but blurry (learned priors)
Source of error	Overfitting (high variance)	Uncertainty / Regression to mean

Table F.1 : Empirical Analysis of Artifacts in 3DGS.


Splatfacto [ 63 ]	GenFusion [ 59 ]	Difix3D+ [ 57 ]	ExploreGS [ 27 ]	GaussFusion (Ours)	Ground Truth

DepthSplat [ 60 ]	MVSplat360 [ 8 ]	Difix3D+ [ 57 ]	ExploreGS [ 27 ]	GaussFusion (Ours)	Ground Truth

Figure I.2 : Additional Qualitative Comparison on Novel-view Refinement on DL3DV [ 33 ] and RE10K [ 74 ] . Top four rows showcase 3DGS optimization methods and bottom four rows feed-forward 3DGS methods as starting point of the refinement.


GenFusion [ 59 ]	Difix3D+ [ 57 ]	ExploreGS [ 27 ]	GaussFusion (Ours)	Ground Truth

Figure I.3 : Additional Qualitative Comparison on 3D Reconstruction on DL3DV [ 33 ] and RE10K [ 74 ] . GaussFusion showcases sharper and cleaner appearance and geometry compared to baseline methods.

3D Gaussian Splatting

3D Gaussian Splatting represents scenes as optimizable 3D Gaussian primitives for fast, photorealistic rendering from new viewpoints.

See all 986 papers

ArtifactWorld: Scaling 3D Gaussian Splatting Artifact Restoration via Video Generation Models
Apr 14, 2026Xinliang Wang, Yifeng Shi, Zhenyu Wu3D Gaussian Splatting Novel View Synthesis
Ke Holdings Inc. Beijing, China
Bi-FlowGS: Bridging Generative View Completion and Gaussian Geometry through Bidirectional Flow Co-Refinement
Sep 15, 2026Yuetong Wang, Jinsheng Quan, Yi Yang +13D Gaussian Splatting 3D Reconstruction
Zhejiang University
GaussVid: Sparse-View Gaussian Splatting with 3D-Aware Video Diffusion Priors
Aug 22, 2026Xinhui Liu, Can Wang, Wei Jiang +2Sparse-View 3D Reconstruction 3D Gaussian Splatting
School of Computing and Data Science, The University of Hong Kong, Hong Kong, China · Futurewei Technologies Inc, Santa Clara, CA, USA

Apr 14, 2026J/K move · Enter open · S save

ArtifactWorld: Scaling 3D Gaussian Splatting Artifact Restoration via Video Generation Models

Xinliang Wang, Yifeng Shi, Zhenyu Wu

Ke Holdings Inc. Beijing, China

3D Gaussian Splatting Novel View Synthesis Sparse-View 3D Reconstruction

Open paper AskPDF arXiv

3D Gaussian Splatting (3DGS) delivers high-fidelity real-time rendering but suffers from geometric and photometric degradations under sparse-view constraints. Current generative restoration approaches are often limited by insufficient temporal coherence, a lack of explicit spatial constraints, and a lack of large-scale training data, resulting in multi-view inconsistencies, erroneous geometric hallucinations, and limited generalization to diverse real-world artifact distributions. In this paper, we present ArtifactWorld, a framework that resolves 3DGS artifact repair through systematic data expansion and a homogeneous dual-model paradigm. To address the data bottleneck, we establish a fine-grained phenomenological taxonomy of 3DGS artifacts and construct a comprehensive training set of 107.5K diverse paired video clips to enhance model robustness. Architecturally, we unify the restoration process within a video diffusion backbone, utilizing an isomorphic predictor to localize structural defects via an artifact heatmap. This heatmap then guides the restoration through an Artifact-Aware Triplet Fusion mechanism, enabling precise, intensity-guided spatio-temporal repair within native self-attention. Extensive experiments demonstrate that ArtifactWorld achieves state-of-the-art performance in sparse novel view synthesis and robust 3D reconstruction. Code available at: https://github.com/fyting/ArtifactWorld.

Abstract

3D Gaussian Splatting

Explore similar work

ArtifactWorld: Scaling 3D Gaussian Splatting Artifact Restoration via Video Generation Models

Bi-FlowGS: Bridging Generative View Completion and Gaussian Geometry through Bidirectional Flow Co-Refinement

GaussVid: Sparse-View Gaussian Splatting with 3D-Aware Video Diffusion Priors