ArtifactWorld: Scaling 3D Gaussian Splatting Artifact Restoration via Video Generation Models

Xinliang Wang; Yifeng Shi; Zhenyu Wu

cs.CVApr 14, 2026

ArtifactWorld: Scaling 3D Gaussian Splatting Artifact Restoration via Video Generation Models

Authors: Xinliang Wang, Yifeng Shi, Zhenyu Wu

Organizations: Ke Holdings Inc. Beijing, China

Abstract

3D Gaussian Splatting (3DGS) delivers high-fidelity real-time rendering but suffers from geometric and photometric degradations under sparse-view constraints. Current generative restoration approaches are often limited by insufficient temporal coherence, a lack of explicit spatial constraints, and a lack of large-scale training data, resulting in multi-view inconsistencies, erroneous geometric hallucinations, and limited generalization to diverse real-world artifact distributions. In this paper, we present ArtifactWorld, a framework that resolves 3DGS artifact repair through systematic data expansion and a homogeneous dual-model paradigm. To address the data bottleneck, we establish a fine-grained phenomenological taxonomy of 3DGS artifacts and construct a comprehensive training set of 107.5K diverse paired video clips to enhance model robustness. Architecturally, we unify the restoration process within a video diffusion backbone, utilizing an isomorphic predictor to localize structural defects via an artifact heatmap. This heatmap then guides the restoration through an Artifact-Aware Triplet Fusion mechanism, enabling precise, intensity-guided spatio-temporal repair within native self-attention. Extensive experiments demonstrate that ArtifactWorld achieves state-of-the-art performance in sparse novel view synthesis and robust 3D reconstruction. Code available at: https://github.com/fyting/ArtifactWorld.

Figures & tables

Figure 1. ArtifactWorld effectively resolves complex 3D Gaussian Splatting degradations under sparse-view constraints. We systematically categorize these failures into nine fine-grained phenomenological artifact types (Sec. 3 ). Compared to recent generative restoration pipelines (e.g., Difix3D ( Wu et al., 2025a ) , GSFixer ( Yin et al., 2025b ) ), our large-scale data scaling and homogeneous video diffusion framework ensure spatio-temporally consistent, high-fidelity restoration.

Figure 2. Overview of ArtifactWorld Data Engine. (a) Our phenomenological taxonomy categorizes 3DGS sparse-view degradations into 4 domains and 9 artifact types. (b) The data curation pipeline extracts 16K pristine scenes and physically simulates 25.6K paired videos, splitting them into a 1.28K benchmark and 24.3K training data. (c) The generative data flywheel leverages a VLM and an Artifact World Model to ultimately produce 107.5K training pairs.

Figure 3. ArtifactWorld Framework. Under a Homogeneous Dual-Model Paradigm within a diffusion transformer: (1) DBA extracts clean boundary references. (2) A predictor (optimized and inferred via LoRA1 ) generates artifact heatmaps. (3) A restoration model (optimized and inferred via LoRA2 ) leverages these heatmaps to synthesize high-fidelity frames via AATF . (4) Closed-loop optimization uses these frames to permanently eliminate 3DGS artifacts.

Figure 4. Qualitative comparison of sparse-view 3D reconstruction. Across varying sparsity ratios (5%, 10%, 15%), ArtifactWorld consistently outperforms existing baselines by generating high-fidelity details closest to the ground truth.

Dataset	Method	PSNR $\uparrow$			SSIM $\uparrow$			LPIPS $\downarrow$
Dataset	Method	5%	10%	15%	5%	10%	15%	5%	10%	15%
DL3DV	3DGS ( Kerbl et al., 2023 )	22.75	26.17	27.76	0.7737	0.8583	0.8880	0.2247	0.1597	0.1372
	Difix3D ∗ ( Wu et al., 2025a )	23.53	26.57	27.96	0.7885	0.8636	0.8913	0.2080	0.1520	0.1315
	GSFixer ( Yin et al., 2025b )	23.44	26.45	27.93	0.7853	0.8615	0.8898	0.2161	0.1552	0.1334
	ArtifactWorld (Ours)	24.56	27.14	28.27	0.8111	0.8722	0.8942	0.1970	0.1498	0.1341
Mip-NeRF-360	3DGS ( Kerbl et al., 2023 )	17.08	20.53	22.35	0.4533	0.5860	0.6440	0.4246	0.3192	0.2811

Table 1. Quantitative comparison on DL3DV and Mip-NeRF 360 datasets under different training view ratios. Best and second-best results are highlighted in bold and underline , respectively. ∗ denotes using the same camera trajectory as GSFixer.

Figure 5. Qualitative comparison of 2D artifact restoration. Across sampled non-consecutive frames ( $T_{a}<T_{b}<T_{c}$ ), ArtifactWorld visibly outperforms existing restoration baselines.

Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
SimpleNeRF ( Somraj et al., 2023 )	13.27	0.283	0.741
ZeroNVS ( Sargent et al., 2024 )	14.44	0.316	0.680
ReconFusion ( Wu et al., 2024b )	15.50	0.358	0.585
3DGS ( Kerbl et al., 2023 )	13.06	0.251	0.576
FSGS ( Zhu et al., 2024 )	14.17	0.318	0.578
Difix3D ( Wu et al., 2025a )	13.92	0.298	0.578

Table 2. Performance under extreme 3-view sparsity.

Methods	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	CLIP-I $\uparrow$	FVD $\downarrow$
DOVE ( Chen et al., 2025 )	20.12	0.6529	0.4062	0.8423	174.05
SeedVR2 ( Wang et al., 2025b )	17.58	0.5840	0.4271	0.8232	197.94
Difix3D ( Wu et al., 2025a )	20.21	0.6442	0.3617	0.8925	66.76
GSFixer ( Yin et al., 2025b )	20.19	0.6769	0.3834	0.8681	114.53
ArtifactWorld (Ours)	24.20	0.7547	0.2893	0.9663	33.92

Table 3. Quantitative comparison of 2D artifact restoration.

DBA	AATF	Data Scaling	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
$\times$	$\times$	$\times$	23.44	0.7330	0.3118
$\checkmark$	$\times$	$\times$	23.99	0.7492	0.2948
$\checkmark$	$\checkmark$	$\times$	24.15	0.7530	0.2900
$\checkmark$	$\checkmark$	$\checkmark$	24.46	0.7572	0.2881

Table 4. Ablation study of our proposed components.

Figure 6. Mechanistic Interpretability. $T_{\text{start}}$ , $T_{x}$ , and $T_{\text{end}}$ denote the initial, intermediate, and terminal frames, respectively. $\text{Frame}_{\text{ref}}$ , $\text{Frame}_{\text{target}}$ (visualized as the generated frame for clarity), and $\text{Frame}_{\text{hm}}$ represent the reference frame, target frame, and artifact heatmap, corresponding to latents $z_{\text{ref}}$ , $z_{t}^{\text{target}}$ , and $z_{\text{heatmap}}$ . $\text{Heatmap}_{\text{ref}}$ and $\text{Heatmap}_{\text{hm}}$ visualize the spatial attention of the query token towards $z_{\text{ref}}$ and $z_{\text{heatmap}}$ , respectively. (Zoom in for details.)

Exp	Timestep Scheduling Strategy (Total: 8 Steps)			PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Exp	Phase 1 ( $t\geq\tau_{2}$ )	Phase 2 ( $\tau_{1}\leq t<\tau_{2}$ )	Phase 3 ( $t<\tau_{1}$ )	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
1	Constant $z_{heatmap}$ (8 steps)			24.428	0.7565	0.2885
2	$z_{full}$ (1 step)	$z_{heatmap}$ (7 steps)		24.433	0.7565	0.2883
3	$z_{heatmap}$ (7 steps)		$z_{null}$ (1 step)	24.435	0.7566	0.2884
4	$z_{full}$ (1 step)	$z_{heatmap}$ (6 steps)	$z_{null}$ (1 step)	24.439	0.7567	0.2882
5	$z_{null}$ (1 step)	$z_{heatmap}$ (6 steps)	$z_{full}$ (1 step)	24.399	0.7566	0.2884

Table 5. Ablation on dynamic timestep scheduling. Evaluated across an 8-step denoising process.

Method	Time (min) $\downarrow$	Memory (GB) $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$
Difix3D ( Wu et al., 2025a )	27.8	13.6	20.80	0.5818
GSFixer ( Yin et al., 2025b )	146.4	50.8	21.03	0.5855
ArtifactWorld	113.3	46.3	22.11	0.6081
ArtifactWorld-Turbo	48.8	41.6	21.95	0.6002

Table 6. Efficiency and restoration quality of generative reconstruction methods.

Explore similar work

GeoQuery: Geometry-Query Diffusion for Sparse-View Reconstruction
May 12, 2026Xiao Cao, Yuze Li, Youmin Zhang +4Novel View Synthesis 3D Reconstruction
University of Electronic Science and Technology of China & Rawmantic AI, China · Rawmantic AI, China · Tianjin University, China +1
SparseGS: Sparse View Synthesis using 3D Gaussian Splatting
Nov 30, 2023Haolin Xiong, Sairisheek Muttukuru, Hanyuan Xiao +4Novel View Synthesis Synthesis
University of California, Los Angeles · USC Institute for Creative Technologies · University of Southern California
Sparse-to-Complete: From Sparse Image Captures to Complete 3D Scenes
May 7, 2026Yiyang Shen, Yin Yang, Kun Zhou +13D Reconstruction 3D Generation
State Key Lab of CAD&CG, Zhejiang University, China · University of Utah, USA · State Key Lab of CAD&CG, Zhejiang University, China and Hangzhou Research Institute of Holographic and AI Technology, China +1

May 12, 2026, cs.CVJ/K move · Enter open · S save

GeoQuery: Geometry-Query Diffusion for Sparse-View Reconstruction

Xiao Cao, Yuze Li, Youmin Zhang, Jiayu Song+3

University of Electronic Science and Technology of China & Rawmantic AI, China · Rawmantic AI, China · Tianjin University, China+1

Novel View Synthesis 3D Reconstruction Cross-View Diffusion Models Geogs-Slam

Open paper AskPDF arXiv

3D Gaussian Splatting (3DGS) has emerged as a prominent paradigm for 3D reconstruction and novel view synthesis. However, it remains vulnerable to severe artifacts when trained under sparse-view constraints. While recent methods attempt to rectify artifacts in rendered views using image diffusion models, they typically rely on multi-view self-attention to retrieve information from reference images. We observe that this mechanism often fails when the rendered novel views output by 3DGS are heavily corrupted: damaged query features lead to erroneous cross-view retrieval, resulting in inconsistent rendering refinement. To address this, we propose GeoQuery, a geometry-guided diffusion framework that integrates generative priors with explicit geometric cues via a novel Geometry-guided Cross-view Attention (GCA) mechanism. First, by leveraging predicted depth maps and camera poses, we construct a geometry-induced correspondence field to sample reference features, forming a geometry-aligned proxy query that replaces the corrupted rendering features. Furthermore, we design a new cross-view feature aggregation pipeline, in which we restrict the cross-view attention to a local window around each proxy query to effectively retrieve useful features while suppressing spurious matches. GeoQuery can be seamlessly integrated into existing diffusion-based pipelines, enabling robust reconstruction even under extreme view sparsity. Extensive experiments on sparse-view novel view synthesis and rendering artifact removal demonstrate the effectiveness of our approach.

Abstract

Explore similar work

GeoQuery: Geometry-Query Diffusion for Sparse-View Reconstruction

SparseGS: Sparse View Synthesis using 3D Gaussian Splatting

Sparse-to-Complete: From Sparse Image Captures to Complete 3D Scenes