ArtiFixer: Enhancing and Extending 3D Reconstruction with Auto-Regressive Diffusion Models

Riccardo de Lutio; Tobias Fischer; Yen-Yu Chang; Yuxuan Zhang; Jay Zhangjie Wu; Xuanchi Ren; Tianchang Shen; Katarina Tothova

cs.CVFeb 28, 2026

ArtiFixer: Enhancing and Extending 3D Reconstruction with Auto-Regressive Diffusion Models

Authors: Riccardo de Lutio, Tobias Fischer, Yen-Yu Chang, Yuxuan Zhang, Jay Zhangjie Wu, Xuanchi Ren, Tianchang Shen, Katarina Tothova, +2 more

Organizations: NVIDIA, USA · NVIDIA, Switzerland · ETHZ, Switzerland · Cornell University, USA · NVIDIA, Canada · University of Toronto, Canada · Vector Institute, Canada

Abstract

Per-scene optimization methods such as 3D Gaussian Splatting provide state-of-the-art novel view synthesis quality but extrapolate poorly to under-observed areas. Methods that leverage generative priors to correct artifacts in these areas hold promise but currently suffer from two shortcomings. The first is scalability, as existing methods use image diffusion models or bidirectional video models that are limited in the number of views they can generate in a single pass (and thus require a costly iterative distillation process for consistency). The second is quality itself, as generators used in prior work tend to produce outputs that are inconsistent with existing scene content and fail entirely in completely unobserved regions. To solve these, we propose a two-stage pipeline that leverages two key insights. First, we train a powerful bidirectional generative model with a novel opacity mixing strategy that encourages consistency with existing observations while retaining the model's ability to extrapolate novel content in unseen areas. Second, we distill it into a causal auto-regressive model that generates hundreds of frames in a single pass. This model can directly produce novel views or serve as pseudo-supervision to improve the underlying 3D representation in a simple and highly efficient manner. We evaluate our method extensively and demonstrate that it can generate plausible reconstructions in scenarios where existing approaches fail completely. When measured on commonly benchmarked datasets, we outperform all existing baselines by a wide margin, exceeding prior state-of-the-art methods by 1-3 dB PSNR.

Figures & tables

Figure 1. ArtiFixer enhances and extends existing 3D reconstructions in a highly efficient and scalable manner. Given an initial reconstruction and optional reference views and text prompt, it auto-regressively generates novel content that maintains a high degree of consistency with existing observations. ArtiFixer can directly produce hundreds of novel views in a single inference pass or serve as pseudo-supervision to improve the underlying 3D reconstruction. Project page: https://research.nvidia.com/labs/sil/projects/artifixer

Figure 2. Method overview. We first train a bidirectional flow matching model that transports degraded RGB renderings into clean outputs. We encode the input RGB into latent space and mix with Gaussian noise using the rendered opacity maps to avoid mode collapse in unseen regions. We inject fine-grained opacity information and camera control along with optional clean reference views and a text prompt. In the second phase of our pipeline, we distill the teacher into an auto-regressive causal model via Self Forcing-style DMD distillation ( Huang et al., 2025 ) , which can be directly used to render novel views or used as pseudo-supervision to distill back into the underlying 3D representation.

Figure 3. Transformer block. We start from a pretrained text-to-video model ( Wan et al., 2025b ) and inject camera and opacity information into each transformer block via linear layers after applying self-attention and layer normalization. We patchify reference views into visual tokens, apply relative camera conditioning via PRoPE ( Li et al., 2025a ) , and add

K_{n}

and

V_{n}

projections to the cross-attention operation. We zero-initialize

f_{r}

f_{o}

, and

V_{n}

to ensure compatibility with the pretrained initialization.

Figure 4. Opacity mixing. Given a degraded rendering and optional reference views and text prompt ( left ), we predict an artifact-free rendering at a target viewpoint. Starting from Gaussian noise and channel concatenating the degraded rendering as in prior work ( Wu et al., 2025c ; Yin et al., 2025b ) produces renderings that are semantically similar to the reference views, but with notable inconsistencies (such as the table in the top row ). Directly starting from the degraded rendering instead of Gaussian noise improves consistency, but degrades quality noticeably when extrapolating to areas outside those covered by the degraded renderings ( bottom row ). Instead, we mix Gaussian noise into the rendering based on its opacity map. The resulting input retains the consistency benefits of the original while enabling a strong generative capability in entirely novel regions.

	Nerfbusters ( Warburg et al., 2023 )				DL3DV ( Ling et al., 2024 )
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$
Nerfacto ( Tancik et al., 2023 )	17.29	0.621	0.402	134.65	17.16	0.581	0.430	112.30
3DGS ( Kerbl et al., 2023 )	17.66	0.678	0.327	113.84	17.18	0.588	0.384	107.23
Nerfbusters ( Warburg et al., 2023 )	17.72	0.647	0.352	116.83	17.45	0.606	0.370	96.61
GANeRF ( Roessle et al., 2023 )	17.42	0.611	0.354	115.60	17.54	0.610	0.342	81.44
NeRFLiX ( Zhou et al., 2023 )	17.91	0.656	0.346	113.59	17.56	0.610	0.359	80.65

Table 1. Artifact removal on Nerfbusters and DL3DV . All ArtiFixer variants outperform prior methods by a considerable margin, improving PSNR by 2 dB.

	PSNR $\uparrow$			SSIM $\uparrow$			LPIPS $\downarrow$
Method	3-view	6-view	9-view	3-view	6-view	9-view	3-view	6-view	9-view
Zip-NeRF ( Barron et al., 2023 )	12.77	13.61	14.30	0.271	0.284	0.312	0.705	0.663	0.633
3DGS ( Kerbl et al., 2023 )	13.06	14.96	16.79	0.251	0.355	0.447	0.576	0.505	0.446
2DGS ( Huang et al., 2024 )	13.07	15.02	16.67	0.243	0.338	0.423	0.580	0.506	0.449
FSGS ( Zhu et al., 2024 )	14.17	16.12	17.94	0.318	0.415	0.492	0.578	0.517	0.468
FreeNeRF ( Yang et al., 2023 )	12.87	13.35	14.59	0.260	0.283	0.319	0.715	0.717	0.695

Table 2. Sparse view reconstruction methods on the Mip-NeRF 360 dataset. We exceed existing work by a wide margin across every metric.

Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$
3DGUT ( Wu et al., 2025b )	16.12	0.537	0.445	92.94
Difix3D (Nerfacto) ( Wu et al., 2025a )	14.16	0.453	0.545	74.59
Difix3D (3DGS) ( Wu et al., 2025a )	16.60	0.599	0.405	52.70
Difix3D+ (Nerfacto) ( Wu et al., 2025a )	13.74	0.434	0.483	30.07
Difix3D+ (3DGS) ( Wu et al., 2025a )	16.34	0.564	0.382	21.77
Fixer (offline) ( NVIDIA, 2025 )	13.09	0.355	0.584	135.43

Table 3. Novel content generation. We reconstruct DL3DV scenes following a protocol that creates large areas unobserved by training views. We outperform the next-best method (GenFusion) by almost 3 dB in PSNR.

Figure 5. ArtiFixer variants. Most visible differences occur in highly corrupted regions. ArtiFixer3D ’s explicit 3D consistency improves fidelity with the source images and mitigates transient corruption ( middle ), at the cost of some sharpness, which ArtiFixer3D+ restores. Nonetheless, all variants outperform prior work by a substantial margin.

Method	Direct Input	Opacity Mixing	Diffusion Forcing	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$
Channel Concatenation	✗	✗	✓	14.52	0.391	0.490	87.551
w/o Opacity Mixing	✓	✗	✓	17.34	0.440	0.429	87.058
w/o Initialization	✓	✓	✗	17.58	0.450	0.416	74.924
Full Method	✓	✓	✓	17.99	0.461	0.408	69.43

Table 4. Diagnostics. We evaluate reconstruction quality on Mip-NeRF 360. Denoising input renderings instead of conditioning via channel concatenation is crucial to producing outputs consistent with source images.

Figure 6. Reference views. Without the initial rendering condition, ArtiFixer can generate predictions from the reference views. Although fidelity drops somewhat, the high-level structure of the scene remains intact.

Figure 7. Text-to-video generation. To illustrate our model’s generative ability, we generate videos from text prompts alone. With opacity mixing, it retains similar quality to its base model ( Wan et al., 2025b ) .

Method	FPS $\uparrow$
Wan 2.1 T2V-14B ( Wan et al., 2025b )	0.12
Wan 2.1 T2V-1.3B ( Wan et al., 2025b )	0.49
ArtiFixer / ArtiFixer3D+ (14B)	8.36
ArtiFixer / ArtiFixer3D+ (1.3B)	34.38
ArtiFixer3D	268

Table 5. Inference speed. Causal distillation yields a

70\times

speedup over the bidirectional Wan 2.1 backbones. ArtiFixer3D renders directly from 3DGUT. Additional configurations are reported in Table 7 .

Figure 8. DL3DV results . We compare ArtiFixer3D+ to its initial 3DGUT ( Wu et al., 2025b ) input, two baselines that build upon bidirectional video diffusion models ( top rows ), and two that leverage image models ( bottom rows ). GenFusion ( Wu et al., 2025c ) ’s video model generates 16 frames at a time, requiring an iterative distillation process that leads to blurry results, especially in empty areas. Gen3C ( Ren et al., 2025 ) ’s renderings are sharper but often do not respect the source content (background in top row ), have incorrect geometry ( second row ), and exhibit color shift ( sixth row ). Methods that directly take renderings as input without opacity mixing ( Wu et al., 2025a ; NVIDIA, 2025 ) fail to reconstruct empty regions. Our method can reconstruct plausible and consistent geometry even when the initial rendering is highly degraded. Please refer to our project website for comparison videos.

Figure 9. Mip-NeRF 360 results . We present visualizations of Mip-NeRF’s most challenging split (3-view). Our results far exceed all prior work both quantitatively and qualitatively. Our method is able to recover the correct geometry from the reference views even in scenarios where the input rendering is completely inaccurate (table in third row ).

Figure 10. Nerfbusters results . As with the other datasets, our method is the only one to generate plausible visuals in unseen regions while preserving the fidelity of the original content.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	$\Delta$ PSNR	$\Delta$ SSIM	$\Delta$ LPIPS
Mip-NeRF 360 (3 views)	+0.14	+0.003	$-$ 0.002
Mip-NeRF 360 (6 views)	+0.07	+0.002	$-$ 0.001
Mip-NeRF 360 (9 views)	+0.03	+0.003	$-$ 0.001
DL3DV	+0.02	0.000	$-$ 0.001
Nerfbusters	$-$ 0.07	+0.001	0.000

Appendix

Table 6. Text conditioning. We measure the impact of VLM-generated prompts vs. no prompt for ArtiFixer3D+ . Text prompts provide a small benefit in sparse settings that diminishes with denser captures.

		FPS $\uparrow$
Method	GPUs	1 step	2 steps	3 steps	4 steps
ArtiFixer (14B)	1	29.42	16.07	11.03	8.36
ArtiFixer (14B)	4	58.72	35.91	24.65	19.18
ArtiFixer (1.3B)	1	86.75	57.76	43.20	34.38
ArtiFixer (1.3B)	4	101.77	69.44	53.77	49.24

Appendix

Table 7. Inference configurations. Fewer denoising steps and context parallelism across multiple GPUs further improve throughput, with the 1.3B variant reaching up to 101.77 FPS.

Figure 11. Denoising steps . We vary the number of denoising steps when beginning from the initial degraded rendering. ArtiFixer can render plausible content in as few as 1 step, although sharpness and temporal consistency suffer somewhat in empty areas.

	PSNR $\uparrow$			SSIM $\uparrow$			LPIPS $\downarrow$
Method	3-view	6-view	9-view	3-view	6-view	9-view	3-view	6-view	9-view
GenFusion ( Wu et al., 2025c )	15.29	17.16	18.36	0.369	0.447	0.496	0.585	0.500	0.465
GSFixer ( Yin et al., 2025b )	15.61	17.27	18.63	0.370	0.426	0.481	0.559	0.478	0.420
CAT3D ( Gao* et al., 2024 )	16.62	17.72	18.67	0.377	0.425	0.460	0.515	0.482	0.460
ArtiFixer3D+ (1.3B)	16.60	18.04	19.44	0.414	0.466	0.513	0.486	0.435	0.394
ArtiFixer3D+ (14B)	17.51	18.95	20.16	0.444	0.498	0.537	0.441	0.396	0.359

Appendix

Table 8. Impact of model scale on Mip-NeRF 360. Our 1.3B variant matches CAT3D within 0.02 dB on the 3-view split and exceeds other video model baselines despite using fewer parameters.

Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$
GenFusion ( Wu et al., 2025c )	17.03	0.624	0.392	132.91
Gen3C ( Ren et al., 2025 )	15.50	0.491	0.476	68.36
ArtiFixer3D+ (1.3B)	19.04	0.635	0.352	22.3
ArtiFixer3D+ (14B)	20.15	0.662	0.307	13.91

Appendix

Table 9. Impact of model scale on novel content generation (DL3DV). Even with a 1.3B backbone, ArtiFixer3D+ outperforms the other video model baselines by a wide margin.

Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
3DGS ( Kerbl et al., 2023 )	9.57	0.108	0.779
SparseNeRF ( Wang et al., 2023 )	9.23	0.191	0.632
DNGaussian ( Li et al., 2024 )	10.23	0.156	0.643
ReconX ( Liu et al., 2026 )	14.28	0.394	0.564
ArtiFixer3D+	14.75	0.464	0.463

Appendix

Table 10. Tanks and Temples (2-view). ArtiFixer3D+ outperforms all baselines.

Method	MASt3R $\downarrow$	RAFT $\downarrow$
Fixer ( NVIDIA, 2025 )	0.1288	0.1236
Difix3D+ ( Wu et al., 2025a )	0.0974	0.0959
GenFusion ( Wu et al., 2025c )	0.0817	0.0786
Gen3C ( Ren et al., 2025 )	0.0766	0.0757
ArtiFixer	0.0749	0.0749
ArtiFixer3D+	0.0697	0.0697

Appendix

Table 11. Multi-view consistency. We measure multi-view consistency via MEt3R ( Asim et al., 2025 ) with MASt3R and RAFT backbones. All ArtiFixer variants outperform baselines, with ArtiFixer3D achieving the best results due to its explicit multi-view-consistent 3D representation.

Explore similar work

ArtifactWorld: Scaling 3D Gaussian Splatting Artifact Restoration via Video Generation Models
Apr 14, 2026Xinliang Wang, Yifeng Shi, Zhenyu Wu3D Gaussian Splatting Novel View Synthesis
Ke Holdings Inc. Beijing, China
BetterScene: 3D Scene Synthesis with Representation-Aligned Generative Model
Feb 26, 2026Yuci Han, Charles Toth, John E. Anderson +2Novel View Synthesis Sparse-View 3D Reconstruction
Dept. of Electrical and Computer Engineering, The Ohio State University · USACE ERDC GRL
Leveling3D: Leveling Up 3D Reconstruction with Feed-Forward 3D Gaussian Splatting and Geometry-Aware Generation
Mar 17, 2026Yiming Huang, Baixiang Huang, Beilei Cui +3Novel View Synthesis 3D Gaussian Splatting
The Chinese University of Hong Kong

Apr 14, 2026J/K move · Enter open · S save

ArtifactWorld: Scaling 3D Gaussian Splatting Artifact Restoration via Video Generation Models

Xinliang Wang, Yifeng Shi, Zhenyu Wu

Ke Holdings Inc. Beijing, China

3D Gaussian Splatting Novel View Synthesis Sparse-View 3D Reconstruction

Open paper AskPDF arXiv

3D Gaussian Splatting (3DGS) delivers high-fidelity real-time rendering but suffers from geometric and photometric degradations under sparse-view constraints. Current generative restoration approaches are often limited by insufficient temporal coherence, a lack of explicit spatial constraints, and a lack of large-scale training data, resulting in multi-view inconsistencies, erroneous geometric hallucinations, and limited generalization to diverse real-world artifact distributions. In this paper, we present ArtifactWorld, a framework that resolves 3DGS artifact repair through systematic data expansion and a homogeneous dual-model paradigm. To address the data bottleneck, we establish a fine-grained phenomenological taxonomy of 3DGS artifacts and construct a comprehensive training set of 107.5K diverse paired video clips to enhance model robustness. Architecturally, we unify the restoration process within a video diffusion backbone, utilizing an isomorphic predictor to localize structural defects via an artifact heatmap. This heatmap then guides the restoration through an Artifact-Aware Triplet Fusion mechanism, enabling precise, intensity-guided spatio-temporal repair within native self-attention. Extensive experiments demonstrate that ArtifactWorld achieves state-of-the-art performance in sparse novel view synthesis and robust 3D reconstruction. Code available at: https://github.com/fyting/ArtifactWorld.

Abstract

Explore similar work

ArtifactWorld: Scaling 3D Gaussian Splatting Artifact Restoration via Video Generation Models

BetterScene: 3D Scene Synthesis with Representation-Aligned Generative Model

Leveling3D: Leveling Up 3D Reconstruction with Feed-Forward 3D Gaussian Splatting and Geometry-Aware Generation