3D Gaussian Splatting (3DGS) has achieved remarkable success in novel view synthesis; however, reconstructions under sparse views often exhibit noticeable artifacts. While recent video diffusion models provide strong spatio-temporal priors for 3DGS restoration, directly fine-tuning them for restoration is suboptimal, as they lack awareness of the underlying multi-camera geometry, resulting in multi-view inconsistencies. In this work, we propose a novel 3D-aware video restoration framework designed to enhance the quality of sparse 3DGS reconstruction. Specifically, we construct a large-scale 3DGS video dataset to enable specialized fine-tuning. To bridge the gap between 2D video generation and 3D multi-view constraints, we introduce a camera-conditioned geometric prior. By using the first and last frames as boundary anchors and encoding the corresponding camera relationships, we explicitly inject spatial structure into the video generation pipeline. This boundary-anchored, camera-aware prior guides the network toward geometrically grounded restoration that remains coherent across viewpoints. Extensive experiments show that, among video-prior restoration methods, our approach attains the best pixel- and structure-level fidelity (PSNR/SSIM) and improves multi-view consistency, while remaining competitive in perceptual quality (LPIPS).
Figures & tables
Fig. 1: Comparison between naive 3DGS and our proposed GaussVid. Sparse 3DGS often produces floaters, distortions, and blurred structures. Our GaussVid restores intermediate frames using a first-to-last frame conditioned video model with a camera injection module and a progressive training, yielding artifact-free, sharp, and geometrically consistent results.
Fig. 2: Overview of the GaussVid framework. Our approach restores artifact-corrupted sparse 3DGS renderings into high-quality, 3D-consistent videos via three core designs: (1) Plücker Token Encoder extracts robust geometric priors by disentangling moment and direction streams via cross-gating fusion (Sec. III-C1 ). (2) Dual-Stream Camera Injection integrates these priors into DiT blocks using asymmetric joint attention, allowing video latents to query 3D context while shielding camera tokens from noise (Sec. III-C2 ). (3) Difficulty-Aware Curriculum Learning : dynamically balances flow matching ( LFM ) and latent reconstruction ( Llatent ) losses based on sample sparsity to stabilize optimization (Sec. III-D ). Note: Snowflake icons denote frozen pre-trained base weights. Standard parameter-efficient fine-tuning details are provided in Sec. IV-A .
Base
Restoration
PSNR ↑
SSIM ↑
LPIPS ↓
3DGS [ 14 ]
Baseline
20.67
0.7034
0.3815
+ Difix3D [ 37 ]
21.19
0.7221
0.2463
+ 4DSloMo [ 5 ]
21.54
0.7052
0.3886
+ GSFIXER [ 41 ]
22.35
0.7239
0.3685
+ Ours
22.74
0.7291
0.3599
FSGS [ 47 ]
Baseline
21.13
0.7289
0.3620
TABLE I: Comparison with existing restoration methods across different sparse-view 3DGS baselines on DL3DV. Each restoration method is applied as post-processing to the same set of baselines. Best results are in bold , second best are underlined .
Fig. 3: Qualitative comparison on DL3DV. From left to right: corrupted 3DGS [ 14 ] , CoR-GS [ 42 ] , 3DGS + DiFix3D [ 37 ] , 3DGS + GSFIXER [ 41 ] , 3DGS + Ours, and Ground Truth. Colored insets highlight key differences: CoR-GS [ 42 ] still leave blurred boundaries in complex regions; DiFix3D shifts geometry and hallucinates content; although GSFIXER incorporates VGGT-derived 3D geometry, the noisy and ambiguous geometry under sparse views confines it to suboptimal results. Our method effectively removes floaters and structural distortions while preserving fine-grained details and multi-view consistency.
Methods
n =6
n =9
n =12
Baseline (Octree-GS [ 31 ] )
21.02
20.62
20.13
+ Difix3D [ 37 ]
21.20
21.04
20.34
+ 4DSloMo [ 5 ]
21.79
21.43
20.98
+ GSFIXER [ 41 ]
21.83
21.12
20.76
+ Ours
21.99
21.84
21.16
TABLE II: Generalization to Zip-NeRF [ 3 ] indoor scenes with Octree-GS [ 31 ] as the base model. PSNR is reported at three sparsity levels ( n=6,9,12 ). Best results in bold .
Fig. 4: Qualitative comparison on Zip-NeRF [ 3 ] indoor scenes at three sparsity levels ( n=6,9,12 ). DiFix3D over-generates content misaligned with the ground truth (green and red insets); although GSFIXER leverages VGGT-derived 3D geometry, the noisy geometry under sparse views leaves less faithful fine structures; 4DSloMo retains residual local artifacts (blue and green insets). Our method yields the cleanest results across all settings.
Variant
PSNR ↑
SSIM ↑
LPIPS ↓
(a)
No anchor (all corrupted)
21.90
0.7160
0.3692
First-only anchor
22.11
0.7192
0.3679
(b)
Random intermediate sampling
22.35
0.7240
0.3685
(c)
Narrow-only ( n=6 )
22.19
0.7134
0.3767
Wide-only ( n=26 )
22.47
0.7255
0.3642
Ours
22.74
0.7291
0.3599
TABLE III: Ablation on data construction pipeline. Each variant modifies exactly one design choice from our full configuration. (a) Number of clean anchor frames. (b) Intermediate frame sampling strategy. (c) Artifact difficulty range during training. Best results in bold .
Injection Strategy
PSNR ↑
SSIM ↑
LPIPS ↓
(a) No Camera Condition
21.27
0.7104
0.3805
(b) Early Latent Fusion
21.71
0.7155
0.3800
(c) Monolithic Dual-Stream
22.09
0.7179
0.3700
(d) Ours
22.74
0.7291
0.3599
TABLE IV: Ablation on camera conditioning injection. We compare strategies of increasing sophistication for injecting camera geometry into the diffusion model. Best results in bold .
Fig. 5: Qualitative ablation on camera conditioning. (a) No Camera Condition; (b) Early Latent Fusion; (c) monolithic dual-stream; (d) our decomposed cross-gate injection. Our design recovers the sharpest geometry, especially near depth discontinuities (see shelf edges).
Fig. 6: Qualitative effect of each training component. Between the corrupted input and the ground truth, we compare a shuffle-trained baseline (a) with each component added incrementally: (b) a difficulty-aware curriculum that gradually raises clip difficulty from easy to hard (small to large sampling interval n ); (c) a progressive latent-reconstruction loss that shifts supervision from latent space toward perceptual quality; and (d) both combined (Ours). The curriculum mainly removes large-scale structural artifacts, the progressive loss restores fine-grained texture, and combining both yields the result closest to the ground truth.
Setting
PSNR ↑
SSIM ↑
LPIPS ↓
(a) Baseline (Shuffle + LFM )
21.50
0.7135
0.3828
(b) + Curriculum
22.06
0.7134
0.3767
(c) + Prog. Loss
22.54
0.7261
0.3658
(d) + Both (Ours)
22.74
0.7291
0.3599
TABLE V: Ablation on training strategy. Each proposed component is incrementally added to the shuffle-trained baseline. All variants use identical data and total training steps. Best results in bold .
Setting
PSNR ↑
SSIM ↑
LPIPS ↓
Baseline (3DGS)
20.67
0.7034
0.3815
(a) + Wan2.1 ∗
20.56
0.6929
0.4050
(b) + Wan2.1 ∗∗
21.11
0.7071
0.3915
(c) + Ours
22.74
0.7291
0.3599
TABLE VI: Ablation on the adaptation strategy. The backbone is fixed to Wan2.1-VACE-14B and only the adaptation strategy is varied. ∗ applies the backbone zero-shot; ∗∗ fine-tunes it on our data with vanilla LoRA, without our camera injection or progressive training. All variants share the same 3DGS baseline. Best results in bold .
3D Gaussian Splatting (3DGS) delivers high-fidelity real-time rendering but suffers from geometric and photometric degradations under sparse-view constraints. Current generative restoration approaches are often limited by insufficient temporal coherence, a lack of explicit spatial constraints, and a lack of large-scale training data, resulting in multi-view inconsistencies, erroneous geometric hallucinations, and limited generalization to diverse real-world artifact distributions. In this paper, we present ArtifactWorld, a framework that resolves 3DGS artifact repair through systematic data expansion and a homogeneous dual-model paradigm. To address the data bottleneck, we establish a fine-grained phenomenological taxonomy of 3DGS artifacts and construct a comprehensive training set of 107.5K diverse paired video clips to enhance model robustness. Architecturally, we unify the restoration process within a video diffusion backbone, utilizing an isomorphic predictor to localize structural defects via an artifact heatmap. This heatmap then guides the restoration through an Artifact-Aware Triplet Fusion mechanism, enabling precise, intensity-guided spatio-temporal repair within native self-attention. Extensive experiments demonstrate that ArtifactWorld achieves state-of-the-art performance in sparse novel view synthesis and robust 3D reconstruction. Code available at: https://github.com/fyting/ArtifactWorld.
Novel view synthesis from sparse inputs remains challenging for 3D Gaussian Splatting (3DGS) due to ambiguous geometry, cross-view inconsistency, and missing details in under-constrained regions, resulting in degraded reconstruction and unstable rendering. To tackle these issues, we propose D3GS, a Depth-DINO-Diffusion guided sparse-view Gaussian reconstruction framework that jointly enhances geometry and appearance. D3GS first recovers a high-resolution, metric depth map via diffusion-based completion and DPT (Dense Prediction Transformer) refinement, providing robust Gaussian initialization and geometric constraints. Then, a DINO-guided view-consistent learning is introduced to augment Gaussian attributes with structural features, improving multi-view consistency. Finally, a diffusion-based Gaussian refinement module injects generative priors into an iterative optimization strategy, enhancing high-frequency geometric and appearance details within the Gaussian representation. Experiments on DTU, LLFF, and Mip-NeRF 360 show that D3GS achieves consistent and substantial improvements over strong baselines, with ablation studies validating the effectiveness and complementary roles of each component.
Yunqi Gao, Zhanfeng Liao, Hanzhang Tu +7
Central China Normal University · Tsinghua University · Fuzhou University +4
Gaussian Splatting has achieved remarkable progress in multi-view surface reconstruction, yet it exhibits notable degradation when only few views are available. Although recent efforts alleviate this issue by enhancing multi-view consistency to produce plausible surfaces, they struggle to infer unseen, occluded, or weakly constrained regions beyond the input coverage. To address this limitation, we present VidSplat, a training-free generative reconstruction framework that leverages powerful video diffusion priors to iteratively synthesize novel views that compensate for missing input coverage, and thereby recover complete 3D scenes from sparse inputs. Specifically, we tackle two key challenges that enable the effective integration of generation and reconstruction. First, for 3D consistent generation, we elaborate a training-free, stage-wise denoising strategy that adaptively guides the denoising direction toward the underlying geometry using the rendered RGB and mask images. Second, to enhance the reconstruction, we develop an iterative mechanism that samples camera trajectories, explores unobserved regions, synthesizes novel views, and supplements training through confidence weighted refinement. VidSplat performs robustly to sparse input and even a single image. Extensive experiments on widely used benchmarks demonstrate our superior performance in sparse-view scene reconstruction.
Jimin Tang, Wenyuan Zhang, Junsheng Zhou +5
School of Software, Tsinghua University, China · Kuaishou Technology, China · Department of Computer Science, Wayne State University, USA