3D Gaussian Splatting (3DGS) has achieved remarkable success in novel view synthesis; however, reconstructions under sparse views often exhibit noticeable artifacts. While recent video diffusion models provide strong spatio-temporal priors for 3DGS restoration, directly fine-tuning them for restoration is suboptimal, as they lack awareness of the underlying multi-camera geometry, resulting in multi-view inconsistencies. In this work, we propose a novel 3D-aware video restoration framework designed to enhance the quality of sparse 3DGS reconstruction. Specifically, we construct a large-scale 3DGS video dataset to enable specialized fine-tuning. To bridge the gap between 2D video generation and 3D multi-view constraints, we introduce a camera-conditioned geometric prior. By using the first and last frames as boundary anchors and encoding the corresponding camera relationships, we explicitly inject spatial structure into the video generation pipeline. This boundary-anchored, camera-aware prior guides the network toward geometrically grounded restoration that remains coherent across viewpoints. Extensive experiments show that, among video-prior restoration methods, our approach attains the best pixel- and structure-level fidelity (PSNR/SSIM) and improves multi-view consistency, while remaining competitive in perceptual quality (LPIPS).
Figures & tables
Fig. 1: Comparison between naive 3DGS and our proposed GaussVid. Sparse 3DGS often produces floaters, distortions, and blurred structures. Our GaussVid restores intermediate frames using a first-to-last frame conditioned video model with a camera injection module and a progressive training, yielding artifact-free, sharp, and geometrically consistent results.
Fig. 2: Overview of the GaussVid framework. Our approach restores artifact-corrupted sparse 3DGS renderings into high-quality, 3D-consistent videos via three core designs: (1) Plücker Token Encoder extracts robust geometric priors by disentangling moment and direction streams via cross-gating fusion (Sec. III-C1 ). (2) Dual-Stream Camera Injection integrates these priors into DiT blocks using asymmetric joint attention, allowing video latents to query 3D context while shielding camera tokens from noise (Sec. III-C2 ). (3) Difficulty-Aware Curriculum Learning : dynamically balances flow matching ( LFM ) and latent reconstruction ( Llatent ) losses based on sample sparsity to stabilize optimization (Sec. III-D ). Note: Snowflake icons denote frozen pre-trained base weights. Standard parameter-efficient fine-tuning details are provided in Sec. IV-A .
Base
Restoration
PSNR ↑
SSIM ↑
LPIPS ↓
3DGS [ 14 ]
Baseline
20.67
0.7034
0.3815
+ Difix3D [ 37 ]
21.19
0.7221
0.2463
+ 4DSloMo [ 5 ]
21.54
0.7052
0.3886
+ GSFIXER [ 41 ]
22.35
0.7239
0.3685
+ Ours
22.74
0.7291
0.3599
FSGS [ 47 ]
Baseline
21.13
0.7289
0.3620
TABLE I: Comparison with existing restoration methods across different sparse-view 3DGS baselines on DL3DV. Each restoration method is applied as post-processing to the same set of baselines. Best results are in bold , second best are underlined .
Fig. 3: Qualitative comparison on DL3DV. From left to right: corrupted 3DGS [ 14 ] , CoR-GS [ 42 ] , 3DGS + DiFix3D [ 37 ] , 3DGS + GSFIXER [ 41 ] , 3DGS + Ours, and Ground Truth. Colored insets highlight key differences: CoR-GS [ 42 ] still leave blurred boundaries in complex regions; DiFix3D shifts geometry and hallucinates content; although GSFIXER incorporates VGGT-derived 3D geometry, the noisy and ambiguous geometry under sparse views confines it to suboptimal results. Our method effectively removes floaters and structural distortions while preserving fine-grained details and multi-view consistency.
Methods
n =6
n =9
n =12
Baseline (Octree-GS [ 31 ] )
21.02
20.62
20.13
+ Difix3D [ 37 ]
21.20
21.04
20.34
+ 4DSloMo [ 5 ]
21.79
21.43
20.98
+ GSFIXER [ 41 ]
21.83
21.12
20.76
+ Ours
21.99
21.84
21.16
TABLE II: Generalization to Zip-NeRF [ 3 ] indoor scenes with Octree-GS [ 31 ] as the base model. PSNR is reported at three sparsity levels ( n=6,9,12 ). Best results in bold .
Fig. 4: Qualitative comparison on Zip-NeRF [ 3 ] indoor scenes at three sparsity levels ( n=6,9,12 ). DiFix3D over-generates content misaligned with the ground truth (green and red insets); although GSFIXER leverages VGGT-derived 3D geometry, the noisy geometry under sparse views leaves less faithful fine structures; 4DSloMo retains residual local artifacts (blue and green insets). Our method yields the cleanest results across all settings.
Variant
PSNR ↑
SSIM ↑
LPIPS ↓
(a)
No anchor (all corrupted)
21.90
0.7160
0.3692
First-only anchor
22.11
0.7192
0.3679
(b)
Random intermediate sampling
22.35
0.7240
0.3685
(c)
Narrow-only ( n=6 )
22.19
0.7134
0.3767
Wide-only ( n=26 )
22.47
0.7255
0.3642
Ours
22.74
0.7291
0.3599
TABLE III: Ablation on data construction pipeline. Each variant modifies exactly one design choice from our full configuration. (a) Number of clean anchor frames. (b) Intermediate frame sampling strategy. (c) Artifact difficulty range during training. Best results in bold .
Injection Strategy
PSNR ↑
SSIM ↑
LPIPS ↓
(a) No Camera Condition
21.27
0.7104
0.3805
(b) Early Latent Fusion
21.71
0.7155
0.3800
(c) Monolithic Dual-Stream
22.09
0.7179
0.3700
(d) Ours
22.74
0.7291
0.3599
TABLE IV: Ablation on camera conditioning injection. We compare strategies of increasing sophistication for injecting camera geometry into the diffusion model. Best results in bold .
Fig. 5: Qualitative ablation on camera conditioning. (a) No Camera Condition; (b) Early Latent Fusion; (c) monolithic dual-stream; (d) our decomposed cross-gate injection. Our design recovers the sharpest geometry, especially near depth discontinuities (see shelf edges).
Fig. 6: Qualitative effect of each training component. Between the corrupted input and the ground truth, we compare a shuffle-trained baseline (a) with each component added incrementally: (b) a difficulty-aware curriculum that gradually raises clip difficulty from easy to hard (small to large sampling interval n ); (c) a progressive latent-reconstruction loss that shifts supervision from latent space toward perceptual quality; and (d) both combined (Ours). The curriculum mainly removes large-scale structural artifacts, the progressive loss restores fine-grained texture, and combining both yields the result closest to the ground truth.
Setting
PSNR ↑
SSIM ↑
LPIPS ↓
(a) Baseline (Shuffle + LFM )
21.50
0.7135
0.3828
(b) + Curriculum
22.06
0.7134
0.3767
(c) + Prog. Loss
22.54
0.7261
0.3658
(d) + Both (Ours)
22.74
0.7291
0.3599
TABLE V: Ablation on training strategy. Each proposed component is incrementally added to the shuffle-trained baseline. All variants use identical data and total training steps. Best results in bold .
Setting
PSNR ↑
SSIM ↑
LPIPS ↓
Baseline (3DGS)
20.67
0.7034
0.3815
(a) + Wan2.1 ∗
20.56
0.6929
0.4050
(b) + Wan2.1 ∗∗
21.11
0.7071
0.3915
(c) + Ours
22.74
0.7291
0.3599
TABLE VI: Ablation on the adaptation strategy. The backbone is fixed to Wan2.1-VACE-14B and only the adaptation strategy is varied. ∗ applies the backbone zero-shot; ∗∗ fine-tunes it on our data with vanilla LoRA, without our camera injection or progressive training. All variants share the same 3DGS baseline. Best results in bold .