3D Gaussian Splatting achieves photorealistic reconstruction within training view distribution, yet it degrades on out-of-distribution novel views, exhibiting holes in unobserved regions and artifacts in observable areas. Recent works formulate this task as extrapolation and interpolation and try to address it with generative models, but remain limited in extrapolation scale and quality. They repeat a generate-reconstruct-shift cycle to progressively build a scene, which introduces accumulated errors with every step conditioning on previous outcomes. In this work, we propose a holistic framework for extrapolation and interpolation. We devise an independent camera view detection mechanism to enable parallel conflict-free extrapolation, circumventing the reliance on the aforementioned error-prone cycle. Building upon this, we design a hierarchical pipeline that extrapolates independent and dependent camera views separately. Additionally, previous methods overlook inconsistency between generated and original images, resulting in compromising well-reconstructed areas. We propose a plug-and-play Quality-Aware Mask (QA-Mask) module, enabling selective utilization of generated data. By calibrating learning weights with pixel-wise rendering quality, it prevents generation-induced degradations on well-constructed areas. Extensive experiments demonstrate the superior performance of our framework, with QA-Mask generalizing to multiple generative reconstruction models.
Figures & tables
Figure 1. Our framework excels at large-scale, geometrically coherent 3D scene extrapolation and interpolation on extreme out-of-distribution novel views.
Figure 2. Independent Camera View Detection. We construct a mesh-based view frustum enclosing the geometry of the inpainted region for each candidate view Vi . Collision detection is performed for frustums to determine whether their camera views are independent. In this example, Vi-1 and Vi are independent, while Vi and Vi+1 collide with each other and are therefore not independent.
Figure 3. An overview of our main pipeline. It consists of extrapolations for independent and dependent camera views. The former achieves extrapolation on the original scene over large areas in a single pass. The latter performs a second-stage residual completion on the remaining un-extrapolated regions, enabling seamless scene extrapolation. DIBR-based video depth alignment provides robust depth priors throughout the process. Blue indicates the generated data used in each stage.
Figure 4. Under-reconstructed areas, including holes and low-quality regions , should be updated using generated images. In contrast, well-reconstructed areas within the original scene should remain unchanged to avoid compromising the overall quality due to potential generative inconsistencies .
Figure 5. Visualizations of rendered RGB images and QA-Mask from extrapolation views on four scenes in Mip-NeRF 360. QA-Mask identified holes and low rendering quality regions (dashed yellow) that align well with human perception. However, Bayes’ Rays and Nerfbusters fail to clearly distinguish between low-quality and well-reconstructed areas, while opacity maps and NVF are unable to accurately identify the low-quality areas within the original scene or maintain high (uncertainty) values across the hole areas.
Figure 6. (a) Rendering degradations on out-of-distribution views. 3D Gaussians overfit on training views but underfit on out-of-distribution (OOD) views which results in degraded rendering. (b) Quality-Aware Mask rendering method. We first prepare a pretrained 3DGS model by setting the SH base color to red, view-dependent color coefficients al,m to 0 and converting all original training images to black. We fine-tune this model with only al,m receiving gradients. After fine-tuning, within the training view distribution with high rendering quality, the base color is canceled out by al,m with the final color being black. While in OOD views with low rendering quality, the lack of black supervisions causes Gaussians to remain the base color.
Figure 7. Qualitative comparisons of extrapolation on Mip-NeRF 360. The first row shows the inpainted images on independent camera views VI . The following rows show the images rendered on the camera views between each pair of VIi,VIi+1 . Our framework achieves high rendering quality on extrapolated areas while also leaves well-reconstructed original areas untouched.
Methods
C-IQA ↑
QUALI-C ↑
MVC ↑
SemSim ↑
Time
Few-Shot GS
0.278
0.352
1792
0.707
24min
DiFix3D+
0.354
0.451
1755
0.687
53min
GenFusion
0.340
0.378
1157
0.712
44min
Guidedvd-3dgs
0.230
0.298
1845
0.691
306min
Ours
0.404
0.498
1972
0.737
77min
Table 1. Non-reference metrics and runtime on Mip-NeRF 360. C-IQA denotes CLIP-IQA. QUALI-C denotes QUALI-CLIP.
Methods
PSNR ↑
SSIM ↑
LPIPS ↓
Few-Shot GS
18.92
0.80
0.3438
DiFix3D+
20.93
0.84
0.2554
GenFusion
21.93
0.85
0.2502
Guidedvd-3dgs
22.10
0.84
0.2662
Ours
23.79
0.88
0.2300
Table 2. Comparisons of reference metrics on ScanNet++. Our method achieves the best performance averaged on 10 scenes.
Methods
PSNR ↑
LPIPS ↓
Exst.
Extrapltd.
Exst.
Extrapltd.
DiFix3D+
28.11
10.23
0.1187
0.4654
w/ Bayes’ Rays
27.87
10.20
0.1235
0.4701
w/ Nerfbusters
27.73
10.09
0.1193
0.4659
w/ NVF
28.19
10.02
0.1093
0.4700
w/ Opacity Maps
28.15
10.17
0.1109
0.4655
Table 3. Generalizability of QA-Mask on ScanNet++. Exst. denotes existing and Extrapltd. denotes extrapolated.
Figure 8. Comparison between w/ and w/o QA-Mask on GenFusion and DiFix3D+. QA-Mask prevents generated data from disrupting well-reconstructed areas in the original scene. Noticeable improvements can be observed for both models.
Mip-NeRF 360
CLIP-IQA ↑
MVC ↑
SemSim ↑
w/o QA-Mask
0.343
1673
0.719
w/o DIBR Propagation
0.316
1468
0.697
Ours
0.404
1972
0.737
ScanNet++
PSNR ↑
SSIM ↑
LPIPS ↓
w/o Idpdt Cam Detection
20.27
0.80
0.3161
w/o Depth Supervision
23.66
0.88
0.2302
Table 4. Ablation study for core components and hyperparameter selection on both Mip-NeRF 360 and ScanNet++. Idpdt Cam Detection denotes Independent Camera View Detection.
We introduce ReconSplat, a feed-forward model for 3D scene reconstruction that aims to address the longstanding trade-off between plausible view generation for unobserved regions and geometric consistency, providing both geometrically aligned novel views and sharp depth estimates. Our approach builds on 3D Gaussian splatting (3DGS) as an intermediate differentiable scene representation and integrates it with a multi-view latent diffusion model (MV-LDM) trained to act simultaneously as a refiner and an inpainter for appearance and scene geometry. We enforce geometric consistency by guiding the diffusion process with variational 3D latent features for appearance and geometry, encoded by the feed-forward 3DGS representation and rasterized to 2D latent space. ReconSplat produces both photorealistic novel views and accurate depth maps on real-world benchmarks, RealEstate10K and DL3DV-10K, outperforming existing methods in challenging extrapolation setups. Notably, ReconSplat allows the extrapolation of unseen and challenging viewpoints jointly with coherent and precise scene geometry.
Giuseppe Stracquadanio, Kevin Raj, Julia Grabinski +1
3D Gaussian Splatting (3DGS) delivers high-fidelity real-time rendering but suffers from geometric and photometric degradations under sparse-view constraints. Current generative restoration approaches are often limited by insufficient temporal coherence, a lack of explicit spatial constraints, and a lack of large-scale training data, resulting in multi-view inconsistencies, erroneous geometric hallucinations, and limited generalization to diverse real-world artifact distributions. In this paper, we present ArtifactWorld, a framework that resolves 3DGS artifact repair through systematic data expansion and a homogeneous dual-model paradigm. To address the data bottleneck, we establish a fine-grained phenomenological taxonomy of 3DGS artifacts and construct a comprehensive training set of 107.5K diverse paired video clips to enhance model robustness. Architecturally, we unify the restoration process within a video diffusion backbone, utilizing an isomorphic predictor to localize structural defects via an artifact heatmap. This heatmap then guides the restoration through an Artifact-Aware Triplet Fusion mechanism, enabling precise, intensity-guided spatio-temporal repair within native self-attention. Extensive experiments demonstrate that ArtifactWorld achieves state-of-the-art performance in sparse novel view synthesis and robust 3D reconstruction. Code available at: https://github.com/fyting/ArtifactWorld.
Generalizable 3D Gaussian Splatting (G-3DGS) has emerged as a promising approach for novel view synthesis undersparse-view settings. However, existing frameworks remain restricted by pixel-aligned Gaussian estimation, whichstruggles in partially observed or occluded regions and often leads to incomplete surfaces or structural collapse. Toaddress these challenges, we propose SeeU (Seeing the Unseen), a novel G-3DGS framework. We frame its core design asSemantic-in-Gaussian: semantic-conditioned refinement in Gaussian space. Specifically, we introduce a Cross-viewEntropy-Aware (CEA) module that aggregates multi-view semantic and geometric cues into compact embeddings. Theseembeddings guide the Conditional Gaussian Transformer, which applies residual updates to coarse Gaussians, helpingrecover under-constrained regions of partially observed structures while preserving surface consistency. Comprehensiveexperiments on multiple benchmarks demonstrate that SeeU consistently improves rendering quality and structuralcompleteness while retaining efficient feed-forward inference. Especially under challenging extrapolation settings,SeeU achieves an average improvement of 2.44 dB in PSNR compared to recent SOTA G-3DGS methods.
Zeyang Bai, Yunpeng Wang, Yunbiao Wang +1
School of Artificial Intelligence, University of Chinese Academy of Sciences, China · Global Institute of Future Technology, Shanghai Jiao Tong University, China