Organizations: Hong Kong University of Science and Technology · National Univeristy of Singapore · KTH Royal Institute of Technology · University of Surrey · Independent Researcher · Imperial College London
The development of generalizable Novel View Synthesis (NVS) models is critically limited by the scarcity of large-scale training data featuring diverse and precise camera trajectories. While real-world captures are photorealistic, they are typically sparse and discrete. Conversely, synthetic data scales but suffers from a domain gap and often lacks realistic semantics. We introduce FreeScale, a novel framework that leverages the power of scene reconstruction to transform limited real-world image sequences into a scalable source of high-quality training data. Our key insight is that an imperfect reconstructed scene serves as a rich geometric proxy, but naively sampling from it amplifies artifacts. To this end, we propose a certainty-aware free-view sampling strategy identifying novel viewpoints that are both semantically meaningful and minimally affected by reconstruction errors. We demonstrate FreeScale's effectiveness by scaling up the training of feedforward NVS models, achieving a notable gain of 2.7 dB in PSNR on challenging out-of-distribution benchmarks. Furthermore, we show that the generated data can actively enhance per-scene 3D Gaussian Splatting optimization, leading to consistent improvements across multiple datasets. Our work provides a practical and powerful data generation engine to overcome a fundamental bottleneck in 3D vision. Project page: https://mvp-ai-lab.github.io/FreeScale.
Figures & tables
Figure 2: FreeScale generation pipeline. Our overall pipeline consists of three phases. First, given an image sequence, we reconstruct the scene as a continuous 3D representation, which allows us to place arbitrary viewpoint candidates. Second, we perform certainty-aware free-view synthesis: we establish a view graph based on a certainty grid and filter redundant candidates. Finally, we apply image rectification to produce the final free-views. The generated data can then be used to train feed-forward models like LVSM and refine the scene Gaussians.
Figure 3: Showcase of predefined camera trajectory modes.
In-Domain
Out-of-Domain (Zero-shot Generalization)
DL3DV
MipNeRF360
Tanks & Temples
Method
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
Small camera motion
LVSM [ 16 ]
22.20
0.680
0.216
15.84
0.285
0.583
13.07
0.336
0.674
LVSM w/ FreeScale
24.20
0.767
0.165
18.30
0.386
0.460
13.80
0.652
0.361
Large camera motion
Table 1: Quantitative comparison of feed-forward models on viewpoint generalization. Joint training with our FVGen data yields consistent improvements across both small and large camera motion settings.
Figure 4: Qualitative comparison for feed-forward model. We use red boxes to highlight challenging regions where the target viewpoint differs significantly from the corresponding areas in the training poses.
DL3DV
Nerfburster
Tanks & Temples
Method
PSNR ↑
SSIM ↑
LPIPS ↓
Time (min.) ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
Nerfbusters [ 46 ]
17.45
0.606
0.370
-
17.72
0.647
0.352
-
-
-
DIFIX3D+ [ 47 ]
17.99
0.601
0.293
81.40
18.07
0.642
0.279
18.59
0.623
0.317
3DGS [ 17 ]
19.18
0.714
0.233
35.19
18.14
0.643
0.265
20.37
0.680
0.253
3DGS w/ DIFIX3D
19.12
0.680
0.211
39.75
17.69
0.606
0.264
19.75
0.630
0.210
3DGS w/ Depth
19.07
0.718
0.227
36.67
17.54
0.630
0.285
20.24
0.678
0.259
Table 2: Quantitative comparison of per-scene reconstruction on the Out-Of-Domain protocol. Our FreeScale achieves consistent advantages in PSNR and SSIM without incurring a significant increase in computational burden.
Figure 5: Qualitative results on the Nerfbusters dataset. The 3DGS baseline exhibits significant artifacts in unobserved areas, such as floaters and geometric noise, particularly in unobserved areas. In contrast, our method ensures high-fidelity results by sampling supplementary views from the reconstructed scene geometry.
Table 7
Feed-forward Model
Per-Scene Optimization
Method
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
Baseline
17.75
0.385
0.465
19.18
0.714
0.233
+ FV (random)
18.68
0.508
0.342
19.20
0.715
0.240
+View-graph
19.11
0.529
0.322
19.57
0.723
0.219
Table 5: Ablation study on free-view images. “FV” indicate generated free-view images, “View-graph” means graph-guided joint training and certainty-guided per-scene reconstruction.
Figure 6: Comparison of reference image selection. Our view graph identifies the shared visible region with the noisy view (red circle), ensuring accurate image rectification.
FreeView Statistics
Feed-Forward Model
Method
Certainty Grid
View Graph
#Image
Per-scene #Image
BRISQUE ↓
Avg. Time (s) ↓
PSNR ↑
SSIM ↑
LPIPS ↓
FVGen
✓
✓
145,528
75
0.36
225.64
19.11
0.529
0.322
✓
-
164,874
91
0.38
608.71
17.63
0.480
0.397
-
✓
166,282
109
0.36
727.93
17.86
0.491
0.354
Baseline
-
-
-
-
-
-
17.75
0.385
0.465
Table 6: Ablation study on Free-View generation. Our certainty-aware generation relies fundamentally on the certainty grid and the established view graph. Without the view graph , selecting the top-500 candidates solely by certainty score results in high redundancy and fails to provide valuable guidance for feed-forward model training. Without the certainty grid , we must resort to calculating inter-view correspondence only via position and rotation distance, which is both inaccurate and computationally inefficient.
Figure 7: Showcase of different freeview generation. Our FVGen maximally captures under-constrained geometry while being minimally contaminated by reconstruction artifacts.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
Small camera motion
Large camera motion
LVSM
22.20
0.680
0.216
18.75
0.522
0.352
wo/ Diffusion
23.41
0.736
0.185
20.89
0.634
0.268
w/ FreeScale
24.20
0.767
0.165
21.45
0.661
0.247
Table 7: Ablation isolating the impact of diffusion-based image rectification. Comparing the LVSM baseline to our method without diffusion ( wo/ Diffusion ) demonstrates that the primary performance boost stems directly from the expanded viewpoint diversity and geometric coverage. Integrating the diffusion prior (FreeScale) resolves remaining artifacts for optimal fidelity.
Figure 8: Consistent showcases of view graph impact. Compared to DIFIX3D+’s distance-based reference selection strategy, our view graph provides better overlap and higher free-view consistency for reference. The red bounding boxes delineate artifacts introduced by inaccurate reference images during the image rectification stage.
Figure 9: Impact of diverse camera trajectory modes. Relying solely on an Orbit trajectory limits viewpoint diversity, leading to noticeable blurring artifacts in under-observed regions (red circle). In contrast, our multi-mode sampling ensures maximal scene coverage, yielding sharper structural details and improved quantitative performance.
Figure 10: Additional showcases of view graph impact on reference image selection. The red circles highlight the shared visible region between the reference view and sampled noisy view, while the blue bounding boxes delineate artifacts introduced by inaccurate reference images during the image rectification stage.
Figure 11: Qualitative comparison of feed-forward on out-of-domain data (MipNeRF360). The results are from LVSM at resolution 256.
Figure 12: Qualitative comparison of per scene reconstruction on Tanks and Temples dataset.
Figure 13: Qualitative comparison on DL3DV dataset for per-scene reconstruction. The blue bounding box indicates the zoom-in area. Despite the apparent clarity of DIFIX3D+, its progressive update and inaccurate reference image selection introduce significant hallucinated content, visible in the spurious reflection of the lamp and the corrupted desktop details on the right side.
Figure 14: Qualitative comparison on DL3DV dataset for per-scene reconstruction. The blue bounding box indicates the zoom-in area.
Figure 15: Qualitative comparison on DL3DV dataset for per-scene reconstruction. The blue bounding box indicates the zoom-in area.
Figure 16: Failure cases of free-view generation. Red circles indicate regions where our pipeline struggles due to diffusion priors, including the incorrect handling of complex reflections (top row) and the over-sharpening of 3DGS floaters (bottom row).
We present BetterScene, an approach to enhance novel view synthesis (NVS) quality for diverse real-world scenes using extremely sparse, unconstrained photos. BetterScene leverages the production-ready Stable Video Diffusion (SVD) model pretrained on billions of frames as a strong backbone, aiming to mitigate artifacts and recover view-consistent details at inference time. Conventional methods have developed similar diffusion-based solutions to address these challenges of novel view synthesis. Despite significant improvements, these methods typically rely on off-the-shelf pretrained diffusion priors and fine-tune only the UNet module while keeping other components frozen, which still leads to inconsistent details and artifacts even when incorporating geometry-aware regularizations like depth or semantic conditions. To address this, we investigate the latent space of the diffusion model and introduce two components: (1) temporal equivariance regularization and (2) vision foundation model-aligned representation, both applied to the variational autoencoder (VAE) module within the SVD pipeline. BetterScene integrates a feed-forward 3D Gaussian Splatting (3DGS) model to render features as inputs for the SVD enhancer and generate continuous, artifact-free, consistent novel views. We evaluate on the challenging DL3DV-10K dataset and demonstrate superior performance compared to state-of-the-art methods.
Yuci Han, Charles Toth, John E. Anderson +2
Dept. of Electrical and Computer Engineering, The Ohio State University · USACE ERDC GRL
Creating flexible 3D scenes from a single image is vital when direct 3D data acquisition is costly or impractical. We introduce NavCrafter, a novel framework that explores 3D scenes from a single image by synthesizing novel-view video sequences with camera controllability and temporal-spatial consistency. NavCrafter leverages video diffusion models to capture rich 3D priors and adopts a geometry-aware expansion strategy to progressively extend scene coverage. To enable controllable multi-view synthesis, we introduce a multi-stage camera control mechanism that conditions diffusion models with diverse trajectories via dual-branch camera injection and attention modulation. We further propose a collision-aware camera trajectory planner and an enhanced 3D Gaussian Splatting (3DGS) pipeline with depth-aligned supervision, structural regularization and refinement. Extensive experiments demonstrate that NavCrafter achieves state-of-the-art novel-view synthesis under large viewpoint shifts and substantially improves 3D reconstruction fidelity.
Hongbo Duan, Peiyu Zhuang, Yi Liu +5
Center for Artificial Intelligence and Robotics, Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, China · Peng Cheng Laboratory, 518108, China · School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University, China +1
While 3D Vision Foundation Models (3DVFMs) have demonstrated remarkable zero-shot capabilities in visual geometry estimation, their direct application to generalizable novel view synthesis (NVS) remains challenging. In this paper, we propose AirSplat, a novel training framework that effectively adapts the robust geometric priors of 3DVFMs into high-fidelity, pose-free NVS. Our approach introduces two key technical contributions: (1) Self-Consistent Pose Alignment (SCPA), a training-time feedback loop that ensures pixel-aligned supervision to resolve pose-geometry discrepancy; and (2) Rating-based Opacity Matching (ROM), which leverages the local 3D geometry consistency knowledge from a sparse-view NVS teacher model to filter out degraded primitives. Experimental results on large-scale benchmarks demonstrate that our method significantly outperforms state-of-the-art pose-free NVS approaches in reconstruction quality. Our AirSplat highlights the potential of adapting 3DVFMs to enable simultaneous visual geometry estimation and high-quality view synthesis.