cs.CVApr 12, 2026
SaveFreeScale: Scaling 3D Scenes via Certainty-Aware Free-View Generation
Organizations: Hong Kong University of Science and Technology · National Univeristy of Singapore · KTH Royal Institute of Technology · University of Surrey · Independent Researcher · Imperial College London
Abstract
The development of generalizable Novel View Synthesis (NVS) models is critically limited by the scarcity of large-scale training data featuring diverse and precise camera trajectories. While real-world captures are photorealistic, they are typically sparse and discrete. Conversely, synthetic data scales but suffers from a domain gap and often lacks realistic semantics. We introduce FreeScale, a novel framework that leverages the power of scene reconstruction to transform limited real-world image sequences into a scalable source of high-quality training data. Our key insight is that an imperfect reconstructed scene serves as a rich geometric proxy, but naively sampling from it amplifies artifacts. To this end, we propose a certainty-aware free-view sampling strategy identifying novel viewpoints that are both semantically meaningful and minimally affected by reconstruction errors. We demonstrate FreeScale's effectiveness by scaling up the training of feedforward NVS models, achieving a notable gain of 2.7 dB in PSNR on challenging out-of-distribution benchmarks. Furthermore, we show that the generated data can actively enhance per-scene 3D Gaussian Splatting optimization, leading to consistent improvements across multiple datasets. Our work provides a practical and powerful data generation engine to overcome a fundamental bottleneck in 3D vision. Project page: https://mvp-ai-lab.github.io/FreeScale.
Figures & tables
Figure 2: FreeScale generation pipeline. Our overall pipeline consists of three phases. First, given an image sequence, we reconstruct the scene as a continuous 3D representation, which allows us to place arbitrary viewpoint candidates. Second, we perform certainty-aware free-view synthesis: we establish a view graph based on a certainty grid and filter redundant candidates. Finally, we apply image rectification to produce the final free-views. The generated data can then be used to train feed-forward models like LVSM and refine the scene Gaussians.
Figure 3: Showcase of predefined camera trajectory modes.
| In-Domain | Out-of-Domain (Zero-shot Generalization) | ||||||||
| DL3DV | MipNeRF360 | Tanks & Temples | |||||||
| Method | PSNR ↑ | SSIM ↑ | LPIPS ↓ | PSNR ↑ | SSIM ↑ | LPIPS ↓ | PSNR ↑ | SSIM ↑ | LPIPS ↓ |
| Small camera motion | |||||||||
| LVSM [ 16 ] | 22.20 | 0.680 | 0.216 | 15.84 | 0.285 | 0.583 | 13.07 | 0.336 | 0.674 |
| LVSM w/ FreeScale | 24.20 | 0.767 | 0.165 | 18.30 | 0.386 | 0.460 | 13.80 | 0.652 | 0.361 |
| Large camera motion | |||||||||
Table 1: Quantitative comparison of feed-forward models on viewpoint generalization. Joint training with our FVGen data yields consistent improvements across both small and large camera motion settings.
Figure 4: Qualitative comparison for feed-forward model. We use red boxes to highlight challenging regions where the target viewpoint differs significantly from the corresponding areas in the training poses.
| DL3DV | Nerfburster | Tanks & Temples | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Method | PSNR ↑ | SSIM ↑ | LPIPS ↓ | Time (min.) ↓ | PSNR ↑ | SSIM ↑ | LPIPS ↓ | PSNR ↑ | SSIM ↑ | LPIPS ↓ |
| Nerfbusters [ 46 ] | 17.45 | 0.606 | 0.370 | - | 17.72 | 0.647 | 0.352 | - | - | - |
| DIFIX3D+ [ 47 ] | 17.99 | 0.601 | 0.293 | 81.40 | 18.07 | 0.642 | 0.279 | 18.59 | 0.623 | 0.317 |
| 3DGS [ 17 ] | 19.18 | 0.714 | 0.233 | 35.19 | 18.14 | 0.643 | 0.265 | 20.37 | 0.680 | 0.253 |
| 3DGS w/ DIFIX3D | 19.12 | 0.680 | 0.211 | 39.75 | 17.69 | 0.606 | 0.264 | 19.75 | 0.630 | 0.210 |
| 3DGS w/ Depth | 19.07 | 0.718 | 0.227 | 36.67 | 17.54 | 0.630 | 0.285 | 20.24 | 0.678 | 0.259 |
Table 2: Quantitative comparison of per-scene reconstruction on the Out-Of-Domain protocol. Our FreeScale achieves consistent advantages in PSNR and SSIM without incurring a significant increase in computational burden.
Figure 5: Qualitative results on the Nerfbusters dataset. The 3DGS baseline exhibits significant artifacts in unobserved areas, such as floaters and geometric noise, particularly in unobserved areas. In contrast, our method ensures high-fidelity results by sampling supplementary views from the reconstructed scene geometry.
| Feed-forward Model | Per-Scene Optimization | |||||
|---|---|---|---|---|---|---|
| Method | PSNR ↑ | SSIM ↑ | LPIPS ↓ | PSNR ↑ | SSIM ↑ | LPIPS ↓ |
| Baseline | 17.75 | 0.385 | 0.465 | 19.18 | 0.714 | 0.233 |
| + FV (random) | 18.68 | 0.508 | 0.342 | 19.20 | 0.715 | 0.240 |
| +View-graph | 19.11 | 0.529 | 0.322 | 19.57 | 0.723 | 0.219 |
Table 5: Ablation study on free-view images. “FV” indicate generated free-view images, “View-graph” means graph-guided joint training and certainty-guided per-scene reconstruction.
Figure 6: Comparison of reference image selection. Our view graph identifies the shared visible region with the noisy view (red circle), ensuring accurate image rectification.
| FreeView Statistics | Feed-Forward Model | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Method | Certainty Grid | View Graph | #Image | Per-scene #Image | BRISQUE ↓ | Avg. Time (s) ↓ | PSNR ↑ | SSIM ↑ | LPIPS ↓ |
| FVGen | ✓ | ✓ | 145,528 | 75 | 0.36 | 225.64 | 19.11 | 0.529 | 0.322 |
| ✓ | - | 164,874 | 91 | 0.38 | 608.71 | 17.63 | 0.480 | 0.397 | |
| - | ✓ | 166,282 | 109 | 0.36 | 727.93 | 17.86 | 0.491 | 0.354 | |
| Baseline | - | - | - | - | - | - | 17.75 | 0.385 | 0.465 |
Table 6: Ablation study on Free-View generation. Our certainty-aware generation relies fundamentally on the certainty grid and the established view graph. Without the view graph , selecting the top-500 candidates solely by certainty score results in high redundancy and fails to provide valuable guidance for feed-forward model training. Without the certainty grid , we must resort to calculating inter-view correspondence only via position and rotation distance, which is both inaccurate and computationally inefficient.
Figure 7: Showcase of different freeview generation. Our FVGen maximally captures under-constrained geometry while being minimally contaminated by reconstruction artifacts.
| Method | PSNR ↑ | SSIM ↑ | LPIPS ↓ | PSNR ↑ | SSIM ↑ | LPIPS ↓ |
|---|---|---|---|---|---|---|
| Small camera motion | Large camera motion | |||||
| LVSM | 22.20 | 0.680 | 0.216 | 18.75 | 0.522 | 0.352 |
| wo/ Diffusion | 23.41 | 0.736 | 0.185 | 20.89 | 0.634 | 0.268 |
| w/ FreeScale | 24.20 | 0.767 | 0.165 | 21.45 | 0.661 | 0.247 |
Table 7: Ablation isolating the impact of diffusion-based image rectification. Comparing the LVSM baseline to our method without diffusion ( wo/ Diffusion ) demonstrates that the primary performance boost stems directly from the expanded viewpoint diversity and geometric coverage. Integrating the diffusion prior (FreeScale) resolves remaining artifacts for optimal fidelity.
Figure 8: Consistent showcases of view graph impact. Compared to DIFIX3D+’s distance-based reference selection strategy, our view graph provides better overlap and higher free-view consistency for reference. The red bounding boxes delineate artifacts introduced by inaccurate reference images during the image rectification stage.
Figure 9: Impact of diverse camera trajectory modes. Relying solely on an Orbit trajectory limits viewpoint diversity, leading to noticeable blurring artifacts in under-observed regions (red circle). In contrast, our multi-mode sampling ensures maximal scene coverage, yielding sharper structural details and improved quantitative performance.
Figure 10: Additional showcases of view graph impact on reference image selection. The red circles highlight the shared visible region between the reference view and sampled noisy view, while the blue bounding boxes delineate artifacts introduced by inaccurate reference images during the image rectification stage.
Figure 11: Qualitative comparison of feed-forward on out-of-domain data (MipNeRF360). The results are from LVSM at resolution 256.
Figure 12: Qualitative comparison of per scene reconstruction on Tanks and Temples dataset.
Figure 13: Qualitative comparison on DL3DV dataset for per-scene reconstruction. The blue bounding box indicates the zoom-in area. Despite the apparent clarity of DIFIX3D+, its progressive update and inaccurate reference image selection introduce significant hallucinated content, visible in the spurious reflection of the lamp and the corrupted desktop details on the right side.
Figure 14: Qualitative comparison on DL3DV dataset for per-scene reconstruction. The blue bounding box indicates the zoom-in area.
Figure 15: Qualitative comparison on DL3DV dataset for per-scene reconstruction. The blue bounding box indicates the zoom-in area.
Figure 16: Failure cases of free-view generation. Red circles indicate regions where our pipeline struggles due to diffusion priors, including the incorrect handling of complex reflections (top row) and the over-sharpening of 3DGS floaters (bottom row).