While single-view novel view synthesis methods have been dramatically enhanced recently, how to quantify their performance has yet to be standardized. In this work, we demonstrate that an important factor lies in how each method's own reference frame is aligned with the so-called ground-truth pose. Existing methods often rely on test-time optimization of target poses for this alignment, which inherently is ill-posed due to the complexity of the optimization landscape of camera poses vs render accuracy, leading to methods being evaluated at an arbitrary local optimum. Some recognize this limitation and thus rely on 3D reconstructions of entire scenes using training and test views to align poses, which may be infeasible. In this work, instead, we propose to use traditional 2D-to-2D and 2D-to-3D image matching techniques to align 3D reconstructions. We also evaluate only within co-visible regions, between different views, which is the only place where a `correct' ground truth can be defined. Our evaluations show that with proper coordinate alignment, novel-view synthesis methods that use explicit 3D Gaussians achieve the state of the art, despite being previously reported in the literature to be outperformed by pure image-based 2D methods. We further provide a simple baseline that directly utilizes an off-the-shelf video generator and a feed-forward geometry estimator. Our simple baseline is already comparable to state-of-the-art methods specifically trained for novel-view synthesis, without any task-specific training.
Figures & tables
Figure 1: – Evaluation can be misleading due to inaccurate test-time pose alignment – We visualize the error map with the conventional test-time optimization and our proposed protocol (red is higher error, blue is lower). With the same 3D scene reconstructed from the reference view, conventional test-time optimization can leave residual misalignment between the rendered and ground-truth views, leading to large quantitative error. In contrast, our evaluation protocol provides a more robust alignment strategy, enabling more reliable performance measurement.
Figure 2: Extracting co-visible regions – We show example co-visible regions extracted by our protocol. We extract by projecting each pixel back and forth via DUSt3R [ 44 ] and checking their relative depths. We make a conservative estimate of co-visible regions to not disadvantage methods that do not extrapolate pixels outside the input view.
Figure 3: Examples of misalignment leading to inaccurate evaluation – We show examples with Flash3D [ 38 ] for RealEstate10K (scene 01a5cc , fr. 50) and DTU (scan103, fr. 12), with test-time optimization only, and our proposed protocol. As shown, optimization alone leaves a large pose offset. Combined with the fact that many methods do not mask via co-visibility [ 6 , 48 , 51 , 5 , 18 ] , the PSNR values can be very low, up to a level where it effectively is at a level where two random images are being compared ( <10 dB). With our image-matching-based pre-alignment, on the other hand, images are better aligned, and reasonable PSNR values are extracted. The yellow tint marks the non-co-visible region that our protocol excludes from computing the metrics.
Method
RealEstate10K [ 54 ] (SV)
RealEstate10K [ 54 ] (VC)
DL3DV-10K [ 29 ] (SV)
DTU [ 15 ]
Tank & Temples [ 22 ] (VC)
Average
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
Test-time optim. only
Ours
18.85
0.714
0.402
16.03
0.666
0.489
12.74
0.341
0.682
11.83
0.397
0.647
14.30
0.454
0.566
14.75
0.514
0.557
Flash3D [ 38 ]
14.52
0.578
0.504
14.75
0.630
0.499
11.71
0.300
0.683
11.12
0.402
0.689
13.75
0.455
0.574
13.17
0.473
0.590
DepthSplat [ 48 ]
16.66
0.612
0.409
16.45
0.633
0.421
12.77
0.300
0.644
10.67
0.346
0.623
13.59
0.441
0.534
14.03
0.466
0.526
Lyra [ 3 ]
17.50
0.653
0.423
16.20
0.662
0.454
12.34
0.295
0.627
12.68
0.432
0.535
15.38
0.478
0.482
14.82
0.504
0.504
AnySplat [ 17 ]
19.40
0.734
0.330
19.03
0.739
0.370
13.54
0.323
0.617
12.82
0.416
0.523
14.81
0.465
0.508
15.92
0.535
0.470
Table 1: Quantitative results – We summarize the Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index Metric (SSIM) [ 45 ] , and Learned Perceptual Image Patch Similarity (LPIPS) [ 52 ] for each method on our benchmark datasets, with the mean over all datasets reported on the right. Within each group, best results are in bold and second best are underlined . Using test-time optimization only reports wildly different results compared to when our evaluation protocol is used: Flash3D [ 38 ] is among the worst performing methods with a naive evaluation but, with better alignment, performs better than DepthSplat, while SHARP [ 32 ] leads the test-time-optimization-only group on average—its Gaussians are regressed directly in the input-camera frame, so it needs little alignment to begin with.
Method
RealEstate10K [ 54 ] (SV)
RealEstate10K [ 54 ] (VC)
DL3DV-10K [ 29 ] (SV)
DTU [ 15 ]
Tank & Temples [ 22 ] (VC)
Average
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
Test-time optim. only
Ours
20.73
0.744
0.315
16.08
0.538
0.462
13.04
0.260
0.668
11.82
0.340
0.652
14.09
0.364
0.536
15.15
0.449
0.527
LVSM [ 18 ]
19.76
0.690
0.283
19.54
0.672
0.270
13.37
0.266
0.632
12.58
0.372
0.602
16.16
0.443
0.440
16.28
0.489
0.445
MVSplat [ 6 ]
15.51
0.589
0.401
14.54
0.535
0.404
11.91
0.227
0.657
10.34
0.275
0.643
12.41
0.351
0.563
12.94
0.395
0.534
DepthSplat [ 48 ]
17.47
0.656
0.360
16.10
0.602
0.378
12.39
0.247
0.648
12.01
0.369
0.539
13.49
0.399
0.517
14.29
0.455
0.488
NoPoSplat [ 49 ]
13.42
0.466
0.538
12.29
0.404
0.584
11.31
0.249
0.740
9.41
0.327
0.801
11.51
0.295
0.623
11.59
0.348
0.657
Table 2: Quantitative results (256 × 256 square evaluation) – PSNR, SSIM [ 45 ] , and LPIPS [ 52 ] for the methods that support the square (256 × 256 centre-crop) setting, under both test-time-optimization-only and our evaluation protocol, with the mean over all datasets reported on the right. Flash3D [ 38 ] and ViewCrafter [ 51 ] are omitted as they cannot produce a comparable 256 × 256 cell. PF3plat [ 14 ] is evaluated with its RealEstate10K checkpoint on every dataset, as are MVSplat [ 6 ] and NoPoSplat [ 49 ] . Within each group, best results are in bold and second best are underlined .
Figure 4: Qualitative results – We show qualitative examples for each dataset under both evaluation resolutions, with the ground-truth target view (co-visibility overlay in blue) on the left. Middle: native-resolution methods, Flash3D [ 38 ] , ViewCrafter [ 51 ] , Lyra [ 3 ] , SHARP [ 32 ] , and our own baseline ( Tab. 1 ). Right: methods that support the 256×256 centre-crop setting, LVSM [ 18 ] , PF3plat [ 14 ] , and our baseline ( Tab. 2 ). Both halves show the same scene and held-out target view for each dataset, listed above each row. All methods are shown with our conservative co-visibility overlay. With our enhanced alignment protocol, the actual qualitative performances of Flash3D [ 38 ] , ViewCrafter [ 51 ] , and our baseline align well with the quantifications in Tab. 1 . Our suggested baseline, while simple, performs on par with Flash3D [ 38 ] .
Init
Train
RE10K [ 54 ]
RE10K [ 54 ] -VC
T&T [ 22 ]
DL3DV-10K [ 29 ]
DTU [ 15 ]
Avg
input only
input only
21.34
20.11
16.32
13.61
16.56
17.59
all views
input only
22.26
21.30
16.80
14.13
17.47
18.39
input only
all views
19.89
20.17
17.24
13.46
16.24
17.40
all views
all views
19.92
20.34
17.40
13.60
16.41
17.53
Table 3: Ablation on using generated views – We ablate when to use generated views for training our 3D Gaussians. Including generated views for initialization only and training Gaussians only with the real input view gives best results.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Example views generated by video models – We show example views generated by video models.
Figure 6: Qualitative effect of generated views – For each scene we show the ground truth and the full init × train factorial of Tab. 3 : rows vary the initialization (input view only vs. all generated views) and columns the training supervision. Training on generated views smears fine geometry (top scene) and hallucinates floaters in disocclusions (bottom scene); using them only for initialization stays sharpest. The teal overlay marks pixels outside the co-visible region, excluded from the metric.
Figure 7: ViewCrafter [ 51 ] point-cloud conditioning vs. diffusion output. ViewCrafter [ 51 ] rasterizes its single-image point cloud from the (canonical) target pose to form the conditioning frame (middle), then a video-diffusion model synthesizes the final view (right). The raster is geometrically and photometrically faithful wherever the single view provides coverage (the speckled regions are sparse splatting where coverage thins); the diffusion model fills the remainder but can hallucinate geometry and drift in color as the viewpoint rotates away from the input. Rows: RealEstate10K (SV), RealEstate10K (VC), DL3DV-10K, DTU, Tanks & Temples (VC), on the same scenes as Fig. 4 .
We study the challenging problem of novel view video synthesis from single images or monocular videos. Existing methods, which operate under the assumption that pre-trained video models lack native novel view synthesis capability and enforce view alignment via camera conditioning, task-specific fine-tuning, or stepwise hard denoising guidance, often suffer from artifacts and compromised global scene consistency. In this paper, we introduce NeoMap, a novel training-free framework designed to locate high-fidelity, view-consistent novel view solutions from general pre-trained video models. The key to our approach is the core insight that promising novel view solutions are inherently encoded within the natural video data manifold learned by pre-trained models, and the core challenge is simply to locate this optimal solution. We solve this via our core mechanism: convergent manifold alternating projection iterations that optimize the initial noise. Extensive experiments demonstrate that NeoMap significantly outperforms all existing methods across 3 standard novel view synthesis benchmarks, including the challenging Tanks-and-Temples, LLFF and DAVIS datasets, achieving state-of-the-art generation fidelity and top-tier view consistency.
Jinxi Li, Tianyi Zhang, Yafei Yang +4
Shenzhen Research Institute, The Hong Kong Polytechnic University · vLAR Group, The Hong Kong Polytechnic University
Generating photorealistic novel views from unposed images requires both 3D geometric understanding and the ability to synthesize unseen content. A natural strategy combines feed-forward 3DGS reconstruction with multi-view diffusion. Yet prior pipelines extract at most one signal from the reconstruction, either pixel rendering or learned features, while none exploits per-Gaussian visibility for occlusion-aware reference selection. This information disconnect leaves renderable geometry, visibility cues, and learned features unused. SplatGuide closes this disconnect by reusing a single 3DGS scene across three complementary roles. Rendered images provide pixel-aligned geometric conditioning. Per-Gaussian source-view indices are rendered into a target-view voting map for occlusion-aware reference selection. Reconstruction tokens supply feature-level guidance via cross-attention. All three signals derive from the same reconstruction forward pass. Across RealEstate10K, DL3DV, Tanks-and-Temples, and Mip-NeRF 360, SplatGuide achieves state-of-the-art pose-free novel view synthesis. On RealEstate10K, with a moderate number of input views, it surpasses the ground-truth-pose baseline.
Yejun Zhang, Zihan Wang, Xu Ji +8
Aalto University, Finland · Deep Render, UK · ELLIS Institute Finland +3
Novel View Synthesis (NVS) aims to generate unseen views of a 3D object given a limited number of known views. Existing methods often struggle to synthesize plausible views for unobserved regions, particularly under single-view input, and still face challenges in maintaining geometry- and appearance-consistency. To address these issues, we propose OrbitNVS, which reformulates NVS as an orbit video generation task. Through tailored model design and training strategies, we adapt a pre-trained video generation model to the NVS task, leveraging its rich visual priors to achieve high-quality view synthesis. Specifically, we incorporate camera adapters into the video model to enable accurate camera control. To enhance two key properties of 3D objects, geometry and appearance, we design a normal map generation branch and use normal map features to guide the synthesis of the target views via attention mechanism, thereby improving geometric consistency. Moreover, we apply a pixel-space supervision to alleviate blurry appearance caused by spatial compression in the latent space. Extensive experiments show that OrbitNVS significantly outperforms previous methods on the GSO and OmniObject3D benchmarks, especially in the challenging single-view setting (\eg, +2.9 dB and +2.4 dB PSNR).
Jinglin Liang, Zijian Zhou, Rui Huang +2
South China University of Technology · Agentic Labs, XGTech · King’s College London +1