Novel View Synthesis (NVS) aims to generate unseen views of a 3D object given a limited number of known views. Existing methods often struggle to synthesize plausible views for unobserved regions, particularly under single-view input, and still face challenges in maintaining geometry- and appearance-consistency. To address these issues, we propose OrbitNVS, which reformulates NVS as an orbit video generation task. Through tailored model design and training strategies, we adapt a pre-trained video generation model to the NVS task, leveraging its rich visual priors to achieve high-quality view synthesis. Specifically, we incorporate camera adapters into the video model to enable accurate camera control. To enhance two key properties of 3D objects, geometry and appearance, we design a normal map generation branch and use normal map features to guide the synthesis of the target views via attention mechanism, thereby improving geometric consistency. Moreover, we apply a pixel-space supervision to alleviate blurry appearance caused by spatial compression in the latent space. Extensive experiments show that OrbitNVS significantly outperforms previous methods on the GSO and OmniObject3D benchmarks, especially in the challenging single-view setting (\eg, +2.9 dB and +2.4 dB PSNR).
Figures & tables
Figure 2: Overview of OrbitNVS. (a) Model architecture. The model incorporates camera conditions by inserting camera adapters before the first and into each subsequent Dit block. A normal map generation branch, which shares parameters with the primary branch, is added to guide the RGB video synthesis. Other conditions are injected via two pathways: 1) channel-wise concatenation of the reference frame’s latent representation and a positional mask with the noised latent, and 2) integration of text embeddings and the first frame’s CLIP embedding via cross-attention at each layer. (b) Loss computation. The latent representations are decoded back to pixel space using the VAE decoder to compute a pixel-space loss, while the original latent loss is retained.
Figure 3: Qualitative comparison of NVS results generated by different methods under the single reference view setting.
GSO
0∘
30∘
60∘
SEVA
18.3
0.87
0.15
16.3
0.84
0.17
13.7
0.80
0.24
EscherNet
20.8
0.85
0.15
19.8
0.84
0.15
19.3
0.83
0.17
SV3D
18.9
0.88
0.18
17.5
0.86
0.22
17.6
0.86
0.22
OrbitNVS
23.7
0.89
0.11
21.9
0.87
0.13
21.0
0.86
0.15
w/o normal
22.3
0.89
0.12
20.8
0.87
0.14
19.9
0.86
0.16
w/o pixel-loss
22.8
0.89
0.11
21.2
0.87
0.13
20.0
0.85
0.16
Table 1: Experimental results on the GSO and OmniObject3D benchmarks under a single reference view and different camera trajectories. The values 0 ∘ , 30 ∘ , and 60 ∘ denote the elevation oscillation amplitude of the sinusoidal camera trajectory (0 ∘ represents a level orbit). “w/o normal” and “w/o pixel-loss” are ablations of our OrbitNVS without the pixel-space loss and normal map branch, respectively. Each entry reports the “PSNR ↑ SSIM ↑ LPIPS ↓ ” metrics, with the best results highlighted in bold .
GSO
1 ref.
2 ref.
3 ref.
SV3D
18.9
0.88
0.18
-
-
SEVA
18.3
0.87
0.15
25.7
0.92
0.07
28.3
0.94
0.04
EscherNet
20.9
0.85
0.15
23.4
0.87
0.13
26.3
0.91
0.06
OrbitNVS
23.7
0.89
0.11
26.5
0.92
0.06
28.8
0.94
0.04
Table 2: Experimental results of different NVS methods under a level camera trajectory (0 ∘ elevation) with varying numbers of reference views. Each entry reports the “PSNR ↑ SSIM ↑ LPIPS ↓ ” metrics, with the best results highlighted in bold .
Figure 4: Ablation on the normal map generation branch. Results with the branch (c, d) show significantly clearer geometric details than those without it (b).
Figure 5: Ablation on the pixel-space post-training. The result with the pixel loss (d) shows superior plausibility in detailed textures compared to the result without it (b).
Figure 6: Controllable Novel View Synthesis is achieved by manipulating the text prompt. (a) Using a VLM to predict a prompt from the reference image as the condition for NVS. (b) Using a manually modified text prompt as the condition for NVS.
Method
0 ∘
30 ∘
60 ∘
OrbitNVS
23.7 0.89 0.11
21.9 0.87 0.13
21.0 0.86 0.15
w/o VLM
24.0 0.90 0.10
21.9 0.87 0.12
21.0 0.87 0.14
Table C : Ablation Study of the VLM using the GSO Dataset.
Method
0 ∘
30 ∘
60 ∘
OrbitNVS
21.0 0.87 0.14
20.4 0.86 0.14
19.9 0.85 0.16
w/o VLM
21.0 0.87 0.13
20.6 0.86 0.14
19.8 0.85 0.15
Table D : Ablation of the VLM on the OmniObject3D Dataset.
Dataset
Mean ↓
< 11.25 ∘ ↑
< 22.5 ∘ ↑
< 30 ∘ ↑
GSO
15.91
59.18
79.13
85.48
OmniObject3D
18.72
48.41
73.78
82.16
Table E : Quantitative evaluation of normal map quality generated by OrbitNVS on both datasets. ‘Mean’ indicates the mean angular error between the predicted and ground-truth normals, and ‘ <x∘ ’ reports the percentage of angles with an error less than x∘ .
Figure G : Example 1. A set of frames from a video generated by OrbitNVS. Each frame includes the reference image, the predicted novel-view image and normal map, and the corresponding ground-truth novel-view image and normal map.
Single-image novel view synthesis remains challenging because the underlying 3D geometry is highly ambiguous. Recent diffusion-based approaches produce plausible results, but they often struggle to preserve the geometric structure and spatial coherence of foreground objects. We present GenNVS, a framework for geometry-enhanced novel view synthesis via a disentangled 3D prior. Specifically, GenNVS models foreground objects and the background with 3D Gaussian Splatting and aligns them through a coarse-to-fine geometric optimization process to form a unified 3D scene. This scene conditions a video diffusion model through the proposed Dual-Stream Masking mechanism, which guides synthesis by jointly exploiting rendered validity masks and geometry-aware warping. Experimental results show that GenNVS performs favorably against recent methods in both visual quality and geometric accuracy, while naturally supporting flexible scene editing.
We present BetterScene, an approach to enhance novel view synthesis (NVS) quality for diverse real-world scenes using extremely sparse, unconstrained photos. BetterScene leverages the production-ready Stable Video Diffusion (SVD) model pretrained on billions of frames as a strong backbone, aiming to mitigate artifacts and recover view-consistent details at inference time. Conventional methods have developed similar diffusion-based solutions to address these challenges of novel view synthesis. Despite significant improvements, these methods typically rely on off-the-shelf pretrained diffusion priors and fine-tune only the UNet module while keeping other components frozen, which still leads to inconsistent details and artifacts even when incorporating geometry-aware regularizations like depth or semantic conditions. To address this, we investigate the latent space of the diffusion model and introduce two components: (1) temporal equivariance regularization and (2) vision foundation model-aligned representation, both applied to the variational autoencoder (VAE) module within the SVD pipeline. BetterScene integrates a feed-forward 3D Gaussian Splatting (3DGS) model to render features as inputs for the SVD enhancer and generate continuous, artifact-free, consistent novel views. We evaluate on the challenging DL3DV-10K dataset and demonstrate superior performance compared to state-of-the-art methods.
Yuci Han, Charles Toth, John E. Anderson +2
Dept. of Electrical and Computer Engineering, The Ohio State University · USACE ERDC GRL
Novel view synthesis requires strong 3D geometric consistency and the ability to generate visually coherent images across diverse viewpoints. While recent camera-controlled video diffusion models show promising results, they often suffer from geometric distortions and limited camera controllability. To overcome these challenges, we introduce GeoNVS, a geometry-grounded novel-view synthesizer that enhances both geometric fidelity and camera controllability through explicit 3D geometric guidance. Our key innovation is the Gaussian Splat Feature Adapter (GS-Adapter), which lifts input-view diffusion features into 3D Gaussian representations, renders geometry-constrained novel-view features, and adaptively fuses them with diffusion features to correct geometrically inconsistent representations. Unlike prior methods that inject geometry at the input level, GS-Adapter operates in feature space, avoiding view-dependent color noise that degrades structural consistency. Its plug-and-play design enables zero-shot compatibility with diverse feed-forward geometry models without additional training, and can be adapted to other video diffusion backbones. Experiments across 9 scenes and 18 settings demonstrate state-of-the-art performance, achieving 11.3% and 14.9% improvements over SEVA and CameraCtrl, with up to 2x reduction in translation error and 7x in Chamfer Distance.