Novel View Synthesis (NVS) aims to generate unseen views of a 3D object given a limited number of known views. Existing methods often struggle to synthesize plausible views for unobserved regions, particularly under single-view input, and still face challenges in maintaining geometry- and appearance-consistency. To address these issues, we propose OrbitNVS, which reformulates NVS as an orbit video generation task. Through tailored model design and training strategies, we adapt a pre-trained video generation model to the NVS task, leveraging its rich visual priors to achieve high-quality view synthesis. Specifically, we incorporate camera adapters into the video model to enable accurate camera control. To enhance two key properties of 3D objects, geometry and appearance, we design a normal map generation branch and use normal map features to guide the synthesis of the target views via attention mechanism, thereby improving geometric consistency. Moreover, we apply a pixel-space supervision to alleviate blurry appearance caused by spatial compression in the latent space. Extensive experiments show that OrbitNVS significantly outperforms previous methods on the GSO and OmniObject3D benchmarks, especially in the challenging single-view setting (\eg, +2.9 dB and +2.4 dB PSNR).
Figures & tables
Figure 2: Overview of OrbitNVS. (a) Model architecture. The model incorporates camera conditions by inserting camera adapters before the first and into each subsequent Dit block. A normal map generation branch, which shares parameters with the primary branch, is added to guide the RGB video synthesis. Other conditions are injected via two pathways: 1) channel-wise concatenation of the reference frame’s latent representation and a positional mask with the noised latent, and 2) integration of text embeddings and the first frame’s CLIP embedding via cross-attention at each layer. (b) Loss computation. The latent representations are decoded back to pixel space using the VAE decoder to compute a pixel-space loss, while the original latent loss is retained.
Figure 3: Qualitative comparison of NVS results generated by different methods under the single reference view setting.
GSO
0∘
30∘
60∘
SEVA
18.3
0.87
0.15
16.3
0.84
0.17
13.7
0.80
0.24
EscherNet
20.8
0.85
0.15
19.8
0.84
0.15
19.3
0.83
0.17
SV3D
18.9
0.88
0.18
17.5
0.86
0.22
17.6
0.86
0.22
OrbitNVS
23.7
0.89
0.11
21.9
0.87
0.13
21.0
0.86
0.15
w/o normal
22.3
0.89
0.12
20.8
0.87
0.14
19.9
0.86
0.16
w/o pixel-loss
22.8
0.89
0.11
21.2
0.87
0.13
20.0
0.85
0.16
Table 1: Experimental results on the GSO and OmniObject3D benchmarks under a single reference view and different camera trajectories. The values 0 ∘ , 30 ∘ , and 60 ∘ denote the elevation oscillation amplitude of the sinusoidal camera trajectory (0 ∘ represents a level orbit). “w/o normal” and “w/o pixel-loss” are ablations of our OrbitNVS without the pixel-space loss and normal map branch, respectively. Each entry reports the “PSNR ↑ SSIM ↑ LPIPS ↓ ” metrics, with the best results highlighted in bold .
GSO
1 ref.
2 ref.
3 ref.
SV3D
18.9
0.88
0.18
-
-
SEVA
18.3
0.87
0.15
25.7
0.92
0.07
28.3
0.94
0.04
EscherNet
20.9
0.85
0.15
23.4
0.87
0.13
26.3
0.91
0.06
OrbitNVS
23.7
0.89
0.11
26.5
0.92
0.06
28.8
0.94
0.04
Table 2: Experimental results of different NVS methods under a level camera trajectory (0 ∘ elevation) with varying numbers of reference views. Each entry reports the “PSNR ↑ SSIM ↑ LPIPS ↓ ” metrics, with the best results highlighted in bold .
Figure 4: Ablation on the normal map generation branch. Results with the branch (c, d) show significantly clearer geometric details than those without it (b).
Figure 5: Ablation on the pixel-space post-training. The result with the pixel loss (d) shows superior plausibility in detailed textures compared to the result without it (b).
Figure 6: Controllable Novel View Synthesis is achieved by manipulating the text prompt. (a) Using a VLM to predict a prompt from the reference image as the condition for NVS. (b) Using a manually modified text prompt as the condition for NVS.
Method
0 ∘
30 ∘
60 ∘
OrbitNVS
23.7 0.89 0.11
21.9 0.87 0.13
21.0 0.86 0.15
w/o VLM
24.0 0.90 0.10
21.9 0.87 0.12
21.0 0.87 0.14
Table C : Ablation Study of the VLM using the GSO Dataset.
Method
0 ∘
30 ∘
60 ∘
OrbitNVS
21.0 0.87 0.14
20.4 0.86 0.14
19.9 0.85 0.16
w/o VLM
21.0 0.87 0.13
20.6 0.86 0.14
19.8 0.85 0.15
Table D : Ablation of the VLM on the OmniObject3D Dataset.
Dataset
Mean ↓
< 11.25 ∘ ↑
< 22.5 ∘ ↑
< 30 ∘ ↑
GSO
15.91
59.18
79.13
85.48
OmniObject3D
18.72
48.41
73.78
82.16
Table E : Quantitative evaluation of normal map quality generated by OrbitNVS on both datasets. ‘Mean’ indicates the mean angular error between the predicted and ground-truth normals, and ‘ <x∘ ’ reports the percentage of angles with an error less than x∘ .
Figure G : Example 1. A set of frames from a video generated by OrbitNVS. Each frame includes the reference image, the predicted novel-view image and normal map, and the corresponding ground-truth novel-view image and normal map.