Single-image novel view synthesis requires viewpoint-dependent structural changes while retaining source-scene information and inferring unobserved content. We formulate this task as a camera-conditioned initialization problem for a frozen pretrained latent diffusion model. At an intermediate DDIM inversion timestep, we decompose the source representation into a structural mean and a source noise residual. A pose-conditioned translation U-Net (TUNet) learns to predict the target-view mean from the source mean. We then introduce a score-guided fusion strategy: the predicted mean and source noise residual form a proposal that is evaluated by the frozen denoiser, whose noise prediction is combined with the translated mean to initialize DDIM sampling. Only TUNet is trained, while the pretrained model synthesizes the final view without explicit reconstruction or rendering of a shared 3D scene. Experiments on MVImgNet and RealEstate10K demonstrate competitive performance. Beyond image-level metrics, Wasserstein comparisons show that the corrected initialization yields trajectory-curvature distributions closer to those of ground-truth target-initialized sampling than raw fusion. These results support structural translation and source-conditioned initialization as a practical approach to reusing frozen diffusion priors for novel view synthesis.
Figures & tables
Figure 2: Overview: Given a single reference image xref , we first apply DDIM inversion up to t=600 to obtain the mean latent zref,μinv . This, together with camera intrinsics/extrinsics, class embeddings, and ray information, is fed into our translation network TUNet . TUNet predicts the target-view mean latent z~tar,μinv , which we combine with the corresponding noise component using our fusion strategy to form the initial DDIM latent z~tarinv . Finally, this latent is used for sampling by a pre-trained diffusion model to synthesize the novel view image.
Figure 3: Mean of the DDIM inverted latent. Top: t=400,600 . Bottom: t=800 and original image.
Figure 4: Qualitative results with our 167-class trained model.
Figure 5: Qualitative results with our 3-class trained model on MVImgNet test set.
Table 1: (a) Full-trajectory curvature Wasserstein distance to the ground-truth target diffusion trajectory (lower is better). (b) Bin-wise curvature analysis. FD is frame distance
Figure 6: (a) View-dependent geometric details absent in the mean prediction but re-synthesized in the final result. (b) Sampling trajectories under different initialization strategies. Row: (i) our diffusion-dynamics aligned latent z~tarinv , (ii) ground-truth target-view latent ztarinv (equation 5 ), and (iii) direct noise addition from the reference latent zfused (equation 3 ).
Table 2: (a) Comparison for 3 classes - chairs, sofa, tables. Resolution is 256×256 . (b) Comparison for 167 classes. Resolution is 90×90 .
Figure 7: (a) Comparison with NViST on unseen test scenes from five classes. (b) Multi-view consistency across a 30-frame sweep from a single input image.
Figure 8: Results on unseen classes of MVImgNet and OOD images
Table 3: (a) Ablation study using 3-class model. (b) Analysis of fusion strategies
t
LPIPS ↓
PSNR ↑
SSIM ↑
FID ↓
400
0.510
15.37
0.528
150.28
600
0.491
15.41
0.522
93.57
800
0.550
15.01
0.502
197.13
Table 4: Effect of diffusion timestep t .
Pipeline
LPIPS ↓
PSNR ↑
SSIM ↑
FID ↓
Stable Diffusion v1.5
0.491
15.41
0.522
93.57
Stable Diffusion 2.1
0.491
15.27
0.522
88.95
Dreamlike Photoreal 2.0
0.507
15.15
0.503
101.84
Table 5: Comparison across different SD pipelines.
Frame Distance
N
LPIPS ↓
PSNR ↑
SSIM ↑
0–5
196
0.4741±0.0455
16.28±1.92
0.543±0.19
6–10
3375
0.4863±0.0483
15.82±2.01
0.528±0.17
11–16
8811
0.4914±0.0499
15.63±2.06
0.525±0.17
17–21
4980
0.4949±0.0508
15.55±2.11
0.517±0.17
21–25
996
0.4996±0.0517
15.44±2.14
0.513±0.17
Table 6: Performance as a function of source–target frame distance. Metrics are reported as mean ± standard deviation across all test pairs in each bucket.
Mid-range (30–60 frames)
Long-range (60–120 frames)
Method
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
VIVID
0.523
13.83
0.439
0.594
12.69
0.410
Ours
0.503
15.04
0.479
0.609
13.44
0.448
Table 7: Results on 1K pairs of RealEstate10K. Images are uniformly sampled at random from different scenes.
Figure 9: We use textual prompts to edit the novel view reconstructions as desired. In the first row, we place a bag on the bench. In the second row, a dog is introduced in the scene. In the third row, a pair of shoes are placed by the bench.
Figure 10: MVImgNet failure cases. Each column represents one scene, while rows correspond to input view, predicted novel view, and target view.
Method
LPIPS ↓
PSNR ↑
SSIM ↑
FID ↓
Ours
0.556
16.162
0.578
69.604
Table S1: Our results at original resolution of 512×512
Figure S1 : Qualitative ablation results.
Figure S2 : Additional Qualitative results with our 167-class trained model on MVImgNet test set.