Sparse-view novel view synthesis is a central problem in 3D content creation, but diffusion-based approaches remain limited by iterative denoising, making multi-view generation expensive at inference time. We introduce NAMVIS, a diffusion-free framework that reformulates multi-view image synthesis as geometry-conditioned next-scale autoregression. Instead of generating target views through repeated denoising, NAMVIS predicts discrete visual tokens through a small number of coarse-to-fine scale steps, while sampling all tokens within each scale and across target views in parallel. To anchor this generation process to explicit camera geometry, we propose Multi-scale Projective Pose Encoding, which injects source and target camera transformations into both target-view self-attention and source-to-target cross-attention at every resolution. NAMVIS further combines global conditioning with dense geometry-aware cross-attention, enabling the model to preserve source-view appearance while maintaining target-view consistency. Across Objaverse, GSO, and OmniObject3D, NAMVIS outperforms diffusion-based baselines in PSNR, SSIM, and LPIPS, while running over 3 times faster than the evaluated diffusion baselines under the same evaluation setting. These results suggest that geometry-conditioned next-scale autoregression is a promising and efficient alternative to diffusion for sparse-view multi-view synthesis. Additional qualitative results, videos, and resources are available at https://corl-team.github.io/namvis/
Figures & tables
Figure 1: NAMVIS performs sparse-view novel view synthesis from posed source images. Purple borders indicate input views; green borders indicate generated target views. NAMVIS generates target views with next-scale autoregressive decoding, producing sharp and more geometrically consistent results.
Figure 2: NAMVIS architecture overview. Source images are encoded by a frozen VQ-VAE into source features, which are pooled into a start-of-sequence token [SOS] and also serve as key–value pairs for dense cross-attention. The autoregressive transformer predicts target residual tokens scale by scale, with multi-scale ProPE injected into both self-attention (among target views) and cross-attention (from target to source). The frozen VQ-VAE decoder maps the final accumulated features to images.
Method
Objaverse (In-Domain)
GSO (Zero-Shot)
OmniObject3D (OO3D)
Time (s) ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
Zero-1-to-3 Liu et al. (2023)
19.097
0.816
0.152
15.614
0.749
0.257
16.094
0.768
0.219
2.6
Zero-1-to-3 XL Deitke et al. (2023a)
19.341
0.827
0.136
15.936
0.747
0.236
16.320
0.767
0.209
2.6
Wonder3D Long et al. (2024)
15.801
0.821
0.209
14.158
0.777
0.285
14.290
0.786
0.243
2.0
SyncDreamer Liu et al. (2024)
19.368
0.826
0.156
16.961
0.781
0.246
16.766
0.793
0.207
16.6
EscherNet Kong et al. (2024)
19.078
0.816
0.147
18.878
0.799
0.156
17.767
0.780
0.167
2.1
Table 1: Quantitative comparison across Objaverse, GSO, and OmniObject3D.
Setup
LPIPS ↓
SSIM ↑
PSNR ↑
Time (s) ↓
VRAM (GB) ↓
2 → 6
0.0654
0.8845
24.49
1.53
4.3
6 → 2
0.0457
0.9017
26.07
0.84
4.1
6 → 6
0.0473
0.9015
26.12
1.56
5.0
Table 2: Scaling to larger source and target view counts on Objaverse.
Method
COLMAP Points ↑
Wonder3D Long et al. (2024)
172
SyncDreamer Liu et al. (2024)
151
EscherNet Kong et al. (2024)
214
NAMVIS 1B (Ours)
221
Table 3: COLMAP reconstructability comparison. Higher is better.
Method
Registered views
Coverage ↑
GT
168 / 240
70.0%
EscherNet
145 / 240
60.4%
NAMVIS
162 / 240
67.5%
Table 4: COLMAP camera-registration coverage across 30 scenes.
Figure 3: Qualitative comparison on OO3D Wu et al. (2023) .
Figure 4: Qualitative comparison on GSO Downs et al. (2022) .
Model Variant
PSNR ↑
SSIM ↑
LPIPS ↓
Geometry & Conditioning
w/ Plücker Rays (instead of ProPE)
20.260
0.899
0.142
Prefix Conditioning (instead of Cross-Attn)
14.539
0.797
0.427
Feature Aggregation
CLIP [SOS] (instead of VQ-VAE)
20.613
0.901
0.126
Learned Dense Features (instead of VQ-VAE)
20.886
0.901
0.134
Table 5: Ablation study. Each row replaces or removes one component of the full NAMVIS architecture. All variants are trained on a 10K subset of Objaverse-XL for 200 epochs.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
Setup
NAMVIS
EscherNet Kong et al. [2024]
Time (s)
LPIPS ↓
SSIM ↑
PSNR ↑
Time (s)
LPIPS ↓
SSIM ↑
PSNR ↑
GSO
1–1
0.60
0.1634
0.8104
19.0542
2.09
0.2924
0.7078
14.0517
1–2
0.81
0.1668
0.8100
19.2163
2.16
0.2705
0.7334
14.5864
1–3
0.94
0.1698
0.8110
19.1860
2.35
0.2683
0.7342
14.8349
1–4
1.11
0.1633
0.8147
19.5188
2.78
0.2498
0.7498
15.6767
2–1
0.62
0.0810
0.8603
23.0655
2.08
0.0927
0.8360
21.2183
Appendix
Table 6: Inference time and image quality for different source–target view configurations. Setup a – b denotes a source views and b target views.
Setup
Time (s)
NAMVIS
0.96
w/o KV-cache
1.06
w/o ProPE-cache
1.28
Appendix
Table 7: Inference time for a 3-to-3 setup for different caching configurations.
Figure 5: Varying number of source views from 1 to 4. When views covering previously unseen areas appear, the generated image quality improves.
Figure 6: Varying number of source views from 1 to 4. When views covering previously unseen areas appear, the generated image quality improves.
Figure 7: Target-view self-attention pattern between two generated views. For a queried token in the first target view, self-attention places high weight on corresponding regions in both generated target views, suggesting that target-target attention supports multi-view consistency.
Figure 8: Cross-attention pattern from a target image to two source images, layer 14. Cross-attention assigns high weights to source-image regions that are visually aligned with the queried target location.
Figure 9: Qualitative results of our method on OO3D Wu et al. [2023] .
Figure 10: Qualitative results of our method on GSO Downs et al. [2022] .
Figure 11: Visualization of image progression as new scales are summed up.
Figure 12: Visualization of self-attention pattern on layers 13-16, scales 4×4 , 6×6 , 8×8 , 12×12 , 16×16 .
Figure 13: Visualization of cross-attention pattern on layers 13-16, scales 6×6 , 8×8 , 12×12 , 16×16 .
Figure 14: Visualization of cross-attention pattern on layers 13-16, scales 1×1 , 2×2 , 4×4 .
Scene
Scene
Scene
chair_002
chair_009
chair_017
chair_026
backpack_003
backpack_016
backpack_022
backpack_033
helmet_004
helmet_011
helmet_013
kettle_002
kettle_008
kettle_017
toy_animals_007
toy_animals_025
toy_animals_049
toy_animals_071
Appendix
Table 8: Evaluation scenes from OmniObject3D (OO3D).
We present VGGT-Diff, a geometry-routed multi-view diffusion model for sparse-view novel view synthesis. Existing novel view synthesis (NVS) methods face a fundamental trade-off: reconstruction-based approaches preserve observed geometry but struggle to synthesize unseen regions, while diffusion-based methods provide strong generative priors yet rely on implicit source-to-query correspondence. VGGT-Diff bridges these regimes by routing visual geometry latents from VGGT-Ω into a pretrained video diffusion model. Each visual token is associated with a 3D point and confidence, then transformed into query-aligned latent conditions through a confidence-aware Visual Geometry Router (VGR) that preserves front and back surface evidence. These conditions guide joint target-view denoising, while Point-Track Residual Consistency (PTRC) regularizes predicted-clean residuals along reliable 3D tracks, improving multi-view stability. We further introduce robust geometry conditioning, combining training-time regularization with inference-time guidance for improved robustness. Experiments show competitive or state-of-the-art performance across interpolation and extrapolation under different viewpoint difficulties. Our code is available at https://github.com/chenkangjie1123/VGGT-Diff.
Kangjie Chen, Xiangyu Li, Dongbin Zhang +12
XPeng Motors · The Chinese University of Hong Kong · Tsinghua University
We present BetterScene, an approach to enhance novel view synthesis (NVS) quality for diverse real-world scenes using extremely sparse, unconstrained photos. BetterScene leverages the production-ready Stable Video Diffusion (SVD) model pretrained on billions of frames as a strong backbone, aiming to mitigate artifacts and recover view-consistent details at inference time. Conventional methods have developed similar diffusion-based solutions to address these challenges of novel view synthesis. Despite significant improvements, these methods typically rely on off-the-shelf pretrained diffusion priors and fine-tune only the UNet module while keeping other components frozen, which still leads to inconsistent details and artifacts even when incorporating geometry-aware regularizations like depth or semantic conditions. To address this, we investigate the latent space of the diffusion model and introduce two components: (1) temporal equivariance regularization and (2) vision foundation model-aligned representation, both applied to the variational autoencoder (VAE) module within the SVD pipeline. BetterScene integrates a feed-forward 3D Gaussian Splatting (3DGS) model to render features as inputs for the SVD enhancer and generate continuous, artifact-free, consistent novel views. We evaluate on the challenging DL3DV-10K dataset and demonstrate superior performance compared to state-of-the-art methods.
Yuci Han, Charles Toth, John E. Anderson +2
Dept. of Electrical and Computer Engineering, The Ohio State University · USACE ERDC GRL
Single-image novel view synthesis remains challenging because the underlying 3D geometry is highly ambiguous. Recent diffusion-based approaches produce plausible results, but they often struggle to preserve the geometric structure and spatial coherence of foreground objects. We present GenNVS, a framework for geometry-enhanced novel view synthesis via a disentangled 3D prior. Specifically, GenNVS models foreground objects and the background with 3D Gaussian Splatting and aligns them through a coarse-to-fine geometric optimization process to form a unified 3D scene. This scene conditions a video diffusion model through the proposed Dual-Stream Masking mechanism, which guides synthesis by jointly exploiting rendered validity masks and geometry-aware warping. Experimental results show that GenNVS performs favorably against recent methods in both visual quality and geometric accuracy, while naturally supporting flexible scene editing.