NAMVIS: Next-Scale Autoregressive Multi-View Image Synthesis

Ramil Khafizov; Ilya Statsenko; Ruslan Rakhimov; Artem Komarichev; Peter Wonka; Evgeny Burnaev

cs.CVOct 3, 2026

NAMVIS: Next-Scale Autoregressive Multi-View Image Synthesis

Authors: Ramil Khafizov, Ilya Statsenko, Ruslan Rakhimov, Artem Komarichev, Peter Wonka, Evgeny Burnaev

Organizations: Applied AI Institute · T-Tech · KAUST · AXXX

Abstract

Sparse-view novel view synthesis is a central problem in 3D content creation, but diffusion-based approaches remain limited by iterative denoising, making multi-view generation expensive at inference time. We introduce NAMVIS, a diffusion-free framework that reformulates multi-view image synthesis as geometry-conditioned next-scale autoregression. Instead of generating target views through repeated denoising, NAMVIS predicts discrete visual tokens through a small number of coarse-to-fine scale steps, while sampling all tokens within each scale and across target views in parallel. To anchor this generation process to explicit camera geometry, we propose Multi-scale Projective Pose Encoding, which injects source and target camera transformations into both target-view self-attention and source-to-target cross-attention at every resolution. NAMVIS further combines global conditioning with dense geometry-aware cross-attention, enabling the model to preserve source-view appearance while maintaining target-view consistency. Across Objaverse, GSO, and OmniObject3D, NAMVIS outperforms diffusion-based baselines in PSNR, SSIM, and LPIPS, while running over 3 times faster than the evaluated diffusion baselines under the same evaluation setting. These results suggest that geometry-conditioned next-scale autoregression is a promising and efficient alternative to diffusion for sparse-view multi-view synthesis. Additional qualitative results, videos, and resources are available at https://corl-team.github.io/namvis/

Figures & tables

Figure 1: NAMVIS performs sparse-view novel view synthesis from posed source images. Purple borders indicate input views; green borders indicate generated target views. NAMVIS generates target views with next-scale autoregressive decoding, producing sharp and more geometrically consistent results.

Figure 2: NAMVIS architecture overview. Source images are encoded by a frozen VQ-VAE into source features, which are pooled into a start-of-sequence token [SOS] and also serve as key–value pairs for dense cross-attention. The autoregressive transformer predicts target residual tokens scale by scale, with multi-scale ProPE injected into both self-attention (among target views) and cross-attention (from target to source). The frozen VQ-VAE decoder maps the final accumulated features to images.

Method	Objaverse (In-Domain)			GSO (Zero-Shot)			OmniObject3D (OO3D)			Time (s) $\downarrow$
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Time (s) $\downarrow$
Zero-1-to-3 Liu et al. (2023)	19.097	0.816	0.152	15.614	0.749	0.257	16.094	0.768	0.219	2.6
Zero-1-to-3 XL Deitke et al. (2023a)	19.341	0.827	0.136	15.936	0.747	0.236	16.320	0.767	0.209	2.6
Wonder3D Long et al. (2024)	15.801	0.821	0.209	14.158	0.777	0.285	14.290	0.786	0.243	2.0
SyncDreamer Liu et al. (2024)	19.368	0.826	0.156	16.961	0.781	0.246	16.766	0.793	0.207	16.6
EscherNet Kong et al. (2024)	19.078	0.816	0.147	18.878	0.799	0.156	17.767	0.780	0.167	2.1

Table 1: Quantitative comparison across Objaverse, GSO, and OmniObject3D.

Setup	LPIPS $\downarrow$	SSIM $\uparrow$	PSNR $\uparrow$	Time (s) $\downarrow$	VRAM (GB) $\downarrow$
2 $\rightarrow$ 6	0.0654	0.8845	24.49	1.53	4.3
6 $\rightarrow$ 2	0.0457	0.9017	26.07	0.84	4.1
6 $\rightarrow$ 6	0.0473	0.9015	26.12	1.56	5.0

Table 2: Scaling to larger source and target view counts on Objaverse.

Method	COLMAP Points $\uparrow$
Wonder3D Long et al. (2024)	172
SyncDreamer Liu et al. (2024)	151
EscherNet Kong et al. (2024)	214
NAMVIS 1B (Ours)	221

Table 3: COLMAP reconstructability comparison. Higher is better.

Method	Registered views	Coverage $\uparrow$
GT	168 / 240	70.0%
EscherNet	145 / 240	60.4%
NAMVIS	162 / 240	67.5%

Table 4: COLMAP camera-registration coverage across 30 scenes.

Figure 3: Qualitative comparison on OO3D Wu et al. (2023) .

Figure 4: Qualitative comparison on GSO Downs et al. (2022) .

Model Variant	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Geometry & Conditioning
w/ Plücker Rays (instead of ProPE)	20.260	0.899	0.142
Prefix Conditioning (instead of Cross-Attn)	14.539	0.797	0.427
Feature Aggregation
CLIP [SOS] (instead of VQ-VAE)	20.613	0.901	0.126
Learned Dense Features (instead of VQ-VAE)	20.886	0.901	0.134

Table 5: Ablation study. Each row replaces or removes one component of the full NAMVIS architecture. All variants are trained on a 10K subset of Objaverse-XL for 200 epochs.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Setup	NAMVIS				EscherNet Kong et al. [2024]
		Time (s)	LPIPS $\downarrow$	SSIM $\uparrow$	PSNR $\uparrow$	Time (s)	LPIPS $\downarrow$	SSIM $\uparrow$	PSNR $\uparrow$
GSO	1–1	0.60	0.1634	0.8104	19.0542	2.09	0.2924	0.7078	14.0517
	1–2	0.81	0.1668	0.8100	19.2163	2.16	0.2705	0.7334	14.5864
	1–3	0.94	0.1698	0.8110	19.1860	2.35	0.2683	0.7342	14.8349
	1–4	1.11	0.1633	0.8147	19.5188	2.78	0.2498	0.7498	15.6767
	2–1	0.62	0.0810	0.8603	23.0655	2.08	0.0927	0.8360	21.2183

Appendix

Table 6: Inference time and image quality for different source–target view configurations. Setup

a

–

b

denotes

a

source views and

b

target views.

Setup	Time (s)
NAMVIS	0.96
w/o KV-cache	1.06
w/o ProPE-cache	1.28

Appendix

Table 7: Inference time for a 3-to-3 setup for different caching configurations.

Figure 5: Varying number of source views from 1 to 4. When views covering previously unseen areas appear, the generated image quality improves.

Figure 6: Varying number of source views from 1 to 4. When views covering previously unseen areas appear, the generated image quality improves.

Figure 7: Target-view self-attention pattern between two generated views. For a queried token in the first target view, self-attention places high weight on corresponding regions in both generated target views, suggesting that target-target attention supports multi-view consistency.

Figure 8: Cross-attention pattern from a target image to two source images, layer 14. Cross-attention assigns high weights to source-image regions that are visually aligned with the queried target location.

Figure 9: Qualitative results of our method on OO3D Wu et al. [2023] .

Figure 10: Qualitative results of our method on GSO Downs et al. [2022] .

Figure 11: Visualization of image progression as new scales are summed up.

Figure 12: Visualization of self-attention pattern on layers 13-16, scales $4\times 4$ , $6\times 6$ , $8\times 8$ , $12\times 12$ , $16\times 16$ .

Figure 13: Visualization of cross-attention pattern on layers 13-16, scales $6\times 6$ , $8\times 8$ , $12\times 12$ , $16\times 16$ .

Figure 14: Visualization of cross-attention pattern on layers 13-16, scales $1\times 1$ , $2\times 2$ , $4\times 4$ .

Scene	Scene	Scene
chair_002	chair_009	chair_017
chair_026	backpack_003	backpack_016
backpack_022	backpack_033	helmet_004
helmet_011	helmet_013	kettle_002
kettle_008	kettle_017	toy_animals_007
toy_animals_025	toy_animals_049	toy_animals_071

Appendix

Table 8: Evaluation scenes from OmniObject3D (OO3D).

Scene	Scene
3D_Dollhouse_Swing	Netgear_Nighthawk_X6_AC3200_TriBand_Gigabit_Wireless_Router
Hasbro_Dont_Wake_Daddy_Board_Game	BUILD_A_ZOO
LEGO_5887_Dino_Defense_HQ	Melissa_Doug_Traffic_Signs_and_Vehicles
New_Super_Mario_BrosWii_Wii_Game	ALPHABET_AZ_GRADIENT
Crayola_Crayons_120_crayons	Tag_Dishtowel_Basket_Weave_Red_18_x_26
VANS_FIRE_ROASTED_VEGGIE_CRACKERS_GLUTEN_FREE	Office_Depot_HP_920XL_920_High_Yield_Black_and_Standard_CMY_Color_Ink_Cartridges

Appendix

Table 9: Evaluation scenes from GSO.

Scene	Scene	Scene
000074a334c541878360457c672b6c2e	001cfadfb9204424bccc45501ce6b90e	0023717f4f564cc99f4ded70db04f590
003199cc6ff2410cb2d8e6f8a9cbb163	003ebdf86df345d39dc166563229fb85	0051724d6efa42de84aaf8467629160f
0002c6eafa154e8bb08ebafb715a8d46	001d1b57e9df4273bede948b26429429	0023b3edbc114be188ca9d8f729dfaaf
003219b0bec442d29725847969e4b6bf	003ed2834130466da3a51bb7fdd6bde5	005f5630a54b442291aeb0a5d487353b
000b76f2b03e44e8ab44e1a1614be0f4	001fe8adb25a49d0b2650a5401dde019	0025c5e2333949feb1db259d4ff08dbe
0032696f5871429fbd0549d9628f812c	0046f208ef8d4988ba7bb9d297f29ec7	00616f328a8b4b5a8e689f61e70758b6

Appendix

Table 10: Evaluation scenes from Objaverse.

Explore similar work

VGGT-Diff: Visual Geometry Meets Diffusion for Sparse-View Novel View Synthesis
Sep 27, 2026Kangjie Chen, Xiangyu Li, Dongbin Zhang +12Novel View Synthesis Conditional Diffusion Models
XPeng Motors · The Chinese University of Hong Kong · Tsinghua University
BetterScene: 3D Scene Synthesis with Representation-Aligned Generative Model
Feb 26, 2026Yuci Han, Charles Toth, John E. Anderson +2Novel View Synthesis Sparse-View 3D Reconstruction
Dept. of Electrical and Computer Engineering, The Ohio State University · USACE ERDC GRL
GenNVS: Geometry-enhanced Novel View Synthesis via Disentangled 3D Prior
Sep 28, 2026Yajiao Xiong, Youyu Luan, Xiaoyu Zhou +1Video Diffusion Models Novel View Synthesis
Peking University

Sep 27, 2026J/K move · Enter open · S save

VGGT-Diff: Visual Geometry Meets Diffusion for Sparse-View Novel View Synthesis

Kangjie Chen, Xiangyu Li, Dongbin Zhang, Chaoda Zheng+11

XPeng Motors · The Chinese University of Hong Kong · Tsinghua University

Novel View Synthesis Conditional Diffusion Models Multi-View Consistency

Open paper AskPDF arXiv

We present VGGT-Diff, a geometry-routed multi-view diffusion model for sparse-view novel view synthesis. Existing novel view synthesis (NVS) methods face a fundamental trade-off: reconstruction-based approaches preserve observed geometry but struggle to synthesize unseen regions, while diffusion-based methods provide strong generative priors yet rely on implicit source-to-query correspondence. VGGT-Diff bridges these regimes by routing visual geometry latents from VGGT-Ω into a pretrained video diffusion model. Each visual token is associated with a 3D point and confidence, then transformed into query-aligned latent conditions through a confidence-aware Visual Geometry Router (VGR) that preserves front and back surface evidence. These conditions guide joint target-view denoising, while Point-Track Residual Consistency (PTRC) regularizes predicted-clean residuals along reliable 3D tracks, improving multi-view stability. We further introduce robust geometry conditioning, combining training-time regularization with inference-time guidance for improved robustness. Experiments show competitive or state-of-the-art performance across interpolation and extrapolation under different viewpoint difficulties. Our code is available at https://github.com/chenkangjie1123/VGGT-Diff.

Abstract

Explore similar work

VGGT-Diff: Visual Geometry Meets Diffusion for Sparse-View Novel View Synthesis

BetterScene: 3D Scene Synthesis with Representation-Aligned Generative Model

GenNVS: Geometry-enhanced Novel View Synthesis via Disentangled 3D Prior