Photorealistic novel view synthesis of people remains challenging at high spatial resolutions and across multiple target cameras, where preserving identity, fine appearance details, and geometric coherence is critical. We build on the next-scale autoregressive paradigm and adapt it for human-centric view synthesis by enabling higher image resolutions, multi-view outputs and stronger cross-view consistency in a single forward pass. We train on a synthetic dataset of human faces spanning diverse identities and apparel. Contrary to diffusion models, this paradigm does not need 2D pre-training and, thanks to its next-scale architecture, it benefits from lower-resolution, general-purpose pre-trainings, with the full-sized purpose-specific images being used only in the last training stages. This enables our architecture to converge with a smaller amount of purpose-specific training data, allowing us to use a smaller but more realistic training dataset. The resulting model produces sharp and realistic views, with the option to synthesize multiple novel viewpoints simultaneously for improved agreement across views. Empirically, we observe gains in perceptual fidelity and cross-view coherence on human subjects, demonstrating that next-scale autoregression is an effective backbone for scalable, multi-output human view synthesis. We also couple our pipeline with an existing transformer-based model for pixel-aligned 3D gaussian lifting from multi-view facial inputs, resulting in accurate and photorealistic 3D models of human faces.
Figures & tables
Figure 1: Face reconstruction from single view: our method achieves more realistic results than competing methods. † Variant of our method trained to handle multiple loosely-posed inputs, it achieves even better results thanks to the additional inputs.
Figure 2: Our architecture : input images are encoded into multi-scale features by the VQ-VAE encoder and used as inputs for the transformer. They are also embedded into a global conditioning along with the camera-pose RT matrix, used both in the AdaLN layers and the SOS tokens. The transformer outputs one scale at a time for every output view simultaneously, up to the final scale. The output features are decoded by the VQ-VAE decoder to produce the final RGB images, which can be lifted to 3D gaussians by an off-the-shelf GS-LRM model. During training, previous scales are teacher-forced, while during inference the model is auto-regressive. The architecture is the same for all our models, with the only difference being the number of input and output views.
Figure 3: Multi-scale VQ-VAE outputs for different resolutions and scale configurations. Notice that the small details appear at the last 2-3 scales, regardless of the resolution and the number of scales.
PSNR ↑
SSIM ↑
LPIPS ↓
DS ↓
ArcFace ↑
SI (PSGS) - I stage [ 41 ]
20.15
0.8640
0.2612
0.02606
0.6761
SI (PSGS) - II stage [ 41 ]
20.16
0.8614
0.2351
0.02297
0.7403
FaceLift-NVS [ 24 ]
17.76
0.7942
0.2785
0.04531
0.6554
FaceLift-NVS (PSGS) [ 24 ]
16.75
0.7900
0.3206
0.02262
0.6680
Ours
22.29
0.8806
0.1905
0.02191
0.6944
Ours MO
22.40
0.8752
0.1927
0.01729
0.7069
Table 1: Quantitative evaluation on PSGS [ 11 ] validation set for NVS models (6 canonical views). Best in bold, second best in italics.
Figure 4: Qualitative results on PSGS [ 11 ] validation set on the 6 canonical views.
Figure 5: Qualitative results on PSGS [ 11 ] validation set on the 32 turntable views.
PSNR ↑
SSIM ↑
LPIPS ↓
DS ↓
ArcFace ↑
SI (PSGS) - I stage [ 41 ]
18.46
0.8548
0.2755
0.03257
0.7108
SI (PSGS) - II stage [ 41 ]
18.32
0.8488
0.2549
0.03184
0.7596
FaceLift [ 24 ]
13.28
0.7791
0.3103
0.1008
0.7121
FaceLift (PSGS) [ 24 ]
15.41
0.8377
0.2668
0.07339
0.7357
Ours + [ 11 ]
20.78
0.8805
0.2248
0.02440
0.7520
Ours MO + [ 11 ]
20.82
0.8772
0.2249
0.02319
0.7608
Table 2: Quantitative evaluation on PSGS [ 11 ] validation set of the full image-to-3D pipelines (32 turntable views). Best in bold, second best in italics.
PSNR ↑
SSIM ↑
LPIPS ↓
DS ↓
ArcFace ↑
SI (PSGS) [ 41 ]
13.06
0.7736
0.2911
0.06242
0.6689
FaceLift [ 24 ]
14.49
0.8038
0.2546
0.03482
0.7699
FaceLift (PSGS) [ 24 ]
14.51
0.8146
0.2489
0.03584
0.5837
Ours + [ 11 ]
15.34
0.8201
0.2357
0.03616
0.6275
Ours MO + [ 11 ]
15.24
0.8197
0.2448
0.03385
0.6405
Ours MI-MO + [ 11 ]
15.67
0.8209
0.2350
0.04187
0.6251
Table 3: Quantitative evaluation on AVA-256, 10 subjects, 11 views, as defined in [ 24 ] . Best in bold, second best in italics.
Figure 6: Qualitative results on AVA-256 [ 25 ] for subject 20210901–0833–LAS440
Depth
Resolution
Checkpoint
1st stage
2nd stage
VQ-VAE
PSNR ↑
ArcFace ↑
d12
256
AV [ 48 ]
default
12.02
17.01
d12
256
AV [ 48 ]
✓
default
19.77
39.20
d12
256
Scratch
✓
default
19.68
34.87
d36
512
VAR [ 43 ]
✓
default
16.72
38.17
d12
512 K=12
AV [ 48 ]
✓
default
15.70
33.71
d12
512
AV [ 48 ]
✓
default
21.43
57.75
Table 4: Ablation study of the NVS-stage of our pipeline on PSGS [ 11 ] validation set (6 canonical views) for single-input single-output models. K=10 unless otherwise specified, and using around half the training steps of the final model.
Figure A.1 : Qualitative results of our method on the PSGS [ 11 ] dataset for human body reconstruction. We show the canonical views produced by our method, compared to the GT and the input views.
Figure A.3 : Example object views from the SS3D dataset.
Figure A.4 : Multi-scale VQ-VAE finetuning results. We show the reconstructions from the base and finetuned VQ-VAEs at both 256 and 512 resolutions, compared to the GT, showing small but noticeable improvements in reconstruction quality after finetuning.
Figure A.5 : Qualitative results on PSGS [ 11 ] validation set on the 6 canonical views for the NVS part of each model.
Figure A.6 : Qualitative results on AVA-256 [ 25 ] for subject 20220310–1128–ZSC414
Sparse-view novel view synthesis is a central problem in 3D content creation, but diffusion-based approaches remain limited by iterative denoising, making multi-view generation expensive at inference time. We introduce NAMVIS, a diffusion-free framework that reformulates multi-view image synthesis as geometry-conditioned next-scale autoregression. Instead of generating target views through repeated denoising, NAMVIS predicts discrete visual tokens through a small number of coarse-to-fine scale steps, while sampling all tokens within each scale and across target views in parallel. To anchor this generation process to explicit camera geometry, we propose Multi-scale Projective Pose Encoding, which injects source and target camera transformations into both target-view self-attention and source-to-target cross-attention at every resolution. NAMVIS further combines global conditioning with dense geometry-aware cross-attention, enabling the model to preserve source-view appearance while maintaining target-view consistency. Across Objaverse, GSO, and OmniObject3D, NAMVIS outperforms diffusion-based baselines in PSNR, SSIM, and LPIPS, while running over 3 times faster than the evaluated diffusion baselines under the same evaluation setting. These results suggest that geometry-conditioned next-scale autoregression is a promising and efficient alternative to diffusion for sparse-view multi-view synthesis. Additional qualitative results, videos, and resources are available at https://corl-team.github.io/namvis/
Ramil Khafizov, Ilya Statsenko, Ruslan Rakhimov +3
Feed-forward novel view synthesis methods achieve strong generalization from posed multi-view inputs, but scaling them to large input view sets remains challenging. Transformer-based approaches that jointly process all input-view tokens incur rapidly increasing computation and memory as the number of views grows, while simple view subsampling discards potentially useful observations. We introduce Anchor-Integrated Multi-View Synthesis (AIMS), a scalable framework that decouples the number of available observations from the number of views processed by the global synthesis model. AIMS selects a fixed set of spatially distributed anchor views using farthest point sampling, groups nearby observations around each anchor, and uses a lightweight learnable integrator to fuse their information into enriched anchor representations. This allows additional observations to contribute to synthesis while keeping the downstream global view budget fixed. Evaluations on RealEstate10K and ScanNet demonstrate a favorable quality--efficiency trade-off against transformer-based and Gaussian-based baselines. AIMS achieves 29.41 dB and 17.73 dB PSNR on the two datasets, respectively, with rendering averaging 7.24 ms per view.
Photorealistic human novel view synthesis from a single image is crucial for democratizing immersive 3D telepresence, eliminating the need for complex multi-camera setups. However, current rendering-centric methods prioritize visual fidelity over explicit geometric understanding and struggle with intricate regions like faces and hands, leading to temporal instability. Meanwhile, human-centric frameworks suffer from memory bottlenecks since they typically rely on an auxiliary model to provide informative structural priors for geometric modeling, which limits real-time performance. To address these challenges, we propose PrismMirror, a geometry-guided framework for instant frontal view synthesis from a single image. By avoiding external geometric modeling and focusing on frontal view synthesis, our model optimizes visual integrity for telepresence. Specifically, PrismMirror introduces a novel cascade learning strategy that enables coarse-to-fine geometric feature learning. It first directly learns coarse geometric features, such as SMPL-X meshes and point clouds, and then refines textures through rendering supervision. To achieve real-time efficiency, we distill this unified framework into a lightweight linear attention model. Notably, PrismMirror is the first monocular human frontal view synthesis model that achieves real-time inference at 24 FPS, significantly outperforming previous methods in both visual authenticity and structural accuracy.