You Only Need One Stage: Novel-View Synthesis From A Single Blind Face Image

Taoyue Wang; Xiang Zhang; Xiaotian Li; Huiyuan Yang; Lijun Yin

cs.CVMar 1, 2026

You Only Need One Stage: Novel-View Synthesis From A Single Blind Face Image

Authors: Taoyue Wang, Xiang Zhang, Xiaotian Li, Huiyuan Yang, Lijun Yin

Organizations: School of Computing, State University of New York at Binghamton · Department of Computer Science, Missouri University of Science and Technology

Abstract

We propose a novel one-stage method, NVB-Face, for generating consistent Novel-View images directly from a single Blind Face image. Existing approaches to novel-view synthesis for objects or faces typically require a high-resolution RGB image as input. When dealing with degraded images, the conventional pipeline follows a two-stage process: first restoring the image to high resolution, then synthesizing novel views from the restored result. However, this approach is highly dependent on the quality of the restored image, often leading to inaccuracies and inconsistencies in the final output. To address this limitation, we extract single-view features directly from the blind face image and introduce a feature manipulator that transforms these features into 3D-aware, multi-view latent representations. Leveraging the powerful generative capacity of a diffusion model, our framework synthesizes high-quality, consistent novel-view face images. Experimental results show that our method significantly outperforms traditional two-stage approaches in both consistency and fidelity.

Figures & tables

Figure 1: We compare our method with typical two-stage pipelines, such as CodeFormer ( Zhou et al. 2022 ) + PanoHead-PTI ( An et al. 2023 ) , which first restore the degraded image and then synthesize novel views. It is evident that when the restoration stage fails to recover accurate details, these errors are further amplified during the novel view synthesis, leading to results that deviate significantly from the original identity and appearance. In contrast, our method generates novel views in a single stage directly from the low-quality input. This end-to-end design suppresses error accumulation, resulting in more reliable and faithful novel-view images.

Figure 2: An overview of the proposed NVB-Face architecture. (a) Our first training step focuses solely image restoration. (b) In the second training step, we update only the parameters of the newly introduced modules (highlighted in dark green), keeping the rest of the network frozen. After training, this two-step process forms our complete inference pipeline.

Figure 3: Qualitative comparisons on NeRSemble ( Kirschstein et al. 2023 ) dataset. As shown in our results, this end-to-end strategy achieves superior perceptual quality and preserves identity and expression information more effectively than two-stage methods, minimizing the loss of critical facial attributes.

Figure 4: Qualitative comparisons on LFW-Test ( Huang et al. 2008 ) dataset. Our method produces consistently stable results across varying levels of input degradation. Compared to other approaches, our generated images preserve the most information from the original input and exhibit higher visual realism, even under severe degradation.

Table 1: (a) Quantitative comparisons with other methods on NeRSemble dataset (novel-view synthesis) / CelebA-Test (single-view reconstruction). (b) Ablation study on NeRSemble dataset to assess the effectiveness of the feature loss in our framework. Specifically, we compare the quantitative performance of our model with and without feature loss.

Figure 5: Qualitative ablation study on LFW-Test dataset to compare our method with and without feature loss.

Figure 6: Feature correction of our training step 2.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Architecture of our Time-aware Image Encoder

Figure 8: Qualitative comparison among CodeFormer-based two-stage method, DiffBIR-based method and ours on LFW-Test dataset

	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	DISTS $\downarrow$	FID $\downarrow$	MUSIQ $\uparrow$	NIQE $\downarrow$	C-IQA $\uparrow$	M-IQA $\uparrow$
CodeFormer	20.98	0.6174	0.4750	0.2636	61.44	66.06	5.6992	0.5608	0.4775
DiffBIR	22.00	0.6000	0.4631	0.2340	32.54	75.61	6.2855	0.7895	0.6256
Ours	20.05	0.5574	0.4969	0.2133	18.43	74.71	5.2747	0.7431	0.6536

Appendix

Table 2: Comparison with other classical blind face restoration (BFR) methods. Bold indicates the best result, and underlined indicates the second-best.

	SSIM $\uparrow$	LPIPS $\downarrow$	DISTS $\downarrow$	FID $\downarrow$	ID $\uparrow$	POSE $\downarrow$
PanoHead-PTI	0.63( 0.73 )	0.54( 0.48 )	0.27( 0.26 )	85.31( 83.19 )	0.30( 0.32 )	0.0484( 0.0446 )
GOAE	0.62( 0.73 )	0.50( 0.45 )	0.27( 0.25 )	108.65( 90.00 )	0.27 ( 0.27 )	0.0392 (0.0425)
TriplaneNet	0.70( 0.75 )	0.52( 0.51 )	0.29 (0.32)	136.21( 111.84 )	0.29( 0.30 )	0.0484( 0.0456 )
Ours	0.78	0.17	0.10	5.67	0.77	0.0084

Appendix

Table 3: Quantitative comparisons with DiffBIR-based two-stage methods on NeRSemble dataset. Values in parentheses indicate results based on the CodeFormer restoration. Underlined values represent the better result between the CodeFormer-based and DiffBIR-based variants.

Figure 9: Limitation of our model

Figure 10: More novel-view results from easy cases

Figure 11: More novel-view results from hard cases

Novel View Synthesis

Novel View Synthesis creates images of a scene from camera viewpoints not present in the input, using learned geometry and appearance models.

See all 289 papers

Photorealistic Novel View Synthesis of Human Faces using Next-Scale Transformers
Aug 24, 2026Federico Stella, Fei Jiang, Zhongshi Jiang +4Novel View Synthesis Multi-View Image Synthesis
École polytechnique fédérale de Lausanne, Switzerland · Meta
NeoMap: Training-free Novel-View Synthesis from Single Images and Videos
Jul 2, 2026Jinxi Li, Tianyi Zhang, Yafei Yang +4Multi-View Consistency Video Diffusion Models
Shenzhen Research Institute, The Hong Kong Polytechnic University · vLAR Group, The Hong Kong Polytechnic University
OrbitNVS: Harnessing Video Diffusion Priors for Novel View Synthesis
Mar 20, 2026Jinglin Liang, Zijian Zhou, Rui Huang +2Novel View Synthesis Single-View 3D Reconstruction
South China University of Technology · Agentic Labs, XGTech · King’s College London +1

Aug 24, 2026J/K move · Enter open · S save

Photorealistic Novel View Synthesis of Human Faces using Next-Scale Transformers

Federico Stella, Fei Jiang, Zhongshi Jiang, Zohar Barzelay+3

École polytechnique fédérale de Lausanne, Switzerland · Meta

Novel View Synthesis Multi-View Image Synthesis Autoregressive Image Generation

Open paper AskPDF arXiv

Photorealistic novel view synthesis of people remains challenging at high spatial resolutions and across multiple target cameras, where preserving identity, fine appearance details, and geometric coherence is critical. We build on the next-scale autoregressive paradigm and adapt it for human-centric view synthesis by enabling higher image resolutions, multi-view outputs and stronger cross-view consistency in a single forward pass. We train on a synthetic dataset of human faces spanning diverse identities and apparel. Contrary to diffusion models, this paradigm does not need 2D pre-training and, thanks to its next-scale architecture, it benefits from lower-resolution, general-purpose pre-trainings, with the full-sized purpose-specific images being used only in the last training stages. This enables our architecture to converge with a smaller amount of purpose-specific training data, allowing us to use a smaller but more realistic training dataset. The resulting model produces sharp and realistic views, with the option to synthesize multiple novel viewpoints simultaneously for improved agreement across views. Empirically, we observe gains in perceptual fidelity and cross-view coherence on human subjects, demonstrating that next-scale autoregression is an effective backbone for scalable, multi-output human view synthesis. We also couple our pipeline with an existing transformer-based model for pixel-aligned 3D gaussian lifting from multi-view facial inputs, resulting in accurate and photorealistic 3D models of human faces.

Abstract

Novel View Synthesis

Explore similar work

Photorealistic Novel View Synthesis of Human Faces using Next-Scale Transformers

NeoMap: Training-free Novel-View Synthesis from Single Images and Videos

OrbitNVS: Harnessing Video Diffusion Priors for Novel View Synthesis