Real-Time Human Frontal View Synthesis from a Single Image

Fangyu Lin; Yingdong Hu; Lunjie Zhu; Zhening Liu; Yushi Huang; Zehong Lin; Jun Zhang

cs.CVMar 16, 2026

Real-Time Human Frontal View Synthesis from a Single Image

Authors: Fangyu Lin, Yingdong Hu, Lunjie Zhu, Zhening Liu, Yushi Huang, Zehong Lin, Jun Zhang

Organizations: HKUST, Hong Kong SAR, China

Abstract

Photorealistic human novel view synthesis from a single image is crucial for democratizing immersive 3D telepresence, eliminating the need for complex multi-camera setups. However, current rendering-centric methods prioritize visual fidelity over explicit geometric understanding and struggle with intricate regions like faces and hands, leading to temporal instability. Meanwhile, human-centric frameworks suffer from memory bottlenecks since they typically rely on an auxiliary model to provide informative structural priors for geometric modeling, which limits real-time performance. To address these challenges, we propose PrismMirror, a geometry-guided framework for instant frontal view synthesis from a single image. By avoiding external geometric modeling and focusing on frontal view synthesis, our model optimizes visual integrity for telepresence. Specifically, PrismMirror introduces a novel cascade learning strategy that enables coarse-to-fine geometric feature learning. It first directly learns coarse geometric features, such as SMPL-X meshes and point clouds, and then refines textures through rendering supervision. To achieve real-time efficiency, we distill this unified framework into a lightweight linear attention model. Notably, PrismMirror is the first monocular human frontal view synthesis model that achieves real-time inference at 24 FPS, significantly outperforming previous methods in both visual authenticity and structural accuracy.

Figures & tables

Figure 1: Real-time high-fidelity human frontal view synthesis. PrismMirror achieves a $5\times$ inference speedup ( $\sim$ 42 ms vs. 206.4 ms) over HumanRAM, while maintaining superior visual quality and reaching real-time frame rates of 24+ FPS.

Figure 2: Overview of the PrismMirror architecture. The framework operates through three cascaded stages: encoding global context, injecting explicit geometric priors (SMPL-X and point clouds), and decoding into NVS or 3DGS (accelerated by a progressive linear attention distillation strategy).

Figure 3: Visualization of geometry feature injection. By generating explicit spatial priors (point clouds and SMPL-X) inside the model, our architecture effectively anchors pure data-driven features to capture precise body topology and high-frequency details.

Figure 5: Qualitative comparisons on THuman2.1 and THumanSit. PrismMirror synthesizes sharper details in high-frequency regions (e.g., faces and hands) compared to baselines, successfully avoiding severe floating artifacts.

res=512	THuman2.1			THumanSit			2K2K			FPS $\uparrow$
res=512	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FPS $\uparrow$
LHM [ 36 ]	24.75	0.9357	0.0623	22.80	0.9081	0.0913	21.06	0.8767	0.0943	0.66
AnySplat [ 14 ]	15.44	0.7365	0.2704	12.98	0.6141	0.3605	14.50	0.6847	0.2860	4.66
WorldMirror [ 27 ]	23.92	0.3289	0.0853	19.76	0.3008	0.1549	20.81	0.3182	0.0986	6.18
LVSM [ 15 ]	26.46	0.9452	0.0603	22.68	0.8851	0.1208	21.78	0.8897	0.0922	3.10
HumanRAM [ 59 ]	27.96	0.9551	0.0371	23.49	0.9010	0.0814	26.81	0.9369	0.0343	4.84

Table 1: Quantitative comparison on static datasets. Best results are in bold and second best are underlined .

Figure 6: Qualitative comparisons on 2K2K.

Figure 7: Spatiotemporal consistency evaluation on ActorsHQ. Unlike baselines that suffer from temporal flickering, our explicit geometric anchors ensure highly stable rendering across continuous motion sequences.

res=512	ActorsHQ
res=512	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
WorldMirror [ 27 ]	21.91	0.3944	0.0980
HumanRAM [ 59 ]	27.52	0.9466	0.0380
Ours	29.62	0.9570	0.0307

Table 2: Comparison on video datasets. Best results are in bold .

Table 10

Figure 8: Zero-shot cross-domain evaluation MVHumanNet. PrismMirror demonstrates exceptional identity preservation and structural fidelity on unseen data with non-training camera settings.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 9: Evaluation of spatial consistency in different directions. Left: Results for 3 IDs during horizontal camera rotation, moving from right to left (from $-30^{\circ}$ to $30^{\circ}$ ). Right: Results for 3 IDs during vertical camera rotation, moving from top to bottom (from $-30^{\circ}$ to $30^{\circ}$ ). In both sets, images are rendered at $5^{\circ}$ intervals. PrismMirror synthesizes sharper details in high-frequency regions, such as faces and hands, compared to baselines, successfully avoiding severe floating artifacts.

Figure 10: Evaluation of temporal consistency on ActorsHQ. Here, we present long-sequence results for Actor01, Actor02, and Actor04 from different viewpoints. Even with significant poses, our method still maintains temporal consistency and produces reasonable NVS results.

Figure 11: Zero-shot evaluation results on in-the-wild cases. To validate the practical applicability of our method, we conduct evaluations using in-the-wild data, including Internet images and real-world photographs. The visually appealing results demonstrate the strong generalization capability and robustness of our approach in unseen and complex scenarios.

Figure 12: Point cloud and human mesh from the encoding stage. This figure illustrates the structural learning capability of our 12-layer transformer encoder. By comparing the predicted point clouds and SMPL-X meshes with their respective ground truth (GT), we observe a high degree of alignment. This indicates that our model effectively extracts precise structural information, ensuring strong geometric priors for the subsequent feature injection stage.

Figure 13: Failure cases. We observe severe distortions in scenarios involving extreme poses or when the subject is in close proximity to the camera. These artifacts are mainly due to OOD issues, which arise from the limited diversity and coverage of such extreme cases in our current training dataset.

Explore similar work

Photorealistic Novel View Synthesis of Human Faces using Next-Scale Transformers
Aug 24, 2026Federico Stella, Fei Jiang, Zhongshi Jiang +4Novel View Synthesis Autoregressive Image Generation
École polytechnique fédérale de Lausanne, Switzerland · Meta
Tele360: Real-Time Feed-Forward Human Reconstruction from Sparse Unposed Cameras
Sep 14, 2026Hanzhang Tu, Zhanfeng Liao, Wei Min +23D Gaussian Splatting Camera Pose Estimation
Tsinghua University, Beijing, China · Shadow AI, Beijing, China
PRISM: Feed-Forward Single-Image 3D Reconstruction via Geometric Warp-Residual Modeling
Jun 24, 2026Zhijie Zheng, Xinhao Xiang, Jiawei ZhangNovel View Synthesis Single-View 3D Reconstruction
University of California, Davis · CA 95616, USA

Aug 24, 2026J/K move · Enter open · S save

Photorealistic Novel View Synthesis of Human Faces using Next-Scale Transformers

Federico Stella, Fei Jiang, Zhongshi Jiang, Zohar Barzelay+3

École polytechnique fédérale de Lausanne, Switzerland · Meta

Novel View Synthesis Autoregressive Image Generation

Open paper AskPDF arXiv

Photorealistic novel view synthesis of people remains challenging at high spatial resolutions and across multiple target cameras, where preserving identity, fine appearance details, and geometric coherence is critical. We build on the next-scale autoregressive paradigm and adapt it for human-centric view synthesis by enabling higher image resolutions, multi-view outputs and stronger cross-view consistency in a single forward pass. We train on a synthetic dataset of human faces spanning diverse identities and apparel. Contrary to diffusion models, this paradigm does not need 2D pre-training and, thanks to its next-scale architecture, it benefits from lower-resolution, general-purpose pre-trainings, with the full-sized purpose-specific images being used only in the last training stages. This enables our architecture to converge with a smaller amount of purpose-specific training data, allowing us to use a smaller but more realistic training dataset. The resulting model produces sharp and realistic views, with the option to synthesize multiple novel viewpoints simultaneously for improved agreement across views. Empirically, we observe gains in perceptual fidelity and cross-view coherence on human subjects, demonstrating that next-scale autoregression is an effective backbone for scalable, multi-output human view synthesis. We also couple our pipeline with an existing transformer-based model for pixel-aligned 3D gaussian lifting from multi-view facial inputs, resulting in accurate and photorealistic 3D models of human faces.

Abstract

Explore similar work

Photorealistic Novel View Synthesis of Human Faces using Next-Scale Transformers

Tele360: Real-Time Feed-Forward Human Reconstruction from Sparse Unposed Cameras

PRISM: Feed-Forward Single-Image 3D Reconstruction via Geometric Warp-Residual Modeling