Photorealistic human novel view synthesis from a single image is crucial for democratizing immersive 3D telepresence, eliminating the need for complex multi-camera setups. However, current rendering-centric methods prioritize visual fidelity over explicit geometric understanding and struggle with intricate regions like faces and hands, leading to temporal instability. Meanwhile, human-centric frameworks suffer from memory bottlenecks since they typically rely on an auxiliary model to provide informative structural priors for geometric modeling, which limits real-time performance. To address these challenges, we propose PrismMirror, a geometry-guided framework for instant frontal view synthesis from a single image. By avoiding external geometric modeling and focusing on frontal view synthesis, our model optimizes visual integrity for telepresence. Specifically, PrismMirror introduces a novel cascade learning strategy that enables coarse-to-fine geometric feature learning. It first directly learns coarse geometric features, such as SMPL-X meshes and point clouds, and then refines textures through rendering supervision. To achieve real-time efficiency, we distill this unified framework into a lightweight linear attention model. Notably, PrismMirror is the first monocular human frontal view synthesis model that achieves real-time inference at 24 FPS, significantly outperforming previous methods in both visual authenticity and structural accuracy.
Figures & tables
Figure 1: Real-time high-fidelity human frontal view synthesis. PrismMirror achieves a 5× inference speedup ( ∼ 42 ms vs. 206.4 ms) over HumanRAM, while maintaining superior visual quality and reaching real-time frame rates of 24+ FPS.
Figure 2: Overview of the PrismMirror architecture. The framework operates through three cascaded stages: encoding global context, injecting explicit geometric priors (SMPL-X and point clouds), and decoding into NVS or 3DGS (accelerated by a progressive linear attention distillation strategy).
Figure 3: Visualization of geometry feature injection. By generating explicit spatial priors (point clouds and SMPL-X) inside the model, our architecture effectively anchors pure data-driven features to capture precise body topology and high-frequency details.
Figure 4: Progressive distillation.
Figure 5: Qualitative comparisons on THuman2.1 and THumanSit. PrismMirror synthesizes sharper details in high-frequency regions (e.g., faces and hands) compared to baselines, successfully avoiding severe floating artifacts.
res=512
THuman2.1
THumanSit
2K2K
FPS ↑
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
LHM [ 36 ]
24.75
0.9357
0.0623
22.80
0.9081
0.0913
21.06
0.8767
0.0943
0.66
AnySplat [ 14 ]
15.44
0.7365
0.2704
12.98
0.6141
0.3605
14.50
0.6847
0.2860
4.66
WorldMirror [ 27 ]
23.92
0.3289
0.0853
19.76
0.3008
0.1549
20.81
0.3182
0.0986
6.18
LVSM [ 15 ]
26.46
0.9452
0.0603
22.68
0.8851
0.1208
21.78
0.8897
0.0922
3.10
HumanRAM [ 59 ]
27.96
0.9551
0.0371
23.49
0.9010
0.0814
26.81
0.9369
0.0343
4.84
Table 1: Quantitative comparison on static datasets. Best results are in bold and second best are underlined .
Figure 6: Qualitative comparisons on 2K2K.
Figure 7: Spatiotemporal consistency evaluation on ActorsHQ. Unlike baselines that suffer from temporal flickering, our explicit geometric anchors ensure highly stable rendering across continuous motion sequences.
res=512
ActorsHQ
PSNR ↑
SSIM ↑
LPIPS ↓
WorldMirror [ 27 ]
21.91
0.3944
0.0980
HumanRAM [ 59 ]
27.52
0.9466
0.0380
Ours
29.62
0.9570
0.0307
Table 2: Comparison on video datasets. Best results are in bold .
Table 10
Figure 8: Zero-shot cross-domain evaluation MVHumanNet. PrismMirror demonstrates exceptional identity preservation and structural fidelity on unseen data with non-training camera settings.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9: Evaluation of spatial consistency in different directions. Left: Results for 3 IDs during horizontal camera rotation, moving from right to left (from −30∘ to 30∘ ). Right: Results for 3 IDs during vertical camera rotation, moving from top to bottom (from −30∘ to 30∘ ). In both sets, images are rendered at 5∘ intervals. PrismMirror synthesizes sharper details in high-frequency regions, such as faces and hands, compared to baselines, successfully avoiding severe floating artifacts.
Figure 10: Evaluation of temporal consistency on ActorsHQ. Here, we present long-sequence results for Actor01, Actor02, and Actor04 from different viewpoints. Even with significant poses, our method still maintains temporal consistency and produces reasonable NVS results.
Figure 11: Zero-shot evaluation results on in-the-wild cases. To validate the practical applicability of our method, we conduct evaluations using in-the-wild data, including Internet images and real-world photographs. The visually appealing results demonstrate the strong generalization capability and robustness of our approach in unseen and complex scenarios.
Figure 12: Point cloud and human mesh from the encoding stage. This figure illustrates the structural learning capability of our 12-layer transformer encoder. By comparing the predicted point clouds and SMPL-X meshes with their respective ground truth (GT), we observe a high degree of alignment. This indicates that our model effectively extracts precise structural information, ensuring strong geometric priors for the subsequent feature injection stage.
Figure 13: Failure cases. We observe severe distortions in scenarios involving extreme poses or when the subject is in close proximity to the camera. These artifacts are mainly due to OOD issues, which arise from the limited diversity and coverage of such extreme cases in our current training dataset.
Photorealistic novel view synthesis of people remains challenging at high spatial resolutions and across multiple target cameras, where preserving identity, fine appearance details, and geometric coherence is critical. We build on the next-scale autoregressive paradigm and adapt it for human-centric view synthesis by enabling higher image resolutions, multi-view outputs and stronger cross-view consistency in a single forward pass. We train on a synthetic dataset of human faces spanning diverse identities and apparel. Contrary to diffusion models, this paradigm does not need 2D pre-training and, thanks to its next-scale architecture, it benefits from lower-resolution, general-purpose pre-trainings, with the full-sized purpose-specific images being used only in the last training stages. This enables our architecture to converge with a smaller amount of purpose-specific training data, allowing us to use a smaller but more realistic training dataset. The resulting model produces sharp and realistic views, with the option to synthesize multiple novel viewpoints simultaneously for improved agreement across views. Empirically, we observe gains in perceptual fidelity and cross-view coherence on human subjects, demonstrating that next-scale autoregression is an effective backbone for scalable, multi-output human view synthesis. We also couple our pipeline with an existing transformer-based model for pixel-aligned 3D gaussian lifting from multi-view facial inputs, resulting in accurate and photorealistic 3D models of human faces.
Federico Stella, Fei Jiang, Zhongshi Jiang +4
École polytechnique fédérale de Lausanne, Switzerland · Meta
Live free-viewpoint visualization of real humans is critical for immersive communication and interactive digital experiences. Existing methods either rely on computationally expensive optimization or require calibrated cameras and low-resolution inputs, making real-time high-resolution deployment impractical. In this work, we present Tele360, the first real-time feed-forward system for dynamic human reconstruction and live free-viewpoint visualization from sparse, unposed RGB streams. Our system jointly estimates camera poses and reconstructs a dynamic 3D Gaussian representation for each time instance in a single forward pass. To achieve this, we start by designing a lightweight sparsity-aware multi-view transformer backbone that tokenizes foreground human regions while preserving global context through a shared scene token. We then employ a fully transformer-based Gaussian decoder to mitigate convolution-induced over-smoothing while keeping decoding sparse and efficient. In addition, we introduce a hybrid feature pyramid that injects multi-scale appearance cues into geometry prediction. We further introduce a lightweight differentiable Levenberg-Marquardt camera refinement layer to enhance multi-view consistency and geometric alignment. Moreover, to stabilize learning under sparse, unposed inputs, we transfer multi-view geometry priors from a large visual-geometry foundation model via teacher-student distillation. Finally, the predicted Gaussian maps are streamed with video codecs to remote devices for interactive free-viewpoint rendering. Extensive experiments show that Tele360 achieves state-of-the-art visual quality on studio benchmarks while supporting real-time 2K input-to-rendering at over 25 FPS on a single consumer GPU. Additional captured sequences illustrate its performance across varied subjects, clothing, and motions under our multi-camera setup.
Hanzhang Tu, Zhanfeng Liao, Wei Min +2
Tsinghua University, Beijing, China · Shadow AI, Beijing, China
Reconstructing 3D scenes from a single image is a fundamental challenge in computer vision, with broad applications in virtual reality, robotics, and content creation. Recent methods achieve outstanding performance by leveraging camera-controlled video diffusion models, but rely on iterative diffusion sampling, which greatly limits their practical deployment. We observe that geometric forward warping alone can cover the majority of a target view directly from the input image, with only a compact residual left for the encoder to correct. Motivated by this observation, we propose PRISM, a feed-forward framework that decomposes multi-view latent prediction into a parameter-free geometric prior and a learned residual correction, with no diffusion sampling required at inference. To enable generalization from purely synthetic training data, we devise a two-stage training strategy combining latents supervised distillation for geometric generalization and perceptual fine-tuning for appearance quality optimization. Extensive experiments on three benchmarks demonstrate that PRISM achieves competitive reconstruction quality compared with diffusion-based methods, while reducing inference time dramatically to only 36 seconds per scene.