The human face is central to communication. For immersive applications, the digital presence of a person should mirror the physical reality, capturing the users idiosyncrasies and detailed facial expressions. However, current 3D head avatar methods often struggle to faithfully reproduce the identity and facial expressions, despite having multi-view data or learned priors. Learning priors that capture the diversity of human appearances, especially, for regions with highly person-specific features, like the mouth and teeth region is challenging as the underlying training data is limited. In addition, many of the avatar methods are purely relying on 3D morphable model-based expression control which strongly limits expressivity. To address these challenges, we are introducing 3DRealHead, a few-shot head avatar reconstruction method with a novel expression control signal that is extracted from a monocular video stream of the subject. Specifically, the subject can take a few pictures of themselves, recover a 3D head avatar and drive it with a consumer-level webcam. The avatar reconstruction is enabled via a novel few-shot inversion process of a 3D human head prior which is represented as a Style U-Net that emits 3D Gaussian primitives which can be rendered under novel views. The prior is learned on the NeRSemble dataset. For animating the avatar, the U-Net is conditioned on 3DMM-based facial expression signals, as well as features of the mouth region extracted from the driving video. These additional mouth features allow us to recover facial expressions that cannot be represented by the 3DMM leading to a higher expressivity and closer resemblance to the physical reality.
Figures & tables
Fig. 2: Overview of 3DRealHead. Based on few-shot input data, an RGB texture is generated which serves as identity conditioning to the Style U-Net which predicts 3D Gaussian primitives in UV-space relative to a FLAME head mesh. To control detailed facial expressions from a driving input video, we extract 3DMM-based position maps [ 35 ] and image gradient features which are projected to the UV texture space. From this control signal, the model learns how to produce dynamically changing 3D Gaussian primitives which can be rendered using 3D Gaussian splatting. The Style U-Net is trained on the NeRSemble [ 24 ] dataset, such that it can act as a prior for few-shot inversion, where only the identity encoder EId is finetuned. During the training of the prior, we employ several reconstruction losses, as well as a multi-head discriminator D for adversarial training.
Studio Data
Method
LPIPS ↓
SSIM ↑
PSNR ↑
ID ↑
t-LPIPS ↓
Ours
0.036
0.944
33.152
0.992
0.073
GAGAvatar [ 9 ]
0.037
0.935
31.595
0.992
0.077
GPAvatar [ 10 ]
0.063
0.927
32.379
0.982
0.101
LAM [ 16 ]
0.081
0.899
25.035
0.974
0.098
In-the-Wild Data – Few-Shot
TABLE I: Quantitative comparison of our method against prior-based and SOTA monocular methods on Studio and In-the-Wild datasets (8 subjects). For studio data, we perform 1-shot inversion.
Fig. 3: We show qualitative results of our method along different views for cases with 1 and 3 target images.
Fig. 4: Qualitative comparison of our approach (single target) against one-shot prior-based methods. We observe that our method produces sharper results and faithfully reproduces the driving expression.
Fig. 5: Qualitative comparison to SynShot [ 61 ] and SOTA monocular approaches on In-the-Wild INSTA dataset [ 60 ] . It is important to note that SOTA monocular methods are trained per subject on minutes of video recording while SynShot and our method are fine-tuned on only 3 frames.
Method
LPIPS ↓
SSIM ↑
PSNR ↑
ID ↑
t-LPIPS ↓
Ours
0.064
0.937
28.016
0.991
0.079
Ours-RGB
0.063
0.937
28.070
0.991
0.078
w/o Adversar. Loss
0.070
0.939
28.137
0.990
0.084
w/o Mouth Cond.
0.065
0.934
27.730
0.989
0.079
Ours-Full
0.068
0.937
27.636
0.990
0.082
RGB-Full
0.061
0.941
29.632
0.991
0.074
TABLE II: Quantitative evaluation of our ablation studies on NeRSemble data.
Fig. 6: Ablation experiments where we replace our gradient-based conditioning with different other types of conditioning, and where we show the influence of the adversarial loss.
Method
LPIPS ↓
SSIM ↑
PSNR ↑
ID ↑
t-LPIPS ↓
Ours
0.068 ± 0.004
0.932 ± 0.002
28.156 ± 0.106
0.991 ± 0.0006
0.084 ± 0.003
DINO
0.071 ± 0.005
0.930 ± 0.001
28.037 ± 0.102
0.990 ± 0.0007
0.086 ± 0.004
HOG
0.079 ± 0.003
0.928 ± 0.002
27.941 ± 0.109
0.990 ± 0.0003
0.089 ± 0.004
TABLE III: Quantitative evaluation of different mouth feature conditionings on NeRSemble data.
Fig. 7: Comparison of different mouth feature conditionings.
Fig. 8: Robustness of our method w.r.t. identity and illumination changes. In the first row, the single enrollment image used is an edited image, from which we recover the 3D avatar which can be driven by a monocular sequence. In the bottom row, one can see that the 3D avatar of a person can be driven by a monocular input in different illumination settings.
Fig. 9: Interpolation between two test identities (leftmost and rightmost columns).
Fig. 10: Cross-reenactment: driving actor (left), output (middle), identity of target (right).
Fig. 11: Inversion Experiment for the No Fine-Tuning, 1-shot and 3-shot cases.
Fig. 12: Comparison against GAN-based baselines on In-the-Wild data.
Fig. 13: Comparison against Avat3r [ 25 ] for different subjects and driving frames rendered along different camera views.
Fig. 14: Novel View Synthesis on In-the-Wild data.
Fig. 15: Qualitative results of using a different UV map (shown on left side of each corresponding experiment) with different number and distribution of Gaussians in the mouth region.
Fig. 16: Qualitative results of using 3DMM-only based control versus using additional mouth gradients (3DMM-G) versus our method (expression + mouth gradients).
Fig. 17: Qualitative comparison of our method against GaussianAvatars [ 33 ] .
Fig. 18: Plots of performance versus number of frames during inversion for LPIPS, SSIM and PSNR metrics.
Method
LPIPS ↓
SSIM ↑
PSNR ↑
ID ↑
t-LPIPS ↓
Ours
0.068 ± 0.004
0.932 ± 0.002
28.156 ± 0.106
0.991 ± 0.001
0.084 ± 0.003
New UV (70 % less)
0.077
0.928
27.83
0.987
0.089
TABLE IV: Quantitative evaluation of the distribution of mouth Gaussians on NeRSemble data.
Method
LPIPS ↓
SSIM ↑
PSNR ↑
ID ↑
t-LPIPS ↓
Ours
0.068 ± 0.004
0.932 ± 0.002
28.156 ± 0.106
0.991 ± 0.001
0.084 ± 0.003
3DMM-only
0.073
0.925
27.102
0.990
0.089
3DMM-Grad.
0.068
0.932
27.551
0.991
0.087
TABLE V: Quantitative evaluation of our method versus 3DMM-only control versus having both 3DMM control with additional Gradient-based mouth maps on NeRSemble data.
Training Set Size
LPIPS ↓
SSIM ↑
PSNR ↑
ID ↑
t-LPIPS ↓
100 % (Ours)
0.049
0.939
30.035
0.990
0.073
50 %
0.049
0.936
30.032
0.989
0.075
25 %
0.055
0.930
29.780
0.982
0.078
5 %
0.072
0.914
27.893
0.967
0.084
TABLE VI: Quantitative evaluation of the size of training set on 8 In-the-Wild subjects.
No. of Frames
LPIPS ↓
SSIM ↑
PSNR ↑
ID ↑
t-LPIPS ↓
5 Frames
0.045
0.941
30.163
0.992
0.071
3 Frames
0.049
0.939
30.035
0.990
0.073
2 Frames
0.049
0.937
30.028
0.990
0.075
1 Frame
0.050
0.935
29.935
0.987
0.076
TABLE VII: Quantitative evaluation of the number of frames during enrollment on 8 In-the-Wild subjects.
Subject Category
No. of Subjects
LPIPS ↓
SSIM ↑
PSNR ↑
ID ↑
t-LPIPS ↓
Similar to training set
3
0.050
0.930
30.20
0.989
0.071
Different ethnicity
2
0.065
0.924
29.21
0.984
0.077
Extreme views
3
0.054
0.929
29.22
0.989
0.075
TABLE VIII: Category-wise quantitative evaluation of our method on 8 In-the-Wild subjects.
Fig. 19: Qualitative comparison of different training set sizes.
Fig. 20: Qualitative comparison between different number of frames used for inversion.
Avatar reconstruction has traditionally relied on per-subject optimization that requires hours of computation or on expensive preprocessing that limits scalability. We introduce FFAvatar, a generalizable feed-forward framework that reconstructs high-quality, animatable 3D Gaussian head avatars from few-shot unposed portrait images in seconds. FFAvatar fuses information from multiple source images into a unified canonical Gaussian representation through Multi-View Query-Former, which is animated via FLAME parameters predicted end-to-end directly from pixels, eliminating the overhead of offline FLAME extraction. We further propose a three-stage training curriculum that achieves both broad generalization and high-fidelity reconstruction: (i) scalable pretraining on extensive monocular video data with over 1M identities to learn strong generalizable priors; (ii) multi-view fine-tuning on a small but high-quality dataset of 360-degree captures to enhance geometric fidelity and extreme-view awareness; and (iii) optional personalization that adapts to specific identities for maximum fidelity within 500 optimization steps. Extensive experiments demonstrate that FFAvatar sets a new standard for identity preservation, geometric consistency, and animation fidelity. On the NeRSemble benchmark, it outperforms the state-of-the-art LAM by a substantial 5.5 PSNR gain. Furthermore, FFAvatar enables real-time deployment, reconstructing avatars in 2 seconds without personalization and 10 seconds with personalization, while supporting 49 FPS animation on a single NVIDIA A100 GPU.
Thuan Hoang Nguyen, Jiahao Luo, Yinyu Nie +3
Snap Inc. · MBZUAI · University of California, Santa Cruz
Reconstructing a complete 3D head from a single portrait remains challenging because existing methods still face a sharp quality-speed trade-off: high-fidelity pipelines often rely on multi-stage processing and per-subject optimization, while fast feed-forward models struggle with complete geometry and fine appearance details. To bridge this gap, we propose Any3DAvatar, a fast and high-quality method for single-image 3D Gaussian head avatar generation, whose fastest setting reconstructs a full head in under one second while preserving high-fidelity geometry and texture. First, we build AnyHead, a unified data suite that combines identity diversity, dense multi-view supervision, and realistic accessories, filling the main gaps of existing head data in coverage, full-head geometry, and complex appearance. Second, rather than sampling unstructured noise, we initialize from a Plücker-aware structured 3D Gaussian scaffold and perform one-step conditional denoising, formulating full-head reconstruction into a single forward pass while retaining high fidelity. Third, we introduce auxiliary view-conditioned appearance supervision on the same latent tokens alongside 3D Gaussian reconstruction, improving novel-view texture details at zero extra inference cost. Experiments show that Any3DAvatar outperforms prior single-image full-head reconstruction methods in rendering fidelity while remaining substantially faster.
We propose OMG-Avatar, a novel One-shot method that leverages a Multi-LOD (Level-of-Detail) Gaussian representation for animatable 3D head reconstruction from a single image in 0.2s. Our method enables LOD head avatar modeling using a unified model that accommodates diverse hardware capabilities and inference speed requirements. To capture both global and local facial characteristics, we employ a transformer-based architecture for global feature extraction and projection-based sampling for local feature acquisition. These features are effectively fused under the guidance of a depth buffer, ensuring occlusion plausibility. We further introduce a coarse-to-fine learning paradigm to support Level-of-Detail functionality and enhance the perception of hierarchical details. To address the limitations of 3DMMs in modeling non-head regions such as the shoulders, we introduce a multi-region decomposition scheme in which the head and shoulders are predicted separately and then integrated through cross-region combination. Extensive experiments demonstrate that OMG-Avatar outperforms state-of-the-art methods in reconstruction quality, reenactment performance, and computational efficiency. The project homepage is https://human3daigc.github.io/OMGAvatar_project_page/ .