We present PartNerFace, a part-based neural radiance fields approach, for reconstructing animatable facial avatar from monocular RGB videos. Existing solutions either simply condition the implicit network with the morphable model parameters or learn an imaginary canonical radiance field, making them fail to generalize to unseen facial expressions and capture fine-scale motion details. To address these challenges, we first apply inverse skinning based on a parametric head model to map an observed point to the canonical space, and then model fine-scale motions with a part-based deformation field. Our key insight is that the deformation of different facial parts should be modeled differently. Specifically, our part-based deformation field consists of multiple local MLPs to adaptively partition the canonical space into different parts, where the deformation of a 3D point is computed by aggregating the prediction of all local MLPs by a soft-weighting mechanism. Extensive experiments demonstrate that our method generalizes well to unseen expressions and is capable of modeling fine-scale facial motions, outperforming state-of-the-art methods both quantitatively and qualitatively.
Figures & tables
Figure 2: Overview of PartNerFace . Our method represents the facial avatar as a neural radiance fields in the canonical space. Given a 3D point in the posed space, it is first transformed to the canonical space through inverse LBS based on the FLAME model, then followed by a part assigner which predicts the probability of its part ascription. This probability is used to aggregate the output from a set of local deformation networks. The weighted deformation is then added to the previous transformed point. Finally, the occupancy and color is evaluated in the canonical space for volume rendering.
Figure 3: Qualitative comparison on the reconstruction of Testset . Some regions are zoomed in for detailed comparisons.
Figure 4: Visual comparison of animation results with unseen poses and expressions.
PSNR ↑
SSIM ↑
L1 ↓
LPIPS ↓
FOMM
22.22
0.87
0.085
0.035
SAFA
18.47
0.85
0.142
0.050
NerFace
26.77
0.93
0.051
0.012
IMavatar
23.78
0.91
0.073
0.018
Ours
25.73
0.93
0.056
0.012
Table 1: Quantitative Comparsion on the reconstruction of Testset . We report mean values for each metrics. Although the quantitative results of NerFace are slightly better than ours, we argue this is due to its overfitting on training data. However, when feed with unseen pose and expressions, its quantitative results declined by a large margin (see supplemental materials).
Figure 5: Qualitative comparison of ablation studies . Columns 1-5: the visual comparison of between different variants and ground truths. Columns 6-8: the visualized face partitions on the learned surface. We use separate colors for each part and use the brightness to indicate the probability of surface points belonging this part (for hard part assignment, its probability is all ones).
PSNR ↑
SSIM ↑
L1 ↓
LPIPS ↓
Global
25.30
0.92
0.059
0.014
Hard
25.60
0.92
0.061
0.013
Soft
25.49
0.92
0.057
0.013
Ours
25.73
0.93
0.056
0.012
Table 2: Quantitative comparsion of ablation studies .
Avatar reconstruction has traditionally relied on per-subject optimization that requires hours of computation or on expensive preprocessing that limits scalability. We introduce FFAvatar, a generalizable feed-forward framework that reconstructs high-quality, animatable 3D Gaussian head avatars from few-shot unposed portrait images in seconds. FFAvatar fuses information from multiple source images into a unified canonical Gaussian representation through Multi-View Query-Former, which is animated via FLAME parameters predicted end-to-end directly from pixels, eliminating the overhead of offline FLAME extraction. We further propose a three-stage training curriculum that achieves both broad generalization and high-fidelity reconstruction: (i) scalable pretraining on extensive monocular video data with over 1M identities to learn strong generalizable priors; (ii) multi-view fine-tuning on a small but high-quality dataset of 360-degree captures to enhance geometric fidelity and extreme-view awareness; and (iii) optional personalization that adapts to specific identities for maximum fidelity within 500 optimization steps. Extensive experiments demonstrate that FFAvatar sets a new standard for identity preservation, geometric consistency, and animation fidelity. On the NeRSemble benchmark, it outperforms the state-of-the-art LAM by a substantial 5.5 PSNR gain. Furthermore, FFAvatar enables real-time deployment, reconstructing avatars in 2 seconds without personalization and 10 seconds with personalization, while supporting 49 FPS animation on a single NVIDIA A100 GPU.
Thuan Hoang Nguyen, Jiahao Luo, Yinyu Nie +3
Snap Inc. · MBZUAI · University of California, Santa Cruz
Mesh-agnostic facial animation retargeting transfers expressions across meshes with different structures, but preserving facial motion without surface artifacts remains challenging. To address this, we present PDB, Point-Based Deformation Blending for facial animation retargeting. PDB predicts a compact set of deformed control points from a source neutral-expression pair and blending weights from the target neutral mesh. The weights are computed once per target and reused across frames, while the control points vary with each source expression. ReLU enforces non-negative weights and permits exact zeros, followed by row-wise normalization. The target mesh is reconstructed directly by multiplying the weights and control points, without a predefined cage, precomputed coordinates, a learned per-element deformation decoder, or a global reconstruction solve. Trained only with self-retargeting reconstruction supervision, PDB supports cross-identity transfer without paired cross-identity training expressions. Experiments demonstrate accurate retargeting, fast inference, and localized support in the learned weights. Joint evaluation of expression accuracy and local surface preservation shows reduced surface artifacts relative to the evaluated dense displacement method while retaining the intended motion. Perceptual evaluations further support expression fidelity and visual quality in both self- and cross-retargeting.
Sihun Cha, Hyeonseung Shin, Suah Yu +1
Visual Media Lab, KAIST, Daejeon, Republic of Korea
Existing full-body Gaussian avatar methods primarily optimize global reconstruction quality and often fail to preserve fine-grained facial geometry and expression details. This challenge arises from limited facial representational capacity that causes difficulties in modeling high-frequency pose-dependent deformations. To address this, we propose F3G-Avatar, a full-body, face-aware avatar synthesis method that reconstructs animatable human representations from multi-view RGB video and regressed pose/shape parameters. Starting from a clothed Momentum Human Rig (MHR) template, front/back positional maps are rendered and decoded into 3D Gaussians through a two-branch architecture: a body branch that captures pose-dependent non-rigid deformations and a face-focused deformation branch that refines head geometry and appearance. The predicted Gaussians are fused, posed with linear blend skinning (LBS), and rendered with differentiable Gaussian splatting. Training combines reconstruction and perceptual objectives with a face-specific adversarial loss to enhance realism in close-up views. Experiments demonstrate strong rendering quality, with face-view performance reaching PSNR/SSIM/LPIPS of 26.243/0.964/0.084 on the AvatarReX dataset. Ablations further highlight contributions of the MHR template and the face-focused deformation. F3G-Avatar provides a practical, high-quality pipeline for realistic, animatable full-body avatar synthesis.
Willem Menu, Erkut Akdag, Pedro Quesado +2
AIMS Group, Department of Electrical Engineering, Eindhoven University of Technology