Reconstructing photorealistic and animatable 4D head avatars from a single portrait image remains a fundamental challenge in computer vision. While diffusion models have enabled remarkable progress in image and video generation for avatar reconstruction, existing methods primarily rely on 2D priors and struggle to achieve consistent 3D geometry. We propose a novel framework that leverages geometry-aware diffusion to learn strong geometry priors for high-fidelity head avatar reconstruction. Our approach jointly synthesizes portrait images and corresponding surface normals, while a pose-free expression encoder captures implicit expression representations. Both synthesized images and expression latents are incorporated into 3D Gaussian-based avatars, enabling photorealistic rendering with accurate geometry. Extensive experiments demonstrate that our method substantially outperforms state-of-the-art approaches in visual quality, expression fidelity, and cross-identity generalization, while supporting real-time rendering.
Figures & tables
Figure 2: Overall architecture. Our system takes a reference image, driving expressions, and head poses as input. Specifically, the reference image is encoded into hierarchical identity embeddings using a pretrained VAE and UNet-based reference network. Driving expressions are compressed into low-dimensional latents via a pose-free expression encoder. Both embeddings are injected into the diffusion model through cross-attention, while head pose maps concatenated with noise serve as inputs. The model then jointly predicts portrait images and surface normals. For 3D reconstruction, a UNet refines FLAME meshes using expression latents through cross-attention, and an MLP captures Gaussian dynamics. Finally, the generated surface normals provide additional geometric supervision that further enhances the reconstruction fidelity.
Figure 3: Cross-view pairing training strategy. For each identity and timestep, frames from different viewpoints are paired with consistent expressions but varying poses, enabling the encoder to learn view-invariant representations.
Figure 4: Self-Reenactment results. We conduct this experiment on a subset of the NeRSemblev2 dataset containing a large number of extreme head poses in both the reference images and driving sequences, enabling a comprehensive evaluation of model performance. We also show surface normals generated by our video generation model.
Method
Self-reenactment
Cross-reenactment
PSNR ↑
SSIM ↑
LPIPS ↓
CSIM ↑
JOD ↑
CSIM ↑
JOD ↑
GAGAvatar [ 7 ]
17.550
0.789
0.229
0.714
6.244
0.588
5.081
Portrait4D-v2 [ 10 ]
13.689
0.701
0.310
0.702
4.933
0.608
4.656
LAM [ 23 ]
16.354
0.759
0.251
0.608
5.772
0.516
5.079
CAP4D [ 44 ]
19.295
0.811
0.195
0.719
6.561
0.655
5.064
Our VGM
21.586
0.831
0.174
0.754
7.127
0.671
5.066
Table 1: Quantitative results of self-reenactment on the NeRSemblev2 dataset and cross-reenactment on a mixture of NeRSemblev2 subset and in-the-wild motion data. Bold and underlined indicate the best and second-best results, respectively.
Figure 5: Cross-Reenactment results. Evaluated on a mixture of NeRSemblev2 and in-the-wild collections spanning diverse real and cartoon identities to assess model generalizability.
Method
Ablation
PSNR ↑
SSIM ↑
LPIPS ↓
CSIM ↑
JOD ↑
AKD ↓
AED ↓
VGM only
w/o joint rep.
20.809
0.825
0.184
0.757
6.960
4.216
2.489
w/o domain attn.
20.984
0.820
0.179
0.743
7.029
4.195
2.556
w/o cv pairing
19.895
0.804
0.191
0.734
6.859
5.367
3.113
w/o synt. data
20.892
0.820
0.181
0.743
6.978
4.339
2.527
Ours
21.586
0.831
0.174
0.754
7.127
4.016
2.340
Recon
w/o hier. refine
19.816
0.821
0.198
0.736
6.758
4.227
2.603
Table 2: Ablation study. For video generation model, we ablate joint representation, Domain-Spatial attention module, cross-view pairing strategy and synthetic data. For avatar reconstruction, we ablate hierarchical refinement, normal regularization and generated normal(vs. normal from DAViD).
Figure 6: Ablation of portrait generation. We ablate joint representation learning, the Domain-Spatial attention module, cross-view pairing, and synthetic data.
Figure 7: Ablation of normal generation. We ablate synthetic data to evaluate its contribution to normal generation quality.
Figure 8: Comparison of ground-truth and pseudo-ground-truth normals. Ground-truth normals from synthetic data are more accurate and capture finer geometric details, thereby improving the quality of generated surface normals.
Figure 9: Ablation on 4D reconstruction. We ablate hierarchical refinement and normal regularization to evaluate their contributions to reconstruction quality.
Figure 10: Ablations on hierarchical refinement and normal regularization for free-view rendering.
Figure 11: Ablations on our normals versus DAViD normals.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
CSIM ↑
JOD ↑
VOODOOXP
13.111
0.714
0.276
0.598
5.116
LivePortrait
18.047
0.787
0.214
0.736
6.230
Wan-Animate
12.805
0.655
0.354
0.712
4.597
X-NeMo
14.202
0.683
0.311
0.732
4.987
Our VGM
21.586
0.831
0.174
0.754
7.127
GeoDiff4D
19.953
0.822
0.195
0.737
6.780
Table 3: Quantitative comparison with more baselines ( best , second-best ).
Figure 12: Qualitative comparison with more baselines.
Figure 13: Cross-view animation results. We generate images using expression sequences from 12 different camera viewpoints while fixing the head pose, demonstrating the view consistency of our pose-free expression encoder.
Steps
PSNR ↑
SSIM ↑
LPIPS ↓
CSIM ↑
JOD ↑
AKD ↓
AED ↓
Speed (s/frame) ↓
10
21.177
0.8286
0.1779
0.7419
7.033
4.215
2.503
1.16
25
21.505
0.8294
0.1746
0.7419
7.093
4.242
2.541
2.74
50
21.529
0.8285
0.1740
0.7421
7.097
4.283
2.573
5.39
100
21.545
0.8280
0.1736
0.7412
7.100
4.261
2.550
10.66
Table 4: Ablation on sampling steps ( best , second-best ).