FG-Portrait: 3D Flow Guided Editable Portrait Animation

Yating Xu; Yunqi Miao; Evangelos Ververas; Jiankang Deng; Jifei Song

cs.CVMar 24, 2026

FG-Portrait: 3D Flow Guided Editable Portrait Animation

Authors: Yating Xu, Yunqi Miao, Evangelos Ververas, Jiankang Deng, Jifei Song

Organizations: National University of Singapore · University of Warwick · Imperial College London · University of Surrey

Abstract

Motion transfer from the driving to the source portrait remains a key challenge in the portrait animation. Current diffusion-based approaches condition only on the driving motion, which fails to capture source-to-driving correspondences and consequently yields suboptimal motion transfer. Although flow estimation provides an alternative, predicting dense correspondences from 2D input is ill-posed and often yields inaccurate animation. We address this problem by introducing 3D flows, a learning-free and geometry-driven motion correspondence directly computed from parametric 3D head models. To integrate this 3D prior into diffusion model, we introduce 3D flow encoding to query potential 3D flows for each target pixel to indicate its displacement back to the source location. To obtain 3D flows aligned with 2D motion changes, we further propose depth-guided sampling to accurately locate the corresponding 3D points for each pixel. Beyond high-fidelity portrait animation, our model further supports user-specified editing of facial expression and head pose. Extensive experiments demonstrate the superiority of our method on consistent driving motion transfer as well as faithful source identity preservation.

Figures & tables

Figure 1: Given a source and driving image, the goal of portrait animation is to synthesize the target image with source identity and driving head pose and expression. APD( $\downarrow$ ) and AED( $\downarrow$ ) denote the average pose and expression error compared to the driving pose and expression, respectively. Compared to other diffusion-based models, e.g . X-Portrait [ 34 ] and Face-Adapter [ 14 ] , we can more accurately mimic the driving motion. Furthermore, we support feed-forward head pose and expression editing in the inference stage.

Figure 2: Visualization of 3D flows. Target portrait is the animated image with the source identity and driving motion. Target 3D head is the head model of target portrait, which is pre-computed by assembling source shape and driving motion parameters. We select some corresponding points both in the 2D and 3D, with red denoting the source and green denoting the target position. The 3D flows (black lines) correctly reflect the displacement from the target to the source position for each point. The yellow circles mark one example of a pair of points and the corresponding 3D flow.

Figure 3: Our Framework. We propose the 3D flow encoding $F_{src\leftarrow tgt}$ as the new motion condition to the ControlNet $G$ . Specifically, we first estimate the FLAME model $M_{tgt}$ and $M_{src}$ from $I_{dri}$ and $I_{src}$ . We perform depth-guided sampling to query the 3D flows in the target space for each pixel to indicate its displacement back to the source location, which are then stacked as the 3D flow encoding. During inference, the driving motion $(\theta_{dri},\psi_{dri})$ can come from the driving image or user-defined editing. During training, $I_{dri}$ is sampled from the video of the same person as $I_{src}$ and the training objective is to reconstruct $I_{tgt}$ as $I_{dri}$ .

Figure 4: Illustration of depth-guided sampling. Arrows show the 3D flows for the selected pixel marked with yellow circle . The sampled points in (b) are closer to the actual 3D location of the pixel than (a), resulting in 3D flows that are more faithfully aligned with the 2D motion.

Table 5

Figure 5: Qualitative results on self-reenactment. “Source” and “Driving” denote the source and driving image, respectively.

Method	FID $\downarrow$	CSIM $\uparrow$	APD $\downarrow$	AED $\downarrow$
EMOPortrait [ 8 ]	105.1	0.384	9.840	0.719
X-Portrait [ 34 ]	131.4	0.654	14.696	0.791
Follow-Your-Emoji [ 22 ]	120.3	0.634	10.624	0.863
Face-Adapter [ 14 ]	135.1	0.437	10.869	0.744
HunyuanPortrait [ 36 ]	113.7	0.505	10.570	0.722
Ours	99.4	0.558	9.297	0.714

Table 3: Comparison on the cross-reenactment task on FFHQ at

512^{2}

\downarrow

means lower the better and

\uparrow

is the opposite. APD is scaled by 100. Best results are marked bold.

Model	S-APD $\downarrow$	S-AED $\downarrow$	C-APD $\downarrow$	C-AED $\downarrow$
Dri-Ldk	4.001	0.373	8.588	0.688
Predicted Flow	4.232	0.384	12.430	0.778
Ours	2.682	0.327	7.764	0.652

Table 4: Comparison of motion conditions. ‘S-APD’ and ‘S-AED’ are the APD and AED for self-reenactment task. ‘C-APD’ and ‘C-AED’ are the APD and AED for cross-reenactment task.

Figure 6: Qualitative results on cross-reenactment. “Source” and “Driving” denote the source and driving image, respectively.

Figure 7: Qualitative results on FFHQ dataset. “Source” and “Driving” denote the source and driving image, respectively.

Method	LPIPS $\downarrow$	CSIM $\uparrow$	APD $\downarrow$	AED $\downarrow$
w/o Depth	0.213	0.770	9.659	0.730
w/ Depth	0.158	0.807	2.682	0.327

Table 5: Ablation study of depth-guided sampling on the self-reenactment task.

Figure 8: Qualitative results on of expression and pose editing. "Original" denotes the original animation using the driving image. The second and third columns show expression control while the rest columns show head movement.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Model	S-APD $\downarrow$	S-AED $\downarrow$	C-APD $\downarrow$	C-AED $\downarrow$
Dri-Img	1.060	0.131	1.216	0.144
Ours	2.682	0.327	7.764	0.652

Appendix

Table 6: Comparison of different motion conditions. ‘Dri-Img’ denotes using driving image as the motion condition. ‘S-APD’ and ‘S-AED’ are the APD and AED for self-reenactment task. ‘C-APD’ and ‘C-AED’ are the APD and AED for cross-reenactment task.

Figure 9: Qualitative comparison between ‘Dri-Img’ and Ours. ‘Dri-Img’ directly copies the driving image as the output. In contrast, we can correctly transfer the driving motion to the source person.

Method	LPIPS $\downarrow$	CSIM $\uparrow$	APD $\downarrow$	AED $\downarrow$	Mem(MB)
N=10	0.164	0.798	2.724	0.332	34110
N=30	0.160	0.807	2.540	0.334	34708
ours	0.158	0.807	2.682	0.327	34402

Appendix

Table 7: Ablation study of

N

in the self-reenactment on VFHQ.

Method	LPIPS $\downarrow$	CSIM $\uparrow$	APD $\downarrow$	AED $\downarrow$	Mem(MB)
$\delta=0.05m$	0.162	0.803	2.742	0.330	34402
$\delta=0.005m$	0.160	0.804	2.641	0.326	34402
ours	0.158	0.807	2.682	0.327	34402

Appendix

Table 8: Ablation study of

\delta

in the self-reenactment on VFHQ.

Figure 10: Qualitative comparisons on cases with diverse motion and appearance change.

Figure 11: Qualitative comparisons on cartoon portraits. (b) shows a challenging case, which is discussed in Sec. C .

Method	EMOPortrait	X-Portrait	FYE	Face-Adapter	Hunyuan	Ours
FVD	567.2	575.3	382.6	472.3	430.2	412.1

Appendix

Table 9: Temporal consistency analysis on the self-reenactment task of VFHQ. Best result is marked bold. ‘FYE’ and ‘Hunyuan’ are short for Follow-Your-Emoji and HunyuanPortrait, respectively.

Video Generation

Video generation studies models that create and edit moving images from text, images, audio, and structured controls.

See all 363 papers

Instant Expressive Gaussian Head Avatars at Over 100 FPS
Dec 18, 2025Kaiwen Jiang, Xueting Li, Seonwook Park +33D Gaussian Splatting Human Avatar Animation
University of California, San Diego, United States · NVIDIA, United States
ViDS: Video Diffusion Shader using 3D Face Tracking
Jul 27, 2026Wenbo Ji, Davide Davoli, Zhe Chen +3Identity-Preserving Video Generation Video Generation
Technical University of Munich · Toyota Motor Europe NV/SA · Woven by Toyota
Loki: Representation over Architecture for Diffusion-Based Portrait Animation
May 22, 2026Pouyan Navard, Sernam LimVideo Diffusion Models 3D Morphable Models
The Ohio State University · University of Central Florida

Dec 18, 2025J/K move · Enter open · S save

Instant Expressive Gaussian Head Avatars at Over 100 FPS

Kaiwen Jiang, Xueting Li, Seonwook Park, Ravi Ramamoorthi+2

University of California, San Diego, United States · NVIDIA, United States

3D Gaussian Splatting Human Avatar Animation

Open paper AskPDF arXiv

Portrait animation has witnessed tremendous quality improvements thanks to recent advances in video diffusion models. However, these 2D methods often compromise 3D consistency and speed, limiting their applicability in real-world scenarios, such as digital twins or telepresence. In contrast, 3D-aware feedforward facial animation methods -- built upon 3D representations, such as neural radiance fields or Gaussian splatting -- ensure 3D consistency and achieve faster inference speed, but come with inferior expression details. In this paper, we address this portrait animation trilemma (speed, 3D consistency, and expressiveness) and propose a pipeline that instantly converts an in-the-wild single image into a 3D-consistent, fast yet expressive animatable representation via a feed-forward encoder. Unlike previous computationally intensive global fusion mechanisms (e.g., multiple attention layers) for fusing 3D structural and animation information, our design employs an efficient lightweight local fusion strategy to achieve high animation expressivity. Furthermore, our animation representation is decoupled from the face's 3D representation and learns motion implicitly from data, eliminating the dependency on pre-defined parametric models that often constrain animation capabilities. Our method runs at 107.31 FPS for animation and pose control, representing a 3-4 order of magnitude speedup versus the state of the art while achieving comparable animation quality, thus surpassing alternative designs that trade speed for quality or vice versa.

Abstract

Video Generation

Explore similar work

Instant Expressive Gaussian Head Avatars at Over 100 FPS

ViDS: Video Diffusion Shader using 3D Face Tracking

Loki: Representation over Architecture for Diffusion-Based Portrait Animation