PerformRecast: Expression and Head Pose Disentanglement for Portrait Video Editing

Jiadong Liang; Bojun Xiong; Jie Tian; Hua Li; Xiao Long; Yong Zheng; Huan Fu

cs.CVMar 20, 2026

PerformRecast: Expression and Head Pose Disentanglement for Portrait Video Editing

Authors: Jiadong Liang, Bojun Xiong, Jie Tian, Hua Li, Xiao Long, Yong Zheng, Huan Fu

Organizations: HUJING Digital Media & Entertainment Group

Abstract

This paper primarily investigates the task of expression-only portrait video performance editing based on a driving video, which plays a crucial role in animation and film industries. Most existing research mainly focuses on portrait animation, which aims to animate a static portrait image according to the facial motion from the driving video. As a consequence, it remains challenging for them to disentangle the facial expression from head pose rotation and thus lack the ability to edit facial expression independently. In this paper, we propose PerformRecast, a versatile expression-only video editing method which is dedicated to recast the performance in existing film and animation. The key insight of our method comes from the characteristics of 3D Morphable Face Model (3DMM), which models the face identity, facial expression and head pose of 3D face mesh with separate parameters. Therefore, we improve the keypoints transformation formula in previous methods to make it more consistent with 3DMM model, which achieves a better disentanglement and provides users with much more fine-grained control. Furthermore, to avoid the misalignment around the boundary of face in generated results, we decouple the facial and non-facial regions of input portrait images and pre-train a teacher model to provide separate supervision for them. Extensive experiments show that our method produces high-quality results which are more faithful to the driving video, outperforming existing methods in both controllability and efficiency. Our code, data and trained models are available at https://youku-aigc.github.io/PerformRecast.

Figures & tables

Figure 2: An overview of our PerformRecast framework. The motion extractor extracts canonical keypoints, head pose, expression deformation, scale factor and translation from source and driving frames. The facial keypoints are obtained via our improved keypoints transformation formula and compared with the tracking results from Pixel3DMM [ 17 ] to calculate the FLAME loss. Finally, the appearance feature volume, source and driving keypoints are sent to the warping module, followed by the decoder to reconstruct the driving frame.

Figure 3: Top: we adopt the boundary alignment module to alleviate the misalignment between the facial and non-facial regions. Bottom: we provide some visual comparisons to demonstrate the necessity of using boundary alignment module (denoted as “BAM”). The red dashed circles highlight the misalignment between the facial and non-facial regions. The bottom-right insets show the driving frames. Please zoom in for better inspection.

Figure 4: The typical usage scenarios for our proposed replacement mode and enhancement mode. The bottom-right insets exhibit driving frames. Please zoom in for better inspection.

Figure 5: Qualitative comparison of portrait video expression editing on replacement mode. The top of the figure shows editing results of different methods. The bottom presents our ablation studies and analysis. The bottom-right insets exhibit driving frames. The red dashed circles highlight the misalignment between the facial and non-facial regions. Please zoom in for better inspection.

Method	Replacement Mode										Enhancement Mode
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	$\mathcal{L}_{1}$ $\downarrow$	CSIM $\uparrow$	MAE(°) $\downarrow$	AED $\downarrow$	APD $\downarrow$	FID $\downarrow$	FVD $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	$\mathcal{L}_{1}$ $\downarrow$	CSIM $\uparrow$	MAE(°) $\downarrow$	AED $\downarrow$	APD $\downarrow$	FID $\downarrow$	FVD $\downarrow$
SkyReels-A1 [ 42 ]	24.9051	0.8586	0.1622	0.0418	0.7163	13.0791	0.7157	0.0155	50.2007	1249.7021	24.9206	0.8584	0.1612	0.0417	0.7176	11.3797	0.676	0.016	51.47	1196.1767
Hunyuan-Portrait [ 70 ]	22.4287	0.7924	0.1691	0.0423	0.7364	10.4001	0.6608	0.0348	38.5997	1925.0654	22.7968	0.7984	0.1649	0.0406	0.736	9.6267	0.6073	0.0332	39.3112	1861.9558
FantasyPortrait [ 60 ]	23.9456	0.8204	0.1883	0.0349	0.7338	16.7765	0.7953	0.0168	60.4381	606.6303	24.2032	0.8239	0.1848	0.0339	0.7387	12.2154	0.6614	0.0155	62.0027	551.448
Wan-Animate [ 7 ]	22.8196	0.8021	0.1319	0.0467	0.614	11.9722	0.7002	0.0238	28.1003	849.2188	22.6417	0.8006	0.1402	0.0491	0.6062	11.2752	0.6847	0.0225	31.2186	885.4313
Act-Two [ 46 ]	20.8344	0.7913	0.1634	0.0459	0.6819	15.9618	0.7901	0.0723	36.1933	322.0726	20.8066	0.7916	0.165	0.0458	0.685	15.4109	0.8019	0.0727	38.0579	330.5131

Table 1: Quantitative results of facial expression editing on our test benchmark on both replacement and enhancement modes. The top of the table shows the comparison with other methods while the bottom presents our ablation studies and analysis.

Figure 6: Qualitative comparison of self-reenactment portrait animation. We show the source frame, driving frame and generative results. These source-driving paired images are from official test split of VFHQ dataset [ 68 ] . Please zoom in for better inspection.

Method	Self-reenactment										Cross-reenactment
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	$\mathcal{L}_{1}$ $\downarrow$	CSIM $\uparrow$	MAE(°) $\downarrow$	AED $\downarrow$	APD $\downarrow$	FID $\downarrow$	FVD $\downarrow$	CSIM $\uparrow$	MAE(°) $\downarrow$	AED $\downarrow$	APD $\downarrow$
GAGAvatar [ 8 ]	-	-	-	-	0.7878	8.1317	0.4057	0.014	-	-	0.6871	12.2879	0.8144	0.0256
Portrait4D-v2 [ 11 ]	14.3781	0.5601	0.4877	0.1268	0.7525	8.4331	0.4434	0.0262	55.0744	506.1298	0.6702	13.4384	0.8385	0.0321
PD-FGC [ 58 ]	16.4477	0.62	0.4252	0.0936	0.3282	11.9368	0.6345	0.0322	76.2318	628.5279	0.2181	15.0314	0.8988	0.0417
EMOPortrait [ 13 ]	18.7677	0.6979	0.2711	0.0648	0.6297	7.7967	0.4676	0.0179	43.2991	444.5448	0.3483	10.3905	0.8012	0.0245
EDTalk [ 53 ]	20.0771	0.7272	0.3054	0.0648	0.674	16.3315	0.572	0.0252	60.5285	369.9713	0.5435	22.5909	1.018	0.0531

Table 2: Quantitative comparisons of different methods on portrait animation task. The top of the table shows the results of non-diffusion-based methods while the bottom presents diffusion-based methods.

Figure 7: Qualitative comparison of cross-reenactment portrait animation. We show the source image, driving image and the generative results. The source images are from FFHQ dataset and driving frames are from VFHQ dataset.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 8: The facial mask calculation process of each frame in training dataset.

Figure 9: The specific location of each keypoint used in our method. Please zoom in for better inspection.

Figure 10: Construction pipeline of our proposed test benchmark.

Figure 11: Qualitative comparison of portrait video expression editing on enhancement mode. The top of the figure shows editing results of different methods. The bottom presents our ablation studies and analysis. The bottom-right insets exhibit driving frames. The light green circles highlight the misalignment between the facial and non-facial regions. Please zoom in for better inspection.

Figure 12: Full qualitative comparison with all the methods mentioned in the main manuscript on our proposed test benchmark. The bottom-right insets exhibit driving frames. Please zoom in for better inspection.

Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	$\mathcal{L}_{1}$ $\downarrow$	CSIM $\uparrow$	MAE(°) $\downarrow$	AED $\downarrow$	APD $\downarrow$	FID $\downarrow$	FVD $\downarrow$
GAGAvatar [ 8 ]	-	-	-	-	0.8946	5.1074	0.3781	0.0101	-	-
Portrait4D-v2 [ 11 ]	20.0907	0.7746	0.3358	0.0617	0.8793	5.4329	0.4353	0.0149	85.3589	460.7595
PD-FGC [ 58 ]	20.8419	0.7824	0.341	0.0573	0.3256	8.3772	0.7156	0.0202	92.8886	1276.3855
EMOPortrait [ 13 ]	26.1748	0.8729	0.1544	0.0287	0.5959	6.6994	0.4992	0.0128	37.5216	443.7428
EDTalk [ 53 ]	26.9246	0.8964	0.1443	0.0319	0.8592	6.218	0.4333	0.0077	43.4199	343.9973
LIA-X [ 63 ]	22.6439	0.8232	0.1816	0.0386	0.8957	5.3919	0.4633	0.0636	32.4902	323.2831

Appendix

Table 3: Quantitative comparisons of self-reenactment portrait animation on MEAD [ 59 ] dataset. The top of the table shows the results of non-diffusion-based methods, while the bottom presents diffusion-based methods.

Figure 13: More generated results on self-reenactment task of different methods. The first two source-driving paired images are from VFHQ dataset [ 68 ] and the last two source-driving paired images are from MEAD dataset [ 59 ] .

Figure 14: More generated results on cross-reenactment task of different methods. The source images are from FFHQ dataset and the driving frames are from famous films and television clips.

Figure 15: A typical failure case when our method generating teeth while the mouth is closed in source video.

Video Editing

Video editing uses AI to alter, compose, relight, or reorganize moving footage while controlling how changes persist across frames.

See all 178 papers

FG-Portrait: 3D Flow Guided Editable Portrait Animation
Mar 24, 2026Yating Xu, Yunqi Miao, Evangelos Ververas +2Video Generation Diffusion Models
National University of Singapore · University of Warwick · Imperial College London +1
ViDS: Video Diffusion Shader using 3D Face Tracking
Jul 27, 2026Wenbo Ji, Davide Davoli, Zhe Chen +3Identity-Preserving Video Generation Video Generation
Technical University of Munich · Toyota Motor Europe NV/SA · Woven by Toyota
ExpPortrait: Expressive Portrait Generation via Personalized Representation
Feb 23, 2026Junyi Wang, Yudong Guo, Boyang Guo +2Video Generation Identity-Preserving Video Generation
University of Science and Technology of China

Mar 24, 2026J/K move · Enter open · S save

FG-Portrait: 3D Flow Guided Editable Portrait Animation

Yating Xu, Yunqi Miao, Evangelos Ververas, Jiankang Deng+1

National University of Singapore · University of Warwick · Imperial College London+1

Video Generation Diffusion Models

Open paper AskPDF arXiv

Motion transfer from the driving to the source portrait remains a key challenge in the portrait animation. Current diffusion-based approaches condition only on the driving motion, which fails to capture source-to-driving correspondences and consequently yields suboptimal motion transfer. Although flow estimation provides an alternative, predicting dense correspondences from 2D input is ill-posed and often yields inaccurate animation. We address this problem by introducing 3D flows, a learning-free and geometry-driven motion correspondence directly computed from parametric 3D head models. To integrate this 3D prior into diffusion model, we introduce 3D flow encoding to query potential 3D flows for each target pixel to indicate its displacement back to the source location. To obtain 3D flows aligned with 2D motion changes, we further propose depth-guided sampling to accurately locate the corresponding 3D points for each pixel. Beyond high-fidelity portrait animation, our model further supports user-specified editing of facial expression and head pose. Extensive experiments demonstrate the superiority of our method on consistent driving motion transfer as well as faithful source identity preservation.

Abstract

Video Editing

Explore similar work

FG-Portrait: 3D Flow Guided Editable Portrait Animation

ViDS: Video Diffusion Shader using 3D Face Tracking

ExpPortrait: Expressive Portrait Generation via Personalized Representation