LiftAvatar: Kinematic-Space Completion for Expression-Controlled 3D Gaussian Avatar Animation

Hualiang Wei; Shunran Jia; Jialun Liu; Wenhui Li

cs.CVMar 2, 2026

LiftAvatar: Kinematic-Space Completion for Expression-Controlled 3D Gaussian Avatar Animation

Authors: Hualiang Wei, Shunran Jia, Jialun Liu, Wenhui Li

Organizations: College of Computer Science and Technology, Jilin University, No. 2699 Qianjin Street, Changchun, 130012, Jilin, China · Impressed Inc DBA SocialBook, 950 Tower Ln, Foster City, 94404, California, USA · Institute of Artificial Intelligence of China Telecom, Zhonghui Building, Dongcheng District, 100007, Beijing, China

Abstract

We present LiftAvatar, a new paradigm that completes sparse monocular observations in kinematic space (e.g., facial expressions and head pose) and uses the completed signals to drive high-fidelity avatar animation. LiftAvatar is a fine-grained, expression-controllable large-scale video diffusion Transformer that synthesizes high-quality, temporally coherent expression sequences conditioned on single or multiple reference images. The key idea is to lift incomplete input data into a richer kinematic representation, thereby strengthening both reconstruction and animation in downstream 3D avatar pipelines. To this end, we introduce (i) a multi-granularity expression control scheme that combines shading maps with expression coefficients for precise and stable driving, and (ii) a multi-reference conditioning mechanism that aggregates complementary cues from multiple frames, enabling strong 3D consistency and controllability. As a plug-and-play enhancer, LiftAvatar directly addresses the limited expressiveness and reconstruction artifacts of 3D Gaussian Splatting-based avatars caused by sparse kinematic cues in everyday monocular videos. By expanding incomplete observations into diverse pose-expression variations, LiftAvatar also enables effective prior distillation from large-scale video generative models into 3D pipelines, leading to substantial gains. Extensive experiments show that LiftAvatar consistently boosts animation quality and quantitative metrics of state-of-the-art 3D avatar methods, especially under extreme, unseen expressions.

Figures & tables

Figure 1: We propose a novel kinematic lifted framework, LiftAvatar, to complement the facial expressions and head poses of the monotonous input video. LiftAvatar can promote subsequent reconstruction and driving tasks, resulting in significant improvements.

Figure 2: Method Pipeline. LiftAvatar is a high-precision, expression‑controlled video diffusion transformer that enriches sparse observations to boost downstream avatar performance. It conditions on three groups of inputs: reference information

(I^{R},S^{R},E^{R})

from the input video; driving information

(S^{D},E^{D})

for the target motion; and the ground‑truth driving video

V^{D}

during training (replaced by noise at inference). The reference images are encoded by a pre‑trained VAE, and the Reference NPHM Encoder encodes their shading maps

S^{R}

. These features are concatenated, projected via Reference Patch Embedding, and summed with the embedded expression coefficients

E^{R}

to form the reference token

x^{R}

. Likewise, the Driven NPHM Encoder processes the driving shading maps

S^{D}

; its output is projected by Driven Patch Embedding and combined with the embedded

E^{D}

to produce the driving token

x^{D}

. The tokens

x^{R}

and

x^{D}

are concatenated and fed as a unified condition into the Wan2.1 Wang et al. (2025a) video diffusion transformer backbone. Optimized with a flow‑matching objective, the model synthesizes high‑fidelity, temporally coherent videos that accurately follow the driving signals, thereby completing the kinematic space of the original input.

Figure 3: Qualitative results for kinematic lifted. We compared our results with non-diffusion-based methods (FOMM Siarohin et al. (2019) and Face Vid2vid Wang et al. (2021) ) as well as diffusion-based models (DiffusionAvatars Kirschstein et al. (2024) and LivePortrait Guo et al. (2024) and HunyuanPortrait Xu et al. (2025) ). It is evident that our method provides better performance in generating extreme expressions, particularly in terms of facial texture details, teeth accuracy, and pose accuracy.

Figure 4: Qualitative results for head avatar animation. We compare the two head avatar animation methods before and after kinematic lifted. The comparison shows that our proposed strategy can effectively enhance subsequent reconstruction and driving.

Figure 5: Lifted results with LiftAvatar.

Figure 6: Additional Results from MonoGaussianAvatar and LAM.

Figure 7: Additional Comparative Results of SplattingAvatar.

Figure 8: We first cropped the original data from $2200\times 3208$ pixels to $512\times 512$ pixels. Next, we performed NPHM Giebenhain et al. (2023) preprocessing on the cropped images, and finally, shaded them into Phong images.

Figure 9: The Number of Reference Images. Optimal performance is achieved with five reference frames selected by K-means (K-means-5) during inference.

Figure 10: User Study (50 users). In the evaluation conducted by 50 users on the kinematic lifted facial expressions, our model achieved the highest scores and significantly outperformed all other compared models.

Figure 11: The Number of Kinematic Lifted Expressions. The impact of the number of expressions on the effectiveness of different kinematic lifted methods.

Method	Image Quality			Other Metrics
	PSNR $\hskip 1.99997pt\uparrow$	LPIPS $\hskip 1.99997pt\downarrow$	SSIM $\hskip 1.99997pt\uparrow$	FID $\hskip 1.99997pt\downarrow$	AED $\hskip 1.99997pt\downarrow$	CSIM $\hskip 1.99997pt\uparrow$
FOMM Siarohin et al. (2019)	21.9167	0.2047	0.6823	60.31	0.7433	0.9721
Face Vid2vid Wang et al. (2021)	21.3089	0.1980	0.7150	54.61	0.7851	0.9718
DiffusionAvatars Kirschstein et al. (2024)	25.3098	0.1479	0.8119	40.84	0.6525	0.9814
LivePortrait Guo et al. (2024)	25.6561	0.1470	0.8237	40.55	0.6372	0.9855
HunyuanPortrait Xu et al. (2025)	27.2258	0.1137	0.8412	39.24	0.6144	0.9891

Table 1: Quantitative comparison of expression kinematic lifted.

Method	FOMM+exp			Face Vid2vid+exp			DiffusionAvatars+exp
	PSNR $\hskip 1.99997pt\uparrow$	SSIM $\hskip 1.99997pt\uparrow$	LPIPS $\hskip 1.99997pt\downarrow$	PSNR $\uparrow$	SSIM $\hskip 1.99997pt\uparrow$	LPIPS $\hskip 1.99997pt\downarrow$	PSNR $\hskip 1.99997pt\uparrow$	SSIM $\hskip 1.99997pt\uparrow$	LPIPS $\hskip 1.99997pt\downarrow$
SplattingAvatar Shao et al. (2024)	20.92	0.683	0.247	21.65	0.710	0.198	24.96	0.840	0.145
MonoGaussianAvatar Chen et al. (2024b)	22.50	0.790	0.232	22.80	0.751	0.192	25.10	0.885	0.137
Method	LivePortrait+exp			HunyuanPortrait+exp			LiftAvatar+exp
	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
SplattingAvatar Shao et al. (2024)	25.06	0.840	0.186	25.32	0.843	0.153	25.71	0.849	0.132

Table 2: Quantitative comparisons of different expression kinematic lifted methods.

Method	PSNR $\hskip 1.99997pt\uparrow$	SSIM $\hskip 1.99997pt\uparrow$	LPIPS $\hskip 1.99997pt\downarrow$
MonoGaussianAvatar	19.73	0.689	0.373
+Kin-Lift (w/o exp. coe.)	25.09	0.849	0.158
+Kin-Lift (w exp. coe.)	26.98	0.903	0.121
SplattingAvatar	18.32	0.657	0.410
+Kin-Lift (w/o exp. coe.)	24.69	0.785	0.264
+Kin-Lift (w exp. coe.)	25.71	0.849	0.132

Table 3: Quantitative comparisons of expression coefficient injection. +Kin-Lift means Kinematic Lifted. (w/o exp. coe.) means without expression coefficient injection, (w exp. coe.) means with expression coefficient injection.

Reference image	PSNR $\hskip 1.99997pt\uparrow$	FVD $\hskip 1.99997pt\downarrow$	FID $\hskip 1.99997pt\downarrow$
K-means-one	28.59	341.55	43.87
K-means-two	28.63	290.94	42.17
K-means-three	28.65	263.67	41.69
K-means-five	28.68	181.90	38.94

Table 4: Quantitative comparison of different reference image counts.

Facial Expression Synthesis

See all 29 papers

Self-Learning Expression Deformations for Data-Efficient Gaussian Avatars
Jun 4, 2026Jiahao Yang, Xiaohang Yang, Qing Wang +33D Gaussian Splatting Self-Supervised Learning
Queen Mary University of London
AGORA: Adversarial Generation Of Real-time Animatable 3D Gaussian Head Avatars
Dec 6, 2025Ramazan Fazylov, Sergey Zagoruyko, Aleksandr Parkin +23D Gaussian Splatting 3D Asset Generation
Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE · Polynome AI, Dubai, UAE · MTS AI, Moscow, Russia
FFAvatar: Feed-Forward 4D Head Avatar Reconstruction from Sparse Portrait Images
Jun 29, 2026Jianjiang Yao, Ke Xian, Renxiang Dai +1Feed-Forward 3D Reconstruction 4D Reconstruction
School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan 430074, China

Jun 4, 2026J/K move · Enter open · S save

Self-Learning Expression Deformations for Data-Efficient Gaussian Avatars

Jiahao Yang, Xiaohang Yang, Qing Wang, Yilan Dong+2

Queen Mary University of London

3D Gaussian Splatting Self-Supervised Learning Dynamic Scene Reconstruction 3D Avatar Generation Human Avatar Animation 3D Face Reconstruction

Open paper AskPDF arXiv

Modeling dynamic facial expressions using 3D Gaussian representations remains challenging due to their unstructured nature. Conventional Gaussian avatar pipelines require extensive multiview and sequential expression data, limiting scalability and accessibility. In this work, we introduce Self-Adaptive Gaussian Expression (SAGE), a framework for self-learning expression-induced Gaussian deformations that enables high-fidelity, animatable avatars from minimal input data. Our method jointly optimizes 2D Gaussian surfels and a Signed Distance Field (SDF) to enforce compact, surface-aligned Gaussian distributions, while a self-supervised expression learning phase replaces long training sequences with geometric and appearance consistency constraints. This design allows flexible deployment across multiple reconstruction regimes: in the multiview setting, only a single frame (timestep) is required instead of thousands; in the monocular setting, only head rotations are needed without expression sequences; and in the one-shot setting, no pretraining or priors are necessary. Experiments demonstrate that our approach achieves reconstruction and animation quality comparable to state-of-the-art methods, while reducing data requirements by several orders of magnitude. Our results highlight the potential of self-supervised Gaussian deformation learning as a step toward accessible, data-efficient avatar creation.

Abstract

Facial Expression Synthesis

Explore similar work

Self-Learning Expression Deformations for Data-Efficient Gaussian Avatars

AGORA: Adversarial Generation Of Real-time Animatable 3D Gaussian Head Avatars

FFAvatar: Feed-Forward 4D Head Avatar Reconstruction from Sparse Portrait Images