LiftAvatar: Kinematic-Space Completion for Expression-Controlled 3D Gaussian Avatar Animation
Authors: Hualiang Wei, Shunran Jia, Jialun Liu, Wenhui Li
Organizations: College of Computer Science and Technology, Jilin University, No. 2699 Qianjin Street, Changchun, 130012, Jilin, China · Impressed Inc DBA SocialBook, 950 Tower Ln, Foster City, 94404, California, USA · Institute of Artificial Intelligence of China Telecom, Zhonghui Building, Dongcheng District, 100007, Beijing, China
We present LiftAvatar, a new paradigm that completes sparse monocular observations in kinematic space (e.g., facial expressions and head pose) and uses the completed signals to drive high-fidelity avatar animation. LiftAvatar is a fine-grained, expression-controllable large-scale video diffusion Transformer that synthesizes high-quality, temporally coherent expression sequences conditioned on single or multiple reference images. The key idea is to lift incomplete input data into a richer kinematic representation, thereby strengthening both reconstruction and animation in downstream 3D avatar pipelines. To this end, we introduce (i) a multi-granularity expression control scheme that combines shading maps with expression coefficients for precise and stable driving, and (ii) a multi-reference conditioning mechanism that aggregates complementary cues from multiple frames, enabling strong 3D consistency and controllability. As a plug-and-play enhancer, LiftAvatar directly addresses the limited expressiveness and reconstruction artifacts of 3D Gaussian Splatting-based avatars caused by sparse kinematic cues in everyday monocular videos. By expanding incomplete observations into diverse pose-expression variations, LiftAvatar also enables effective prior distillation from large-scale video generative models into 3D pipelines, leading to substantial gains. Extensive experiments show that LiftAvatar consistently boosts animation quality and quantitative metrics of state-of-the-art 3D avatar methods, especially under extreme, unseen expressions.
Figures & tables
Figure 1: We propose a novel kinematic lifted framework, LiftAvatar, to complement the facial expressions and head poses of the monotonous input video. LiftAvatar can promote subsequent reconstruction and driving tasks, resulting in significant improvements.
Figure 2: Method Pipeline. LiftAvatar is a high-precision, expression‑controlled video diffusion transformer that enriches sparse observations to boost downstream avatar performance. It conditions on three groups of inputs: reference information (IR,SR,ER) from the input video; driving information (SD,ED) for the target motion; and the ground‑truth driving video VD during training (replaced by noise at inference). The reference images are encoded by a pre‑trained VAE, and the Reference NPHM Encoder encodes their shading maps SR . These features are concatenated, projected via Reference Patch Embedding, and summed with the embedded expression coefficients ER to form the reference token xR . Likewise, the Driven NPHM Encoder processes the driving shading maps SD ; its output is projected by Driven Patch Embedding and combined with the embedded ED to produce the driving token xD . The tokens xR and xD are concatenated and fed as a unified condition into the Wan2.1 Wang et al. (2025a) video diffusion transformer backbone. Optimized with a flow‑matching objective, the model synthesizes high‑fidelity, temporally coherent videos that accurately follow the driving signals, thereby completing the kinematic space of the original input.
Figure 3: Qualitative results for kinematic lifted. We compared our results with non-diffusion-based methods (FOMM Siarohin et al. (2019) and Face Vid2vid Wang et al. (2021) ) as well as diffusion-based models (DiffusionAvatars Kirschstein et al. (2024) and LivePortrait Guo et al. (2024) and HunyuanPortrait Xu et al. (2025) ). It is evident that our method provides better performance in generating extreme expressions, particularly in terms of facial texture details, teeth accuracy, and pose accuracy.
Figure 4: Qualitative results for head avatar animation. We compare the two head avatar animation methods before and after kinematic lifted. The comparison shows that our proposed strategy can effectively enhance subsequent reconstruction and driving.
Figure 5: Lifted results with LiftAvatar.
Figure 6: Additional Results from MonoGaussianAvatar and LAM.
Figure 7: Additional Comparative Results of SplattingAvatar.
Figure 8: We first cropped the original data from 2200×3208 pixels to 512×512 pixels. Next, we performed NPHM Giebenhain et al. (2023) preprocessing on the cropped images, and finally, shaded them into Phong images.
Figure 9: The Number of Reference Images. Optimal performance is achieved with five reference frames selected by K-means (K-means-5) during inference.
Figure 10: User Study (50 users). In the evaluation conducted by 50 users on the kinematic lifted facial expressions, our model achieved the highest scores and significantly outperformed all other compared models.
Figure 11: The Number of Kinematic Lifted Expressions. The impact of the number of expressions on the effectiveness of different kinematic lifted methods.
Method
Image Quality
Other Metrics
PSNR ↑
LPIPS ↓
SSIM ↑
FID ↓
AED ↓
CSIM ↑
FOMM Siarohin et al. (2019)
21.9167
0.2047
0.6823
60.31
0.7433
0.9721
Face Vid2vid Wang et al. (2021)
21.3089
0.1980
0.7150
54.61
0.7851
0.9718
DiffusionAvatars Kirschstein et al. (2024)
25.3098
0.1479
0.8119
40.84
0.6525
0.9814
LivePortrait Guo et al. (2024)
25.6561
0.1470
0.8237
40.55
0.6372
0.9855
HunyuanPortrait Xu et al. (2025)
27.2258
0.1137
0.8412
39.24
0.6144
0.9891
Table 1: Quantitative comparison of expression kinematic lifted.
Method
FOMM+exp
Face Vid2vid+exp
DiffusionAvatars+exp
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
SplattingAvatar Shao et al. (2024)
20.92
0.683
0.247
21.65
0.710
0.198
24.96
0.840
0.145
MonoGaussianAvatar Chen et al. (2024b)
22.50
0.790
0.232
22.80
0.751
0.192
25.10
0.885
0.137
Method
LivePortrait+exp
HunyuanPortrait+exp
LiftAvatar+exp
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
SplattingAvatar Shao et al. (2024)
25.06
0.840
0.186
25.32
0.843
0.153
25.71
0.849
0.132
Table 2: Quantitative comparisons of different expression kinematic lifted methods.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
MonoGaussianAvatar
19.73
0.689
0.373
+Kin-Lift (w/o exp. coe.)
25.09
0.849
0.158
+Kin-Lift (w exp. coe.)
26.98
0.903
0.121
SplattingAvatar
18.32
0.657
0.410
+Kin-Lift (w/o exp. coe.)
24.69
0.785
0.264
+Kin-Lift (w exp. coe.)
25.71
0.849
0.132
Table 3: Quantitative comparisons of expression coefficient injection. +Kin-Lift means Kinematic Lifted. (w/o exp. coe.) means without expression coefficient injection, (w exp. coe.) means with expression coefficient injection.
Reference image
PSNR ↑
FVD ↓
FID ↓
K-means-one
28.59
341.55
43.87
K-means-two
28.63
290.94
42.17
K-means-three
28.65
263.67
41.69
K-means-five
28.68
181.90
38.94
Table 4: Quantitative comparison of different reference image counts.
Modeling dynamic facial expressions using 3D Gaussian representations remains challenging due to their unstructured nature. Conventional Gaussian avatar pipelines require extensive multiview and sequential expression data, limiting scalability and accessibility. In this work, we introduce Self-Adaptive Gaussian Expression (SAGE), a framework for self-learning expression-induced Gaussian deformations that enables high-fidelity, animatable avatars from minimal input data. Our method jointly optimizes 2D Gaussian surfels and a Signed Distance Field (SDF) to enforce compact, surface-aligned Gaussian distributions, while a self-supervised expression learning phase replaces long training sequences with geometric and appearance consistency constraints. This design allows flexible deployment across multiple reconstruction regimes: in the multiview setting, only a single frame (timestep) is required instead of thousands; in the monocular setting, only head rotations are needed without expression sequences; and in the one-shot setting, no pretraining or priors are necessary. Experiments demonstrate that our approach achieves reconstruction and animation quality comparable to state-of-the-art methods, while reducing data requirements by several orders of magnitude. Our results highlight the potential of self-supervised Gaussian deformation learning as a step toward accessible, data-efficient avatar creation.
The generation of high-fidelity, animatable 3D human avatars remains a core challenge in computer graphics and vision, with applications in VR, telepresence, and entertainment. Existing approaches based on implicit representations like NeRFs suffer from slow rendering and dynamic inconsistencies, while 3D Gaussian Splatting (3DGS) methods are typically limited to static head generation, lacking dynamic control. We bridge this gap by introducing AGORA, a novel framework that extends 3DGS within a generative adversarial network to produce animatable avatars. Our key contribution is a lightweight, FLAME-conditioned deformation branch that predicts per-Gaussian residuals, enabling identity-preserving, fine-grained expression control while allowing real-time inference. Identity is further preserved through spatial shape conditioning of the identity branch, and expression fidelity is enforced via a dual-discriminator training scheme leveraging synthetic renderings of the parametric mesh. AGORA generates avatars that are not only visually realistic but also precisely controllable. Quantitatively, we outperform state-of-the-art NeRF-based methods on expression accuracy while rendering at 250 FPS on a single GPU and, notably, at ∼9 FPS under CPU-only inference -- to our knowledge the first demonstration of CPU-only animatable 3DGS avatar synthesis. This work represents a significant step toward practical, high-performance digital humans. Project website: https://ramazan793.github.io/AGORA/
Ramazan Fazylov, Sergey Zagoruyko, Aleksandr Parkin +2
Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE · Polynome AI, Dubai, UAE · MTS AI, Moscow, Russia
We present FFAvatar, a Transformer-based 3D Gaussian framework for fast construction of high-quality and animatable 4D head avatars from one or more reference portrait images. Unlike existing feed-forward approaches that require a fixed number of input views, FFAvatar supports incremental reconstruction, progressively refining the avatar representation as additional reference images become available. At the core of our method is an alternating attention mechanism that disentangles identity appearance from expression and viewpoint variations, enabling the reconstruction of a canonical 3D appearance that remains consistent across poses and facial expressions. To balance visual fidelity and computational efficiency, we introduce a sparse-to-dense learning paradigm. Coarse appearance features are first learned using sparse primitives anchored to the FLAME vertex level and are subsequently densified in the UV domain to capture fine-grained geometric and texture details. We further propose a plug-and-play motion refinement module that enables subject-specific dynamic personalization by modeling residual motion beyond parametric deformation. Extensive experiments demonstrate that FFAvatar efficiently produces high-fidelity and controllable 4D head avatars, achieving superior flexibility, driving efficiency, and identity-consistent rendering across diverse expressions and viewpoints.
Jianjiang Yao, Ke Xian, Renxiang Dai +1
School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan 430074, China