A photorealistic and immersive human avatar experience demands capturing fine, person-specific details such as cloth and hair dynamics, subtle facial expressions, and characteristic motion patterns. Achieving this requires large, high-quality datasets, which often introduce ambiguities and spurious correlations when very similar poses correspond to different appearances. Models that fit these details during training can overfit and produce unstable, abrupt appearance changes for novel poses. We propose a 3D Gaussian Splatting avatar model with a spatial MLP backbone that is conditioned on both pose and an appearance latent. The latent is learned during training by an encoder, yielding a compact representation that improves reconstruction quality and helps disambiguate pose-driven renderings. At driving time, our predictor autoregressively infers the latent, producing temporally smooth appearance evolution and improved stability. Overall, our method delivers a robust and practical path to high-fidelity, stable avatar driving.
Figures & tables
Figure 1: We represent the posed avatar with a hierarchical 3D Gaussian structure controlled by per-anchor spatial MLPs. Each anchor receives localized driving features via skinning-weight-based masking of pose parameters θ (and face-region masking of ϕ ), together with an appearance latent li . During training, li is obtained by encoding a per-frame UV texture into a 2D feature map and sampling it at the anchor’s UV coordinates; at test time, a transformer autoregressively predicts temporally smooth latents from a short pose history for stable driving. The encoder reconstructs training poses extremely well (a) and generalizes to novel poses and unseen textures (b), while our appearance predictor yields realistic appearances and smooth transitions on test sequences when textures are not available (c).
Figure 2: Following Zhan et al. [ 46 ] , we initialize a hierarchical point cloud on the template mesh, consisting of anchors/control points/Gaussians (300/10k/200k for our example, colored as orange/blue/red). Every anchor holds an MLP, whose outputs are interpolated by control points and Gaussians from the closest three anchors to calculate their positional displacement and Gaussian correctives.
Figure 3: (a) Providing pose parameter to every anchor leads to spurious correlations between unrelated regions. (b) We locally mask out pose parameters per anchor based on the skinning weights of the template mesh to restrict them to their local region.
Figure 4: (a) Similar poses can exhibit vastly different appearances, leading to an ambiguous one-to-many mapping. Therefore, we model appearance through a texture encoder that receives (b) a multi-view projected UV texture as input (face region masked).
Figure 5: Architecture of our appearance predictor. The transformer predicts the next latents from the previous Nb poses—value, plus their velocity and acceleration calculated via finite differences—and the previous latents.
Test
Train
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
#Params
Ours
32.99
0.939
0.063
36.21
0.950
0.053
192.1M
MMLPs †
32.34
0.937
0.066
36.13
0.950
0.054
187.2M
nRFGCA †
32.74
0.939
0.065
35.94
0.950
0.055
155.9M
Ours (re-init. 30)
33.85
0.942
0.060
–
–
–
192.1M
Ours (encoder)
34.78
0.946
0.056
–
–
–
189.8M
Table 1: Image metrics on all compared methods. As appearance is inherently ambiguous and drifts over time, we initialize the appearance predictor for Ours with the encoder output. Additionally, we show results for re-initializing every 30 frames, and for using the encoder during driving.
Test
Train
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
Ours
32.99
0.939
0.063
36.21
0.950
0.053
Ours w/o local pca
32.86
0.938
0.064
36.23
0.950
0.053
Ours w/o local pose ↓
32.87
0.938
0.064
36.29
0.951
0.052
Table 2: Ablation of our components.
Figure 6: Qualitative comparison of global (MMLPs † ) versus localized (Ours) pose conditioning. We manually perturb a single pose parameter (top: neck twist; bottom: wrist rotate) between two frames: global conditioning induces large, non-local deformations and spurious appearance changes (visualized as ℓ1 error in the cutouts), whereas localized conditioning confines the effect to local, physically plausible motion.
Figure 7: Two temporally adjacent frames from a test sequence and the corresponding cutout ℓ1 error against the previous frame. Our method remains temporally stable due to explicit pose–appearance disentanglement and the learned appearance predictor, while nRFGCA † and MMLPs † exhibit abrupt, implausible appearance changes.
Figure 8: Test-sequence comparison of our method using the texture encoder (unseen texture input) versus our appearance predictor (no textures), alongside MMLPs † . The encoder provides an upper bound closest to the ground truth, while the predictor yields temporally smooth, plausible appearance evolution without texture inputs.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9: Our dataset consists of six captures of varying difficulty.
Figure 10: Multiple image pairs showing the generalization capabilities of our texture encoder on the test set, i.e . , novel poses and unseen UV textures. The left image shows ground truth and the right image is the appearance encoder result.
Figure 11: Test-sequence comparison of our method using the texture encoder (unseen texture input) versus our appearance predictor (no textures), alongside MMLPs † .
PSNR ↑
Actor 1
Actor 2
Actor 3
Actor 4
Actor 5
Actor 6
Mean
Ours
31.52
31.90
28.82
35.03
34.38
36.31
32.99
MMLPs †
31.29
31.79
28.41
33.68
34.31
34.55
32.34
nRFGCA †
31.47
31.94
28.95
34.84
34.29
34.97
32.74
Ours (re-init. 30)
32.39
32.93
30.04
35.80
35.02
36.92
33.85
Ours (encoder)
33.33
33.85
31.48
36.71
35.63
37.67
34.78
SSIM ↑
Actor 1
Actor 2
Actor 3
Actor 4
Actor 5
Actor 6
Mean
Appendix
Table 3: Test set image metrics on all compared methods. We initialize the appearance predictor for Ours with the encoder output and show additional results for re-initializing every 30 frames, and for using the texture encoder on the test sequences.
PSNR ↑
Actor 1
Actor 2
Actor 3
Actor 4
Actor 5
Actor 6
Mean
Ours
34.97
35.40
34.58
37.14
36.52
38.63
36.21
MMLPs †
35.00
35.36
34.22
37.07
36.51
38.62
36.13
nRFGCA †
34.69
35.58
33.42
36.84
36.53
38.56
35.94
SSIM ↑
Actor 1
Actor 2
Actor 3
Actor 4
Actor 5
Actor 6
Mean
Ours
0.939
0.956
0.949
0.956
0.934
0.969
0.950
MMLPs †
0.939
0.956
0.947
0.955
0.934
0.969
0.950
Appendix
Table 4: Training set image metrics on all compared methods.
We introduce AvatarPointillist, a novel framework for generating dynamic 4D Gaussian avatars from a single portrait image. At the core of our method is a decoder-only Transformer that autoregressively generates a point cloud for 3D Gaussian Splatting. This sequential approach allows for precise, adaptive construction, dynamically adjusting point density and the total number of points based on the subject's complexity. During point generation, the AR model also jointly predicts per-point binding information, enabling realistic animation. After generation, a dedicated Gaussian decoder converts the points into complete, renderable Gaussian attributes. We demonstrate that conditioning the decoder on the latent features from the AR generator enables effective interaction between stages and markedly improves fidelity. Extensive experiments validate that AvatarPointillist produces high-quality, photorealistic, and controllable avatars. We believe this autoregressive formulation represents a new paradigm for avatar generation, and we will release our code inspire future research.
The generation of high-fidelity, animatable 3D human avatars remains a core challenge in computer graphics and vision, with applications in VR, telepresence, and entertainment. Existing approaches based on implicit representations like NeRFs suffer from slow rendering and dynamic inconsistencies, while 3D Gaussian Splatting (3DGS) methods are typically limited to static head generation, lacking dynamic control. We bridge this gap by introducing AGORA, a novel framework that extends 3DGS within a generative adversarial network to produce animatable avatars. Our key contribution is a lightweight, FLAME-conditioned deformation branch that predicts per-Gaussian residuals, enabling identity-preserving, fine-grained expression control while allowing real-time inference. Identity is further preserved through spatial shape conditioning of the identity branch, and expression fidelity is enforced via a dual-discriminator training scheme leveraging synthetic renderings of the parametric mesh. AGORA generates avatars that are not only visually realistic but also precisely controllable. Quantitatively, we outperform state-of-the-art NeRF-based methods on expression accuracy while rendering at 250 FPS on a single GPU and, notably, at ∼9 FPS under CPU-only inference -- to our knowledge the first demonstration of CPU-only animatable 3DGS avatar synthesis. This work represents a significant step toward practical, high-performance digital humans. Project website: https://ramazan793.github.io/AGORA/
Ramazan Fazylov, Sergey Zagoruyko, Aleksandr Parkin +2
Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE · Polynome AI, Dubai, UAE · MTS AI, Moscow, Russia
Reconstructing a photorealistic 3D face avatar from a single unconstrained photograph is challenging: feed-forward 3D Gaussian Splatting (3DGS) models degrade on out-of-distribution inputs, while pretrained diffusion models produce high-fidelity images but lack multi-view consistency. We observe that these paradigms are fundamentally complementary: explicit 3D representations guarantee geometric consistency, whereas 2D diffusion priors ensure photorealism. Building on this, we propose SplatShot, a training-free framework that couples these representations directly within the denoising process. Given a base 3DGS face model and a single reference image, we jointly denoise all target views using a per-step 3D feedback loop. At each timestep, we predict clean images from the noisy latents, refit the 3DGS to these multi-view predictions, and back-propagate the photometric discrepancy between the 3DGS re-renderings and 2D predictions into the noise estimate. This steers the sampling trajectory toward strictly 3D-coherent, identity-faithful outputs. Experiments on diverse in-the-wild images demonstrate that SplatShot produces 3D avatars with superior identity preservation, photorealism, and multi-view consistency.
Hao Liang, Zhixuan Ge, Soumendu Majee +3
Rice University Houston, TX 77005 · Samsung Research America Plano, TX 75023