Generating high-fidelity 3D avatars from text or image prompts is highly sought after in virtual reality and human-computer interaction. However, existing text-driven methods often rely on iterative Score Distillation Sampling (SDS) or CLIP optimization, which struggle with fine-grained semantic control and suffer from excessively slow inference. Meanwhile, image-driven approaches are severely bottlenecked by the scarcity and high acquisition cost of high-quality 3D facial scans, limiting model generalization. To address these challenges, we first construct a novel, large-scale dataset comprising over 100,000 pairs across four modalities: fine-grained textual descriptions, in-the-wild face images, high-quality light-normalized texture UV maps, and 3D geometric shapes. Leveraging this comprehensive dataset, we propose PromptAvatar, a framework featuring dual diffusion models. Specifically, it integrates a Texture Diffusion Model (TDM) that supports flexible multi-condition guidance from text and/or image prompts, alongside a Geometry Diffusion Model (GDM) guided by text prompts. By learning the direct mapping from multi-modal prompts to 3D representations, PromptAvatar eliminates the need for time-consuming iterative optimization, successfully generating high-fidelity, shading-free 3D avatars in under 10 seconds. Extensive quantitative and qualitative experiments demonstrate that our method significantly outperforms existing state-of-the-art approaches in generation quality, fine-grained detail alignment, and computational efficiency.
Figures & tables
Figure 1: PromptAvatar generates realistic and animatable 3D avatars from a single text prompt, image, or both, compatible with 3D rendering engines like Blender. The top-left corner uses a text prompt to create an accurate texture UV-map and mesh. The bottom-left corner combines text with an FLUX-dev-1.0-generated [ 1 ] image to guide high-quality texture UV-map creation. When an image prompt is used, facial geometry is extracted via a pre-trained 3D face reconstruction network [ 2 , 3 ] . On the right, image prompts enable detailed texture effects like crow’s feet and beards.
Figure 2: Our dataset creation pipeline consists of three main modules: De-lighting and Re-lighting face image generation, Incomplete UV-map correction and completion, and Identity coefficients estimation and facial attribute description.
Figure 3: Architecture of PromptAvatar. The framework comprises a Texture Diffusion Model (TDM), which targets high-quality normalized texture generation, and a Geometry Diffusion Model (GDM) for geometric identity coefficients. Both models are guided by multi-modal prompts embedded via CLIP. For image prompts, incomplete textures are encoded into the latent space to provide localized guidance.
Instruction
Provide a detailed description of this portrait’s appearance features within 200 tokens, ignoring hair and expression. Please answer in the following format, for example:
{{ Pbase : A middle-aged male about 50s, Asian yellow human},
{ Ptex : Yellow skin with highland red undertones, large pores, thick and long eyebrows, chin and cheeks covered with beard, deep wrinkles around the forehead and eyes, wrinkles between the eyes, freckles distributed on the cheeks, slight eye bags, red lips, deep red nose, dark circles, deep gray beard, black eyebrows, melanin deposits},
{ Pgeo : rectangular face, overall slender, forehead slightly broad and protruding, willow leaf eyebrows, large almond eyes, relatively narrow eye spacing, high nasal bridge, rounded nose tip, slightly high cheekbones, cherry lips, round but slightly upturned narrow chin, overall carrying a hint of a calm demeanor.}}
Note that this is very important to my work. Your description should be as detailed and fair as possible.
Table 1: The instruction template used for Qwen2.5-VL-32B-Instruct to generate structured facial attribute descriptions.
Figure 4: The network architecture of GDM.
Similarity ↑
left-front
left-right
front-right
Ours
0.754
0.801
0.774
FFHQ-UV
0.634
0.544
0.632
Table 2: Quantitative evaluation of Identity Similarity on multi-view images.
Figure 5: Visual comparison with DreamFace and Describe3D. Our results demonstrate superior alignment with fine-grained text prompts (highlighted in red). For instance, in the first row, observe the distribution of facial hair, eyebrow shape, and chin, and in the second row, note the facial shape, eye bags, and other features.
Metric
DreamFusion
Describe3D
DreamFace
Ours
CLIP score ↑
20.16
19.81
20.56
21.14
Infer time(s) ↓
2400
15
300
10
Table 4: Quantitative comparison with other text-to-avatar methods.
Figure 6: Demonstration of fine-grained facial attribute editing via text prompts. Starting from an initial generated UV-map (left), our Texture Diffusion Model (TDM) accurately manipulates specific semantic features—such as gender, facial hair, age, and ethnicity—by simply modifying the corresponding keywords in the prompts (highlighted in red). The results show that TDM achieves precise localized editing while preserving the unedited features.
Figure 7: Visual comparison of the image-to-avatar methods. Each group consists of the original image, the re-lit PromptAvatar, and FFHQ-UV. It can be observed that our method is able to preserve more facial details and skin tone of the original image. Please zoom in for better visibility of the details.
Similarity ↑
FFHQ-UV
FlameTex
Ours
w/emb
w/both
FFHQ
0.354
0.1589
0.370
0.284
0.338
CMask-HQ
0.324
0.114
0.334
0.242
0.304
Infer times(s) ↓
80
72
6
/
/
Table 5: Quantitative experiments of Image-to-Avatar and Ablation studies. CMask-HQ is CelebAMask-HQ.
Figure 8: Ablation study on VAE fine-tuning. Left: Original in-the-wild image. Middle: Texture UV-map generated using the frozen pre-trained SD-2.1 VAE, which results in overly smoothed high-frequency details. Right: Texture generated using our fine-tuned VAE. The fine-tuned VAE successfully adapts to the UV-map data distribution, effectively recovering realistic facial features such as crow’s feet and wrinkles (highlighted in the red boxes).
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Part I: Coarse-grained Prompts
55-year-old Caucasian male, rosy cheeks, noticeably thin face with slightly sunken cheeks.
African child, round face, small eyes, thick lips, short curly hair.
Elderly Caucasian male, elegant appearance, big eyes, long pointed nose with visible wrinkles, wide lips.
30-year-old Middle Eastern male, prominent nose, thick eyebrows, medium beard, calm and regal facial expression.
Table 6: Extended list of text prompts (Part I). These coarse-grained descriptions focus on general demographic attributes and basic facial features, explicitly omitting non-facial body descriptions.
Part II: Fine-grained Prompts
A Caucasian male in his 30s with medium-tone skin and neutral undertones has some visible pores, very light stubble on the chin and upper lip, a few faint smile lines around the mouth, minimal creases under the eyes, and slightly noticeable frown lines on the forehead.
A mixed man in his 40s with a personable appearance has fair skin and rosy cheeks. His complexion is clear with subtle lines under his eyes and faint laugh lines around his mouth.
A young adult Caucasian male in his 30s with a professional appearance has fair skin with slight pink undertones, small pores, and a clean-shaven look.
A 30-year-old light-skinned adult male with olive skin tone and faint pink undertones has small pores and slight nasolabial fold visibility.
A Caucasian male in his 30s has light skin with subtle pink undertones and visible pores. He has a small number of faint freckles on the cheeks and minimal signs of wrinkles, mainly in the nasolabial fold area.
A middle-aged Asian male in his 50s with a professional demeanor. His skin features relatively small pores, faint fine lines, and wrinkles mainly on the forehead and around the eyes, with some visible around the mouth.
Appendix
Table 7: Extended list of text prompts (Part II). These fine-grained descriptions incorporate diverse and challenging facial details such as pores, wrinkles, acne, and pigmentation.
Figure 9: Visual examples of the multi-modal facial dataset. Each row displays a unique identity from our collection. From left to right: multi-view source images, unwrapped high-resolution texture maps, facial renderings under novel lighting, and the corresponding 3D geometry.
Figure 10: More visual comparison of the image-to-avatar methods. The [red dashed boxes] indicate that our method exhibits better details than FFHQ-UV, while the [blue dashed boxes] indicate flaws in FFHQ-UV. The bottom-right corner of each image in the first column is the environment map used for rendering. The FlameTex and Deep3D results use textures directly estimated by each method for visualization.
We present GenLCA, a diffusion-based generative model for generating and editing photorealistic full-body avatars from text and image inputs. The generated avatars are faithful to the inputs, while supporting high-fidelity facial and full-body animations. The core idea is a novel paradigm that enables training a full-body 3D diffusion model from partially observable 2D data, allowing the training dataset to scale to millions of real-world videos. This scalability contributes to the superior photorealism and generalizability of GenLCA. Specifically, we scale up the dataset by repurposing a pretrained feed-forward avatar reconstruction model as an animatable 3D tokenizer, which encodes unstructured video frames into structured 3D tokens. However, most real-world videos only provide partial observations of body parts, resulting in excessive blurring or transparency artifacts in the 3D tokens. To address this, we propose a novel visibility-aware diffusion training strategy that replaces invalid regions with learnable tokens and computes losses only over valid regions. We then train a flow-based diffusion model on the token dataset, inherently maintaining the photorealism and animatability provided by the pretrained avatar reconstruction model. Our approach effectively enables the use of large-scale real-world video data to train a diffusion model natively in 3D. We demonstrate the efficacy of our method through diverse and high-fidelity generation and editing results, outperforming existing solutions by a large margin. The project page is available at https://onethousandwu.com/GenLCA-Page.
Yiqian Wu, Rawal Khirodkar, Egor Zakharov +6
State Key Laboratory of CAD&CG, Zhejiang University, China · Codec Avatars Lab, Meta, USA · State Key Lab of CAD&CG, Zhejiang University, China
Single-image 3D human reconstruction often suffers from over-smoothed textures and geometric inconsistencies. While diffusion models improve generative quality, their reliance on multi-view synthesis prior to 3D reconstruction is computationally expensive and prone to view inconsistency. We propose DiGS-Avatar, which reformulates this task as an efficient, diffusion-based UV-latent completion task, ensuring 3D consistency by design. To capture accurate spatial structure, we introduce a teacher-student framework where a multi-view teacher provides geometrically aligned pseudo-ground-truth latents to supervise a single-view diffusion student. Treating this inferred latent as a robust structural skeleton, our method injects high-level semantic features to accurately recover fine textural details without disrupting spatial integrity. The refined representation is then decoded into 3D Gaussian primitives. Extensive experiments demonstrate that DiGS-Avatar achieves state-of-the-art or highly competitive visual fidelity and zero-shot generalization, while reconstructing a fully animatable 3D avatar in just 0.71 seconds. Code is available at https://github.com/KLMAV-CUC/DiGS-Avatar.
Jiakun Li, Li Fang, Hao Zhu +4
Key Laboratory of Media Audio and Video (Communication University of China), Ministry of Education, Beijing 100024, China · School of Intelligence Science and Technology, Nanjing University, Nanjing 210023, China
We propose S-Avatar, a novel method for generating photorealistic 3D head avatars from a single image using a diffusion-guided 3D model generation module and strategies for animating 3D Gaussian Splatting (3DGS). While single-image head avatar reconstruction is crucial for lifelike Virtual Reality (VR) applications, existing approaches often struggle to preserve 3D consistency under unseen viewpoints. S-Avatar addresses this limitation through a three-stage pipeline. First, a high-resolution 3DGS is synthesized directly from a single image using a diffusion-based Gaussian splat generation module. Next, the parametric head model FLAME is aligned with the generated 3DGS by optimizing its parameters and spatial transformations. Finally, to adapt the 3DGS to FLAME variations, we construct a binding template that encodes the spatial relationship between the initial splats and FLAME. The dynamic 3D head avatar can then be rendered in real time by deforming the 3DGS with the binding template. By combining diffusion-guided canonical 3DGS generation with FLAME-based control, our method achieves efficient and accurate reconstruction with enhanced 3D consistency. Evaluations on public datasets demonstrate that S-Avatar outperforms state-of-the-art methods in novel-view and expression generation, achieving superior realism and consistency. Consequently, our approach represents a significant advance in accessible avatar creation, applicable to a wide range of VR/AR applications. The project page is available at https://github.com/hailsong/savatar.