Recovering identity-faithful, physically based surface appearance for 3D human heads from sparse images remains challenging. Existing methods often rely on UV-based pipelines, introducing seam artefacts, or operate in image space, limiting surface consistency. We present VISAGE, Vertex-based Intrinsic Surface Appearance GEneration, a surface-based conditional diffusion framework that synthesises per-vertex albedo and roughness directly on arbitrary mesh topologies, without requiring consistent UV parameterisation at training time. VISAGE combines global spectral processing with local geodesic K-Nearest-Neighbour refinement, enforcing structural coherence while restoring detail beyond the truncated spectral basis. Multi-view RGB images condition the network via visibility-aware cross-attention, enabling occlusion-robust synthesis across the full head surface. Region-specific modules specialise in semantically demanding and distinct areas (eyes, mouth, skin and hair), while a tangent-space output projection recovers fine iris detail. We further introduce a scalable procedural augmentation pipeline that enriches raw geometry scans with physically plausible albedo, roughness and normal maps for training. The predicted attributes render directly as a single RGBA vertex-colour channel. VISAGE reduces FID by 42.7% on NPHM and 22.5% and 4.4% on zero-shot FaceVerse and i3DMM, respectively, relative to the strongest baseline, with a 7.2M-parameter core network.
Figures & tables
Figure 1: Given sparse 1-5 RGB images and an untextured head mesh (left), VISAGE predicts per-vertex albedo and roughness directly on arbitrary mesh topologies (center). The predicted intrinsic materials enable relighting under unseen illumination (right).
Figure 2: Overview of VISAGE pipeline. Noisy vertex features xt are concatenated with geometric descriptors and conditioned on multi-view image features before entering the dual-stream diffusion network. The network combines global spectral propagation with local geodesic refinement to predict the denoised representation x0 , which is projected to per-vertex albedo and roughness.
Figure 3: Enhanced Diffusion Block (EDB). Input features xin are first grounded with multi-view image evidence through image-to-mesh cross-attention. The conditioned features are processed by two complementary parallel streams: noise-aware spectral heat diffusion for global propagation and geodesic KNN self-attention for local refinement. The fused representation is modulated by the diffusion noise level σ and projected to xout .
Figure 4: Qualitative comparison on NPHM, FaceVerse, and in-the-wild images. VISAGE better preserves identity-specific appearance, including skin tone, facial hair, and eye colour, while producing more coherent skin and better-resolved eye detail than competing methods. Fine-scale wrinkles around the eyes and mouth (in-the-wild rows) are recovered by none of the compared methods, including ours. Eyes are blurred for FaceVerse [ 38 ] examples following the dataset policy.
Figure 5: Perceptual user study. Left: overall preference across all participants. Right: subject-wise preference distribution. VISAGE is preferred over competing methods in the majority of trials.
Figure 6: Effect of region specialisation and tangent refinement. Removing semantic region splitting degrades iris structure, while removing tangent-space refinement produces blurry eye details. Combining both components restores sharp, realistic eye appearance.
Figure 7: Scaling behaviour. (a,b) Appearance quality against mesh resolution: the model trained at 120K vertices, evaluated without retraining on NPHM [ 18 ] decimated to 30K–250K vertices. The intrinsic operators depend on the surface rather than a fixed tessellation, so a single set of weights covers the whole range and resolution can be traded against computational cost at inference time. (c) Effect of the number of conditioning images, with diminishing returns beyond three views.
Figure 8: Albedo illumination invariance. Results on the NPHM [ 18 ] validation split under 10 environment maps. Spread is the mean CIEDE2000 [ 28 ] distance between each illumination’s predicted albedo and the per-vertex mean prediction; GT is the distance to ground-truth albedo, and Observed is the corresponding variation in the input observations. Table: regional mean ± std across validation subjects. Plot: results by environment map. Predicted variation remains substantially below observed variation.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9: Visual variance of our procedural PBR augmentation pipeline. Top Row: Generated roughness maps showcasing varying simulated skin types, ranging from very dry (left) to oily (right). Bottom Row: Generated tangent-space normal maps demonstrating the effect of our randomised factors α and β . Ranging from the lowest possible strength (left), to the mid-point of strength (middle), and the highest possible strength (right). The overlaid insets highlight the procedural multi-scale noise used to synthesise micro-pores and directional lip wrinkles.
Region
Parameter
Sampling
Range / Values
Skin
Age Factor α
Uniform
[0,1]
Skin Type
Categorical
5 classes
Roughness baseline shift
Uniform (per class)
[0.0,1.0]
Normal map strength
Derived from α
[0.25,1.6]
Mouth/Lips
Lip fissure strength β
Uniform
[0.15,0.5]
Wetness Factor
Uniform
[0.0,1.0]
Appendix
Table 3: Per-region augmentation parameters and sampling distributions. All continuous parameters are sampled uniformly within the stated range unless otherwise noted.
Hyperparameter
Albedo
Roughness
Base feature dimensionality d
64
64
Attention heads
4
4
Cross-attention patch size r
5
5
KNN neighbourhood size k
21
– (omitted)
EDBs with cross-attention
{1, 4, 5, 6, 7, 9}
{1, 2, 3, 4}
EDBs with k -NN attention
{1, 4, 5, 6, 7, 9}
– (omitted)
Appendix
Table 4: Architectural and noise-scheduling hyperparameters for the albedo and roughness prediction networks.
Block
Layer Type
Kernel / Stride
Out Channels
Activation, Norm
Mapping
Conv2d
3×3 / 1
16
GroupNorm(2), SiLU
Conv2d
3×3 / 1
32
GroupNorm(4), SiLU
Downsample 1
Conv2d
3×3 / 2
32
GroupNorm(4), SiLU
Downsample 2
Conv2d
3×3 / 2
32
GroupNorm(4), SiLU
Downsample 3
Conv2d
3×3 / 2
32
GroupNorm(4), SiLU
Appendix
Table 5: Architecture of the Image Pre-Processing CNN. The mapping block projects input images to feature space, followed by three sequential downsampling blocks.
Block
Layer Type
Kernel / Stride
Out Channels
Activation, Norm
Encoder 1
Conv2d × 2
3×3 / 1
32
GroupNorm(8), SiLU
Down 1
Conv2d
3×3 / 2
64
–
Encoder 2
Conv2d
3×3 / 1
64
GroupNorm(16), SiLU
Down 2
Conv2d
3×3 / 2
128
–
Bottleneck
Conv2d
3×3 / 1
128
GroupNorm(32), SiLU
Up 1
ConvTranspose2d
2×2 / 2
64
–
Appendix
Table 6: Architecture of the Tangent Refinement U-Net. This lightweight network processes 64×64 tangent-space patches to recover high-frequency eye and iris details.
Figure 10: Spectral reconstruction trade-off. Increasing the number of Laplace–Beltrami eigenpairs reduces reprojection error but increases inference time. We use K=300 as a practical trade-off between accuracy and efficiency.
Figure 11: Robustness to geometric noise. LPIPS under Gaussian noise, quantization, and surface spikes. VISAGE degrades gracefully under increasing mesh perturbations, indicating robustness to imperfect input geometry.
Figure 12: Contribution of predicted roughness on NPHM dataset [ 18 ] . Predicted albedo rendered with a single global constant roughness ( r=0.59 , the area-weighted training-set mean) against the predicted per-vertex roughness, alongside the ground-truth render. All conditions share geometry, albedo, camera and illumination.
Figure 13: Qualitative ablation of key architectural components on the NPHM dataset. Top Row: Removing the local k -NN refinement stream forces the network to rely on the EDB’s MLP, resulting in grainy artefacts and a loss of high-frequency sharpness, like beard. Middle Row: Disabling noise-level conditioning in the heat diffusion step prevents dynamic spatial adaptation, causing texture bleeding and boundary collapse in occluded areas. Bottom Row: Without semantic region splitting, a shared global MLP fails to capture the unique statistical distributions of specialised regions such as the mouth and eyes, degrading realism.
Figure 14: Effect of the local high-frequency stream. Removing Stream 2 leaves the spectral branch alone, whose truncated Laplace–Beltrami basis acts as a low-pass filter, and the MLP in each EDB block needs to retain and enhance local features, which leads to a noisy, grainy look.
Figure 15: Typical failure cases of VISAGE. Top Block: Results from the NPHM dataset [ 18 ] , where sometimes the colour in the occluded hair region shifts slightly and gets brighter, compared to hair colour in no occluded areas on the right. Bottom Block: Results from FaceVerse dataset [ 38 ] , in which strong directional lighting and harsh cast shadows are occasionally baked into the predicted albedo rather than being fully decomposed.
Digitizing humans and synthesizing photorealistic avatars with explicit 3D pose and camera controls are central to VR, telepresence, and entertainment. Existing skinning-based workflows require laborious manual rigging or template-based fittings, while neural volumetric methods rely on canonical templates and re-optimization for each unseen pose. We present PoseCraft, a diffusion framework built around tokenized 3D interface: instead of relying only on rasterized geometry as 2D control images, we encode sparse 3D landmarks and camera extrinsics as discrete conditioning tokens and inject them into diffusion via cross-attention. Our approach preserves 3D semantics by avoiding 2D re-projection ambiguity under large pose and viewpoint changes, and produces photorealistic imagery that faithfully captures identity and appearance. To train and evaluate at scale, we also implement GenHumanRF, a data generation workflow that renders diverse supervision from volumetric reconstructions. Our experiments show that PoseCraft achieves significant perceptual quality improvement over diffusion-centric methods, and attains better or comparable metrics to latest volumetric rendering SOTA while better preserving fabric and hair details.
We present a single-image head mesh reconstruction framework that addresses the longstanding challenge of simultaneously preserving facial identity and producing industry-grade topology. Our framework adopts a coarse-to-fine optimization pipeline that refines a rigged template across three stages -- rig, joint, and vertex -- achieving stable convergence and consistent topology. To mitigate the ill-posed nature of single-image 3D face reconstruction and ensure identity preservation, we employ a normal consistency objective jointly with landmark alignment. To further preserve local surface structure and enforce topological regularity, we introduce geometry-aware constraints based on Gaussian curvature and conformal consistency, along with auxiliary regularizations that correct fine artifacts such as lip seams and eyelid discontinuities. Our hierarchical optimization with geometry-aware regularization yields meshes with semantically meaningful edge flow and industry-grade topology. After geometry reconstruction, we extract UV-space texture and normal maps to preserve appearance details for visualization and downstream use. In a user study with 22 professional technical artists, our results were assessed as approaching industry-grade usability, and 95% of participants ranked our method as the top-performing approach, underscoring its effectiveness for real-world digital human production.
We present GeoFace, a geometry-constrained multi-view diffusion framework for consistent face generation from a single input. % While recent multi-view diffusion models achieve photorealistic synthesis at the per-view level, they lack an explicit mechanism to enforce a shared 3D structure across views, often leading to inconsistent geometry across viewpoints. To address this, GeoFace proposes a unified dual-stream framework for joint generation of multi-view RGB images and 3D face geometry, where the appearance and geometry streams interact through shared attention layers. To encourage the two streams to mutually constrain each other, we introduce a geometry-guided attention alignment loss that supervises the cross-attention between appearance and geometry tokens with 3D-consistent correspondences, enabling the appearance stream to correctly reference pose-invariant geometric cues for robust alignment across viewpoints. Geometry is represented as a canonical UV position map, derived from a FLAME mesh fitted to multi-view observations, serving as a view-invariant shared constraint across all generated views. Experiments on RenderMe-360 and NeRSemble demonstrate that GeoFace consistently outperforms existing methods in both visual quality and cross-view geometric consistency, facilitating more efficient 3D reconstruction.