HairFlow: Modeling 3D Hair Strands with Multi-Modal Conditioned Diffusion

Yating Wang; Kui Wu; Ran Yi; Zixuan Lu; Tongtong Wang; Zhongtian Zheng; Lizhuang Ma; Yuefan Shen

cs.CVOct 12, 2026

HairFlow: Modeling 3D Hair Strands with Multi-Modal Conditioned Diffusion

Authors: Yating Wang, Kui Wu, Ran Yi, Zixuan Lu, Tongtong Wang, Zhongtian Zheng, Lizhuang Ma, Yuefan Shen

Organizations: Shanghai Jiao Tong University · LIGHTSPEED

Abstract

Modeling 3D hair strands is challenging due to complex hairstyle geometry, scarce high-quality training data, and incomplete or noisy input observations. Foundation models offer broader hairstyle priors, but their outputs are not directly usable for strand-based rendering and simulation. Existing approaches require costly per-instance optimization to fit hair priors to meshes from foundation models and 2D orientations from specialized predictors trained on limited hair data, restricting efficiency and generalization. We present HairFlow, a multimodal conditioned diffusion framework for efficient and controllable hair strands modeling. HairFlow conditions strand generation on two complementary representations: a coarse mesh that constrains global shape and multi-view sketches that specify local strand flow. HairFlow combines an Orientation VAE with a conditioned diffusion transformer to generate volumetric fields that are traced into explicit strands. To overcome the lack of paired sketch-strand training data, we introduce an automatic synthesis pipeline that generates geometrically aligned multi-view sketches with diverse styles representative of inference-time inputs. Experiments show that HairFlow generates plausible 3D strands that faithfully follow the input conditions across diverse hairstyles, substantially reduces inference time, and supports flexible multimodal control.

Figures & tables

Figure 1: HairFlow uses conditioned diffusion to recover 3D strands from image inputs, guided by 3D mesh and sketches inferred by foundation models. It handles complex hairstyles and stylized images while closely adhering to the input conditions.

Figure 2: Overview of HairFlow. HairFlow takes two geometric inputs—a coarse hair mesh and multi-view sketches—to specify global shape and local strand flow. These conditions can be obtained from images or text using foundation models, or provided by artists. Mesh and camera-aware sketch encoders produce tokens that condition a diffusion transformer through cross-attention. The transformer generates latents in the space of a pretrained Orientation VAE, which decodes them into an orientation field for explicit strand extraction.

Figure 3: Hair sketch synthesis pipeline. (a) Input strands and template head mesh. (b) Multi-view orientation maps; colors encode directed 2D tangents as shown in the circular legend. (c) Bidirectional tracing from sampled seeds; arrows show local directions, and the inset details tracing from the highlighted seed. (d) Sketch variations through stroke density, line width, and localized seed sampling.

Figure 4: Qualitative comparison of single-view 3D hair reconstruction. HairLRM and HairFlow share the same input mesh, supplemented by HairStep orientation maps or sketches from image generation models, respectively. Inputs appear to the left of the corresponding strands. The color wheel below HairLRM illustrates the direction-to-color mapping.

Metric	HairStep	DiffLocks	Im2Haircut	HairLRM	HairFlow (Ours)
IoU $\uparrow$	0.625	0.583	0.773	0.855	0.912
Ori-Err $\downarrow$	$56^{\circ}$	$43^{\circ}$	$35^{\circ}$	$29^{\circ}$	$22^{\circ}$

Table 1: Quantitative comparison. Silhouette IoU (

\uparrow

) is evaluated on 30 real images. Angular error ( ∘ ,

\downarrow

) is measured between 3D orientation fields derived from reconstructed and ground-truth strands on 326 validation hairstyles.

Figure 5: Effect of the sketch condition. From left: input image, mesh, mesh-only result, sketch, and mesh+sketch result. Results within each row share the same mesh and random seed. Colors encode 3D tangent directions.

Method	Training set	Validation set	Mesh Only	Mesh + 2D cues
HairLRM ( Shen et al., 2026 )	10.47	$12.70$	$36.81^{\circ}$	$29.52^{\circ}$
HairFlow (Ours)	$\mathbf{4.01}$	$\mathbf{6.25}$	$27.43^{\circ}$	$22.92^{\circ}$

Table 2: Additional quantitative comparisons with HairLRM. We report mean angular errors ( ∘ , lower is better) for VAE reconstruction on the training and test sets, and for strand generation conditioned on meshes alone or with additional 2D cues.

Figure 6: Flexible Sketch Guidance: (a) Sketches with different stroke widths and styles yield similar strand geometry. (b) Joint conditioning on sketches from two viewpoints produces a single hairstyle consistent with both views. (c) Two sets of artist-drawn 3D strokes on the same mesh are projected into multi-view sketches, guiding the generated strands to follow different flow patterns.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Qualitative comparisons with HairGPT ( Luo et al., 2026 ) and Hao et al. (2026) . Their input images and reconstruction results are taken from the respective papers, as code is unavailable.

Figure 8: Multi-view renderings of reconstructed strands. Rows 3 and 7 use anime and oil-painting inputs, respectively; row 6 uses multi-view image inputs. The results demonstrate reconstruction across diverse hairstyles, image styles, and input views.

Figure 9: Additional multi-view renderings of reconstructed strands, showing hairstyle geometry and strand flow from different viewpoints.

Figure 10: Additional multi-view results. In the first row, an artist-provided text prompt is used to generate a hair mesh, and sketches are extracted from its rendered images, forming our unified geometric interface for strand generation.

Conditional Diffusion Models

Conditional diffusion models generate or infer data by steering iterative denoising with observations, labels, text, physical constraints, or other inputs.

See all 277 papers

HairGPT: Strand-as-Language Autoregressive Modeling for Realistic 3D Hairstyle Synthesis
May 9, 2026Haimin Luo, Min Ouyang, Lan Xu +13D Asset Generation 3D Shape Generation
ShanghaiTech University, China · ShanghaiTech University and Deemos Technology Co., Ltd., China
HairCS: Reconstructing Strand-Based Hair from Hair Cards
Sep 15, 2026Zixuan Lu, Tongtong Wang, Yuefan Shen +43D Reconstruction
University of Utah · LIGHTSPEED · UCLA
HairLRM: Strand-based Hair Modeling via Large Reconstruction Models
Jun 13, 2026Yuefan Shen, Yican Dong, Xiufeng Huang +33D Reconstruction Generative Modeling
LIGHTSPEED, China · State Key Lab of CAD and CG, Zhejiang University, China · Hong Kong Baptist University, China +1

May 9, 2026J/K move · Enter open · S save

HairGPT: Strand-as-Language Autoregressive Modeling for Realistic 3D Hairstyle Synthesis

Haimin Luo, Min Ouyang, Lan Xu, Jingyi Yu

ShanghaiTech University, China · ShanghaiTech University and Deemos Technology Co., Ltd., China

3D Asset Generation 3D Shape Generation Autoregressive Generation Conditional Generative Modeling

Open paper AskPDF arXiv

Hair is a rich medium of visual and cultural expression, yet its digital modeling remains challenging due to the duality of fluidity and structure. Many existing generative approaches rely primarily on continuous diffusion fields, which entangle global topology with local texture and obscure the semantic and structural organization of hairstyles. To address this, we propose HairGPT, a strand-centric framework that treats strands as generative primitives and formulates realistic 3D hairstyle synthesis as a dual-decoupled autoregressive sequence modeling problem. Our method applies spatial decoupling across semantic scalp regions and structural decoupling along a hierarchical strand representation, progressing from global layout to fine-grained style. We further introduce a geometric tokenizer and region-aware semantic annotations to guide strand-level generation, enabling compositional editing, synthesis of rare and complex hairstyles, and adaptation to stylized domains. By aligning generative modeling with the workflow of digital grooming, HairGPT turns hair generation from opaque texture synthesis into a structured and semantically controllable authoring process, supporting robust semantic conditioning and high-fidelity results across realistic and stylized domains. Project Page: https://haiminluo.github.io/hairgpt/

Abstract

Conditional Diffusion Models

Explore similar work

HairGPT: Strand-as-Language Autoregressive Modeling for Realistic 3D Hairstyle Synthesis

HairCS: Reconstructing Strand-Based Hair from Hair Cards

HairLRM: Strand-based Hair Modeling via Large Reconstruction Models