GarmentPainter: Efficient 3D Garment Texture Synthesis with Character-Guided Diffusion Model

Jinbo Wu; Xiaobo Gao; Xing Liu; Chen Zhao; Jialun Liu

cs.CVMar 9, 2026

GarmentPainter: Efficient 3D Garment Texture Synthesis with Character-Guided Diffusion Model

Authors: Jinbo Wu, Xiaobo Gao, Xing Liu, Chen Zhao, Jialun Liu

Organizations: Baidu Inc. · TeleAI

Abstract

Generating high-fidelity, 3D-consistent garment textures remains a challenging problem due to the inherent complexities of garment structures and the stringent requirement for detailed, globally consistent texture synthesis. Existing approaches either rely on 2D-based diffusion models, which inherently struggle with 3D consistency, require expensive multi-step optimization or depend on strict spatial alignment between 2D reference images and 3D meshes, which limits their flexibility and scalability. In this work, we introduce GarmentPainter, a simple yet efficient framework for synthesizing high-quality, 3D-aware garment textures in UV space. Our method leverages a UV position map as the 3D structural guidance, ensuring texture consistency across the garment surface during texture generation. To enhance control and adaptability, we introduce a type selection module, enabling fine-grained texture generation for specific garment components based on a character reference image, without requiring alignment between the reference image and the 3D mesh. GarmentPainter efficiently integrates all guidance signals into the input of a diffusion model in a spatially aligned manner, without modifying the underlying UNet architecture. Extensive experiments demonstrate that GarmentPainter achieves state-of-the-art performance in terms of visual fidelity, 3D consistency, and computational efficiency, outperforming existing methods in both qualitative and quantitative evaluations.

Figures & tables

Figure 1: The results of our method. Given garment type, we showcase three types of garment texture generation: top, bottom and one-piece . Each image consists of two sections: The left side displays the reference image alongside the UV position map, which is obtained through the rasterization of the corresponding mesh. The right side presents the rendered output of the generated texture, including both the front view and back view. Our method produces high-quality and 3D consistent textures across a wide range of garment styles.

Figure 2: We show three types of our data, i.e. , one-piece, top and bottom . Specifically, we show reference image, UV texture map, UV position map and UV mask image.

Figure 3: The GarmentPainter framework starts by encoding the reference image and UV position map with a VAE Kingma (2013) . At the same time, the masked UV texture map is encoded to preserve background regions that should not be regenerated. The reference latent captures style information, while the UV position latent captures structure. These latents, along with the UV texture map and reference image, are transformed into texture latents, then noised to form the noisy latent. The UV mask image is downsampled by 8× to restrict generation to the intended area. All latent features are concatenated and passed into the diffusion model. A type encoder also embeds garment type labels, which are added to the time embedding to provide fine-grained control over garment regions.Note: Each transformer block omits cross-attention for text interaction.

Figure 4: Attention maps for $z^{2}\rightarrow z^{1}$ and $z^{1}\rightarrow z^{2}$ demonstrate accurate and effective information exchange between the two diffusion noise predictors ${\epsilon_{\theta}^{1},\epsilon_{\theta}^{2}}$

Figure 5: Qualitative comparison with SOTA methods. Our approach produces garment textures that closely match the reference images, even when characters are present. Compared to other methods, the results are more detailed and maintain better 3D consistency. Please zoom in to view details.

Methods	FID $\downarrow$	KID( $\times 10^{-3}$ ) $\downarrow$	Runtime $\downarrow$
TEXTure Richardson et al. (2023)	75.33	17.79	$\sim$ 140s
Text2Tex Chen et al. (2023a)	77.28	13.39	$\sim$ 940s
Paint3D Zeng et al. (2024)	71.32	7.35	$\sim$ 220s
TEXGen Yu et al. (2024)	46.90	3.37	$\sim$ 10s
TEXGen ${\dagger}$ Yu et al. (2024)	44.82	2.96	$\sim$ 10s
Ours	39.79	1.01	$\sim$ 4s

Table 1: Quantitative comparison against state-of-the-art methods.

{\dagger}

denotes that TexGen is trained on our dataset. We report the FID, KID and runtime. Our method achieves the best performance.

Figure 6: Comparison with Paint3D Zeng et al. (2024) using a character image as reference.

Methods	FID $\downarrow$	KID $\downarrow$
w/o type select module	45.08	1.30
w/o uv position map	47.82	1.45
Ours	39.79	1.01

Table 2: Evaluation of modules in our method. This demonstrates the effectiveness of each component. Our full model achieves optimal performance.

Figure 7: The visual comparison with and without type selection module.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 8: Additional results of our method. We showcase a variety of reference images, including cartoon characters, real people, and scenes with complex backgrounds. Across these diverse inputs, our method consistently generates high-quality texture maps that accurately reflect the garment depicted in each reference image.

Figure 9: Evaluation on GarmentCodeData Korosteleva et al. (2024) demonstrates the strong generalization ability of our method across diverse garment meshes.

Figure 10: Evaluation under challenging setting: the reference image and UV map are mismatched.

Methods	FID $\downarrow$	KID( $\times 10^{-3}$ ) $\downarrow$
w/o character	38.54	0.98
w/ character	39.79	1.01

Appendix

Table 3: Comparison of our method using reference images with and without characters, showing robust performance and visually consistent textures in both cases.

Figure 11: Dynamic results. We present a dynamic example that demonstrates the effectiveness and robustness of our method.

Texture Synthesis

Texture synthesis creates and edits realistic visual patterns and surface materials, from image patches to coherent, view-consistent 3D assets.

See all 34 papers

OmniFabric: Coherent UV Space Texture Synthesis for 3D Garment Reconstruction
Sep 24, 2026Ding-Jiun Huang, Yuanhao Wang, Cheng Zhang +43D Asset Generation 3D Reconstruction
Carnegie Mellon University, United States of America · University of Washington, United States of America · Texas A&M University, United States of America +2
Fashion-3DLR: A Controllable 3D Garment Generation Using Pairwise Fashion Elements for Intelligent Design
Jul 25, 2026Shenghao Yang, Hongtao Zhang, Yuhan Yi +63D Asset Generation Conditional Generative Modeling
Donghua University Shanghai, China · Shanghai Artificial Intelligence Laboratory Shanghai, China
SwiftTailor: Efficient 3D Garment Generation with Geometry Image Representation
Mar 19, 2026Phuc Pham, Uy Dieu Tran, Binh-Son Hua +13D Mesh Generation 3D Asset Generation
Qualcomm AI Research · Trinity College Dublin, Ireland.

Sep 24, 2026J/K move · Enter open · S save

OmniFabric: Coherent UV Space Texture Synthesis for 3D Garment Reconstruction

Ding-Jiun Huang, Yuanhao Wang, Cheng Zhang, Hugo Bertiche+3

Carnegie Mellon University, United States of America · University of Washington, United States of America · Texas A&M University, United States of America+2

3D Asset Generation 3D Reconstruction 3D Garment Reconstruction Texture Synthesis

Open paper AskPDF arXiv

Automated generation of production-ready 3D garment assets from a single image is a central challenge in digital content creation. While recent generative models have significantly advanced 3D geometry reconstruction, synthesizing high-quality textures remains a bottleneck. Existing methods often bake environmental illumination and shadows directly into the texture map, or they fail to maintain global structural coherence, making the resulting assets unusable for physical simulation and relighting. In this work, we introduce OmniFabric, a novel approach that synthesizes globally coherent texture maps directly within the 2D sewing pattern space. Given a single reference image, our pipeline utilizes an estimated 3D mesh and generative priors of powerful Vision-Language Models (VLM) to establish a complete but coarse texture initialization across the unwrapped sewing patterns. We then leverage a specialized diffusion transformer, trained via an automated synthetic data engine and conditioned on 3D positional features, to refine this initialization directly in the canonical UV domain. This effectively removes distortion and baked-in artifacts to extract a clean and normalized texture map that preserves the original garment design. Extensive experiments demonstrate that OmniFabric significantly outperforms state-of-the-art baselines, yielding photorealistic 3D garments with high-quality textures.

Abstract

Texture Synthesis

Explore similar work

OmniFabric: Coherent UV Space Texture Synthesis for 3D Garment Reconstruction

Fashion-3DLR: A Controllable 3D Garment Generation Using Pairwise Fashion Elements for Intelligent Design

SwiftTailor: Efficient 3D Garment Generation with Geometry Image Representation