Authors: Yingjie Tang, Di Luo, Zixiong Wang, Xiaoli Ling, jian Yang, Beibei Wang
Organizations: College of Computer Science, Nankai University, Tianjin, China · School of Intelligence Science and Technology, Nanjing University, Suzhou, China
Woven fabric materials are widely used in rendering applications, yet designing realistic examples typically involves multiple stages, requiring expertise in weaving principles and texture authoring. Recent advances have explored diffusion models to streamline this process; however, pre-trained diffusion models often struggle to generate intricate yarn-level details that conform to weaving rules. To address this, we present FabricGen, an end-to-end framework for generating high-quality woven fabric materials from textual descriptions. A key insight of our method is the decomposition of macro-scale textures and micro-scale weaving patterns. To generate macro-scale textures free from microstructures, we fine-tune pre-trained diffusion models on a collected dataset of microstructure-free fabrics. As for micro-scale weaving patterns, we develop an enhanced procedural geometric model capable of synthesizing natural yarn-level geometry with yarn sliding and flyaway fibers. The procedural model is driven by a specialized large language model, WeavingLLM, which is fine-tuned on an annotated dataset of formatted weaving drafts, and prompt-tuned with domain-specific fabric expertise. Through fine-tuning and prompt tuning, WeavingLLM learns to design weaving drafts and fabric parameters from textual prompts, enabling the procedural model to produce diverse weaving patterns that stick to weaving principles. The generated macro-scale texture, along with the micro-scale geometry, can be used for fabric rendering. Consequently, our framework produces materials with significantly richer detail and realism compared to prior generative models.
Figures & tables
Figure 2: Overview of FabricGen. Our fabric material creation framework consists of two main components: (a) an LLM-driven procedural microstructure generator that generates yarn-level microstructures along with fabric parameters, and (b) a texture generator that generates tileable, macro-scale textures from a text prompt (and optional image). Note that the two components require distinct prompts.
Figure 3: Woven fabrics are typically constructed by interlacing weft and warp yarns following specific patterns (b), which can be represented via a binary matrix, named weaving draft (a). Each yarn may consist of multiple plies, and each ply can be further divided into a number of fibers. Some fibers escape from the surface, resulting in irregular flyaway fibers (c).
Figure 4: Curved helix model for multi-ply yarns. The model achieves analytical normal/orientation formulation, where the yarn space arc parameters u,v are linear mapped from UV space.
Figure 5: Visualization of the normal, orientation and height maps of a 3-ply basket weave. The parameters of this case are provided in the supplementary material.
Figure 6: Comparison on text conditioning. Given text prompt, DressCode [ 4 ] and MatFuse [ 22 ] generate PBR textures, which struggle to capture fine fabric details, resulting in blurring and distortion in close-up views. In contrast, our method generates procedural microstructure together with macroscale texture, ensuring a realistic appearance in both macro and close-up views.
MatFuse
DressCode
Ours
Text conditioning
CLIP Score ↑
0.240
0.307
0.317
User study ↑
1.43%
16.02%
82.55%
Text and image conditioning
CLIP-I Score ↑
0.722
N/A
0.827
Table 1: Quantitative comparisons between our method, MatFuse [ 22 ] , and DressCode [ 4 ] .
Figure 7: Comparison on text and image conditioning . Given both text and image prompts, our generated materials achieving better consistency and richer detail than MatFuse [ 17 ] .
Figure 8: Comparison among WeavingLLM, Qwen2.5 [ 20 ] , and GPT-5 [ 13 ] . COSSIM indicates the cosine similarity of Fourier spectra, and PERF indicates the user preferences ratio. All renderings use a pure gray albedo map. More reference drafts are provided in the supplementary material.
Figure 9: Impact of prompt-tuning . The parameters predicted by WeavingLLM successfully reproduce the rough appearance of herringbone twill and the anisotropic sheen of satin, whereas parameters predicted by the base LLM fail to reproduce this effect.
Figure 10: Impact of macro-scale texture generator . The diffusion model pre-trained on natural images occasionally produces unintended 3D artifacts (first row) or implicit stripes (third row), whereas our fine-tuned model is capable of generating microstructure-free albedo maps. The results in the fourth column use the regular twill pattern in Fig 2 .
Figure 11: Impact of irregular effects . Irregular yarn displacements in real fabrics often result in strip-like gaps between yarns, and fibers escaping from the surface cause irregular highlights.
Automated generation of production-ready 3D garment assets from a single image is a central challenge in digital content creation. While recent generative models have significantly advanced 3D geometry reconstruction, synthesizing high-quality textures remains a bottleneck. Existing methods often bake environmental illumination and shadows directly into the texture map, or they fail to maintain global structural coherence, making the resulting assets unusable for physical simulation and relighting. In this work, we introduce OmniFabric, a novel approach that synthesizes globally coherent texture maps directly within the 2D sewing pattern space. Given a single reference image, our pipeline utilizes an estimated 3D mesh and generative priors of powerful Vision-Language Models (VLM) to establish a complete but coarse texture initialization across the unwrapped sewing patterns. We then leverage a specialized diffusion transformer, trained via an automated synthetic data engine and conditioned on 3D positional features, to refine this initialization directly in the canonical UV domain. This effectively removes distortion and baked-in artifacts to extract a clean and normalized texture map that preserves the original garment design. Extensive experiments demonstrate that OmniFabric significantly outperforms state-of-the-art baselines, yielding photorealistic 3D garments with high-quality textures.
Ding-Jiun Huang, Yuanhao Wang, Cheng Zhang +4
Carnegie Mellon University, United States of America · University of Washington, United States of America · Texas A&M University, United States of America +2
Multimodal Sewing pattern generation aims to infer executable sewing patterns from design cues such as sketches and textual descriptions. As an interpretable and simulation-compatible representation, sewing patterns are particularly valuable for digital garment creation. However, existing methods often model garment specifications as flat long sequences, which entangles garment structure with detailed parameters and leads to redundant components, inaccurate local details, and poor simulation compatibility. In this paper, we present GarmentWeaver, a schema-aware framework for multimodal Sewing pattern generation. GarmentWeaver constructs compact hierarchical targets by activating garment-relevant structural branches and predicts executable Sewing patterns in a structured manner. Specifically, we introduce a schema-aware target construction strategy, build the generator on top of a pretrained vision-language model for multimodal garment understanding, and impose feasibility-aware regularization to encourage structurally valid and simulation-compatible outputs. Extensive experiments show that GarmentWeaver produces more accurate and more executable sewing patterns than strong baselines, while also yielding better simulation results. These findings demonstrate the effectiveness of schema-aware structured generation for reliable multimodal Sewing pattern prediction.
Yinwen Lu, Weihao Luo, Yueqi Zhong
Donghua University · College of Textiles · Shanghai, China +3
Standard clothing asset generation---restoring forward-facing flat-lay garment images from diverse real-world contexts---holds immense commercial value yet demands both macroscopic topological accuracy and microscopic physical fidelity. Although our previous work RAGDiffusion effectively eradicated large-scale structural hallucinations via retrieval-augmented macro-constraints, achieving industrial-grade micro-texture realism remains an unsolved bottleneck. We formally identify this limitation as High-Frequency Trajectory Collapse: supervised fine-tuning (SFT) converges to the conditional mean of the training distribution, which is dominated by smooth, low-frequency textures, causing high-frequency patterns (e.g., fabric weaves, intricate logos) to become nearly un-sampleable. Naively applying Reinforcement Learning (RL) post-training further triggers Artifact Hacking, where models exploit semantic biases in generic reward models by generating deceptive checkerboard noise. Our key insight is that RL can fundamentally reshape the sampling distribution of flow models---elevating the probability of high-fidelity trajectories under accurate reward guidance---while adversarial regularization prevents exploitation of reward blind spots. Realizing this principle requires three prerequisites: (i)inherent capacity, established through a 27,725-pair high-complexity garment dataset (STGarment-Plus) and a Dual-Image-Stream FLUX architecture upgrade; (ii)perceptive reward, provided by a novel attribute-aware reward model (Garment-RM) trained on 500K images via fine-grained contrastive learning, achieving 84.67% human preference accuracy; and (iii)hacking prevention, enforced by our Adversarial-Regularized GRPO (AR-GRPO) strategy that integrates a dynamic discriminator into the RL sampling trajectory to penalize artifacts while enriching authentic high-frequency details.