DiT-Garment: Garment Dynamics with Diffusion Transformers

Antoine Dumoulin; Laurence Boissieux; Joao Regateiro; Pierre Hellier; Stefanie Wuhrer

cs.CVSep 16, 2026

DiT-Garment: Garment Dynamics with Diffusion Transformers

Authors: Antoine Dumoulin, Laurence Boissieux, Joao Regateiro, Pierre Hellier, Stefanie Wuhrer

Organizations: Inria Centre at the University Grenoble Alpes · InterDigital Inc. · Inria, University of Rennes, CNRS, IRISA-UMR 6074

Abstract

We present DiT-Garment to model dynamic 3D clothing over human body models in arbitrary motion. Unlike existing methods, DiT-Garment can animate garments with unseen designs and physical materials, while allowing for direct inference of deformations for any target pose. To achieve this, we leverage a 2D diffusion transformer architecture to learn 3D deformations in a 2D UV-space. As the result is non-deterministic, our generative model learns the distribution of possible outcomes. The template garment is represented as a 3D triangle mesh spatially aligned with a 3D human body model in a standardized pose. To work with different garment designs without the need of a common template or complex graph convolution operations, the diffusion transformer is conditioned on a 3D position map of the template, represented in UV-space, which allows to implicitly learn a deformation of the 3D space around the body in standard pose. Further conditioning on body motion and physical parameters allows to physically ground the model. We quantitatively and qualitatively evaluate DiT-Garment on both synthetic and real data. While only trained on synthetic simulations of automatically generated cloth designs, our method generalizes to captured and artist-made garment designs. Code and data are available for research purposes at https://dumoulina.github.io/dit-garment/.

Figures & tables

Figure 1: DiT-Garment animates arbitrary garment designs over any human body in any target pose. Given a body motion and physical parameters, DiT-Garment directly infers dynamic garment deformations for a design seen during training (left) and generalizes to three unseen designs (right), without retraining.

Graph representation
Method	Gar.	Motion	Phys.	Posing
TailorNet [ 38 ]	✗	✗	✗	✓
Santesteban et al. [44]	✗	✓	✗	✗
PBNS [ 4 ]	✗	✓	✓	✗
SNUG [ 43 ]	✗	✓	✗	✗
GAPS [ 8 ]	✗	✓	✗	✗

Table 1: Positioning of our method compared to existing learned-based approaches. Gar. indicates whether the method can generalize to garments unseen during training. Motion indicates whether the method models physical dynamics given body motion. Phys. allows for control of the physical material. Posing indicates whether the method can directly infer deformations for any target pose. Bold indicates compared methods in experiments.

Figure 2: DiT-Garment generates garment deformations for a given template representing a garment design conditioned on body shape $\boldsymbol{\beta}$ , motion $\boldsymbol{\theta}_{t-l:t}$ and cloth material $\boldsymbol{\gamma}$ (see Sec. 3.1 ). It builds upon a 2D diffusion transformer model (DiT) to learn how to deform a template in UV -space. 3D geometric features are parameterized by the UV parametrization of the template, and we concatenate the position map to the diffusion noise. At inference, our model deforms the garment by iteratively denoising the Gaussian noise to the displacement map.

Figure 3: Visual comparison of the simulation ground truth (a), DiT-Garment (b), Pyramid-Drape [ 20 ] (c), DPF [ 40 ] (d) and ContourCraft [ 13 ] (e). DiT-Garment generates realistic deformations for unseen garment designs, physical materials and motions.

	Shape Sim.			Phys. Validity
	$E_{v}^{\downarrow}$	$E_{CD}^{\downarrow}$	$E_{n}^{\downarrow}$	$E_{c}^{\downarrow}$	$E_{b}^{\downarrow}$	$E_{s}^{\downarrow}$	$E_{d}^{\downarrow}$	FPS
Method with access to ground truth correspondence and normals
Pyramid-Drape ∗	-	0.01	0.31	0.63	-	-	2.24	12.6
Methods without ground truth information
DiT-Garment	5.20	0.17	0.38	0.62	0.51	1.32	2.66	0.4
DPF	6.95	0.18	0.42	0.64	0.49	0.22	3.94	0.05

Table 2: Comparison on synthetic test set to Pyramid-Drape [ 20 ] , DPF [ 40 ] and ContourCraft [ 13 ] . Best scores among methods without ground truth information are in bold , and - means that the method does not produce the output.

Panel of Figure 4 — Figure 4: We use DiT-Garment to reproduce captured sequences from 4D-Dress [ 51 ] and 4DHumanOutfit [ 1 ] . For each sequence, we show the captured avatar (left) and the predicted garment on top of the fitted body (right). We use reconstructed garment templates for 4D-Dress and an artist designed template for 4DHumanOutfit.

Figure 5: Effect of our guidance strategy to reduce cloth-body intersections. Left: without guidance, the generated garment intersects with the body. Right: with guidance, the generated garment does not intersect with the body.

	Seen templates							Unseen templates
Patch	Shape Similarity			Physical Validity				Shape Similarity			Physical Validity
size	$E_{v}^{\downarrow}$	$E_{CD}^{\downarrow}$	$E_{n}^{\downarrow}$	$E_{c}^{\downarrow}$	$E_{b}^{\downarrow}$	$E_{s}^{\downarrow}$	$E_{d}^{\downarrow}$	$E_{v}^{\downarrow}$	$E_{CD}^{\downarrow}$	$E_{n}^{\downarrow}$	$E_{c}^{\downarrow}$	$E_{b}^{\downarrow}$	$E_{s}^{\downarrow}$	$E_{d}^{\downarrow}$	fps
8	3.07	0.10	0.28	0.65	0.48	0.15	1.44	5.99	0.25	0.44	0.64	0.51	1.17	2.89	11.9
2	2.87	0.09	0.28	0.65	0.48	0.12	1.32	5.67	0.18	0.39	0.63	0.50	0.64	3.38	6.1
1	3.09	0.09	0.28	0.64	0.48	0.12	1.31	5.38	0.20	0.38	0.62	0.50	0.62	2.99	2.6
full	3.02	0.09	0.28	0.64	0.48	0.13	1.33	5.20	0.17	0.38	0.62	0.51	1.32	2.66	0.4

Table 3: Comparison of different transformer input patch size without guidance on seen and unseen templates. Smaller patch size improves the performance on unseen templates at the cost of inference speed. Guidance and patch size of 1 are used in the full model.

Unseen	Shape Similarity			Physical Validity
factor	$E_{v}^{\downarrow}$	$E_{CD}^{\downarrow}$	$E_{n}^{\downarrow}$	$E_{c}^{\downarrow}$	$E_{b}^{\downarrow}$	$E_{s}^{\downarrow}$	$E_{d}^{\downarrow}$
garment design	4.30	0.11	0.36	1.55	0.53	1.47	2.30
body motion	3.40	0.03	0.22	0.68	0.40	0.10	0.74
body shape	2.07	0.03	0.22	1.44	0.43	0.11	1.01
phys. material	0.94	0.01	0.19	1.68	0.43	0.11	0.85

Table 4: Evaluation of our method on one unseen factor at a time. Each test set uses the same sampled sequences from our training set, but with one factor replaced by an unseen value (garment design, body motion, body shape, or physical material).

3D Mesh Deformation

See all 16 papers

Fashion-3DLR: A Controllable 3D Garment Generation Using Pairwise Fashion Elements for Intelligent Design
Jul 25, 2026Shenghao Yang, Hongtao Zhang, Yuhan Yi +63D Asset Generation Conditional Generative Modeling
Donghua University Shanghai, China · Shanghai Artificial Intelligence Laboratory Shanghai, China
Stitched Embeddings: A Unified Latent Space for 3D Garments and 2D Patterns
Jul 1, 2026Andrea Sanchietti, Riccardo Marin, Bharat Lal Bhatnagar +23D Reconstruction 3D Garment Reconstruction
University of Tübingen, Germany · Tübingen AI Center · Technical University of Munich, Germany +3
SwiftTailor: Efficient 3D Garment Generation with Geometry Image Representation
Mar 19, 2026Phuc Pham, Uy Dieu Tran, Binh-Son Hua +13D Mesh Generation 3D Asset Generation
Qualcomm AI Research · Trinity College Dublin, Ireland.

Jul 25, 2026J/K move · Enter open · S save

Fashion-3DLR: A Controllable 3D Garment Generation Using Pairwise Fashion Elements for Intelligent Design

Shenghao Yang, Hongtao Zhang, Yuhan Yi, Zhihao Tang+5

Donghua University Shanghai, China · Shanghai Artificial Intelligence Laboratory Shanghai, China

3D Asset Generation Conditional Generative Modeling

Open paper AskPDF arXiv

AI-generated content (AIGC) has made significant progress, with 2D generative models becoming ready-to-use tools for the digital fashion industry. However, 3D garment generation remains in its nascent stage, where in the realm of fashion, the semantic information of diverse design elements exhibits intricate coupling relationships in 3D representations, posing substantial challenges for generating diverse 3D garments. In this work, to handle the above problem, We introduce Fashion-3DLR, a novel 3D garment generation framework that utilizes diverse design elements to create high-quality, versatile 3D garment assets. Specifically, to bridge the semantic gaps between different fashion elements, we propose a Garment Feature Fusion Diffusion Transformer (GFF-DiT) module to integrate 2D fashion design elements, e.g., sketch and texture, into latent space. Within the latent space, we then employ a rectified flow transformer to generate geometry latents, which can be decoded into various 3D garment representations, including 3D Gaussians and meshes. Furthermore, we integrate Fashion-3DLR into downstream tasks, achieving the 3D Gaussian Splatting (3DGS)-driven cloth physical simulation and mesh-based virtual try-on. Experimental results indicate that Fashion-3DLR surpass the previous state-of-the-art methods, which verify that the proposed work can generate well-structured, non-watertight garments capable of physical simulation and virtual try-on, underscoring its potential as a versatile 3D garment design tool.

Abstract

3D Mesh Deformation

Explore similar work

Fashion-3DLR: A Controllable 3D Garment Generation Using Pairwise Fashion Elements for Intelligent Design

Stitched Embeddings: A Unified Latent Space for 3D Garments and 2D Patterns

SwiftTailor: Efficient 3D Garment Generation with Geometry Image Representation