We study the problem of inverse garment design, reconstructing simulation-ready 3D outfits with arbitrary layers from a single image, with visual agents. Unlike differentiable optimization that is time-consuming, or feed-forward methods limited by training data, we introduce DressAgent, a hierarchical agentic system that recovers garment sewing patterns through planning and closed-loop refinement grounded in physical simulation. To this end, we first present an enhanced representation based on GarmentCode for diverse styles and arbitrary dressing layers, and build the first multi-layer sewing pattern dataset with over 15K outfits for supervision and benchmarking. Inverse design then proceeds coarse to fine: In layer planning and coarse estimation, agents infer layer order and relations, peel each layer into its own view, and estimate coarse patterns. In fine-grained refinement, an agentic trio of Tailor, Critic, and Orchestrator refines patterns in a closed loop of editing, simulating, and judging against the reference with visual and measured evidence. Carefully designed tools and an evolving memory make this refinement converge on the target geometry. We also design an agentic generation method for high-fidelity textures. DressAgent outperforms feed-forward methods by a large margin, surpasses or matches differentiable optimization in a fraction of the time, and generalizes to complex layering, diverse poses, and body shapes.
Figures & tables
Figure 1: We introduce DressAgent for inverse garment design with agentic planning and refinement. (a) Results on in-the-wild photos with arbitrary layering, diverse poses, and body shapes. (b) The outfits drape and animate under physics. (c) Every garment is a sewing pattern with high-fidelity texture. (d) DressAgent pushes the Pareto frontier of quality and speed.
Figure 2: Enhanced Garment Representation and Dataset. Top : Examples of enhanced expressivity and layered representation in GarmentCode-XL. Bottom : Examples from GarmentCodeData-XL. Its 15,205 outfits contain 4,640 one-layer, 9,282 two-layer, 721 three-layer, and 562 four-layer examples. From left to right, colors indicate one to four layers. For more details, see Appendix B .
Figure 3: Layer Planning and Coarse Pattern Estimation. The Layer Planner reasons ordered layers and peels each into view. The Visual Reader generates a Garment Style Template per layer, then an encoder-decoder with distilled GarmentCode-XL semantics maps it to a coarse program.
Figure 4: Fine-Grained Refinement with an Agentic Trio. Top : illustration of one refinement round (from Iter 1 to Iter 2 of the inner layer). The Critic scores each garment and identifies discrepancies, the Orchestrator saves a checkpoint and instructs the Tailor, and the Tailor applies the requested edits and simulates the pattern. Bottom : examples of rollout trajectories. For agent implementation, detailed prompts, and the complete record of this example, see Appendix D .
Figure 5: Texture Generation and Simulation. We design an agentic generative approach to recover textures for the sewing patterns, and use an XPBD-based cloth solver for dynamic simulation.
Figure 6: Qualitative comparison on in-the-wild images. Top five : reconstructing outfits with one to three layers (left to right). Bottom five : generalization to diverse poses and body shapes. ChatGarment, ISP, and GarmentRecovery lack texture generation, so we show only their geometry.
Structure Acc. ↑
Geometry (CloSe)
Geometry (4D-Dress)
Method
Panel
Edge
Stitch
CD ↓
IoU ↑
CD ↓
IoU ↑
Avg. Time ↓ (min)
SewFormer
18.67
28.80
7.24
7.40
0.348
7.58
0.496
2.0
ChatGarment
51.84
57.15
70.18
2.39
0.580
1.94
0.763
4.2
GarmentGPT
43.62
38.54
65.43
5.50
0.381
10.46
0.204
0.4
DressCode
19.67
26.47
30.97
2.77
0.542
2.56
0.685
2.2
ISP
-
-
-
1.79
0.687
1.16
0.859
90.0
Table 1: Quantitative comparison. We report the structure accuracy of sewing patterns and the geometry accuracy of draped garments. ISP and GarmentRecovery are excluded from structure accuracy, as they do not produce explicit sewing patterns with panel geometry. Our method achieves the best quality on most metrics, and is significantly faster than differentiable optimization.
Figure 8Figure 9
GarmentCodeData-XL
DeepFashion2
Comparison
Struct. Acc. ↑
IoU ↑
IoU ↑
w/o Refinement
63.25
0.623
0.562
w/o GarmentCode-XL
57.99
0.574
0.650
w/o Critic
69.56
0.822
0.765
w/o Measure
67.21
0.804
0.772
Full
73.59
0.839
0.783
Table 3: Ablation on representation and the agentic trio. DeepFashion2 IoU is measured against the annotated clothing masks.
Appendix figures & tables25 assets
Supplementary material from the paper’s appendix.
Appendix
Symbol
Meaning
Garment Representation
S=P(B)
Sewing pattern of one garment: its flat panels and the stitches that join them, drafted by P on body B .
P=⟨F,D⟩
Garment program: panel constructors with their design parameters.
F={fi}
Panel constructors, one per component such as a sleeve or a bodice.
fi
The i -th constructor, drafting a component as parametric 2D curves controlled by di and B .
D={di}
Design parameters, 131 in GarmentCode-XL.
Appendix
Table 4: Notations. Summary of the main symbols and notations used in the paper.
Figure 10: Examples of Construction details of GarmentCode-XL. New panel constructors and outfit fields are marked + , and expanded ranges of existing parameters are marked ∼ . The first seven widen the design space to real-world styles, and the last adds the layered outfit structure with dressing order and relations.
Figure 11: Example of a two-layer GarmentCode-XL outfit. The tee is tucked into the skirt within the inner layer L1 . The cardigan forms the outer layer L2 , with L1 worn under L2 .
Parameter type
Count
Options
Answer format
Length (float)
83
308
3 to 6 ordered visual states
Count (int)
15
55
ordered states, or one option per admissible count
Boolean
14
28
the two geometric states of the flag
Categorical
11
56
one option per legal value
Categorical with absence
8
34
one option per legal value, plus absence
Total
131
481
Appendix
Table 5: The frozen answer space. Here, every design parameter of GarmentCode-XL becomes exactly one multiple choice question.
Symmetric
Asymmetric
Garment type
Questions
Numeric
Questions
Numeric
Top
48
34
79
56
Skirt
42
33
42
33
Pants
15
12
15
12
Dress
90
67
121
89
Jumpsuit
63
46
94
68
Appendix
Table 6: Layer-specific questionnaire size. Number of questions retained from Q for each garment type and symmetry state.
Figure 12: Architecture of the decoder Ψ . The frozen encoder E embeds each question with its selected answer. A projection MLP maps the embeddings to the decoder width, and a causal Transformer with 12 blocks decodes the numeric program parameters.
Tool
Access
Arguments
Description
Observation
Render
T
pictures
Views of any step: the photograph, the target and drape pair of either view, the before/after strip, per-garment mask diffs, and the flat sewing pattern.
Measure
T
step
Measurement tables of one simulated step: every garment edge on the drape against the target, and the same sizes read off the drafted pattern, in percent of body height.
Modification
Draft
T
edits
Drafts the current program with the edits and returns the garment sizes of the flat pattern, with the change against the current step.
Fit
T
part , channel , lever , target
Value of one numeric lever that lands one measured size on its target, found through drafting probes (Appendix D.4 ). Costs no simulation.
Appendix
Table 7: The complete tool interface of Stage 2. Access: T = Tailor, C = Critic, O = Orchestrator.
Figure 13: The observation Ot given to the agents in the recorded round. Left : the paired front view of step 1 (Iter 1 of the inner layer in Fig. 4 ), which the Critic and the Tailor read. The skirt has been corrected, and the tee is still wide, deep at the neck, and long in the sleeve. Right : the same observation at step 2, after the five fitted edits below. Both panels carry the same landmark lines and body ruler in percent of body height.
Figure 14: Examples of the five texture classes. From left to right: a solid fabric, a knit without an exact repeat, placed artwork over a base fabric, an exact repeating lattice, and a non-repeating print, each handled by the tool named below it.
Canonical A-pose
Target Pose
Method
ΔE↓
CLIP-I ↑
DINO ↑
PSNR ↑
SSIM ↑
LPIPS ↓
ΔE↓
CLIP-I ↑
DINO ↑
SewFormer
20.61
0.661
0.252
8.39
0.543
0.433
14.05
0.681
0.324
DressCode
9.67
0.633
0.305
8.68
0.565
0.407
12.93
0.652
0.275
Dress 1-to-3
-
-
-
8.26
0.531
0.410
21.26
0.650
0.229
Ours
9.57
0.821
0.658
9.94
0.615
0.352
8.43
0.800
0.597
Appendix
Table 8: Quantitative comparison on texture generation. We evaluate the texture fidelity of each method against the input photo on different metrics. Dress 1-to-3 reconstructs the garment directly in the target pose, so we do not report its results in the canonical A-pose.
GarmentCodeData-XL (max 4 layers)
DeepFashion2 (max 3 layers)
Method
Count Acc. ↑
Struct. Acc. ∣ Count ↑
Count Acc. ↑
Struct. Acc. ∣ Count ↑
DressCode
34.0%
76.5%
31.7%
52.6%
ISP
50.0%
87.0%
65.0%
84.6%
GarmentRecovery
49.5%
85.9%
61.7%
83.8%
Ours
95.5%
100.0%
96.7%
98.3%
Appendix
Table 9: Quantitative comparison on layer structure. We report the accuracy of the layer count, and the accuracy of the layer structure conditional on a correct count, on GarmentCodeData-XL and DeepFashion2. Only the methods that can model more than one layer are included.
CloSe
4D-Dress
Comparison
CLIP-I ↑
DINO ↑
ΔE↓
CLIP-I ↑
DINO ↑
ΔE↓
Other outfits vs. photo
0.869
0.678
7.55
0.705
0.391
23.29
Second photo vs. photo
-
-
-
0.917
0.776
3.46
Canonical view vs. photo
0.907
0.845
5.49
0.806
0.729
4.31
Appendix
Table 10: Faithfulness of the canonical view. We compare garment crops with the input photograph. Second photo is another frame of the same 4D-Dress capture, a real view of the same outfit. Other outfits compares the canonical view with the photographs of all other outfits in the dataset.
Figure 15: From one T-shirt to any garment. We show four examples of refinement runs that start from the same plain T-shirt (left). The trio first intelligently modifies the construction, then refines the sizes to match the photo.
Figure 16: Refinement from a plain T-shirt. Silhouette IoU on CloSe and 4D-Dress over the refinement iterations, starting from the Stage-1 program (solid) or from a plain T-shirt (dashed). Bands show the 95% bootstrap confidence interval.
Geometry (CloSe)
Geometry (4D-Dress)
Starting point
CD ↓
IoU ↑
CD ↓
IoU ↑
From Plain T-shirt
1.75
0.699
1.31
0.871
From Stage-1 Program (Ours)
1.20
0.764
1.21
0.880
Appendix
Table 11: Refinement from a plain T-shirt. Comparison of the geometry of the reposed garment against the scan, with the protocol of Table 1 .
CloSe
4D-Dress
Setting
CD ↓
IoU ↑
CD ↓
IoU ↑
Input only
1.36
0.743
2.28
0.782
Front only
1.29
0.750
1.73
0.833
No peeling
-
-
1.64
0.846
Full
1.20
0.764
1.21
0.880
Appendix
Table 12: Ablation on synthesized views. We report the geometry of the reposed garment against the scan, graded as in Table 1 . Since CloSe has few multi-layer outfits, while 4D-Dress captures outfits with an outer garment and labels it as its own class, we test no peeling only on 4D-Dress.
GarmentCodeData-XL
DeepFashion2
Comparison
Struct. Acc. ↑
IoU ↑
Self-Intersection (‰) ↓
w/o Rollback
58.70
0.736
0.16
Full
73.59
0.839
0.04
Appendix
Table 13: Ablation on Rollback. We ablate the Rollback tool of the agentic trio. Self-intersection is the share of cloth edges that intersect the garment itself on the delivered drape.
Figure 17: Ablation on single-agent refinement. Per-garment silhouette IoU and measurement error of the delivered garment after each simulation on CloSe and 4D-Dress. Bands show the 95% bootstrap confidence interval. The single agent oscillates and plateaus early, while our agentic trio converges progressively.
Method
IoU ↑
Meas. Err. ↓
Edits / Iter.
Tokens (k) ↓
Time (s) ↓
Single agent
0.820
1.80%
6.2
38.7
748
Single agent + self-selection
0.833
1.69%
6.2
38.7
748
Agentic trio (ours)
0.846
1.65%
2.8
15.0
412
Appendix
Table 14: Ablation on single-agent refinement. All methods use the same coarse pattern, backbone, tools, memory, and 10 simulations per layer. Measurement error is in percent of body height. Tokens and time are the output tokens and VLM time per case. We find that the single agent is worse in quality, biased in self-evaluation, and higher in cost.
Figure 18: Ablation on texture generation. (a) Naive UV mapping without Grounded SAM mixes garments: the stripes of the tee bleed onto the pants. (b) With Grounded SAM, the garments separate, but unobserved cloth admits no extrapolation and the observed print lands warped. (c) Our agentic generation method continues the stripes at their measured repeat, restore the logo, and keep the pants a uniform black.
Figure 19: Screenshot of the user study question on geometry.
Figure 20: Screenshot of the user study question on plausibility.
Figure 21: Screenshot of the user study question on texture.
Figure 22: Failure case. The sash hanging free at the waist cannot be recovered by the current representation. Future work may let the agent write code to extend the representation on its own.
Figure 23: Additional results on in-the-wild photos with different layering, poses, and body shapes.
Recovering sewing patterns from draped 3D garments is a challenging problem in human digitization research. In contrast to the well-studied forward process of draping designed sewing patterns using mature physical simulation engines, the inverse process of recovering parametric 2D patterns from deformed garment geometry remains fundamentally ill-posed for existing methods. We propose a two-stage framework that centers on a structured intermediate representation, BoxMesh, which serves as the key to bridging the gap between 3D garment geometry and parametric sewing patterns. BoxMesh encodes both garment-level geometry and panel-level structure in 3D, while explicitly disentangling intrinsic panel geometry and stitching topology from draping-induced deformations. This representation imposes a physically grounded structure on the problem, significantly reducing ambiguity. In Stage I, a geometry-driven autoregressive model infers BoxMesh from the input 3D garment. In Stage II, a semantics-aware autoregressive model parses BoxMesh into parametric sewing patterns. We adopt autoregressive modeling to naturally handle the variable-length and structured nature of panel configurations and stitching relationships. This decomposition separates geometric inversion from structured pattern inference, leading to more accurate and robust recovery. Extensive experiments demonstrate that our method achieves state-of-the-art performance on the GarmentCodeData benchmark and generalizes effectively to real-world scans and single-view images.
Leyang Jin, Zirong Jin, Zisheng Ye +4
Mohamed bin Zayed University of Artificial Intelligence · The Chinese University of Hong Kong, Shenzhen · School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen, FNii-Shenzhen, and Guangdong Provincial Key Laboratory of Future Networks of Intelligence +1
Recent advances in garment pattern generation have shown promising progress. However, existing feed-forward methods struggle with diverse poses and viewpoints, while optimization-based approaches are computationally expensive and difficult to scale. This paper focuses on sewing pattern generation for garment modeling and fabrication applications that demand editable, separable, and simulation-ready garments. We propose DressWild, a novel feed-forward pipeline that reconstructs physics-consistent 2D sewing patterns and the corresponding 3D garments from a single in-the-wild image. Given an input image, our method leverages vision-language models (VLMs) to normalize pose variations at the image level, then extract pose-aware, 3D-informed garment features. These features are fused through a transformer-based encoder and subsequently used to predict sewing pattern parameters, which can be directly applied to physical simulation, texture synthesis, and multi-layer virtual try-on. Extensive experiments demonstrate that our approach robustly recovers diverse sewing patterns and the corresponding 3D garments from in-the-wild images without requiring multi-view inputs or iterative optimization, offering an efficient and scalable solution for realistic garment simulation and animation.
While garments are essential for realistic digital humans, their topological variety makes them much harder to model than parametric bodies. Traditional tailoring relies on 2D sewing patterns, yet bridging these patterns to 3D geometry currently requires physical simulations. We present Stitched Embeddings, the first simulation-free framework to unify 3D garment reconstruction and sewing pattern inference within a single bidirectional latent space. By leveraging the geometric priors of a pretrained 3D foundation model, our approach overcomes the data scarcity typically associated with high-quality garment modeling. We propose to use the BoxMesh as a critical intermediate representation to align 2D panels into 3D configurations without the computational overhead of a simulator. This architecture achieves state-of-the-art accuracy in pattern reconstruction while significantly improving efficiency. Furthermore, our differentiable pipeline enables novel applications, including pattern recovery from meshes and 3D editing from 2D patterns. Finally, this work provides a scalable link between neural 3D vision and the physical garment manufacturing pipeline. Project Page: https://andreus00.github.io/stitchedembeddings
Andrea Sanchietti, Riccardo Marin, Bharat Lal Bhatnagar +2
University of Tübingen, Germany · Tübingen AI Center · Technical University of Munich, Germany +3