MagicGarment: Benchmarking Single-Image Standalone 3D Garment Reconstruction
Authors: Jia Chen, Ruyang Zhao, Jun Bai, Chao Huang, Saishang Zhong, Xinrong Hu
Organizations: School of Computer Science and Artificial Intelligence, Wuhan Textile University, Wuhan 430200, China · Engineering Research Center of Hubei Province for Clothing Information, Wuhan Textile University, Wuhan 430200, China · School of Computer Science, McGill University, Montreal, QC H3A 0E9, Canada
Single-image 3D reconstruction has advanced rapidly, yet standalone garments remain challenging due to thin-shell geometry, open boundaries, and layered structures. Existing datasets and evaluation protocols provide limited support for systematic evaluation in this setting. We introduce MagicGarment, a benchmark for single-image standalone 3D garment reconstruction with 1,518 professionally constructed assets across 17 composite garment types, aligned multimodal annotations, and a unified evaluation protocol. We further establish a garment-aware reference baseline by adapting a pretrained image-to-3D backbone at two levels: Grounded Semantic Refinement (GSR) provides garment-focused visual conditioning, while Flow-Time-Route LoRA (FTR-LoRA) adapts the frozen 3D generative backbone using flow-timestep, token-routing, and expert-load information. Experiments demonstrate MagicGarment's benchmark utility and the effectiveness of garment-aware adaptation. Project page: https://1019538157.github.io/MagicGarment/.
Figures & tables
Figure 1: Overview of MagicGarment. Left: representative garment assets and aligned multimodal annotations. Right: the single-image standalone 3D garment reconstruction task.
Figure 2: Overview of the garment-aware reference baseline built on pretrained Hunyuan3D. GSR performs input-level garment refinement, while FTR-LoRA provides model-level adaptation of the frozen Hunyuan3D-DiT backbone.
Figure 3: Representative 3D garment assets across the 17 composite types in MagicGarment.
Dataset
Avg. #Vertices
Avg. #Faces
#Types
Method
#Annotations
MagicGarment
375,817
665,488
17
Professional 3D artist modeling
9
SewFactory
8,813
16,834
13
Simulation
3
CLOTH3D
7,473
14,329
6
Simulation
3
DeepFashion3D
21,782
42,613
10
Real scans
4
GarmentCode v2
14,035
26,384
N/A
Simulation
4
Table 1: Comparison of representative 3D garment datasets.
Figure 4: Qualitative comparison on MagicGarment. Each example shows the input image, ground-truth mesh, and reconstructions from our garment-aware baseline and representative image-to-3D methods.
Geometry
Appearance
Normal
Semantic
Method
CD ↓
F-score ↑
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR-N ↑
LPIPS-N ↓
U1-I ↑
U1-T ↑
U2-I ↑
U2-T ↑
Uni-I ↑
Uni-T ↑
Hunyuan3D-2.1
4.2322
12.5148
17.6060
0.8193
0.2294
16.0559
0.2386
0.1139
0.0612
0.3975
0.2845
0.3411
0.2601
AIpparel
34.4063
5.0719
14.1433
0.7452
0.4192
9.5482
0.4608
0.0613
-0.0336
0.1276
0.0731
0.0837
0.0952
ChatGarment
39.4414
4.6322
14.1324
0.7617
0.4161
9.8198
0.4453
0.0607
-0.0324
0.1203
0.0696
0.0899
0.1046
CRM
11.5174
6.5935
18.5177
0.8252
0.2431
15.3413
0.2646
0.1253
0.0688
0.3078
0.2043
0.2712
0.1966
LGM
16.5124
5.2953
18.1696
0.8254
0.2531
15.5809
0.2594
0.1392
0.0599
0.2844
0.1855
0.2487
0.1987
Table 2: Zero-shot benchmark results on MagicGarment. U1/U2 denote ULIP-1/2, Uni denotes Uni3D, and I/T denote image/text similarity.
Geometry
Appearance
Normal
Semantic
Method
CD ↓
F-score ↑
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR-N ↑
LPIPS-N ↓
U1-I ↑
U1-T ↑
U2-I ↑
U2-T ↑
Uni-I ↑
Uni-T ↑
Hunyuan3D-2.1
2.7026
14.0527
19.0146
0.8380
0.1889
17.6048
0.2069
0.1162
0.0597
0.3900
0.2813
0.3475
0.2584
InstantMesh
11.3486
8.0174
17.6216
0.8475
0.1962
18.0185
0.2140
0.1402
0.0586
0.3751
0.2657
0.3198
0.2455
TRELLIS
4.3491
11.6548
17.9737
0.7390
0.2810
13.6614
0.2659
0.1227
0.0592
0.3629
0.2418
0.3094
0.2199
TRELLIS.2
3.2618
15.2347
17.8024
0.7824
0.2518
14.6542
0.2292
0.1334
0.0599
0.3588
0.2342
0.3035
0.2156
Wonder3D
11.0617
10.4935
16.0713
0.8296
0.2480
16.6064
0.2332
0.1359
0.0340
0.3642
0.2461
0.3008
0.2467
Table 3: In-domain adaptation results on MagicGarment. U1/U2 denote ULIP-1/2, Uni denotes Uni3D, and I/T denote image/text similarity.
Reconstructing animatable 3D human avatars from a single image remains particularly challenging for loose garments, whose geometry and motion cannot be adequately represented by body-aligned topology and skinning. We present Forwardrobe, a feed-forward framework for reconstructing garment-aware Gaussian avatars from a single image. Forwardrobe explicitly separates clothing from the body in canonical Gaussian space and equips the garment layer with continuity-aware geometry and skinning initialization, pose-conditioned non-rigid deformation, and appearance adaptation. These designs improve garment reconstruction and visual quality during animation, particularly for skirts and dresses. The separated garment layer additionally forms an independently controllable 3D asset, enabling garment editing, transfer, and 3D virtual try-on. Experiments demonstrate improved garment reconstruction quality and greater flexibility in garment manipulation compared with existing single-image avatar reconstruction methods.
Daisheng Jin, Shuyun Wang, Ying He
Nanyang Technological University · Singapore · The University of Queensland +1
While garments are essential for realistic digital humans, their topological variety makes them much harder to model than parametric bodies. Traditional tailoring relies on 2D sewing patterns, yet bridging these patterns to 3D geometry currently requires physical simulations. We present Stitched Embeddings, the first simulation-free framework to unify 3D garment reconstruction and sewing pattern inference within a single bidirectional latent space. By leveraging the geometric priors of a pretrained 3D foundation model, our approach overcomes the data scarcity typically associated with high-quality garment modeling. We propose to use the BoxMesh as a critical intermediate representation to align 2D panels into 3D configurations without the computational overhead of a simulator. This architecture achieves state-of-the-art accuracy in pattern reconstruction while significantly improving efficiency. Furthermore, our differentiable pipeline enables novel applications, including pattern recovery from meshes and 3D editing from 2D patterns. Finally, this work provides a scalable link between neural 3D vision and the physical garment manufacturing pipeline. Project Page: https://andreus00.github.io/stitchedembeddings
Andrea Sanchietti, Riccardo Marin, Bharat Lal Bhatnagar +2
University of Tübingen, Germany · Tübingen AI Center · Technical University of Munich, Germany +3
Reconstructing 3D clothed humans from monocular images and videos is a fundamental problem with applications in virtual try-on, avatar creation, and mixed reality. Despite significant progress in human body recovery, accurately reconstructing garment geometry, particularly for loose-fitting clothing, remains an open challenge. We propose a unified framework for high-fidelity 3D garment reconstruction from both single images and video sequences. Our approach combines Implicit Sewing Patterns (ISP) with a generative diffusion model to learn expressive garment shape priors in 2D UV space. Leveraging these priors, we introduce a mapping model that establishes correspondences between image pixels, UV pattern coordinates, and 3D geometry, enabling accurate and detailed garment reconstruction from single images. We further extend this formulation to dynamic reconstruction by introducing a spatio-temporal diffusion scheme with test-time guidance to enforce long-range temporal consistency. We also develop analytic projection-based constraints that preserve image-aligned geometry in visible regions while enforcing coherent completion in occluded areas over time. Although trained exclusively on synthetically simulated cloth data, our method generalizes well to real-world imagery and consistently outperforms existing approaches on both tight- and loose-fitting garments. The reconstructed garments preserve fine geometric detail while exhibiting realistic dynamic motion, supporting downstream applications such as texture editing, garment retargeting, and animation.
Yingxuan You, Ren Li, Corentin Dumery +3
CVLab, École Polytechnique Fédérale de Lausanne (EPFL), Switzerland · Mohamed bin Zayed University of Artificial Intelligence, United Arab Emirates · Pinscreen and Mohamed bin Zayed University of Artificial Intelligence, United Arab Emirates