Organizations: CVLab, École Polytechnique Fédérale de Lausanne (EPFL), Switzerland · Mohamed bin Zayed University of Artificial Intelligence, United Arab Emirates · Pinscreen and Mohamed bin Zayed University of Artificial Intelligence, United Arab Emirates
Reconstructing 3D clothed humans from monocular images and videos is a fundamental problem with applications in virtual try-on, avatar creation, and mixed reality. Despite significant progress in human body recovery, accurately reconstructing garment geometry, particularly for loose-fitting clothing, remains an open challenge. We propose a unified framework for high-fidelity 3D garment reconstruction from both single images and video sequences. Our approach combines Implicit Sewing Patterns (ISP) with a generative diffusion model to learn expressive garment shape priors in 2D UV space. Leveraging these priors, we introduce a mapping model that establishes correspondences between image pixels, UV pattern coordinates, and 3D geometry, enabling accurate and detailed garment reconstruction from single images. We further extend this formulation to dynamic reconstruction by introducing a spatio-temporal diffusion scheme with test-time guidance to enforce long-range temporal consistency. We also develop analytic projection-based constraints that preserve image-aligned geometry in visible regions while enforcing coherent completion in occluded areas over time. Although trained exclusively on synthetically simulated cloth data, our method generalizes well to real-world imagery and consistently outperforms existing approaches on both tight- and loose-fitting garments. The reconstructed garments preserve fine geometric detail while exhibiting realistic dynamic motion, supporting downstream applications such as texture editing, garment retargeting, and animation.
Figures & tables
Fig. 2: Pipeline . Given an image of a clothed person, we first estimate the front normal nF of the target garment, and the SMPL body model which is used to render the body part segmentation ( sF , sB ) and depth ( dFb , dBb ) images. The back normal nB of the garment is estimated subsequently by the diffusion model ϵθn . We then predict the UV-coordinate ( cF , cB ) and the depth ( dFg , dBg ) images from the garment normal and body estimations with the mapping model ϵθm . The incomplete UV positional map U~ is produced from them using the camera backprojection. Finally, we fit U~ to DISP to recover the complete UV positional map U^ and the corresponding garment mesh g , which is further improved by the refinement.
Fig. 3: Mapping between pixel, 3D, and UV spaces. The pixel (x,y) is mapped to (X,Y,Z) in the 3D space using the estimated depth d and the camera backprojection P−1 , and to (u,v) in the UV space using the estimated UV coordinates (u,v,σ) . The dash line indicates that (X,Y,Z) and (u,v) are connected indirectly through (x,y) .
Fig. 4: Recovering garment rest geometry . Given (a) the incomplete panel mask M~ , we fit (b) the complete panel mask M by Eq. 11 . (c) shows the overlay of M~ in gray and M in white. (d) is the corresponding rest-state garment mesh gˉ for (b).
Fig. 5: Processing a video sequence. Given a set of images with extracted body segmentations S , body depths Db , and garment normals NF , our method produces a sequence of garment meshes G in three steps. First, the back-view normals NB of the garment are inferred. By design, our method ensures these predictions are temporally consistent. Second, a mapping network estimates the 2D/3D positions of each pixel, where the 2D positions (CF,CB) are in a reference pattern space, and the 3D positions are represented as depth maps (DFg,DBg) . We introduce novel guidance on this generation to match normal estimations and prevent intersection with the body. Finally, this mapping is unwrapped into a partial 2D pattern U~ where pixel value encodes the 3D position, and our temporal inpainting diffusion completes these partial observations into a full garment sequence while ensuring the partial constraints are respected.
Fig. 6: Architecture of the spatio-temporal diffusion model. The model decouples spatial and temporal knowledge into separate modules: the spatial module (red) captures per-frame spatial structure, while the temporal module (blue) models per-pixel motion across frames. Together, these components form the foundation for high-fidelity and temporally smooth 4D garment reconstruction.
Method
Skirt
Trousers
Tshirt
Jacket
CD ↓
NC ↑
IoU ↑
CD ↓
NC ↑
IoU ↑
CD ↓
NC ↑
IoU ↑
CD ↓
NC ↑
IoU ↑
SMPLicit [ 11 ]
5.80
0.34
53.72
1.46
-0.12
74.53
2.77
0.09
58.64
2.40
0.03
54.81
ISP [ 45 ]
4.18
0.78
68.10
1.53
0.78
75.39
1.36
0.82
68.70
1.69
0.79
71.31
GaRec † [ 44 ]
2.67
0.88
96.33
1.67
0.85
92.55
1.19
0.83
87.15
1.21
0.79
91.40
D 3 -Human † [ 8 ]
3.64
0.78
76.42
1.80
0.79
82.49
1.79
0.79
78.51
1.71
0.76
56.60
DMap-Static (Ours)
2.26
0.89
88.80
1.36
0.84
86.23
1.06
0.83
80.20
1.29
0.83
78.28
TABLE I: Quantitative comparison with SOTA methods. “ † ” denotes models that use refinement. Bold : best; Underline : second best.
Fig. 7: Qualitative comparison on an in-the-wild image. The top and bottom rows show the front and the back of the reconstructions produced by our method DMap, BCNet [ 30 ] , SMPLicit [ 11 ] , ISP [ 45 ] , GaRec [ 44 ] and ECON [ 72 ] , respectively.
Fig. 8: Qualitative comparison on a video sequence. Our method (second row) achieves finer geometric detail, better temporal consistency, and avoids the garment–body collisions frequently observed in REC-MV [ 59 ] (bottom row), where the reconstructed garment often penetrates the body surface.
Fig. 9: Reconstruction for in-the-wild images. Our method can recover realistic 3D models for diverse garments in different shapes and deformations.
Fig. 10: Qualitative results on real-world videos. For each sample, we show the input, the front- and back-view reconstructions, and the 2D renderings.
Fig. 11: Ablation of fitting strategy . (a) The input image and its normal estimations for the front and back. (b) Our full reconstruction. (c) Reconstruction without post-refinement. (d) Reconstruction refined by optimizing only the neural displacement field. (e) Reconstruction refined by optimizing only the vertex positions. (f) Reconstruction using only the front normal.
Method
Time/Frame
Hardware
VRAM Usage
I/O
Image-based
SMPLicit [9]
8 min
A100-80G
3.8 GB
1/1
ISP [14]
19 min
A100-80G
2.3 GB
1/1
GaRec † [15]
1 min
A100-80G
12.0 GB
1/1
DMap-Static
7 min
A100-80G
6.8 GB
1/1
DMap-Static †
24 min
A100-80G
6.8 GB
1/1
Video-based
REC-MV † [16]
12 min
A100-80G
19.6 GB
1/1
TABLE II: Inference time. “ † ” denotes models that use refinement.
TABLE IV: Ablation study of the proposed spatial guidance in DMap-dynamic on the CLOTH3D dataset.
Fig. 13: Ablation on spatial guidance. (a) 2D observations, including input image, front normal, and back normal. (b) Reconstruction after refinement. (c) Reconstruction after inference. (d) Inferred result without using generated back-view normals. (e) Inferred result without depth-to-normal guidance GD2N . (f) Inferred result without interpenetration-aware guidance Ginter .
Figure 17Figure 18
Fig. 18: Qualitative results under profile views, occlusions, cropping, and out-of-distribution garment types.
Fig. 19: Qualitative results on a profile-view and occluded video sequence.
Fig. 20: Reconstruction results on multi-layered structures.
Fig. A: Qualitative comparison with the pattern-draping-fitting method on images.
Fig. B: Qualitative comparison with the pattern-draping-fitting method on a video sequence.
Fig. C: Comparison between DMap-Static and GaRec [ 43 ] .
Fig. D: More reconstruction results for in-the-wild images. DMap-Static can handle both the tight-fitting and the loose-fitting garments and recover high-fidelity 3D meshes for them.
Fig. E: Additional qualitative comparison for in-the-wild videos. DMap-Dynamic achieves both geometric accuracy and temporal consistency.
Fig. F: Visualization of error distribution on the reconstructed jacket mesh. The bright colors indicate large errors.
Fig. G: Visualization of error distribution on the reconstructed skirt mesh. The bright colors indicate large errors.
Fig. H: Recovering the rest garment using accumulated partial masks in DMap-Dynamic .
Eq. 11
λarea=0.5,λz=0.02
Eq. 12
ρ=20
Eq. 14
λm=1,λd=2,λn=2,λu=100,λp=1
TABLE A: The values of hyper-parameters.
CD ↓
IoU ↑
NC ↑
w/o NB
1.67
90.04
0.78
w/o fϕ , w/o V
1.45
92.67
0.81
w/o fϕ , w/ V
1.32
93.34
0.82
w/ fϕ , w/o V
1.27
94.71
0.82
Ours
1.21
95.32
0.83
TABLE B: Ablation study. w/o NB means optimization without using the back normal estimation NB . fϕ denotes optimization using the neural displacement field, and V denotes optimization conducted directly on vertex positions.
Fig. I: Clothing looseness variation. Columns correspond to garment categories from left to right: T-shirt, trousers, jacket, and skirt. In each column, examples of the same garment category are ordered from top to bottom from tight-fitting to loose-fitting.
Fig. J: Loose-garment reconstruction from Internet videos . First row: loose-fitting T-shirt and trousers. Second row: skirt.
Fig. K: CLOTH3D dataset on the 33 garments in each category.
Fig. L: Re-simulated CLOTH3D garments using AMASS motion sequences.
While garments are essential for realistic digital humans, their topological variety makes them much harder to model than parametric bodies. Traditional tailoring relies on 2D sewing patterns, yet bridging these patterns to 3D geometry currently requires physical simulations. We present Stitched Embeddings, the first simulation-free framework to unify 3D garment reconstruction and sewing pattern inference within a single bidirectional latent space. By leveraging the geometric priors of a pretrained 3D foundation model, our approach overcomes the data scarcity typically associated with high-quality garment modeling. We propose to use the BoxMesh as a critical intermediate representation to align 2D panels into 3D configurations without the computational overhead of a simulator. This architecture achieves state-of-the-art accuracy in pattern reconstruction while significantly improving efficiency. Furthermore, our differentiable pipeline enables novel applications, including pattern recovery from meshes and 3D editing from 2D patterns. Finally, this work provides a scalable link between neural 3D vision and the physical garment manufacturing pipeline. Project Page: https://andreus00.github.io/stitchedembeddings
Andrea Sanchietti, Riccardo Marin, Bharat Lal Bhatnagar +2
University of Tübingen, Germany · Tübingen AI Center · Technical University of Munich, Germany +3
Automated generation of production-ready 3D garment assets from a single image is a central challenge in digital content creation. While recent generative models have significantly advanced 3D geometry reconstruction, synthesizing high-quality textures remains a bottleneck. Existing methods often bake environmental illumination and shadows directly into the texture map, or they fail to maintain global structural coherence, making the resulting assets unusable for physical simulation and relighting. In this work, we introduce OmniFabric, a novel approach that synthesizes globally coherent texture maps directly within the 2D sewing pattern space. Given a single reference image, our pipeline utilizes an estimated 3D mesh and generative priors of powerful Vision-Language Models (VLM) to establish a complete but coarse texture initialization across the unwrapped sewing patterns. We then leverage a specialized diffusion transformer, trained via an automated synthetic data engine and conditioned on 3D positional features, to refine this initialization directly in the canonical UV domain. This effectively removes distortion and baked-in artifacts to extract a clean and normalized texture map that preserves the original garment design. Extensive experiments demonstrate that OmniFabric significantly outperforms state-of-the-art baselines, yielding photorealistic 3D garments with high-quality textures.
Ding-Jiun Huang, Yuanhao Wang, Cheng Zhang +4
Carnegie Mellon University, United States of America · University of Washington, United States of America · Texas A&M University, United States of America +2
Reconstructing animatable 3D human avatars from a single image remains particularly challenging for loose garments, whose geometry and motion cannot be adequately represented by body-aligned topology and skinning. We present Forwardrobe, a feed-forward framework for reconstructing garment-aware Gaussian avatars from a single image. Forwardrobe explicitly separates clothing from the body in canonical Gaussian space and equips the garment layer with continuity-aware geometry and skinning initialization, pose-conditioned non-rigid deformation, and appearance adaptation. These designs improve garment reconstruction and visual quality during animation, particularly for skirts and dresses. The separated garment layer additionally forms an independently controllable 3D asset, enabling garment editing, transfer, and 3D virtual try-on. Experiments demonstrate improved garment reconstruction quality and greater flexibility in garment manipulation compared with existing single-image avatar reconstruction methods.
Daisheng Jin, Shuyun Wang, Ying He
Nanyang Technological University · Singapore · The University of Queensland +1