Garment flattening and folding require robots to identify garment structure during manipulation and adapt their actions as the cloth deforms. We present ClothDojo, a simulation platform that generates adaptive manipulation trajectories at scale across diverse garments and initial states and evaluates policies in closed loop. ClothDojo encodes each garment's canonical mesh coordinates as an NOCS texture, providing canonical surface-position cues through the visual interface of a pretrained vision-language-action model. We fine-tune the model on human demonstrations and corrections to obtain a data-generation policy. The policy autonomously generates simulated manipulation trajectories, and we retain rollouts that satisfy task-specific success criteria. Each retained trajectory is re-rendered under a randomly sampled appearance to provide RGB observations for downstream policies that do not receive NOCS. Across both tasks, the policy produces nearly 8,000 successful trajectories over hundreds of garments. RGB policies trained on these generated trajectories outperform those trained directly on the original human demonstrations and corrections across all evaluated garment splits and appearance settings. Evaluations across multiple policy models show that overall performance and appearance robustness vary across models and tasks, while generalization to unseen garments remains a common challenge.
Figures & tables
Fig. 2: Autonomous policies generate long-horizon behaviors with varied action transitions and adjustments during execution. Flattening trajectories combine midair regrasping, dynamic unfolding, and local adjustments; folding includes hand switching, garment rotation, and post-fold adjustments. Each row shows keyframes from an independent rollout, ordered from left to right with nonuniform time intervals. Box labels describe observed behaviors; the images are re-rendered from recorded simulation states.
Fig. 3: Overview of the NOCS-guided data-generation pipeline. Expert demonstrations DE initialize a data-generation policy. For selected failures encountered during autonomous execution, an operator restores a chosen state and provides a correction DH . The two sources of human data update the policy, while successful rollouts are re-rendered under randomized appearances to train an RGB policy.
Fig. 4: The core garment pool spans diverse geometries. The figure shows most of the 500 assets, covering different garment lengths, widths, sleeve types, necklines, and asymmetric cuts. All garments are naturally settled and shown at their relative physical sizes.
Task
Expert
HIL corrections
raw → aug. / garments
raw → aug. / garments
Flatten
890 → 1,780 / 182
201 → 402 / 88
Fold
537 → 1,074 / 156
213 → 426 / 125
TABLE I: Human training data for the data-generation policies.
Method
SR (%) ↑
Seen
Unseen
Def. ↓
Rot. ↓
White
24.8
26.3
23.3
0.081
0.024
+ NOCS
62.9
66.3
59.5
0.080
0.024
+ Mirror
69.6
76.0
63.3
0.080
0.025
+ HIL
76.5
82.3
70.8
0.080
0.024
Expert
–
–
–
0.068
0.019
TABLE II: Data-generation-policy performance on flattening.
Method
SR (%) ↑
Seen
Unseen
Q↑
White
55.1
61.0
44.3
0.809
+ NOCS
57.6
61.2
51.1
0.779
+ Mirror
60.5
64.2
53.6
0.846
+ HIL
70.0
73.7
63.2
0.825
Expert
–
–
–
1.037
TABLE III: Data-generation-policy performance on folding.
Fig. 5: The data-generation policy produces successful terminal states with visual quality close to human demonstrations. Rows show selected released states at P10, P50, and P90 of each source’s eligible subset, ranked jointly by deformation and orientation errors for flattening, and by Q for folding. The two values below each flattening image are ddef and drot , respectively (lower is better); folding images report Q (higher is better). Flattening images use later released states, whereas Table II measures first confirmed success.
Task
Human
Generated
Combined
Flatten
1,091 / 217
3,992 / 400
5,083 / 470
Fold
750 / 242
3,856 / 399
4,606 / 458
TABLE IV: RGB-policy training data (trajectories / garments).
Task
Training set
Seen garments
Unseen garments
Avg.
S app.
U app.
S app.
U app.
Flatten
Human
33.3
30.5
22.3
19.3
26.3
Generated
41.3
33.3
23.5
23.8
30.4
Combined
32.8
28.0
20.3
19.5
25.1
Fold
Human
68.0
61.8
40.0
37.8
51.9
Generated
72.3
69.0
47.8
47.0
59.0
TABLE V: RGB-policy success rates (%) by training set.
Task
Method
Seen garments
Unseen garments
Avg.
S app.
U app.
S app.
U app.
Flatten
π0.5
41.3
33.3
23.5
23.8
30.4
StarVLA
25.0
14.3
12.8
8.3
15.1
GR00T
16.0
17.3
10.8
12.3
14.1
Diffusion
3.5
4.5
1.3
3.0
3.1
Fold
π0.5
72.3
69.0
47.8
47.0
59.0
TABLE VI: RGB-policy success rates (%) by method.
Robot
Flatten → Fold
Fold
Flatten SR
Fold SR
Fold SR
Piper
49.4
20.0
17.8
UR5e
68.9
7.1
27.8
TABLE VII: Task composition and cross-embodiment reuse.
Fig. 6: Potential task extensions of ClothDojo. Left: humanoid dressing. Right: bimanual garment hanging. Both settings introduce more complex cloth–rigid-body contacts than tabletop flattening and folding.