RFW-A: Agentic Procedural Generation of 400K Articulated Assets over 500 Categories
Authors: Yuanbo Lyu, Changjian Pan, Saining Zhang, Yi Yang, Peishuo Li, Zhiheng Li, Jialu Liu, Junhao Chen, Jiawei Weng, Yupeng Gao, Weiqing Xiao, Wenyi Li, Jiarui Guo, Rui Zhang, Jingrun Du, Weiyi Lin, Zenan Meng, Hu Wei, Lin Qu, Zongzheng Zhang, Hao Zhao, Chaojian Li
Organizations: Alibaba Group · Tsinghua University · Peking University · Nanyang Technological University · Nanjing University · The Hong Kong University of Science and Technology
High-quality articulated assets are essential for embodied perception, simulation, and manipulation, yet constructing them at scale remains difficult. Manual annotation and procedural modeling require substantial human effort. Recent agentic systems automate asset construction through program synthesis, but repeat costly reasoning and refinement for every new asset. Our key observation is that assets within a category can share construction logic despite variations in structure and appearance. Agentic synthesis and procedural generation are therefore complementary: agents can construct reusable programs, while procedural execution reuses the resulting logic across many samples. We therefore present RFW-A, which combines the two in a new articulated-asset generation paradigm. Starting from existing articulated programs, RFW-A organizes reusable components and their dependencies into category-level generators. Agents test and refine these generators across sampled configurations and motion states. Once constructed and validated, the generators produce diverse assets through procedural execution with no further model calls or token charges. The estimated API cost of generator authoring averages 10.62,amortizingto0.05 per asset over 200 assets, versus $1.24 for the sampled per-asset agentic synthesis baseline. Using RFW-A, we built RFW-A-400K, containing over 400K sim-ready articulated assets across more than 500 categories. Experiments show high structural and kinematic validity and stable execution across physics simulators. Beyond asset validity, training with RFW-A-400K improves part-level articulation estimation on seen and unseen categories, while policies trained in simulation with retrieved and program-edited assets transfer zero-shot to real articulated objects.
Figures & tables
Figure 2: Cabinet families produced by reusable procedural generators. Across cabinet configurations, the generators vary materials, dimensions, door types, drawer layouts, and appearance while preserving shared rules for panel sizing, part placement, and joint attachment.
Figure 3: Comparison of articulated-asset construction paradigms. Manual annotation operates per asset; procedural generation amortizes sampling but requires category-specific generator authoring; and agentic synthesis repeats model inference for each asset. RFW-A uses agentic synthesis to construct a reusable generator for each object family, then produces assets through procedural sampling.
Figure 4: Overview of the RFW-A generator-construction pipeline. Panel (a) shows how agents extract reusable components from source assets and executable programs, organize them into a category blueprint, and compile the blueprint into a generator. Panel (b) shows procedural execution by sampling choices, resolving dependent parameters, and assembling an asset, followed by validation and repair. Panel (c) shows that a released generator produces a family of articulated variants from different seeds, while exposing the joint motion encoded by the generator.
Dataset
Assets
Cats.
Type
Physical Grounding
Metric
Texture
F-mat.
Density
E
ν
Friction
e
PartNet-Mobility
2,347
46
Manual
×
△
×
×
×
×
×
×
Infinite Mobility
On-demand
22
Procedural
×
△
×
×
×
×
×
×
Infinigen-Articulated
8,226
17
Procedural
△
✓
×
△
×
×
△
×
PhysX-Mobility
2,024
45
Manual
×
△
×
✓
✓
✓
×
×
Artiverse
3,544
84
Manual
×
✓
×
✓
×
×
×
×
Table 1: Release-level comparison of articulated-object datasets, their construction types, and available annotations. Cats. denotes categories, Proc. denotes procedural generation, and Metric denotes metric-scale geometry. Physical Grounding groups the physical-property fields (Section 4.2 ): E is Young’s modulus, ν is Poisson’s ratio, Friction includes static and dynamic coefficients, and e is the coefficient of restitution. F-mat. marks fine-grained material annotation, i.e., material assignments below the articulated-part level. ✓ indicates that a field or resource is included in the released data or documented format, △ indicates partial or format-dependent availability, and × indicates that it is absent. The RFW-A-400K row follows the release specification and evaluation manifest.
Figure 5: Qualitative comparison of Articraft, LAM, and RFW-A -400K. The three rows are the compared datasets; each row contains four quality dimensions (geometry, part segmentation, motion, and material appearance).
Figure 6: API-cost scaling of reusable category-level generators versus per-asset agentic synthesis. (a) Our mean generator-authoring cost is amortized over N assets, while the baseline averages N recorded synthesis costs. (b) Our one-time API cost stays fixed, while the baseline accumulates a new charge for each asset.
Figure 7: How a frozen generator’s per-surface material-class identifiers become concrete materials and physical properties at export: each identifier selects a frozen material pool, a seeded draw binds one pool member to the surface, and that material’s physical properties are copied into the asset’s physics record.
Figure 8: Directed 3D editing enabled by agent-generated procedural programs: (a) structural editing, (b) count editing, (c) material editing, and (d) dimensional editing.
Training data
Rest-Pose Segmentation
Articulated Geometry
mIoU ↑
gIoU ↑
PC ↓
gIoU ↑
PC ↓
OC ↓
In-domain
PM-only
0.60262
0.46364
0.09552
0.44339
0.15760
0.01253
PM+Articraft
0.71442
0.62972
0.05371
0.60164
0.10196
0.00778
PM+ RFW-A -400K
0.78873 (+10.4%)
0.73550 (+16.8%)
0.03501 (+34.8%)
0.70295 (+16.8%)
0.06268 (+38.5%)
0.00661 (+15.0%)
Category OOD
Table 2: Articulation estimation with different training datasets. Bold marks the best result for each metric within each evaluation split.
Figure 9: Qualitative comparison of articulation estimation. Rows show PM-only, PM+Articraft, PM+ RFW-A -400K, and ground truth (GT). Red arrows and circles highlight local differences in supports, attachments, and movable parts.
Setting
Retrieved Asset
Edited Variants
Sim. Demos
Retrieval
1
0
500
Retrieval + 4 Edit
1
4
500
Retrieval + 9 Edit
1
9
500
Table 3: Simulation configurations for robotic manipulation. All settings use exactly 500 simulation demonstrations per task.
Figure 10: Sim-to-real robotic manipulation. Simulation training environments (top) and corresponding real-world evaluations (bottom) across five articulated-object manipulation tasks.
Setting
Cabinet
Folding Chair
Double Door
Toaster
Folding Ruler
Avg.
Retrieval
5/10
7/10
8/10
7/10
4/10
6.2/10
Retrieval + 4 Edit
7/10
7/10
8/10
8/10
6/10
7.2/10
Retrieval + 9 Edit
8/10
8/10
8/10
9/10
7/10
8.0/10
Table 4: Zero-shot sim-to-real manipulation. Real-world successes over 10 trials per task. All settings use the same 500 simulation demonstrations per task.
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
Scale
Movable joints
Structural variation
Assets
Parsed links
Mean
P-90
Multi-joint (%)
Within-cat. var.
nTED
PartNet-Mobility
2,347
16,665
5.10
9
59.86
24.52
0.411
Infinigen-Articulated
8,226
48,427
3.89
9
73.86
2.15
0.393
PhysX-Mobility
2,024
26,016
4.89
7
55.09
14.83
0.354
Artiverse
3,544
28,971
4.60
7
77.65
12.76
0.372
LAM
3,217
18,306
3.00
5
53.59
2.19
0.373
Appendix
Table 5: Dataset scale and structural variation. Joint statistics are computed over successfully parsed assets. Within-category joint-count variance is averaged equally across each source’s categories. nTED is averaged over all pairs of 250 valid trees per source, including cross-category pairs. Bold marks the best value in each ranked column, where a larger value indicates greater scale or structural diversity; mean and P-90 movable-joint counts are descriptive and are not ranked.
Dataset
(a) Validity
(b) Geometry
(c) Simulation
Full release
Core-200 (%)
Core-200
Resource resol. ↑
Non-deg. ↑
Kinematic pass ↑
Joint gap P95 ↓
Axis support ↑
Import ↑
DoF mapping ↑
Stable rollout ↑
Inertials ↑
PartNet-Mobility
0.9859
0.9979
0.9928
39.138
57.952
0.420
0.258
0.415
0.000
Infinigen-Articulated
0.9995
0.8053
0.6944
30.223
50.591
0.930
0.903
0.930
0.000
PhysX-Mobility
1.0000
0.2664
0.5924
45.481
47.427
0.995
0.998
0.990
0.000
Artiverse
0.9997
0.9704
0.9405
37.433
35.673
0.770
0.724
0.770
1.000
Appendix
Table 6: Asset evaluation across three test groups. (a) File and kinematic validity on each complete release. (b) Rest-pose assembly geometry on Core-200. (c) Import, DoF mapping, inertial completeness, and 1-second numerical stability on Core-200. Values in (a) and (c) are fractions; (b) uses percentages. Lower is better for Joint Gap. Bold marks the best value within each column. See Appendix A.13 for the definitions of the geometry metrics and Appendix Table 9 for the motion-sweep collision results.
Dataset
Resource resol. ↑
Valid tree ↑
Joint spec. valid ↑
Collision cov. ↑
Strict URDF ↑
PartNet-Mobility
0.9859
1.0000
0.9979
0.0000
0.0000
Infinigen-Articulated
0.9995
1.0000
1.0000
0.0000
0.0000
PhysX-Mobility
1.0000
1.0000
1.0000
0.0000
0.0000
Artiverse
0.9997
0.9949
0.9989
0.9757
0.9715
LAM
0.9714
0.9226
0.9568
0.4597
0.4109
SketchMobility
0.9903
0.9986
0.9996
0.3860
0.3858
Appendix
Table 7: URDF validity on the complete release of each dataset. Strict URDF is the per-asset conjunction of resource resolution, tree validity, joint specification, and collision coverage. All values are fractions; the best result in each column is bold. See Appendix A.11 for the complete criteria.
Dataset
Valid range ↑
Joint sweep ↑
Non-deg. ↑
Joint-level pass ↑
Asset-level pass ↑
PartNet-Mobility
0.9960
1.0000
0.9979
0.9939
0.9928
Infinigen-Articulated
1.0000
1.0000
0.8053
0.8053
0.6944
PhysX-Mobility
1.0000
1.0000
0.2664
0.2664
0.5924
Artiverse
0.9993
0.9939
0.9704
0.9697
0.9405
LAM
0.9949
0.8604
0.8574
0.8530
0.8775
SketchMobility
0.9998
0.9991
0.8230
0.8229
0.7203
Appendix
Table 8: Kinematic executability on the complete release of each dataset. Joint-level pass evaluates individual declared movable joints; asset-level pass requires every movable joint of an asset to pass. All values are fractions; the best result in each column is bold. See Appendix A.12 for the sweep and denominator rules.
Dataset
Rest-Pose Geometry
Motion Sweep ( K=9 )
Joint Support ↑
All-joint Support ↑
Joint Gap P95 ↓
Axis Support ↑
Pose Axis Support ↑
Collision-free Poses ↑
Penetr. Growth P95 ↓
PartNet-Mobility
20.124
15.500
39.138
57.952
48.688
83.023
29.402
Infinigen-Articulated
60.344
42.500
30.223
50.591
39.028
76.840
37.200
PhysX-Mobility
13.148
9.000
45.481
47.427
39.754
78.152
36.497
Artiverse
16.657
5.500
37.433
35.673
17.611
84.599
32.763
LAM
21.933
10.500
54.218
32.361
17.623
71.075
43.443
Appendix
Table 9: Geometric support and collision behavior on Core-200. Rest-pose metrics evaluate assembly support, part separation, and axis placement; motion metrics evaluate support retention and additional penetration over nine joint states. All values are percentages. Lower is better for Joint Gap and Penetration Growth; the best result in each column is bold. See Appendix A.13 for definitions.
Dataset
N
Import ↑
DoF mapping ↑
Complete non-placeholder inertials ↑
Genesis 10 s simulation validity ↑
Trajectory coverage ↑
PartNet-Mobility
200
0.950
0.920
0.000
0.950
0.000
Infinigen-Articulated
200
0.930
0.903
0.000
0.930
0.000
PhysX-Mobility
200
1.000
1.000
0.000
0.990
1.000
Artiverse
200
0.995
0.992
1.000
0.960
0.955
LAM
200
0.875
0.989
0.000
0.870
0.958
SketchMobility
200
0.995
0.990
0.400
0.995
0.801
Appendix
Table 10: Runtime compatibility and physical stability on Core-200. Panel (a1) evaluates import and rollout behavior in Genesis; panel (a2) evaluates free-root center-of-mass stability in Genesis; and panel (b) requires each asset or joint to pass the corresponding test in Genesis, PyBullet, MuJoCo, and Isaac Sim. All reported rates are fractions.
Genesis
PyBullet
MuJoCo
Isaac Sim
Dataset
Imp. ↑
DoF ↑
Stable ↑
Imp. ↑
DoF ↑
Stable ↑
Imp. ↑
DoF ↑
Stable ↑
Imp. ↑
DoF ↑
Stable ↑
PartNet-Mobility
0.950
0.920
0.945
0.985
0.994
0.985
0.420
0.258
0.420
0.975
0.984
0.970
Infinigen-Articulated
0.930
0.903
0.930
1.000
1.000
1.000
0.995
0.995
0.995
1.000
1.000
1.000
PhysX-Mobility
1.000
1.000
0.995
1.000
1.000
0.995
1.000
1.000
0.995
0.995
0.998
0.990
Artiverse
0.995
0.992
0.960
1.000
1.000
1.000
0.850
0.855
0.845
0.915
0.839
0.915
LAM
0.875
0.989
0.870
0.880
0.994
0.880
0.440
0.546
0.420
0.920
1.000
0.895
Appendix
Table 11: Per-engine cross-simulator rates on Core-200: native import success, canonical DoF mapping (joint-level), and passive stable rollout (asset-level), each evaluated independently per engine. The main-paper panel (b) reports the per-asset and per-joint intersection across all four engines, not an average of these per-engine rates.
Dataset
Classes
Renders
Metric N
DINOv2 Top-1 ↑
DINOv2 Top-4 ↑
DINOv2 silh. ↑
CLIP Top-1 ↑
CLIP Top-4 ↑
CLIP silh. ↑
PartNet-Mobility
46
230
230
0.696
0.567
0.196
0.552
0.365
0.056
Artiverse
84
368
363
0.716
0.604
0.233
0.540
0.395
0.074
LAM
660
1,279
860
0.226
0.149
-0.301
0.152
0.113
-0.442
Articraft
244
1,193
1,187
0.530
0.413
0.073
0.405
0.266
-0.051
RFW-A-400K (Ours)
531
2,655
2,655
0.881
0.704
0.289
0.592
0.373
0.064
Appendix
Table 12: Feature-space class consistency under standardized Blender rendering using all successfully rendered assets. Classes and renders include every available class and image; metric N excludes only images from singleton classes when a positive same-class pair is required. Top-1 and Top-4 are the same-class fractions among the nearest one and four neighbors, respectively. The cosine silhouette is (bi−ai)/max(ai,bi) over L2-normalized features. The best value in each metric column is bold.
Figure 11: Joint all-available t-SNE overviews in DINOv2-base (left) and CLIP ViT-B/32 (right) feature spaces. Within each encoder, panels use that encoder’s joint embedding coordinates and axis limits; the two encoders are embedded separately. Colors identify datasets ( RFW-A uses larger outlined markers); all renders, including classes with fewer than five renders, are shown.
Figure 12: All-available feature-space metrics reported in Table 12 . Singleton classes are excluded only from metrics requiring a positive same-class pair; triangles mark the best value within each encoder.
Figure 24Figure 25
Category
PM
Articraft
RFW-A (new)
Total
Dishwasher
4
4
20
28
Oven
1
11
20
32
Refrigerator
4
12
20
36
Storage furniture
55
11
20
86
Table
9
17
20
46
Washing machine
1
6
20
27
Appendix
Table 13: Category and source composition of the in-domain evaluation split. The added objects belong to the same six categories as the original PartNet-Mobility evaluation split. All three models are evaluated on the complete combined split with equal weight per object.
Training data
Rest-Pose Segmentation
Articulated Geometry
mIoU ↑
gIoU ↑
PC ↓
gIoU ↑
PC ↓
OC ↓
PM-only
0.79658
0.75910
0.02554
0.73732
0.09812
0.01114
PM+Articraft
0.77070
0.73300
0.02276
0.70978
0.09562
0.01232
PM+ RFW-A -400K
0.78706
0.74752
0.02442
0.72360
0.08508
0.01028
Appendix
Table 14: Articulation estimation on the original PartNet-Mobility evaluation split using the three 150K checkpoints. Results average three training seeds × five evaluation seeds; bold marks the best result. This original split contains 55 storage-furniture objects and 19 objects across the other five categories.
Variant
Mean Judge
Strict Valid
Avg. Iter.
Est. Cost ($)
w/o Component Extraction
0.500
35/108
9.33
3.80
w/o Structural Blueprint
0.401
72/108
11.33
3.73
w/o Validation & Repair
0.241
0/108
0.00
0.83
RFW-A (full)
0.580
108/108
8.00
6.73
Appendix
Table 15: Aggregate generator-component ablation outcomes across flip_phone , microscope , and folding_chair . Mean Judge and Avg. Iter. are category means; Strict Valid reports pooled counts.
Variant
Geometry
Aesthetics
Semantic
Judge
w/o Component Extraction
5.17
4.83
6.50
0.500
w/o Structural Blueprint
4.67
3.83
5.33
0.401
w/o Validation & Repair
2.42
2.83
3.25
0.241
RFW-A (full)
6.25
5.42
7.00
0.580
Appendix
Table 16: Mean component scores across flip_phone , microscope , and folding_chair for the same ablations. Each category receives equal weight; higher is better and bold marks the best mean in each column.
Figure 18: Qualitative comparison of generator-component ablations. Rows show (a) flip phone, (b) microscope, and (c) folding chair. From left to right, columns compare full RFW-A with variants removing Component Extraction, Structural Blueprint, and Validation & Repair, respectively. Quantitative results are reported in Tables 15 and 16 . The panels are qualitative visual examples; mechanical validity is measured by the strict-validity probes in these tables.
Figure 19: Cabinet generator families. Each panel shows assets sampled from one reusable procedural cabinet generator: (top left) a cabinet generator with drawers, (top right) a cabinet generator with drawers and doors, (bottom left) a cabinet generator with doors, and (bottom right) the hutch cabinet generator of Figure 2 , whose row labeled Original reproduces the reference asset; all other rows and columns are generated variants. Within each panel, rows vary component choices, materials, drawer and door layouts, and appearance while the generators preserve shared rules for panel sizing, part placement, and joint attachment.
A bottleneck in learning to understand articulated 3D objects is the lack of large and diverse datasets. In this paper, we propose to leverage large language models (LLMs) to close this gap and generate articulated assets at scale. We reduce the problem of generating an articulated 3D asset to that of writing a program that builds it. We then introduce a new agentic system, Articraft, that writes such programs automatically. We design a programmatic interface and harness to help the LLM do so effectively. The LLM writes code against a domain-specific SDK for defining parts, composing geometry, specifying joints, and writing tests to validate the resulting assets. The harness exposes a restricted workspace and interface to the LLM, validates the resulting assets, and returns structured feedback. In this way, the LLM is not distracted by details such as authoring a URDF file or managing a complex software environment. We show that this produces higher-quality assets than both state-of-the-art articulated-asset generators and general-purpose coding agents. Using Articraft, we build Articraft-10K, a curated dataset of over 10K articulated assets spanning 245 categories, and show its utility both for training models of articulated assets and in downstream applications such as robotics simulation and virtual reality.
Matt Zhou, Ruining Li, Xiaoyang Lyu +6
University of Cambridge · University of Oxford · Nanyang Technological University
Articulated objects are central to interactive 3D applications, including embodied AI, robotics, and VR/AR, where functional part decomposition and kinematic motion are essential. Yet producing high-fidelity articulated assets remains difficult to scale because it requires reliable part decomposition and kinematic rigging. Existing approaches largely fall into two paradigms: optimization-based reconstruction or distillation, which can be accurate but often takes tens of minutes to hours per instance, and inference-time methods that rely on template or part retrieval, producing plausible results that may not match the specific structure and appearance in the input observation. We introduce a part-centric generative framework for articulated object creation that synthesizes part geometry, composition, and articulation under explicit part-aware conditioning. Our representation models an object as a set of movable parts, each encoded by latent tokens augmented with part identity and articulation cues. Conditioned on a single image, the model generates articulated 3D assets that preserve instance-level correspondence while maintaining valid part structure and motion. The resulting approach avoids per-instance optimization, enables fast feed-forward inference, and supports controllable assembly and articulation, which are important for embodied interaction. Experiments on common articulated categories (e.g., drawers and doors) show improved input consistency, part accuracy, and articulation plausibility over optimization-based and retrieval-driven baselines, while substantially reducing inference time.
Qingming Liu, Xinyue Yao, Shuyuan Zhang +4
The Chinese University of Hong Kong, Shenzhen, China · DexForce Technology, China
Robot learning in simulation depends on the objects the simulator offers. Many tasks need objects with separate parts, joints that allow the required motion, and physical properties that remain valid under contact. Existing methods recover this structure anew for every image: generative models predict parts and joints that mostly fail to settle or move in simulation, and general-purpose agents need a long session of model calls for each photograph. AffordCraft builds such an asset from a single RGB image and a task instruction by retrieval instead of generation: it locates the object and the part to operate, selects a matching entry from a library of articulated assets, and fits it to the image while keeping its parts and joints intact. Without any box or mask marking the object, AffordCraft produces a physically valid asset for 1,703 of 2,000 photographs from 31 categories. Five generative methods pass on at most 45% of the same photographs and, at the median, need 10 to 78 times our GPU time per valid asset. On 50 cluttered images, 162 of 237 annotated objects pass the same physical test after automatic detection. Growing the library from 141 to 11,372 entries needs no change to the method and raises category coverage from 46% to 100% and the share of selections with the requested label from 18% to 51%. We also build manipulation tasks from the constructed assets, both with single objects and in composed scenes; policies trained on scripted demonstrations complete both kinds of tasks from initial states unseen in training.
Haoyun Yang, Xueyang Zhou, Ziyi Xie +1
College of AI, Tsinghua University · School of Computer Science, Chongqing University · School of Cyber Science and Engineering, Huazhong University of Science and Technology +1