Creating interactive digital environments for gaming, robotics, and simulation relies on articulated 3D objects whose functionality emerges from their part geometry and kinematic structure. However, existing approaches remain fundamentally limited: optimization-based reconstruction methods require slow, per-object joint fitting and typically handle only simple, single-joint objects, while retrieval-based methods assemble parts from a fixed library, leading to repetitive geometry and poor generalization. To address these challenges, we introduce ArtLLM, a novel framework for generating high-quality articulated assets directly from complete 3D meshes. At its core is a 3D multimodal large language model trained on a large-scale articulation dataset curated from both existing articulation datasets and procedurally generated objects. Unlike prior work, ArtLLM autoregressively predicts a variable number of parts and joints, inferring their kinematic structure in a unified manner from the object's point cloud. This articulation-aware layout then conditions a 3D generative model to synthesize high-fidelity part geometries. Experiments on the PartNet-Mobility dataset show that ArtLLM significantly outperforms state-of-the-art methods in both part layout accuracy and joint prediction, while generalizing robustly to real-world objects. Finally, we demonstrate its utility in constructing digital twins, highlighting its potential for scalable robot learning.
Figures & tables
Figure 2: Architecture Overview. Given an input point cloud, ArtLLM first predicts a tokenized articulation blueprint that specifies part layouts and kinematic structures . This blueprint then conditions a part-aware generative model to synthesize high-fidelity link geometries , followed by a physics-based joint-limit correction module refines the articulation, producing simulation-ready articulated assets.
Dataset Source
#Objects
#Categories
PartNet-Mobility [ 54 ]
2168
43
PhysX3D [ 2 ]
7672
23
Infinite-Mobility [ 22 ]
10833
13
Total
20673
43
Table 1: Statistics of our curated dataset for ArtLLM training.
Figure 3: Physical limit calcualtion. Illustration for our physical based limit correction process.
Figure 4: Qualitative Comparison . Baseline methods rely on retrieving parts from a fixed asset library, hence often fail to recover accurate geometry and frequently generate incorrect articulations with mismatched joint types or misaligned joint positions. In contrast, our approach produces geometry that closely matches the input and recovers correct, coherent articulations.
Method
mIoU
CD
Type Acc
Joint-Axis-Err
Joint-Pivot-Err
Joint-Range-IoU
Graph Acc
Time(s)
Ours
0.6884
0.028
0.9084
0.1271
0.0801
0.7398
0.7741
19
ArtAny
0.3381
0.072
0.8457
0.4529
0.5361
0.8653
0.6142
522
Singapo
0.4330
0.044
0.7649
0.2445
0.2567
0.5256
0.4564
84
Singapo*
0.4705
0.048
0.9065
0.2463
0.1465
0.6184
0.6851
84
urdformer
0.1225
0.249
0.6068
0.7377
0.6095
0.7032
0.0791
183
Table 2: Quantitative Comparison. We evaluate all methods on the seven categories of the PartNet-Mobility dataset (splitted by SINGAPO [ 25 ] ). Metrics are computed per category, and the final score is obtained by an average over all seven categories. * denotes retraining on our dataset. Our method attains high performance, highlighting its strong ability to recover accurate articulated structures.
IoU
TA
JAE
JPE
JRI
GA
Full
0.473
0.898
0.141
0.135
0.582
0.780
A
0.352
0.823
0.277
0.235
0.575
0.775
B
0.464
0.825
0.289
0.131
0.510
0.737
C
0.412
0.894
0.142
0.138
0.577
0.754
D
0.463
0.890
0.143
0.175
0.511
0.780
Table 3: Quantitative Ablation. Ablation experiments evaluating the impact of our key components.
Figure 5: Qualitative result for physical limit correction. Before correction, the predicted joint ranges cause noticeable self-collisions during articulation. After applying our physics-based limit refinement, the articulated parts move smoothly without collision, yielding physically plausible and stable motion.
Figure 6: We teleoperate a Franka Panda robot to execute three articulation tasks and record its pose trajectories. Using ArtLLM, we reconstruct the corresponding articulated assets from real scenes and replay the trajectories in simulation. The simulated objects reproduce the real articulation behavior, showing that our generated assets accurately capture real-world kinematics and joint constraints.
Figure 7: Part number statistic of our curated dataset for training ArtLLM
Category
Method
mIoU
Type Acc
Joint-Axis-Err
Joint-Pivot-Err
Range-IoU
Graph Acc
Table
Ours
0.4528
0.7000
0.1125
0.1393
0.6451
0.7000
ArtAnything
0.2803
0.7500
0.2992
0.1458
0.8029
0.2500
Singapo
0.4716
1.0000
0.0017
0.1992
0.4906
0.4000
URDFormer
0.0999
0.6000
1.0472
0.8528
0.5941
0.1000
StorageFurniture
Ours
0.7759
0.9091
0.1771
0.1192
0.7354
0.9091
ArtAnything
0.4464
0.9200
0.2537
0.4336
0.8555
0.8000
Table 4: Quantitative Comparison. We show the metrics on each category of the 3 method against our method. The ’-’ denotes that no limit is predicted due to the error joint type.
Figure 8: Additional results generated by our ArtLLM , from the test set of the PartNet-Mobility [ 54 ] dataset. The input geometry is generated by Hunyuan3D 3.0 from rendered images. In each pair of images, the left one is the canoical state, and the right one is a sampled articulated state.