Geometrically faithful and functional articulated 3D assets are essential for real-to-sim robot manipulation, where policies trained in simulation must transfer to physical objects. Recent mesh-based methods learn to infer articulation from annotated 3D assets, but deployment remains challenging when real-world objects fall outside the training distribution or their meshes are incomplete or corrupted. To address these limitations, we formulate articulated asset reconstruction as programmatic modeling grounded in partial geometric evidence and introduce USDCraft, a framework in which a pretrained LLM writes and revises executable programs for simulation-ready articulated assets without task-specific training. We propose source geometry analysis, which converts the source mesh into a metric textual description that distinguishes observed surface from unknown space, and iterative geometric rechecking, which re-encodes each candidate in the same representation so that discrepancies point to program edits while unobserved regions remain open to completion. Visual feedback and physical authoring guidance complete the modeling process, which produces articulated USD assets with explicit physical properties that load into Isaac Sim without manual adjustment. Experiments demonstrate leading articulation recovery on two benchmarks and validate USDCraft's effectiveness for real-to-sim-to-real robot manipulation.
Figures & tables
Figure 1: USDCraft enables generation, reconstruction, and real-to-sim-to-real manipulation.
Figure 2: USDCraft overview. The modeling agent refines an asset program, choosing which tools to use and in what order. Blue modules measure the source geometry (reconstruction only), purple modules are shared tools, and orange marks downstream use.
Table 3
Figure 3: USDCraft-bench reconstructions; each pair shows rest (left) and articulated (right) poses. Red rotation glyphs and yellow double arrows indicate revolute and prismatic joints, respectively.
Figure 4: Harness comparison on scanned toaster (a) and drawer (b), and generated chair (c) and game console (d). Pairs show first-run reconstructions at rest (left) and articulated (right) poses.
Table 6
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
USDCraft-bench
Chair
Laundry hamper
Stove
Clothes dryer
Manipulation assembly
Task lamp
Control knob
Microwave
Teapot
Dishwasher
Oven
Telescope
Door
Paper cutter
Toaster
Drawer
Recycling bin
Toaster oven
Appendix
Table 8: Object categories in USDCraft-bench and Lightwheel.
Metric
Mini Workflow–Astra
USDCraft–Astra
Part match
F1(%) ↑
74.48±1.00
84.83±0.88
Static geometry
gIoU ↑
0.512±0.023
0.694±0.010
PC ↓
0.070±0.005
0.052±0.002
mIoU ↑
0.646±0.013
0.784±0.009
Articulated geometry
gIoU ↑
0.495±0.022
0.670±0.009
PC ↓
0.119±0.008
0.093±0.001
Appendix
Table 9: Reconstruction consistency over three Astra runs on USDCraft-bench. Entries report mean ± standard deviation across runs.
Figure 5: Additional reconstruction comparisons. Each method shows rest and articulated poses from left to right; colors distinguish parts and joint markers follow Figure 3 .
Figure 6: USDCraft-bench reconstruction gallery. Each row shows the input image, source mesh, reconstructed appearance, and part colors with joint markers. Source and reconstruction share a camera and scale.
Figure 7: Lightwheel reconstruction gallery using USDCraft–Astra without +G. Columns follow Figure D.3 .
Figure 8: Text- and image-conditioned asset generation. Each example shows its input evidence, the generated mesh with authored appearance, and the same geometry colored by rigid part with joint markers.
Method
Input
Cache read
Output
Cost ($)
Mini Workflow–Sol
513.4
402.3
18.9
0.6878
Mini Workflow–Astra
128.3
101.0
5.7
0.4619
USDCraft–Sol
1,164.1
1,000.3
16.5
0.9704
USDCraft–Astra
560.0
508.5
3.8
0.8510
USDCraft–Claude
6,493.0
6,313.2
101.1
4.5115
USDCraft–Gemini
4,916.8
4,250.4
80.5
0.8643
Appendix
Table 12: Mean modeling-agent token usage (thousands) and estimated cost (USD) for the methods in Table 3 . Cache reads are included in input.
Figure 9: Asset preparation for robot manipulation: reference images, scanned geometry, and USDCraft reconstructions. Scan and reconstruction share the same camera and scale within each row.
Figure 10: Simulation and real-world execution of the three manipulation tasks. Each row shows four chronological video frames; timestamps refer to its own recording.
Figure 11: Fine-lattice reconstruction: the shopping cart retains its overall form but differs in local wire geometry. The overlay compares source and reconstruction in their shared coordinate frame.
Figure 12: Generated assets with flexible members, shown with native materials and part colors. The umbrella’s kinematic display uses annotated attachments.
Object
Object-specific request
Baby grand piano
Create a baby grand piano with an opening lid and keyboard cover. Include the curved case on three rigid legs, a large top lid hinged along the straight long side, a separate hinged keyboard fallboard and a hinged lid-prop stick mounted to the case. Model a complete recognizable keyboard as fixed geometry, a recessed soundboard and simplified strings. The lid, fallboard and prop are independent limited revolute joints, without a closed-loop support constraint. Start with both covers closed and prop stowed. Approximate length 1.5 m.
Compact excavator
Create a compact excavator with a three-stage digging arm. Include a fixed track-shaped undercarriage, a cab and upper platform on one vertical slew joint, a boom on a horizontal shoulder hinge, a stick on a parallel elbow hinge and a hollow toothed bucket on a wrist hinge. Add one side-hinged cab door. Author simplified hydraulic-cylinder appearances without redundant closed-loop constraints. Use limited boom, stick and bucket travel with meaningful folded and reaching poses. Tracks remain rigid decorative assemblies. Approximate chassis length 2.5 m. Start with door closed and arm in a compact collision-free resting pose.
Desktop 3D printer
Create an enclosed desktop Cartesian 3D printer. Include a rigid cubic frame, one left-hinged transparent front door, fixed transparent side panels, a print head sliding along an X carriage nested on a vertically sliding Z gantry, and a print bed sliding front to back on Y rails. Provide three independent prismatic axes plus the door hinge. Represent belts and wiring as fixed visual details rather than articulated links. Start with door closed, bed centered and nozzle safely above the bed. Approximate outer width 0.5 m.
Appendix
Table 13: Representative text prompts for the generation gallery; append the shared instructions to each row.
Department of Mechanical Engineering, Seoul National University, Seoul, Korea · Department of Robotics and Mechatronics Engineering, DGIST, Daegu, Korea