Towards Physically Grounded Articulated 3D Assets: A Multi-Agent Pipeline and Benchmark

Xudong Cai; Ruijie Zhu; Haoliang Shang; Ruining Li; Marc Pollefeys; Shangzhe Wu; Andrea Vedaldi; Chuanxia Zheng

cs.CVOct 9, 2026

Towards Physically Grounded Articulated 3D Assets: A Multi-Agent Pipeline and Benchmark

Authors: Xudong Cai, Ruijie Zhu, Haoliang Shang, Ruining Li, Marc Pollefeys, Shangzhe Wu, Andrea Vedaldi, Chuanxia Zheng

Organizations: PVG, Nanyang Technological University · ETH Zurich · VGG, University of Oxford · Microsoft · University of Cambridge

Abstract

In this paper, we study physically grounded articulated 3D asset generation---moving the goal from reproducing how objects look to capturing how they move and function. Inspired by the literature on procedural and agentic generation, we present Artisan, a system that extends the latest method Articraft by introducing a multi-agent pipeline with physical simulation feedback, enabling comprehensive attributes beyond geometry, appearance, and physically plausible articulation, such as collision-free motion, stability under gravity, and accurate mass and material properties. Furthermore, we construct a unified benchmark spanning four key aspects: spatial fidelity, physical plausibility, downstream task performance, and overall quality. The benchmark encompasses 100 reference assets from ArtVIP and LightWheel, 50 stability prompts, and 50 ABO objects with reference masses, with protocols evaluating various metrics. Through extensive evaluation of more than 10 representative models, Artisan achieves competitive spatial fidelity while attaining the lowest collision rates and highest stability rates among the evaluated methods. Together, our system and benchmark establish insights for advancing 3D generation towards physically plausible, functionally usable assets.

Figures & tables

Figure 1: Physically grounded articulated 3D objects generated by Artisan (assembled into scenes). Orange and blue annotations indicate revolute and prismatic axes, respectively. Insets list the physical attributes composition of individual parts for exampled objects. Zoom in for more details .

Figure 2: The staged pipeline of the three agents: Design Agent, Coding Agent, and Review Agent.

Figure 3: Illustrations of the six families of parameterized standard parts.

Method	Appearance					Rest-Pose Geometry			Articulated Geometry
Method	IoU $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	LPIPS $\downarrow$	CLIP $\uparrow$	oCD $\downarrow$	pCD $\downarrow$	pIoU $\uparrow$	oCD $\downarrow$	pCD $\downarrow$	pIoU $\uparrow$
Instruct-Particulate	0.6843	0.8581	17.12	0.2705	0.9167	0.0027	0.1143	0.2436	0.0010	0.1132	0.2732
PhysX-Anything	0.5259	0.8457	15.26	0.3899	0.8474	0.0098	0.2703	0.0965	0.0123	0.2989	0.0920
URDF-Anything+	0.6294	0.8598	16.11	0.3819	0.8319	0.0062	0.2913	0.1262	0.0068	0.3034	0.1236
Articulate-Anything	0.6054	0.8197	14.44	0.3893	0.8444	0.0087	0.1923	0.1223	0.0097	0.2076	0.1254
Articraft (GPT-5.5)	0.7195	0.8580	17.48	0.2781	0.9033	0.0026	0.0211	0.2795	0.0040	0.0235	0.2731

Table 1: Spatial fidelity scores on ArtVIP ( Jin et al., 2026 ) , reported in the rest and the open state. For simplicity, we only report the up-to-date results of Articraft and Codex, while the full results are available in Table 7 . Bold marks the best value and underline marks the second-best value per metric.

Figure 4: Qualitative comparison of articulated assets generated by different models. Orange rods indicate revolute axes; blue arrows indicate prismatic directions.

Method	Collision			Stability		Material			Mass
Method	Single $\downarrow$	Double $\downarrow$	Multiple $\downarrow$	Rest $\uparrow$	Open $\uparrow$	Base color $\uparrow$	Metallic $\uparrow$	Roughness $\uparrow$	MnRE $\uparrow$	ADE (kg) $\downarrow$	APE $\downarrow$
NeRF2Physics	$-$	$-$	$-$	$-$	$-$	$-$	$-$	$-$	0.601	10.78	0.509
PUGS	$-$	$-$	$-$	$-$	$-$	$-$	$-$	$-$	0.412	44.50	3.016
Instruct-Particulate	70.4	90.4	98.7	68%	42%	12.00	18.02	10.40	$-$	$-$	$-$
PhysX-Anything	58.4	77.6	81.3	74%	22%	10.06	$-$	$-$	0.302	198.07	19.292
URDF-Anything+	84.6	99.3	100.0	64%	48%	$-$	$-$	$-$	$-$	$-$	$-$

Table 2: Physical attribute scores on ArtVIP ( Jin et al., 2026 ) ( Collision , Material ), our own dataset ( Stability ), and ABO-500 ( Collins et al., 2022 ) ( Mass ). The full results are available in Table 9 . Bold marks the best value and underline marks the second-best value per metric.

-

indicates an unavailable result due to the problem scope of each method.

Panel of Figure 5 — Figure 5: Qualitative collision and stability comparisons. (a) Visualization of Collision. Red and blue denote the major intersected part pair. Artisan is clean, whereas the other baselines involve collisions between components. (b) Visualization of stability under gravity. Artisan remains upright, whereas all five baselines topple after gravity is applied.

Assets	RoboTwin	RoboCasa
Native only	80%	39%
Articraft only	60%	5%
Artisan only	63%	27%
Native + Artisan	84%	43%

Table 3: Downstream robotic evaluation of articulated assets. (a) Task success rate (%) of

\pi_{0.5}

( Black et al., 2025 ) fine-tuned with different training-asset compositions, averaged over 3 RoboTwin 2.0 ( Chen et al., 2026 ) tasks and 8 RoboCasa365 ( Nasiriany et al., 2026 ) tasks. We evaluate all policies on held-out native assets not seen during fine-tuning. (b) Percentage of all assets that pass the qualification test (Round 1) and support successful demonstration generation (Round 2).

Figure 6: Human evaluation results. Each bar stacks the rates at which a method is ranked first, second, and third; labels give each segment’s rate.

Appendix figures & tables24 assets

Supplementary material from the paper’s appendix.

Appendix

Family	Tool(s)	C	R	Purpose
Edit	write_file , replace	✓		Create or revise the asset program and geometry scripts.
Edit	write_test , edit_test		✓	Create or revise independent tests against the blueprint.
Execute	run_blender	✓		Execute a geometry script and return the mesh and its diagnostics.
	compile_model	✓		Build the asset and return structural, geometric, and test feedback.
	run_tests		✓	Run the Review Agent’s tests and return failures or errors.
Inspect	probe_model	✓	✓	Query part and joint structure or geometric measurements.

Appendix

Table 4: Tools available to the Coding (C) and Review (R) Agents. The Coding Agent edits the asset program; the Review Agent edits only its own tests.

Figure 7: Composition of the spatial-fidelity evaluation subsets. (a) The LightWheel subset contains 50 objects across 14 categories, but all for kitchen-appliance categories. (b) The ArtVIP subset contains 50 objects across 9 large categories with 37 subcategories. (c) The ArtVIP objects have 1–13 movable joints each, with fixed joints excluded.

Category	Count	Object names
Furniture	18	GULLABERG_Shoe_cabinet_with_2_compartments_grey_88x30x122cm, KILBERGET_chest_of_6_drawers_white_anchor_unlock_function_144x78cm, MUSKEN_bedside_table_ brown_45x58cm, NORDKISA_Dressing_table_bamboo_76x47cm(2), RIDSPO_Desk_oak_140X70cm, STOCKHOLM_2025_Sideboard_oak_veneer_161x42x83cm, bookcase006, cabinet_1, chest_of_drawers_1, clock007, double_door_4, lab_bench001, office_chair_3, poison_cabinet001, table_3, wardrobe_1, wardrobe_21, wardrobe_22
Kitchen appliances	9	capsule_coffee_maker_3, dishwasher_2, fridge_1, l_dishwasher_1, microwave_2, oven_2, refrigerator_6, rice_cooker_7, steamer
Mechanical equipment	7	dolly, elevator, incubator, medical_cart001, medical_monitor, microscope001, oscillator
Appliances	5	big_electric_fan_9, desk_lamp_008, electric_fan_10, television003, washing_machine_2
Cleaning tools	4	mop_10, mop_9, steping_dustbin_20, stepping_dustbin_6
Storage	3	basket_21, carton_4, clamshell _tool_box_2

Appendix

Table 5: The 50 ArtVIP objects used for evaluation. Names follow the source asset identifiers.

Figure 8: Composition of the stability dataset. Left: The 50 objects cover 9 destabilization mechanisms. Right: The objects span 10 categories defined by their primary use.

Figure 9: Composition of the mass-evaluation dataset. Left: The ground-truth masses range from

0.086

205

kg. Right: The 50 objects span 12 categories grouped by primary use.

ASIN	Category	Mass (kg)	ASIN	Category	Mass (kg)
B006ZT4VA0	tablet stand	0.086	B07TFQM3JZ	utility cart	10.478
B082L6NQZ7	speaker wall mount	0.617	B084W2S628	barstool	10.977
B075HXHKVF	jewelry box	0.717	B075X463J2	ottoman	12.247
B00WRDS8H0	laptop stand	1.102	B073GGGF7H	storage bench	17.236
B0742D9X4G	table lamp	1.361	B082L28P18	file cabinet	18.144
B07MSYK43S	projector ceiling bracket	1.383	B007IHF36E	sideboard	19.999

Appendix

Table 6: The 50 ABO objects used for mass evaluation, ordered by ground-truth mass.

Panel of Figure 10 — Figure 10: User study interface. (a) Closed- and open-state renders of the six anonymized candidates from a shared viewpoint, next to the reference images and text prompt; the right panel lists the three criteria and collects the top-three ranking. (b) The same page on the 3D interaction tab, cropped to the interaction area. Each candidate is loaded into an in-browser MuJoCo viewer, where the participant drags a part (red arrow: applied force); buttons below the viewer locate the movable parts, and the bottom bar tracks each candidate’s interactions. (c) The same moment with Color by part enabled, which draws each rigid body in its own color to reveal the part decomposition.

Method	Appearance					Rest-Pose Geometry			Articulated Geometry
Method	IoU $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	LPIPS $\downarrow$	CLIP $\uparrow$	oCD $\downarrow$	pCD $\downarrow$	pIoU $\uparrow$	oCD $\downarrow$	pCD $\downarrow$	pIoU $\uparrow$
Instruct-Particulate	0.6843	0.8581	17.12	0.2705	0.9167	0.0027	0.1143	0.2436	0.0010	0.1132	0.2732
PhysX-Anything	0.5259	0.8457	15.26	0.3899	0.8474	0.0098	0.2703	0.0965	0.0123	0.2989	0.0920
URDF-Anything+	0.6294	0.8598	16.11	0.3819	0.8319	0.0062	0.2913	0.1262	0.0068	0.3034	0.1236
Articulate-Anything	0.6054	0.8197	14.44	0.3893	0.8444	0.0087	0.1923	0.1223	0.0097	0.2076	0.1254
Articraft (GPT-5.5)	0.7195	0.8580	17.48	0.2781	0.9033	0.0026	0.0211	0.2795	0.0040	0.0235	0.2731

Appendix

Table 7: Complete spatial fidelity results on ArtVIP . Bold and underline indicate the best and second-best values per metric, including ties at the reported precision.

Method	Appearance					Rest-Pose Geometry			Articulated Geometry
Method	IoU $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	LPIPS $\downarrow$	CLIP $\uparrow$	oCD $\downarrow$	pCD $\downarrow$	pIoU $\uparrow$	oCD $\downarrow$	pCD $\downarrow$	pIoU $\uparrow$
Instruct-Particulate	0.8517	0.9250	20.80	0.1085	0.9347	0.0039	0.0845	0.2631	0.0033	0.0938	0.2890
PhysX-Anything	0.6225	0.9021	16.69	0.2158	0.8948	0.0082	0.1743	0.0986	0.0163	0.1967	0.0762
URDF-Anything+	0.7126	0.9145	16.17	0.2455	0.8368	0.0063	0.3620	0.1200	0.0087	0.3728	0.1144
Articulate-Anything	0.6120	0.9066	16.45	0.2306	0.8596	0.0103	0.1627	0.1098	0.0149	0.1928	0.0924
Articraft (GPT-5.5)	0.7891	0.9110	18.49	0.1534	0.9117	0.0019	0.0516	0.2050	0.0041	0.0635	0.1726

Appendix

Table 8: Complete spatial fidelity results on LightWheel . Bold and underline indicate the best and second-best values per metric.

Method	Collision (%)						Stability		Material (dB)			Mass
	ArtVIP			LightWheel			Rest $\uparrow$	Open $\uparrow$	Base color $\uparrow$	Metallic $\uparrow$	Roughness $\uparrow$	MnRE $\uparrow$	ADE (kg) $\downarrow$	APE $\downarrow$
	Single $\downarrow$	Double $\downarrow$	Multi. $\downarrow$	Single $\downarrow$	Double $\downarrow$	Multi. $\downarrow$	Rest $\uparrow$	Open $\uparrow$	Base color $\uparrow$	Metallic $\uparrow$	Roughness $\uparrow$	MnRE $\uparrow$	ADE (kg) $\downarrow$	APE $\downarrow$
NeRF2Physics	$-$	$-$	$-$	$-$	$-$	$-$	$-$	$-$	$-$	$-$	$-$	0.601	10.78	0.509
PUGS	$-$	$-$	$-$	$-$	$-$	$-$	$-$	$-$	$-$	$-$	$-$	0.412	44.50	3.016
Instruct-Particulate	70.4	90.4	98.7	45.1	66.2	80.6	68%	42%	12.00	18.02	10.40	$-$	$-$	$-$
PhysX-Anything	58.4	77.6	81.3	58.6	86.5	96.3	74%	22%	10.06	$-$	$-$	0.302	198.07	19.292

Appendix

Table 9: Complete physical attribute results . Collision rates are reported on ArtVIP and LightWheel; direct material PSNR uses ArtVIP, stability our stability dataset, and mass uses the ABO subset. Bold and underline mark the best and second-best values per metric;

-

denotes unavailable results.

Method	Base color $\uparrow$	Metallic $\uparrow$	Roughness $\uparrow$
Instruct-Particulate	10.07	18.14	10.54
PhysX-Anything	8.84	$-$	$-$
Articulate-Anything	7.55	$-$	$-$
Articraft (GPT-5.5)	11.15	$-$	$-$
Articraft (GPT-5.6-Sol)	11.36	$-$	$-$
Articraft (GPT-6-Astra)	13.24	$-$	$-$

Appendix

Table 10: Material transfer evaluation on ArtVIP . PSNR (dB) uses predicted materials transferred to ground-truth geometry. Bold and underline mark the best and second-best scores;

-

denotes unavailable results due to the problem scope of methods.

Variant	Appearance		Geometry			Parts	Collision (%)			Stability		Mass
Variant	PSNR $\uparrow$	CLIP $\uparrow$	oCD $\downarrow$	pCD $\downarrow$	pIoU $\uparrow$	$N_{p}$	Single $\downarrow$	Double $\downarrow$	Multi. $\downarrow$	Rest $\uparrow$	Open $\uparrow$	MnRE $\uparrow$	ADE $\downarrow$	APE $\downarrow$
w/o Design	17.84	0.9130	0.0035	0.0195	0.2788	5.82	1.0	3.5	15.1	96%	96%	0.630	9.57	0.710
w/o Review	17.95	0.9128	0.0030	0.0211	0.2951	6.50	3.1	5.8	11.8	96%	94%	0.640	8.44	0.736
w/o Physics	17.90	0.9091	0.0030	0.0199	0.2952	6.66	35.7	54.1	73.2	90%	90%	0.649	7.68	0.628
Full	17.63	0.9150	0.0022	0.0214	0.2993	6.78	1.9	4.8	16.4	98%	96%	0.670	8.25	0.566

Appendix

Table 11: Component ablations with GPT-5.6-Sol . Each variant removes the Design Agent, Review Agent, or physical validation. Appearance, geometry, collision, and mean part count are measured on ArtVIP; stability and mass use their respective datasets. Geometry scores average the rest and open states, and mass ADE is in kilograms. Bold and underline mark the best and second-best scores per metric; part count is descriptive.

Figure 11: Additional qualitative comparisons on the ArtVIP and LightWheel benchmarks.

Figure 12: Additional visualizations of stability under gravity, each shown at rest and after ten seconds in MuJoCo. A green frame marks an asset that passes the stability check and a red frame one that does not.

Figure 13: Additional collision comparisons. Red and blue highlight the most severely interpenetrating pair of parts, enlarged in the lower-right inset.

Method	Model	Avg. cost (USD)	Avg. time (min)
Ours ( Artisan )	GPT-5.6-Sol	6.03	34.7
Ours ( Artisan )	GPT-6-Astra	5.79	34.3
Ours ( Artisan )	GPT-5.5	7.35	33.9
Vanilla Codex	GPT-5.6-Sol	0.33	9.9
Vanilla Codex	GPT-6-Astra	1.07	6.3
Vanilla Codex	GPT-5.5	0.51	1.9

Appendix

Table 12: Average API cost and generation time per asset.

Figure 14: API cost and generation time breakdown of Artisan . The left three pies show API cost shares of the Design, Coding, and Review Agents; the right three show generation time shares, including the simulator.

Task	Train	Test	Artisan	Train + Artisan	Articraft
OpenMicrowave	1	1	6	7	3
OpenLaptop	9	2	9	18	4
TurnSwitch	6	2	6	12	6
Total	16	5	21	37	13

Appendix

Table 13: Asset split in each RoboTwin 2.0 task. Native assets are split into disjoint training and held-out test sets. Artisan and Articraft denote generated assets that are physically stable and can be used for downstream evaluation and policy training.

Task	Train	Artisan	Articraft
OpenMicrowave	1/1	6/6	3/6
OpenLaptop	9/9	9/9	4/9
TurnSwitch	6/6	6/6	6/6

Appendix

Table 14: Round 2 qualification test results on RoboTwin 2.0. Number of assets that pass the qualification test and are successful in generating demonstrations on the RoboTwin 2.0 tasks, for the native assets, the assets generated by Artisan , and the assets generated by Articraft.

Task	Train	Train + Artisan	Artisan	Articraft
Open Microwave	64%	78%	40%	46%
Open Laptop	98%	97%	97%	86%
Turn Switch	78%	76%	53%	49%
Average	80%	84%	63%	60%

Appendix

Table 15: Task execution success rate (%) of

\pi_{0.5}

on RoboTwin 2.0. All policies are evaluated on held-out native test assets unseen during training. Native, Ours , Native + Ours , and Articraft denote policies fine-tuned on demonstrations collected from the corresponding asset sets. Higher is better.

Task	Train	Test	Artisan	Train + Artisan	Articraft
OpenElectricKettleLid	8	9	10	18	0
OpenFridgeDrawer	5	11	1	6	0
OpenStandMixerHead	9	12	8	17	0
OpenToasterOvenDoor	6	5	3	9	1
SlideDishwasherRack	10	4	1	11	0
SlideOvenRack	3	9	2	5	0

Appendix

Table 16: Asset split in each RoboCasa365 task. Native assets are split into disjoint training and held-out test sets. Artisan and Articraft denote generated assets physically able to be used for downstream evaluation and policy training. Each training asset contributes 50 demonstration episodes.

Task	Train	Artisan	Articraft
OpenElectricKettleLid	8/10	10/10	0/10
OpenFridgeDrawer	5/10	1/10	0/10
OpenStandMixerHead	9/10	8/10	0/10
OpenToasterOvenDoor	6/10	3/10	1/10
SlideDishwasherRack	10/10	1/10	0/10
SlideOvenRack	3/10	2/10	0/10

Appendix

Table 17: Round 2 qualification test results on RoboCasa365. Number of assets that pass the qualification test and are successful in generating demonstrations on the RoboCasa365 tasks, for the native assets, the assets generated by Artisan , and the assets generated by Articraft.

Task	Train	Train + Artisan	Artisan	Articraft
OpenElectricKettleLid	11%	12%	14%	0%
OpenFridgeDrawer	18%	23%	4%	0%
OpenStandMixerHead	17%	29%	12%	0%
OpenToasterOvenDoor	30%	35%	21%	23%
SlideDishwasherRack	6%	13%	72%	0%
SlideOvenRack	87%	84%	59%	0%

Appendix

Table 18: Task execution success rate (%) of

\pi_{0.5}

on RoboCasa365. All policies are evaluated on the held-out native test assets listed in Table 16 , with 50 evaluation rollouts per test asset. Native, Native + Artisan , Artisan , and Articraft denote policies fine-tuned on demonstrations collected from the corresponding asset sets. For tasks with no qualified training assets, we assign a success rate of 0% when computing the benchmark average. Higher is better.

Explore similar work

Articraft: An Agentic System for Scalable Articulated 3D Asset Generation
May 14, 2026Matt Zhou, Ruining Li, Xiaoyang Lyu +63D Asset Generation LLM-Based Program Synthesis
University of Cambridge · University of Oxford · Nanyang Technological University
UniPhysGen: Unified Physical Grounding for Simulation-Ready 3D Assets
Jul 15, 2026Xian Li, Rong Wei, Lujie Yang +6Physical Reasoning Robotics Simulation
Zhejiang University · Manycore Tech Inc. · University of Electronic Science and Technology of China
Artiverse: A Diverse and Physically Grounded Dataset for Articulated Objects
May 23, 2026Denys Iliash, Jiayi Liu, Egor Fokin +4
Simon Fraser University · 2Canada-CIFAR AI Chair, Amii

May 14, 2026J/K move · Enter open · S save

Articraft: An Agentic System for Scalable Articulated 3D Asset Generation

Matt Zhou, Ruining Li, Xiaoyang Lyu, Zhaomou Song+5

University of Cambridge · University of Oxford · Nanyang Technological University

3D Asset Generation LLM-Based Program Synthesis Synthetic Data Generation for Robotics

Open paper AskPDF arXiv

A bottleneck in learning to understand articulated 3D objects is the lack of large and diverse datasets. In this paper, we propose to leverage large language models (LLMs) to close this gap and generate articulated assets at scale. We reduce the problem of generating an articulated 3D asset to that of writing a program that builds it. We then introduce a new agentic system, Articraft, that writes such programs automatically. We design a programmatic interface and harness to help the LLM do so effectively. The LLM writes code against a domain-specific SDK for defining parts, composing geometry, specifying joints, and writing tests to validate the resulting assets. The harness exposes a restricted workspace and interface to the LLM, validates the resulting assets, and returns structured feedback. In this way, the LLM is not distracted by details such as authoring a URDF file or managing a complex software environment. We show that this produces higher-quality assets than both state-of-the-art articulated-asset generators and general-purpose coding agents. Using Articraft, we build Articraft-10K, a curated dataset of over 10K articulated assets spanning 245 categories, and show its utility both for training models of articulated assets and in downstream applications such as robotics simulation and virtual reality.

Abstract

Explore similar work

Articraft: An Agentic System for Scalable Articulated 3D Asset Generation

UniPhysGen: Unified Physical Grounding for Simulation-Ready 3D Assets

Artiverse: A Diverse and Physically Grounded Dataset for Articulated Objects