Converting static 3D meshes into interactable articulated assets is crucial for embodied AI and robotic simulation. However, existing zero-shot pipelines struggle with complex assets due to a critical lack of physical grounding. Specifically, ungrounded Vision-Language Models (VLMs) frequently suffer from kinematic hallucinations, while unconstrained joint estimation inevitably leads to catastrophic mesh inter-penetration during physical simulation. To bridge this gap, we propose MotionAnymesh, an automated zero-shot framework that seamlessly transforms unstructured static meshes into simulation-ready digital twins. Our method features a kinematic-aware part segmentation module that grounds VLM reasoning with explicit SP4D physical priors, effectively eradicating kinematic hallucinations. Furthermore, we introduce a geometry-physics joint estimation pipeline that combines robust type-aware initialization with physics-constrained trajectory optimization to rigorously guarantee collision-free articulation. Extensive experiments demonstrate that MotionAnymesh significantly outperforms state-of-the-art baselines in both geometric precision and dynamic physical executability, providing highly reliable assets for downstream applications.
Figures & tables
Figure 1: MotionAnymesh: Physics-Grounded Articulation of Static 3D Assets. While current SOTAs (e.g., Articulate-AnyMesh) rely on ungrounded semantics and often suffer from severe inter-penetration, MotionAnymesh ensures physically plausible articulation. Through kinematic-aware perception and physics-constrained optimization , our zero-shot framework transforms static meshes into collision-free, simulation-ready URDF digital twins for direct deployment in embodied AI tasks.
Figure 2: Overview of the MotionAnymesh framework. Our pipeline consists of three integrated stages: (1) Kinematic-Aware Part Segmentation, which extracts 3D-native primitives and clusters them using SP4D kinematic priors and VLM reasoning; (2) Joint Estimation and Optimization, featuring type-aware geometric initialization and physics-constrained trajectory refinement to ensure collision-free articulation ; and (3) Simulation-Ready Asset Finalization, providing simulation-ready URDF models with preserved textures.
Figure 3: Qualitative comparison of articulated object modeling. Compared with Articulate-Anything and Articulate-AnyMesh, MotionAnymesh produces cleaner geometric boundaries and more accurate kinematic structures, especially for complex mechanical objects like the robot arm and multi-part lamps.
Method
Part Segmentation
Joint Estimation
Simulation
mIoU ↑
Count Acc ↑
Type Err ↓
Axis Err ↓
Pivot Err ↓
Executability ↑
PARIS [ 25 ]
0.17
0.23
0.67
1.56
1.14
11%
URDFormer [ 3 ]
0.21
0.33
0.72
1.31
1.53
21%
SINGAPO [ 24 ]
0.52
0.66
0.24
0.73
0.57
43%
Articulate-Anything [ 20 ]
0.47
0.61
0.21
0.86
0.64
46%
Articulate-AnyMesh [ 35 ]
0.59
0.74
0.35
0.64
0.44
35%
Table 1: Quantitative comparison against state-of-the-art baselines across diverse 3D assets. We evaluate our method against others on three core dimensions: Part Segmentation, Joint Parameter Estimation, and Physical Executability. ↑ indicates higher is better, and ↓ indicates lower is better. Best results are highlighted in bold .
Clustering Strategy
mIoU ↑
Count Acc ↑
Pure VLM Semantics (w/o SP4D)
0.68
0.81
SP4D-Guided Clustering (Ours)
0.86
0.92
Table 2: Ablation on SP4D kinematic priors. Removing SP4D guidance ("w/o SP4D") causes the VLM to rely solely on semantic logic, leading to severe kinematic hallucinations and over-segmentation.
Joint Estimation Method
Axis Err ↓
Pivot Err ↓
Executability ↑
Kinematic Initialization (w/o Opt.)
0.23
0.27
65%
Physics-Constrained Opt. (Ours)
0.12
0.10
87%
Table 3: Impact of physics-constrained optimization. While geometric initialization provides a reasonable starting point, our non-linear optimization is essential to eliminate micro-misalignments and guarantee high physical executability in simulators.
Figure 4: Versatile articulation across diverse domains. MotionAnymesh successfully processes handcrafted assets and AI-generated surface meshes, transforming unstructured geometry into interactive digital twins without manual intervention.
Figure 5: Real-to-Sim-to-Real application. (Left) A static mesh is reconstructed from a single image. (Middle) MotionAnymesh generates a collision-free URDF for policy learning in simulation. (Right) The learned manipulation policy is deployed onto a physical robot, validating the sim-to-real fidelity of our estimated parameters.
High-quality articulated 3D assets are indispensable for embodied AI and physical simulation, yet 3D generation still focuses on static meshes, leaving a gap in "sim-ready" interactive objects. Most recent articulated object creation methods rely on multi-stage pipelines that accumulate errors across decoupled modules. Alternatively, unified MLLMs offer a single-stage path to joint static asset understanding and sim-ready asset generation. However dense voxel-based 3D tokenization yields long 3D token sequences and high memory overhead, limiting scalability to complex articulated objects. To address this, we propose SIMART, a unified MLLM framework that jointly performs part-level decomposition and kinematic prediction. By introducing a Sparse 3D VQ-VAE, SIMART reduces token counts by 70% vs. dense voxel tokens, enabling high-fidelity multi-part assemblies. SIMART achieves state-of-the-art performance on PartNet-Mobility and in-the-wild AIGC datasets, and enables physics-based robotic simulation.
Creating interactive digital environments for gaming, robotics, and simulation relies on articulated 3D objects whose functionality emerges from their part geometry and kinematic structure. However, existing approaches remain fundamentally limited: optimization-based reconstruction methods require slow, per-object joint fitting and typically handle only simple, single-joint objects, while retrieval-based methods assemble parts from a fixed library, leading to repetitive geometry and poor generalization. To address these challenges, we introduce ArtLLM, a novel framework for generating high-quality articulated assets directly from complete 3D meshes. At its core is a 3D multimodal large language model trained on a large-scale articulation dataset curated from both existing articulation datasets and procedurally generated objects. Unlike prior work, ArtLLM autoregressively predicts a variable number of parts and joints, inferring their kinematic structure in a unified manner from the object's point cloud. This articulation-aware layout then conditions a 3D generative model to synthesize high-fidelity part geometries. Experiments on the PartNet-Mobility dataset show that ArtLLM significantly outperforms state-of-the-art methods in both part layout accuracy and joint prediction, while generalizing robustly to real-world objects. Finally, we demonstrate its utility in constructing digital twins, highlighting its potential for scalable robot learning.
Physically grounded 3D assets are increasingly important for embodied AI and robotic simulation. However, most existing 3D assets lack unified physical semantics, including articulation semantics and intrinsic physical properties, required for realistic interaction. Current approaches either treat these semantics independently or rely on canonicalized object structures, limiting robustness across heterogeneous 3D assets. We present UniPhys, a scalable framework for automatically transforming raw 3D assets into simulation-ready assets with unified physical semantics. Based on UniPhys, we construct UniPhys-40K, a large-scale physically grounded dataset, together with UniPhys-Bench, a carefully verified benchmark for unified physical grounding evaluation. We further introduce UniPhysGen, a unified physical grounding model that jointly reasons over articulation semantics and intrinsic physical properties. UniPhysGen incorporates geometry-robust articulation grounding to mitigate geometric shortcut bias under heterogeneous part decompositions. Extensive experiments demonstrate state-of-the-art performance across articulation grounding and intrinsic physical property estimation tasks, while the resulting assets can be directly deployed in robotic simulation environments for realistic physical interaction. Our code and dataset will be available at https://github.com/breezexian/UniPhysGen.
Xian Li, Rong Wei, Lujie Yang +6
Zhejiang University · Manycore Tech Inc. · University of Electronic Science and Technology of China