URDF-Anything+: End-to-End Generation for Simulation-Ready Articulated Assets
Organizations: Peking University · Visual Geometry Group, University of Oxford · University of Washington
Abstract
Articulated objects are fundamental for robotics, simulation of physics, and interactive virtual environments. However, recovering them from visual observations is inherently challenging, as images provide only partial and ambiguous cues about both part geometry and their underlying kinematic structure. Existing approaches typically rely on multi-stage pipelines, retrieval from asset libraries, or explicit part segmentation. We present URDF-Anything+, an end-to-end autoregressive diffusion framework that generates simulation-ready URDF models directly from a single RGB image. Conditioned on visual observations and object geometry, URDF-Anything+ operates in a structured latent space and jointly models part geometry and articulation in a unified generation process. Specifically, the model sequentially predicts each articulated part together with its associated joint parameters, while a termination token dynamically determines the number of parts. This design enables direct generation of fully executable URDFs without external retrieval or post-processing stages. Experiments on large-scale articulated object benchmarks demonstrate that URDF-Anything+ outperforms prior methods in geometric reconstruction quality, joint parameter estimation, and physical executability, while being substantially more efficient than existing multi-stage approaches. Furthermore, the generated URDFs serve as faithful digital twins, enabling the zero-shot transfer of manipulation policies trained purely in simulation.
Figures & tables
| Method | Parts | Whole Object | Joint Accuracy | Time / Part (s) | ||||||
| IoU | F-Score | CD | IoU | F-Score | CD | Axis Err. | Origin Err. | Limit Err. | ||
| URDFormer Chen et al. (2024) | 0.202 | 0.389 | 0.354 | 0.243 | 0.413 | 0.286 | 1.071 ±0.497 | 0.504 ±0.331 | 0.882 ±0.679 | 41.88 |
| Articulate-Anything Le et al. (2025) | 0.563 | 0.452 | 0.064 | 0.627 | 0.510 | 0.053 | 0.230 ±0.505 | 0.267 ±0.357 | 1.108 ±0.676 | 380.57 |
| SINGAPO Liu et al. (2025a) | 0.382 | 0.434 | 0.165 | 0.462 | 0.481 | 0.132 | 0.670 ±0.472 | 0.404 ±0.297 | 0.817 ±0.525 | 52.30 |
| PhysX-Anything Cao et al. (2025b) | 0.784 | 0.697 | 0.076 | 0.813 | 0.715 | 0.068 | 0.191 ±0.337 | 0.103 ±0.192 | 0.532 ±0.423 | 578.38 |
| URDF-Anything+ (Ours) | 0.879 | 0.721 | 0.033 | 0.930 | 0.742 | 0.009 | 0.129 ±0.404 | 0.062 ±0.096 | 0.225 ±0.440 | 34.70 |
| Objectives | Variation | Geometry Quality | Joint Accuracy | |||||||
| Parts | Whole Object | Axis Err. | Origin Err. | Limit | ||||||
| IoU | F-Score | CD | IoU | F-Score | CD | |||||
| Input Modality | Image Only | 0.627 | 0.560 | 0.106 | 0.682 | 0.634 | 0.098 | 0.349 | 0.197 | 0.540 |
| Image + 3D Guid.(Ours) | 0.823 | 0.740 | 0.028 | 0.841 | 0.766 | 0.009 | 0.182 | 0.091 | 0.281 | |
| Link Ordering | Random Order | 0.630 | 0.686 | 0.145 | 0.710 | 0.704 | 0.072 | 0.203 | 0.168 | 0.442 |
| Order by Joint XYZ (Ours) | 0.805 | 0.742 | 0.096 | 0.816 | 0.753 | 0.034 | 0.120 | 0.102 | 0.323 | |
| Comparison of deployment paradigms | Comparison of URDF generation methods (under Real-Follow-Sim) | Comparison of training data sources | ||||||
| Method | Close Laptop | Close Drawer | Method | Close Laptop | Close Drawer | Method | Close Laptop | Close Drawer |
| Sim-to-Real | 80 | 75 | Articulate-Anything | 75 | 55 | Real-Only | 85 | 75 |
| Real-Follow-Sim (Ours) | 100 | 90 | URDF-Anything+ (Ours) | 100 | 90 | Simulation-Only (Ours) | 100 | 90 |
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.