I2CD: Direct Image-to-Convex Decomposition for Simulation-Ready Collision Geometry

Qian Wang; Liam Merz Hoffmeister; Brian Scassellati; Daniel Rakita

cs.ROOct 2, 2026

I2CD: Direct Image-to-Convex Decomposition for Simulation-Ready Collision Geometry

Authors: Qian Wang, Liam Merz Hoffmeister, Brian Scassellati, Daniel Rakita

Organizations: Department of Computer Science, Yale University

Abstract

Physics simulators and motion planners require convex collision geometry, yet image-to-3D generative models output dense, frequently non-manifold visual meshes. Bridging the two today takes a slow, brittle reconstruct-then-decompose pipeline of repair, decimation, and approximate convex decomposition. We present I2CD, which predicts a convex decomposition directly from a single RGB image. Rather than train a new image-to-3D model, I2CD freezes the pretrained Hunyuan3D-2 image-conditioned diffusion transformer and shape decoder and trains only a lightweight cross-attention head (38M parameters, under ten GPU-hours) whose learned "convex-slot" tokens emit the halfplane parameters of $K$ convex polytopes. The output is compact, convex by construction, and loads into physics engines without any post-processing, in ${\sim}0.5$ s per image. On $227$ held-out OmniObject3D and Google Scanned Objects instances, I2CD attains the highest volumetric IoU among eight reconstruct-then-decompose pipelines while running $6$ - $37\times$ faster end-to-end. In a cross-simulator study in MuJoCo, PyBullet, Genesis, and Isaac Sim, every engine uses I2CD geometry as delivered, whereas raw generated meshes "load" everywhere but are silently replaced by a different collision shape in most cases or need seconds to minutes of per-object preprocessing. On a physical xArm7, I2CD produces planner-ready geometry for a $20$ -object cluttered scene in $11$ s versus $328$ s for the strongest baseline, at comparable pick-and-place execution success ( $85$ vs. $90$ of $100$ trials).

Figures & tables

Fig. 1: From a single RGB image, I2CD predicts a union of convex polytopes in ${\sim}0.5$ s that physics simulators consume directly, with no mesh extraction, repair, or post-hoc decomposition.

Fig. 2: Pipeline. The frozen Hunyuan3D-2 DiT samples a shape latent from the image and the frozen ShapeVAE decoder $\mathcal{D}$ expands it into per-token shape features. Our trainable convex head uses $K$ learned convex-slot tokens that cross-attend to these features and emit halfplane parameters $\{\mathbf{n}_{k,h},d_{k,h},\mathbf{t}_{k},\beta_{k}\}$ for $K$ convex polytopes $\mathcal{C}_{k}$ ; their union is the predicted occupancy, from which per-convex meshes are extracted for simulator use. During training (top), the latent comes from the frozen encoder $\mathcal{E}$ instead.

Method	IoU (%) $\uparrow$	Omni	GSO	F (%) $\uparrow$	CD (mm) $\downarrow$	$t_{\mathrm{total}}$ (s) $\downarrow$
HY3D + CoACD	$62.8\pm 19.0$	$62.9$	$62.7^{\ast}$	$16.4\pm 12.9$	$6.67\pm 7.07$	$7.82$
HY3D + V-HACD	$61.0\pm 18.6$	$61.1$	$60.8^{\ast}$	$16.1\pm 15.3$	$6.76\pm 6.77$	$7.50$
HY3D + CvxNet	$58.7\pm 15.9$	$56.1$	$62.0^{\ast}$	$13.2\pm 10.9$	$8.17\pm 11.63$	$3.05$
HY3D + BSP-Net	$59.7\pm 16.4$	$59.7$	$59.7^{\ast}$	$14.4\pm 11.3$	$8.73\pm 9.46$	$3.67$
SAM3D + CoACD	$61.5\pm 20.2$	$61.3$	$62.0$	$\mathbf{19.2\pm 14.7}$	$\mathbf{5.87\pm 6.82}$	$19.04$
SAM3D + V-HACD	$62.2\pm 18.3$	$61.0$	$63.8^{\ast}$	$17.1\pm 14.8$	$6.12\pm 6.78$	$13.76$

TABLE I: Single-image convex decomposition on OmniObject3D

+

GSO (

n{=}227

), mean

\pm

std over objects, with IoU also split by dataset (OmniObject3D uses a hull-proxy reference, GSO the exact mesh).

t_{\mathrm{total}}

for baselines is mesh generation (

1.56

s HY3D,

13.30

s SAM3D) plus decomposition. ∗ GSO means for baselines are recovered from the pooled and OmniObject3D means (exact up to rounding).

Fig. 3: Qualitative comparison. Each color is one convex part; the time under each column is the rounded mean end-to-end runtime. I2CD produces compact, volumetrically faithful decompositions in one pass, whereas reconstruct-then-decompose pipelines inherit and amplify reconstruction artifacts into fragmented or misaligned parts.

Engine	Geometry	Preserved	Load	Penetr.	At rest
		(%)	(s)	(mm)	(%)
MuJoCo	ours	99.2	0.05	0.22	99.2
MuJoCo	raw, default ( $\to$ hull)	40.2	2.77	0.26	83.5
PyBullet	ours	99.2	0.03	0.01	100.0
	raw, default ( $\to$ hull)	40.2	2.48	0.01	78.7
	raw, concave mesh	100 †	0.01	0.01	100.0

TABLE II: Simulation-readiness beyond loading (

127

OmniObject3D objects, four engines). “Raw” is the HY3D mesh; “default” is the engine’s untouched mesh handling. “Preserved”: round-trip IoU

\geq 0.9

between engine-effective and input geometry (not fidelity to the object). All rows load

100\%

of objects except ‡ , where PhysX cooking crashed on

1

(hull) and

6

(SDF) objects, excluded from that row. Genesis’s SDF collider is omitted: in a

20

-object pilot it exceeded

180

GB RAM or

15

min on

13

objects and took

1

–

6

min on the rest. † exact shape, but dynamic concave meshes register zero object–object contacts in PyBullet.

Fig. 4: Real cluttered scene. From one RGB-D observation (left), SAM3D reconstructs dense per-object meshes; auto-hull collapses each to one convex hull; SAM3D+CoACD decomposes post hoc; I2CD predicts compact convex geometry directly. Times are wall-clock to a planner-ready $20$ -object scene.

Fig. 5: Physical execution. The xArm7 executing a planned pick in the $20$ -object scene of Fig. 4 . Every trial in Table III is run on this robot.

Method	$t_{\mathrm{geom}}$ (s) $\downarrow$	$t_{\mathrm{load}}$ (s) $\downarrow$	Success $\uparrow$
SAM3D + auto-hull	$198$	$0.9$	$\;\;5/100$
SAM3D + CoACD	$328$	$1.7$	$\mathbf{90/100}$
I2CD (ours)	$\mathbf{11}$	$\mathbf{0.9}$	$85/100$

TABLE III: Physical pick-and-place in a

20

-object scene.

t_{\mathrm{geom}}

: time to planner-ready geometry for all objects;

t_{\mathrm{load}}

: one-time MuJoCo load. Success over

20

targets

\times

5

seeds, executed on the xArm7.

Loss	IoU (%) $\uparrow$	F (%) $\uparrow$	CD (mm) $\downarrow$
BCE $+$ NCE (ours)	$\mathbf{63.8}$	$\mathbf{12.5}$	$\mathbf{5.98}$
Class-balanced BCE only	$62.2$	$11.4$	$6.49$
Unbalanced BCE	$54.7$	$9.7$	$8.89$
MSE	failed: $127/127$ empty

TABLE IV: Training objective on OmniObject3D (

n{=}127

); each row is that objective’s validation-best checkpoint (ours is the Table I model; its

63.8

vs.

63.1

there is DiT sampling noise).

(a) Parts $K$ vs. halfplanes $H$				(b) Head depth
Cell: IoU / F / CD				(cross-attn, self-attn)
$K\backslash H$	$6$	$12$	$24$	Head (layers)	IoU
$6$	61.4 / 11.9 / 6.66	65.5 / 13.1 / 6.52	65.3 / 12.8 / 6.76	(1, 0) minimal	61.4
$12$	61.3 / 12.4 / 6.79	63.1 / 13.1 / 6.23	64.4 / 12.8 / 6.00
$24$	58.6 / 12.4 / 6.49	60.2 / 12.0 / 6.59	61.6 / 12.6 / 6.11	(2, 1) default	62.0

TABLE V: Capacity on OmniObject3D at a matched

100

-epoch budget.

Fig. 6: Qualitative ablation of the objective. I2CD yields compact, coherent parts; class-balanced BCE alone tiles the interior with thin slabs; unbalanced BCE (uBCE) under-fills the shape.

Learning Convex Decomposition via Feature Fields
Mar 10, 2026Yuezhi Yang, Qixing Huang, Mikaela Angelina Uy +1
NVIDIA · The University of Texas at Austin
SimuScene: Simulation-Ready Compositional 3D Scene Reconstruction from a Single Image
Jun 2, 2026Inhee Lee, Sangwon Baik, Sungjoo Kim +33D Reconstruction Robotics Simulation
Seoul National University
Tetris3D: 3D Scene Generation With Objects That Fit Together
Oct 7, 2026Jaeyeong Kim, Jinhyuk Jang, Jongmin Lee +23D Scene Generation Image-to-3D Scene Generation
KAIST AI

Mar 10, 2026J/K move · Enter open · S save

Learning Convex Decomposition via Feature Fields

Yuezhi Yang, Qixing Huang, Mikaela Angelina Uy, Nicholas Sharp

NVIDIA · The University of Texas at Austin

Open paper AskPDF arXiv

This work proposes a new formulation to the long-standing problem of convex decomposition through learning feature fields, enabling the first feed-forward model for open-world convex decomposition. Our method produces high-quality decompositions of 3D shapes into a union of convex bodies, which are essential to accelerate collision detection in physical simulation, amongst many other applications. The key insight is to adopt a feature learning approach and learn a continuous feature field that can later be clustered to yield a good convex decomposition via our self-supervised, purely-geometric objective derived from the classical definition of convexity. Our formulation can be used for single shape optimization, but more importantly, feature prediction unlocks scalable, self-supervised learning on large datasets resulting in the first learned open-world model for convex decomposition. Experiments show that our decompositions are higher-quality than alternatives and generalize across open-world objects as well as across representations to meshes, CAD models, and even Gaussian splats. https://research.nvidia.com/labs/sil/projects/learning-convex-decomp/

Abstract

Explore similar work

Learning Convex Decomposition via Feature Fields

SimuScene: Simulation-Ready Compositional 3D Scene Reconstruction from a Single Image

Tetris3D: 3D Scene Generation With Objects That Fit Together