Seeing Both the Forest and the Trees: Reusing Holistic 3D Priors for Part-Decomposed Generation

Yiqun Zhao; Binbin Huang; Haobin Duan; Zibo Zhao; Shenghua Gao

cs.CVOct 15, 2026

Seeing Both the Forest and the Trees: Reusing Holistic 3D Priors for Part-Decomposed Generation

Authors: Yiqun Zhao, Binbin Huang, Haobin Duan, Zibo Zhao, Shenghua Gao

Organizations: The University of Hong Kong · Transengram · Tencent · Shenzhen Loop Area Institute

Abstract

Generating 3D assets with explicit parts requires two properties at once: geometric detail inside each part and structural coherence across the whole object. Recent image-conditioned 3D generation models adopt a structured 3D representation that anchors tokens to explicit spatial positions, capturing both properties, but their output remains a single fused mesh. We ask how a part-decomposed generation model can inherit both properties of these holistic priors, and answer with STRUCTPARTS, a framework whose core technical contribution is a dual-frame coordinate interleaving mechanism: each shape latent is interpreted under both a part-local and an object-global reference frame, and processed through a global-local inter-leaved transformer that simultaneously reuses the prior's geometric detail at the part scale and its structural coherence at the whole-object scale. As its structural input, STRUCTPARTS uses a segmented mesh scaffold, a coarse mesh paired with a face-level segmentation mask kept entirely within the prior's native 3D representation. With only lightweight finetuning, both intra-part detail and inter-part coherence emerge from the same pretrained weights. On PartObjaverse-Tiny, STRUCTPARTS matches or surpasses prior part-decomposed models at a fraction of their training cost, while naturally supporting part-mesh refinement and part-level editing.

Figures & tables

Figure 1: Diverse decomposed 3D objects generated by our approach using images from recent open-source image generation models Z-Image Team, Alibaba Group (2025) ; Black Forest Labs (2025) . Among them, the guns, train, crab, and motorcycle are specifically displayed in exploded views

Figure 2: Overview of StructParts ’s pipeline. Given a single image, our method first constructs a 3D segmented scaffold using the structure generation model followed by a segmentation model. We then sample points from each part surface to construct the voxel indices in both the per-part normalized space and the shared global-shape space. These voxel indices serve as position embeddings to guide the diffusion model’s sampling process. The segmented scaffold and part meshes depicted here are dataset-provided ground truth (GT), used to illustrate the part-wise data flow.

Figure 3: Layout-conditioned part decomposition. Comparison among methods that accept an explicit layout (HoloPart, X-Part, FullPart, OmniPart): each row shows assembled shapes for part coherency and exploded views for per-part geometry. Baselines drop fine details, leave incomplete shapes, or introduce floaters; StructParts preserves both by reusing the holistic prior.

Table 4

Figure 4: Image-to-decomposed-parts comparison. Top two rows: in-the-wild prompts generated by flux-2 Black Forest Labs (2025) ; bottom row: an in-distribution test sample. For each input we show the assembled shape and a zoom-in view. PartPacker / PartCrafter struggle to produce semantically meaningful decompositions; FullPart and OmniPart suffer from boundary collisions;

Method	Part-level			Shape-level
Method	CD $\downarrow$	F-0.05 $\uparrow$	F-0.10 $\uparrow$	CD $\downarrow$	F-0.05 $\uparrow$	F-0.10 $\uparrow$
StructParts	0.107	0.67	0.77	0.008	0.93	0.97
GRoPE + part embed.	0.125	0.63	0.74	0.012	0.83	0.91
GRoPE only	0.143	0.63	0.75	0.019	0.70	0.90
LRoPE only	0.331	0.20	0.36	0.054	0.51	0.73

Table 3: Ablation Study of dual-frame positional embedding strategies. Quantitative results evaluated at both part-level and shape-level.

Figure 5: Ablation on positional-embedding strategies. Three variants compared against StructParts ’ dual frame: LRoPE only loses global structure (parts treated identically in their own normalized frames); GRoPE only recovers global layout but degrades small or off-center parts under unnormalized coordinates; GRoPE + part embed. (as in prior work) still misses fine geometric detail. Only the dual-frame design recovers both global structure and per-part detail.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Quantity	StructParts	X-Part (matched)
GPUs	8 H200	8 H200
Total allocation	768 GPU-hours	768 GPU-hours
Trainable parameters	3.051 B	3.315 B
TFLOPs/object	92.7	137.4
Allocated GPU memory	82.71 GB	115 GB

Appendix

Table 4: Training allocation and computational cost. Both methods use 512 tokens per part and the same GPU-hour allocation. TFLOPs/object include forward and backward computation under the same PyTorch profiler. Optimizer steps and processed-part counts are reported with checkpoint quality in Tab. 5 .

Run	Steps	Processed parts	Mean CD $\downarrow$	Median CD $\downarrow$	[email protected] $\uparrow$	[email protected] $\uparrow$
StructParts	20K	2.56M	0.129	0.013	0.64	0.76
X-Part (matched)	28K	3.59M	0.169	0.028	0.55	0.69
StructParts	40K	5.12M	0.107	0.014	0.67	0.77

Appendix

Table 5: Checkpoint quality and training-schedule accounting. Part-level evaluation on PartObjaverse-Tiny. The 40K StructParts and 28K X-Part runs use the same GPU-hour allocation; the 20K StructParts checkpoint is included to compare quality after fewer optimizer steps and processed parts.

Method / condition	Mean CD $\downarrow$	Median CD $\downarrow$	[email protected] $\uparrow$	[email protected] $\uparrow$
Input Hunyuan scaffold	0.049	0.038	0.63	0.84
StructParts (clean)	0.045	0.037	0.63	0.83
Geometry noise 1%	0.046	0.037	0.63	0.83
Geometry noise 2%	0.047	0.039	0.61	0.80
Geometry noise 5%	0.052	0.041	0.51	0.73
Label corruption 5%	0.045	0.037	0.62	0.83

Appendix

Table 6: Geometry preservation and scaffold perturbations. The input scaffold and generated outputs are evaluated against the same GT shapes with identical normalization and surface sampling. Noise is normal-direction vertex displacement as a percentage of the bounding-box diagonal; label corruption reassigns boundary voxels. All other inference settings are fixed.

Method / condition	Mean CD $\downarrow$	Median CD $\downarrow$	[email protected] $\uparrow$	[email protected] $\uparrow$
Clean	0.107	0.014	0.67	0.77
Label corruption 5%	0.108	0.014	0.67	0.77
Label corruption 10%	0.113	0.019	0.61	0.73
Label corruption 20%	0.117	0.038	0.58	0.71

Appendix

Table 7: Part-level sensitivity to segmentation errors. We corrupt GT boundary labels to retain a known correspondence between each generated and GT part. These results are distinct from the assembled-shape experiment in Tab. 6 .

Method / condition	Mean CD $\downarrow$	Median CD $\downarrow$	[email protected] $\uparrow$	[email protected] $\uparrow$
StructParts	0.021	0.012	0.68	0.79
X-Part (released)	0.023	0.013	0.68	0.80
X-Part (matched)	0.045	0.037	0.58	0.69
HoloPart	0.026	0.018	0.62	0.70
FullPart	0.028	0.021	0.66	0.71
OmniPart	0.095	0.057	0.49	0.49

Appendix

Table 8: Cross-dataset evaluation on ABO. A fixed 200-object subset is evaluated with identical inputs and the same whole-shape CD/F-score protocol for all methods. Released and matched-budget X-Part checkpoints are distinguished.

Category	FullPart	HoloPart	OmniPart	X-Part	StructParts
Animals	0.395/0.317	0.120/0.031	0.182/0.205	0.040 /0.013	0.095/ 0.009
Buildings & Outdoor	0.483/0.389	0.166/0.068	0.403/0.386	0.100/0.027	0.068/0.013
Daily-used	0.259/0.122	0.199/0.043	0.229/0.191	0.119/0.024	0.066/0.011
Electronics	0.453/0.256	0.150/0.061	0.165/0.146	0.152/0.029	0.137/0.022
Human-shaped	0.332/0.273	0.081/0.028	0.177/0.148	0.049/0.011	0.041/0.009
Transportation	0.377/0.238	0.181/0.065	0.332/0.211	0.208/0.044	0.144/0.022

Appendix

Table 9: Category-wise part-level CD on PartObjaverse-Tiny. Each cell reports mean / median CD, using the six largest evaluation categories. X-Part denotes its released checkpoint.

Category	Objects	Paired parts	StructParts	X-Part
Human-Shape	29	423	0.74/0.83	0.74/0.85
Animals	23	344	0.74 /0.82	0.72/0.84
Daily-Use	25	335	0.71/0.81	0.62/0.74
Buildings & Outdoor	25	370	0.67/0.79	0.61/0.72
Transportation	38	615	0.59/0.70	0.52/0.64
Electronics	34	523	0.63/0.74	0.58/0.71

Appendix

Table 10: Category counts and paired-part F-scores. Object counts describe the evaluation set; paired parts have complete matched predictions. F-score pairs are [email protected] / [email protected]. X-Part denotes its released checkpoint.

Metric difference	Estimate	95% paired bootstrap interval
Mean CD	$-0.0089$	$[-0.0198,\,+0.0009]$
[email protected]	$+0.04$	$[+0.02,\,+0.06]$
[email protected]	$+0.03$	$[-0.00,\,+0.05]$

Appendix

Table 11: Paired bootstrap differences against released X-Part. Objects with complete paired predictions are resampled 10,000 times, retaining all paired parts within each sampled object. Differences are StructParts minus X-Part. F-score estimates and interval bounds use two decimals; a negative bound that rounds to zero retains its sign. Statistical conclusions use the unrounded intervals.

3D Shape Generation

3D shape generation creates and edits 3D objects from learned geometric priors and text, images, sketches, or structural constraints.

See all 104 papers

Seg3DParts: Segmentation-Grounded Controllable Part-Level 3D Generation
Sep 29, 2026Jiantao Lin, Meixi Chen, Yingjie Xu +53D Mesh Generation Part-Level 3D Generation
The Hong Kong University of Science and Technology (Guangzhou) · Knowin AI · The Hong Kong University of Science and Technology
CubePart: An Open-Vocabulary Part-Controllable 3D Generator
May 27, 2026Yiheng Zhu, Kangle Deng, Jean-Philippe Fauconnier +93D Mesh Generation Part-Level 3D Generation
Roblox, USA · Carnegie Mellon University, USA · Stanford University, USA
SCULPT: Subtractive Composition for 3D Part Generation
Aug 13, 2026Sikuang Li, Chen Yang, Jiemin Fang +5Part-Level 3D Generation 3D Asset Generation
Shanghai Jiao Tong University · This work was done during an internship at Huawei. · Huawei

Sep 29, 2026J/K move · Enter open · S save

Seg3DParts: Segmentation-Grounded Controllable Part-Level 3D Generation

Jiantao Lin, Meixi Chen, Yingjie Xu, Chenbo Fu+4

The Hong Kong University of Science and Technology (Guangzhou) · Knowin AI · The Hong Kong University of Science and Technology

3D Mesh Generation Part-Level 3D Generation 3D Asset Generation 3D Shape Generation

Open paper AskPDF arXiv

Part-level 3D assets are essential for editing, reassembly, and interaction, yet recovering such structure from a single image remains challenging due to occlusion, ambiguous boundaries, and the need for coherent multi-part reasoning. Existing approaches struggle to achieve both controllable part-level generation and coherent multi-part structure, as part identity and spatial allocation are typically inferred implicitly. We present Seg3DParts, a segmentation-grounded framework for controllable part-level 3D generation from a single image. By treating segmentation as an explicit grounding signal, our method defines part identity during generation, enabling each component to be anchored to a corresponding image region. To ensure coherent assemblies, we introduce structured cross-part interaction that allows components to exchange global context throughout the generative process. As a result, Seg3DParts directly generates well-aligned part meshes in a shared canonical space without post-hoc alignment, supporting flexible and controllable decomposition. We further introduce PartObjectNet, a large-scale dataset with over 200K objects and 1M annotated parts. Experiments demonstrate that Seg3DParts achieves superior geometry quality, cross-part coherence, and part-level controllability over existing methods.

Abstract

3D Shape Generation

Explore similar work

Seg3DParts: Segmentation-Grounded Controllable Part-Level 3D Generation

CubePart: An Open-Vocabulary Part-Controllable 3D Generator

SCULPT: Subtractive Composition for 3D Part Generation