Organizations: University of Hong Kong, Hong Kong · Tencent Technology, China · The Chinese University of Hong Kong, Hong Kong · Nanjing University, China
3D part decomposition is a central capability for editable asset creation, yet existing workflows typically segment a completed mesh as a post-processing step, making the result sensitive to geometric artifacts, texture ambiguity, and limited part-level annotations. We present OmniLatent, a unified latent modeling framework for 3D part decomposition. Instead of predicting part labels in isolation, OmniLatent learns a shared latent space that jointly encodes geometry, deformation, appearance, and binary part IDs, allowing data-rich geometric and visual priors to support data-scarce part learning. Given a single input image, the base model generates geometry, deformation, texture, and surface part IDs in one pass, producing a part-aware 3D asset without a separate segmentation stage. We further tune the pretrained model for two extensions: a point-prompt adapter that maps arbitrary 3D point prompts with binary part IDs into latent tokens for full segmentation, refinement, and part subdivision, and an amodal part completion model that completes partial surface masks into a coherent full-object decomposition through dynamic per-part token allocation, cross-attention to the given geometry latent, and alternating part-local and global attention. Extensive experiments show that OmniLatent produces semantically meaningful part segmentations with precise boundaries, supports interactive part editing, and completes partial part observations while preserving global object consistency.
Figures & tables
Figure 1. OmniLatent generates geometry, deformation, appearance, and coherent surface part IDs from a single image in a unified latent space.
Figure 2. Unified Latent Modeling and Pretraining . OmniLatent performs 3D part decomposition through unified latent modeling. Geometry, deformation, appearance, and binary part IDs are encoded into a shared OmniLatent space, then generated by an MM-DiT backbone from an input image with optional 2D mask conditions. The aligned latent space allows geometric and visual priors to support surface part segmentation, and the pretrained model can further be adapted for point-prompted control and amodal part completion. The left branch shows the cascaded VAE decoder: SDF reconstruction defines the surface, whose vertices subsequently query deformation, appearance, and Part IDs from the same latent tokens.
Figure 3. Unified part segmentation and refinement using images, 2D masks, and point prompts. We adopt an MM-DiT architecture that integrates multiple control signals: interactive regions and geometric conditions are injected via channel concatenation with the noisy latent, while other conditions are incorporated through sequence concatenation.
Figure 4. Amodal part completion. Given full geometry and partial surface masks for all parts, OmniLatent allocates tokens dynamically according to visible part size. RoPE-anchored tokens preserve observed surface evidence, while no-RoPE completion tokens represent unobserved regions. A geometry latent enters through cross-attention, and alternating part-local and global attention layers complete all parts into a consistent full-object part layout.
Figure 5. Surface part segmentation and amodal completion results. For each example, OmniLatent produces complete surface part segmentation and further completes partial surface masks into complete object-level parts, preserving global structure while recovering missing components.
Figure 6. Qualitative comparison of surface part segmentation on PartObjaverse-Tiny-wt. OmniLatent produces semantically coherent regions and cleaner boundaries by decoding part IDs from a latent space aligned with geometry and appearance.
Method
Avg. ↑
Human-Shape
Animals
Daily-Used
Buildings
Transportations
Plants
Food
Electronics
Find3D ( Ma et al., 2025b )
20.76
25.08
24.86
21.71
14.93
12.52
22.43
27.54
17.02
SAMPart3D ( Yang et al., 2024 )
50.02
48.49
58.66
49.00
39.52
42.68
58.37
54.94
48.48
PartField ( Liu et al., 2025b )
52.61
54.50
60.36
56.38
38.43
44.57
55.70
58.59
52.31
P3 -SAM ( Ma et al., 2025a )
58.10
64.10
62.04
60.74
43.66
54.83
65.36
55.73
58.37
OmniLatent (Ours)
74.18
79.36
82.68
76.98
58.94
65.04
78.73
80.72
70.98
Table 1. Surface part segmentation on PartObjaverse-Tiny-wt, reported in mIoU (%). OmniLatent is evaluated in the mask-guided setting.
Surface
Method
mIoU ↑
Generated
Find3D ( Ma et al., 2025b )
19.21
Generated
SAMPart3D ( Yang et al., 2024 )
48.73
Generated
PartField ( Liu et al., 2025b )
51.93
Generated
P3 -SAM ( Ma et al., 2025a )
56.01
Generated
OmniLatent
74.18
GT watertight
Ours w/o mask
62.42
Table 2. Additional surface part segmentation comparisons in mIoU (%). The same-surface setting evaluates every method on meshes generated by OmniLatent. We also evaluate OmniLatent on the ground-truth watertight surface with and without 2D mask guidance.
Figure 7. Point-prompted part control. OmniLatent supports three prompt-driven editing modes: (a) Full segmentation from sparse point prompts, (b) Part refine for correcting an initialized segmentation, and (c) Part subdivision for splitting a coarse part into finer components.
Method
Full Seg. ↑
Interactive Seg. ↑
Point-SAM ( Zhou et al., 2024 )
–
24.16
P3 -SAM ( Ma et al., 2025a )
58.10
49.11
OmniLatent
67.32
55.75
Table 3. Point-prompted surface part segmentation using 10 prompts, reported in mIoU (%). “Full Seg.” evaluates complete segmentation from prompts, while “Interactive Seg.” evaluates prompt-guided editing.
Figure 8. Qualitative comparison on amodal part completion. Given full geometry and partial surface masks, OmniLatent preserves visible part evidence and completes coherent full-object parts, producing cleaner part geometry and more consistent object structure than X-Part, HoloPart, PartPacker, and OmniPart.
Method
CD ↓
F-score-0.1 ↑
F-score-0.05 ↑
Part123 ( Liu et al., 2024a )
0.42
0.36
0.20
HoloPart ( Yang et al., 2025b )
0.09
0.88
0.73
PartCrafter ( Lin et al., 2025 )
0.20
0.66
0.45
PartPacker ( Tang et al., 2025b )
0.11
0.85
0.65
OmniPart ( Yang et al., 2025c )
0.08
0.91
0.77
X-Part ( Yan et al., 2025 )
0.08
0.92
0.78
Table 4. Holistic geometric evaluation for amodal part completion. Given full geometry and partial surface masks, we evaluate the union of completed parts with Chamfer Distance (CD) and F-scores following X-Part.
Method
CD ↓
F-score-0.1 ↑
F-score-0.05 ↑
SAMPart3D ( Yang et al., 2024 )
0.15
0.73
0.63
PartField ( Liu et al., 2025b )
0.17
0.68
0.57
HoloPart ( Yang et al., 2025b )
0.26
0.59
0.43
OmniPart ( Yang et al., 2025c )
0.23
0.63
0.46
X-Part ( Yan et al., 2025 )
0.11
0.80
0.71
Ours (w/o surface RoPE)
0.15
0.75
0.64
Table 5. Part-level evaluation for amodal part completion. Given full geometry and partial surface masks, we evaluate completed parts against ground-truth part geometries with Chamfer Distance (CD) and F-scores following X-Part.
Figure 9. Visual comparison of geometry generation . Despite its unified design, our method still produces high-quality generations, providing strong geometric understanding that benefits part decomposition.
Figure 10. Unified generation and mask-guided surface part segmentation. OmniLatent jointly generates geometry, deformation, appearance, and surface part IDs from a single image, and can use 2D segmentation masks as controllable guidance for part granularity.
Method
ULIP-T
ULIP-I
Uni-T
Uni-I
Michelangelo ( Zhao et al., 2024 )
0.075
0.115
0.213
0.261
Craftsman 1.5 ( Li et al., 2024 )
0.074
0.129
0.237
0.298
TRELLIS ( Xiang et al., 2024 )
0.076
0.126
0.249
0.311
Hunyuan3D 2.0 ( Zhao et al., 2025c )
0.077
0.130
0.251
0.315
Hi3DGen ( Ye et al., 2025 )
0.066
0.112
0.246
0.299
Direct3D-s2 ( Wu et al., 2025b )
0.074
0.122
0.247
0.314
Table 6. Quantitative comparison of geometry generation.
Method
cFID ↓
CMMD ↓
CLIP ↑
LPIPS ↓
Paint3D ( Zeng et al., 2024 )
26.86
2.400
0.887
0.126
TexGen ( Yu et al., 2024 )
28.23
2.447
0.882
0.133
Hunyuan3D-2 ( Zhao et al., 2025c )
26.43
2.318
0.889
0.126
RomanTex ( Feng et al., 2025 )
24.78
2.191
0.891
0.121
MaterialMVP ( He et al., 2025b )
24.78
2.191
0.921
0.121
OmniLatent (Ours)
22.95
2.079
0.903
0.111
Table 7. Quantitative comparison of appearance generation.
Figure 11. Ablation study on Part ID representation and pretraining.
Figure 12. Ablation study on Condition-Aware 3D Rotary Position Embedding.
Category
Part ID-only Latent
W/o CA-RoPE
OmniLatent (Ours)
Avg.
67.69
69.20
74.18
Human-Shape
67.34
74.99
79.36
Animals
76.34
72.97
82.68
Daily-Used
68.89
72.60
76.98
Buildings
51.34
52.33
58.94
Transportations
59.27
60.49
65.04
Table 8. Ablation study on unified latent modeling and CA-RoPE for surface part segmentation on PartObjaverse-Tiny-wt, reported in mIoU (%).
Figure 13. Limitations. OmniLatent can still produce asymmetric part layouts on symmetric structures and may mistakenly merge semantically distinct regions into a single part.
Existing 3D part decomposition methods do not necessarily partition the original shape into non-overlapping parts that collectively cover the entire shape, allowing overlaps or gaps that hinder downstream part-level applications. We instead formulate part decomposition as a joint partitioning of the entire shape, where the predicted parts are non-overlapping and jointly recover the entire shape. Our key insight is that part decomposition should consider all desired parts jointly, rather than modeling each part independently. To this end, we develop a promptable model for 3D part decomposition from images or meshes. Users can specify desired parts through 3D point prompts for controllable decomposition. Given one point prompt per desired part, our model produces the corresponding parts as a complete partition of the entire shape. We build on a pretrained 3D generation model and first obtain a shape latent from either an input image or mesh. We then introduce a prompt encoder that maps each 3D point prompt to a part token while attending to the shape latent. To decode the desired parts, we propose a novel part decoder jointly scoring the entire shape against all part tokens in a coarse-to-fine manner, assigning every position within the shape volume to exactly one part. We perform part decomposition in this shared shape latent space, enabling a unified model for image-to-part generation, mesh-to-part generation, and part segmentation. Our method outperforms existing works on all part-quality metrics across all three tasks, and improves compatibility among parts by an order of magnitude over previous SOTA methods. Code and models will be released.
Part-level 3D generation has recently attracted increasing attention for producing structured and editable 3D assets. However, existing methods typically decompose objects according to functional semantics rather than the editable material boundaries (e.g., fabric, wood, metal) required in practical 3D applications such as interior design. Additionally, current methods often generate parts independently, causing computational costs to scale linearly with the part count. To address these limitations, we present PartMat, an efficient material-aware 3D part decomposition pipeline that represents multi-part geometry with a single global latent. Given a reference image and a single whole-object geometry, PartMat decomposes the object into parts that follow material boundaries. First, we propose PartVAE to learn such a unified representation and decode all material parts in a single forward pass, thereby decoupling inference cost from the number of parts. Second, with this representation, a diffusion model is trained for part generation and refined via reinforcement learning for accurate material assignment and overlap suppression. Finally, to recover fine-grained geometric details, we introduce a sparse-voxel flow-matching model with part attention for geometry post-processing. Extensive experiments demonstrate that PartMat significantly outperforms existing baselines in material-aware decomposition accuracy and achieves comparable geometric quality, while maintaining efficient inference.
Part segmentation is a fundamental problem in computer graphics and 3D vision. Recent works have expanded 3D part segmentation beyond fixed taxonomies, but existing approaches typically only address a specific setting, such as text-guided part segmentation or point-based interaction. In this work, we argue that these settings can be unified as an intent-conditioned generative problem, where different prompts specify the desired part decomposition. To this end, we introduce PartLLM, a unified multimodal model that formulates 3D part segmentation as autoregressive semantic decomposition. Conditioned on an input shape and a user prompt, PartLLM autoregressively generates semantic part hypotheses as queries for mask prediction and feeds them to a decomposition-aware decoder that jointly predicts coherent part masks. This unified design supports text-guided part segmentation, interactive segmentation, and full-shape semantic decomposition with controllable granularity within a single model. Extensive experiments across these task settings show that PartLLM consistently outperforms task-specific baselines, demonstrating the effectiveness of unifying 3D part segmentation under an intent-conditioned generative formulation.
Zhe Zhu, Yiheng Zhang, Peng Li +9
Taiyuan University of Technology, Tencent Visvise, China · Hong Kong University of Science and Technology, Tencent Visvise, China · Taiyuan University of Technology, China +6