OmniLatent: Unified Latent for 3D Part Modeling

Yunhan Yang; Yunfei Zhao; Zeqiang Lai; Jiachen Xu; Minxiu Wang; Xintong Han; Xiangyu Yue; Xihui Liu

cs.CVNov 6, 2026

OmniLatent: Unified Latent for 3D Part Modeling

Authors: Yunhan Yang, Yunfei Zhao, Zeqiang Lai, Jiachen Xu, Minxiu Wang, Xintong Han, Xiangyu Yue, Xihui Liu,

Organizations: University of Hong Kong, Hong Kong · Tencent Technology, China · The Chinese University of Hong Kong, Hong Kong · Nanjing University, China

Abstract

3D part decomposition is a central capability for editable asset creation, yet existing workflows typically segment a completed mesh as a post-processing step, making the result sensitive to geometric artifacts, texture ambiguity, and limited part-level annotations. We present OmniLatent, a unified latent modeling framework for 3D part decomposition. Instead of predicting part labels in isolation, OmniLatent learns a shared latent space that jointly encodes geometry, deformation, appearance, and binary part IDs, allowing data-rich geometric and visual priors to support data-scarce part learning. Given a single input image, the base model generates geometry, deformation, texture, and surface part IDs in one pass, producing a part-aware 3D asset without a separate segmentation stage. We further tune the pretrained model for two extensions: a point-prompt adapter that maps arbitrary 3D point prompts with binary part IDs into latent tokens for full segmentation, refinement, and part subdivision, and an amodal part completion model that completes partial surface masks into a coherent full-object decomposition through dynamic per-part token allocation, cross-attention to the given geometry latent, and alternating part-local and global attention. Extensive experiments show that OmniLatent produces semantically meaningful part segmentations with precise boundaries, supports interactive part editing, and completes partial part observations while preserving global object consistency.

Figures & tables

Figure 1. OmniLatent generates geometry, deformation, appearance, and coherent surface part IDs from a single image in a unified latent space.

Figure 2. Unified Latent Modeling and Pretraining . OmniLatent performs 3D part decomposition through unified latent modeling. Geometry, deformation, appearance, and binary part IDs are encoded into a shared OmniLatent space, then generated by an MM-DiT backbone from an input image with optional 2D mask conditions. The aligned latent space allows geometric and visual priors to support surface part segmentation, and the pretrained model can further be adapted for point-prompted control and amodal part completion. The left branch shows the cascaded VAE decoder: SDF reconstruction defines the surface, whose vertices subsequently query deformation, appearance, and Part IDs from the same latent tokens.

Figure 3. Unified part segmentation and refinement using images, 2D masks, and point prompts. We adopt an MM-DiT architecture that integrates multiple control signals: interactive regions and geometric conditions are injected via channel concatenation with the noisy latent, while other conditions are incorporated through sequence concatenation.

Figure 4. Amodal part completion. Given full geometry and partial surface masks for all parts, OmniLatent allocates tokens dynamically according to visible part size. RoPE-anchored tokens preserve observed surface evidence, while no-RoPE completion tokens represent unobserved regions. A geometry latent enters through cross-attention, and alternating part-local and global attention layers complete all parts into a consistent full-object part layout.

Figure 5. Surface part segmentation and amodal completion results. For each example, OmniLatent produces complete surface part segmentation and further completes partial surface masks into complete object-level parts, preserving global structure while recovering missing components.

Figure 6. Qualitative comparison of surface part segmentation on PartObjaverse-Tiny-wt. OmniLatent produces semantically coherent regions and cleaner boundaries by decoding part IDs from a latent space aligned with geometry and appearance.

Method	Avg. $\uparrow$	Human-Shape	Animals	Daily-Used	Buildings	Transportations	Plants	Food	Electronics
Find3D ( Ma et al., 2025b )	20.76	25.08	24.86	21.71	14.93	12.52	22.43	27.54	17.02
SAMPart3D ( Yang et al., 2024 )	50.02	48.49	58.66	49.00	39.52	42.68	58.37	54.94	48.48
PartField ( Liu et al., 2025b )	52.61	54.50	60.36	56.38	38.43	44.57	55.70	58.59	52.31
$\text{P}^{3}$ -SAM ( Ma et al., 2025a )	58.10	64.10	62.04	60.74	43.66	54.83	65.36	55.73	58.37
OmniLatent (Ours)	74.18	79.36	82.68	76.98	58.94	65.04	78.73	80.72	70.98

Table 1. Surface part segmentation on PartObjaverse-Tiny-wt, reported in mIoU (%). OmniLatent is evaluated in the mask-guided setting.

Surface	Method	mIoU $\uparrow$
Generated	Find3D ( Ma et al., 2025b )	19.21
Generated	SAMPart3D ( Yang et al., 2024 )	48.73
Generated	PartField ( Liu et al., 2025b )	51.93
Generated	$\text{P}^{3}$ -SAM ( Ma et al., 2025a )	56.01
Generated	OmniLatent	74.18
GT watertight	Ours w/o mask	62.42

Table 2. Additional surface part segmentation comparisons in mIoU (%). The same-surface setting evaluates every method on meshes generated by OmniLatent. We also evaluate OmniLatent on the ground-truth watertight surface with and without 2D mask guidance.

Figure 7. Point-prompted part control. OmniLatent supports three prompt-driven editing modes: (a) Full segmentation from sparse point prompts, (b) Part refine for correcting an initialized segmentation, and (c) Part subdivision for splitting a coarse part into finer components.

Method	Full Seg. $\uparrow$	Interactive Seg. $\uparrow$
Point-SAM ( Zhou et al., 2024 )	–	24.16
$\text{P}^{3}$ -SAM ( Ma et al., 2025a )	58.10	49.11
OmniLatent	67.32	55.75

Table 3. Point-prompted surface part segmentation using 10 prompts, reported in mIoU (%). “Full Seg.” evaluates complete segmentation from prompts, while “Interactive Seg.” evaluates prompt-guided editing.

Figure 8. Qualitative comparison on amodal part completion. Given full geometry and partial surface masks, OmniLatent preserves visible part evidence and completes coherent full-object parts, producing cleaner part geometry and more consistent object structure than X-Part, HoloPart, PartPacker, and OmniPart.

Method	CD $\downarrow$	F-score-0.1 $\uparrow$	F-score-0.05 $\uparrow$
Part123 ( Liu et al., 2024a )	0.42	0.36	0.20
HoloPart ( Yang et al., 2025b )	0.09	0.88	0.73
PartCrafter ( Lin et al., 2025 )	0.20	0.66	0.45
PartPacker ( Tang et al., 2025b )	0.11	0.85	0.65
OmniPart ( Yang et al., 2025c )	0.08	0.91	0.77
X-Part ( Yan et al., 2025 )	0.08	0.92	0.78

Table 4. Holistic geometric evaluation for amodal part completion. Given full geometry and partial surface masks, we evaluate the union of completed parts with Chamfer Distance (CD) and F-scores following X-Part.

Method	CD $\downarrow$	F-score-0.1 $\uparrow$	F-score-0.05 $\uparrow$
SAMPart3D ( Yang et al., 2024 )	0.15	0.73	0.63
PartField ( Liu et al., 2025b )	0.17	0.68	0.57
HoloPart ( Yang et al., 2025b )	0.26	0.59	0.43
OmniPart ( Yang et al., 2025c )	0.23	0.63	0.46
X-Part ( Yan et al., 2025 )	0.11	0.80	0.71
Ours (w/o surface RoPE)	0.15	0.75	0.64

Table 5. Part-level evaluation for amodal part completion. Given full geometry and partial surface masks, we evaluate completed parts against ground-truth part geometries with Chamfer Distance (CD) and F-scores following X-Part.

Figure 9. Visual comparison of geometry generation . Despite its unified design, our method still produces high-quality generations, providing strong geometric understanding that benefits part decomposition.

Figure 10. Unified generation and mask-guided surface part segmentation. OmniLatent jointly generates geometry, deformation, appearance, and surface part IDs from a single image, and can use 2D segmentation masks as controllable guidance for part granularity.

Method	ULIP-T	ULIP-I	Uni-T	Uni-I
Michelangelo ( Zhao et al., 2024 )	0.075	0.115	0.213	0.261
Craftsman 1.5 ( Li et al., 2024 )	0.074	0.129	0.237	0.298
TRELLIS ( Xiang et al., 2024 )	0.076	0.126	0.249	0.311
Hunyuan3D 2.0 ( Zhao et al., 2025c )	0.077	0.130	0.251	0.315
Hi3DGen ( Ye et al., 2025 )	0.066	0.112	0.246	0.299
Direct3D-s2 ( Wu et al., 2025b )	0.074	0.122	0.247	0.314

Table 6. Quantitative comparison of geometry generation.

Method	cFID $\downarrow$	CMMD $\downarrow$	CLIP $\uparrow$	LPIPS $\downarrow$
Paint3D ( Zeng et al., 2024 )	26.86	2.400	0.887	0.126
TexGen ( Yu et al., 2024 )	28.23	2.447	0.882	0.133
Hunyuan3D-2 ( Zhao et al., 2025c )	26.43	2.318	0.889	0.126
RomanTex ( Feng et al., 2025 )	24.78	2.191	0.891	0.121
MaterialMVP ( He et al., 2025b )	24.78	2.191	0.921	0.121
OmniLatent (Ours)	22.95	2.079	0.903	0.111

Table 7. Quantitative comparison of appearance generation.

Figure 11. Ablation study on Part ID representation and pretraining.

Figure 12. Ablation study on Condition-Aware 3D Rotary Position Embedding.

Category	Part ID-only Latent	W/o CA-RoPE	OmniLatent (Ours)
Avg.	67.69	69.20	74.18
Human-Shape	67.34	74.99	79.36
Animals	76.34	72.97	82.68
Daily-Used	68.89	72.60	76.98
Buildings	51.34	52.33	58.94
Transportations	59.27	60.49	65.04

Table 8. Ablation study on unified latent modeling and CA-RoPE for surface part segmentation on PartObjaverse-Tiny-wt, reported in mIoU (%).

Figure 13. Limitations. OmniLatent can still produce asymmetric part layouts on symmetric structures and may mistakenly merge semantically distinct regions into a single part.

3D Part Segmentation

3D part segmentation divides objects and scenes into meaningful components using geometry, appearance, language, and motion cues.

See all 37 papers

Point2Part: Unified 3D Partitioning from Point Prompts
Sep 29, 2026Hao-Tang Tsui, Yu-Rou Tuan, Xiaoxuan Ma +33D Part Segmentation
Carnegie Mellon University
PartMat: Material-Aware 3D Part Decomposition with a Single Global Latent
Aug 3, 2026Guangming Fu, Jin Song, Yiyun Fei +3Part-Level 3D Generation 3D Asset Generation
1Nankai University, China · 2Alibaba Group, China
PartLLM: A Unified Multimodal Foundation for 3D Part Segmentation
Sep 22, 2026Zhe Zhu, Yiheng Zhang, Peng Li +93D Part Segmentation 3D VLMs
Taiyuan University of Technology, Tencent Visvise, China · Hong Kong University of Science and Technology, Tencent Visvise, China · Taiyuan University of Technology, China +6

Sep 29, 2026J/K move · Enter open · S save

Point2Part: Unified 3D Partitioning from Point Prompts

Hao-Tang Tsui, Yu-Rou Tuan, Xiaoxuan Ma, Nicolas Ugrinovic+2

Carnegie Mellon University

3D Part Segmentation

Open paper AskPDF arXiv

Existing 3D part decomposition methods do not necessarily partition the original shape into non-overlapping parts that collectively cover the entire shape, allowing overlaps or gaps that hinder downstream part-level applications. We instead formulate part decomposition as a joint partitioning of the entire shape, where the predicted parts are non-overlapping and jointly recover the entire shape. Our key insight is that part decomposition should consider all desired parts jointly, rather than modeling each part independently. To this end, we develop a promptable model for 3D part decomposition from images or meshes. Users can specify desired parts through 3D point prompts for controllable decomposition. Given one point prompt per desired part, our model produces the corresponding parts as a complete partition of the entire shape. We build on a pretrained 3D generation model and first obtain a shape latent from either an input image or mesh. We then introduce a prompt encoder that maps each 3D point prompt to a part token while attending to the shape latent. To decode the desired parts, we propose a novel part decoder jointly scoring the entire shape against all part tokens in a coarse-to-fine manner, assigning every position within the shape volume to exactly one part. We perform part decomposition in this shared shape latent space, enabling a unified model for image-to-part generation, mesh-to-part generation, and part segmentation. Our method outperforms existing works on all part-quality metrics across all three tasks, and improves compatibility among parts by an order of magnitude over previous SOTA methods. Code and models will be released.

Abstract

3D Part Segmentation

Explore similar work

Point2Part: Unified 3D Partitioning from Point Prompts

PartMat: Material-Aware 3D Part Decomposition with a Single Global Latent

PartLLM: A Unified Multimodal Foundation for 3D Part Segmentation