SegviGen: Repurposing 3D Generative Model for Part Segmentation
Authors: Lin Li, Haoran Feng, Zehuan Huang, Haohua Chen, Wenbo Nie, Shaohua Hou, Keqing Fan, Pan Hu, Sheng Wang, Buyu Li, Lu Sheng
Organizations: Renmin University of China, Beijing, China · Tsinghua University, Shenzhen, China · Beihang University, Beijing, China · Beijing Jiaotong University, Beijing, China · Bambu Lab, Shenzhen, China
We introduce SegviGen, a framework that repurposes native 3D generative models for 3D part segmentation. Existing pipelines either lift strong 2D priors into 3D via distillation or multi-view mask aggregation, often suffering from cross-view inconsistency and blurred boundaries, or explore native 3D discriminative segmentation, which typically requires large-scale annotated 3D data and substantial training resources. In contrast, SegviGen leverages the structured priors encoded in pretrained 3D generative model to induce segmentation through distinctive part colorization, establishing a novel and efficient framework for part segmentation. Specifically, SegviGen encodes a 3D asset and predicts part-indicative colors on active voxels of a geometry-aligned reconstruction. It supports interactive part segmentation, full segmentation, and full segmentation with 2D guidance in a unified framework. Extensive experiments show that SegviGen improves over the prior state of the art by 40% on interactive part segmentation and by 15% on full segmentation, while using only 0.32% of the labeled training data. It demonstrates that pretrained 3D generative priors transfer effectively to 3D part segmentation, enabling strong performance with limited supervision. See our project page at https://fenghora.github.io/SegviGen-Page/.
Figures & tables
Figure 1. SegviGen enables diverse and accurate 3D part segmentation by leveraging priors from large-scale 3D generative models. With substantially less training data, it produces high-fidelity segmentation results with sharp part boundaries and strong generalization across object categories.
Figure 2. Pipeline of SegviGen . We reformulate 3D part segmentation as a conditional colorization task. During training, given a 3D mesh and its part-color ground truth, we encode both with a pretrained 3D VAE, add noise to the ground-truth latent, and then concatenate the geometry latent, noisy color latent, and point-condition tokens to form the final latent input. Conditioned on the sampled timestep and a task embedding, the multi-task flow transformer predicts the noise residual for flow-matching training.
Method
PartObjaverse-Tiny
PartNeXT
IoU@1
IoU@3
IoU@5
IoU@7
IoU@10
IoU@1
IoU@3
IoU@5
IoU@7
IoU@10
Point-SAM ( Zhou et al., 2025 )
24.87
48.99
59.67
64.33
67.99
23.90
47.50
56.71
61.23
65.04
P3-SAM ( Ma et al., 2025a )
33.04
50.57
53.78
54.74
55.51
35.61
51.26
52.03
52.61
53.81
SegviGen
42.49
61.14
67.53
71.50
75.02
54.86
71.15
78.11
79.96
82.73
Table 1. Comparison of interactive part segmentation performance. We report IoU at different numbers of clicks, compared with Point-SAM ( Zhou et al., 2025 ) and P3-SAM ( Ma et al., 2025a ) on PartObjaverse-Tiny ( Yang et al., 2024a ) and PartNeXT ( Wang et al., 2025 ) .
Figure 3. Interactive part-segmentation results. We compare SegviGen with existing representative baselines, including Point-SAM ( Zhou et al., 2025 ) and P3-SAM ( Ma et al., 2025a ) . In the figure, yellow points denote user clicks, while the predicted target part is highlighted in red. Leveraging priors from pretrained 3D generative models, SegviGen achieves more accurate results with sharper boundaries than prior methods, while requiring substantially less training data.
Figure 4. Full segmentation results. We compare SegviGen against a broad set of prior methods, where different colors indicate different segmented parts. From the results, SegviGen achieves high-accuracy full segmentation with sharp part boundaries using only 3D input.
Figure 5. Results with different view 2D guidance. Visible regions are consistently segmented, while occluded regions may receive different color assignments under different views.
Figure 6. Generalization results on AI-generated meshes. The results show that SegviGen can produce plausible interactive and full part segmentations on AI-generated 3D assets without additional training.
Method
PartObjaverse-Tiny
PartNext
Find3D ( Ma et al., 2025b )
15.62
19.04
SAMPart3D ( Yang et al., 2024b )
59.05
29.62
PartField ( Liu et al., 2025 )
51.72
41.50
P3-SAM ( Ma et al., 2025a )
45.36
31.94
SegviGen
50.64
55.40
SegviGen (w. 2D Map)
62.98
71.53
Table 2. Quantitative results (IoU) for full segmentation. “w. 2D Map” denotes the setting with 2D segmentation-map guidance.
Figure 7. Failure cases. Due to semantic ambiguity, SegviGen may produce slightly more or fewer parts than expected in interactive and full segmentation. In the 2D-guided setting, extremely fine structures may still be difficult to segment clearly and consistently from the guidance map.
Method
IoU@1
IoU@3
IoU@5
IoU@7
IoU@10
Explicit Coord
41.75
60.19
67.43
71.61
75.40
Label-based
42.49
61.14
67.53
71.50
75.02
Table 3. Ablation study on point embedding mechanisms. We compare the performance of Explicit Coordinate Encoding and Label-based Semantic Embedding under varying numbers of clicks on PartObjaverse ( Yang et al., 2024a ) .
Steps
IoU@1
IoU@3
IoU@5
IoU@7
IoU@10
Time
1
42.90
59.98
65.86
69.50
72.85
0.44s
4
44.51
60.40
66.65
70.64
73.58
1.02s
8
44.21
61.14
67.64
71.14
74.49
1.81s
12
42.49
61.14
67.53
71.50
75.02
2.63s
25
43.82
61.67
68.30
71.87
74.99
5.12s
Table 4. Effect of sampling steps on segmentation performance. We choose 12 steps as a practical trade-off between accuracy and efficiency.
Figure 8. Interactive demo. Users specify clicks on the 3D asset to perform interactive part segmentation, and can adjust visualization settings.
Figure 9. More qualitative comparisons for full segmentation.
Figure 10. More qualitative segmentation results of our SegviGen .
Figure 11. Interactive 3D editing results with SegviGen and VoxHammer ( Li et al., 2025a ) . SegviGen provides precise part segmentation to facilitate downstream editing models. The target region to be edited is indicated by green points. It demonstrates the practical utility of SegviGen in downstream 3D editing pipelines.
Taiyuan University of Technology, Tencent Visvise, China · Hong Kong University of Science and Technology, Tencent Visvise, China · Taiyuan University of Technology, China +6