We introduce the problem of material-aware part grouping in untextured meshes. Many real-world shapes, such as scales of pinecones or windows of buildings, contain repeated structures that share the same material but exhibit geometric variations. When assigning materials to such meshes, these repeated parts often require piece-by-piece manual identification and selection, which is tedious and time-consuming. To address this, we propose Material Magic Wand, a tool that allows artists to select part groups based on their estimated material properties -- when one part is selected, our algorithm automatically retrieves all other parts likely to share the same material. The key component of our approach is a part encoder that generates a material-aware embedding for each 3D part, accounting for both local geometry and global context. We train our model with a supervised contrastive loss that brings embeddings of material-consistent parts closer while separating those of different materials; therefore, part grouping can be achieved by retrieving embeddings that are close to the embedding of the selected part. To benchmark this task, we introduce a curated dataset of 100 shapes with 241 part-level queries. We verify the effectiveness of our method through extensive experiments and demonstrate its practical value in an interactive material assignment application.
Figures & tables
Figure 1: Material Magic Wand. Given an untextured 3D mesh (left) with existing part segmentation, which is often obtained by finding connected components of the mesh, a user can apply our tool to select a group of material-consistent parts by clicking on one single representative part. In the middle, we show example selections of petal , base , sepal , stem , leaf , and grass . For each selection, the tool automatically finds all other parts in the shape that are likely to share the same material (right) through geometric and contextual cues, accelerating the material assignment process. Colors may appear darker due to backface shading.
Figure 2: Method Overview. Left: Our view selection process renders each part with nearby context from multiple viewpoints sampled randomly over a hemisphere. We choose the one with minimal occlusion, Ictx , and use the same viewpoint to render the part in isolation, Ipart . Ifull captures the entire mesh. We highlight the part with a different color from the rest of the mesh. Right: For each part, its corresponding images are passed through an encoder and their embeddings are concatenated. During training, embeddings of parts with the same material are pulled together, while those with different materials are pushed apart.
Method
AUC PR
R-Prec
mAP
Recall@K
F1
R@5
R@10
R@20
R@100
Histogram Matching [ 5 ]
26.85
25.45
30.71
4.55
8.89
15.97
45.64
23.84
SigLIP-v2 [ 43 ]
62.83
56.02
60.58
22.88
31.20
40.72
68.44
39.44
PartField [ 26 ]
75.30
67.74
70.52
26.88
37.12
47.92
71.31
56.57
DINO-v3 small† [ 41 ]
81.14
78.32
83.49
34.57
45.49
56.63
82.18
59.36
Ours
89.74
88.33
91.70
37.99
50.98
62.79
83.67
75.94
Table 1: Comparison of geometry-based, vision-embedding, and segmentation-based baselines on our material-aware grouping task. All metrics are macro-averaged and reported in percentage. Our method achieves consistent improvements across all evaluation metrics.
Figure 3: Ambiguities in raw material IDs. Using meshes from Objaverse directly for material grouping can introduce ambiguity in the testing benchmark due to artistic intent, noisy labeling, or coarse material assignment. For example, some shingles on the roof exhibit mixed materials, and fence or wall slats alternate between different material labels (left). To reduce such inconsistencies, we manually refine the material annotations to make the material assignment more uniform and fine-grained (right).
Figure 4: Precision–Recall curve. We sweep the similarity threshold to evaluate retrieval performance. Our method consistently maintains higher precision across all recall levels, achieving the highest AUC (89.7), followed by DINO (81.1).
Figure 5: Qualitative comparison. For each mesh, the red part denotes the query, and orange parts indicate the retrieved matches. Our method retrieves components that are both geometrically and contextually similar with the query, while baselines often miss structurally related parts or include visually similar but contextually incorrect ones.
Figure 6: Effect of changing the grouping tolerance parameter. The similarity threshold λ controls the cut-off distance in the parts’ embedding space for part retrieval. With our method (right), at low values of λ , only highly similar parts are selected; increasing λ gradually expands the selection to include less similar, yet related parts. In contrast, baselines exhibit less stable behavior by either retrieving unrelated components at higher thresholds or lacking fine-grained control at lower thresholds.
Figure 7: Effect of multiple queries. Additional query parts improve retrieval by capturing a more complete set of desired components. The initial query (left) retrieves only a subset of sails, while adding an additional example (right) helps recover the rest.
Method
AUC
R-Prec
mAP
R@20
w/o isolated part
86.90
84.97
88.53
61.11
w/o part-with-context
87.30
85.37
88.91
61.09
w/o full-object
88.89
87.72
90.75
62.36
Only isolated part
86.18
81.88
85.88
59.87
Init from Dino-v2 L
86.51
84.58
88.61
61.13
Random initialization
78.54
74.52
79.62
56.25
Table 2: Ablation results. We examine the contribution of our design choices. Removing any component reduces performance, with the full model achieving the highest scores across all metrics.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8: Qualitative results of Histogram Matching approach. Histogram matching successfully retrieves near-identical components (e.g., duplicated branches or similar jewels) but fails to capture structurally similar yet non-identical parts, such as the rear car window or varied tree leaves.
Figure 9: Connected components as segments. Examples of part segmentation produced using connected components. Each color corresponds to a different part. N denotes the number of parts in the mesh, illustrating the fine granularity of the decomposition.
Figure 10: Data sampling. (M: #materials, C: #segments, S: #sampled after balancing). Top-left (a): Within-mesh imbalance where one material (purple bricks) dominates. Top-right (b): Long-tail distribution with 76 materials but most appearing in few parts. Bottom-left (c): Insufficient samples requiring multi-view augmentation. Bottom-right (d): Mesh with 56K several near-duplicate parts requiring capping.
Inference
#Parts †
Time(s) †
AUC ↑
mAP ↑
No Dedup
265; 1144.2
48.91; 208.05
88.13
89.68
Dedup
60; 181.0
11.14; 33.01
89.74
91.70
† Reported numbers are (Median; IQR) over full test set.
Appendix
Table 3: Effect of deduplication on inference runtime.
Figure 11: Deduplication error when using similar pieces across categories (e.g., cuboid for stairs and fence ).
Method
AUC PR
R-Prec
mAP
Recall@K
R@5
R@10
R@20
R@100
Dino v2 (small)
80.13
77.33
82.23
33.39
45.14
57.03
81.60
Dino v2 (base)
78.99
76.24
81.98
33.37
45.10
56.73
81.00
Dino v2 (large)
78.28
74.41
80.59
33.94
44.70
55.92
81.00
Dino v2 (giant)
78.30
76.51
81.78
33.57
45.15
56.29
80.47
Dino v3 (small)
81.14
78.32
83.49
34.57
45.49
56.63
82.18
Appendix
Table 4: Performance comparison of different DINO backbone variants on our material-aware part retrieval benchmark. Results show that scaling to larger models provides only marginal or inconsistent gains across metrics. All metrics in (%).
Figure 12: Additional qualitative results on Objaverse test set.
Figure 13: Results from Texverse. For each mesh in a row, the red part denotes the query and the orange parts indicate the retrieved matches. While baselines often miss non-identical instances or include unrelated parts, our method can retrieve material-consistent components (window frames, ship masts, stairs, and roof logs).
Figure 14: Geometric variation within a material. Given a query part (red), we show retrieved parts (left in each pair) versus ground truth (right in each pair). Left: Cable endpoints in the machinery exhibit different geometry from the cable body due to folding and knotting. Right: Bike cables near the down tube are tightly coiled, appearing geometrically distinct from the loosely suspended cable segments near the handlebars. Such cases can result in incomplete retrieval.
Figure 15: Geometric similarity across different materials. Given a query part (red), we show retrieved parts (left in each pair) versus ground truth (right in each pair). Left: Chess pieces have identical geometry across opposing teams, differing only in material. Right: Tail and wing feathers share similar geometric structure, leading to incorrect grouping despite different material assignments. These cases could be mitigated by training on more examples with similar contextual patterns.
Figure 16: Ambiguity arising from artistic intent. Given a query part (red), we show retrieved parts (left in each pair) versus ground truth (right in each pair). Left: Our method retrieves all pentagon-shaped cake toppings based on geometric and contextual similarity, but the actual assignments had different materials (shown in varying colors). Right: We retrieve all foliage when querying one leaf cluster, but the mesh contains intentional material variation across the tree. Such artistic choices represent inherent ambiguity that cannot be resolved from geometry and context.
Part-level 3D generation has recently attracted increasing attention for producing structured and editable 3D assets. However, existing methods typically decompose objects according to functional semantics rather than the editable material boundaries (e.g., fabric, wood, metal) required in practical 3D applications such as interior design. Additionally, current methods often generate parts independently, causing computational costs to scale linearly with the part count. To address these limitations, we present PartMat, an efficient material-aware 3D part decomposition pipeline that represents multi-part geometry with a single global latent. Given a reference image and a single whole-object geometry, PartMat decomposes the object into parts that follow material boundaries. First, we propose PartVAE to learn such a unified representation and decode all material parts in a single forward pass, thereby decoupling inference cost from the number of parts. Second, with this representation, a diffusion model is trained for part generation and refined via reinforcement learning for accurate material assignment and overlap suppression. Finally, to recover fine-grained geometric details, we introduce a sparse-voxel flow-matching model with part attention for geometry post-processing. Extensive experiments demonstrate that PartMat significantly outperforms existing baselines in material-aware decomposition accuracy and achieves comparable geometric quality, while maintaining efficient inference.
This paper aims to generate materials for 3D meshes from text descriptions. Unlike existing methods that synthesize texture maps, we propose to generate segment-wise procedural material graphs as the appearance representation, which supports high-quality rendering and provides substantial flexibility in editing. Instead of relying on extensive paired data, i.e., 3D meshes with material graphs and corresponding text descriptions, to train a material graph generative model, we propose to leverage the pre-trained 2D diffusion model as a bridge to connect the text and material graphs. Specifically, our approach decomposes a shape into a set of segments and designs a segment-controlled diffusion model to synthesize 2D images that are aligned with mesh parts. Based on generated images, we initialize parameters of material graphs and fine-tune them through the differentiable rendering module to produce materials in accordance with the textual description. Extensive experiments demonstrate the superior performance of our framework in photorealism, resolution, and editability over existing methods. Project page: https://zju3dv.github.io/MaPa
Shangzhan Zhang, Sida Peng, Tao Xu +7
Zhejiang University · Ant Group · Shenzhen University
Existing 3D part decomposition methods do not necessarily partition the original shape into non-overlapping parts that collectively cover the entire shape, allowing overlaps or gaps that hinder downstream part-level applications. We instead formulate part decomposition as a joint partitioning of the entire shape, where the predicted parts are non-overlapping and jointly recover the entire shape. Our key insight is that part decomposition should consider all desired parts jointly, rather than modeling each part independently. To this end, we develop a promptable model for 3D part decomposition from images or meshes. Users can specify desired parts through 3D point prompts for controllable decomposition. Given one point prompt per desired part, our model produces the corresponding parts as a complete partition of the entire shape. We build on a pretrained 3D generation model and first obtain a shape latent from either an input image or mesh. We then introduce a prompt encoder that maps each 3D point prompt to a part token while attending to the shape latent. To decode the desired parts, we propose a novel part decoder jointly scoring the entire shape against all part tokens in a coarse-to-fine manner, assigning every position within the shape volume to exactly one part. We perform part decomposition in this shared shape latent space, enabling a unified model for image-to-part generation, mesh-to-part generation, and part segmentation. Our method outperforms existing works on all part-quality metrics across all three tasks, and improves compatibility among parts by an order of magnitude over previous SOTA methods. Code and models will be released.