Self-supervised Vision Transformers (ViTs) like DINO encode rich semantics, enabling training-free segmentation of objects from the background and discovery of their parts via feature clustering. This paper focuses on part discovery: partitioning a foreground object into semantic parts. Existing approaches produce imprecise part boundaries. We trace this to the rigid ViT patch grid used in training DINO, where patches straddling boundaries contain pixels from multiple parts. While DINO's invariance-based training could absorb this pixel-level mixing into clean features, experiments with over 26K foreground patches show it does not: boundary-overlapping patches produce features with higher entropy and lower assignment confidence, blurring predicted part boundaries. Rather than make the massive effort of retraining DINO over geometrically coherent regions, an easier approach is to take boundaries directly from a boundary detector, which supplies the geometric information DINO's features lack. Motivated by this, we introduce Part-DINO, a training-free framework that combines boundaries from low-level image segmentation with DINO's semantic features. We use the boundaries to partition the image into regions, pool DINO features within each region, and group them to discover parts. Without any training, Part-DINO produces structurally coherent part assignments, outperforming training-free baselines across four standard benchmarks (CUB, CelebA, PartImageNet-OOD, PartImageNet-Seg) and remaining competitive with weakly-supervised and unsupervised methods.
Figures & tables
Figure 1: Mixing of pixels from across part boundaries present in boundary-area patches blurs semantic boundaries. (a) Input image. (b) A boundary patch straddling the bird’s head and body pools pixels from both regions, illustrating geometric mixing in pixel space. (c) Across 26K foreground patches, feature entropy rises and the top-2 assignment margin falls as patches approach part boundaries, showing mixing in feature space, which causes ambiguity in inferences of semantic part boundaries as can be seen in (d) where edges from DINO feature similarity (left) blur at part boundaries, while edges from image contrast (right) align with them. Recovery of sharp boundaries from DINO features may thus be expected to benefit from complementary geometric information.
Figure 2: Overview of Part-DINO. Given a set of reference images I , the Region Generation module (Sec. 3.1 , top) extracts a foreground mask of the object Mfg and performs low-level image segmentation to produce atomic regions. In parallel, the Prototype Generation module (Sec. 3.2 , bottom) encodes the reference images through a frozen DINO ViT to obtain dense feature maps. These semantic features are pooled within each atomic region and clustered via K-Means to establish K global part prototypes {c1,c2,…,cK} . At inference (right), the Region Assignment module (Sec. 3.3 ) applies the same region generation pipeline to a test image and assigns each region to its nearest semantic prototype, producing the final part labels without any training.
Method
K=4
K=8
K=16
NME ↓
NMI ↑
ARI ↑
NME ↓
NMI ↑
ARI ↑
NME ↓
NMI ↑
ARI ↑
Weakly-supervised part discovery
Huang and Li (2020)
11.51
29.74
14.04
11.60
35.72
15.90
12.60
43.92
21.01
Van Der Klis et al. (2023)
9.12
37.82
15.26
8.52
50.08
26.96
7.64
56.87
38.05
Choudhury et al. (2021)
9.20
43.50
19.60
-
51.50
28.30
-
-
-
Van Der Klis et al. (2023) + ViT-B/14
7.70
52.59
26.66
6.34
65.01
37.90
5.95
68.63
43.41
Table 1: Quantitative comparisons on CUB dataset across part granularities K ∈{4,8,16} . We report NME ( ↓ ), NMI ( ↑ ), and ARI ( ↑ ) against weakly-supervised, unsupervised, and training-free methods. † indicates replicated with SAM foreground mask
Method
K=4
K=8
K=16
NME ↓
NMI ↑
ARI ↑
NME ↓
NMI ↑
ARI ↑
NME ↓
NMI ↑
ARI ↑
Weakly-supervised part discovery
Huang and Li (2020)
8.75
56.69
34.74
7.96
54.80
34.74
7.62
62.22
41.01
Van Der Klis et al. (2023)
11.11
75.97
69.53
9.82
62.61
51.89
9.46
77.43
70.58
Unsupervised part discovery
GANSeg He et al. (2022b)
12.26
41.71
28.06
6.18
67.28
56.23
-
-
-
Table 2: Quantitative comparison on CelebA across part granularities K ∈{4,8,16} . We report NME ( ↓ ), NMI ( ↑ ), and ARI ( ↑ ) against weakly-supervised, unsupervised, and training-free methods. † indicates replication with SAM foreground masks.
Figure 3: Qualitative comparison across CUB ( K=16 ), CelebA ( K=16 ), PartImageNet-S ( K=50 ), and PartImageNet-O ( K=50 ). MPAE is an unsupervised method that requires dataset-specific training, while Amir et al. (2021) , Amir et al. (2021) + SAM, and our Part-DINO are all training-free.
Table 6
Method
PartImage _O (%)
PartImage _S (%)
K=8
K=25
K=50
K=8
K=25
K=50
NMI ↑
ARI ↑
NMI ↑
ARI ↑
NMI ↑
ARI ↑
NMI ↑
ARI ↑
NMI ↑
ARI ↑
NMI ↑
ARI ↑
Weakly-supervised part discovery
Huang and Li (2020)
5.88
1.53
7.56
1.25
10.19
1.05
-
-
-
-
-
-
Van Der Klis et al. (2023)
27.13
8.76
32.41
10.69
41.49
14.17
10.68
32.43
18.44
40.70
21.98
39.40
Van Der Klis et al. (2023) (ViT-B/14)
19.28
34.72
28.23
25.93
29.48
28.68
16.68
32.01
46.81
40.70
34.13
39.40
Table 5: Quantitative comparison on PartImageNet Out-of-Distribution (PartImageNet O ) and Segmentation (PartImageNet S ) splits across K∈{8,25,50} . Part-DINO shows consistent improvements over the training-free baseline on PartImageNet S across all granularities and metrics, and remains competitive on PartImageNet O . † indicates replicated with SAM foreground mask
Figure 8
Figure 5: Qualitative experiments on region generation. (a) The structural edges from RAMP align with object contours, while DINO’s semantic edges are blurred. (b) Watershed regions are separable (head, body, wing, and tail), while SLIC prioritizes spatial compactness and cuts across them.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Regions/img
NME ↓
NMI ↑
ARI ↑
Amir et al. (2021) †
–
4.87
56.92
35.10
Part-DINO ( smin=10 , default, rerun)
506.7
4.33
64.98
45.83
Part-DINO ( smin=0 , no merge)
525.2
4.36
65.01
43.92
Appendix
Table 7: Effect of the DINO-based micro-region merge on CUB ( K=16 ). With smin=0 , region generation uses no DINO features. Both Part-DINO rows come from the same run. † replicated with SAM foreground mask.
Runtime Breakdown (ms)
Metrics
Method
DINO
SAM
RAMP
Wshed
Merge
Pool
Assign
Total
Avg. Reg.
NME ( ↓ )
NMI ( ↑ )
ARI ( ↑ )
( Amir et al., 2021 ) (patch)
40.0
552.4
0.0
0.0
0.0
6.0
1.1
599.5
1024
4.87
56.92
35.10
Part-DINO ( dmin=15 )
40.2
558.3
32.4
23.8
5.2
1.3
0.4
664.9
23.1
5.20
42.57
25.21
Part-DINO ( dmin=5 )
39.7
562.9
32.8
24.5
6.1
1.3
1.4
672.2
151.1
4.44
61.96
43.13
Part-DINO ( dmin=2 )
39.6
563.1
31.9
29.0
7.7
1.4
3.7
680.2
562.1
4.35
65.45
46.00
Appendix
Table 8: Timing breakdown (ms), average regions, and clustering performance metrics.
Figure 6: Extended Qualitative Analysis (CelebA). Comparison of edge map modalities. Structural boundaries consistently eliminate the semantic bleeding artifacts present in the patch-based baseline.
Figure 7: Quantifying the boundary bottleneck. Across 26,459 foreground patches, all three reliability metrics correlate monotonically with distance to the nearest part boundary: spatial purity rises ρ(d,π)=+0.71 ), while feature entropy falls (( ρ(d,H)=−0.38 ), and top-2 assignment margin grows ( ρ(d,m) = +0.36), confirming that geometric mixing propagates into feature space
Figure 8: Extended Qualitative Analysis (CUB). Comparison of edge map modalities on fine-grained bird categories, highlighting precision improvements in part boundaries.
Figure 9: Extended Qualitative Analysis (PartImageNet). Comparison of structural vs. semantic edge modalities on Out-of-Distribution (OOD) and Segmentation (Seg) examples. Structural boundaries demonstrate robust geometric alignment regardless of semantic category.
Figure 10: Extended Ablation on Region Generation (Part 1). SLIC enforces arbitrary spatial compactness, forcing grid-like boundaries that ignore true anatomical shapes. In contrast, our Watershed approach naturally follows the topographic valleys of the structural gradient, seamlessly wrapping around physical object contours.
Figure 11: Extended Ablation on Region Generation (Part 2). Continued comparison across complex PartImageNet categories, demonstrating the robustness of Watershed boundaries against rigid SLIC superpixels.
Tongji University Shanghai, China · Hong Kong Polytechnic University Hong Kong, China · University of Electronic Science and Technology of China Chengdu, China +1