K-CLUE: A Segmentation-to-Clustering Framework for Ki67 Cell Typing

Ana Beatriz Vieira; Sara P. Oliveira; Diana Montezuma; João Fraga; Jaime S. Cardoso; Arlindo L. Oliveira

cs.CVOct 23, 2026

K-CLUE: A Segmentation-to-Clustering Framework for Ki67 Cell Typing

Authors: Ana Beatriz Vieira, Sara P. Oliveira, Diana Montezuma, João Fraga, Jaime S. Cardoso, Arlindo L. Oliveira

Organizations: Instituto Superior Técnico, Universidade de Lisboa, Portugal · INESC-ID, Lisboa, Portugal · The Netherlands Cancer Institute Amsterdam, The Netherlands · IMP Diagnostics, Porto, Portugal ·

Abstract

Ki67 is a key marker of tumor proliferation in breast cancer, but its quantification from immunohistochemistry images normally depends on supervised classifiers trained per dataset. We examine the role of frozen pretrained visual representations in annotation-free Ki67 cell typing. To this end, we use K-CLUE, a framework that segments individual nuclei, represents each nucleus with a pretrained encoder, reduces the resulting embeddings with principal component analysis, and separates Ki67-positive from Ki67-negative nuclei using k-means clustering, without task-specific training or manual annotations. Clustering the raw components is highly sensitive to the choice of encoder: across seven encoders, the Ki67-positive F1-score on matched MIST nuclei varies by 44 points, although Ki67 status remains linearly decodable from every representation, with supervised probe AUCs above 0.98. Analysis of the embedding spaces shows that clustering succeeds when DAB-related variation occupies the leading directions of variance. Whitening the retained principal components removes this dependence, reducing the spread across encoders to 1.00 F1-score point. Handcrafted color representations are also competitive, confirming that Ki67 status is strongly reflected in chromatic information, but they require an explicit choice of color space. In end-to-end evaluation, K-CLUE reaches F1-scores of 85.00% and 81.91% for Ki67-positive and Ki67-negative nuclei on MIST, and 79.59% and 81.16% on BCData, outperforming DeepLIIF and IHCScoreGAN on both datasets. These results show that pretrained representations can support annotation-free Ki67 typing when their variance structure is accounted for.

Figures & tables

Panel of Figure 1 — Figure 1: MIST tile.

Figure 3: Overview of the proposed method pipeline. An input RGB tile is first segmented using Cellpose-DINOv3 to obtain instance-level nuclear masks. Each nucleus is extracted as a centered crop with background masking. Deep representations are computed with a pretrained feature encoder, reduced via PCA, whitened, and clustered using $k$ -means to separate Ki67-positive from Ki67-negative nuclei.

Model	Precision(%) $\uparrow$	Recall(%) $\uparrow$	F1-Score(%) $\uparrow$	MAE $\downarrow$
Cellpose-DINOv3 [ 17 ]	91.49	80.26	85.51	13.82
HoVer-NeXt [ 3 ]	91.00	54.00	67.78	42.00
CellViT++ [ 10 ]	91.22	50.11	64.69	46.84
StarDist [ 25 ]	90.27	63.65	74.66	33.46
CellDETR [ 18 ]	90.18	35.57	51.02	62.55

Table 1: Quantitative comparison of nuclei segmentation/detection models on the MIST dataset for cell detection (all nuclei).

Figure 4: Qualitative comparison of nuclei segmentation and detection results on MIST dataset tiles. Results from Cellpose-DINOv3, HoVer-NeXt, CellViT++, CellDETR, and StarDist are compared against the ground-truth annotations.

Model	Class	Precision(%) $\uparrow$	Recall(%) $\uparrow$	F1-Score(%) $\uparrow$	MAE $\downarrow$	MCC $\uparrow$
EfficientNet-B2	Ki67+	94.06	94.99	94.52	5.61	0.9210
EfficientNet-B2	Ki67-	97.79	97.37	97.58	12.14	0.9210
ResNet-101	Ki67+	83.15	93.01	87.81	6.68	0.8229
ResNet-101	Ki67-	96.76	91.72	94.18	14.46	0.8229
ConvNeXt V2 Tiny	Ki67+	94.16	91.11	92.61	6.41	0.8947
ConvNeXt V2 Tiny	Ki67-	96.15	97.52	96.83	11.45	0.8947

Table 2: Comparison of pretrained encoders for Ki67 cell typing on MIST before PCA whitening. Classification metrics are computed over Cellpose-DINOv3 detections matched to ground-truth nuclei, while MAE reports per-tile counting error over the predicted cell set.

Model	Class	Precision(%) $\uparrow$	Recall(%) $\uparrow$	F1-Score(%) $\uparrow$	MAE $\downarrow$	MCC $\uparrow$
EfficientNet-B2	Ki67+	96.01	93.37	94.67	5.75	0.9239
EfficientNet-B2	Ki67-	97.12	98.30	97.70	11.36	0.9239
ResNet-101	Ki67+	94.80	94.00	94.40	5.62	0.9196
ResNet-101	Ki67-	97.37	97.74	97.56	11.91	0.9196
ConvNeXt V2 Tiny	Ki67+	95.42	94.14	94.78	5.52	0.9251
ConvNeXt V2 Tiny	Ki67-	97.44	98.02	97.73	11.77	0.9251

Table 3: Comparison of pretrained encoders for Ki67 cell typing on MIST after PCA whitening. Classification metrics are computed over Cellpose-DINOv3 detections matched to ground-truth nuclei, while MAE reports per-tile counting error over the predicted cell set.

Model	Dim.	AUC $\uparrow$	AP $\uparrow$	F1-Score + (%) $\uparrow$	Bal. acc.(%) $\uparrow$
EfficientNet-B2	1408	0.9902	0.9813	95.22	96.79
ResNet-101	2048	0.9833	0.9641	93.01	95.77
ConvNeXt-V2 Tiny	768	0.9926	0.9869	95.53	97.21
DINOv3 ViT-S/16	384	0.9911	0.9757	95.51	97.09
HoVer-NeXt	1024	0.9925	0.9852	95.77	97.34
UNI2	1536	0.9957	0.9924	95.94	97.45

Table 4: Supervised linear probe performance on the frozen embeddings of each encoder, using ground-truth labels of the 4643 matched nuclei with cross-validation grouped by tile ID. AUC and AP denote the area under the ROC curve and the average precision. F1-Score + is the Ki67-positive F1-score.

Model	DAB	Hematoxylin	Saturation	Luminance	Area	Tile
EfficientNet-B2	0.829	0.118	0.747	0.774	0.488	0.626
ResNet-101	0.728	0.271	0.584	0.595	0.639	0.586
ConvNeXt-V2 Tiny	0.854	0.162	0.753	0.758	0.532	0.594
DINOv3 ViT-S/16	0.774	0.092	0.683	0.732	0.568	0.593
HoVer-NeXt	0.238	0.121	0.335	0.234	0.654	0.501
UNI2	0.579	0.105	0.574	0.628	0.422	0.634

Table 5: Absolute Pearson correlation between the first principal component of each frozen representation and per-nucleus covariates measured from the image. The tile column reports the correlation ratio

\eta

, as tile of origin is categorical.

	Ki67-positive cells				Ki67-negative cells
Model	P(%) $\uparrow$	R(%) $\uparrow$	F1(%) $\uparrow$	MAE $\downarrow$	P(%) $\uparrow$	R(%) $\uparrow$	F1(%) $\uparrow$	MAE $\downarrow$
DAB baseline	96.86	61.75	75.42	10.84	80.15	77.96	79.04	9.34
HED+HSV+L ∗ a ∗ b ∗	91.19	77.90	84.02	6.36	86.72	76.87	81.50	10.64
DeepLIIF	71.40	77.78	74.46	6.09	61.50	78.42	68.94	25.61
IHCScoreGAN	84.88	63.75	72.82	7.82	66.43	70.86	68.57	21.07
K-CLUE	89.41	81.00	85.00	5.52	88.25	76.41	81.91	11.77

Table 6: End-to-end Ki67-positive and Ki67-negative cell detection and classification on MIST using Hungarian matching with a

3\,\mu\mathrm{m}

tolerance.

Figure 5: Qualitative comparison of K-CLUE, IHCScoreGAN, and DeepLIIF for Ki67-positive (red) and Ki67-negative (blue) cell detection on representative MIST tiles.

	Ki67-positive cells				Ki67-negative cells
Model	P(%) $\uparrow$	R(%) $\uparrow$	F1(%) $\uparrow$	MAE $\downarrow$	P(%) $\uparrow$	R(%) $\uparrow$	F1(%) $\uparrow$	MAE $\downarrow$
DAB baseline	82.48	70.45	75.99	2.28	77.81	80.68	79.22	4.39
HED+HSV+L ∗ a ∗ b ∗	78.76	81.97	80.33	2.02	84.08	79.58	81.77	4.08
DeepLIIF	55.05	87.71	67.65	5.86	64.86	83.60	73.05	6.73
IHCScoreGAN	72.92	77.13	74.96	2.22	77.17	67.00	71.73	4.51
K-CLUE	73.94	86.17	79.59	2.69	86.38	76.53	81.16	4.45

Table 7: End-to-end Ki67-positive and Ki67-negative cell detection and classification on BCData using Hungarian matching with a

3\,\mu\mathrm{m}

tolerance.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Detected	TP	FP	FN	P(%) $\uparrow$	R(%) $\uparrow$	F1(%) $\uparrow$	MAE $\downarrow$
Cellpose-SAM	4773	4454	319	1331	93.32	76.99	84.37	18.57
Cellpose-SAMv2	4818	4488	330	1297	93.15	77.58	84.66	17.52
Cellpose-DINOv3 ViT-B	5075	4643	432	1142	91.49	80.26	85.51	13.82
Cellpose-DINOv3 ViT-L	5003	4586	417	1199	91.67	79.27	85.02	15.46

Appendix

Table 8: Detection performance of the four Cellpose variants on the MIST dataset using default inference parameters, against 5785 ground-truth annotations. Detected denotes the total number of predicted nuclei, TP, FP, FN, precision, recall, and F1-score are computed after matching predicted nuclei to the annotations (Hungarian matching with a

3\,\mu\mathrm{m}

tolerance), and MAE from the per-image cell counts.

Figure 6: Examples of crops with neighboring or real context background.

Model	Class	Precision(%) $\uparrow$	Recall(%) $\uparrow$	F1-Score(%) $\uparrow$	MAE $\downarrow$	MCC $\uparrow$
EfficientNet-B2	Ki67+	38.80	94.34	54.98	37.79	0.3023
EfficientNet-B2	Ki67-	93.19	34.24	50.08	50.43	0.3023
ResNet-101	Ki67+	40.63	48.67	44.29	16.52	0.1649
ResNet-101	Ki67-	75.14	68.57	71.71	22.02	0.1649
ConvNeXt V2 Tiny	Ki67+	67.13	40.28	50.35	14.68	0.3757
ConvNeXt V2 Tiny	Ki67-	77.57	91.29	83.87	13.71	0.3757

Appendix

Table 9: Ki67 cell typing performance on MIST with real context background, after PCA whitening. Precision, recall, F1-score, and MCC are computed over Cellpose-DINOv3 detections matched to ground-truth nuclei, while MAE is computed from the per-tile predicted and ground-truth cell counts.

Figure 7: Examples of crops extracted with a white background.

Model	Class	Precision(%) $\uparrow$	Recall(%) $\uparrow$	F1-Score(%) $\uparrow$	MAE $\downarrow$	MCC $\uparrow$
EfficientNet-B2	Ki67+	95.98	93.43	94.68	5.59	0.9240
EfficientNet-B2	Ki67-	97.13	98.27	97.70	11.48	0.9240
ResNet-101	Ki67+	94.87	94.48	94.67	5.34	0.9233
ResNet-101	Ki67-	97.56	97.74	97.65	12.12	0.9233
ConvNeXt V2 Tiny	Ki67+	95.27	95.87	95.57	4.88	0.9361
ConvNeXt V2 Tiny	Ki67-	98.17	97.90	98.03	12.20	0.9361

Appendix

Table 10: Ki67 cell typing performance on MIST with the white background fill, after PCA whitening. Precision, recall, F1-score, and MCC are computed over Cellpose-DINOv3 detections matched to ground-truth nuclei, while MAE is computed from the per-tile predicted and ground-truth cell counts.

		Ki67-positive cells				Ki67-negative cells
Methods	Whitening	P(%) $\uparrow$	R(%) $\uparrow$	F1(%) $\uparrow$	MAE $\downarrow$	P(%) $\uparrow$	R(%) $\uparrow$	F1(%) $\uparrow$	MAE $\downarrow$
DAB baseline	No	96.86	61.75	75.42	10.84	80.15	77.96	79.04	9.34
HED	No	97.22	53.07	68.66	13.36	77.25	77.96	77.60	9.89
HED	Yes	93.99	68.43	79.20	8.54	82.69	77.45	79.99	8.86
HSV	No	87.66	77.23	82.12	7.50	86.12	75.42	80.42	11.39
HSV	Yes	88.29	80.09	83.99	5.66	88.36	76.46	81.98	11.80

Appendix

Table 11: End-to-end comparison of color-space feature sets for Ki67-positive and Ki67-negative cell detection and classification on MIST, without and with PCA whitening, using one-to-one Hungarian matching with a

3\,\mu\mathrm{m}

tolerance.

		Ki67-positive cells				Ki67-negative cells
Methods	Whitening	P(%) $\uparrow$	R(%) $\uparrow$	F1(%) $\uparrow$	MAE $\downarrow$	P(%) $\uparrow$	R(%) $\uparrow$	F1(%) $\uparrow$	MAE $\downarrow$
DAB baseline	No	82.48	70.45	75.99	2.28	77.81	80.68	79.22	4.39
HED	No	82.92	66.15	73.59	2.58	75.87	80.74	78.23	4.62
HED	Yes	83.05	66.41	73.80	2.56	75.93	80.74	78.26	4.61
HSV	No	78.33	81.94	80.10	2.08	84.13	79.40	81.70	4.05
HSV	Yes	52.60	63.76	57.65	4.58	73.48	63.44	68.09	6.00

Appendix

Table 12: End-to-end comparison of color-space feature sets for Ki67-positive and Ki67-negative cell detection and classification on BCData, without and with PCA whitening, using one-to-one Hungarian matching with a

3\,\mu\mathrm{m}

tolerance.

Figure 8: Association between the first ten principal components of each frozen representation and the per-nucleus covariates measured from the image.

Model	PC	0.0	0.1	0.2	0.3	0.4	0.5	0.6	0.7	0.8	0.9	1.0
EfficientNet-B2	114	94.52	94.66	94.66	94.68	94.76	94.79	94.85	94.85	94.84	94.98	94.67
ResNet-101	201	87.81	88.51	89.53	90.04	91.52	92.40	92.65	93.31	93.60	94.11	94.40
ConvNeXt-V2 Tiny	64	92.61	92.64	92.67	93.06	93.25	93.51	93.79	93.98	94.34	94.69	94.78
DINOv3 ViT-S/16	54	92.81	93.07	93.29	93.63	93.66	93.99	94.24	94.63	94.77	95.12	94.58
HoVer-NeXt	16	50.37	50.45	50.63	50.60	50.83	50.63	51.87	93.62	93.67	94.25	94.19
UNI2	90	66.83	66.92	68.15	68.81	69.21	84.25	88.89	91.33	93.79	94.53	94.32

Appendix

Table 13: Ki67-positive F1-score (%) as a function of the whitening exponent

\alpha

. The PC column gives the number of components retained at the 90% variance cutoff.

Model	Class	Precision(%) $\uparrow$	Recall(%) $\uparrow$	F1-Score(%) $\uparrow$	MAE $\downarrow$	MCC $\uparrow$
EfficientNet-B2	Ki67+	89.51	98.23	93.67	2.73	0.9034
EfficientNet-B2	Ki67-	99.02	93.96	96.42	4.43	0.9034
ResNet-101	Ki67+	63.38	87.69	73.58	5.67	0.5813
ResNet-101	Ki67-	91.91	73.42	81.64	6.94	0.5813
ConvNeXt V2 Tiny	Ki67+	90.51	93.87	92.16	2.13	0.8797
ConvNeXt V2 Tiny	Ki67-	96.72	94.84	95.77	4.32	0.8797

Appendix

Table 14: Comparison of pretrained encoders for Ki67 cell typing on BCData before PCA whitening. Classification metrics are computed over Cellpose-DINOv3 detections matched to ground-truth nuclei, while MAE reports per-tile counting error over the predicted cell set.

Model	Class	Precision(%) $\uparrow$	Recall(%) $\uparrow$	F1-Score(%) $\uparrow$	MAE $\downarrow$	MCC $\uparrow$
EfficientNet-B2	Ki67+	90.37	98.44	94.23	2.77	0.9121
EfficientNet-B2	Ki67-	99.14	94.50	96.76	4.52	0.9121
ResNet-101	Ki67+	84.85	99.18	91.46	3.34	0.8709
ResNet-101	Ki67-	99.53	90.71	94.92	4.92	0.8709
ConvNeXt V2 Tiny	Ki67+	90.65	98.53	94.42	2.69	0.9150
ConvNeXt V2 Tiny	Ki67-	99.19	94.67	96.88	4.45	0.9150

Appendix

Table 15: Comparison of pretrained encoders for Ki67 cell typing on BCData after PCA whitening. Classification metrics are computed over Cellpose-DINOv3 detections matched to ground-truth nuclei, while MAE reports per-tile counting error over the predicted cell set.

Model	Probe AUC	PC1–DAB	Strongest DAB PC
EfficientNet-B2	0.9958	0.861	PC1 (0.861)
ResNet-101	0.9945	0.597	PC1 (0.597)
ConvNeXt-V2 Tiny	0.9949	0.854	PC1 (0.854)
DINOv3 ViT-S/16	0.9956	0.870	PC1 (0.870)
HoVer-NeXt	0.9957	0.256	PC2 (0.777)
UNI2	0.9959	0.297	PC2 (0.672)

Appendix

Table 16: Representation analysis on BCData. Probe AUC measures linear decodability of Ki67 status. DAB associations are absolute Pearson correlations with the corresponding principal component.

Histopathology Image Classification

Histopathology image classification uses computational models to identify tissue types, tumors, grades, and biomarkers from stained pathology images.

See all 146 papers

Apr 10, 2026

Cluster-First Labelling: An Automated Pipeline for Segmentation and Morphological Clustering in Histology Whole Slide Images

Labelling tissue components in histology whole slide images (WSIs) is prohibitively labour-intensive: a single slide may contain tens of thousands of structures--cells, nuclei, and other morphologically distinct objects--each requiring manual boundary delineation and classification. We present a cloudnative, end-to-end pipeline that automates this process through a cluster-first paradigm. Our system tiles WSIs, filters out tiles deemed unlikely to contain valuable information, segments tissue components with Cellpose-SAM (including cells, nuclei, and other morphologically similar structures), extracts neural embeddings via a pretrained ResNet-50, reduces dimensionality with UMAP, and groups morphologically similar objects using DBSCAN clustering. Under this paradigm, a human annotator labels representative clusters rather than individual objects, reducing annotation effort by orders of magnitude. We evaluate the pipeline on 3,696 tissue components across 13 diverse tissue types from three species (human, rat, rabbit), measuring how well unsupervised clusters align with independent human labels via per-tile Hungarian-algorithm matching. Our system achieves a weighted cluster-label alignment accuracy of 96.8%, with 7 of 13 tissue types reaching perfect agreement. The pipeline, a companion labelling web application, and all evaluation code are released as open-source software.

Muhammad Haseeb Ahmad, Sharmila Rajendran, Damion Young +1

Department of Engineering Science, University of Oxford, United Kingdom · Department of Physiology, Anatomy and Genetics, University of Oxford, United Kingdom · Medical Sciences Division, University of Oxford, United Kingdom

Whole-Slide Image Analysis Microscopy Image Analysis Computational Pathology Histopathology Image Segmentation Image Clustering

Ask PDF

Sep 21, 2026

MiTHras: Task-specific Hierarchical Semi-supervised Contrastive Masked Autoencoder for Mitotic Figure Analysis

Mitotic figure (MF) analysis supports tumor grading and prognostic assessment, but automated models remain sensitive to differences in tissue type and image acquisition. We present MiTHras, a task-specific pretraining framework that combines pseudo-label-guided image- and token-level contrastive learning with masked reconstruction. We construct TCGA-MF-Pseudo, a corpus of 1.8 million cell-centered images from 14 TCGA cohorts spanning 11 organ sites. Comprehensive evaluation on MF classification, detection, count-based survival prediction, and subtype classification demonstrates the efficacy of MiTHras. It achieves the highest mean F1 on all three MF classification benchmarks and both subtype benchmarks. MiTHras also outperforms general-purpose and pathology foundation encoders by a larger margin under frozen-encoder linear probing than under full fine-tuning. Although detection gains are modest due to a shared candidate-detection stage, ablations confirm that token-level supervision improves typical-versus-atypical classification and linear probing. These findings establish that MiTHras yields robust, transferable representations for automated mitotic activity assessment.

Trinh T. L. Vuong, Simon Graham, Quoc Dang Vu +5

Department of Pathology, Brigham and Women’s Hospital, Harvard Medical School, Boston, USA · Histofy Ltd, Coventry, United Kingdom · Department of Pathology, University Medical Center Ho Chi Minh City, Ho Chi Minh City, Vietnam +1

Contrastive Learning Masked Autoencoders Histopathology Image Classification Medical Image Representation Learning Microscopy Image Analysis

Ask PDF

Sep 15, 2026

InfoTaxa: Information-Calibrated Label-Free Clustering for Fine-Grained Visual Taxonomy

Label-free clustering of frozen pretrained visual embeddings offers a scalable route to biodiversity monitoring, but image-only fine-grained taxonomy exhibits a consistent coarse-to-fine failure mode: clusters recover broad taxonomic structure yet plateau at species level. We study this behaviour on BIOSCAN-5M through an information-calibrated clustering analysis. BioCLIP~2 features with UMAP and HDBSCAN reach

0.79

AMI at family and

0.67

at genus, substantially improving over the prior image baseline and remaining competitive with oracle-

K

, graph-based, and learned clustering heads on the same frozen features. To diagnose whether the remaining plateau is method-limited or information-limited, we introduce InfoTaxa, which combines clustering efficiency---the fraction of probe-estimated image information recovered by an unsupervised partition---with paired DNA as an audit signal only, not an inference input. The density pipeline recovers approximately

0.90

and

0.81

of the image-available information at order and family, respectively. Held-out late-fusion probes show that adding DNA to the image embedding reduces species-level prediction error by approximately two bits. Robustness analyses cover multiple image encoders, described-species and rare-class subsets, probe diagnostics, and held-out-species coarse-rank generalisation and same-species retrieval. Thus, in the tested setting, species-level label-free clustering is both clustering-limited and representation-limited: improved clustering may recover additional image-exposed structure, but cannot close the DNA-audited information gap alone.

David Ahmedt-Aristizabal, Mohammad Ali Armin, Lars Petersson

Imaging and Computer Vision Group, CSIRO, Australia

Visual Representation Learning Clustering Unsupervised Clustering Image Clustering

Ask PDF

Abstract

Histopathology Image Classification

Explore similar work

Cluster-First Labelling: An Automated Pipeline for Segmentation and Morphological Clustering in Histology Whole Slide Images

MiTHras: Task-specific Hierarchical Semi-supervised Contrastive Masked Autoencoder for Mitotic Figure Analysis

InfoTaxa: Information-Calibrated Label-Free Clustering for Fine-Grained Visual Taxonomy