Label-Space Mismatch Distorts Cross-Cohort Evaluation in Breast Cancer Molecular Subtyping

Suxing Liu; Byungwon Min; Zhijian Zheng; Jiahao Wang

cs.LGOct 18, 2026

Label-Space Mismatch Distorts Cross-Cohort Evaluation in Breast Cancer Molecular Subtyping

Authors: Suxing Liu, Byungwon Min, Zhijian Zheng, Jiahao Wang

Organizations: Department of IT Engineering, Mokwon University, Daejeon 35349, Republic of Korea · School of Digital Media Technology, Jiangxi Arts & Ceramics Technology Institute, Jingdezhen 333001, China · University of Alabama, Tuscaloosa, AL 35487, USA · School of Computer Sciences, Universiti Sains Malaysia, Minden Heights, Penang 11800, Malaysia

Abstract

Cross-cohort evaluation of breast cancer subtype classifiers is unreliable when source and target cohorts use different annotation standards: METABRIC labels six classes, PAM50 defines five, and a claudin-low prediction cannot be correct under a five-class standard. A classifier transferred from 1,369 METABRIC patients (train n=958) to TCGA-BRCA (n=987) and GSE21653 (n=266) separates three evaluation operations routinely conflated in cross-site comparisons. (1) Label-space restriction - an evaluation-rule change, not a retraining gain - is the dominant correction (73.76%->79.74% on TCGA-BRCA; 63.16%->68.42% on GSE21653); (2) EM prior correction on the reconciled space is an optional calibration step (Brier fidelity up, macro F1 down; to 80.65% and 74.06%); and (3) the target label space is not recoverable from unlabelled predictions (EM retains prior 0.0751 on the absent class; calibrated detectors fail), so compliance must come from cohort documentation. The protocol exposes why internal validation overstates cross-cohort value: multimodal fusion gains do not transfer (no configuration beats mRNA alone, n=694; McNemar p=0.13-1.00), and the genomic panel adds nothing to pathological complete response prediction over clinical variables (DeltaAUC = +0.0045). Clinically, without label-space compliance, cross-site accuracy comparisons misattribute annotation-standard differences to model skill; Luminal A/B and claudin-low boundary errors map to distinct treatment decisions; and subtype accuracy is a weaker portable endpoint than survival discrimination (external C-index 0.577-0.584). Reporting the cohort's allowed label set and applying label-space compliance are therefore prerequisites before any cross-cohort performance claim.

Figures & tables

Modality Combination	Dim	Val Acc (%)	5-fold CV (%)
Clinical only	69	63.9	64.1 $\pm$ 2.7
mRNA only	412	74.6	73.8 $\pm$ 1.6
Methylation only	232	56.6	54.7 $\pm$ 2.4
Clinical + mRNA	481	75.6	74.5 $\pm$ 1.9
Clinical + Methylation	301	68.3	65.7 $\pm$ 1.9
mRNA + Methylation	644	74.6	73.1 $\pm$ 2.6

TABLE I: Modality-combination ablation on the METABRIC validation set (n=205). Five-fold cross-validation results use the training subset (n=958). The clinical + mRNA + methylation model exceeds the all-modality model by 3.4 percentage points on the validation split.

Model	CV Accuracy (%)
Logistic Regression ( $C=0.01$ )	75.3
Logistic Regression ( $C=0.1$ )	74.1
Logistic Regression ( $C=1.0$ )	72.9
Calibrated LogReg (sigmoid)	74.4
Random Forest (300 trees)	72.4
XGBoost	72.0

TABLE II: Model comparison using five-fold cross-validation on the METABRIC training subset (n=958). Models use the selected clinical + mRNA + methylation feature set (713 dimensions), unless otherwise stated.

Subtype	Precision	Recall	F1-score	Support
Luminal A	0.700	0.851	0.768	74
Luminal B	0.812	0.780	0.796	50
HER2-enriched	0.741	0.769	0.755	26
Claudin-low	0.733	0.579	0.647	19
Basal-like	0.778	0.667	0.718	21
Normal-like	0.625	0.312	0.417	16

TABLE III: Per-class classification metrics on the METABRIC test set (n=206).

Fig. 1 : Confusion matrix for the six-class PAM50 prediction task on the held-out METABRIC test set (n=206) for the 3-modality logistic regression (Clinical $+$ mRNA $+$ Methylation, 713 dimensions; accuracy 73.8%). Cell values are predicted counts with row percentages. Luminal A/Luminal B confusion dominates the error budget.

Fig. 2 : External validation on TCGA-BRCA (n=987). The within-platform rank-normalized mRNA model reaches 73.8% accuracy and a survival C-index of 0.577 ( $p$ =0.047).

Configuration	Dims	METABRIC CV	METABRIC test	TCGA	TCGA macro F1	$p$
mRNA	412	72.91	72.82	80.55	0.768	—
mRNA + clinical	422	73.43	75.24	80.40	0.774	1.00
mRNA + methylation	644	72.91	72.33	78.67	0.747	0.13
clinical + mRNA + methylation	654	73.17	74.76	79.11	0.760	0.30
original flagship	713	75.92	73.79	n/a	n/a	—

TABLE IV: Modality ablation in the harmonized three-layer space shared by METABRIC and TCGA-BRCA. CV is five-fold cross-validated accuracy on pooled METABRIC train+val (n=1,163). TCGA accuracy is on n=694 patients annotated with one of the five PAM50 classes present in that cohort. The McNemar column is a paired test against mRNA alone. The final row is the original 713-dimension model, shown for reference: its clinical block includes METABRIC-specific fields and has no external counterpart.

Fig. 3 : The multimodal advantage does not transfer. Panel A: internal cross-validated and held-out METABRIC accuracy for each configuration, with the original 713-dimension flagship as a reference line. Panel B: TCGA-BRCA accuracy over the five PAM50 classes present (n=694) with bootstrap 95% confidence intervals. Every configuration’s interval overlaps the mRNA-only interval; the harmonized clinical block helps internally and is negligible externally.

	TCGA-BRCA (n=987)		GSE21653 (n=266)
Estimator	Acc (%)	Macro F1	Acc (%)	Macro F1
base (no reconciliation)	73.76	0.585	63.16	0.504
mask_only	79.74	0.757	68.42	0.626
em_pam50	80.65	0.737	74.06	0.683
em_full (no mask)	75.99	0.586	68.42	0.541

TABLE V: Decomposition of the cross-cohort correction. Within each cohort, all rows share one classifier and feature set, using logistic regression (

C

=0.01) and within-platform rank normalization. TCGA-BRCA uses 412 genes; GSE21653 uses the 369 genes mapped to GPL570, with a source model refitted on those genes. Rows differ only in label-space handling. mask_only uses the annotation-standard fact that claudin-low is not a PAM50 subtype; em_pam50 adds EM prior correction on the reconciled space; em_full runs EM without label-space knowledge. Neither the mask nor any row uses target labels.

Fig. 4 : Accuracy and macro F1 for the same mRNA-only estimator across METABRIC validation, METABRIC test, and TCGA-BRCA. TCGA accuracy is similar to the internal estimates, while the six-class macro F1 is affected by the absent claudin-low class.

Estimator	CV acc	Base acc	Mask acc	$\Delta$
Logistic Regression ( $C$ =0.01)	72.9%	73.34%	80.55 %	+7.20
Random Forest (300 trees)	70.4%	75.36%	78.82 %	+3.46
XGBoost (300 trees)	72.2%	72.05%	77.95 %	+5.91

TABLE VI: Mask-only label-space correction is effective across estimator families on TCGA-BRCA (harmonised mRNA,

n

=694). Base: argmax over the full six-class space. Mask: the absent claudin-low class is dropped from the candidate set and the remaining five re-argmaxed. All estimators are trained on METABRIC train+val (

n

=1,163).

Detector	Micro P	Micro R	F1	Exact	Null FP
Uncalibrated EM ( em_orig )	0.920	0.685	0.786	0.550	17/120
Calibrated EM ( em_cal )	0.577	0.876	0.696	0.362	73/120
Importance-weighted EM ( em_cal_iw )	0.540	0.544	0.542	0.242	56/120
Probability frequency ( pfreq )	0.548	0.902	0.682	0.372	87/120
Geometric coherence ( geo )	0.467	0.154	0.232	0.193	27/120
Never predicted ( naive )	1.000	0.006	0.012	0.167	0/120

TABLE VII: Absent-class detectors scored with the standard empty-set convention (20 seeds

\times

6 absent sets

\times

3 covariate-shift levels

\times

2 prior-shift settings

=

720 cohorts). Null false positives count cohorts with no absent class on which the detector returned a non-empty set. The naïve exact-match rate of 0.167 is the fraction of settings in which nothing was in fact absent.

TCGA-BRCA (n=987)
Cohort	Estimator	Acc (%)	Macro F1	ECE	Brier
6-class base		73.76	0.585	0.0352	0.3756
6-class mask_only		79.74	0.757	0.0557	0.2906
6-class em_pam50		80.65	0.737	0.0460	0.2745
5-class drop (n=867)		80.04	0.774	0.0560	0.2892
5-class merge		76.90	0.695	0.0344	0.3284

TABLE VIII: Five-class training baselines against the six-class estimator with label-space reconciliation. All rows use one rank-normalised mRNA feature set and logistic regression (

C{=}0.01

); rows differ only in training label space and post-hoc handling. Paired McNemar comparisons against the 6-class base are reported in the text; where

p\geq 0.05

the difference is not statistically distinguishable and is described as such rather than as an advantage.

Fig. 5 : Label-space reconciliation and prior correction. (A) Accuracy before and after label-space reconciliation on TCGA-BRCA and GSE21653, showing the mask-only and EM-corrected stages separately. (B) The identifiability failure: unconstrained EM retains substantial prior mass on claudin-low, whose true prevalence in both cohorts is zero, and recovers only a fraction of the available gain. (C) Controlled simulation separating label-space mismatch from prior shift; the correction is near zero without prior shift and consistently large with it.

Predicted Subtype	HR	95% CI	p-value
Luminal B	1.50	[1.14, 1.97]	$<0.005$
HER2-enriched	1.43	[1.08, 1.89]	0.01
Basal-like	1.46	[1.07, 1.99]	0.02
Claudin-low	0.82	[0.58, 1.16]	0.25
Normal-like	1.31	[0.86, 2.00]	0.21

TABLE IX: Cox regression hazard ratios for predicted subtypes (Luminal A = reference) on METABRIC (n=411).

Fig. 6 : SHAP-based modality attribution. mRNA expression accounts for 63.3% of model decisions, methylation 26.7% and clinical features 10.0%; copy-number and mutation blocks contribute under 1% each.

Treated subgroup	n	Events	AUC (panel)	AUC (subtype)	$\Delta$
Chemotherapy	275	120	0.624	0.571	$+0.053$
Hormone therapy	768	210	0.679	0.577	$+0.102$
Radiotherapy	767	225	0.718	0.640	$+0.078$

TABLE X: 5-year outcome discrimination within treatment subgroups, genomic panel versus PAM50 subtype alone. These are administered-treatment cohorts, not response assessments.

Fig. 7 : Treatment-response prediction in GSE20194 (n=278; 56 pCR, 222 residual disease). (A) Discrimination of pCR after neoadjuvant chemotherapy. The genomic panel does not improve on clinical variables: the increment is

+0.0045

with a 95% CI spanning zero, and the genomic-only model (0.773) is worse than clinical-only (0.791). (B) pCR rate by predicted subtype. Despite adding no predictive value over clinical features, the predicted subtype reproduces the observed response pattern, with pCR rates of 53.6% for HER2-enriched and 42.4% for basal-like compared with 6.0% for Luminal A.

Cancer Subtype Classification

See all 14 papers

Multimodal Survival Modeling and Fairness-Aware Clinical Machine Learning for 5-Year Breast Cancer Risk Prediction
Feb 25, 2026Toktam KhatibiCancer Survival Prediction Multimodal Survival Analysis
Faculty of Industrial and Systems Engineering, Tarbiat Modares University, Tehran, Iran
Cross-Dataset Generalization in Breast MRI Tumor Classification via Class-Wise Dataset Mixing
Jul 21, 2026Mohammad Ali Dadrast, Hamid UsefiAlgorithmic Bias Medical Image Classification
Data Science Program Memorial University of Newfoundland St. John’s, Canada · Department of Mathematics and Statistics Memorial University of Newfoundland St. John’s, Canada
The MAMA-MIA Challenge: Advancing Generalizability and Fairness in Breast MRI Tumor Segmentation and Treatment Response Prediction
Mar 1, 2026Lidia Garrucho, Smriti Joshi, Kaisar Kushibar +43Medical Image Segmentation Medical Image Benchmarks

Feb 25, 2026J/K move · Enter open · S save

Multimodal Survival Modeling and Fairness-Aware Clinical Machine Learning for 5-Year Breast Cancer Risk Prediction

Toktam Khatibi

Faculty of Industrial and Systems Engineering, Tarbiat Modares University, Tehran, Iran

Cancer Survival Prediction Multimodal Survival Analysis Survival Analysis Breast Cancer

Open paper AskPDF arXiv

Clinical risk prediction models often underperform in real-world settings due to poor calibration, limited transportability, and subgroup disparities. These challenges are amplified in high-dimensional multimodal cancer datasets characterized by complex feature interactions and a p >> n structure. We present a fully reproducible multimodal machine learning framework for 5-year overall survival prediction in breast cancer, integrating clinical variables with high-dimensional transcriptomic and copy-number alteration (CNA) features from the METABRIC cohort. After variance- and sparsity-based filtering and dimensionality reduction, models were trained using stratified train/validation/test splits with validation-based hyperparameter tuning. Two survival approaches were compared: an elastic-net regularized Cox model (CoxNet) and a gradient-boosted survival tree model implemented using XGBoost. CoxNet provides embedded feature selection and stable estimation, whereas XGBoost captures nonlinear effects and higher-order interactions. Performance was assessed using time-dependent area under the ROC curve (AUC), average precision (AP), calibration curves, Brier score, and bootstrapped 95 percent confidence intervals. CoxNet achieved validation and test AUCs of 98.3 and 96.6, with AP values of 90.1 and 80.4. XGBoost achieved validation and test AUCs of 98.6 and 92.5, with AP values of 92.5 and 79.9. Fairness diagnostics showed stable discrimination across age groups, estrogen receptor status, molecular subtypes, and menopausal state. This work introduces a governance-oriented multimodal survival framework emphasizing calibration, fairness auditing, robustness, and reproducibility for high-dimensional clinical machine learning.

Abstract

Cancer Subtype Classification

Explore similar work

Multimodal Survival Modeling and Fairness-Aware Clinical Machine Learning for 5-Year Breast Cancer Risk Prediction

Cross-Dataset Generalization in Breast MRI Tumor Classification via Class-Wise Dataset Mixing

The MAMA-MIA Challenge: Advancing Generalizability and Fairness in Breast MRI Tumor Segmentation and Treatment Response Prediction