Ensemble Classification and Cancer-Specific Gene Signatures from Quality-Aware Hybrid Gene Selection in TCGA RNA-Seq data

Random Forest--based selectors such as Boruta depend on tree depth, yet Boruta's most used implementation, BorutaPy, recommends depths of 3 to 7 without published evidence. We tested the effect of depth using pan-cancer data from The Cancer Genome Atlas (10,469 tumors, 33 cancer types) and breast cancer PAM50 subtyping (981 tumors, 5 subtypes). Label-dependent steps were fitted inside training folds, effects were measured against seed-to-seed variation, and each task kept a locked test set. In pan-cancer data, depth 5 changed the selection far more than a new seed did: the top-500 Jaccard overlap with unlimited depth was 0.37, against 0.84 (depth 5) and 0.93 (unlimited) between seeds. Depth-5 trees confirmed 472 instead of 1,659 genes, covered fewer large-class markers (16% vs 62%) and gave less accurate 50-gene panels (balanced accuracy 0.809 vs 0.889). The effect vanished once the cap reached 61 to 76% of the median depth of unrestricted trees, a depth that depends mainly on the number of classes, not sample size. In PAM50 subtyping (median depth 12) the effect was small, and leaving depth uncapped cost no accuracy in either task. It replicated in a second cross-validation repeat and under permutation importance. About a third of pre-filtered candidates correlated within class with the number of detected genes, a sample-quality measure; a within-class filter removed them without loss of test accuracy (0.943 vs 0.942 balanced accuracy, 1,000 genes). We recommend uncapped trees, or a cap above 75% of the realized median depth, and a within-class quality filter before Boruta.

Cross-cohort evaluation of breast cancer subtype classifiers is unreliable when source and target cohorts use different annotation standards: METABRIC labels six classes, PAM50 defines five, and a claudin-low prediction cannot be correct under a five-class standard. A classifier transferred from 1,369 METABRIC patients (train n=958) to TCGA-BRCA (n=987) and GSE21653 (n=266) separates three evaluation operations routinely conflated in cross-site comparisons. (1) Label-space restriction - an evaluation-rule change, not a retraining gain - is the dominant correction (73.76%->79.74% on TCGA-BRCA; 63.16%->68.42% on GSE21653); (2) EM prior correction on the reconciled space is an optional calibration step (Brier fidelity up, macro F1 down; to 80.65% and 74.06%); and (3) the target label space is not recoverable from unlabelled predictions (EM retains prior 0.0751 on the absent class; calibrated detectors fail), so compliance must come from cohort documentation. The protocol exposes why internal validation overstates cross-cohort value: multimodal fusion gains do not transfer (no configuration beats mRNA alone, n=694; McNemar p=0.13-1.00), and the genomic panel adds nothing to pathological complete response prediction over clinical variables (DeltaAUC = +0.0045). Clinically, without label-space compliance, cross-site accuracy comparisons misattribute annotation-standard differences to model skill; Luminal A/B and claudin-low boundary errors map to distinct treatment decisions; and subtype accuracy is a weaker portable endpoint than survival discrimination (external C-index 0.577-0.584). Reporting the cohort's allowed label set and applying label-space compliance are therefore prerequisites before any cross-cohort performance claim.

Accurate classification of breast cancer subtypes from gene expression data is critical for diagnosis and treatment selection. However, such datasets are characterized by high dimensionality and limited sample size, posing challenges for machine learning models. In this study, we evaluate the impact of model complexity and feature selection on subtype classification performance using TCGA-BRCA gene expression data. Logistic regression, random forest, and support vector machine (SVM) models were trained using varying numbers of highly variable genes (50 to 20,518). Performance was evaluated using stratified 5-fold cross-validation and assessed with accuracy and macro F1 score. While all models achieved high accuracy, macro F1 analysis revealed substantial differences in subtype-level performance. Logistic regression demonstrated the most stable and balanced performance across subtypes, including improved detection of rare classes. Random forest underperformed on minority subtypes despite strong overall accuracy, while SVM showed sensitivity to feature dimensionality. These findings highlight the importance of model simplicity, evaluation metrics, and feature selection in high-dimensional biological classification tasks.