Random Forest--based selectors such as Boruta depend on tree depth, yet Boruta's most used implementation, BorutaPy, recommends depths of 3 to 7 without published evidence. We tested the effect of depth using pan-cancer data from The Cancer Genome Atlas (10,469 tumors, 33 cancer types) and breast cancer PAM50 subtyping (981 tumors, 5 subtypes). Label-dependent steps were fitted inside training folds, effects were measured against seed-to-seed variation, and each task had a locked test set. In pan-cancer data, depth 5 changed the selection far more than a new seed did: the top-500 Jaccard overlap with unlimited depth was 0.37, against 0.84 (depth 5) and 0.93 (unlimited) between seeds. Depth-5 trees confirmed 472 instead of 1,659 genes, covered fewer large-class markers (16% vs 62%) and gave less accurate 50-gene panels (balanced accuracy 0.809 vs 0.889). The effect vanished once the cap reached 61 to 76% of the median depth of unrestricted trees, a depth set mainly by the number of classes, not sample size. In PAM50 subtyping (median depth 12) the effect was small, and leaving depth uncapped cost no accuracy in either task. The effect replicated in a second cross-validation repeat and under permutation importance. About a third of pre-filtered candidates correlated within class with the number of detected genes, a sample-quality measure; a within-class filter removed them without loss of test accuracy (0.943 vs 0.942 balanced accuracy, 1,000 genes). We recommend uncapped trees, or a cap above 75% of the realized median depth, and a within-class quality filter before Boruta.
Figures & tables
Figure 1: Study design. Two tasks from TCGA RNA-seq data: pan-cancer classification (10,469 tumors, 33 classes) and breast cancer PAM50 subtyping (981 tumors, 5 classes). Each task has a locked test set (20% of patients). Inside every training fold: expression filter, within-class quality filter (quality-aware pipeline only), category-aware ANOVA F pre-filter to 3,000 candidates, and Boruta at each depth setting with three random seeds. Selections are compared with seed-to-seed variation; panels of the top K genes are evaluated with logistic regression (LR) and LightGBM.
Pan-cancer
PAM50
Classes
33 cancer types
5 subtypes
Tumors after QC
10,469
981
Development set
8,374
797
Locked test set
2,095
184
Training samples per CV fold
about 6,700
637
Smallest class
CHOL (35)
Normal-like (36)
Table 1: Cohorts and Splits
Figure 2: Effect of Random Forest maximum depth on Boruta feature selection in TCGA pan-cancer data (33 classes; 5-fold cross-validation, 3 seeds per depth). Shaded: depth range recommended in the BorutaPy documentation (3–7); “unl.” = unlimited depth. (a) Genes confirmed by Boruta (mean ± SD over folds and seeds). (b) Jaccard overlap of the top 500 genes: blue, between each depth and unlimited depth, compared across different seeds; dashed, between two seeds at the same depth. (c) Stability of the top 500 genes across folds (Nogueira index). (d) Validation balanced accuracy of logistic regression and LightGBM trained on the top 50 genes (mean ± SD over folds); differences between depths are consistent within folds (Section 4.2 ). (e) Share of each class’s 20 strongest one-vs-rest marker genes found among the top 500, averaged over the 8 smallest and the 8 largest classes. (f) Share of non-coding “other” gene types, mostly processed pseudogenes, among the top 500.
Figure 3: The depth effect follows the depth that unrestricted trees reach. (a) Depth gap, defined as the seed-to-seed overlap of the top 500 genes minus their overlap with unlimited-depth selections from different seeds, in the pan-cancer and PAM50 tasks; 0 means changing the depth alters the selection no more than changing the seed. Shaded: BorutaPy-recommended range. (b) Realized depth of unrestricted Random Forest trees (100 trees per training fold, 5 folds), on the same depth scale as (a); boxes show quartiles, whiskers the 5th–95th percentiles.
Depth
3
5
8
10
15
20
25
30
Unlimited
Confirmed genes
249
472
911
1,224
1,550
1,638
1,661
1,660
1,659
Seed-to-seed Jaccard
0.823
0.842
0.831
0.779
0.906
0.923
0.931
0.924
0.930
Jaccard vs unlimited (cross-seed)
0.322
0.370
0.479
0.581
0.782
0.911
0.931
0.926
–
Nogueira Φ
0.849
0.863
0.869
0.858
0.888
0.900
0.897
0.898
0.900
Balanced accuracy, LR, K=50
0.806
0.809
0.840
0.858
0.881
0.886
0.888
0.893
0.889
Balanced accuracy, LightGBM, K=50
0.827
0.831
0.838
0.851
0.876
0.872
0.874
0.868
0.870
Table 2: Pan-Cancer Selections by Random Forest Depth (5 Folds, Seeds 0–2; Top 500 Genes Unless Stated)
Figure 4: Candidate genes associated with sample quality in the pan-cancer task. A gene is quality-associated when its within-class correlation with log10 of the number of detected genes exceeds 0.3 in absolute value (training fold). (a) Share of quality-associated candidates by RNA category (blue) and by the most affected gene types (gray), over 5 folds. (b) Share among the top 500 Boruta genes at each depth; dashed, share among all 3,000 candidates. (c) Share among the top 500 genes re-measured on validation samples, without and with the within-class quality filter.
Task, classifier, K
Quality-aware (95% CI)
Baseline
Depth 5
PAM50 genes
Quality-aware minus baseline (95% CI)
Pan-cancer, LR, 1,000
0.943 (0.927–0.958)
0.942
–
–
+0.001 ( −0.006 to 0.007)
Pan-cancer, LightGBM, 1,000
0.928 (0.912–0.945)
0.923
–
–
+0.006 ( −0.001 to 0.013)
Pan-cancer, LR, 100
0.921 (0.904–0.936)
0.925
–
–
−0.005 ( −0.013 to 0.002)
Pan-cancer, LightGBM, 100
0.905 (0.887–0.923)
0.904
–
–
+0.001 ( −0.012 to 0.013)
PAM50, LR, 50
0.889 (0.814–0.951)
0.897
0.902
0.894
−0.008 ( −0.044 to 0.017)
PAM50, LightGBM, 20
0.833 (0.741–0.920)
0.834
0.811
–
−0.001 ( −0.039 to 0.031)
Table 3: Balanced Accuracy on the Locked Test Sets
Figure 5: Balanced accuracy on the locked test sets, with 95% confidence intervals from 2,000 bootstrap resamples drawn within class. (a) Pan-cancer task (2,095 samples, 33 classes), panels of 1,000 genes (chosen by the one-standard-error rule) and 100 genes. (b) PAM50 task (184 samples, 5 subtypes), panels of 20 or 50 genes; the 50 PAM50 genes are an upper reference, as the labels were derived from them.
Depth
Depth gap
Seed Jaccard
Nogueira Φ
Confirmed
5
0.468 / 0.468
0.838 / 0.839
0.863 / 0.872
472 / 469
8
0.353 / 0.352
0.831 / 0.831
0.869 / 0.877
911 / 920
10
0.199 / 0.203
0.779 / 0.784
0.858 / 0.859
1,224 / 1,227
15
0.121 / 0.130
0.904 / 0.909
0.888 / 0.893
1,550 / 1,554
Unl.
0.000 / 0.000
0.927 / 0.932
0.900 / 0.907
1,659 / 1,670
Values: repeat 0 / repeat 1 (5 folds × 3 seeds each; top-500 Jaccard; depth gap measured across different seeds). Unl.: unlimited depth.
Table 4: Pan-Cancer Selections in Two Independent CV Repeats
Figure 6: Realized depth and the depth effect versus sample size and number of classes (pan-cancer subsamples). (a) Median realized depth of unrestricted trees (100 trees per training fold) against training-set size, all 33 classes; mean ± SD over 5 folds. (b) Median realized depth against the number of randomly drawn cancer types at 637 training samples (3 draws per fold). (c) Depth gap of the top 500 genes for the full pan-cancer data, for a 637-sample subsample with all 33 classes (depths 5–20 and unlimited), and for PAM50 (637 training samples, 5 subtypes). At 637 samples fewer than 500 genes were confirmed, so the top 500 also includes tentative genes ranked by hit rate.
Gradient saliency from deep sequence models surfaces candidate biomarkers efficiently, but the resulting gene lists can be contaminated by tissue-composition confounders that degrade downstream classifiers. We study whether LLM chain-of-thought (CoT) reasoning can filter these confounders, and whether reasoning quality is associated with downstream performance. We train a Mamba SSM on TCGA-BRCA RNA-seq and extract the top-50 genes by gradient saliency; DeepSeek-R1 evaluates every candidate with structured CoT to produce a final 17-gene set. On the held-out test split, the raw 50-gene saliency set (no LLM) performs worse than a 5,000-gene variance baseline (AUC 0.832 vs. 0.903), while the LLM-filtered set surpasses it (AUC 0.927), using 294x fewer features. A faithfulness audit (COSMIC CGC, OncoKB, PAM50) shows that 6 of 17 selected genes (35.3%) are validated BRCA biomarkers, while 10 of 16 known BRCA genes present in the input were missed - including FOXA1. This divergence between downstream performance and reasoning faithfulness suggests selective faithfulness in this setting: targeted confounder removal can improve predictive performance without comprehensive recall.
Pushpa Kumar Balan, Aijing Feng
Department of Computer Science and Cybersecurity, University of Central Missouri
Most feature selection algorithms, especially wrapper methods, run inefficiently on CPU based platforms because of their high computational complexity. This inefficiency makes them unsuitable for processing large scale datasets. To address this challenge, the present study proposed two GPU accelerated versions of the Boruta feature selection procedure, in which Boruta-Permut relies on permutation based feature importance and Boruta-TreeImp employs importance based on impurity reduction. To evaluate these methods we conducted experiments on both a self constructed dataset and several publicly available datasets. The experimental results show that the proposed GPU accelerated algorithms greatly improve computational efficiency while preserving feature selection accuracy comparable to the original Boruta algorithm. In our analysis we also observe that the impurity reduction based version can overestimate the importance of some features. Overall these findings suggest that performing Boruta feature selection on GPUs offers an effective and cost efficient solution for large scale data analysis, which is a good deal.
Xurui Li, Zhiguo Gan, Jiaming Zhang +2
Department of Chemical Engineering, Tsinghua University, 2026 Shuangqing Road, Beijing, 100084, China.
Single-cell ribonucleic acid sequencing (scRNA-seq) is a foundational technology for precision-medicine workflows that contribute to United Nations Sustainable Development Goal 3 on Good Health and Well-being, and unsupervised clustering is the analytical step that turns raw expression matrices into interpretable cell populations. Practitioners therefore face a recurring engineering decision: is an additional deep representation stage worth its compute and tuning cost, or do classical principal component analysis (PCA) pipelines already suffice? We address this question with a diagnostic benchmark of nine clustering pipelines on ten real datasets (90-5,685 cells, 19,046-41,480 genes, 4-11 cell types), augmented by a partial scVI V2 specialized comparison on seven datasets. The protocol integrates Optuna hyperparameter search, repeated-run robustness, Friedman/Wilcoxon-Holm/TOST testing, and Sobol total-order sensitivity analysis. The contrastive autoencoder achieved the highest mean Adjusted Rand Index (0.7872), but Holm-corrected tests did not establish dominance over the strongest baselines. Per-dataset analysis reveals three reproducible regimes: probabilistic variational autoencoder (VAE) variants help on the smallest datasets, deep autoencoders win on mid-scale data with multi-batch or many-type structure, and classical PCA pipelines remain competitive when linear projection already captures the dominant variation. Sobol indices identify learning rate (ST=0.70) and latent dimensionality (ST=0.56) as the dominant variance contributors, indicating where limited tuning budgets should be allocated. The contribution is therefore a dataset-aware and compute-conscious decision framework for biomedical AI pipelines supporting sustainable healthcare analytics, rather than a universal superiority claim.
Nguyen Thanh Phong, Truong Viet Vu, Nguyen Ha Thu +4
Faculty of Information Technology, Van Lang School of Technology, Van Lang University, Ho Chi Minh City, Vietnam · Faculty of Law, Van Lang University, Ho Chi Minh City, Vietnam