Global-Local and Window-Concatenated Functional Random Forests: Early and Late Fusion of Window-Specific Functional Principal Components
Authors: Fabrizio Maturo, Annamaria Porreca
Organizations: Department of Economics, Statistics and Business, Faculty of Technological and Innovation Sciences, Universitas Mercatorum, Piazza Mattei 10, 00186 Rome, Italy. · Department for the Promotion of Human Science and Quality of Life, San Raffaele University, Via di Val Cannuta 247, 00166 Rome, Italy · IRCCS San Raffaele Roma, Via della Pisana 235, 00163 Rome, Italy.
Functional random forests built on a single functional principal component decomposition of the whole domain may dilute features confined to short intervals, whose location is rarely known in advance. We propose two random forests based on window-specific functional principal components. The global-local functional random forest (GL-FRF) is a late-fusion method that stacks a tuned full-curve forest and one forest per window with penalised convex weights; it contains the full-curve forest as a feasible solution, and its weights indicate which parts of the domain drive the prediction. The window-concatenated functional random forest (WC-FRF) is an early-fusion method that trains a single tuned forest on the concatenated window-specific scores. On twenty UCR datasets, GL-FRF improved on a tuned full-curve forest with the same number of trees on 16, with Brier score reductions of up to 0.21, significantly after Holm adjustment. No method was best on more than six datasets. WC-FRF was more accurate than GL-FRF on 13 datasets but failed when the signal required the full curve, whereas GL-FRF had the smallest mean and maximum excess over the best method across all 32 simulated and real settings, which makes it the safer choice when the location of the signal is unknown.
Figures & tables
Scenario
Class signal si(t)
Expected behaviour
S1 Global
zi{0.75sin(2πt)+0.5cos(4πt)}
global signal
S2 LocalEarly
1.3ziϕ(t;0.16,0.035)
early transient
S3 LocalLate
1.3ziϕ(t;0.83,0.035)
late transient
S4 OppositePatches
1.1zi{ϕ(t;0.20,0.028)−ϕ(t;0.77,0.028)}
two opposite transients
S5 CrossWindowXOR
1.45{Aiϕ(t;0.19,0.04)+Biϕ(t;0.82,0.04)}
cross-window interaction
S6 Null
0
no signal
Table 1: Class signals of the simulated scenarios in model ( 17 ). Further details are given in Section 5.1 .
Figure 1: Simulated training datasets for the twelve scenarios (first replicate). Each panel displays 25 training curves per group, with colours indicating the outcome group.
Figure 2: Held-out Brier score of all methods in Scenarios 1–6 over 20 independent replicates. Methods are ordered by median; the white diamond indicates the mean. Vertical scales differ across panels.
Figure 3: Held-out Brier score of all methods in Scenarios 7–12 over 20 independent replicates. Methods are ordered by median; the white diamond indicates the mean. Vertical scales differ across panels.
Figure 4: Simulation study: mean paired difference in held-out Brier score between each competitor and GL-FRF. Blue cells indicate that GL-FRF has a lower Brier score, red cells that the competitor has a lower Brier score.
Figure 5: Training curves of the twenty UCR datasets: up to 20 curves per class from the official training set, coloured by class. The number of classes is given in parentheses; t indexes the points of the sampling grid and X(t) is the value provided in the archive.
Figure 6: UCR datasets (first ten in alphabetical order): held-out Brier score of all methods over the official split and four stratified resamples (FordA: official split only). Methods are ordered by median; the white diamond indicates the mean.
Figure 7: UCR datasets (last ten in alphabetical order): held-out Brier score of all methods over the official split and four stratified resamples. Methods are ordered by median; the white diamond indicates the mean.
Figure 8: UCR datasets: mean paired difference in held-out Brier score between each competitor and GL-FRF over the analysed splits. Blue cells indicate that GL-FRF has a lower Brier score.
Figure 9: UCR datasets: distribution over datasets and splits of the paired difference in held-out Brier score between each competitor of the main comparisons and GL-FRF. Positive values favour GL-FRF.
Competitor
Datasets
Median diff.
GL-FRF better/worse
V
p
pHolm
G-tuned-bud
20
0.0146
16/ 4
186
0.001
0.006
Scr-stack
20
0.0153
16/ 4
181
0.003
0.009
WC-FRF
20
-0.0071
7/13
65
0.143
0.286
IF-TSF
20
0.0019
12/ 8
121
0.571
0.571
Stack-0
20
0.0015
14/ 6
170
0.014
–
WC-0
20
-0.0042
7/13
84
0.452
–
Table 8: UCR datasets: median over datasets of the dataset-level mean difference in Brier score (competitor minus GL-FRF; positive values favour GL-FRF), number of datasets on which GL-FRF was better or worse, and two-sided Wilcoxon signed-rank test. The first 4 rows form the family of main comparisons (Holm adjustment); the remaining rows are descriptive.
Figure 10: Paired difference in held-out Brier score between WC-FRF and GL-FRF (mean ± two standard errors over replicates or splits). Positive values favour GL-FRF, negative values WC-FRF.
Group (settings)
GL-FRF
WC-FRF
Stack-0
Scr-stack
G-tuned-bud
IF-TSF
FkNN
Simulation (12)
0.003 ( 0.013 )
0.127 (0.416)
0.003 (0.016)
0.004 (0.022)
0.003 (0.027)
0.132 (0.298)
0.112 (0.287)
UCR (20)
0.031 (0.187)
0.021 ( 0.112 )
0.035 (0.187)
0.067 (0.366)
0.073 (0.399)
0.033 (0.156)
0.100 (0.446)
All (32)
0.021 ( 0.187 )
0.060 (0.416)
0.023 ( 0.187 )
0.043 (0.366)
0.047 (0.399)
0.070 (0.298)
0.105 (0.446)
Table 9: Excess mean Brier score over the best of the twelve methods in each simulated scenario or dataset: mean (maximum) over the settings of each group. Smallest mean and smallest maximum of each row in bold. Descriptive summary.
Figure 11: Weight a0 assigned to the tuned global expert by GL-FRF, over the replicates of each simulated scenario and the splits of each UCR dataset. Violin: distribution; box: interquartile range; line: median; white diamond: mean.
Figure 12: Mean stacking weight assigned by GL-FRF to the local experts, by relative position of the window centre on the domain. For the simulated scenarios the location of the class signal is known (Table 1 ).
Building trustworthy machine learning algorithms for biological applications requires adapting to data heterogeneity from different sources, batches, distributions, or studies. We propose the 'Cross-Cluster Weighted Forest' (CCWF), an ensembling approach that explicitly leverages heterogeneity in the feature distribution to produce more accurate and more generalizable predictors than the standard Random Forest in cases when data can be naturally clustered. CCWF generalizes the RF architecture to an outer unsupervised layer, supervised subtasks, and ensembling. Specifically it involves unsupervised clustering of the training data, fitting a Random Forest on each cluster, and combining the forests via stacked regression weights that reward cross-cluster generalizability. We provide a theoretical analysis of an analytically tractable forest model showing that cluster-based ensembling is asymptotically more accurate than training a single forest on the full data, with the gain driven by bias reduction. In simulations, we find that CCWF is robust across data-generating regimes and outcome models; furthermore, we explore the influence of data partitioning and ensemble weighting strategies on the benefits of our method. Finally, we apply our approach to cancer molecular profiling and gene expression datasets that are naturally divisible into clusters; in both simulations and real data examples, we illustrate that our approach outperforms classic Random Forest by margins of 30-40%, aligning with our theoretical results. Overall, we show that CCWF provides a statistically grounded prediction algorithm for data spanning multiple domains or sub-populations, a structure common in biological applications.
Maya Ramchandran, Rajarshi Mukherjee, Giovanni Parmigiani
Department of Biostatistics, Harvard T.H. Chan School of Public Health · Department of Data Science, Dana-Farber Cancer Institute
Multi-center survival prediction can improve robustness and generalizability, yet privacy regulations and institutional governance often prevent pooling patient-level clinical and genomic data across institutions. In practice, deployment is further complicated by feature-space heterogeneity, in which sites collect different covariates or use different sequencing panels, resulting in only partially overlapping feature sets. We present FederatedRSF, a Python package that implements federated random survival forests, aggregating locally trained survival trees and redistributing only feature-compatible trees to each site, enabling inference with partial overlap without sharing raw data. We evaluate FederatedRSF on the GBSG2 breast cancer cohort distributed with the scikit-survival package, simulating feature heterogeneity across clients by withholding subsets of features, and assessing discrimination using Harrell's concordance index (C-Index) under repeated cross-validation and site-splits. The results demonstrated that the federated model can achieve performance comparable to that of the centralized training setting.
Maryam Moradpour, Jonas Harriehausen, Amirreza Aleyasin +3
Institute for Predictive Deep Learning in Medicine and Healthcare, Justus Liebig University Gießen, Germany · Hessian Center for Artificial Intelligence (hessian.AI), Darmstadt, Germany · Department of Medical Informatics, University Medical Center Göttingen, Germany +1
Self-supervised pretraining reshaped prediction in language and vision, and brain foundation models (BFMs) inherited its promise. Representations learned from large unlabelled corpora should capture individual functional dynamics and generalise across cohorts. However, kernel ridge regression (KRR) fitted on functional connectivity (FC) matrices still predicts individual phenotypes more accurately than any BFM we tested. In this paper, we show that KRR is weighted by the eigenvalues of the FC which are miscalibrated for phenotype prediction. We apply an efficient spectral filter to recalibrate the eigenvalues of each subject's FC matrix, enabling the model to exploit more inter-individual variance. Across the 5 datasets, 11 parcellations and 6 prediction targets we tested, we match or exceed the KRR baseline. Based on this finding, we then pretrain a small encoder model on about 4,000 hours of fMRI from 162 open datasets, whereby we align the pairwise similarities between the embeddings of recording snippets with those between the recalibrated connectomes. Our model performs on par with the best of the 6 published BFMs we tested while having an order of magnitude fewer parameters. Our encoder performs better than FC on short scans and in smaller cohorts, especially in fingerprinting. We release the pretrained model weights, the code and the pretraining data, preprocessed and parcellated.
Giovanni Marraffini, Victoria Shevchenko, Carlo Alberto Barbano +1