Bias, Measurement Error, and Double-Dipping: When Can GNN Convolutions Help Brain Connectome Prediction?
Organizations: Columbia University, USA · Rice University, USA · Washington University in St. Louis, USA · Christian-Albrechts University Kiel, Germany ·
Abstract
Graph Neural Networks (GNNs) are an important tool for fMRI prediction, where empirical functional connectivity matrices serve both as features and for graph construction. However, recent studies and our own experiments show that these methods often fail to outperform simpler graph-agnostic baselines. We propose a statistical explanation for this failure by formulating connectome prediction as an errors-in-variables (EIV) problem: the true connectome is latent, while the observed covariance matrix is a noisy finite-time-series estimate. In a linear asymptotic setting, we decompose the effect of graph convolution into three mechanisms: smoothing-induced bias and variance reduction, graph estimation error, and a double-dipping effect arising when the same noisy matrix determines both graph and features. Under a latent community model adapted to neuroscience, we identify a narrow favorable regime, set by the brain community structure, the predictive signal, the graph construction and the recording length, in which message passing can improve prediction. Simulations validate this decomposition, while real-data experiments illustrate that connectome GNN pipelines often fall outside this regime. Our results provide a first step towards explaining when graph convolutions can help brain connectome prediction, and why they often do not.
Figures & tables
| Graph | Effects isolated | Main prediction conclusion |
|---|---|---|
| (population) | Fixed smoothing | Cannot improve optimized raw prediction. |
| (oracle) | Subject-specific smoothing | Can help when variance reduction outweighs squared bias and raw refitting. |
| (independent) | Graph estimation | Same tradeoff, with an additional graph-error bound. |
| (same-sample) | Noise reuse | Close to independent prediction when graph estimation is reliable and regression is stable. |
| 5 | 0.560\hbox to0.0pt{{}_{\pm.033}\hss} | 0.582\hbox to0.0pt{{}^{***}_{\pm.034}\hss} | 0.561\hbox to0.0pt{{}_{\pm.031}\hss} | 0.563\hbox to0.0pt{{}_{\pm.035}\hss} | 0.562\hbox to0.0pt{{}_{\pm.039}\hss} | +0.022\hbox to0.0pt{{}^{***}\hss} | -0.021\hbox to0.0pt{{}^{***}\hss} | +0.002\hbox to0.0pt{$$\hss} | +0.001\hbox to0.0pt{$$\hss} |
|---|---|---|---|---|---|---|---|---|---|
| 15 | 0.625\hbox to0.0pt{{}_{\pm.040}\hss} | 0.630\hbox to0.0pt{{}_{\pm.033}\hss} | 0.619\hbox to0.0pt{{}_{\pm.040}\hss} | 0.619\hbox to0.0pt{{}_{\pm.039}\hss} | 0.618\hbox to0.0pt{{}_{\pm.032}\hss} | +0.004\hbox to0.0pt{$$\hss} | -0.010\hbox to0.0pt{{}^{*}\hss} | +0.000\hbox to0.0pt{$$\hss} | -0.007\hbox to0.0pt{$$\hss} |
| 25 | 0.651\hbox to0.0pt{{}_{\pm.037}\hss} | 0.668\hbox to0.0pt{{}^{**}_{\pm.030}\hss} | 0.641\hbox to0.0pt{{}_{\pm.047}\hss} | 0.650\hbox to0.0pt{{}_{\pm.044}\hss} | 0.653\hbox to0.0pt{{}_{\pm.037}\hss} | +0.017\hbox to0.0pt{{}^{**}\hss} | -0.027\hbox to0.0pt{{}^{***}\hss} | +0.009\hbox to0.0pt{{}^{*}\hss} | +0.002\hbox to0.0pt{$$\hss} |
| 40 | 0.682\hbox to0.0pt{{}_{\pm.034}\hss} | 0.678\hbox to0.0pt{{}_{\pm.031}\hss} | 0.669\hbox to0.0pt{{}^{**}_{\pm.030}\hss} | 0.663\hbox to0.0pt{{}^{**}_{\pm.036}\hss} | 0.672\hbox to0.0pt{{}^{*}_{\pm.029}\hss} | -0.004\hbox to0.0pt{$$\hss} | -0.009\hbox to0.0pt{{}^{**}\hss} | -0.006\hbox to0.0pt{$$\hss} | -0.010\hbox to0.0pt{{}^{*}\hss} |
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
| Dataset | Task | Parcellation | # ROIs | Stride | Subjects | |
|---|---|---|---|---|---|---|
| ABIDE | Regression | rois_cc200 | 200 | 150 | 15 | 785 |
| ADHD | Classification | cc200 | 190 | full TS | N/A | 768 |
| Frankfurt | Classification | Gordon286 + ASEG8 | 294 | 200 | 20 | 72 |
| Model | Hyperparameters |
|---|---|
| Random Forest | n_estimators=400 , max_depth=None , min_samples_leaf=5 , max_features=sqrt |
| Ridge | alpha=100 |
| SVM | C=1 , gamma=scale , kernel=rbf |
| SVR | C=1 , gamma=scale , kernel=rbf |
| MLP | lr=1e-2 , weight_decay=1e-3 , l1_lambda=1e-4 , hidden_dim=64 |
| GAT (1-conv) | lr=1e-2 , weight_decay=0 , l1_lambda=1e-4 , hidden_dim=64 |
| ReLU activation (vectorized readout, fixed) | |||||||||
| 5 | 0.558\hbox to0.0pt{{}_{\pm.037}\hss} | 0.567\hbox to0.0pt{{}_{\pm.044}\hss} | 0.537\hbox to0.0pt{{}^{**}_{\pm.037}\hss} | 0.546\hbox to0.0pt{{}_{\pm.038}\hss} | 0.550\hbox to0.0pt{{}_{\pm.031}\hss} | +0.009\hbox to0.0pt{$$\hss} | -0.029\hbox to0.0pt{{}^{***}\hss} | +0.009\hbox to0.0pt{$$\hss} | -0.008\hbox to0.0pt{$$\hss} |
| 15 | 0.617\hbox to0.0pt{{}_{\pm.051}\hss} | 0.632\hbox to0.0pt{{}_{\pm.044}\hss} | 0.615\hbox to0.0pt{{}_{\pm.051}\hss} | 0.620\hbox to0.0pt{{}_{\pm.045}\hss} | 0.609\hbox to0.0pt{{}_{\pm.051}\hss} | +0.015\hbox to0.0pt{$$\hss} | -0.017\hbox to0.0pt{{}^{*}\hss} | +0.005\hbox to0.0pt{$$\hss} | -0.009\hbox to0.0pt{$$\hss} |
| 25 | 0.643\hbox to0.0pt{{}_{\pm.045}\hss} | 0.660\hbox to0.0pt{{}^{*}_{\pm.032}\hss} | 0.641\hbox to0.0pt{{}_{\pm.048}\hss} | 0.653\hbox to0.0pt{{}_{\pm.030}\hss} | 0.640\hbox to0.0pt{{}_{\pm.046}\hss} | +0.017\hbox to0.0pt{{}^{*}\hss} | -0.019\hbox to0.0pt{{}^{*}\hss} | +0.012\hbox to0.0pt{$$\hss} | -0.003\hbox to0.0pt{$$\hss} |
| 40 | 0.680\hbox to0.0pt{{}_{\pm.035}\hss} | 0.676\hbox to0.0pt{{}_{\pm.031}\hss} | 0.666\hbox to0.0pt{{}^{**}_{\pm.033}\hss} | 0.662\hbox to0.0pt{{}^{***}_{\pm.031}\hss} | 0.656\hbox to0.0pt{{}^{**}_{\pm.046}\hss} | -0.005\hbox to0.0pt{$$\hss} | -0.009\hbox to0.0pt{{}^{**}\hss} | -0.004\hbox to0.0pt{$$\hss} | -0.024\hbox to0.0pt{{}^{**}\hss} |
| ReLU activation and free (vectorized readout) | |||||||||
| Simulation-like scenario (identity activation, vectorized readout, fixed) | |||||||||
|---|---|---|---|---|---|---|---|---|---|
| 5 | 0.558\hbox to0.0pt{{}_{\pm.033}\hss} | 0.585\hbox to0.0pt{{}^{***}_{\pm.031}\hss} | 0.547\hbox to0.0pt{{}_{\pm.037}\hss} | 0.557\hbox to0.0pt{{}_{\pm.031}\hss} | 0.554\hbox to0.0pt{{}_{\pm.034}\hss} | +0.027\hbox to0.0pt{{}^{***}\hss} | -0.038\hbox to0.0pt{{}^{***}\hss} | +0.010\hbox to0.0pt{$$\hss} | -0.004\hbox to0.0pt{$$\hss} |
| 15 | 0.621\hbox to0.0pt{{}_{\pm.040}\hss} | 0.643\hbox to0.0pt{{}^{***}_{\pm.035}\hss} | 0.608\hbox to0.0pt{{}^{*}_{\pm.032}\hss} | 0.618\hbox to0.0pt{{}_{\pm.038}\hss} | 0.605\hbox to0.0pt{{}^{***}_{\pm.032}\hss} | +0.022\hbox to0.0pt{{}^{***}\hss} | -0.035\hbox to0.0pt{{}^{***}\hss} | +0.010\hbox to0.0pt{$$\hss} | -0.016\hbox to0.0pt{{}^{***}\hss} |
| 25 | 0.649\hbox to0.0pt{{}_{\pm.037}\hss} | 0.654\hbox to0.0pt{{}_{\pm.033}\hss} | 0.637\hbox to0.0pt{{}^{*}_{\pm.035}\hss} | 0.643\hbox to0.0pt{{}_{\pm.036}\hss} | 0.638\hbox to0.0pt{{}^{**}_{\pm.032}\hss} | +0.005\hbox to0.0pt{$$\hss} | -0.017\hbox to0.0pt{{}^{**}\hss} | +0.006\hbox to0.0pt{$$\hss} | -0.011\hbox to0.0pt{{}^{**}\hss} |
| 40 | 0.680\hbox to0.0pt{{}_{\pm.047}\hss} | 0.663\hbox to0.0pt{{}^{**}_{\pm.037}\hss} | 0.662\hbox to0.0pt{{}^{***}_{\pm.035}\hss} | 0.665\hbox to0.0pt{{}^{***}_{\pm.038}\hss} | 0.654\hbox to0.0pt{{}^{***}_{\pm.035}\hss} | -0.018\hbox to0.0pt{{}^{**}\hss} | -0.001\hbox to0.0pt{$$\hss} | +0.003\hbox to0.0pt{$$\hss} | -0.027\hbox to0.0pt{{}^{***}\hss} |
| Deployed pipeline (ReLU activation, mean pooling, free) | |||||||||
| Simulation-like scenario (identity activation, vectorized readout, fixed) | |||||||||
|---|---|---|---|---|---|---|---|---|---|
| 5 | 0.243\hbox to0.0pt{{}_{\pm.060}\hss} | 0.261\hbox to0.0pt{{}_{\pm.063}\hss} | 0.219\hbox to0.0pt{{}^{*}_{\pm.069}\hss} | 0.225\hbox to0.0pt{{}_{\pm.063}\hss} | 0.251\hbox to0.0pt{{}_{\pm.057}\hss} | +0.018\hbox to0.0pt{$$\hss} | -0.042\hbox to0.0pt{{}^{***}\hss} | +0.006\hbox to0.0pt{$$\hss} | +0.008\hbox to0.0pt{$$\hss} |
| 7 | 0.318\hbox to0.0pt{{}_{\pm.047}\hss} | 0.333\hbox to0.0pt{{}^{**}_{\pm.057}\hss} | 0.267\hbox to0.0pt{{}^{***}_{\pm.061}\hss} | 0.312\hbox to0.0pt{{}_{\pm.058}\hss} | 0.320\hbox to0.0pt{{}_{\pm.055}\hss} | +0.015\hbox to0.0pt{{}^{**}\hss} | -0.066\hbox to0.0pt{{}^{***}\hss} | +0.045\hbox to0.0pt{{}^{***}\hss} | +0.002\hbox to0.0pt{$$\hss} |
| 15 | 0.463\hbox to0.0pt{{}_{\pm.043}\hss} | 0.469\hbox to0.0pt{{}_{\pm.057}\hss} | 0.443\hbox to0.0pt{{}^{**}_{\pm.051}\hss} | 0.448\hbox to0.0pt{{}^{*}_{\pm.053}\hss} | 0.466\hbox to0.0pt{{}_{\pm.059}\hss} | +0.005\hbox to0.0pt{$$\hss} | -0.026\hbox to0.0pt{{}^{***}\hss} | +0.006\hbox to0.0pt{$$\hss} | +0.002\hbox to0.0pt{$$\hss} |
| 25 | 0.531\hbox to0.0pt{{}_{\pm.035}\hss} | 0.526\hbox to0.0pt{{}_{\pm.043}\hss} | 0.511\hbox to0.0pt{{}^{***}_{\pm.043}\hss} | 0.516\hbox to0.0pt{{}^{**}_{\pm.040}\hss} | 0.532\hbox to0.0pt{{}_{\pm.046}\hss} | -0.006\hbox to0.0pt{$$\hss} | -0.015\hbox to0.0pt{$$\hss} | +0.005\hbox to0.0pt{$$\hss} | +0.001\hbox to0.0pt{$$\hss} |
| 40 | 0.543\hbox to0.0pt{{}_{\pm.063}\hss} | 0.523\hbox to0.0pt{{}^{***}_{\pm.058}\hss} | 0.526\hbox to0.0pt{{}^{*}_{\pm.051}\hss} | 0.525\hbox to0.0pt{{}^{**}_{\pm.054}\hss} | 0.529\hbox to0.0pt{{}^{*}_{\pm.050}\hss} | -0.020\hbox to0.0pt{{}^{***}\hss} | +0.003\hbox to0.0pt{$$\hss} | -0.002\hbox to0.0pt{$$\hss} | -0.014\hbox to0.0pt{{}^{*}\hss} |
| Simulation-like scenario (identity activation, vectorized readout, fixed) | |||||||||
|---|---|---|---|---|---|---|---|---|---|
| 5 | 0.069\hbox to0.0pt{{}_{\pm.067}\hss} | 0.126\hbox to0.0pt{{}^{***}_{\pm.090}\hss} | 0.066\hbox to0.0pt{{}_{\pm.072}\hss} | 0.060\hbox to0.0pt{{}^{***}_{\pm.068}\hss} | 0.066\hbox to0.0pt{{}_{\pm.071}\hss} | +0.057\hbox to0.0pt{{}^{***}\hss} | -0.060\hbox to0.0pt{{}^{***}\hss} | -0.007\hbox to0.0pt{$$\hss} | -0.003\hbox to0.0pt{$$\hss} |
| 8 | 0.141\hbox to0.0pt{{}_{\pm.052}\hss} | 0.193\hbox to0.0pt{{}^{***}_{\pm.070}\hss} | 0.145\hbox to0.0pt{{}_{\pm.059}\hss} | 0.144\hbox to0.0pt{{}_{\pm.056}\hss} | 0.145\hbox to0.0pt{{}_{\pm.047}\hss} | +0.052\hbox to0.0pt{{}^{***}\hss} | -0.048\hbox to0.0pt{{}^{***}\hss} | -0.002\hbox to0.0pt{$$\hss} | +0.005\hbox to0.0pt{$$\hss} |
| 14 | 0.246\hbox to0.0pt{{}_{\pm.073}\hss} | 0.262\hbox to0.0pt{{}_{\pm.082}\hss} | 0.271\hbox to0.0pt{{}^{***}_{\pm.070}\hss} | 0.252\hbox to0.0pt{{}^{**}_{\pm.073}\hss} | 0.249\hbox to0.0pt{{}_{\pm.073}\hss} | +0.016\hbox to0.0pt{$$\hss} | +0.009\hbox to0.0pt{$$\hss} | -0.018\hbox to0.0pt{{}^{***}\hss} | +0.004\hbox to0.0pt{$$\hss} |
| 25 | 0.374\hbox to0.0pt{{}_{\pm.063}\hss} | 0.369\hbox to0.0pt{{}_{\pm.085}\hss} | 0.398\hbox to0.0pt{{}^{***}_{\pm.066}\hss} | 0.381\hbox to0.0pt{{}^{*}_{\pm.064}\hss} | 0.379\hbox to0.0pt{{}^{**}_{\pm.064}\hss} | -0.004\hbox to0.0pt{$$\hss} | +0.029\hbox to0.0pt{{}^{**}\hss} | -0.017\hbox to0.0pt{{}^{***}\hss} | +0.005\hbox to0.0pt{{}^{**}\hss} |
| 45 | 0.461\hbox to0.0pt{{}_{\pm.065}\hss} | 0.445\hbox to0.0pt{{}_{\pm.083}\hss} | 0.488\hbox to0.0pt{{}^{***}_{\pm.076}\hss} | 0.461\hbox to0.0pt{{}_{\pm.071}\hss} | 0.477\hbox to0.0pt{{}^{***}_{\pm.066}\hss} | -0.016\hbox to0.0pt{$$\hss} | +0.044\hbox to0.0pt{{}^{***}\hss} | -0.028\hbox to0.0pt{{}^{***}\hss} | +0.016\hbox to0.0pt{{}^{***}\hss} |
| 5 | 0.560\hbox to0.0pt{{}_{\pm.033}\hss} | 0.580\hbox to0.0pt{{}^{**}_{\pm.034}\hss} | 0.563\hbox to0.0pt{{}_{\pm.033}\hss} | 0.563\hbox to0.0pt{{}_{\pm.033}\hss} | 0.567\hbox to0.0pt{{}_{\pm.035}\hss} | +0.019\hbox to0.0pt{{}^{**}\hss} | -0.016\hbox to0.0pt{{}^{**}\hss} | -0.000\hbox to0.0pt{$$\hss} | +0.006\hbox to0.0pt{$$\hss} |
|---|---|---|---|---|---|---|---|---|---|
| 15 | 0.625\hbox to0.0pt{{}_{\pm.040}\hss} | 0.628\hbox to0.0pt{{}_{\pm.035}\hss} | 0.616\hbox to0.0pt{{}_{\pm.039}\hss} | 0.616\hbox to0.0pt{{}_{\pm.039}\hss} | 0.623\hbox to0.0pt{{}_{\pm.040}\hss} | +0.003\hbox to0.0pt{$$\hss} | -0.013\hbox to0.0pt{{}^{**}\hss} | +0.001\hbox to0.0pt{$$\hss} | -0.003\hbox to0.0pt{$$\hss} |
| 25 | 0.651\hbox to0.0pt{{}_{\pm.037}\hss} | 0.654\hbox to0.0pt{{}_{\pm.043}\hss} | 0.649\hbox to0.0pt{{}_{\pm.051}\hss} | 0.651\hbox to0.0pt{{}_{\pm.045}\hss} | 0.652\hbox to0.0pt{{}_{\pm.045}\hss} | +0.003\hbox to0.0pt{$$\hss} | -0.005\hbox to0.0pt{$$\hss} | +0.002\hbox to0.0pt{$$\hss} | +0.001\hbox to0.0pt{$$\hss} |
| 40 | 0.682\hbox to0.0pt{{}_{\pm.034}\hss} | 0.661\hbox to0.0pt{{}^{**}_{\pm.030}\hss} | 0.668\hbox to0.0pt{{}^{**}_{\pm.033}\hss} | 0.660\hbox to0.0pt{{}^{**}_{\pm.035}\hss} | 0.668\hbox to0.0pt{{}_{\pm.035}\hss} | -0.021\hbox to0.0pt{{}^{**}\hss} | +0.007\hbox to0.0pt{$$\hss} | -0.008\hbox to0.0pt{$$\hss} | -0.014\hbox to0.0pt{$$\hss} |
Graph Neural Networks
Graph Neural Networks learn from graph-structured data by combining information across connected nodes, edges, and larger structures.
See all 780 papers