Diagnosing GNN Failure Modes with Layer-Wise Probe Entropy
Organizations: University of Cambridge
Abstract
Graph neural networks (GNNs) are typically evaluated only at their final layer, obscuring how task-relevant information evolves during message passing. We introduce a post-hoc diagnostic that freezes a trained GNN, fits an independent linear probe to each layer, and tracks probe accuracy alongside normalized predictive entropy. Across real and synthetic graph settings, layer-wise probe entropy distinguishes beneficial, neutral, and harmful smoothing, a capability not demonstrated by existing geometric measures. Moreover, by comparing shallow and deep models, we show that the onset of entropy degradation separates the two main GNN failure modes: structural mismatch under heterophily, which appears from the earliest layers even in shallow models, and excessive-depth oversmoothing, which emerges only as propagation depth increases. These results establish probe-entropy trajectories as a tool for analyzing information flow in GNNs and motivate future work on adaptive depth selection and early-exit mechanisms.
Figures & tables
| Pearson | Spearman | |||||
|---|---|---|---|---|---|---|
| Diagnostic | 95% CI | 95% CI | ||||
| Probe entropy | ||||||
| Dirichlet energy | ||||||
| MAD | ||||||
| GCN | GAT | |||
|---|---|---|---|---|
| Regime | 4 layers | 10 layers | 4 layers | 10 layers |
| Homophilous ( ) | ||||
| Heterophilous ( ) | ||||
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
| Dataset | Nodes | Classes | Train nodes | Split protocol | |
|---|---|---|---|---|---|
| Coauthor-Physics | 34,493 | 5 | 20,695 | Fixed 60/20/20 | |
| Amazon-Photo | 7,650 | 8 | 4,590 | Fixed 60/20/20 | |
| Coauthor-CS | 18,333 | 15 | 10,999 | Fixed 60/20/20 | |
| Cora | 2,708 | 7 | 140 | 20 nodes per class | |
| PubMed | 19,717 | 3 | 60 | 20 nodes per class | |
| Amazon-Computer | 13,752 | 10 | 8,251 | Fixed 60/20/20 |
| GCN | GAT | ||||
|---|---|---|---|---|---|
| Regime | Edge homophily | 4 layers | 10 layers | 4 layers | 10 layers |
| High | |||||
| Intermediate | |||||
| Low | |||||
Graph Neural Networks
Graph Neural Networks learn from graph-structured data by combining information across connected nodes, edges, and larger structures.
See all 726 papers