Diagnosing GNN Failure Modes with Layer-Wise Probe Entropy

Shanzi Bao; Mar Gonzàlez I Català; Pietro Liò

cs.LGOct 15, 2026

Diagnosing GNN Failure Modes with Layer-Wise Probe Entropy

Authors: Shanzi Bao, Mar Gonzàlez I Català, Pietro Liò

Organizations: University of Cambridge

Abstract

Graph neural networks (GNNs) are typically evaluated only at their final layer, obscuring how task-relevant information evolves during message passing. We introduce a post-hoc diagnostic that freezes a trained GNN, fits an independent linear probe to each layer, and tracks probe accuracy alongside normalized predictive entropy. Across real and synthetic graph settings, layer-wise probe entropy distinguishes beneficial, neutral, and harmful smoothing, a capability not demonstrated by existing geometric measures. Moreover, by comparing shallow and deep models, we show that the onset of entropy degradation separates the two main GNN failure modes: structural mismatch under heterophily, which appears from the earliest layers even in shallow models, and excessive-depth oversmoothing, which emerges only as propagation depth increases. These results establish probe-entropy trajectories as a tool for analyzing information flow in GNNs and motivate future work on adaptive depth selection and early-exit mechanisms.

Figures & tables

Figure 1: Beneficial, neutral, and harmful predictive outcomes under geometric smoothing in four-layer GCNs. Geometric trajectories are normalized by their input-layer values. Lines and markers show means across five seeds; shaded bands and error bars denote one standard deviation.

	Pearson			Spearman
Diagnostic	$r$		95% CI	$\rho$		95% CI
Probe entropy	$\mathbf{-0}$	$\mathbf{.67}$	$[-0.86,\,-0.34]$	$\mathbf{-0}$	$\mathbf{.71}$	$[-0.89,\,-0.26]$
Dirichlet energy	$-0$	$.07$	$[-0.41,\,0.19]$	$-0$	$.27$	$[-0.59,\,0.15]$
MAD	$0$	$.01$	$[-0.49,\,0.70]$	$0$	$.17$	$[-0.43,\,0.66]$
$R_{\mathrm{Group}}$	$0$	$.34$	$[-0.09,\,0.72]$	$0$	$.62$	$[0.03,\,0.82]$

Table 1: Correlations between diagnostic changes and probe-accuracy changes across 24 dataset–architecture settings. Boldface indicates the largest absolute correlation for each coefficient.

Figure 2: Controlled GCN failure-onset trajectories. (a,b): Layer-wise probe accuracy and normalized entropy at

D=4,10

. (c): High-homophily final readouts across

D\in\{2,4,6,8,10,12,16,20\}

. Curves show means over five runs, with bands and error bars indicating one standard deviation.

	GCN		GAT
Regime	4 layers	10 layers	4 layers	10 layers
Homophilous ( $8$ )	$0.63\pm 0.22$	$0.44\pm 0.20$	$0.68\pm 0.22$	$0.61\pm 0.22$
Heterophilous ( $4$ )	$0.13\pm 0.11$	$0.19\pm 0.06$	$0.12\pm 0.13$	$0.16\pm 0.08$

Table 2: Per-node entropy–confidence alignment

\overline{\rho}

. Values report mean

\pm

sample standard deviation across datasets within each regime, after averaging five runs per dataset. Parentheses indicate dataset counts.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Nodes	Classes	Train nodes	$h_{\mathrm{adj}}$	Split protocol
Coauthor-Physics	34,493	5	20,695	$0.87$	Fixed 60/20/20
Amazon-Photo	7,650	8	4,590	$0.79$	Fixed 60/20/20
Coauthor-CS	18,333	15	10,999	$0.79$	Fixed 60/20/20
Cora	2,708	7	140	$0.77$	20 nodes per class
PubMed	19,717	3	60	$0.69$	20 nodes per class
Amazon-Computer	13,752	10	8,251	$0.68$	Fixed 60/20/20

Appendix

Table 3: Real-world multiclass datasets and split protocols. For datasets with multiple splits, training-set sizes refer to the first split.

h_{\mathrm{adj}}

denotes adjusted homophily [ 28 ] .

Figure 3: Input-to-final-layer probe-accuracy change versus changes in probe entropy, scale-normalized Dirichlet energy, global MAD, and

R_{\mathrm{Group}}

across 24 real dataset–architecture settings with four-layer models. Geometric changes are relative to input-layer values. Circles and triangles denote GCN and GAT, respectively, and colour indicates adjusted homophily. Points and error bars show the mean and standard deviation across five runs. Dashed lines show linear fits; panel titles report Pearson and Spearman correlations.

Figure 4: Controlled failure-onset patterns across eight depths for GCN and GAT. Panel (a) shows first-step entropy changes under high and low homophily. Panels (b)–(c) show final entropy and accuracy under high homophily. Error bars denote one standard deviation across five runs.

Figure 5: Controlled GAT failure-onset trajectories at

D\in\{4,10,20\}

. Curves show mean

\pm

standard deviation across five runs.

Figure 6: Representative layer-wise probe trajectories on Cora and Roman-Empire for shallow and deep GCNs and GATs. Lines and shaded bands show the mean and one standard deviation across five runs.

		GCN		GAT
Regime	Edge homophily	4 layers	10 layers	4 layers	10 layers
High	$0.865$	$0.50\pm 0.03$	$0.36\pm 0.07$	$0.51\pm 0.04$	$0.47\pm 0.06$
Intermediate	$0.512$	$0.17\pm 0.04$	$0.13\pm 0.04$	$0.06\pm 0.02$	$0.14\pm 0.04$
Low	$0.110$	$-0.10\pm 0.03$	$0.02\pm 0.05$	$-0.27\pm 0.07$	$0.02\pm 0.05$

Appendix

Table 4: Per-node entropy–confidence Spearman correlation

\overline{\rho}

on controlled synthetic graphs. Values report mean

\pm

standard deviation across five seeds.

Graph Neural Networks

Graph Neural Networks learn from graph-structured data by combining information across connected nodes, edges, and larger structures.

See all 726 papers

Declarative Explanations for Graph Neural Networks: A Demonstration
Oct 19, 2026Cibo Yu, Haitong Tang, Yinghui Wu +4GNN Explainability Graph Neural Networks
Zhejiang University, Hangzhou, China · Case Western Reserve University, Cleveland, USA · Bowling Green State University, Bowling Green, USA
Beyond ReLU: Bifurcation, Oversmoothing, and Topological Priors
Feb 17, 2026Erkan Turan, Gaspard Abel, Maysam Behmanesh +2GNN Oversmoothing Graph Neural Networks
LIX, Ecole Polytechnique, IP Paris · Université Paris Saclay, Université Paris Cité, ENS Paris Saclay, CNRS, SSA, INSERM, Centre Borelli, F-91190, Gif-sur-Yvette, France · Centre d’Analyse et de Mathématique Sociales, EHESS, CNRS, 75006 Paris, France..
Local-Global Geometric Insights for Graph Neural Networks via Entropic Curvature
Jul 24, 2026Rachid Caich, Yassine AbbahaddouOversmoothing Graph Rewiring
LIX, École Polytechnique IP Paris Palaiseau, France

Oct 19, 2026J/K move · Enter open · S save

Declarative Explanations for Graph Neural Networks: A Demonstration

Cibo Yu, Haitong Tang, Yinghui Wu, Arijit Khan+3

Zhejiang University, Hangzhou, China · Case Western Reserve University, Cleveland, USA · Bowling Green State University, Bowling Green, USA

GNN Explainability Graph Neural Networks

Open paper AskPDF arXiv

Graph Neural Networks (GNNs) have proven effective for graph analytical tasks, yet their "black-box" nature complicates the understanding of their layer-wise behavior and hinders their application to large graphs. We demonstrate SliceGXQ, an end-to-end declarative and interactive system for querying and exploring layer-wise explanations for GNNs over large-scale graphs. In particular, SliceGXQ provides (1) an interactive graphical user interface for dataset, model, and explanation configuration; (2) a lightweight SPARQL-like parser that allows users to specify target nodes, layers, explanation modes, and constraints; and (3) an integrated execution and visualization workflow that invokes SliceGX as its underlying explanation engine. We demonstrate how users can diagnose a misclassified node, identify problematic layers and influential messages, and perform targeted model debugging, together with large-scale graph interpretation and fraud-detection analysis. Our demonstration video and source code are available at https://github.com/jinzhaoqet/SliceGXQ.

Abstract

Graph Neural Networks

Explore similar work

Declarative Explanations for Graph Neural Networks: A Demonstration

Beyond ReLU: Bifurcation, Oversmoothing, and Topological Priors

Local-Global Geometric Insights for Graph Neural Networks via Entropic Curvature