HoTS: Homophily-Aware Temperature Scaling for Graph Neural Network Calibration

Inwoo Tae; Yoontae Hwang; Yongjae Lee

cs.LGSep 26, 2026

HoTS: Homophily-Aware Temperature Scaling for Graph Neural Network Calibration

Authors: Inwoo Tae, Yoontae Hwang, Yongjae Lee

Organizations: UNIST · Pusan National University · UNIST LinqAlpha

Abstract

For graph node classification, calibrated class probabilities are needed when confidence scores, usually the maximum predicted class probability, are used to rank predictions, defer uncertain nodes to human review, or control risk. Existing post-hoc calibrators either apply one global temperature or use graph-aware modules without a principled structural form. We study how local graph structure should enter node-level calibration. Our first results show that a logit-only temperature rule is insufficient when nodes with identical logits but different local homophily require different optimal temperatures. We then analyze a population-concentration contextual stochastic block model with Gaussian features and a one-layer linear GCN. Under equidistant class means, the Bayes posterior over class-template scores is a temperature-scaled softmax whose inverse-temperature is governed by a homophily-dependent signal strength. In the positive-signal homophilic regime, the resulting temperature decreases approximately inversely with normalized local homophily. This law motivates Homophily-aware Temperature Scaling (HoTS), a simple post-hoc calibrator that assigns each node a positive scalar temperature from entropy-based logit concentration and estimated local homophily. HoTS has three temperature parameters, preserves the predicted class, and learns the strength of the structural correction from calibration data. Across 18 node-classification benchmarks, two GNN backbones, and eight calibration baselines, HoTS achieves the best mean Expected Calibration Error (ECE) of 4.79%, the best average rank, and the most reliable confidence ranking in selective classification. Code is available at https://github.com/inu0104/HoTS.

Figures & tables

Homophilic
Dataset	Uncal	TS	VS	ETS	HTS	CaGCN	GATS	GETS	WATS	HoTS (ours)
Cora	$19.64\pm 2.62$	$3.14\pm 0.82$	$\bm{2.90\pm 0.78}$	$\underline{2.94\pm 0.82}$	$3.58\pm 0.56$	$4.12\pm 0.78$	$2.99\pm 0.96$	$3.86\pm 0.82$	$3.08\pm 0.73$	${\color[rgb]{1,0,0}2.50\pm 0.72}$
CiteSeer	$18.57\pm 4.19$	$\bm{3.04\pm 0.62}$	$3.20\pm 0.56$	${\color[rgb]{1,0,0}2.93\pm 0.63}$	$3.78\pm 0.86$	$4.81\pm 0.79$	$3.19\pm 0.86$	$5.08\pm 1.07$	$\underline{3.10\pm 0.57}$	$3.50\pm 0.80$
PubMed	$12.15\pm 2.15$	$1.04\pm 0.28$	$1.16\pm 0.39$	$\underline{1.00\pm 0.31}$	$1.22\pm 0.65$	$1.11\pm 0.99$	$\bm{0.88\pm 0.28}$	$1.14\pm 0.38$	${\color[rgb]{1,0,0}0.83\pm 0.30}$	$1.26\pm 0.30$
CoraFull	$32.93\pm 4.51$	$2.76\pm 0.81$	$2.84\pm 0.63$	${\color[rgb]{1,0,0}2.20\pm 0.63}$	$3.94\pm 0.63$	$\bm{2.56\pm 0.61}$	$2.77\pm 0.80$	$3.72\pm 0.98$	$3.85\pm 1.65$	$\underline{2.73\pm 0.81}$
Computers	$5.86\pm 1.42$	$2.19\pm 0.45$	$2.07\pm 0.45$	$2.12\pm 0.50$	$1.78\pm 0.55$	${\color[rgb]{1,0,0}1.39\pm 0.34}$	$1.91\pm 0.49$	$2.66\pm 0.73$	$\bm{1.62\pm 0.60}$	$\underline{1.76\pm 0.49}$

Table 1: Calibration ECE (%, mean

\pm

std over 10 seeds per backbone, pooled across GCN and GAT). Best per row in red , second in bold , third underlined (excluding Uncal).

Figure 1: Controlled CSBM validation. (a) Fitted temperature over the full homophily range; the blue curve and shaded band show the mean and one standard deviation across 20 seeds. The dashed curve shows the inverse-homophily trend. Pearson correlation is computed over the eight sweep points in

0.25\leq h\leq 0.95

. We mark

h_{0}

and

1/K

and shade

\Gamma(h)<0

; the displayed vertical range is truncated. (b) The fitted exponent

\alpha

is near the oracle value under noiseless homophily and decreases as estimation noise increases, matching Proposition 2 ’s attenuation.

Coverage	Uncal	TS	HTS	HoTS (ours)
100%	67.39	67.39	67.39	67.39
95%	68.87	68.90	68.92	68.94
90%	70.01	70.08	70.13	70.15
85%	71.04	71.12	71.17	71.18
80%	71.94	72.05	72.13	72.11
75%	72.81	72.92	72.94	72.96

Table 2: Selective classification (top prediction-preserving baselines): retained-node accuracy (%) at coverage

c

, macro-averaged over 18 datasets, 2 backbones, and 10 seeds. Restricted to baselines that strictly preserve the predicted class (see Section C.6 ). Best per row in bold (excluding Uncal).

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Metric	TS	ETS	HTS	CaGCN	GATS	GETS	WATS	HoTS (ours)
NLL	0.97	0.94	0.94	0.97	0.98	0.95	1.02	0.92
deg_ECE	6.50	6.32	6.13	5.81	6.45	6.16	7.16	5.90

Appendix

Table 3: Pooled (GCN

+

GAT) mean NLL and degree-stratified ECE (%) across 18 datasets and 10 seeds per backbone.

Homophilic
Dataset	Backbone	Uncal	TS	VS	ETS	HTS	CaGCN	GATS	GETS	WATS	HoTS
Cora	GCN	$21.49\pm 1.15$	$3.42\pm 0.70$	$\bm{3.05\pm 0.97}$	$\underline{3.19\pm 0.77}$	$3.66\pm 0.45$	$4.08\pm 0.88$	$3.30\pm 0.96$	$4.16\pm 0.92$	$3.20\pm 0.69$	${\color[rgb]{1,0,0}2.28\pm 0.76}$
Cora	GAT	$17.79\pm 2.37$	$2.86\pm 0.83$	$2.75\pm 0.48$	$\bm{2.69\pm 0.78}$	$3.49\pm 0.64$	$4.17\pm 0.67$	${\color[rgb]{1,0,0}2.68\pm 0.85}$	$3.55\pm 0.56$	$2.95\pm 0.75$	$\underline{2.72\pm 0.59}$
CiteSeer	GCN	$21.87\pm 3.46$	$\bm{2.91\pm 0.42}$	$3.16\pm 0.73$	${\color[rgb]{1,0,0}2.90\pm 0.44}$	$3.68\pm 0.98$	$4.93\pm 1.08$	$3.42\pm 0.77$	$5.66\pm 1.10$	$\underline{3.01\pm 0.47}$	$3.36\pm 1.02$
CiteSeer	GAT	$15.26\pm 1.17$	$\underline{3.17\pm 0.74}$	$3.24\pm 0.32$	${\color[rgb]{1,0,0}2.96\pm 0.78}$	$3.88\pm 0.71$	$4.69\pm 0.22$	${\color[rgb]{1,0,0}2.96\pm 0.87}$	$4.49\pm 0.63$	$3.18\pm 0.65$	$3.65\pm 0.44$
PubMed	GCN	$13.90\pm 1.58$	$1.18\pm 0.20$	$1.44\pm 0.31$	$1.18\pm 0.23$	$1.45\pm 0.65$	$1.28\pm 1.31$	$\bm{0.91\pm 0.30}$	$\underline{1.14\pm 0.38}$	${\color[rgb]{1,0,0}0.74\pm 0.24}$	$1.25\pm 0.37$

Appendix

Table 4: Per-backbone ECE (%, mean

\pm

std over 10 seeds per backbone). Best per (dataset, backbone) row in red , second in bold , third underlined (excluding Uncal).

Homophilic
Dataset	Backbone	Uncal	TS	VS	ETS	HTS	CaGCN	GATS	GETS	WATS	HoTS
Cora	GCN	$21.47\pm 1.15$	$2.79\pm 0.53$	$\underline{2.72\pm 0.82}$	$2.85\pm 0.58$	$\bm{2.70\pm 0.43}$	$3.10\pm 0.51$	$2.89\pm 0.57$	$3.23\pm 0.54$	$2.83\pm 0.54$	${\color[rgb]{1,0,0}2.59\pm 0.41}$
Cora	GAT	$17.79\pm 2.37$	$\bm{2.17\pm 0.71}$	${\color[rgb]{1,0,0}2.09\pm 0.73}$	$2.35\pm 0.75$	$2.30\pm 0.51$	$2.72\pm 0.73$	$\bm{2.17\pm 0.79}$	$2.53\pm 0.48$	$2.25\pm 0.57$	$\bm{2.17\pm 0.59}$
CiteSeer	GCN	$21.86\pm 3.45$	${\color[rgb]{1,0,0}2.71\pm 0.43}$	$\underline{2.76\pm 0.59}$	${\color[rgb]{1,0,0}2.71\pm 0.55}$	$3.23\pm 0.80$	$3.32\pm 0.54$	$3.15\pm 0.75$	$4.04\pm 0.91$	$2.85\pm 0.54$	$2.91\pm 0.93$
CiteSeer	GAT	$15.24\pm 1.18$	$\bm{2.58\pm 0.49}$	$2.83\pm 0.52$	${\color[rgb]{1,0,0}2.46\pm 0.39}$	$3.36\pm 0.76$	$2.77\pm 0.48$	$2.94\pm 0.87$	$3.04\pm 0.88$	$\underline{2.67\pm 0.39}$	$3.13\pm 0.45$
PubMed	GCN	$13.94\pm 1.52$	$2.58\pm 0.34$	$2.92\pm 0.34$	$2.72\pm 0.33$	$\bm{1.83\pm 0.36}$	$\underline{1.84\pm 1.47}$	$2.19\pm 0.39$	$2.16\pm 0.57$	${\color[rgb]{1,0,0}0.92\pm 0.18}$	$1.98\pm 0.25$

Appendix

Table 5: Per-backbone degree-stratified ECE (%, mean

\pm

std). Best per (dataset, backbone) row in red , second in bold , third underlined (excluding Uncal).

Homophilic
Dataset	Backbone	Uncal	TS	VS	ETS	HTS	CaGCN	GATS	GETS	WATS	HoTS
Cora	GCN	$0.681\pm 0.025$	$0.518\pm 0.031$	${\color[rgb]{1,0,0}0.481\pm 0.026}$	$0.512\pm 0.030$	$\underline{0.503\pm 0.028}$	$0.572\pm 0.053$	$0.518\pm 0.030$	$0.528\pm 0.032$	$0.516\pm 0.030$	$\bm{0.497\pm 0.028}$
Cora	GAT	$0.636\pm 0.036$	$0.516\pm 0.027$	$\bm{0.506\pm 0.022}$	$0.515\pm 0.026$	$\underline{0.513\pm 0.024}$	$0.553\pm 0.026$	$0.520\pm 0.026$	$0.523\pm 0.021$	$0.516\pm 0.027$	${\color[rgb]{1,0,0}0.504\pm 0.023}$
CiteSeer	GCN	$0.977\pm 0.066$	$\underline{0.849\pm 0.046}$	${\color[rgb]{1,0,0}0.809\pm 0.023}$	$\bm{0.844\pm 0.042}$	$0.853\pm 0.052$	$0.878\pm 0.042$	$0.853\pm 0.046$	$0.884\pm 0.039$	$\underline{0.849\pm 0.048}$	$0.859\pm 0.050$
CiteSeer	GAT	$0.878\pm 0.009$	$\underline{0.801\pm 0.012}$	${\color[rgb]{1,0,0}0.790\pm 0.011}$	$\bm{0.800\pm 0.012}$	$0.803\pm 0.013$	$0.832\pm 0.017$	$0.807\pm 0.011$	$0.832\pm 0.016$	$\underline{0.801\pm 0.012}$	$0.805\pm 0.013$
PubMed	GCN	$0.459\pm 0.017$	$0.380\pm 0.005$	$0.377\pm 0.005$	$0.377\pm 0.005$	$0.376\pm 0.006$	$\bm{0.368\pm 0.017}$	$0.375\pm 0.006$	$\underline{0.371\pm 0.005}$	${\color[rgb]{1,0,0}0.365\pm 0.005}$	$0.376\pm 0.005$

Appendix

Table 6: Per-backbone NLL (mean

\pm

std). Best per (dataset, backbone) row in red , second in bold , third underlined (excluding Uncal).

Backbone	TS	VS	ETS	HTS	CaGCN	GATS	GETS	WATS	HoTS
GCNII	6.82	6.08	6.29	6.23	5.79	6.35	6.79	7.74	5.84
GPR-GNN	6.85	5.95	6.47	5.73	6.05	5.87	7.11	7.78	5.49
GIN	6.32	5.67	5.37	5.23	6.37	5.31	6.33	8.98	4.97
GraphSAGE	6.60	6.10	5.53	5.75	6.63	5.79	6.51	9.20	5.11

Appendix

Table 7: Calibration across additional GNN backbones. Mean ECE (%) over 17 datasets, with 10 seeds per dataset. Reddit is excluded. Lower is better; best per row in bold .

Figure 2: Reliability diagrams on Reddit with a GCN backbone, pooling test nodes across 10 seeds into 10 equal-width confidence bins. TS reduces the calibration gap, and HoTS further improves calibration by allowing the temperature to vary across nodes.

Figure 3: Reliability diagrams across the 18 benchmark datasets. Each pair compares the uncalibrated GCN and HoTS, pooling test nodes across 10 seeds into 10 equal-width confidence bins. The figure is intended as a diagnostic complement to Table 1 ; pooled ECE differs from the mean per-seed ECE reported in the tables.

Figure 4: Node-level diagnostic for HoTS on representative datasets. Points show confidence gap versus the homophily-dependent temperature correction evaluated with oracle homophily. The two rows color the same nodes by oracle normalized homophily $\tilde{h}$ and entropy complement $1-e$ . Oracle homophily is computed from true labels solely for this diagnostic.

Figure 5: Node-level diagnostic across datasets, colored by oracle normalized homophily $\tilde{h}$ , computed from true labels for this diagnostic. The effect of HoTS is most structured on datasets where local homophily varies substantially across nodes.

Figure 6: Node-level diagnostic across datasets, colored by entropy complement $1-e$ . Entropy complement captures logit concentration and is complementary to the homophily signal in Figure 5 .

Figure 7: KL bound validation on CSBM. Three independent sweeps over

n

\bar{d}

, and

K

collapse when plotted against

K^{2}\bar{d}^{2}/n

, supporting the residual scaling derived in Appendix A.8 .

Dataset	Method	0%	5%	10%	20%	30%	50%
Cora	HoTS	2.27	2.72	2.21	2.55	2.77	3.03
Cora	TS	3.39	3.24	2.81	3.73	3.69	4.16
CiteSeer	HoTS	2.83	3.09	3.21	3.10	2.72	3.92
CiteSeer	TS	3.09	2.95	3.09	3.11	3.13	3.79
CS	HoTS	0.99	0.95	0.95	1.26	1.19	1.15
CS	TS	1.49	1.49	1.54	1.93	1.89	2.44

Appendix

Table 8: ECE (%) under edge perturbation. HoTS is comparatively stable on the homophilic benchmarks shown here. On heterophilic graphs, random perturbation can improve calibration for both TS and HoTS by weakening the original heterophilic structure.

Figure 8: Confidence-conditioned oracle-homophily diagnostic with a 50/50 split. For each dataset, test nodes are stratified by the base-model confidence and separated into low- and high-

h

groups using a rank-based 50/50 split of oracle local homophily within each bin; ties retain the pooled node order. Oracle homophily is used to examine the structural reliability signal separately from estimation error. Curves show empirical accuracy within each confidence bin. The title of each panel reports the mean high-minus-low empirical accuracy gap

\Delta_{\mathrm{acc}}

across the five confidence bins in percentage points.

Figure 9: Confidence-conditioned oracle-homophily diagnostic with three groups. This figure repeats the diagnostic in Figure 8 , but separates nodes by oracle-homophily rank into low-, middle-, and high-

h

groups of approximately equal size within each confidence bin. The title of each panel reports the mean high-minus-low empirical accuracy gap

\Delta_{\mathrm{acc}}

across the five confidence bins in percentage points. The three-way split visualizes how accuracy varies with neighborhood label composition at comparable confidence.

Structural statistic	Category	Pooled	Macro
Oracle normalized homophily	Homophily	$+0.384$	$+0.283$
Local clustering coefficient	Motifs	$+0.044$	$+0.012$
PageRank	Centrality	$+0.003$	$-0.003$
Mean incident augmented Forman–Ricci curvature	Curvature	$-0.026$	$-0.052$

Appendix

Table 9: Partial correlation with prediction correctness after controlling for confidence and log-degree. Results use GCN and five seeds. Homophily and PageRank cover 18 datasets; clustering and curvature cover 17, excluding Reddit.

	TS	Entropy-only	Homophily-only †	HoTS ( $\alpha=1$ )	HoTS (ours)
Mean ECE (%)	$6.04$	$4.83$	$5.56$	$5.90$	$\bm{4.79}$

Appendix

Table 10: HoTS component ablation. Mean ECE (%) across 18 datasets, pooling GCN and GAT with 10 seeds per backbone. HoTS and HoTS (

\alpha=1

) use the same observed-label-only homophily estimates. † Homophily-only retains the previously reported result and has not been rerun with the current observed-label-only estimator. Lower is better; best in bold .

Dataset	Nodes	Edges	Classes	Features	Homophily
Citation (Planetoid)
Cora	2,708	13,264	7	1,433	0.871 $\pm$ 0.211
CiteSeer	3,327	12,431	6	3,703	0.820 $\pm$ 0.245
PubMed	19,717	108,365	3	500	0.865 $\pm$ 0.217
Amazon / Coauthor
Computers	13,381	491,556	10	767	0.802 $\pm$ 0.239

Appendix

Table 11: Dataset statistics. Homophily is the mean (

\pm

std) of node-level homophily

h_{i}

over all nodes. All datasets use random 20/10/70 train/val/test splits regenerated for each seed.

Group	Hidden	Layers	LR	Weight decay	Dropout	Patience
G1	16	2	$10^{-2}$	$5\times 10^{-4}$	0.5	50 / –
G2	64	2	$10^{-2}$	$10^{-3}$	0.5	50
G3	256	2	$10^{-2}$	$0$	0.5	–

Appendix

Table 12: GNN training hyperparameters by dataset group. Optimizer is Adam in all cases; both backbones train for at most 200 epochs; “–” under Patience indicates no early stopping (full 200 epochs are run). The same configuration is used for GCN and GAT, with GAT additionally using 2 attention heads and feature/attention dropout equal to the listed Dropout. Group G3 (ogbn-arxiv) additionally uses BatchNorm between layers, following the OGB convention.

Method	Method-specific settings
TS, ETS, HTS	Scalar parameters only; no architectural hyperparameters.
VS	Per-class affine logit transform $z_{c}\mapsto w_{c}\,z_{c}+b_{c}$ .
CaGCN, GATS	Per-dataset config (auxiliary GCN/attention width and dropout) following the original releases.
GETS	Per-dataset config (mixture-of-experts width and number of experts) following the original release.
WATS	Per-dataset config (graph wavelet basis size) following the original release.
HoTS (ours)	Three scalar temperature parameters $(T_{\mathrm{base}},\beta,\alpha)$ , enforced through softplus parameterizations $T_{\mathrm{base}}=\operatorname{softplus}(\hat{T})+0.1$ , $\beta=\operatorname{softplus}(\hat{\beta})+0.01$ , $\alpha=\operatorname{softplus}(\hat{\alpha})+0.01$ on unconstrained underlying parameters $\hat{T},\hat{\beta},\hat{\alpha}\in\mathbb{R}$ . We initialize $\hat{T}=\hat{\beta}=0.5$ and $\hat{\alpha}=0$ , giving effective initial values $T_{\mathrm{base}}\approx 1.07$ , $\beta\approx 0.98$ , $\alpha\approx 0.70$ . The stabilizer in the homophily denominator is fixed to $\varepsilon=0.02$ . The homophily predictor that produces $\hat{h}_{i}$ is trained separately; see Section C.5 .

Appendix

Table 13: Method-specific calibration settings. Common settings (Adam,

\text{lr}{=}10^{-2}

, weight decay

0

1{,}000

epochs, patience

50

) apply to all methods. “Per-dataset config” indicates that learning rate, hidden width, attention heads, or other architectural choices follow the per-dataset configuration files released with the original method.

Method	Calibrator form	Preserves	Rate
Global methods
TS	$\mathrm{softmax}(z/T)$ , $T>0$ scalar	Yes	$0.0\%$
ETS	$\sum_{k}w_{k}\,p^{(k)}$ , mixture	No ∗	$0.3\%$
VS	$z^{\mathrm{cal}}_{c}=w_{c}z_{c}+b_{c}$ per-class affine	No	$96.1\%$
Node-wise methods
HTS	$\mathrm{softmax}(z_{i}/T_{i})$ , $T_{i}=a+b\,e_{i}$ , $T_{i}>0$	Yes	$0.0\%$

Appendix

Table 14: Prediction change rates across all

360

experiments (

18

datasets

\times

2

backbones

\times

10

seeds). “Preserves” indicates whether the method theoretically preserves the predicted class. “Rate” is the fraction of experiments in which at least one test node’s

\arg\max

differs from the uncalibrated baseline. Lower is more conservative. ∗ ETS can in principle reweight class probabilities but empirically learns mixture weights very close to

(1,0,0)

. † CaGCN and GATS produce strictly positive

T_{i}

, but rare numerical edge cases (near-tied logits) flip a small number of predictions. ‡ Reddit GATS is omitted due to OOM during fitting (Section C.2 ); the rate for GATS is computed over the remaining

340

experiments.

Model Calibration

Model calibration aligns predicted confidence or model outputs with observed outcomes and measurements to support more reliable decisions.

See all 218 papers

Jul 12, 2026

When does distribution shift break graph neural networks calibration?

Graph neural networks (GNNs) are increasingly deployed in real-world applications where distribution shift is un-avoidable. However, how such shifts affect model calibration, defined as the agreement between predictive confidence and actual accuracy, remains poorly understood, and existing graph calibration methods typically rely on labeled validation data from the deployment distribution. In this work, I present the first closed-form theoretical characterization of GNN calibration under distribution shift. I show that calibration is governed by a single scalar quantity that explicitly depends on structural changes between the source and target graphs, as well as feature quality. This characterization precisely identifies when a model becomes over-confident, under-confident, or remains calibrated, and directly yields the optimal temperature scaling strategy. I further extend the analysis to graph convolutional networks with symmetric normalization, multi-class classification, and covariate shift, and derive a theoretical upper bound on the expected calibration error. My analysis also reveals that, under homogeneous distribution shift, a single global temperature is theoretically optimal, providing a principled explanation for why more complex node-wise recalibration methods offer no additional benefit. Building on these theoretical insights, I propose STAC, a source-free, label-free calibration method. Experiments on synthetic benchmarks demonstrate substantial calibration improvements, while evaluations on five real-world graph datasets show that reliable calibration without target labels remains challenging despite the strong predictive power of the theory.

Abderaouf Bahi

Computer Science and Applied Mathematics Laboratory (LIMA), Faculty of Science and Technology, Chadli Bendjedid University, El Tarf 36000, Algeria

Graph Neural Networks Model Calibration Uncertainty Calibration

Ask PDF

Aug 14, 2026

Disentangling Homophily and Rarity: Explaining Failure in Graph Neural Networks

Are heterophilic nodes in a graph harder to classify because they are heterophilic or because they are rare? Some existing work frames classification of such nodes as a subgroup generalisation problem, where a model performs well on the majority group at the expense of the rare group. Others explain this as a problem of neighbourhood aggregation in graph neural networks (GNNs). We assess these two viewpoints through a detailed evaluation of six GNNs on five datasets of varying homophily, and find that homophilic nodes tend to be easier to classify, even when they are rare---challenging the subgroup framing. However, our findings also nuance existing beliefs about how GNNs misrepresent heterophilic nodes. We demonstrate that the information needed to classify heterophilic nodes correctly is often recoverable by retraining the classification head of a model, or even just the final linear classification layer.

Preben M. Ness, Fariz Ikhwantri, Dusica Marijan

Simula Research Laboratory

Graph Neural Networks Node Classification Graph Heterophily

Ask PDF

Sep 16, 2026

Not All Nodes Are Created Equal: Homophily-Aware Stratification for Stable GNN Evaluation

Graph neural networks are widely used for transductive node classification, with accuracy typically measured on randomly drawn train/validation/test splits. Reported accuracy has been shown to shift substantially across different random splits of the same dataset, making published comparisons between architectures unreliable. The classical remedy in non-graph settings is stratified

k

-fold cross-validation, which ensures each test fold reflects the full class distribution of the dataset. We argue that class stratification alone is insufficient for graphs: nodes are not isolated but connected, and folds that differ in their distribution of local neighbourhood homophily expose the model to systematically different relational conditions that directly affect message-passing behaviour. The resulting cross-fold variation reflects the homophily composition of each split, inflating reported variance beyond what model behaviour alone would produce. To address this, we propose \hp{}, a topology-aware stratification procedure that treats node homophily as the primary stratification axis, aligning folds with respect to local relational consistency alongside the class marginal that standard stratification already controls. Stratifying on homophily alone does not guarantee class balance, so \hp{} incorporates class label as a secondary axis, preserving class representativeness as a natural consequence of the procedure. We evaluate \hp{} on a broad benchmark suite comprising 15 node-classification datasets spanning the full homophily spectrum and 7 GNN architectures. \hp{} achieves a mean stability rank of 1.49 compared to 2.31 for random

k

-fold, achieving the lowest mean stability rank on 13 of 15 datasets while preserving class balance close to class-stratified splits and substantially better than random. We argue that homophily-aware split construction merits broader adoption for GNN evaluation.

Naga Venkata Sai Jitin Jami, Thomas Altstidl, Sebastian Hoefler +4

Chair for Machine Learning in Medicine, Universität Bayreuth · Department Artiﬁcial Intelligence in Biomedical Engineering, FAU Erlangen-Nürnberg · Chair of AI-supported Therapy Decisions, Ludwig-Maximilians-Universität München +1

Graph Neural Networks Node Classification

Ask PDF

Abstract

Model Calibration

Explore similar work

When does distribution shift break graph neural networks calibration?

Disentangling Homophily and Rarity: Explaining Failure in Graph Neural Networks

Not All Nodes Are Created Equal: Homophily-Aware Stratification for Stable GNN Evaluation