Disentangling Homophily and Rarity: Explaining Failure in Graph Neural Networks

Preben M. Ness; Fariz Ikhwantri; Dusica Marijan

cs.LGAug 14, 2026

Disentangling Homophily and Rarity: Explaining Failure in Graph Neural Networks

Authors: Preben M. Ness, Fariz Ikhwantri, Dusica Marijan

Organizations: Simula Research Laboratory

Abstract

Are heterophilic nodes in a graph harder to classify because they are heterophilic or because they are rare? Some existing work frames classification of such nodes as a subgroup generalisation problem, where a model performs well on the majority group at the expense of the rare group. Others explain this as a problem of neighbourhood aggregation in graph neural networks (GNNs). We assess these two viewpoints through a detailed evaluation of six GNNs on five datasets of varying homophily, and find that homophilic nodes tend to be easier to classify, even when they are rare---challenging the subgroup framing. However, our findings also nuance existing beliefs about how GNNs misrepresent heterophilic nodes. We demonstrate that the information needed to classify heterophilic nodes correctly is often recoverable by retraining the classification head of a model, or even just the final linear classification layer.

Figures & tables

Figure 1: Overview of the diagnostic preparations. The diagram shows the trainable and frozen components, the initialisation source, and the optimisation objective under each preparation.

Panel of Figure 2 — Figure 2: Distribution of nodes across local-homophily bins. Bar widths show the bin intervals and heights show within-dataset percentages. Dashed lines give mean $h_{v}$ ; thick outlines mark the selected tails.

Figure 3: RQ1 . Average test macro-F1 across 12 equal-width local-homophily bins. Grey bars show the mean test-node count per run for each bin, and grey regions mark each dataset’s rare homophily tail. Roman-empire macro-F1 is omitted where a bin lacks test nodes in at least one split.

Figure 4: RQ1, RQ1a . Macro-F1 tail gaps for all models on all datasets, with homophilic datasets on the top and heterophilic datasets on the bottom. Unadjusted and class-adjusted values are shown on the left and right, respectively, of each subplot, and lines connect values for a given model. Vertical lines denote 95% confidence intervals, and models whose interval contains zero are denoted with an unfilled marker symbol.

Figure 5: RQ1b, RQ1c . Whole-test and tail macro-F1 for all models on all datasets, with models coloured by model family. Within each dataset, models are ordered from left to right as MLP, GCN, GraphSAGE-mean, H2GCN-2, GPR-GNN, ACM-GCN, and LINKX. Blue denotes MLP, orange the standard aggregation models, green the heterophily-designed aggregation models, and purple denotes LINKX.

Figure 6: RQ2, RQ2a, RQ2b, RQ2c . Mean tail and non-tail macro-F1 for different training procedures across the six graph models. Within each model, points from left to right are

A_{\mathrm{ERM}}

A_{\mathrm{last}}

(RQ2a),

A_{\mathrm{head}}

(RQ2b), and

A_{\mathrm{full}}

(RQ2c). Circles show tail scores and squares show non-tail scores. Note that each sub-figure uses its own vertical scale. An increase in the blue curve indicates recovered performance on the rare-homophily tail, while a decrease in the orange curve indicates corresponding performance degradation on the rest of the test set.

Appendix figures & tables22 assets

Supplementary material from the paper’s appendix.

Appendix

Deviation
The registered code used scikit-learn’s default label set for macro-F1. This takes the union of the labels in the true values and predictions, so the classes included in the average could vary between evaluations. We replaced this with the classes present in the evaluated subset’s true labels. The Roman-empire recovery experiments were rerun because the error also affected the validation score used for checkpoint selection. All macro-F1 scores were then recalculated using the corrected definition.
The preregistration described a rule where only classes with at least 10 tail test nodes should be included in class-wise accuracy comparisons. In contrast, for macro-F1, all classes represented among the tail nodes. Although motivated by removing noisy results, we felt the rule was too arbitrary and changed it so that class-wise accuracy now uses the same rule as the existing macro-F1. All dependent estimates, tables, and figures were recalculated. The only visible difference on the plotted figures was that the class-adjustment of Roman-empire in Figure 4 became less conclusive, with the adjusted models landing either side of 0.
Minor grammatical, nomenclature, and notation adjustments were made to the RQ formulations. For example, we use the term rare homophily tail in this paper, instead of minority pattern tail , which is used in the preregistration.

Appendix

Table 1: Ledger of deviations from the preregistered analysis and experimental protocol.

Model	Encoder	Training
MLP [ 24 ]	Shared backbone; model-specific operation is the identity, so each block has no graph access.	AdamW, lr $3{\cdot}10^{-5}$ , wd 0, 1000 steps.
GCN [ 26 ]	Shared backbone; parameter-free symmetric degree-normalised neighbour aggregation ( $1/\sqrt{d_{u}d_{v}}$ ). Self-loops added.	AdamW, lr $3{\cdot}10^{-5}$ , wd 0, 1000 steps.
GraphSAGE-mean [ 27 ]	Shared backbone; concatenation of the node state and mean neighbour state (1024-dimensional), followed by an FFN with widths $1024{\to}512{\to}512$ . No self-loops.	AdamW, lr $3{\cdot}10^{-5}$ , wd 0, 1000 steps.
H2GCN-2 [ 11 ]	Ego and neighbour separation over exact 1-hop and 2-hop indicator matrices, each symmetrically normalised; network_setup = M73-R-T1-G-V-T2-G-V-C1-C2-D0.5 ; internal width 73; input-feature normalisation on; dropout 0.5. $z_{v}=$ the native 7-block concatenation (511-dim); the source output layer is removed.	Adam, lr 0.01, L2 $5{\cdot}10^{-4}$ , 2000 epochs.
GPR-GNN [ 12 ]	Two-layer MLP feature extractor (width 512), then learned GPR propagation: $K=10$ , $\alpha=0.1$ , personalised PageRank initialisation; feature dropout 0.5, propagation dropout 0.5. $z_{v}=\sum_{k=0}^{K}\gamma_{k}H^{(k)}$ (512-dim).	Adam, lr 0.002, wd $5{\cdot}10^{-4}$ on the linear layers and 0 on $\gamma$ , 1000 epochs.
ACM-GCN [ 13 ]	Plain acmgcn : one layer, one hop; three channels (low-pass aggregation, high-pass diversification, identity) mixed per node by learned attention; variant=0 , structure_info=0 ; width 512. $z_{v}=$ the mixed representation (512-dim).	AdamW, lr 0.01, wd $10^{-3}$ , dropout 0.5, 500 epochs (the authors’ large-scale protocol).

Appendix

Table 2: Fixed encoder and ERM training settings. Here, lr is learning rate and wd is weight decay. The shared classification head and the recovery-preparation settings are given in Sections 3 and 4 .

Procedure	Runs	GPU-hours
$A_{\mathrm{ERM}}$	1,050	179.714
$A_{\mathrm{last}}$	1,050	19.095
$A_{\mathrm{head}}$	1,050	53.941
$A_{\mathrm{full}}$	1,050	177.189
Total	4,200	429.939

Appendix

Table 3: Total GPU-hours by training procedure.

Dataset	Bin	Nodes	% of nodes
PubMed	$\mathbf{[0,0.25)}$	2,696	13.7
(homophilic, mean $h_{v}$ 0.792)	$[0.25,0.5)$	524	2.7
	$[0.5,0.75)$	1,891	9.6
	$[0.75,1]$	14,606	74.1
ogbn-arxiv	$\mathbf{[0,0.25)}$	30,364	17.9
(homophilic, mean $h_{v}$ 0.635)	$[0.25,0.5)$	20,524	12.1

Appendix

Table 4: The local homophily bins used throughout. Bold type marks the rare

h_{v}

tail—the low

h_{v}

bin on homophilic datasets and the high-

h_{v}

bin on heterophilic datasets.

Figure 7: RQ1 . Average test accuracy across 12 equal-width local-homophily bins. Grey bars show the mean test-node count per run for each bin, and grey regions mark each dataset’s rare homophily tail. Roman-empire accuracy is omitted where a bin lacks test nodes in at least one split.

	Low-homophily tail
Model	PubMed	ogbn-arxiv
MLP	$+.208$ [-1pt] $[.203,.213]$	$+.102$ [-1pt] $[.102,.103]$
GCN	$+.364$ [-1pt] $[.358,.369]$	$+.332$ [-1pt] $[.331,.333]$
GraphSAGE-mean	$+.353$ [-1pt] $[.348,.358]$	$+.356$ [-1pt] $[.354,.357]$
H2GCN-2	$+.299$ [-1pt] $[.292,.305]$	$+.301$ [-1pt] $[.300,.303]$
GPR-GNN	$+.350$ [-1pt] $[.345,.356]$	$+.305$ [-1pt] $[.303,.306]$

Appendix

Table 5: ERM signed tail gap: whole-test macro-F1 minus tail macro-F1. Each cell gives the mean paired difference over 30 matched split–seed runs and its 95% percentile-bootstrap interval. Positive values indicate lower tail performance; grey cells contain zero.

	Low-homophily tail
Model	PubMed	ogbn-arxiv
MLP	$+.211$ [-1pt] $[.206,.217]$	$+.270$ [-1pt] $[.269,.271]$
GCN	$+.369$ [-1pt] $[.363,.375]$	$+.520$ [-1pt] $[.519,.521]$
GraphSAGE-mean	$+.359$ [-1pt] $[.354,.365]$	$+.555$ [-1pt] $[.553,.557]$
H2GCN-2	$+.302$ [-1pt] $[.295,.308]$	$+.493$ [-1pt] $[.492,.495]$
GPR-GNN	$+.355$ [-1pt] $[.349,.360]$	$+.517$ [-1pt] $[.515,.519]$

Appendix

Table 6: ERM accuracy gap, whole-test accuracy minus tail accuracy. Each cell gives the mean paired difference over 30 matched split–seed runs and its 95% percentile-bootstrap interval. Positive values indicate lower tail accuracy; grey cells contain zero.

	Low-homophily tail
Model	PubMed	ogbn-arxiv
MLP	$+.229$ [-1pt] $[.224,.235]$	$+.151$ [-1pt] $[.149,.152]$
GCN	$+.419$ [-1pt] $[.412,.428]$	$+.498$ [-1pt] $[.496,.501]$
GraphSAGE-mean	$+.406$ [-1pt] $[.399,.414]$	$+.516$ [-1pt] $[.513,.518]$
H2GCN-2	$+.344$ [-1pt] $[.336,.352]$	$+.415$ [-1pt] $[.412,.417]$
GPR-GNN	$+.401$ [-1pt] $[.393,.408]$	$+.401$ [-1pt] $[.398,.404]$

Appendix

Table 7: Per-class non-tail accuracy minus tail accuracy, averaged equally over classes. Each cell reports the mean over the 30 runs and the 95% percentile-bootstrap interval. Grey cells denote confidence intervals that contain zero.

	Low-homophily tail
Model	PubMed	ogbn-arxiv
MLP	$+.197$ [-1pt] $[.191,.203]$	$+.188$ [-1pt] $[.187,.190]$
GCN	$+.376$ [-1pt] $[.369,.383]$	$+.498$ [-1pt] $[.497,.500]$
GraphSAGE-mean	$+.366$ [-1pt] $[.360,.373]$	$+.524$ [-1pt] $[.523,.527]$
H2GCN-2	$+.306$ [-1pt] $[.298,.313]$	$+.427$ [-1pt] $[.426,.429]$
GPR-GNN	$+.354$ [-1pt] $[.346,.361]$	$+.451$ [-1pt] $[.449,.452]$

Appendix

Table 8: Class-adjusted accuracy tail gaps for all models on all datasets. Entries are the mean and 95% bootstrap interval across 30 runs. Grey cells include zero, and

\dagger

marks cells whose unadjusted accuracy tail gap includes zero.

	Low-homophily tail
Model	PubMed	ogbn-arxiv
MLP	$+.197$ [-1pt] $[.192,.203]$	$+.106$ [-1pt] $[.105,.107]$
GCN	$+.364$ [-1pt] $[.358,.371]$	$+.350$ [-1pt] $[.348,.351]$
GraphSAGE-mean	$+.350$ [-1pt] $[.345,.357]$	$+.366$ [-1pt] $[.364,.367]$
H2GCN-2	$+.298$ [-1pt] $[.291,.305]$	$+.299$ [-1pt] $[.298,.301]$
GPR-GNN	$+.348$ [-1pt] $[.340,.354]$	$+.303$ [-1pt] $[.301,.304]$

Appendix

Table 9: Class-adjusted macro-F1 tail gaps for all models on all datasets. Entries are the mean and 95% bootstrap interval across 30 runs. Grey cells include zero, and

\dagger

marks cells whose unadjusted macro-F1 tail gap includes zero.

Figure 8: Macro-F1 tail gap of the five aggregation GNNs minus the tail gap of the graphless MLP model for all five datasets. Each point is the mean over 30 matched split–seed differences; error bars show 95% percentile-bootstrap intervals.

Figure 9: Class-composition adjusted macro-F1 tail gap of the five aggregation GNNs minus the tail gap of the graphless MLP model for all five datasets. Each point is the mean over 30 matched split–seed differences; error bars show 95% percentile-bootstrap intervals.

		Low-homophily tail
Designed model	Standard baseline	PubMed	ogbn-arxiv
H2GCN-2	GCN	$-.064$ [-1pt] $[-.071,-.059]$	$-.031$ [-1pt] $[-.033,-.029]$
H2GCN-2	GraphSAGE-mean	$-.053$ [-1pt] $[-.059,-.048]$	$-.054$ [-1pt] $[-.056,-.053]$
GPR-GNN	GCN	$-.013$ [-1pt] $[-.019,-.008]$	$-.027$ [-1pt] $[-.029,-.025]$
GPR-GNN	GraphSAGE-mean	$-.002$ [-1pt] $[-.009,+.004]$	$-.051$ [-1pt] $[-.053,-.049]$
ACM-GCN	GCN	$-.031$ [-1pt] $[-.039,-.023]$	$-.107$ [-1pt] $[-.109,-.105]$

Appendix

Table 10: RQ1c . Macro-F1 tail-gap comparison between each heterophily-designed model and each standard aggregation baseline. Each cell is heterophily-model gap minus standard-baseline gap, with its 95% percentile-bootstrap interval over the 30 matched split–seed runs. Negative values indicate a smaller tail gap for the heterophily-designed model; grey cells contain zero and are not statistically significant.

Figure 10: RQ1c . Class-composition-standardised whole-test and tail macro-F1 for all models on all datasets, with models coloured by model family. The tail is reweighted to the whole-test proportions of true classes represented in the tail; the whole-test score uses the same class set. Within each dataset, models are ordered from left to right as MLP, GCN, GraphSAGE-mean, H2GCN-2, GPR-GNN, ACM-GCN, and LINKX. Blue denotes MLP, orange the standard aggregation models, green the heterophily-designed aggregation models, and purple LINKX.

	Low-homophily tail
Model	PubMed	ogbn-arxiv
MLP	$+.016$ [-1pt] $[.011,.020]$	$+.008$ [-1pt] $[.006,.009]$
GCN	$+.025$ [-1pt] $[.016,.034]$	$+.036$ [-1pt] $[.035,.038]$
GraphSAGE-mean	$+.116$ [-1pt] $[.105,.127]$	$+.053$ [-1pt] $[.051,.055]$
H2GCN-2	$+.057$ [-1pt] $[.042,.072]$	$+.076$ [-1pt] $[.074,.078]$
GPR-GNN	$+.022$ [-1pt] $[.014,.030]$	$+.093$ [-1pt] $[.091,.096]$

Appendix

Table 11: RQ2a . Change in tail macro-F1 after last layer retraining,

A_{\mathrm{last}}-A_{\mathrm{ERM}}

. Each cell gives the mean difference over the 30 runs and its 95% percentile-bootstrap interval. Grey cells contain zero and hence are not statistically significant.

	Low-homophily tail
Model	PubMed	ogbn-arxiv
MLP	$+.021$ [-1pt] $[.017,.026]$	$+.032$ [-1pt] $[.031,.033]$
GCN	$+.108$ [-1pt] $[.099,.118]$	$+.090$ [-1pt] $[.088,.091]$
GraphSAGE-mean	$+.156$ [-1pt] $[.148,.163]$	$+.121$ [-1pt] $[.120,.123]$
H2GCN-2	$+.048$ [-1pt] $[.038,.059]$	$+.069$ [-1pt] $[.067,.072]$
GPR-GNN	$+.017$ [-1pt] $[.009,.024]$	$+.129$ [-1pt] $[.125,.132]$

Appendix

Table 12: RQ2b . Change in tail macro-F1 after head retraining,

A_{\mathrm{head}}-A_{\mathrm{ERM}}

. Each cell gives the mean difference over the 30 runs and its 95% percentile-bootstrap interval. Grey cells contain zero and hence are not statistically significant.

	Low-homophily tail
Model	PubMed	ogbn-arxiv
MLP	$+.004$ [-1pt] $[-.001,.010]$	$+.023$ [-1pt] $[.022,.025]$
GCN	$+.122$ [-1pt] $[.115,.129]$	$+.094$ [-1pt] $[.090,.097]$
GraphSAGE-mean	$+.162$ [-1pt] $[.156,.169]$	$+.128$ [-1pt] $[.126,.130]$
H2GCN-2	$+.064$ [-1pt] $[.056,.072]$	$+.081$ [-1pt] $[.078,.084]$
GPR-GNN	$+.095$ [-1pt] $[.089,.102]$	$+.074$ [-1pt] $[.071,.077]$

Appendix

Table 13: RQ2c . Change in tail macro-F1 after full model retraining,

A_{\mathrm{full}}-A_{\mathrm{ERM}}

. Each cell gives the mean difference over the 30 runs and its 95% percentile-bootstrap interval. Grey cells contain zero and hence are not statistically significant.

	Low-homophily tail		High-homophily tail
Model	PubMed	ogbn-arxiv	Roman-empire	Amazon-ratings	arxiv-year
GCN	$0.208$	$0.388$	$-0.001$	$1.632$	$1.899$
GraphSAGE-mean	$0.713$	$0.417$	$-0.011$	$0.201$	$2.686$
H2GCN-2	$0.884$	$0.940$	$0.489$	$-2.376$	$-0.396$
GPR-GNN	$0.228$	$1.255$	$0.742$	$0.015$	$1.545$
ACM-GCN	$0.053$	$1.125$	$0.755$	$0.336$	$0.424$

Appendix

Table 14: RQ2a . Recovered share

\rho_{\mathrm{last}}

after final-layer refitting; cells are grey when the 95% paired-bootstrap interval for the numerator contains zero or

\hat{A}_{\mathrm{full}}-\hat{A}_{\mathrm{ERM}}<0.03

	Low-homophily tail		High-homophily tail
Model	PubMed	ogbn-arxiv	Roman-empire	Amazon-ratings	arxiv-year
GCN	$0.888$	$0.957$	$0.421$	$-0.026$	$3.951$
GraphSAGE-mean	$0.961$	$0.948$	$-0.032$	$-0.503$	$3.680$
H2GCN-2	$0.751$	$0.852$	$0.530$	$-3.185$	$-1.657$
GPR-GNN	$0.175$	$1.733$	$0.997$	$-0.008$	$1.786$
ACM-GCN	$0.177$	$0.864$	$1.325$	$0.148$	$4.402$

Appendix

Table 15: RQ2b . Recovered share

\rho_{\mathrm{head}}

after head refitting; cells are grey when the 95% paired-bootstrap interval for the numerator contains zero or

\hat{A}_{\mathrm{full}}-\hat{A}_{\mathrm{ERM}}<0.03

Model	$A_{\mathrm{ERM}}$	$A_{\mathrm{last}}$	$A_{\mathrm{head}}$	$A_{\mathrm{full}}$	$A_{\mathrm{DFR}}$
ACM-GCN	$.557$ [-1pt] $[.550,.564]$	$.562$ [-1pt] $[.555,.569]$	$.575$ [-1pt] $[.568,.581]$	$.658$ [-1pt] $[.652,.665]$	$.593$ [-1pt] $[.586,.600]$
GCN	$.512$ [-1pt] $[.506,.518]$	$.537$ [-1pt] $[.531,.543]$	$.620$ [-1pt] $[.614,.626]$	$.634$ [-1pt] $[.628,.639]$	$.551$ [-1pt] $[.542,.560]$
GraphSAGE-mean	$.532$ [-1pt] $[.526,.538]$	$.648$ [-1pt] $[.635,.660]$	$.688$ [-1pt] $[.678,.697]$	$.694$ [-1pt] $[.689,.700]$	$.629$ [-1pt] $[.621,.636]$

Appendix

Table 16: PubMed tail macro-F1 for ERM, the three registered retraining procedures, and a DFR-style frozen-representation refit. Each cell gives the mean over 30 runs and the 95% percentile-bootstrap interval.

Model	Procedure	FG-CCDB gain [95% CI]	GroupDRO gain
GraphSAGE-mean	$A_{\mathrm{last}}$	$+.007$ [-1pt] $[.004,.010]$	+.116
	$A_{\mathrm{head}}$	$+.043$ [-1pt] $[.039,.048]$	+.156
	$A_{\mathrm{full}}$	$+.013$ [-1pt] $[.006,.020]$	+.162
GCN	$A_{\mathrm{last}}$	$-.015$ [-1pt] $[-.019,-.011]$	+.025
	$A_{\mathrm{head}}$	$+.012$ [-1pt] $[.007,.016]$	+.108
	$A_{\mathrm{full}}$	$+.002$ [-1pt] $[-.005,.008]$	+.122

Appendix

Table 17: PubMed tail macro-F1 gain relative to

A_{\mathrm{ERM}}

under FG-CCDB and GroupDRO. Values are means over 30 matched split–seed runs. FG-CCDB entries include 95% percentile-bootstrap intervals; grey intervals contain zero.

Figure 11: Tail macro-F1 after rewiring increasing percentages of the rare homophily tail. Treatment rewiring changes each selected node’s local homophily by one neighbour towards the graph-wide mean, while the control rewires randomly, leaving the nodes’ local homophily unchanged. Both procedures preserve every node’s degree. Curves show means over five graph replicates after averaging over 30 fixed ERM checkpoints per model; shaded bands show one standard deviation across graph-replicate means. Treatment increases local homophily for the homophilic datasets PubMed and ogbn-arxiv in (a)–(d), and decreases it for the heterophilic dataset arxiv-year in (e) and (f).

Explore similar work

Revealing the Pitfalls and Re-Evaluating the Advancement of Heterophilic Graph Learning
Sep 9, 2024Sitao Luan, Qincheng Lu, Chenqing Hua +3Graph Neural Networks Node Classification
Mila - Quebec Artiﬁcial Intelligence Institute · 1McGill University
Not All Nodes Are Created Equal: Homophily-Aware Stratification for Stable GNN Evaluation
Sep 16, 2026Naga Venkata Sai Jitin Jami, Thomas Altstidl, Sebastian Hoefler +4Graph Neural Networks Node Classification
Chair for Machine Learning in Medicine, Universität Bayreuth · Department Artiﬁcial Intelligence in Biomedical Engineering, FAU Erlangen-Nürnberg · Chair of AI-supported Therapy Decisions, Ludwig-Maximilians-Universität München +1
Inductive Subgraphs as Shortcuts: Causal Disentanglement for Heterophilic Graph Learning
Apr 21, 2026Xiangmeng Wang, Qian Li, Haiyang Xia +3Disentangled Representation Learning Graph Neural Networks
Curtin University Perth, Australia · The Education University of Hong Kong New Territories, Hong Kong

Sep 9, 2024J/K move · Enter open · S save

Revealing the Pitfalls and Re-Evaluating the Advancement of Heterophilic Graph Learning

Sitao Luan, Qincheng Lu, Chenqing Hua, Xinyu Wang+2

Mila - Quebec Artiﬁcial Intelligence Institute · 1McGill University

Graph Neural Networks Node Classification Graph Heterophily

Open paper AskPDF arXiv

Over the past decade, Graph Neural Networks (GNNs) have achieved great success on machine learning tasks with relational data. However, recent studies have found that heterophily can cause significant performance degradation of GNNs, especially on node-level tasks. Numerous heterophilic benchmark datasets have been put forward to validate the efficacy of heterophily-specific GNNs, and various homophily metrics have been designed to help recognize these challenging datasets. Nevertheless, there still exist multiple pitfalls that severely hinder the proper evaluation of new models and metrics: 1) lack of hyperparameter tuning; 2) insufficient evaluation on the truly challenging heterophilic datasets; 3) missing quantitative evaluation for homophily metrics on synthetic graphs. To overcome these challenges, we first train and fine-tune baseline models on

27

most widely used benchmark datasets, and categorize them into three distinct groups: malignant, benign and ambiguous heterophilic datasets. We identify malignant and ambiguous heterophily as the truly challenging subsets of tasks, and to our best knowledge, we are the first to propose such taxonomy. Then, we re-evaluate

11

state-of-the-arts (SOTA) GNNs, covering six popular methods, with fine-tuned hyperparameters on different groups of heterophilic datasets. Based on the model performance, we comprehensively reassess the effectiveness of different methods on heterophily. At last, we evaluate

11

popular homophily metrics on synthetic graphs with three different graph generation approaches. To overcome the unreliability of observation-based comparison and evaluation, we conduct the first quantitative evaluation and provide detailed analysis.

Abstract

Explore similar work

Revealing the Pitfalls and Re-Evaluating the Advancement of Heterophilic Graph Learning

Not All Nodes Are Created Equal: Homophily-Aware Stratification for Stable GNN Evaluation

Inductive Subgraphs as Shortcuts: Causal Disentanglement for Heterophilic Graph Learning