A Comparative Study of Counterfactual Explainers for Graph Neural Networks Enabling Multiple Types of Graph Edit
Authors: Maria Myrto Villia, Filippos Gouidis, Theodore Patkos, Panos Trahanias
Organizations: Institute of Computer Science, Foundation for Research and Technology - Hellas (FORTH) · Computer Science Department, University of Crete Greece
Counterfactual explanations for graph-structured data seek to determine minimal and realistic modifications required in an input graph to alter a model's prediction to a predefined output. Although counterfactual explainers that support modifying the graph by both adding and removing edges have recently emerged, there is still a lack of general and efficient methods, especially when considering the quality of the generated explanations. Moreover, the problem remains far from solved, as existing methods exhibit different strengths and weaknesses, often trading off between explanation size, coverage and quality. For this reason, it is important to identify where each method performs well and where it falls short, so as to guide future research in the field. Thus, our study compares six state-of-the-art (SOTA) models on a diverse set of real-world and synthetic datasets, covering both binary and multi-class graph and node classification tasks, and evaluates their performance using diverse quantitative and qualitative metrics.
Figures & tables
Figure 1. (a) Predictions on drug-drug interaction networks constitute a prominent application domain of Graph Neural Networks (b) subgraph-based explanations identify important edges of the input data (c) We argue that supergraph-based explanations can also offer useful insights
Dataset
# of classes
# of features
Avg. # of nodes
Avg. # of edges
# of train graphs
# of val. graphs
# of test graphs
BA-2Motifs
2
10
25
25,49
800
100
100
BA-2Motifs-3Classes
3
10
25
25,23
800
100
100
BA-3Motifs
3
10
25
27,06
800
100
100
BA-4Motifs
4
10
25
27,09
800
100
100
BBBP
2
9
24.06
25,35
1631
203
205
Twitter
3
768
21.103
20,35
4,998
1,250
692
Table 1. Dataset statistics.
Dataset
# of classes
# of features
# of nodes
# of edges
# of train nodes
# of val. nodes
# of test nodes
BA-Shapes
4
10
700
2055
560
70
70
Tree-Cycles
2
10
871
971
696
87
88
Cora
7
1433
2708
5278
1647
479
479
PubMed
3
500
19717
44324
13243
3000
3000
Table 2. Statistics of the node classification datasets.
Institute of Computer Science, Foundation for Research and Technology - Hellas (FORTH), Heraklion, Greece · Computer Science Department, University of Crete, Greece
Project Management Department, Research Center, China Life Insurance Company Ltd. · School of Computer Science and Technology, Beijing Institute of Technology