cs.LGOct 2, 2026

Beyond Trained Models: Compiling GNNs for a Sound Explainer Benchmark

Authors: Steve Azzolin, Francesco Paolo Nerini, Stefano Teso, Francesco Bonchi, Bruno Lepri, André Panisson, Andrea Passerini

Organizations: University of Trento Trento, Italy

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Explainers for Graph Neural Networks (GNNs) are commonly evaluated by their plausibility, i.e., how well their explanations recover a predefined ground truth, such as a motif planted in the data. This protocol implicitly assumes that a GNN trained on such data relies on the intended motif. Although prior work has questioned this assumption, plausibility remains widespread. First, we show that the assumption is violated on several widely used benchmarks, where, e.g., degree statistics alone suffice to solve the task. Then, we remove this confounder by replacing training with compilation. We achieve this by introducing Gracr\mathsf{Gracr}, the first compiler translating graded modal logic formulas into GNN weights, yielding models that replicate the behaviour of the corresponding formulas. Since the behaviour of the model is now known by construction, we can define its ground truth explanation formally and compute it exactly. Building on this, we introduce GracrBench\mathsf{Gracr}\mathsf{Bench}, a benchmark of compiled GNNs for the evaluation of explainers against this exact ground truth. Experiments on eleven explainers across six tasks show its effectiveness for fine-grained diagnostic evaluation: notably, we discover that most explainers are not robust to indirect influences or alternative implementations of the same formula. These results position GracrBench\mathsf{Gracr}\mathsf{Bench} as a novel, rigorous evaluation setting for graph post-hoc explainability.

Figures & tables

Figure 1: We manually reviewed 155155 published GNN explainability papers ( 20192019 – 20252025 ), recording how they evaluate explanation quality. Plausibility remains a core criterion , used alone in at least 20%20\% of papers each year. Numbers above bars indicate the number of papers analysed; details in Appendix G .
Figure 2: Evaluating the plausibility of explanations is misleading when the model is not behaving as expected. Example of a model trained on Mutagenicity , where plausibility penalizes well-behaved explainers. a) A correctly predicted mutagenic molecule, with the intended ground truth NO2\mathrm{NO_{2}} pattern highlighted as the explanation μ∗\mu^{*} . b) The same molecule, where we substituted the original μ∗\mu^{*} with a single hydrogen atom. Crucially, the model prediction does not change, suggesting it has learned an alternative decision rule μˉ∗\bar{\mu}^{*} (e.g., the carbon ring). As a result, a good explainer correctly providing μˉ∗\bar{\mu}^{*} as the explanation would be penalized in favour of a bad one providing μ∗\mu^{*} . In this work, we fix this issue by building GNNs whose behaviour is known, as described in Section 4.1 .
SubformulaCompiled weights
φi=¬φk\varphi_{i}=\neg\varphi_{k}Cki(ℓ)=−1C^{(\ell)}_{ki}=-1bi(ℓ)=1b^{(\ell)}_{i}=1
φi=φj∧φk\varphi_{i}=\varphi_{j}\wedge\varphi_{k}Cji(ℓ)=Cki(ℓ)=1C^{(\ell)}_{ji}=C^{(\ell)}_{ki}=1bi(ℓ)=−1b^{(\ell)}_{i}=-1
φi=◊≥Nφk\varphi_{i}=\Diamond_{\geq N}\varphi_{k}Aki(ℓ)=1A^{(\ell)}_{ki}=1bi(ℓ)=1−Nb^{(\ell)}_{i}=1-N
Table 1: Weights realising each connective.
Figure 4
 RedBlueLeafRedundancyCompetingRedundancy
ExplainerprecisionrecallF1F_{1}precisionrecallF1F_{1}precisionrecallF1F_{1}
Saliency1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}0.04\text{{\pm 0.00}}1.00\text{{\pm 0.00}}0.09\text{{\pm 0.00}}
IG1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}0.50\text{{\pm 0.00}}0.67\text{{\pm 0.00}}
Deconv0.22\text{{\pm 0.00}}1.00\text{{\pm 0.00}}0.37\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}0.04\text{{\pm 0.00}}1.00\text{{\pm 0.00}}0.09\text{{\pm 0.00}}
GuidedBP1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}0.04\text{{\pm 0.00}}1.00\text{{\pm 0.00}}0.09\text{{\pm 0.00}}
GNNExplainer1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}0.04\text{{\pm 0.00}}1.00\text{{\pm 0.00}}0.09\text{{\pm 0.00}}
Table 2: Plausibility on graph classification , averaged over 55 seeds.
 RedBlueLeaf-NCMajority1HopMajority2Hop
ExplainerprecisionrecallF1F_{1}precisionrecallF1F_{1}precisionrecallF1F_{1}
Saliency1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}0.87\text{{\pm 0.02}}0.92\text{{\pm 0.02}}
IG1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}0.96\text{{\pm 0.01}}1.00\text{{\pm 0.00}}0.97\text{{\pm 0.01}}
Deconv0.87\text{{\pm 0.00}}0.58\text{{\pm 0.00}}0.69\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}0.94\text{{\pm 0.02}}1.00\text{{\pm 0.00}}0.96\text{{\pm 0.01}}
GuidedBP1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}0.87\text{{\pm 0.02}}0.92\text{{\pm 0.02}}
GNNExplainer1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}0.96\text{{\pm 0.01}}0.99\text{{\pm 0.00}}0.97\text{{\pm 0.01}}
Table 3: Plausibility on node classification, averaged over 55 seeds. A ‘ OOT ’ marks an explainer that exceeded the per-explanation time budget (see Appendix H ) on every seed; a †\dagger marks one whose mean uses only seeds completed within the budget.
Figure 3: Ablation study on GNNExplainer over Redundancy . The plots report: Top: metric values for different values of the L1\mathrm{L}_{1} penalty regularization term controlling the sparsity strength; across all strength values, GNNExplainer maintains perfect precision and recall. Bottom: average edge-wise relevance scores for different values of the L1\mathrm{L}_{1} penalty regularization term, confirming that despite the penalty strength getting stronger, GNNExplainer does not sparsify redundant edges.
Figure 4: Deconv can assign high importance to irrelevant-by-design edges . To show this, we take a graph from Redundancy and progressively re-colour background nodes into red. Recall the underlying formula is ∃u.black(u)∧◊≥1 red\exists u.\mathrm{black}(u)\wedge\Diamond_{\geq 1}\,\mathrm{red} . In the original graph, grey-like colours play no role in the formula, and are correctly assigned zero importance. Hence, Deconv achieves perfect precision, as indicated also in Table 2 . However, as red nodes are added to the graph, precision steadily drops. This is because red nodes, although they do not contribute to the formula when present alone, do appear in the formula, which is enough for Deconv to highlight them as important. This is the same mechanisms that brings Deconv to perform at chance-level on RedBlueLeaf , as the background contains nodes firing intermediate sub-formulas (and intermediate sub-formulas only).
Figure 5: Explainers struggle to recover indirect influences. We illustrate this limitation using two representative examples from Majority2Hop evaluated across GuidedBP and GNNExplainer . Majority2Hop requires the target node to have strictly more 1-hop neighbors with a majority of red neighbors than neighbors with a majority of blue neighbors, with node colors reflecting their original features. On the left example graph, GNNExplainer assigns high importance ( ≥0.5\geq 0.5 ) to edges (101,713)(101,713) , (101,817)(101,817) , and (101,773)(101,773) ; however, because node 101101 has a blue-majority neighborhood, it does not contribute to the positive prediction of the target. On the right example graph, instead, GuidedBP assigns zero importance to edges (80,167)(80,167) , (80,52)(80,52) , (80,837)(80,837) , (110,837)(110,837) , and (528,837)(528,837) . Yet, these edges ensure that nodes 167167 , 5252 , and 837837 maintain an equal balance of red and blue neighbors, which prevents a higher number of blue-ish nodes in the neighborhood of node 8080 : if they were removed, the prediction at target node 8080 would flip. In both scenarios, the misattributed edges correspond to messages aggregated two hops away from the target. Notably, the true importance of (80,52)(80,52) , (80,837)(80,837) , and (80,167)(80,167) is directional (from node 8080 outward): while baseline explainers and our UPI\mathsf{UPI} algorithm evaluate directed edges, we display undirected attributions here for visual clarity.
Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Left: Degree distribution for each node class in BaShapes . Classes 00 and 11 are fully distinguishable, whereas classes 22 and 33 overlap. Right: Distribution of average neighbour degrees for classes 22 and 33 , showing that 2-hop degree statistics cleanly separate the remaining two classes.
Figure 7: Distribution of average node degree in BaMultiShapes , showing a clear separation between graphs in class 00 and class 11 .
Figure 8: 44 -hop neighbourhood of node 77 in Tree-Cycles . Original graph has all nodes predicted correctly. However, removing an edge outside of the ground truth explanation yields a prediction change for node 77 .
LayersTest AccuracyValidation Accuracy
220.9720±0.00450.9720\pm 0.00450.9818±0.00450.9818\pm 0.0045
330.9460±0.01410.9460\pm 0.01410.9939±0.00450.9939\pm 0.0045
440.9600±0.02450.9600\pm 0.02450.9859±0.00900.9859\pm 0.0090
Appendix
Table 4: Test and validation accuracy averaged across five random seeds for Tree-Cycles .
Figure 9: 44 -hop neighbourhood of node 1515 on Tree-Grid . Given a high-accuracy model, all nodes are predicted correctly in the original graph. Yet, removing edges outside of the ground truth explanation (right) yields a prediction change for node 1515 .
LayersTest AccuracyValidation Accuracy
20.8774±0.00600.8774\pm 0.00600.9073±0.00400.9073\pm 0.0040
30.9774±0.00940.9774\pm 0.00940.9870±0.00400.9870\pm 0.0040
40.9694±0.02360.9694\pm 0.02360.9740±0.01300.9740\pm 0.0130
50.9581±0.01390.9581\pm 0.01390.9707±0.01510.9707\pm 0.0151
Appendix
Table 5: Test and validation accuracy averaged across five random seeds for Tree-Grid .
 AccuracyF1F_{1} score
 MutagenicityBenzeneAlkaneCarbonylFluorideCarbonyl
GCN0.77 ±\pm 0.050.85 ±\pm 0.070.42 ±\pm 0.310.61 ±\pm 0.14
GIN0.80 ±\pm 0.020.91 ±\pm 0.030.70 ±\pm 0.170.71 ±\pm 0.10
GraphSAGE0.77 ±\pm 0.060.85 ±\pm 0.040.43 ±\pm 0.280.63 ±\pm 0.12
PNA0.80 ±\pm 0.020.99 ±\pm 0.020.71 ±\pm 0.180.77 ±\pm 0.11
GAT0.77 ±\pm 0.040.93 ±\pm 0.060.49 ±\pm 0.270.71 ±\pm 0.09
Appendix
Table 6: Mean 5-fold cross-validation test performance (accuracy for Mutagenicity and Benzene ; F1F_{1} score for AlkaneCarbonyl and FluorideCarbonyl ) for the best hyperparameter configuration across each architecture and dataset for the additional analysis provided in Section B.2 .
ModelPred. change (%)
GCN39%
GIN43%
GraphSAGE46%
PNA37%
GAT41%
Appendix
Table 7: Percentage of prediction changes on Mutagenicity across perturbed graphs for positive, mutagenic graphs containing NO2\mathrm{NO}_{2} or NH2\mathrm{NH}_{2} groups.
 Original ( ◊>\Diamond_{>} expanded)Alternative ( ◊>\Diamond_{>} direct)
ExplainerprecisionrecallF1F_{1}precisionrecallF1F_{1}
Majority1Hop
Saliency1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}0.90\text{{\pm 0.01}}1.00\text{{\pm 0.00}}0.94\text{{\pm 0.00}}
IG1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}
Deconv1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}
GuidedBP1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}
Appendix
Table 8: Plausibility under the ablation of the comparative operator , averaged over 55 seeds. The original columns compile ◊>(φ,ψ)\Diamond_{>}(\varphi,\psi) as the capped disjunction of Eq. 6 , the alternative ones as a single aggregation layer defined in Eq. 8 . The graphs, the labels, and UPI\mathsf{UPI} are identical in both cases and equal those of Table 3 ; only the compiled GNN changes, from 44 to 11 layers on Majority1Hop and from 88 to 22 on Majority2Hop , and from 203203 to 33 and 505505 to 55 hidden dimensions respectively. A ‘ oot ’ marks an explainer that exceeded the per-explanation time budget on every seed.
TaskprecisionrecallF1F_{1}
RedBlueLeaf1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}
Redundancy1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}
CompetingRedundancy1.00\text{{\pm 0.00}}0.50\text{{\pm 0.00}}0.67\text{{\pm 0.00}}
RedBlueLeaf-NC1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}
Majority1Hop1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}
Majority2Hop1.00\text{{\pm 0.00}}0.87\text{{\pm 0.02}}0.92\text{{\pm 0.02}}
Appendix
Table 9: IG under the surrogate backward pass of Eq. 9 , averaged over 55 seeds. The graphs, the labels, UPI\mathsf{UPI} , and the forward pass are identical to those in Table 2 and Table 3 .
 Original ( ∨\vee via De Morgan)Alternative ( ∨\vee flat gate)
ExplainerprecisionrecallF1F_{1}precisionrecallF1F_{1}
Saliency0.04\text{{\pm 0.00}}1.00\text{{\pm 0.00}}0.09\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}
IG1.00\text{{\pm 0.00}}0.50\text{{\pm 0.00}}0.67\text{{\pm 0.00}}1.00\text{{\pm 0.00}}0.50\text{{\pm 0.00}}0.67\text{{\pm 0.00}}
Deconv0.04\text{{\pm 0.00}}1.00\text{{\pm 0.00}}0.09\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}
GuidedBP0.04\text{{\pm 0.00}}1.00\text{{\pm 0.00}}0.09\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}
GNNExplainer0.04\text{{\pm 0.00}}1.00\text{{\pm 0.00}}0.09\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}
Appendix
Table 10: Plausibility on CompetingRedundancy over 55 seeds, for two compilations of the same disjunction , see Section C.5 . The original columns report the implementation of the disjunction “ φ∨ψ\varphi\vee\psi ” with De Morgan’s law “ ¬(¬φ∧¬ψ)\neg(\neg\varphi\wedge\neg\psi) ”. The alternative ones use the flat nn -ary gate ( Eq. 7 ), which puts no negation on any path.
 Original formula ( φ∧\varphi_{\wedge} )Alternative formula ( φ∨\varphi_{\vee} )
ExplainerprecisionrecallF1F_{1}precisionrecallF1F_{1}
Saliency1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}0.22\text{{\pm 0.00}}1.00\text{{\pm 0.00}}0.37\text{{\pm 0.00}}
IG1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}
Deconv0.22\text{{\pm 0.00}}1.00\text{{\pm 0.00}}0.37\text{{\pm 0.00}}0.22\text{{\pm 0.00}}1.00\text{{\pm 0.00}}0.37\text{{\pm 0.00}}
GuidedBP1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}0.22\text{{\pm 0.00}}1.00\text{{\pm 0.00}}0.37\text{{\pm 0.00}}
GNNExplainer1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}1.00\text{{\pm 0.00}}
Appendix
Table 11: Plausibility on RedBlueLeaf over 55 seeds, for two different formulas , see Section C.5 . The graphs, the labels, and UPI\mathsf{UPI} are identical for both formulas and equal to those evaluated in Table 2 . Only the compiled GNN changes. Given that the alternative, more complex formula yields a GNN of greater depth, execution is slower and FlowX exceeds the time budget.
Figure 10: The path graph of Example 2 (left) and the ROBDD of its edge-cover function f=x1∧x3f=x_{1}\wedge x_{3} under the order x1<x2<x3x_{1}<x_{2}<x_{3} (right). Dashed arcs are lo\mathrm{lo} (the variable is 00 , the edge is deleted), solid arcs are hi\mathrm{hi} (the variable is 11 , the edge is kept).
Figure 23
  cudd , static π\picudd , dynamic π\piautorefbrute force
NNN2N^{2}time (s)nodestime (s)nodestime (s)nodestime (s)
390.006<< 1k0.006<< 1k0.001<< 1k0 0.004
4160.006<< 1k0.006<< 1k0.001<< 1k0 0.050
5250.0071k0.0071k0.0031k48.041
6360.0022k0.0022k0.0062koot
8640.0027k0.0227k0.0207koot
Appendix
Table 12: Cost of extracting UPI\mathsf{UPI} with Algorithm 1 on complete graphs on NN nodes for the edge-cover classifier of Example 2 , against a brute-force baseline implementing Section 4.2 . Times are for one full extraction, nodes is the number of nodes in the ROBDD\mathsf{ROBDD} , and oot marks a configuration that was dropped after exceeding the budget of 11 second on a smaller size.
     time (s)
Dataset∣V∣|\mathcal{V}|∣E∣|\mathcal{E}|∣ET∣|\mathcal{E}_{T}|Num. nodesBUILD\mathrm{BUILD}Algorithm 2Algorithm 3total
graph classification
RedBlueLeaf329898.04.6k (155k)0.0124 ±\pm 0.01850.0002 ±\pm 0.00000.0003 ±\pm 0.00000.0130 ±\pm 0.0185
Redundancy37134134.0338 (338)0.0009 ±\pm 0.00010.0004 ±\pm 0.00000.0005 ±\pm 0.00000.0018 ±\pm 0.0002
CompetingRedundancy37134134.0342 (342)0.0022 ±\pm 0.00470.0004 ±\pm 0.00000.0005 ±\pm 0.00000.0030 ±\pm 0.0047
node classification
Appendix
Table 13: Cost of extracting UPI\mathsf{UPI} with Algorithm 1 on the benchmark datasets. Times are wall-clock seconds for one complete extraction on a single explained instance (one graph for graph classification, one target node for node classification), split into the three stages of the algorithm, as mean ±\pm standard deviation over the first 200200 explained instances of each dataset. ∣V∣|\mathcal{V}| and ∣E∣|\mathcal{E}| are the size of the graph, and ∣ET∣|\mathcal{E}_{T}| is the average size of the receptive field of the target. The ‘ Num. nodes ’ column indicates the mean number of nodes in the ROBDD\mathsf{ROBDD} , with the maximum reported in parentheses.
 time (s)
DatasetGNNExplainerGraphMaskSubgraphX
graph classification
RedBlueLeaf0.554 ±\pm 0.3800.609 ±\pm 0.2921.523 ±\pm 0.409
Redundancy0.066 ±\pm 0.0020.019 ±\pm 0.00242.464 ±\pm 1.846
CompetingRedundancy1.001 ±\pm 0.0290.023 ±\pm 0.00144.986 ±\pm 1.890
node classification
Appendix
Table 14: Wall-clock seconds to explain one instance, per explainer and per dataset, averaged over random seeds. A †\dagger marks an explainer that did not finish every instance within the per-explanation time-out, so that its mean is computed on the instances that did finish.
DatasetgraphsnodesedgesUPI\mathsf{UPI} edgesUPI\mathsf{UPI} fraction
RedBlueLeaf1000320 98.022.00.22
Redundancy100037134.00 6.00.04
CompetingRedundancy100037134.00 6.00.04
RedBlueLeaf-NC00 47502192.010.70.29
Majority1Hop3–45001992.00 2.30.60
Majority2Hop3–45001992.017.10.43
Appendix
Table 15: Size of the datasets, averaged over the positive samples of the 55 evaluation seeds. Nodes and edges are counted per graph, and edges are counted as directed, i.e., an undirected edge contributes two. For node classification we keep sampling graphs until the dataset holds at least 500500 explainable targets, so the number of graphs is a range over the seeds rather than a fixed size. The last column is the fraction of edges that the explainers are asked to recover. For node classification, the denominator is the receptive field of the target.
ExplainerHyper-parameters and ranges
GNNExplainerepochs ∈{50,100,1000}\in\{50,100,1000\} ; lr ∈[10−4,10−1]log⁡\in[10^{-4},10^{-1}]_{\log} ; edge_size ∈[10−4,10−1]log⁡\in[10^{-4},10^{-1}]_{\log} ; edge_ent ∈[10−4,10−1]log⁡\in[10^{-4},10^{-1}]_{\log}
GraphMaskepochs ∈{5,10,25,50}\in\{5,10,25,50\} ; lr ∈[10−5,10−2]log⁡\in[10^{-5},10^{-2}]_{\log} ; penalty_scaling ∈{1,…,10}\in\{1,\dots,10\} ; allowance ∈[0,0.3]\in[0,0.3]
SubgraphXmax_nodes ∈{5,…,20}\in\{5,\dots,20\} ; min_atoms ∈{3,…,max_nodes}\in\{3,\dots,\texttt{max\_nodes}\} ; sample_num ∈{10,20,50}\in\{10,20,50\} ; rollout ∈{3,5,10,20}\in\{3,5,10,20\} ; expand_atoms ∈{2,…,10}\in\{2,\dots,10\} ; local_radius ∈{1,2,3}\in\{1,2,3\}
PGMExplainernum_samples ∈{50,100,200,500}\in\{50,100,200,500\} ; perturbation_mode ∈{randint,zero,mean,max}\in\{\texttt{randint},\texttt{zero},\texttt{mean},\texttt{max}\} ; significance_threshold ∈[10−3,10−1]log⁡\in[10^{-3},10^{-1}]_{\log} ; pred_threshold ∈[0,0.5]\in[0,0.5] ; is_perturbation_scaled ∈{false,true}\in\{\texttt{false},\texttt{true}\} ; max_subgraph_size ∈{1,…,50}\in\{1,\dots,50\} (graph tasks only)
GraphSVXnum_samples ∈{100,200,400,800}\in\{100,200,400,800\} ; S ∈{1,…,4}\in\{1,\dots,4\} ; args_coal ∈{Smarter,Smart,Random}\in\{\texttt{Smarter},\texttt{Smart},\texttt{Random}\} ; args_g ∈{WLS,WLR_sklearn}\in\{\texttt{WLS},\texttt{WLR\_sklearn}\} ; use_abs ∈{true,false}\in\{\texttt{true},\texttt{false}\}
Appendix
Table 16: Hyper-parameter search space of each explainer , shared across all tasks. [a,b]log⁡[a,b]_{\log} denotes a log-uniform range, [a,b][a,b] a uniform one, and {⋅}\{\cdot\} a categorical choice. The other explainers are run in their default configuration.
 Majority2HopRedBlueLeaf-NC
Explainerprecisionrecallprecisionrecall
FlowX0.888\text{{\pm 0.150}}0.942\text{{\pm 0.103}}0.994\text{{\pm 0.028}}0.998\text{{\pm 0.014}}
FlowX (original)0.888\text{{\pm 0.150}}0.943\text{{\pm 0.103}}0.994\text{{\pm 0.028}}0.998\text{{\pm 0.014}}
Appendix
Table 17: Comparison in performance of the new implementation of FlowX , reporting mean ±\pm standard deviation for explanations across sample nodes, for a single seed.
More in this topic

GNN Explainability

Graph neural network explainability studies how to identify and communicate the features, nodes, edges, and subgraphs behind graph-model predictions.

See all 80 papers

Explore similar work

CardsList
Jul 6, 2026

Measuring What Matters: A Unified Evaluation Framework for GNN Explainability

Graph eXplainable AI (G-XAI) is increasingly important for making Graph Neural Networks interpretable and accountable. While a growing number of explainers are available, choosing the right method and assessing the trustworthiness of its outputs remains unclear. Consistent evaluation practices and actionable guidance are still missing, hindering practical adoption. In this paper, we introduce a unified, quantitative benchmarking framework for G-XAI that requires no ground-truth assumptions. We formalize tabular explainability metrics for graph data, evaluating topological structure and node features as independent components. Our large-scale benchmarking study identifies explainers that consistently lie on the Pareto front across metric pairs and tasks, establishing robustly non-dominated solutions - while confirming that no single explainer achieves universal superiority. We distill our findings into actionable G-XAI usability guidelines to support Machine Learning practitioners in evaluating and deploying trustworthy GNN-based pipelines.

Francesco Paolo Nerini, Mirko Zaffaroni, Paolo Baracco +2

Sapienza University of Rome, Rome, Italy · Intesa Sanpaolo AI Research, Turin, Italy · Anti Financial Crime Digital Hub, Turin, Italy +1

Explainability EvaluationGNN Explainability
AskPDF
Sep 26, 2026

Controllable GNN Explanations via Multi-Metric Preference Selection

Mechanisms for generating GNN explanations are crucial for building trust and mitigating biases in Graph Neural Networks (GNNs), especially in high-stakes scenarios. Most current methods optimize only for fidelity under the sparsity constraint. However, this discounts the need for interpretable explanations (those that consist of familiar motif patterns) and stable explanations (those that remain unchanged under structural perturbations). We propose a novel approach that optimizes GNN explanations across these metrics, exposing their relative weighing as a control. Experiments on various real-world datasets, including MUTAG, BA-2Motif, BAMultiShapes, and PROTEINS, suggest that our method produces higher-fidelity explanations than a state-of-the-art baseline on MUTAG and PROTEINS across all evaluated budgets, and on BA-2Motif at larger budgets, while being faster in the regime of small explanation budgets. They also indicate that our method produces more stable explanations for the MUTAG and BA-2Motif datasets. We also examine the relationship among different metrics via their induced trade-off surface and explore its dependence on the nature of the motif library.

Rachit Verma, Yashraj J. Deshmukh, Anirban Dasgupta

Indian Institute of Technology Gandhinagar

GNN ExplainabilityExplanation StabilityExplainability EvaluationMulti-Objective Optimization
AskPDF
Oct 19, 2026

Declarative Explanations for Graph Neural Networks: A Demonstration

Graph Neural Networks (GNNs) have proven effective for graph analytical tasks, yet their "black-box" nature complicates the understanding of their layer-wise behavior and hinders their application to large graphs. We demonstrate SliceGXQ, an end-to-end declarative and interactive system for querying and exploring layer-wise explanations for GNNs over large-scale graphs. In particular, SliceGXQ provides (1) an interactive graphical user interface for dataset, model, and explanation configuration; (2) a lightweight SPARQL-like parser that allows users to specify target nodes, layers, explanation modes, and constraints; and (3) an integrated execution and visualization workflow that invokes SliceGX as its underlying explanation engine. We demonstrate how users can diagnose a misclassified node, identify problematic layers and influential messages, and perform targeted model debugging, together with large-scale graph interpretation and fraud-detection analysis. Our demonstration video and source code are available at https://github.com/jinzhaoqet/SliceGXQ.

Cibo Yu, Haitong Tang, Yinghui Wu +4

Zhejiang University, Hangzhou, China · Case Western Reserve University, Cleveland, USA · Bowling Green State University, Bowling Green, USA

GNN ExplainabilityGraph Neural Networks
AskPDF