The Interpretable and Effective Graph Neural Additive Networks

Maya Bechler-Speicher; Amir Globerson; Ran Gilad-Bachrach

cs.LGJun 3, 2024

The Interpretable and Effective Graph Neural Additive Networks

Authors: Maya Bechler-Speicher, Amir Globerson, Ran Gilad-Bachrach

Organizations: Blavatnik School of Computer Science, Tel-Aviv University · Department of Bio-Medical Engineering, Edmond J. Safra Center for Bioinformatics, Tel-Aviv University

Abstract

Graph Neural Networks (GNNs) have emerged as the predominant approach for learning over graph-structured data. However, most GNNs operate as black-box models and require post-hoc explanations, which may not suffice in high-stakes scenarios where transparency is crucial. In this paper, we present a GNN that is interpretable by design. Our model, Graph Neural Additive Network (GNAN), is a novel extension of the interpretable class of Generalized Additive Models, and can be visualized and fully understood by humans. GNAN is designed to be fully interpretable, offering both global and local explanations at the feature and graph levels through direct visualization of the model. These visualizations describe exactly how the model uses the relationships between the target variable, the features, and the graph. We demonstrate the intelligibility of GNANs in a series of examples on different tasks and datasets. In addition, we show that the accuracy of GNAN is on par with black-box GNNs, making it suitable for critical applications where transparency is essential, alongside high accuracy.

Figures & tables

Panel of Figure 1 — Figure 1: Visualization of the distance and feature functions, learned on Mutagenicity. As the features are binary, the feature functions are evaluated only on the value $1$ . These plots provide an exact description of the functions’ signal processing and a global explanation of how the model uses the distances and features.

Figure 2: Visualization of products of the outputs of the distance function and the feature functions, trained on Mutagenicity. Each cell shows the exact contribution, positive or negative, of features at a certain distance to the prediction. Positive values (green) contribute to classifying a molecule as mutagenic, and negative values (red) contribute to classifying a molecule as non-mutagenic.

Figure 3: Visualization of the distance shape function learned on the PubMed dataset. As the output of the function is of dimension three, we plot it as three shape functions, one for each class. We plot them on the same figure to compare them. The shape functions show that the model uses only the local neighborhood of each node. It also shows a difference between the classes; while for type 2 diabetes, the longer the distance, the less their information is used (converges to 0), for type 1 and gestational diabetes, nodes of long-distance have a negative effect.

Figure 4: Visualization of nine features’ shape functions, learned over the PubMed dataset.

Figure 5: Visualization of the products between the outputs of the ’children’ feature function over the input range $[0,1]$ and the outputs of the distance function, learned over the PubMed dataset.

Panel of Figure 6 — Figure 6: Local explanations of two molecules from the Mutagenicity datasets, through visualizations of the molecules. The area of each atom corresponds to the node importance according to Equation 3 .

Model	Cornell	Tolokers	Cora	Citeseer	PubMed	ogb-arxiv
GraphConv	65.9 ± 0.5	83.5 ± 0.7	81.3 ± 1.1	75.9 ± 2.0	85.9 ± 0.5	72.4 ± 0.1
GraphSAGE	75.9 ± 5.0	82.4 ± 0.4	81.4 ± 0.7	76.4 ± 0.8	88.4 ± 0.4	71.7 ± 0.2
GIN	69.0 ± 1.3	81.0 ± 0.4	80.0 ± 1.2	77.1 ± 1.9	85.3 ± 0.9	73.8 ± 1.4
GATv2	72.5 ± 0.7	83.8 ± 1.1	83.1 ± 0.9	73.9 ± 1.5	84.4 ± 0.5	74.0 ± 2.1
GTransformer	70.5 ± 1.7	83.3 ± 0.9	80.7 ± 0.5	76.0 ± 0.9	85.3 ± 1.6	73.1 ± 0.2
FSGNN	86.0 ± 4.1	83.1 ± 0.6	83.0 ± 1.3	76.2 ± 1.3	85.0 ± 1.3	72.9 ± 1.7

Table 1: Evaluation of GNAN on node (top) and graph (bottom) tasks. The best and second-best models are marked in cyan and violet colors, respectively. We report accuracy and std for all tasks, except for the Tolokers dataset where we report ROC-AUC and std, and the

\mu

\alpha

\alpha_{HOMO}

datasets where we report MAE and std.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7:

Figure 8: Visualization of the products between the outputs of the ’fat’ feature function over the input range [0, 1] and the outputs of the distance function, learned over the PubMed dataset

Figure 9: Visualization of the products between the outputs of the ’young’ feature function over the input range [0, 1] and the outputs of the distance function, learned over the PubMed dataset

Panel of Figure 10 — Figure 10: Visualization of the feature functions learned over the Tolokers dataset.

Figure 11: isualization of the distance shape function learned on the Tolokers dataset.

Figure 12: Visualization of the products between the outputs of the continuous feature functions over the input range $[0,1]$ and the outputs of the distance function, learned over the Tolokers dataset.

Figure 13: Visualization of products of the outputs of the distance function and the feature functions, trained on Tolokers. Each cell shows the exact contribution, positive or negative, of features at a certain distance to the prediction. Positive values (green) contribute to classifying a toloker as ’banned’, and negative values (red) contribute to classifying a toloker as ’not banned’.

Dataset	# Graphs	Avg # Nodes	Avg # Edges	# Node Features	# Classes
Proteins [ 59 ]	1,113	39.06	72.82	3	2
NCI1 [ 59 ]	4,110	29.87	32.3	37	2
Mutagenicity [ 59 ]	4,337	30.32	30.37	7	2
PTC [ 59 ]	344	14	14	19	2
QM9 [ 60 ]	130,831	18	37.3	11	-
Cora [ 55 ]	1	2,708	10,556	1,433	7

Appendix

Table 2: Statistics of the real-world datasets used in our evaluation.

Graph Neural Networks

Graph Neural Networks learn from graph-structured data by combining information across connected nodes, edges, and larger structures.

See all 751 papers

CIExplainer++: Generating Causal and Interpretable Explanations for Graph Neural Networks
Jun 17, 2026Francisco Caldas, Sahil Satish Kumar, Ruben Belo +1Natural Language Explanation Generation Causal Interpretability
NOVA LINCS, NOVA School of Science and Technology, Lisbon, Portugal
GRAFT: Auditing Graph Neural Networks via Global Feature Attribution
May 5, 2026Rishi Raj Sahoo, Subhankar MishraFeature Attribution Graph Neural Networks
National Institute of Science Education and Research (NISER), Bhubaneswar, India · Homi Bhabha National Institute, Mumbai, India
Graph Additive Models disentangling feature, topology, and interactions
Oct 14, 2026Clara Alvarez-Rodriguez, Steve Azzolin, Vincenzo Marco De Luca +3Graph Neural Networks GNN Explainability
The Arctic University of Norway

Jun 17, 2026J/K move · Enter open · S save

CIExplainer++: Generating Causal and Interpretable Explanations for Graph Neural Networks

Francisco Caldas, Sahil Satish Kumar, Ruben Belo, Cláudia Soares

NOVA LINCS, NOVA School of Science and Technology, Lisbon, Portugal

Natural Language Explanation Generation Causal Interpretability GNN Explainability

Open paper AskPDF arXiv

Explainable Artificial Intelligence aims to make black-box models more trustworthy by presenting, in a human-understandable manner, the elements that lead to the model's output. This involves both (i) identifying components and connections with genuine causal influence on outputs and (ii) translating such structures into an interpretable representation. For the former, we introduce CIExplainer, a novel perturbation-based method grounded in causal inference for explaining Graph Neural Networks (GNNs). CIExplainer identifies the subgraph with the highest causal effects on GNN predictions using the Potential Outcome Framework. We evaluate and compare CIExplainer on various GNN architectures (GCN, GraphSAGE, GAT, GIN) and datasets. To bridge subgraph explanations with human interpretability, we further propose G2TeXplainer, a method that transforms causal subgraphs into natural language explanations that capture both feature-level and relational information.

Abstract

Graph Neural Networks

Explore similar work

CIExplainer++: Generating Causal and Interpretable Explanations for Graph Neural Networks

GRAFT: Auditing Graph Neural Networks via Global Feature Attribution

Graph Additive Models disentangling feature, topology, and interactions