Value or Attention: Investigating Layer-wise Relevance Propagation for Graph Attention Networks

Ken Schreiber; René P. Klausen; Sebastian Lapuschkin; Wojciech Samek; Jonas Naujoks

cs.LGNov 5, 2026

Value or Attention: Investigating Layer-wise Relevance Propagation for Graph Attention Networks

Authors: Ken Schreiber, René P. Klausen, Sebastian Lapuschkin, Wojciech Samek, Jonas Naujoks

Organizations: Department of Artificial Intelligence, Fraunhofer Heinrich Hertz Institute, Berlin, Germany · Centre of eXplainable Artificial Intelligence, Technological University Dublin, Ireland · Department of Electrical Engineering, Technische Universität Berlin, Germany · BIFOLD – Berlin Institute for the Foundations of Learning and Data, Berlin, Germany

Abstract

Graph Attention Networks (GATs) are increasingly used in domains where scientists need to read a model's reasoning, from medicine to materials science, yet the routing they learn remains opaque to most explainers. Layer-wise Relevance Propagation (LRP) follows the model's computation, but in an attention layer the relevance reaching a node is the sum of two mechanically different quantities: its content relevance through the value path and its routing relevance through the attention path. Existing rules either drop one or add them into a single number. We write a single product rule with free weights on the two factors that recovers current approaches as special cases. The value path measures what content a node supplied against a zero baseline; the attention path measures how the node shaped what was passed relative to its neighborhood, and its sign records whether the node's features moved the gating the way the prediction demanded. On a ground-truth classification task, we show that the same node receives negative, positive and near-zero relevance under three settings of the rule. The separated relevance maps yield a hypothesis about the model: it gains by not reading that node. We give reading rules for both paths and argue that they should be interpreted separately.

Figures & tables

Figure 1: Node relevance on one BA-House [ 15 ] graph, red positive, blue negative.

Figure 2: Node relevance for the house logit against degree, all nodes of $600$ positive graphs across 5 initializations. The 4.6% of nodes with degree higher than 7 are omitted here for clarity; Fig. 6 shows the full range. Grey: ordinary (non-anchor) nodes, from both the BA graph and the house; black bars: their per-degree mean. Blue / orange: anchor nodes of degree three / four; dark blue / dark orange bars: their means. $\Delta R$ : mean anchor relevance minus mean ordinary-node relevance at the same degree. $\Delta z$ : $\Delta R$ divided by the map’s standard deviation over all nodes, including those not drawn, which makes it independent of each method’s overall scale (per initialization, then averaged).

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

	value path	attention path
		$n$ as source		$n$ as destination
node map	$R^{\mathrm{val}}_{n}$	$\sum_{i\,:\,n\in\mathcal{N}(i)}r^{\mathrm{att}}_{in}u_{in}$		$\sum_{j\in\mathcal{N}(n)}r^{\mathrm{att}}_{nj}v_{nj}$
edge term	$\alpha_{in}\,g_{i}^{\top}m_{n}$	$r^{\mathrm{att}}_{in}\,u_{in}$		$r^{\mathrm{att}}_{nj}\,v_{nj}$
reference	$0$ baseline	neighborhoods reading $n$		$\mathcal{N}(n)$ , contrasted twice
scales with	content $m_{n}$	contrast in demand		contrast in demand and in supply
supply (below)	—	$u_{in}>0$	$u_{in}<0$	$v_{nj}>0$	$v_{nj}<0$

Appendix

Table 1: Readings of a positive attribution. The value path does not branch; the attention path branches by role and, within each role, by the direction on which demand and supply agreed. The negative case is given in Table 2 .

	value path	attention path
		$n$ as source		$n$ as destination
edge term	$\alpha_{in}\,g_{i}^{\top}m_{n}$	$r^{\mathrm{att}}_{in}\,u_{in}$		$r^{\mathrm{att}}_{nj}\,v_{nj}$
supply (below)	—	$u_{in}>0$	$u_{in}<0$	$v_{nj}>0$	$v_{nj}<0$
		$n$ pushes logit up	$n$ pushes logit down	$n$ rates $j$ above	$n$ rates $j$ below
demand (above)	—	$r^{\mathrm{att}}_{in}<0$	$r^{\mathrm{att}}_{in}>0$	$r^{\mathrm{att}}_{nj}<0$	$r^{\mathrm{att}}_{nj}>0$
		score edge down	score edge up	shift away from $j$	shift toward $j$

Appendix

Table 2: Readings of a negative attribution: the mirror of Table 1 , where demand and supply disagree. Note that within one neighborhood

\sum_{j\in\mathcal{N}(n)}r^{\mathrm{att}}_{nj}=0

, so a node acting as destination always faces demands of both signs.

	QKV-BA	BA-House	ESOL
graphs	BA $n{=}8$ , $m{=}2$	pos.: BA $n{=}15$ , $m{=}2$ ${+}$ HouseMotif ; neg.: BA $n{=}20$ , $m{=}2$ ; balanced	$1116$ Delaney molecules ( $1128$ before de-duplication)
node feat.	$x_{i}=[q_{i}\|k_{i}\|v_{i}]\in\mathbb{R}^{15}$ , $d_{q}{=}d_{k}{=}d_{v}{=}5$ , iid $U(-1,1)$	one-hot true degree, $10$ bins (loops excluded from the degree)	$35$ -dim one-hot (element, degree, charge, $n$ H, hybrid., aromaticity, ring)
edge feat.	—	—	$9$ -dim one-hot bond
task	regress one Transformer head (below), MSE	graph classification, cross-entropy	graph regression on standardized $\log S$ , MSE
model	GATv2 $h{=}16$ , $L{=}1$ , $629$ par.	GATv2 $h{=}32$ , $L{=}1$ , $834$ par.	GATv2 $h{=}64$ , $L{=}4$ , $32449$ par.
training	Adam, lr $10^{-2}$ , cosine, $60$ epochs, batch $64$	Adam, lr $10^{-2}$ , cosine, $30$ epochs, batch $32$	AdamW, lr $10^{-3}$ , cosine, $600$ epochs, batch $32$ , best-validation weights restored

Appendix

Table 3: Datasets, models and evaluation. All models share one skeleton:

L

message-passing layers of width

h

, one head, ReLU between convolutions, linear readout, sum-pooled where the task is graph-level. QKV-BA has no activation after the last convolution, its target being linear in

v

Model	$R^{2}$	Value path (q/k/v)	Attention path (q/k/v)
TransformerConv h=5	1.000	0.000 / 0.000 / 1.000	0.323 / 0.677 / 0.000
GATv2 h=5	0.413	0.019 / 0.043 / 0.939	0.168 / 0.166 / 0.666
GATv2 h=10	0.780	0.008 / 0.019 / 0.974	0.303 / 0.611 / 0.087
GATv2 h=16	0.950	0.005 / 0.003 / 0.992	0.326 / 0.610 / 0.064

Appendix

Table 4: QKV-BA: Mean fraction of

|R|

mass per block in node 0.

Figure 3: Shows separation of attributions by input block on separate path explanations.

Figure 4: ESOL : attention-path share

\sigma_{c}=|R^{\mathrm{att}}_{,c}|/(|R^{\mathrm{att}}_{,c}|+|R^{\mathrm{val}}_{,c}|)

per live input coordinate (

27

35

; the other

8

are always zero on ESOL ), at the layer-1 split with the value path in all layers above, attributed over all

1116

molecules and sorted by

\sigma_{c}

. Markers: mean

\pm

SD over 5 initializations. Grey bars:

95\%

bootstrap interval over molecules (

2000

resamples) for the best-validation model, centered on that model’s own estimate. Dashed line: global attention-path share, averaged over initializations (

0.240

; per initialization

0.201

–

0.280

). Hollow markers: coordinates present on fewer than

300

atoms, excluded from all statistics.

Figure 5: Three molecules spanning the solubility range. Top row: the value map summed over features per atom. Bottom row: the routing map, same atoms, same layer-1 split. Color is signed relevance normalized by the column maximum — value against routing on the same molecule is the comparison that is meaningful; magnitudes are not comparable across columns. Molecules are chosen deterministically for legibility, not for what they show.

Figure 6: Node relevance for the house logit against degree, full version of Fig. 2 .

Figure 7: (a) Every content reading crosses between degree three and four; each curve is divided by its own largest value, since only the ordering and the sign boundary are the claim. (b) Margin gain from suppressing the anchor minus the same quantity for an ordinary node of the same degree in the same graph, split into what removing content buys and what rerouting the freed attention buys on top. Degree-three anchors are the control. Bars are

95\%

cluster-bootstrap intervals.

Figure 8: Fidelity metrics of Yuan et al. [32] , with Fid+ and sparsity going back to Pope et al. [22] , which remove a node by masking the messages from and to it. Fid+: drop in predicted-class probability when the explanation is removed, so high means necessary. Fid-: the same drop when only the explanation is kept, so low means sufficient. Sparsity is the fraction of the graph not in the explanation; characterization score [ 3 ] is the harmonic mean of Fid+ and (1 - Fid-).

Figure 9: Fidelity+ with attention knockout instead of message masking: the most positive edges of each edge attribution map have their logits set to

-\infty

together, allowing neighborhood attention to renormalize. Same models and graphs as Fig. 8 .

\mathcal{E}

is the set of all edges. Up to sparsity of 0.70: 30% of edges removed.

GNN Explainability

Graph neural network explainability studies how to identify and communicate the features, nodes, edges, and subgraphs behind graph-model predictions.

See all 86 papers

Relevant Walk Search for Explaining Graph Neural Networks
May 22, 2026Ping Xiong, Thomas Schnake, Michael Gastegger +3Layer-Wise Relevance Propagation GNN Explainability
Technische Universität Berlin (TU Berlin) · BIFOLD – Berlin Institute for the Foundations of Learning and Data · Freie Universität Berlin (FU Berlin) +4
Efficient Higher-order Subgraph Attribution via Message Passing
May 21, 2026Ping Xiong, Thomas Schnake, Grégoire Montavon +2Feature Attribution Layer-Wise Relevance Propagation
Technische Universität Berlin (TU Berlin) · BIFOLD – Berlin Institute for the Foundations of Learning and Data · Department of Artificial Intelligence, Korea University, Seoul 136-713, Korea +2
GRAFT: Auditing Graph Neural Networks via Global Feature Attribution
May 5, 2026Rishi Raj Sahoo, Subhankar MishraFeature Attribution Graph Neural Networks
National Institute of Science Education and Research (NISER), Bhubaneswar, India · Homi Bhabha National Institute, Mumbai, India

May 22, 2026J/K move · Enter open · S save

Relevant Walk Search for Explaining Graph Neural Networks

Ping Xiong, Thomas Schnake, Michael Gastegger, Grégoire Montavon+2

Technische Universität Berlin (TU Berlin) · BIFOLD – Berlin Institute for the Foundations of Learning and Data · Freie Universität Berlin (FU Berlin)+4

Layer-Wise Relevance Propagation GNN Explainability

Open paper AskPDF arXiv

Graph Neural Networks (GNNs) have become important machine learning tools for graph analysis, and its explainability is crucial for safety, fairness, and robustness. Layer-wise relevance propagation for GNNs (GNN-LRP) evaluates the relevance of walks to reveal important information flows in the network, and provides higher-order explanations, which have been shown to be superior to the lower-order, i.e., node-/edge-level, explanations. However, identifying relevant walks by GNN-LRP requires {\em exponential} computational complexity with respect to the network depth, which we will remedy in this paper. Specifically, we propose {\em polynomial-time} algorithms for finding top-

K

relevant walks, which drastically reduces the computation and thus increases the applicability of GNN-LRP to large-scale problems. Our proposed algorithms are based on the max-product algorithm -- a common tool for finding the maximum likelihood configurations in probabilistic graphical models -- and can find the most relevant walks exactly at the neuron level and approximately at the node level. Our experiments demonstrate the performance of our algorithms at scale and their utility across application domains, i.e., on epidemiology, molecular, and natural language benchmarks. We provide our codes under github.com/xiong-ping/rel_walk_gnnlrp.

Abstract

GNN Explainability

Explore similar work

Relevant Walk Search for Explaining Graph Neural Networks

Efficient Higher-order Subgraph Attribution via Message Passing

GRAFT: Auditing Graph Neural Networks via Global Feature Attribution