Probing Graph Neural Network Activation Patterns Through Graph Topology
Authors: Floriano Tori, Lorenzo Bini, Marco Sorbi, Stéphane Marchand-Maillet, Vincent Ginis
Organizations: Business Technology and Operations, Vrije Universiteit Brussels, Brussels, Belgium · Department of Computer Science, University of Geneva, Geneva, Switzerland · Research Institute for Statistics and Information Science, Geneva, Switzerland · School of Engineering and Applied Sciences, Harvard University
Curvature notions on graphs provide a theoretical description of graph topology, highlighting bottlenecks and denser connected regions. Artifacts of the message passing paradigm in Graph Neural Networks, such as oversmoothing and oversquashing, have been attributed to these regions. However, it remains unclear how the topology of a graph interacts with the learned preferences of GNNs. Through Massive Activations, which correspond to extreme edge activation values in Graph Transformers, we probe this correspondence. Our findings on synthetic graphs and molecular benchmarks reveal that MAs do not preferentially concentrate on curvature extremes, despite their theoretical link to information flow. On the Long Range Graph Benchmark, we identify a systemic curvature shift: global attention mechanisms exacerbate topological bottlenecks, drastically increasing the prevalence of negative curvature. Our work reframes curvature as a diagnostic probe for understanding when and why graph learning fails.
Figures & tables
Type
Name
Description
0
Cl - Cl
Between clique nodes
1
S - BS
Source node to bridge node
2
BT - T
Bridge node to target node
3
Bridge
Bridge edge (between source and target)
4
BridgeDummy
Dummy bridge edge (between dummy and target)
Table 1: Edge type encoding for the (modified) barbell graph.
Figure 1: The standard barbell graph (a.), the simple modified barbell graph (b.) and the extended modified barbell graph (c.) considered. The source node (orange) contains a signal to be received by the target node (blue). The modified barbells add one or more dummy cliques containing nodes with no information about the task. These dummy cliques contain a dummy signal node (pink). The green bridge connects the source and target cliques (task-relevant), while pink bridges connect dummy cliques to the target (task-irrelevant).
Table 2: Signal reconstruction MSE on barbell graph variants (Train / Test).
Figure 2: Activation Ratios for the three barbell graphs: standard (a., d.), modified (b., e.) and extended modified (c., f.). The top row (a., b., c.) shows the ratios when the edge features are topologically accurate, while the bottom row (d., e., f.) when the features are permuted. The dummy bridges in the modified and extended activation ratios for the task-relevant bridge edge (green) and the dummy bridges (blue) in the modified datasets show the same (or higher) activation ratios. Note the log-scale on the y -axis.
Figure 3: Proportions of massively activated edges (based on 95th percentile) within each curvature value per model and per dataset ( (a.) ZINC (b.) Tox21 ) We show for each curvature value ( x-axis ) the relative proportion (model-wise) of edges that have been designated as massively active. For each model we shade the region based on number of edges that are present relative to all massively activated edges of that model
Figure 4: Enrichment values for all three models over both datasets: ZINC (a.) and Tox21 (b.). The enrichment values show that the relation between curvature and massive activations is non-monotonic, where topological bottlenecks (indicated by extreme negative curvatures) are marginally over-represented. Additionally, positively curved edges are also over-represented. The dotted line indicates E=1 meaning no over- or underrepresentation.
Figure 5: Layer-wise evolution of activation ratios of MAs across curvature bins for Tox21 (a) and ZINC (b). Each panel corresponds to a BFc value and lines show mean activation ratio per layer. ZINC exhibits uniform early-layer decay across all curvature values, while Tox21 maintains elevated ratios throughout, with GT showing late-layer spikes at negative curvatures.
Dataset
Static Graph
Activation Graph
peptides-func
57%
84%
peptides-struct
57%
82%
Table 3: Prevalence of Negative Curvature Edges. The significant increase showcases the ”Curvature Collapse” phenomenon.
Figure 6: Shift in Effective Geometry. The distribution of Activation-Weighted Curvature (Red) is shifted significantly to the left compared to the Static Curvature (Blue), indicating a system-wide increase in negative curvature (bottlenecks).
Pruning Target
Region Type
Func ( Δ Loss)
Struct ( Δ Loss)
Set A (MA + Neg)
Bottleneck
+0.1124
+0.1327
Set B (MA + Pos)
Grid/Flat
+0.0153
+0.0302
Set C (NoMA + Neg)
Bottleneck
+0.0506
+0.0790
Table 4: Impact of Causal Pruning on Loss. Retrieving Set A (MA on bottlenecks) consistently hurts performance, confirming the model relies on these inefficient structures. Details in Appendix C .
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Type
Name
Description
0
Cl - Cl
Intra-clique edge
1
S - BS
Source node to bridge-connected node
2
BS - T
Bridge-connected node to Target node
3
Bridge
The critical bottleneck edge
4
BridgeDummy
Bridge to a dummy clique
Appendix
Table 5: Edge type encoding for the Barbell graph.
Figure 7: The Geometry Shift. The distribution of edge curvature weights shifts significantly to the left (more negative) in the Activation Graph (Red) compared to the Static Graph (Blue). This visualizes the Curvature Collapse .
Figure 8: Spectral Gap Comparison. The Activation Graph (Red) consistently exhibits a lower spectral gap than the Static Graph (Blue), indicating reduced global connectivity.
Figure 9: MA Length Distribution. MAs are overwhelmingly concentrated on short-range (0-1 hop) interactions.
Figure 10: Diagnostic Plots. (a) High entropy persists at negative curvature. (b) Geometric distortion correlates with higher error (Slope: -1.662).