Organizations: Department of Mathematics, Faculty of Science National University of Singapore Singapore · School of Physical and Mathematical Sciences Nanyang Technological University Singapore
To combat oversmoothing in Graph Convolutional Networks, Sheaf Neural Networks (SNNs) were proposed as a generalization by equipping the graph with a sheaf structure and replacing the graph Laplacian with a sheaf Laplacian L. Existing analyses connect sheaf diffusion to oversmoothing via the harmonic space (kerL), taking its absolute dimension as an indicator of anti-oversmoothing capacity. However, absolute dimension alone is not a reliable measure: certain sheaf configurations inflate dimkerL while their harmonic sections remain entirely constant, without enriching discriminative capacity. We instead introduce the first relative, geometric approach, yielding a precise characterisation of anti-oversmoothing capacity. Under natural conditions on stalk transportation and global sheaf structure, we establish an index-theoretic comparison criterion showing that one sheaf's harmonic space genuinely contains another's beyond trivial inflation. We illustrate this with a concrete instance and further introduce GyroSheaf, a sheaf with curved gyrovector-space stalks, extending the criterion to the non-linear setting via local tangent-space linearization. Experiments across ten models confirm the theoretical criterion: sheaf models violating the criterion collapse despite possessing index jumps, while compliant models maintain depth-stable representations.
Figures & tables
Figure 1: Normalized Dirichlet energy (top) and MAD (bottom) on Cora, Citeseer, and Texas in the untrained regime. Curves and shaded bands show per-seed mean and min–max range over five seeds.
Figure 2: Normalized Dirichlet energy (left), MAD (middle), and test accuracy (right) on Cora in trained regime. Legend and shading as in Figure 1 .
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 3: Asymptotic decay of R(F) in trivial sheaves ( dim(H0)=1) .
Figure 4: Mitigation of representation collapse through intermediate harmonic enrichment.
Figure 5: Robust information preservation in high-capacity geometric sheaf frameworks.
Hyperparameter
Value
Learning rate
0.004
Weight decay
10−4
Epochs
220
Patience
50
Label smoothing
0.05
LR scheduler
ReduceLROnPlateau
Appendix
Table 1: Training hyperparameters (shared across all model families).
Figure 6: PCA projections of input features (left), GyroSheaf (middle), and SPDSheaf (right) at the clean ( η=0 , top) and strongly-perturbed ( η=0.8 , bottom) endpoints; ellipses show per-class covariance. Under strong perturbation, SPDSheaf collapses into a narrow band while GyroSheaf retains separable class structure. Right: test accuracy versus AffErr(X) . SPDSheaf degrades monotonically as AffErr grows; GyroSheaf remains flat.
Sheaf Neural Networks (SNNs) were introduced as an extension of Graph Convolutional Networks to address oversmoothing on heterophilous graphs by attaching a sheaf to the input graph and replacing the adjacency-based operator with a sheaf Laplacian defined by (learnable) restriction maps. Prior work motivates this design through theoretical properties of sheaf diffusion and the kernel of the sheaf Laplacian, suggesting that suitable non-identity restriction maps can avoid representations converging to constants across connected components. Since oversmoothing can also be mitigated through residual connections and normalization, we revisit a trivial sheaf construction to ask whether the additional complexity of learning restriction maps is necessary. We introduce an Identity Sheaf Network baseline, where all restriction maps are fixed to the identity, and use it to ablate the empirical improvements reported by sheaf-learning architectures. Across five popular heterophilic benchmarks, the identity baseline achieves comparable performance to a range of SNN variants. Finally, we introduce the Rayleigh quotient as a normalized measure for comparing oversmoothing across models and show that, in trained networks, the behavior predicted by the diffusion-based analysis of SNNs is not reflected empirically. In particular, Identity Sheaf Networks do not appear to suffer more significant oversmoothing than their SNN counterparts.
Ferran Hernandez Caralt, Mar Gonzàlez i Català, Adrián Bazaga +1
Department of Computer Science and Technology, University of Cambridge, Cambridge, United Kingdom · Microsoft
Neural Sheaf Diffusion (NSD) generalizes diffusion-based Graph Neural Networks by replacing scalar graph Laplacians with sheaf Laplacians whose learned restriction maps define a task-adapted geometry. While the diffusion limit of NSD is known to be the space of global sections, the representation-theoretic structure of this harmonic space remains largely implicit. We develop a quiver-theoretic interpretation of NSD by identifying cellular sheaves on graphs with representations of the associated incidence quiver. Under this correspondence, learned sheaf geometries become points in a finite-dimensional representation space. We show that direct-sum decompositions of the underlying incidence-quiver representation induce decompositions of the harmonic space reached in the diffusion limit. This gives an algebraic interpretation of oversmoothing as representation degeneration: learned sheaves may collapse toward low-complexity summands whose global sections fail to preserve discriminative information. Building on this viewpoint, we connect sheaf diffusion to stability and moment-map principles from Geometric Invariant Theory. We introduce moment-map-inspired regularizers that bias restriction maps toward balanced representation geometries, and identify a structural obstruction in equal-stalk architectures: when dv=de, admissibility for learnable stability parameters forces the trivial all-object summand onto a stability wall. Non-uniform stalk dimensions remove this obstruction, making adaptive stability meaningful. Experiments on heterophilic benchmarks are consistent with this mechanism: breaking stalk symmetry can reduce variance or improve validation behavior, and adaptive stability becomes more effective in selected rectangular settings. Overall, our framework reframes oversmoothing as a degeneration phenomenon in the representation geometry underlying learned sheaf diffusion.
Arif Dönmez, Axel Mosig, Ellen Fritsche +1
IUF – Leibniz Research Institute for Environmental Medicine, Düsseldorf, Germany · DNTOX GmbH, Düsseldorf, Germany · Bioinformatics Group, Ruhr University Bochum, Bochum, Germany +1
Graph Neural Networks (GNNs) often struggle to capture long-range dependencies due to over-squashing -- a phenomenon in which the repeated compression of node embeddings into finite-size messages causes representations to collapse. Over-squashing is most often diagnosed as a property of the graph topology, with effective resistance serving as a principled measure of the bottleneck. We provide a complementary view on the matter: building on cellular sheaves, we introduce sheaf effective resistance, a generalization of effective resistance that depends on the sheaf attached to the graph, and we prove that for flat vector bundles, the over-squashing sensitivity in the Jacobian sense is upper bounded by a quantity related to the sheaf effective resistance between the nodes. The bottleneck thus need not lie in the graph itself: it can be relocated, and reduced, by adjusting the sheaf. We instantiate this idea in FlatNSD, a simple message-passing variant of Neural Sheaf Diffusion, and show that it implicitly learns to modulate total sheaf effective resistance, performing well on benchmarks designed to stress over-squashing without altering the original graph topology.
André Ribeiro, Germano Barcelos, Amauri H. Souza +2
Getulio Vargas Foundation · Federal University of Viçosa · Federal Institute of Ceará, 2δ AI +2