A Bochner Weitzenböck Decomposition for Cellular Sheaf Laplacians, Curvature and Neural Sheaf Diffusion
Authors: Francesco Maria Guadagnuolo, Alessio Borgi, Dorina Thanou, Pietro Liò
Organizations: Sapienza University of Rome, Rome, Italy · Italian Institute of Artificial Intelligence (AI4I), Turin, Italy · University of Cambridge, Cambridge, United Kingdom · ´Ecole Polytechnique F´ed´erale de Lausanne (EPFL), Lausanne, Switzerland
We extend the Bochner-Weitzenböck decomposition to cellular sheaf Laplacians, splitting it into a positive semidefinite term, the Bochner sheaf Laplacian, and a block diagonal term, the Forman curvature. We show that the Bochner Laplacians form a class of cellular sheaves that extends both connection and matrix-weighted Laplacians, and generalizes signed graphs to d dimensions. We give a holonomy characterization of their kernel, prove the related Cheeger inequalities, and attach to them a random walk that generalizes the Gremban expansion. For the Forman term, we prove a discrete Bochner theorem, which bounds the first cohomology of a cellular sheaf with nonnegative curvature, and we read the whole decomposition as an opinion-dynamics model. We then apply this machinery to Sheaf Neural Networks, whose learnt Laplacian is otherwise unobserved. The Bochner interpretation yields a node-level conviction score that acts as an operational importance measure under frozen-model interventions, while the Forman profile provides a local descriptor of the learnt operator.
Figures & tables
Figure 1: The Sd−1 and O(d) synchronization problems on a single edge, d=2 . (a) Each node carries a unit vector. The angular factor W10⊤ , here a rotation by 90∘ , transports x0 to the stalk of node 1 , and the frustration of the edge is the discrepancy between x1 and W10⊤x0 (dashed), weighted by the radial factor M10 . The ellipse below the edge draws M10 , with principal axes labelled by its largest and smallest eigenvalues w10+ and w10− : a discrepancy along the first costs more than one along the second, while the dashed circle is the isotropic case of a connection Laplacian. (b) Each node carries an orthogonal frame gv∈O(d) , drawn as its two columns, and the frustration is g1−W10⊤g0M10,F .
Figure 2: The random walk of a Bochner matrix as a generalized Gremban expansion. (a) For the signed triangle with w01=−3 (dashed) and w02=1 , the Gremban expansion doubles each node into v+ and v− . From node 0 the walker moves to node 1 with probability 3/4 and to node 2 with probability 1/4 , in proportion to ∣w0u∣ ; crossing the negative edge flips its sign, 0±→1∓ , and crossing the positive one keeps it, 0±→2± . (b) For d>1 each node is lifted to a sphere Sd−1 , and the jump probability p0u(x^)=x^⊤P0ux^ depends on the direction the walker carries: in this example, from y it reaches nodes 1 and 2 with probabilities 3/4 and 1/4 , from x with 1/2 each (Definition 4.3 ).
Figure 3: The three examples of this section, with d=1 . (a) The empty triangle of Example 5.1 : each arrow carries the restriction map from a vertex to an edge, all equal to 1 except F0⊴[0,2]=F1⊴[1,2]=2 . (b) The star S4 of Example 5.2 , with maps 1 from the center and A from the leaves. (c) The heptagonal bipyramid of Example 5.3 seen from above: the vertices A and B , one on each side of the equator, project onto the same point at the center. Next to each vertex is its value of q , which sets the restriction maps of the construction.
Zero masking
Budget (fraction of the training pool)
Selection
1%
5%
10%
20%
30%
High conviction
0.51±0.17
1.34±0.39
2.18±0.70
3.53±1.26
4.55±1.83
Low conviction
0.02±0.01
0.04±0.04
0.07±0.07
0.20±0.14
0.39±0.32
High degree
0.37±0.16
0.75±0.28
1.06±0.40
1.76±0.68
2.46±0.98
Low degree
0.03±0.02
0.19±0.10
0.38±0.21
0.85±0.44
1.35±0.74
Degree-matched (high)
0.06±0.07
0.20±0.17
0.38±0.32
0.57±0.51
0.93±0.62
Table 1: Relative drop in performance (%) under zero masking, by selection strategy and budget. Mean ±s.e. across 8 datasets, each averaged over its folds. In each column, the most damaging and the least damaging selection are highlighted.
Table 2: 1 -Wasserstein distance between the Forman profile of the learnt normalized GeneralSheaf and the topological reference, by training epoch (rows; 0 is the untrained sheaf) and relative depth of the layer (columns). Darker cells are farther from the reference: the distance shrinks with training and with depth. Mean ±s.e. across 10 datasets, each averaged over its folds; 8 for the middle layer, which networks with L=2 lack.
Appendix figures & tables20 assets
Supplementary material from the paper’s appendix.
Appendix
Checkpoint
Observable
Untrained
Best epoch
Spearman ρ with degree
0.07±0.10
0.33±0.07
Spearman ρ with betweenness
0.05±0.08
0.28±0.06
Spearman ρ with confidence
0.17±0.07
0.66±0.05
Spearman ρ with saliency
0.07±0.10
−0.18±0.09
∣ Cohen’s d∣ , train split
0.21±0.05
0.54±0.09
Appendix
Table 3: Spearman correlation between the conviction and four node properties, and absolute Cohen’s d between the conviction of correctly and incorrectly classified nodes, at the last diffusion layer. Untrained is the epoch- 0 checkpoint, a control.
Mean masking
Budget (fraction of the training pool)
Selection
1%
5%
10%
20%
30%
High conviction
0.50±0.16
1.34±0.39
2.19±0.72
3.61±1.30
4.66±1.85
Low conviction
0.01±0.02
0.08±0.05
0.14±0.10
0.36±0.17
0.57±0.38
High degree
0.38±0.16
0.94±0.38
1.35±0.59
2.13±0.94
2.91±1.17
Low degree
0.03±0.02
0.18±0.11
0.41±0.22
0.88±0.48
1.43±0.80
Degree-matched (high)
0.09±0.07
0.30±0.21
0.52±0.37
0.77±0.63
1.21±0.71
Appendix
Table 4: Budget sweep under mean masking, laid out as Table 1 . High conviction is the most damaging selection at every budget, low conviction the least.
Hard gating
Budget (fraction of the training pool)
Selection
1%
5%
10%
20%
30%
High conviction
0.81±0.29
1.80±0.41
2.77±0.66
4.32±1.27
5.67±1.89
Low conviction
0.05±0.03
0.11±0.04
0.18±0.09
0.43±0.22
0.71±0.45
High degree
0.65±0.20
1.37±0.37
1.91±0.45
2.83±0.73
3.67±1.04
Low degree
0.05±0.02
0.26±0.12
0.58±0.27
1.20±0.59
1.91±0.98
Degree-matched (high)
0.12±0.05
0.37±0.19
0.68±0.35
1.02±0.60
1.44±0.86
Appendix
Table 5: As Table 4 , under hard gating. At the 1% budget low degree is marginally the least damaging.
Pruning
Budget (fraction of the training pool)
Selection
1%
5%
10%
20%
30%
High conviction
1.65±0.96
2.95±1.10
4.02±1.16
5.64±1.49
7.08±2.04
Low conviction
0.01±0.01
0.07±0.05
0.16±0.11
0.42±0.28
0.81±0.57
High degree
1.36±0.77
2.68±1.16
3.48±1.24
4.82±1.38
5.77±1.57
Low degree
0.04±0.03
0.24±0.14
0.54±0.30
1.13±0.67
1.88±1.10
Degree-matched (high)
0.20±0.05
0.52±0.21
0.89±0.41
1.29±0.76
1.73±1.09
Appendix
Table 6: As Table 4 , under pruning.
Zero masking
Ranking criterion
Bin
Conviction
Confidence
Degree
1
0.07±0.07
0.35±0.21
0.38±0.21
2
0.09±0.08
0.36±0.21
0.42±0.22
3
0.06±0.20
0.39±0.26
0.41±0.24
4
0.17±0.25
0.42±0.25
0.37±0.27
5
0.37±0.26
0.48±0.38
0.45±0.29
Appendix
Table 7: Decile experiment under zero masking. The training pool is split into ten equal bins by each ranking criterion (columns), from the lowest-scoring bin 1 to the highest-scoring bin 10 (rows), and one bin is perturbed at a time. The conviction has the largest drop at bin 10 , and most of its growth occurs in the last two bins.
Zero masking, avg_z
Budget (fraction of the training pool)
Selection
1%
5%
10%
20%
30%
High conviction
0.56±0.17
1.68±0.57
2.65±0.98
4.10±1.61
5.18±2.21
Low conviction
0.01±0.01
0.02±0.03
0.02±0.06
0.13±0.18
0.34±0.29
High degree
0.37±0.16
0.75±0.28
1.06±0.40
1.76±0.69
2.46±0.98
Low degree
0.03±0.02
0.19±0.10
0.38±0.21
0.85±0.44
1.35±0.74
Degree-matched (high)
0.07±0.04
0.19±0.15
0.27±0.26
0.52±0.49
0.77±0.55
Appendix
Table 8: As Table 1 , with nodes ranked by the layer-averaged score avg_z . The effect is unchanged, and slightly sharper.
Soft gating
Ranking score
Relative drop (%)
Confidence
0.67±0.37
Conviction
3.09±0.95
Degree
4.19±1.95
Appendix
Table 9: Relative drop under soft gating, which attenuates the outgoing message of every training node in proportion to its score (gate floor gmin=0.1 ); with no budget, each ranking score gives one number.
Zero masking, high
Budget (fraction of the training pool)
Dataset
Selection
1%
5%
10%
20%
30%
Amazon Ratings
Degree
1.25±0.17
1.75±0.28
2.14±0.32
3.14±0.32
3.90±0.27
Conviction
1.16±0.17
2.31±0.26
2.93±0.29
3.81±0.28
4.48±0.28
Citeseer
Degree
−0.04±0.16
−0.39±0.21
−0.65±0.35
−1.25±0.82
−1.74±1.05
Conviction
0.06±0.17
0.41±0.21
0.62±0.22
1.20±0.31
0.67±0.46
Cora
Degree
0.46±0.17
0.87±0.20
0.99±0.37
1.53±0.44
2.21±0.45
Appendix
Table 10: Budget sweep under zero masking, dataset by dataset, for the high-degree and high-conviction selections. Unlike the other tables, the spread is the standard error across the ten folds of each dataset.
Zero masking, low
Budget (fraction of the training pool)
Dataset
Selection
1%
5%
10%
20%
30%
Amazon Ratings
Degree
0.02±0.05
−0.02±0.08
0.05±0.12
0.48±0.18
0.73±0.20
Conviction
−0.02±0.03
−0.02±0.08
0.07±0.09
0.13±0.10
0.56±0.22
Citeseer
Degree
−0.04±0.04
−0.05±0.11
−0.12±0.16
−0.15±0.22
−0.29±0.35
Conviction
0.08±0.09
−0.12±0.18
−0.30±0.20
−0.62±0.27
−1.60±0.67
Cora
Degree
0.00±0.00
0.09±0.06
0.09±0.14
0.11±0.19
0.14±0.18
Appendix
Table 11: As Table 10 , for the low-degree and low-conviction selections.
Pruning
Budget (fraction of the training pool)
Selection
1%
5%
10%
20%
30%
High conviction
7.7±3.1
15.9±3.5
22.0±3.4
31.3±3.1
39.4±2.8
Low conviction
0.5±0.1
2.7±0.5
5.6±1.0
11.6±1.8
18.0±2.5
High degree
10.2±3.4
23.5±5.1
32.7±5.5
44.2±5.1
51.6±4.3
Low degree
0.4±0.1
1.9±0.6
4.2±1.2
8.7±2.5
13.3±3.7
Degree-matched (high)
5.0±1.3
11.5±1.4
16.6±1.3
23.6±1.1
28.3±1.7
Appendix
Table 12: Percentage of the graph’s edges deleted under pruning, by selection strategy and budget; the performance cost of the same selections is in Table 6 . Selecting by degree deletes the most edges.
Table 13: W1 distance of the unnormalized GeneralSheaf from the topological reference (top) and the zero reference (bottom). The profile aligns with neither.
Table 14: W1 distance of the other Laplacian, with maps trained unnormalized and evaluated normalized (Appendix D.2 ), from the topological reference. Once normalized, the maps align with it as training proceeds.
Table 15: W1 distance of the other Laplacian, with maps trained normalized and evaluated unnormalized (Appendix D.2 ), from the topological reference (top) and the zero reference (bottom). The maps lose their alignment and stay far from the zero reference.
Table 16: W1 distance between the two Laplacians evaluated normalized, built from the maps trained normalized and unnormalized (cross-comparison, Appendix D.2 ). They stay apart and neither converge nor diverge.
Table 17: As Table 16 , with both Laplacians evaluated unnormalized. The distances are of the same size.
Table 18: W1 distance of the unnormalized JointSheafParams, started at the identity sheaf, from the topological reference (top) and the zero reference (bottom), the latter measuring the deviation from the identity. The maps deviate through the layers and are drawn back with training.
Table 19: W1 distance of the normalized JointSheafParams, started at the identity sheaf, from the topological reference, which here measures the deviation from the identity. The deviation grows with training. Tolokers is excluded since its training is unstable ( 9 datasets, 7 for the middle layer). On Cora, PubMed and Questions training was interrupted, and these datasets enter with the folds that completed (between two and six of ten).
Table 20: Normalized GeneralSheaf on Roman Empire, d=5 , five layers, against the topological reference.
Table 21: Unnormalized GeneralSheaf on Citeseer, d=5 , four layers, against the topological and the zero reference.
Table 22: Unnormalized JointSheafParams on Questions, d=5 , four layers, started at the identity sheaf, against the topological and the zero reference. In the first layer the profile is constant, so its correlation is undefined.
We construct a cellular sheaf from any feedforward ReLU neural network by placing one vertex for each intermediate quantity in the forward pass and encoding each computational step - affine transformation, activation, output - as a restriction map on an edge. The restricted coboundary operator on the free coordinates is unitriangular, so its determinant is 1 and the restricted Laplacian is positive definite for every activation pattern. It follows that the relative cohomology vanishes and the forward pass output is the unique harmonic extension of the boundary data. The sheaf heat equation converges exponentially to this output despite the state-dependent switching introduced by piecewise linear activations. Unlike the forward pass, the heat equation propagates information bidirectionally across layers, enabling pinned neurons that impose constraints in both directions, training through local discrepancy minimization without a backward pass, and per-edge diagnostics that decompose network behavior by layer and operation type. We validate the framework experimentally on small synthetic tasks, confirming the convergence theorems and demonstrating that sheaf-based training, while not yet competitive with stochastic gradient descent, obeys quantitative scaling laws predicted by the theory.
Vicente Bosca, Robert Ghrist
Department of Mathematics, University of Pennsylvania
Sheaf Neural Networks (SNNs) were introduced as an extension of Graph Convolutional Networks to address oversmoothing on heterophilous graphs by attaching a sheaf to the input graph and replacing the adjacency-based operator with a sheaf Laplacian defined by (learnable) restriction maps. Prior work motivates this design through theoretical properties of sheaf diffusion and the kernel of the sheaf Laplacian, suggesting that suitable non-identity restriction maps can avoid representations converging to constants across connected components. Since oversmoothing can also be mitigated through residual connections and normalization, we revisit a trivial sheaf construction to ask whether the additional complexity of learning restriction maps is necessary. We introduce an Identity Sheaf Network baseline, where all restriction maps are fixed to the identity, and use it to ablate the empirical improvements reported by sheaf-learning architectures. Across five popular heterophilic benchmarks, the identity baseline achieves comparable performance to a range of SNN variants. Finally, we introduce the Rayleigh quotient as a normalized measure for comparing oversmoothing across models and show that, in trained networks, the behavior predicted by the diffusion-based analysis of SNNs is not reflected empirically. In particular, Identity Sheaf Networks do not appear to suffer more significant oversmoothing than their SNN counterparts.
Ferran Hernandez Caralt, Mar Gonzàlez i Català, Adrián Bazaga +1
Department of Computer Science and Technology, University of Cambridge, Cambridge, United Kingdom · Microsoft
We introduce the Clifford Sheaf Neural Network (CSNN), an equivariant sheaf neural network for geometric graphs that places a Clifford algebra on each stalk of a cellular sheaf and transports multivector features along edges. The canonical choice of restriction map for sheaves with algebra-valued stalks is algebra homomorphism. Adding the constraint of equivariance, the naive choice becomes versor conjugation. However, versor conjugation is expressively weak, so we drop algebra homomorphism and arrive at the K-term sandwich. The resulting sheaf Laplacian is positive semidefinite by construction, needs no versor constraint, and still mixes grades. Our main contribution characterizes the resulting family of restriction maps along three axes: which grades a map couples, how much of the endomorphism space it reaches, and how well it is conditioned. The K-term sandwich spans half of the endomorphism space, and in Cl(3, 0, 0) it corresponds to the maps that commute with the central pseudoscalar. The number of terms controls expressivity. CSNN is the reversion member, a first-order model by construction and the grade-mixing corner of this family, developed as a sheaf construction for graph-level equivariant regression.