GIST: Gauge-Invariant Spectral Transformers for Scalable Graph Neural Operators

Mattia Rigotti; Nicholas Thumiger; Thomas Frick

cs.LGMar 17, 2026

GIST: Gauge-Invariant Spectral Transformers for Scalable Graph Neural Operators

Authors: Mattia Rigotti, Nicholas Thumiger, Thomas Frick

Organizations: IBM Research

Abstract

Neural operators on irregular meshes face a fundamental tension. Spectral positional encodings, the natural choice for capturing geometry, require cubic-complexity eigendecomposition and inadvertently break gauge invariance through numerical solver artifacts; existing efficient approximations sacrifice gauge symmetry by design. Both failure modes break discretization invariance: models fail to transfer across mesh resolutions of the same domain, and similarly across different graphs of related structure in inductive settings. We propose GIST (Gauge-Invariant Spectral Transformer), a scalable neural operator that resolves this tension by restricting attention to pairwise inner products of efficient approximate spectral embeddings. We prove these inner products estimate an exactly gauge-invariant graph kernel at end-to-end $\mathcal{O}(N)$ complexity, and establish a formal connection between gauge invariance and discretization-invariant learning with bounded mismatch error. To our knowledge, GIST is the first scalable graph neural operator with a provable discretization-mismatch bound. Empirically, GIST sets state-of-the-art on the AirfRANS, ShapeNet-Car, DrivAerNet, and DrivAerNet++ mesh benchmarks (up to 750K nodes), and additionally matches strong baselines on standard graph benchmarks (e.g., 99.50% micro-F1 on PPI).

Figures & tables

Figure 1: Gauge-Invariant Spectral Transformer. Left : Gauge-Invariant Spectral Self-Attention uses spectral embeddings

\tilde{\phi}

as queries and keys and node features

x

as values; restricting

\tilde{\phi}

to the query-key role preserves gauge invariance. Softmax shown for clarity; GIST uses linear attention in practice. Right : the multi-scale GIST block combines three parallel branches inspired by EfficientViT.

Architecture	Train $R^{2}$ (coarse mesh) $\uparrow$	Test $R^{2}$ (fine mesh) $\uparrow$
GIST (gauge-invariant)	0.989 $\pm$ 0.012	0.840 $\pm$ 0.051
Non-gauge-invariant	0.991 $\pm$ 0.010	0.526 $\pm$ 0.081

Table 1: Controlled discretization-invariance check on a single DrivAerNet geometry: surface pressure trained on a 50% decimated mesh and evaluated on the full-resolution mesh. The gauge-invariant architecture transfers across resolutions; the matched non-gauge-invariant baseline collapses, isolating gauge invariance as the responsible factor (

R^{2}

as mean

\pm

std over 6 seeds).

Model	Cora (accuracy $\uparrow$ )	PubMed (accuracy $\uparrow$ )	Photo (accuracy $\uparrow$ )	ArXiv (accuracy $\uparrow$ )	PPI (micro-F1 $\uparrow$ )
GCN [ 61 , 62 , 63 , 13 ]	85.10 $\pm$ 0.67	81.12 $\pm$ 0.52	88.26 $\pm$ 0.37 ∗	71.74 $\pm$ 0.29	51.50 $\pm$ 0.60 †
GraphSAGE [ 61 , 64 , 63 ]	83.88 $\pm$ 0.65	79.72 $\pm$ 0.50	–	71.49 $\pm$ 0.27	76.80
GAT [ 61 , 64 , 63 , 13 ]	84.46 $\pm$ 0.55	80.28 $\pm$ 0.64	90.94 $\pm$ 0.34 ∗	71.59 $\pm$ 0.38	97.30 $\pm$ 0.02
GATv2 [ 65 , 63 ]	82.90	78.70	–	71.87 $\pm$ 0.25	96.30
GaAN [ 66 ]	–	–	–	–	98.71
Cluster-GCN [ 67 ]	–	–	–	–	99.36

Table 2: Node classification on standard graph benchmarks. GIST is competitive across all five benchmarks at end-to-end

\mathcal{O}(N)

complexity, achieving SOTA on Photo and matching SOTA on PPI. Cora, PubMed, Photo, and ArXiv report test accuracy; PPI reports micro-F1; all metrics are percentages and higher-is-better, reported as mean

\pm

std across 6 random seeds for GIST. Sources are cited per model for baselines. ∗ source reports 95% CI rescaled to std. † source does not specify the uncertainty type.

Table 4

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 2: Sensitivity study of GIST spectral embeddings parameters. The plots show the final test accuracy of a two-block Gauge-Invariant Spectral Self-Attention linear transformer trained on Cora while sweeping over the power iteration parameter

k

with

r=256

(left panel), and sweeping over the embedding dimension

r

with

k=32

(right panel). Test accuracy is fairly robust around the best value of either parameter. As expected,

r

is monotonically related to higher performance, as higher

r

correspond to better approximations of the eigenmaps. Accuracy conveniently saturates relatively fast, justifying the use of reasonably low

r

. The plots show mean test accuracy averaged across 10 seeds and corresponding standard deviation as error bars.

Figure 3: Scalability study of GIST. All experiments use a fixed 3-layer model while varying the hidden dimensionality. VRAM consumption was measured as a function of the number of nodes in the input graph. Graph sizes were controlled using random node dropout applied to samples from the DrivAerNet dataset, enabling a systematic evaluation of memory scaling behavior.

Ablation	Test Accuracy (% baseline)	$\Delta$ (%)
No ablation	100.0	0.0
Branch 1 (feature)	$95.0\pm 2.8$	$-5.0$
Branch 2 (local)	$92.1\pm 1.8$	$-7.9$
Branch 3 (global)	$95.7\pm 3.7$	$-4.3$

Appendix

Table 5: Ablation study on PPI dataset showing the contribution of each branch of the Multi-Scale GIST (see Figure 1 , right panel). Test accuracy is reported as percentage of baseline performance. Results averaged over 10 seeds with standard deviations indicated as uncertainty intervals.

Graph Neural Operators

See all 10 papers

Joint Spatiotemporal Spectral Neural Operators for Learning PDEs on Irregular Domains
Aug 30, 2026Abdolmehdi Behroozi, Chaopeng ShenPDE Surrogate Modeling PDE Operator Learning
Department of Civil and Environmental Engineering, Penn State University, University Park, PA 16802, USA
Topology-Preserving Neural Operator Learning via Hodge Decomposition
May 13, 2026Dongzhe Zheng, Tao Zhong, Christine Allen-BlanchetteHodge Decomposition PDE Operator Learning
Princeton University
Gauge-Equivariant Intrinsic Neural Operators for Geometry-Consistent Learning of Elliptic PDE Maps
Mar 16, 2026Pengcheng ChengPDE Operator Learning Neural Operators
School of Mathematics, Jilin University, Changchun, 130012, China

Aug 30, 2026J/K move · Enter open · S save

Joint Spatiotemporal Spectral Neural Operators for Learning PDEs on Irregular Domains

Abdolmehdi Behroozi, Chaopeng Shen

Department of Civil and Environmental Engineering, Penn State University, University Park, PA 16802, USA

PDE Surrogate Modeling PDE Operator Learning Spectral Methods Neural Operators

Open paper AskPDF arXiv

Learning solution operators for partial differential equations (PDEs) on irregular and geometry-dependent domains remains a central challenge in scientific machine learning. While spectral methods provide strong inductive biases for modeling global interactions, they are typically limited to regular domains, and existing neural approaches often require domain warping, interpolation, or costly geometric embeddings. We introduce the Graph Spectral Neural Operator (GSNO), a neural operator that combines spatial graph spectral decompositions with temporal Fourier transforms through a unified space--time spectral kernel. This formulation enables globally coherent operator learning on non-Cartesian discretizations without domain warping or autoregressive rollouts. By replacing learned geometric embeddings with a graph Laplacian spectral basis, GSNO provides geometry-aware spectral learning with low parameter complexity. Across steady and unsteady PDE benchmarks on irregular and geometry-dependent domains, GSNO achieves strong accuracy with reduced runtime and parameter counts, while demonstrating robust zero-shot generalization across mesh resolutions and geometry families.

Abstract

Graph Neural Operators

Explore similar work

Joint Spatiotemporal Spectral Neural Operators for Learning PDEs on Irregular Domains

Topology-Preserving Neural Operator Learning via Hodge Decomposition

Gauge-Equivariant Intrinsic Neural Operators for Geometry-Consistent Learning of Elliptic PDE Maps