Modern neural networks have shown promise for solving partial differential equations over surfaces, often by discretizing the surface as a mesh and learning with a mesh-aware graph neural network. However, graph neural networks suffer from oversmoothing, where a node's features become increasingly similar to those of its neighbors. Unitary graph convolutions, which are mathematically constrained to preserve smoothness, have been proposed to address this issue. Despite this, in many physical systems, such as diffusion processes, smoothness naturally increases and unitarity may be overconstraining. In this paper, we systematically study the smoothing effects of different GNNs for dynamics modeling and prove that unitary convolutions hurt performance for such tasks. We propose relaxed unitary convolutions that balance smoothness preservation with the natural smoothing required for physical systems. We also generalize unitary and relaxed unitary convolutions from graphs to meshes. In experiments on PDEs such as the heat and wave equations over complex meshes and on weather forecasting, we find that our method outperforms several strong baselines, including mesh-aware transformers and equivariant neural networks.
Figures & tables
Figure 1: Qualitative comparison of autoregressive model predictions for the heat equation on the armadillo mesh at timestep T=190 . Our R-UniMesh model remains faithful to the ground truth during each step of the rollout, whereas the EMAN model over smooths and the Hermes model under smooths. A more complete comparison over several timesteps is in Section C.4 , Table 3 .
Figure 2: Top: After zero padding, individual unitary blocks are stacked and the output is fed into an unconstrained smoothness-breaking component. Bottom: Each block uses Taylor truncated unitary convolution.
Model
MSE (↓)
MRE (↓)
GCN
1.08⋅10−2
5.99⋅10−2
Lie Uni
0.14⋅10−2
8.86⋅10−2
R-UniGraph (Ours)
0.11⋅10−2
2.07⋅10−2
Table 1: MSE and mean Rayleigh quotient error (MRE) of GCN, a Lie unitary convolution network, and R-UniGraph . The best run for each method out of 5 runs is shown. The best performance is bold.
Figure 3: Left: The dotted lines indicate the mean Rayleigh quotient for input heat graphs at T=3 and target graphs at T=4 . We also show the mean Rayleigh quotient for the best performing GCN, R-UniGraph , and Lie unitary models. R-UniGraph is best at capturing the true smoothness. Right: Validation MSE of the same three models. R-UniGraph has the best performance. Results for the full set of runs are provided in Section B.3 .
Convolutional
Attentional
Message Passing
Metric
GCN
GemCNN
R-UniMesh (Ours)
EMAN
Transformer
MPNN
EGNN
Hermes
Heat (α=1)
NRMSE (↓)
–
–
51.9±3.6
73.50±3.8
92.5±5.6
99.45±4.8
344.25±110.5
73.02±4.7
SMAPE (↓)
–
375.4±0.53
79.7±5.6
110.9±13.3
213.9±2.7
223.6±1.5
319.33±7.59
107.6±7.4
RE (↓)
–
52.21±9.4
9.1±7.4
14.2±1.4
46.0±3.7
76.06±3.6
81.5±8.77
39.76±4.7
Wave (c=1)
Table 2: NRMSE, SMAPE, and RE averaged over all rollouts on all test meshes for the heat, wave, and Cahn-Hilliard equations. The best values are in bold and second best are underlined. Errors and standard deviations are reported over all test meshes and initializations. Cells with a dash (–) indicate models that do not converge for a given metric. Errors are scaled by ×196 , the number of rollout timesteps. R-UniMesh is competitive across all tasks and excels at solving the heat equation on unseen meshes. Models are grouped together by flavor: convolutional, attention, or message passing ( Bronstein et al., 2021 ) .
Figure 4: The Rayleigh quotient for each timestep on an unseen mesh for Hermes, EMAN, and R-UniMesh models. The R-UniMesh is the best at capturing the true smoothness for heat.
Figure 5: RMSE and ACC as a function of lead time for all models temperature prediction. R-UniMesh has a competitive RMSE, especially at early lead time. R-UniMesh also maintains viability for lead times of roughly 2 days according to the ECMWF baseline. Exact values recorded in Table 13 ( Section D.4 ).
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Illustration of an intrinsic Delaunay edge flip performed by the Robust Laplacian edge rewiring algorithm. This figure is a reproduced version of Figure 7, Sharp and Crane (2020) .
Figure 7: Sample heat diffusion process on a grid discretized as a graph. Node neighbors are the nodes that sit adjacent in the grid.
Figure 8: KL divergence between distribution of Rayleigh quotients before and after applying the model. Results are averaged over 10 runs.
Figure 9: Validation MSE and mean Rayleigh quotient for 5 different Lie Unitary Convolution runs. MSE tends to increase as Tmax increases. Tmax=3 offers the best balance between accuracy under MSE loss and smoothness errors.
Figure 10: Top: Validation MSE for an ensemble of 5 runs for a GCN (left), R-UniGraph (middle), and a Lie unitary convolution network (right) at timestep t=3 . R-UniGraph significantly outperforms the GCN and also outperforms the Lie unitary network. Bottom: The average Rayleigh quotient over all graphs for an ensemble of 5 runs for the same models at timestep t=3 . The GCN is under constrained and biased towards oversmoothing at initialization. R-UniGraph is able to roughly match the true smoothness of the labeled graphs. The Lie unitary network is overconstrained and can not model the Rayleigh quotient of the labels because it is forced to preserve the Rayleigh quotient of the input graphs.
Table 3: Qualitative comparison of model performance for the heat equation on the armadillo mesh. Our R-UniMesh model remains faithful to the ground truth during each step of the rollout, whereas the EMAN model over smooths and the Hermes model under smooths.
Figure 11: Smoothness for a Hermes model as measured by the 2− point correlation function. The plot indicates undersmoothing in each radial bin.
Heat ( α=1 )
Model
errsmooth(↓)
GemCNN
–
EMAN
4.04×10−3
Hermes
9.71×10−3
Wave ( c=1 )
Model
errsmooth(↓)
Appendix
Table 4: errsmooth for Gauge Equivariant models on the MeshPDE datasets. Dashes (–) indicate non-convergence. Best performing model is indicated with bold text .
Metric
R-UniMesh
MeshGraphNet
Heat (α=1)
NRMSE (↓)
51.9±3.6
108.1±3.2
SMAPE (↓)
79.7±5.6
144.1±6.3
RE (↓)
9.1±7.4
53.4±1.6
Wave (c=1)
NRMSE (↓)
236.5±6.4
253.6±28.2
Appendix
Table 5: NRMSE, SMAPE, and RE averaged over all rollouts on all test meshes for the heat, wave, and Cahn-Hilliard equations. The best values are in bold. R-UniMesh outperforms MeshGraphNet on heat and is competitive on the other PDEs.
Metric
Unitary GCN ( T=10 )
Unitary GCN ( T=3 )
Unconstrained GCN
Heat
NRMSE (↓)
51.9±3.6
70.1±6.3
416.9±14.3
SMAPE (↓)
79.7±5.6
76.7±10.4
248.8±6.6
RE (↓)
9.1±7.4
9.0±0.6
31.5±8.6
Wave
NRMSE (↓)
236.5±6.4
242.2±4.1
196.0±0.0
Appendix
Table 6: Comparison for the smoothness-preserving component type on NRMSE, SMAPE, and RE averaged over all rollouts on all test meshes for the heat, wave, and Cahn-Hilliard equations. The best values are in bold.
Metric
Zero Pad
GCN
Heat
NRMSE (↓)
51.9±3.6
4,689.6±232.0
SMAPE (↓)
79.7±5.6
367.7±0.6
RE (↓)
9.1±7.4
153.9±13.4
Wave
NRMSE (↓)
236.5±6.4
12,532.0±242.2
Appendix
Table 7: Comparison of Zero Pad and GCN mapping strategies for increasing the latent dimension on NRMSE, SMAPE, and RE averaged over all rollouts on all test meshes for the heat, wave, and Cahn-Hilliard equations. The best values are in bold.
Metric
GroupSort
ReLU
Heat
NRMSE (↓)
51.9±3.6
87.7±10.5
SMAPE (↓)
79.7±5.6
82.9±3.6
RE (↓)
9.1±7.4
9.0±10.5
Wave
NRMSE (↓)
236.5±6.4
825.7±12.3
Appendix
Table 8: Comparison of GroupSort and ReLU activation functions on NRMSE, SMAPE, and RE averaged over all rollouts on all test meshes for the heat, wave, and Cahn-Hilliard equations. The best values are in bold.
Metric
MLP (Depth = 1)
MLP (Depth = 3)
Heat
NRMSE (↓)
99.9±13.1
51.9±3.6
SMAPE (↓)
181.6±4.0
79.7±5.6
RE (↓)
21.2±7.1
9.1±7.4
Wave
NRMSE (↓)
−
236.5±6.4
Appendix
Table 9: Effect of MLP depth on NRMSE, SMAPE, and RE averaged over all rollouts on all test meshes for the heat, wave, and Cahn-Hilliard equations. The best values are in bold.
Metric
MLP (Width = 64)
MLP (Width = 128)
Heat
NRMSE (↓)
51.9±3.6
88.2±6.8
SMAPE (↓)
79.7±5.6
136.3±4.7
RE (↓)
9.1±7.4
10.7±2.3
Wave
NRMSE (↓)
236.5±6.4
1,433.0±35.0
Appendix
Table 10: Effect of MLP width on NRMSE, SMAPE, and RE averaged over all rollouts on all test meshes for the heat, wave, and Cahn-Hilliard equations. The best values are in bold.
Table 11: Sample initializations for the heat and wave equations on the PyVista meshes.
Table 12: Sample initializations for the Cahn-Hilliard equation on the PyVista meshes.
Figure 12: RMSE and ACC as a function of lead time for all models geopotential prediction. R-UniMesh has a competitive RMSE, especially at early lead time. R-UniMesh also maintains viability for lead times of roughly 2 days according to the ECMWF baseline.
Model
Metric
6h
12h
18h
24h
30h
36h
42h
48h
240h
EGNN
Z500 ACC
0.909
0.946
0.948
0.912
0.835
0.747
0.639
0.538
-0.134
Z500 RMSE
347.83
266.96
261.09
340.43
467.68
581.20
706.01
812.70
3769.31
T850 ACC
0.840
0.923
0.894
0.817
0.725
0.637
0.541
0.456
0.093
T850 RMSE
1.94
1.32
1.54
2.02
2.46
2.81
3.17
3.47
6.71
EMAN
Z500 ACC
0.114
0.125
0.121
0.123
0.120
0.122
0.123
0.123
0.127
Z500 RMSE
39530.01
52028.53
65037.68
89210.70
103179.47
119529.81
139314.85
156786.23
4898180.46
Appendix
Table 13: Weather Forecasting Results, ACC and RMSE at Different Lead Times for Temperature and Geopotential.
Model
RE Temperature (↓)
RE Geopotential (↓)
GCN
6.8⋅10−2±1.7⋅10−2
1.2⋅10−3±1.0⋅10−3
MPNN
3.0⋅10−3±2.7⋅10−3
1.3⋅10−3±8.5⋅10−4
Hermes
3.1⋅10−1±7.5⋅10−1
1.0⋅10−1±3.6⋅10−1
GemCNN
8.9⋅10−4±7.0⋅10−4
9.4⋅10−4±6.1⋅10−4
EMAN
7.7⋅100±5.2⋅100
4.1±2.6⋅10−2
EGNN
1.1⋅10−3±7.3⋅10−4
8.3⋅10−4±1.0⋅10−3
Appendix
Table 14: Rayleigh error for all models for all initializations on WB 2 temperature and geopotential. Best performing model is indicated with bold text . Errors scaled up by ×40 .
Unitarity is a useful principle for stabilizing deep neural networks, but in graph neural networks (GNNs) instability is induced not only by learnable parameters but also by the graph propagation operator. Motivated by this distinction, we propose Graph Unitary Message Passing (GUMP), a message-passing framework that uses a unitary propagation operator on a transformed graph to avoid graph-induced exponential decay under repeated propagation. GUMP combines (i) a graph transformation that maps an input graph to an Eulerian line-graph construction admitting unitary adjacency matrices, and (ii) a practical unitary projection procedure based on Newton-Schulz iteration. Theoretical analysis clarifies that, under standard analysis assumptions, unitary propagation keeps the graph-propagation term depth-stable, while vanilla normalized propagation exhibits exponential decay in its non-trivial spectral components. Across synthetic long-range tasks, TUDataset benchmarks, and LRGB datasets, GUMP improves over vanilla message passing and achieves competitive or superior performance against strong baselines. Code is available at https://github.com/ucker/gump_code.
Haiquan Qiu, Quanming Yao
Department of Electrical Engineering, Tsinghua University · 2Beijing National Research Center for Information Science and Technology · 3State Key laboratory of Space Network and Communications
Oversmoothing is a fundamental limitation of deep graph neural networks (GNNs), where repeated message passing causes node representations to become increasingly similar, eventually collapsing toward a low-dimensional subspace. This phenomenon limits the effective depth of message-passing architectures and motivates the search for mechanisms that preserve representation diversity. In this paper, we study a recurrent graph neural network in which independent Gaussian noise is injected after every propagation step and analyze the resulting architecture as a stochastic dynamical system. Under a standard global contraction assumption on the deterministic update, we prove that the hidden representations form a geometrically ergodic Markov chain admitting a unique invariant probability measure. Our main theoretical result establishes an explicit positive lower bound on the expected stationary Dirichlet energy, proportional to both the noise variance and the spectral gap of the underlying graph. Consequently, the stationary representations cannot collapse onto the constant manifold, providing a rigorous guarantee that asymptotic oversmoothing is prevented in the sense of non-vanishing Dirichlet energy. Our analysis reveals persistent stochastic perturbations as a fundamentally different mechanism for combating oversmoothing, complementing existing deterministic approaches based on residual connections, normalization, and graph rewiring. Finally, numerical experiments on both linear and nonlinear recurrent graph neural networks closely match the theoretical predictions, illustrating the emergence of a stationary distribution and the predicted dependence of the limiting Dirichlet energy on the noise intensity.
Mostafa Haghir Chehreghani
Department of Computer Engineering Amirkabir University of Technology (Tehran Polytechnic) Tehran, Iran
Graph Neural Networks (GNNs) have emerged as a cornerstone of deep learning, with most existing methods rooted in graph signal processing and diffusion equations to model message passing. However, these approaches inherently suffer from the oversmoothing problem, where node features become indistinguishable as the network depth increases. Inspired by the Navier Stokes equations, we introduce Graph Navier Stokes Networks (GNSN), a novel architecture that transcends conventional diffusion-based message passing by incorporating convection into graph structures. GNSN defines a dynamic velocity field on the graph to govern convection, enabling more efficient and direct message propagation. By adaptively balancing convection and diffusion, GNSN is able to efficiently handle datasets with varying levels of homophily. Extensive evaluations across twelve real-world datasets demonstrate that GNSN consistently outperforms state-of-the-art baselines in classification accuracy. Moreover, experimental results further emphasize its effectiveness in alleviating the oversmoothing problem.
Zexing Zhao, Guangsi Shi, Yu Gong +4
Northwest A&F University Yangling, Shaanxi, China · Corporate Research Center, Midea Group Shanghai, China · Bosch Shanghai, China