Structure-preserving fidelity diagnostics for partial differential equations
Authors: Kyriakos Flouris
Organizations: MRC Biostatistics Unit, University of Cambridge, Cambridge, United Kingdom · King’s College, University of Cambridge, Cambridge, United Kingdom
Machine learning surrogates of partial differential equations (PDEs) are commonly trained on data generated by numerical solvers. Therefore, the training data obey not the PDE but the modified equation of the numerical scheme, and the test loss, computed on data from the same solver, cannot distinguish a model that learned the physics from one that learned the discretization error. Here, we show that the difference is nevertheless visible in the trained model. We propose a suite of structure-preserving fidelity diagnostics, simple scores that measure whether a learned model respects one known property of the physics, such as a conserved quantity, a symmetry or a Hamiltonian structure, without access to the training data or the governing equation. Over five systems, the diagnostics track the error with respect to the true dynamics with a rank correlation of up to 0.94, while the test loss is uncorrelated or anti-correlated with it. For the Burgers, Korteweg-de Vries and Navier-Stokes equations, the diagnostics separate numerical schemes that the test loss ranks as equivalent, including schemes of the same order. order.
dispersive error, Burgers 6.0×103 (blind to viscosity); time reversal, oscillators 1.4×103
Structure
Table 1: The diagnostic suite. Each diagnostic is computed from the trained model alone, Eq. ( 2 ), and requires only the listed property of the physics. Last column: what the diagnostic detects in this work, with its spread (largest over smallest value) across the learned models of each system. The last two rows are diagnostics that cannot detect a solver (Section III.4 ).
Figure 1: Test loss against fidelity diagnostics. Each marker is one learned model (scheme and seed); both axes are relative to the model trained on the reference scheme ( JA for Navier–Stokes). (a) Test loss and (b) the largest diagnostic of the suite against the transfer error. (c) Spearman rank correlation with the transfer error per system, for the test loss (open) and the diagnostic (filled). Navier–Stokes (grey) is discussed in Section IV.4 .
Figure 2: Linearized learned Burgers models. (a) Effective viscosity at uˉ=0 ; (b) relative error of the phase speed at uˉ=0.6 . Markers: learned models trained on data from each scheme (median over three seeds). Lines: analytic values of the scheme; dashed: the PDE. The upwind line (grey, dotted) is the averaged prediction ν+21⟨∣u∣⟩h . Shaded: wavenumbers at which the training data carry less than 10−6 of the peak power, so the model is not constrained by the data.
scheme
Dj
Da[M]
Da[P]
Da[H]
Da[P] of the learned model
spectral
<10−30
<10−30
<10−30
<10−30
7.7×10−9
skew, 4th
8.8×10−12
<10−30
<10−30
7.9×10−13
6.6×10−9
skew, 2nd
2.6×10−11
<10−30
<10−30
2.7×10−8
1.8×10−9
flux, 2nd
1.4×10−9
<10−30
5.6×10−8
2.0×10−9
5.6×10−8
advective, 2nd
7.3×10−9
<10−30
2.2×10−7
2.1×10−7
2.2×10−7
Table 2: KdV, N=127 . Diagnostics of the numerical operators for the structure ( Dj ), mass ( M ), momentum ( P ) and energy ( H ), averaged over 16 probe states. Values below 10−30 are round-off. Last column: momentum diagnostic of the model trained on data from each scheme (median over three seeds).
Figure 3: 2D Navier–Stokes, 48×48 , ν=0.05 . (a) Energy and enstrophy diagnostics of the four second-order schemes and the spectral reference; values below 10−27 (shaded) are round-off and shown at 10−30 . (b) The same diagnostics for models trained on data from each scheme (median over three seeds). (c) Transfer error of the learned models against the rollout horizon T . The model with the correct structure ( JA , black) has the third-lowest error at every horizon.
Figure 4: Semigroup error and fidelity diagnostics on the same learned time-steppers, coupled harmonic oscillators, median over three seeds. (a) Data from four solvers, one model class: the semigroup error is constant, while the energy diagnostic Da[H] and the structure diagnostic Dj separate explicit Euler by more than four orders of magnitude. (b) One data set, three model classes, of which only the first is a valid time-stepper: the semigroup error separates them.
Neural PDE surrogates are trained on numerical solver outputs that contain both physical evolution and solver-specific discretization errors. Because surrogates are also evaluated against held-out trajectories from the same solver, standard benchmarks cannot distinguish fidelity to the exact evolution from imitation of the numerical scheme. We introduce an empirical Fourier-symbol diagnostic that probes a trained surrogate's linearized one-step operator with individual Fourier modes and compares it with both exact-evolution and training-scheme references. To address architectural spectral bias, we train identical networks on schemes with orthogonal dissipative and dispersive signatures and compare their learned operators. In linear advection, the learned surrogates reproduce the training schemes' amplitude and phase errors, with the twin-scheme difference reaching more than 99.8% of the analytically predicted full-imitation ceiling. The same behavior occurs for a non-local Fourier neural operator and at the operator level for nonlinear Burgers dynamics. These results show that agreement with solver-generated test data does not by itself establish fidelity to the exact evolution. Fourier-symbol measurements provide a direct diagnostic of numerical provenance.
Ridham Patel
Computer Science & Engineering Indian Institute of Technology Gandhinagar Gandhinagar, India
We present the first method to directly use a learned continuous Lagrangian to forecast the dynamics of systems governed by partial differential equations, exploiting the inherent conservative structure to achieve stable long-range predictions. We develop an optimization-based integrator that minimizes the squared Euler--Lagrange residual via a mesh-free near-symplectic construction on local space-time patches. Different from integrators for analytical models, integrators for learned models should decouple model error (phase error) from integration error (conservation error). By relying on optimization rather than time-stepping, we bypass the global coupling inherent to fixed discretizations, which slows time- and space-stepping and complicates learning. Our method scales linearly with domain size via Jacobi iteration, and places no structural requirements on the learned network, allowing it to be coupled with existing physics-guided machine learning (ML) methods. We validate our approach on a learned representation of a double pendulum, a one-dimensional wave equation, and a two-dimensional wave equation. Our method achieves error comparable to classical symplectic methods while generalizing to spatially varying dynamics and arbitrary boundary conditions without retraining.
Lyra Zhornyak, Eric Forgoston, M. Ani Hsieh
University of Pennsylvania · Montclair State University
Neural PDE surrogates increasingly incorporate structural priors, yet it is often unclear whether their gains arise from physics-specific information or simply from regularization and training choices. We evaluate several such priors under a common protocol against a matched from-scratch neural operator baseline. Our central result is that a known-equation residual consistently outperforms the best generic regularizer at equal tuning budget. At fixed capacity this benefit appears across linear and nonlinear PDEs, but a capacity sweep reveals a sharp distinction: the advantage persists and grows for Burgers, KdV, and Allen-Cahn, while collapsing toward or below parity for linear heat and advection-diffusion. Thus, the durable value of the residual is specific to nonlinear operators. We further falsify a pre-registered hypothesis that the benefit is activated only by data sparsity: the residual remains advantageous even under full supervision. Its usefulness does, however, have a clear boundary. Under grid under-resolution, nonlinear coarse fields no longer satisfy the naive governing-equation residual, and enforcing it becomes actively harmful. In contrast, cross-family pretraining and in-context conditioning fail to outperform the strong from-scratch baseline in the regime studied. Together, these results identify when known physics provides non-redundant information to neural PDE models, when it does not, and when enforcing it introduces bias.
Zahra Farazpay, Aniruddha Bora
Department of Physics Louisiana Tech University Ruston, LA, USA · Department of Computer Science Texas State University San Marcos, TX, USA