Learning Physical Operators using Neural Operators
Authors: Vignesh Gopakumar, Ander Gray, Dan Giles, Lorenzo Zanisi, Matt J. Kusner, Timo Betcke, Stanislas Pamela, Marc Peter Deisenroth
Organizations: UCL Centre for Artificial Intelligence · UK Atomic Energy Authority · LIX, CNRS, ´Ecole Polytechnique · Polytechnique Montr´eal · Mila - Quebec AI Institute
Neural operators have emerged as promising surrogate models for solving partial differential equations (PDEs), but struggle to generalise beyond training distributions and are often constrained to a fixed temporal discretisation. This work introduces a physics-informed training framework that addresses these limitations by decomposing PDEs using operator splitting methods, training separate neural operators to learn individual non-linear physical operators while approximating linear operators with fixed finite-difference convolutions. This modular mixture-of-experts architecture enables generalisation to novel physical regimes by explicitly encoding the underlying operator structure. We formulate the modelling task as a neural ordinary differential equation (ODE) where these learned operators constitute the right-hand side, enabling continuous-in-time predictions through standard ODE solvers and implicitly enforcing PDE constraints. Demonstrated on incompressible and compressible Navier--Stokes equations, our approach achieves better convergence and superior performance when generalising to unseen physics. The method remains parameter-efficient, enabling temporal extrapolation beyond training horizons, and provides interpretable components whose behaviour can be verified against known physics.
Figures & tables
Figure 1: Comparison of operator learning approaches for PDE solving. The figure contrasts three methods: (top left) the traditional autoregressive approach where a single neural operator learns the solution operator mapping directly from state un to un+1 ; (bottom left) the neural ODE approach where a neural operator learns the dynamics operator \derivativeut integrated with an ODE solver; and (right) the proposed OpsSplit method where individual neural operators learn specific physical operators (∇\crossproduct,∇⋅) and are combined via operator splitting to compute \derivativeut which is integrated using an ODE solver. The OpsSplit approach decomposes the PDE into its constituent physical operators, with linear operators approximated by convolutions and non-linear operators learned by neural networks, enabling physics-informed and modular PDE solving.
Train Time
NRMSE
Operator Learning
Model
Time Stepping
OpsSplit
Parameters
(hrs:mins)
Test
t-extrapolate
OOD
OOD+t-extrapolate
Solution
FNO
Autoregressive
False
13437314
1:03
0.2770 ± 0.0491
0.6860 ± 0.26476
0.4821 ± 0.0639
0.8986 ± 0.2727
Dynamics
FNO
Euler
False
13437314
1:04
0.0325 ± 0.0165
0.0536 ± 0.0246
0.1276 ± 0.0081
0.3038 ± 0.0119
Physical
FNO
Euler
True
13437314
1:06
0.0297 ± 0.0017
0.0630 ± 0.0111
0.1160 ± 0.0162
0.2185 ± 0.0846
Solution
U-Net
Autoregressive
False
31384322
1:22
0.9508 ± 0.1369
1.3183 ± 0.1318
0.8192 ± 0.2007
1.1872 ± 0.2534
Dynamics
U-Net
Euler
False
31384322
1:22
0.1006 ± 0.0133
0.2212 ± 0.0315
0.2823 ± 0.0154
0.5135 ± 0.0331
Table 1: Incompressible Navier–Stokes: Performance comparison across methods and model architectures. The models and methods are set up to have comparable parameter sizes while exploring a certain architecture. Best performance within each test setting is given in bold. Our method of using neural operators to learn physical operators offers the best performance across most architectures. OOD refers to a different parameterisation of the initial condition and viscosity than that used for training ( appendix J ).
Figure 2: Rollout error for incompressible Navier–Stokes equations. FNO ( figs. 2(a) and 2(b) ) and CNO ( figs. 2(c) and 2(d) ) predictive error growth for in-distribution and out-of-distribution cases. The temporal extrapolation region is shaded orange. OpsSplit accumulates fewer errors and provides more stable temporal rollout than autoregressive and neural ODE methods across architectures and scenarios.
Figure 3: Neural operator learned convection ( fig. 3(a) ) versus numerical method. NO captures convection nuances and advection-driven flow trends. Qualitative interpretability study: learned convection exists in latent space; numerical convection shown in physical space, normalised to [-1, 1].
Figure 4: Continuity equation ( eq. 14 ) violation across FNO predictions for OOD modeling.
Train Time
NRMSE
Operator Learning
Model
Time Stepping
OpsSplit
Parameters
(hrs:mins)
Test
t-extrapolate
OOD
OOD+t-extrapolate
Solution
FNO
Autoregressive
False
13437828
1:53
0.0938 ± 0.0131
0.2462 ± 0.0443
0.1209 ± 0.0133
0.2792 ± 0.0530
Dynamics
FNO
Euler
False
13437828
1:56
0.0245 ± 0.0039
0.0854 ± 0.0102
0.0763 ± 0.0130
0.1185 ± 0.0154
Physical
FNO
Euler
True
13454787
3:08
0.0602 ± 0.0090
0.0959 ± 0.0182
0.0573 ± 0.0063
0.0751 ± 0.0105
Solution
U-Net
Autoregressive
False
31385604
1:53
0.5963 ± 0.0716
1.1914 ± 0.2025
0.6069 ± 0.1153
1.2384 ± 0.1610
Dynamics
U-Net
Euler
False
31385604
1:54
0.1222 ± 0.0196
0.2912 ± 0.0320
0.1477 ± 0.0222
0.2989 ± 0.0538
Table 2: Compressible Navier–Stokes: Performance comparison across methods and model architectures. The models and methods are set up to have comparable parameter sizes while exploring a certain architecture. Best performance within each test setting is given in bold. Our method of using neural operators to learn physical operators offers the best performance across most architectures. OOD refers to a different parameterisation of the initial condition and adiabatic index than that used for training ( appendix K ). The higher computational times for OpsSplit is observed due to the several forward passes required to estimate the physical operators.
Figure 5: Rollout error for the compressible Navier–Stokes equations. Similar to fig. 2 , show the rollout error of various methods for an FNO and U-Net for both in and out-of-distribution scenarios. In most cases, across neural operator architectures, our method of deploying operator splitting to learn the physical operator accumulates less error and provides a more stable temporal rollout than autoregressive and neural ODE-based methods.
Train Time
NRMSE
Operator Learning
Model
Time Stepping
OpsSplit
Parameters
(hrs:mins)
In-Distribution
Out-of-Distribution
Solution
GNO
Autoregressive
False
330,370
6:25
0.4131±0.0475
0.5697±0.0530
Dynamics
GNO
Euler
False
330,370
6:26
0.3276±0.0398
0.4510±0.0571
Physical
GNO
Euler
True
660,740
9:31
0.2963±0.0043
0.3621±0.0510
Table 3: Incompressible Navier–Stokes on irregular geometries: Performance comparison across methods and model architectures. Best performance within each test setting is shown in bold. Our method of using neural operators to learn physical operators achieves the best performance. In-distribution refers to modelling within the training regime, whereas out-of-distribution refers to extrapolation further in time on a different validation dataset. The higher computational time for OpsSplit is due to the multiple models required for each physical operator.
Figure 6: Loss convergence for OpsSplit models on incompressible Navier–Stokes. The pre-trained model learns all operators from scratch. The fine-tuned model initialises the convection operator with trained weights from the compressible case, showing quicker convergence. For more details, refer appendix I .
Appendix figures & tables46 assets
Supplementary material from the paper’s appendix.
Appendix
Work (citation)
Physics-informed
Continuous-in-time
Modular/ Mixture of Experts
Interpretable
Extrapolation / Generalisation
[ Li et al., 2021 ] (FNO)
✗
✗
✗
✗
✗
[ Lu et al., 2021 ] (DeepONet)
✗
✓
✓
✗
✗
[ Raissi et al., 2019 ] (PINNs)
✓
✓
✗
✓
✓
[ Rackauckas et al., 2021 ] (UDE)
✓
✓
✓
✗
✗
[ Li et al., 2024 ] (PINO)
✓
✗
✗
✗
✓
[ Koch et al., 2025 ]
✓
✓
✓
✗
✓
Appendix
Table 4: Comparison of different approaches for Neural PDE solvers. ✓: Yes, ✗ : No, ✓ : Partial.
Figure 7: Incompressible Navier–Stokes: comparing across convection operator obtained numerically against those learnt using various neural operators.
Figure 8: Compressible Navier–Stokes: comparing across convection and vector divergence operator obtained numerically against those learnt using various neural operators.
Figure 9: Incompressible Navier–Stokes: Rollout Error of the the various OpsSplit ablations as described in table 5 . We experiment each ablation across both in and out-of-distribution. While we notice that higher order finite-difference stencils deployed to model the linear operators perform best within distribution, they fail to extend this to outside distribution. Learnt approximations of the linear diffusion operator either using a linear convolutional operator (5) or a neural operator (6) allows for better generalisation.
Ablation
Convection
Diffusion
Train Time
NRMSE
(v⋅∇)v
∇2v
Parameters
(hrs:mins)
Test
t-extrapolate
OOD
OOD+t-extrapolate
1
NO
FD (2)
13437321
1:11
0.0270
0.0796
0.1471
0.4731
2
NO
FD (4)
13437339
1:12
0.0244
0.0600
0.1993
0.6995
3
NO
FD (6)
13437350
1:13
0.0221
0.0504
0.1576
0.5303
4
NO
FD (8)
13437378
1:13
0.0214
0.0568
0.4195
1.5992
5
NO
Linear
13438196
1:13
0.0283
0.0823
0.1581
0.3693
Appendix
Table 5: Incompressible Navier–Stokes: Performance comparison across different methods of operator splitting and subsequent operator choices. FD(n) represents a fixed finite difference kernel of order n , Linear corresponds to a learnable linear convolution, and NO represents an FNO.
Figure 10: Compressible Navier–Stokes: Rollout Error of the the various OpsSplit ablations as described in table 6 . We experiment each ablation across both in and out-of-distribution. We notice that when we approximate nonlinear operators using FD stencils the prediction blows up when extrapolated in time. The general trend we notice is that approximating using neural operators offer better stability both in and out-of-distribution.
Ablation
Continuity
Convection
Pressure
Pressure
Pressure
Train Time
NRMSE
Gradient
Advection
Expansion
∇⋅(ρv)
(v⋅∇)v
∇P
v⋅∇P
P(∇⋅v)
Parameters
(hrs:mins)
Test
t-extrapolate
OOD
OOD+t-extrapolate
1
NO 1
NO 2
FD (4)
FD (4)
FD (4)
26874627
3:56
0.0824
-
0.0851
-
2
NO 1
NO 2
FD (4)
FD (4)
NO 3
40311940
5:43
0.0762
0.1441
0.0785
0.1564
3
NO 1
NO 2
Linear
Linear
Linear
26874666
3:52
0.0639
0.1115
0.2061
0.4201
4
NO 1
NO 2
Linear
Linear
NO 3
40311979
5:41
0.0866
0.1177
0.1329
0.2839
Appendix
Table 6: Compressible Navier–Stokes: Performance comparison across different methods of operator splitting and subsequent operator choices. FD(n) represents a fixed finite difference kernel of order n , Linear corresponds to a learnable linear convolution, and NO represents an FNO. NOi represents the ith neural operator within the system.
Figure 11: Incompressible Navier–Stokes: Rollout length of the FNO deployed across each method for both in and out-of-distribution.
Figure 12: Incompressible Navier–Stokes: Data Efficiency of the FNO deployed across each method for both in and out-of-distribution.
Figure 13: Incompressible Navier–Stokes: Data Efficiency of the FNO deployed across each method for both in and out-of-distribution.
Figure 14: Incompressible Navier–Stokes: Impact of finesse of ODE solver used for temporal integration of the FNO deployed across each method for both in and out-of-distribution.
Figure 15: Incompressible Navier–Stokes: Train and test loss convergences across training methods for the FNO
Figure 16: Compressible Navier–Stokes: Train and test loss convergences across training methods for the FNO
Figure 17: Incompressible Navier–Stokes: Comparison of training and test loss convergence for FNO-based OpsSplit models. The pre-trained baseline learns all operators from scratch with random initialisation. The fine-tuned model initialises the convection operator using weights pre-trained on the compressible Navier–Stokes equations ( eq. 21 ), demonstrating accelerated convergence through cross-PDE transfer learning.
Figure 18: Compressible Navier–Stokes: Comparison of training and test loss convergence for FNO-based OpsSplit models. The pre-trained baseline learns all operators from scratch with random initialisation. The fine-tuned model initialises the convection operator using weights pre-trained on the incompressible Navier–Stokes equations ( eq. 16 ), demonstrating that transfer learning benefits are bidirectional across fluid dynamics regimes.
Parameter
Training
OOD Testing
Type
Velocity x-axis (u0)
[0.5,1.0]
[0.1,0.5]
Continuous
Velocity y-axis (v0)
[0.5,1.0]
[0.1,0.5]
Continuous
viscosity ν
0.001
0.01
Discrete
Appendix
Table 7: Parameterisations of the 2D incompressible Navier–Stokes equations utilised for training and OOD testing
Table 8: Model configuration details of neural operators used for modelling the incompressible Navier–Stokes equations. All three methods - AR, NODE, and OpsSplit utilised NO models with the same configuration as mentioned above.
Figure 19: Incompressible Navier–Stokes: Model prediction within test distribution for Autoregressive FNO
Figure 20: Incompressible Navier–Stokes: Model prediction within test distribution for Neural-ODE FNO
Figure 21: Incompressible Navier–Stokes: Model prediction within test distribution for OpsSplit FNO
Figure 22: Incompressible Navier–Stokes: Model prediction within test distribution for Autoregressive U-Net
Figure 23: Incompressible Navier–Stokes: Model prediction within test distribution for Neural-ODE U-Net
Figure 24: Incompressible Navier–Stokes: Model prediction within test distribution for OpsSplit U-Net
Figure 25: Incompressible Navier–Stokes: Model prediction within test distribution for Autoregressive ViT
Figure 26: Incompressible Navier–Stokes: Model prediction within test distribution for Neural-ODE ViT
Figure 27: Incompressible Navier–Stokes: Model prediction within test distribution for OpsSplit ViT
Figure 28: Incompressible Navier–Stokes: Model prediction within test distribution for Autoregressive CNO
Figure 29: Incompressible Navier–Stokes: Model prediction within test distribution for Neural-ODE CNO
Figure 30: Incompressible Navier–Stokes: Model prediction within test distribution for OpsSplit CNO
Figure 31: Incompressible Navier–Stokes: Model prediction within test distribution for Autoregressive UNO
Figure 32: Incompressible Navier–Stokes: Model prediction within test distribution for Neural-ODE UNO
Figure 33: Incompressible Navier–Stokes: Model prediction within test distribution for OpsSplit UNO
Parameter
Training
OOD Testing
Type
α
[0.1,0.5]
[0.5,1.0]
Continuous
β
[1.0,5.0]
[5.0,10.0]
Continuous
γ
35
32
Discrete
Appendix
Table 9: Parameterisation of the 2D Euler Fluid equations utilised for training and OOD testing
Table 10: Model configuration details of neural operators used for modelling the compressible Navier–Stokes equations. Both AR and NODE-based models have the same configuration, whereas the two models within OpsSplit have slightly different setups to ensure the same parameterisation levels across all methods.
Figure 34: Compressible Navier–Stokes: Model prediction within test distribution for Autoregressive FNO
Figure 35: Compressible Navier–Stokes: Model prediction within test distribution for Neural-ODE FNO
Figure 36: Compressible Navier–Stokes: Model prediction within test distribution for OpsSplit FNO
Figure 37: Compressible Navier–Stokes: Model prediction within test distribution for Autoregressive U-Net
Figure 38: Compressible Navier–Stokes: Model prediction within test distribution for Neural-ODE U-Net
Figure 39: Compressible Navier–Stokes: Model prediction within test distribution for OpsSplit U-Net
Figure 40: Compressible Navier–Stokes: Model prediction within test distribution for Autoregressive ViT
Figure 41: Compressible Navier–Stokes: Model prediction within test distribution for Neural-ODE ViT
Figure 42: Compressible Navier–Stokes: Model prediction within test distribution for OpsSplit ViT
Figure 43: Compressible Navier–Stokes: Model prediction within test distribution for Autoregressive UNO
Figure 44: Compressible Navier–Stokes: Model prediction within test distribution for Neural-ODE UNO
Figure 45: Compressible Navier–Stokes: Model prediction within test distribution for OpsSplit UNO
Recent advances in scientific machine learning (SciML) have enabled neural operators (NOs) to serve as powerful surrogates for modeling the dynamic evolution of physical systems governed by partial differential equations (PDEs). While existing approaches focus primarily on learning simulations from the target PDE, they often overlook more fundamental physical principles underlying these equations. Inspired by how numerical solvers are compatible with simulations of different settings of PDEs, we propose a multiphysics training framework that jointly learns from both the original PDEs and their simplified basic forms. Our framework enhances data efficiency, reduces predictive errors, and improves out-of-distribution (OOD) generalization, particularly in scenarios involving shifts of physical parameters and synthetic-to-real transfer. Our method is architecture-agnostic and demonstrates consistent improvements in normalized root mean square error (nRMSE) across a wide range of 1D/2D/3D PDE problems. Through extensive experiments, we show that explicit incorporation of fundamental physics knowledge significantly strengthens the generalization ability of neural operators. We will release models and codes at https://sites.google.com/view/sciml-fundemental-pde.
Siying Ma, Mehrdad M. Zadeh, Mauricio Soroco +3
Simon Fraser University · Georgia Institute of Technology
Developing foundational neural simulators for Partial Differential Equations (PDEs) requires robust generalization across diverse physical parameters and boundary conditions. However, current deep learning approaches largely face a structural trade-off between condition-agnostic deployment and physical fidelity. Purely data-driven operators infer the underlying physics implicitly and thus lack the explicit constraints needed to ensure physically valid solutions across varying domains, rendering the learning problem ill-posed. On the other hand, Physics-Informed Neural Networks (PINNs) enforce rigorous physical constraints but necessitate costly, instance-specific optimization. Furthermore, the massive scale of emerging foundational operators has severely degraded their inference speeds, making them computationally uncompetitive with traditional numerical solvers. To address this bottleneck between condition-agnostic deployment, physical rigor, and inference efficiency, we propose a Generalized Neural Operator. By formalizing the classical conditions for well-posedness within neural operators, our framework demonstrates the theoretical benefits of explicitly conditioning on PDE parameters and boundary conditions. To implement this synthesis without compromising computational speed, we introduce three novel architectural components: a parameter-gated mixture of kernels for efficient parameter generalization, a generalized boundary transfer operator that projects arbitrary boundary constraints into a unified latent Dirichlet representation, and a specialized training objective to ensure stability. Extensive experiments demonstrate that our theoretically grounded approach achieves superior generalization across heterogeneous physical regimes while maintaining strict inference efficiency comparable to conventional numerical baselines.
Partial differential equations (PDEs) govern diverse physical phenomena, yet high-fidelity numerical solutions are computationally expensive and Machine Learning approaches lack generalization. While Scientific Foundation Models (SFMs) aim to provide universal surrogates, typical encoding-decoding approaches suffer from high pretraining costs and limited interpretability. In this paper, we propose Compositional Neural Operators (CompNO) for 2D systems, a framework that decomposes complex PDEs into a library of Foundation Blocks. Each block is a specialized Neural Operator pretrained on elementary physics. This modular library contains convection, diffusion, and nonlinear convection blocks as well as a Poisson Solver, enabling the framework to address the pressure-velocity coupling. These experts are assembled via an Adaptation Block featuring an Aggregator. This aggregator learns nonlinear interactions by minimizing data loss and physics-based residuals driven from governing equations. The proposed approach has been evaluated on the Convection-Diffusion equation, the Burgers' equation, and the Incompressible Navier-Stokes equation. Our results demonstrate that learning from elementary operators significantly improves adaptability, enhances model interpretability and facilitates the reuse of pretrained blocks when adapting to new physical systems.
Hamda Hmida, Hsiu-Wen Chang, Youssef Mesri
Mines Paris - PSL University, Centre for Material Forming (CEMEF) · Mines Paris - PSL University, Centre for Robotics (CAOR)