Learning Data-Efficient and Generalizable Neural Operators via Fundamental Physics Knowledge

Siying Ma; Mehrdad M. Zadeh; Mauricio Soroco; Wuyang Chen; Jiguo Cao; Vijay Ganesh

cs.LGFeb 16, 2026

Learning Data-Efficient and Generalizable Neural Operators via Fundamental Physics Knowledge

Authors: Siying Ma, Mehrdad M. Zadeh, Mauricio Soroco, Wuyang Chen, Jiguo Cao, Vijay Ganesh

Organizations: Simon Fraser University · Georgia Institute of Technology

Abstract

Recent advances in scientific machine learning (SciML) have enabled neural operators (NOs) to serve as powerful surrogates for modeling the dynamic evolution of physical systems governed by partial differential equations (PDEs). While existing approaches focus primarily on learning simulations from the target PDE, they often overlook more fundamental physical principles underlying these equations. Inspired by how numerical solvers are compatible with simulations of different settings of PDEs, we propose a multiphysics training framework that jointly learns from both the original PDEs and their simplified basic forms. Our framework enhances data efficiency, reduces predictive errors, and improves out-of-distribution (OOD) generalization, particularly in scenarios involving shifts of physical parameters and synthetic-to-real transfer. Our method is architecture-agnostic and demonstrates consistent improvements in normalized root mean square error (nRMSE) across a wide range of 1D/2D/3D PDE problems. Through extensive experiments, we show that explicit incorporation of fundamental physics knowledge significantly strengthens the generalization ability of neural operators. We will release models and codes at https://sites.google.com/view/sciml-fundemental-pde.

Figures & tables

Figure 1: Can SciML models (e.g., neural operators trained on advanced PDEs) also understand fundamental physics knowledge (basic terms like diffusion, advection)?

Figure 2: On 2D incompressible Navier-Stokes, neural operators and SciML foundation models (MPP ( McCabe et al., 2023 ) , DPOT ( Hao et al., 2024 ) , Hyena ( Patil et al., 2023 ) ) exhibit correlated yet worse performance on fundamental physics (x-axis).

Figure 3: Overview of our method. Decomposed PDEs encode rich fundamental physical knowledge and introduce cheaper simulations. By jointly training on both the full PDE and its decomposed basic form, we bring multiple benefits to neural operators.

Figure 4: Visualizations of simulations of PDEs and their decomposed basic forms (Section 3.1 ). From left to right: Diffusion-Reaction (activator concentration), 2D Navier-Stokes (fluid velocity), 3D Navier-Stokes (smoke density), and Kuramoto-Sivashinsky (perturbation amplitude). Basic PDE forms are used for training neural operators with fundamental physics knowledge (Section 3.2 ), and the OOD settings are used for evaluating the generalization of neural operators (Section 4.4 ). $D_{v},D_{u}$ : diffusion coefficients (Equation 2 ). $\nu$ : viscosity (Equation 4 ). $a,b,k_{1},k_{2}$ : magnitudes and wavenumbers (Equation 7 ).

PDE	Spatial Resolution	Temporal Steps	Target Variables	Simulation Costs (sec. per step)	Sample Mixture Ratio (PDE : Basic Form)
Diffusion-Reaction (Eq. 2 )	$128\times 128$	100	Activator $u$ , Inhibitor $v$	$1.864\times 10^{-2}$	1:3
Basic Form (Eq. 3 )				$6.610\times 10^{-3}$
Navier-Stokes (2D) (Eq. 4 )	$256\times 256$	1000	Velocity $\bm{u}$ , Density $s$	2.775	1:24
Basic Form (2D) (Eq. 5 )				0.113
Navier-Stokes (3D) (Eq. 4 )	$50\times 50\times 89$	150	Velocity $\bm{u}$ , Density $s$	$1.047$	1:3
Basic Form (3D) (Eq. 5 )				0.300

Table 1: Summary of simulations of PDEs and their decomposed basic forms. GPU: NVIDIA RTX 6000 Ada.

Panel of Figure 5 — Figure 5: Joint training neural operators on data of the original PDE and the basic form improves performance and data efficiency. Y-axis: normalized RMSE. X-axis: simulation costs (seconds).

Panel of Figure 6 — Figure 6: Joint training neural operators on data of the original PDE and the basic form improves performance with autoregressive inference at different unrolled steps. Models are evaluated using the best-performing checkpoints from training, shown in Figure 5 .

PDE	Model	Source		Target 1		Target 2
		Setting	nRMSE	Setting	nRMSE	Setting	nRMSE
Diffusion-Reaction (2D)	Baseline	$\frac{D_{v}}{D_{u}}=5$	0.0289	$\frac{D_{v}}{D_{u}}=1$	0.0413	$\frac{D_{v}}{D_{u}}=100$	0.0770
	Ours		0.0231		0.0331		0.0538
Navier-Stokes (2D)	Baseline	$\nu=0.01$	0.0487	$\nu=0.05$	0.0825	$\nu=0.0001$	0.0369
	Ours		0.0175		0.0222		0.0125
Navier-Stokes (3D)	Baseline	$\nu=0.01$	0.0675	$\nu=0.1$	0.0393	$\nu=0.0001$	0.0836

Table 2: Comparisons of OOD generalization for different training methods. Models are evaluated using the best-performing checkpoints from training, as shown in Figure 5 , under comparable simulation cost settings.

Figure 7: Visualizations of the last time step in the ScalarFlow and its predictions derived by baseline and our model.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

PDE	Spatial Resolution	Temporal Steps	Target Variables	Simulation Costs (sec. per step)	Sample Mixture Ratio (PDE : Basic Form)
Diffusion-Reaction (Eq. 2 )	$128\times 128$	100	Activator $u$ , Inhibitor $v$	$1.864\times 10^{-2}$	1:8
Basic Form (Eq. 3 ) with	$96\times 96$	50		$2.390\times 10^{-3}$
Reduced Spatiotemporal Resolution	$96\times 96$	50		$2.390\times 10^{-3}$

Appendix

Table 3: Summary of 2D Diffusion-Reaction simulation and its decomposed basic forms with reduced spatiotemporal resolution. “Sample Mixture Rate”: We replace simulations of the original PDE with its decomposed basic form with reduced spatiotemporal resolution and make sure the total simulation cost of the training data can be comparable. GPU: NVIDIA RTX 6000 Ada.

Figure 8: Our method is agnostic to specific architectures of neural operators: we always share the backbone of the model between learning the original PDE and its basic form, and decouple their predictions in the last layer.

	2D DR (FNO)	2D DR (Transformer)	2D NS (FNO)	2D NS (Transformer)	3D NS (FNO)	3D NS (Transformer)	1D KS (FNO)
Input Shape Format	H $\times$ W $\times$ T $\times$ C (C = 2)		H $\times$ W $\times$ T $\times$ C (C = 3)		X $\times$ Y $\times$ Z $\times$ T $\times$ C (C = 4)		T $\times$ S
Number of Training Samples (PDE Simulations)	2, 4, 8, 16, 32, 64, 128		2, 4, 8, 16, 32, 48, 64		2, 4, 8, 16, 32, 48, 64		1024 $\sim$ 8192
Input Time Steps ( $\ell$ in Section 2.1 )	10	10	10	10	10	10	10
Sample Mixture Ratio	1:3	1:3	1:24	1:6	1:3	1:3	1:12
Learning Rate	0.001	0.0003	0.001	0.001	0.001	0.00015	0.001
Batch Size for Primary Data	4	8	16	16	8	8	64

Appendix

Table 4: Training details. “DR”: Diffusion-Reaction.“NS”: Navier-Stokes.“KS”: Kuramoto–Sivashinsky.

Panel of Figure 9 — Figure 9: Joint training neural operators on data of the original PDE and the basic form improves performance and data efficiency. “Spatiotemporal”: short for “Spatiotemporal Downsampling”. Y-axis: normalized RMSE. X-axis: simulation costs (seconds).

PDE	Model	Source		Target 1		Target 2
		Setting	nRMSE	Setting	nRMSE	Setting	nRMSE
Diffusion-Reaction (2D, FNO)	Baseline	$\frac{D_{v}}{D_{u}}=5$	0.0289	$\frac{D_{v}}{D_{u}}=1$	0.0413	$\frac{D_{v}}{D_{u}}=100$	0.0770
	Baseline@Spatiotemporal		0.0234		0.0303		0.0663
	Ours		0.0231		0.0331		0.0538
	Ours@Spatiotemporal		0.0218		0.0298		0.0596
Diffusion-Reaction (2D, Transformer)	Baseline	$\frac{D_{v}}{D_{u}}=5$	0.1056	$\frac{D_{v}}{D_{u}}=1$	0.1249	$\frac{D_{v}}{D_{u}}=100$	0.1976

Appendix

Table 5: Comparisons of OOD generalization for different training methods with the Transformer. Models are evaluated using the best checkpoints from training in Figure 5 , under comparable simulation cost settings. “Spatiotemporal“: short for “Spatiotemporal Downsampling”.

Figure 10: Joint training neural operators on data of the original 3D Navier-Stokes equation and the basic form improves performance and data efficiency.

PDE	Model	Source		Target 1		Target 2
		Setting	nRMSE	Setting	nRMSE	Setting	nRMSE
3D Navier-Stokes	Baseline	$\nu=0.01$	0.0114	$\nu=0.1$	0.0327	$\nu=0.0001$	0.0816
	Ours		0.0064		0.0124		0.0322

Appendix

Table 6: Comparisons of OOD generalization on 3D NS for different training methods using Transformer. Models are evaluated using the best checkpoints from training in Figure 10 .

Figure 11: Joint training neural operators on data of the original PDE and the basic form improves performance with autoregressive inference at different unrolled steps using Transformer. Models are evaluated using the best-performing checkpoints from training shown in Figure 9 .

Figure 12: Model performance averaged over three random seeds. Joint training neural operators on data of the original PDE and the basic form consistently improves performance and data efficiency. Dash lines indicate the mean performance, with error bars representing standard deviation. Legends align with the descriptions in Section 4.2 . Columns: (left) 2D Diffusion-Reaction, (right) 2D Navier-Stokes. Y-axis: nRMSE. X-axis: Simulation Costs (seconds).

Figure 13: Model performance averaged over three random seeds. Joint training neural operators on data of the original PDE and the basic form consistently improves long-term physical consistency. Dash lines indicate the mean performance, with error bars representing standard deviation. Models are evaluated using the best-performing checkpoints from training shown in Figure 12 .

PDE	Model	Source		Target 1		Target 2
		Setting	nRMSE Mean (SD)	Setting	nRMSE Mean (SD)	Setting	nRMSE Mean (SD)
Diffusion-Reaction (2D, FNO)	Baseline	$\frac{D_{v}}{D_{u}}=5$	0.0287 (0.0006)	$\frac{D_{v}}{D_{u}}=1$	0.0411 (0.0008)	$\frac{D_{v}}{D_{u}}=100$	0.0754 (0.0014)
	Baseline@Spatiotemporal		0.0235 (0.0001)		0.0305 (0.0005)		0.0664 (0.0041)
	Ours		0.0232 (0.0003)		0.0329 (0.0003)		0.0532 (0.0014)
	Ours@Spatiotemporal		0.0216 (0.0005)		0.0296 (0.0002)		0.0548 (0.0053)
Navier-Stokes (2D, FNO)	Baseline	$\nu=0.01$	0.0522 (0.0030)	$\nu=0.05$	0.0855 (0.0028)	$\nu=0.0001$	0.0367 (0.0002)

Appendix

Table 7: Comparisons of OOD generalization for different training methods averaged over three random seeds. Models are evaluated using the best checkpoints from training in Figure 12 , under comparable simulation cost settings. “Spatiotemporal“: short for “Spatiotemporal Downsampling”.

Simulation Costs (Seconds)	Weight = 0.5	Weight = 0.7	Weight = 1
3.73	0.1255	0.1233	0.1214
7.46	0.0674	0.0659	0.0636
14.92	0.0508	0.0478	0.0445
29.84	0.0349	0.0330	0.0314
59.68	0.0274	0.0266	0.0249
119.36	0.0245	0.0238	0.0224

Appendix

Table 8: Ablation study on the auxiliary loss weight in joint training using FNO for the 2D Diffusion-Reaction across three settings: auxiliary weights of 0.5, 0.7, and 1.0. Results indicate that our model is robust to the choice of auxiliary loss weighting. Simulation costs are aligned with the ones used in Figure 5 (top-left).

Figure 14: Ablation study of joint training neural operators with two different decomposed terms: the fundamental diffusion term, and reaction term in 2D Diffusion-Reaction, and the fundamental advection term and diffusion term in 2D Navier-Stokes, shows the importance of choice on fundamental terms from PDE equation. Y-axis: nRMSE. X-axis: Simulation Costs (Seconds).

Figure 15: Qualitative visualization of the original PDE and its decomposed forms for 2D Diffusion-Reaction. This results combined with the quantitative results obtained from the RMSE experiment demonstrate that our choice of fundamental term is principled.

Figure 16: Data efficiency on 2D Diffusion–Reaction using the DPOT foundation model, suggesting joint training on the original and basic form yields consistently lower error across simulation-cost budgets

Figure 17: Joint training neural operators on data of the original PDE and the basic form, as a complementary data augmentation orthogonal to Lie-transform augumentation, can further improve performance and data efficiency. Y-axis: normalized RMSE. X-axis: simulation costs (seconds).

Figure 18: Qualitative visualization of model predictions for 2D Diffusion-Reaction, 2D Incompressible Navier-Stokes, 3D Incompressible Navier-Stokes systems and 1D Kuramoto–Sivashinsky using FNO trained with our joint framework. For the first three cases, the initial state and predicted states at intermediate and final rollout times are shown. For 1D Kuramoto–Sivashinsky, we show the whole predicted trajectory here. The results demonstrate accurate temporal evolution and spatial coherence.

Explore similar work

Learning Physical Operators using Neural Operators
Feb 26, 2026Vignesh Gopakumar, Ander Gray, Dan Giles +5PDE Surrogate Modeling Physics-Informed Neural Operators
UCL Centre for Artificial Intelligence · UK Atomic Energy Authority · LIX, CNRS, ´Ecole Polytechnique +2
Generalized Neural Operator for Parametric and Boundary-Value Problems
Jul 24, 2026Ruoyan Li, Yizhou Sun, Wei WangEfficient Neural Network Inference PDE Solving
University of California, Los Angeles
Compositional Neural Operators for Multi-Dimensional Fluid Dynamics
May 12, 2026Hamda Hmida, Hsiu-Wen Chang, Youssef MesriPDE Operator Learning Physics-Informed Neural Operators
Mines Paris - PSL University, Centre for Material Forming (CEMEF) · Mines Paris - PSL University, Centre for Robotics (CAOR)

Feb 26, 2026J/K move · Enter open · S save

Learning Physical Operators using Neural Operators

Vignesh Gopakumar, Ander Gray, Dan Giles, Lorenzo Zanisi+4

UCL Centre for Artificial Intelligence · UK Atomic Energy Authority · LIX, CNRS, ´Ecole Polytechnique+2

PDE Surrogate Modeling Physics-Informed Neural Operators Computational Fluid Dynamics Modular Neural Networks

Open paper AskPDF arXiv

Neural operators have emerged as promising surrogate models for solving partial differential equations (PDEs), but struggle to generalise beyond training distributions and are often constrained to a fixed temporal discretisation. This work introduces a physics-informed training framework that addresses these limitations by decomposing PDEs using operator splitting methods, training separate neural operators to learn individual non-linear physical operators while approximating linear operators with fixed finite-difference convolutions. This modular mixture-of-experts architecture enables generalisation to novel physical regimes by explicitly encoding the underlying operator structure. We formulate the modelling task as a neural ordinary differential equation (ODE) where these learned operators constitute the right-hand side, enabling continuous-in-time predictions through standard ODE solvers and implicitly enforcing PDE constraints. Demonstrated on incompressible and compressible Navier--Stokes equations, our approach achieves better convergence and superior performance when generalising to unseen physics. The method remains parameter-efficient, enabling temporal extrapolation beyond training horizons, and provides interpretable components whose behaviour can be verified against known physics.

Abstract

Explore similar work

Learning Physical Operators using Neural Operators

Generalized Neural Operator for Parametric and Boundary-Value Problems

Compositional Neural Operators for Multi-Dimensional Fluid Dynamics