PDE Operator Learning

PDE: Partial Differential Equation

Follow this topic Ask about this topic

Momentum

24 papers in the last four weeks, up 140% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 148

May 8, 2026cs.LG

When Attention Beats Fourier: Multi-Scale Transformers for PDE Solving on Irregular Domains

We study the problem of \emph{architecture selection} for deep learning models trained to solve partial differential equations (PDEs), asking when transformer-based architectures with learned attention outperform Fourier-domain neural operators. We introduce the \textbf{Multi-Scale Attention Transformer} (\msat{}), a deep learning architecture that encodes spatiotemporal solution histories as token sequences and trains end-to-end via a composite supervised objective with optional physics-informed regularization terms. We conduct a comprehensive empirical evaluation against nine baselines -- including physics-informed neural networks (PINNs), neural operators (FNO, DeepONet, GNOT), and state-space models (Mamba-NO) -- across five benchmark problems from the PINNacle suite, using identical train/test splits and reference data for all methods. \msat{} achieves state-of-the-art generalization on complex geometry problems (

L^2_\mathrm{rel} = 0.0101

on Heat2D-CG, a

3.7\times

improvement over FNO) at

34\,\mathrm{s}

total inference vs.\

120{,}812\,\mathrm{s}

for Mamba-NO. Ablation studies over the physics regularization component reveal a precise inductive bias tradeoff: physics priors reduce test error on diffusion-dominated problems but degrade generalization on chaotic and recirculating-flow regimes, directly characterizing the prior misspecification boundary. Approximation error bounds as a function of domain boundary complexity

κ

provide a theoretical basis for these empirical findings and a principled rule for architecture selection.

Brandon Yee, Pairie Koh, Jack Rodriguez +1

Physics Lab, Yee Collins Research Group

Neural PDE Solvers PDE Solving PDE Operator Learning Physics-Informed ML

May 8, 2026cs.LG

QuadNorm: Resolution-Robust Normalization for Neural Operators

Normalization layers in neural operators usually compute statistics by uniformly averaging discrete grid values, making the normalization itself discretization-dependent and thereby a source of transfer error across different resolutions or meshes. To enable discretization robustness, we introduce a quadrature normalization family that replaces existing uniform averaging in normalization layers with numerical quadrature: QuadNorm and BlendQuadNorm. On endpoint-inclusive uniform grids, the proposed quadrature moments are

O(h^2)

-consistent across discretizations, meaning that their cross-resolution mismatch decays quadratically with grid spacing. A transfer-error bound then predicts how normalization-induced mismatch scales with both the resolution gap and network depth. The experiments show the same gap- and depth-scaling trends predicted by the transfer-error bound. On Darcy, QuadNorm delivers the best cross-resolution performance at every tested target resolution from

64^2

to

256^2

; on real-data benchmarks, Transolver with QuadNorm achieves nearly resolution-invariant transfer. The largest gains appear on nonperiodic PDEs and nonspectral architectures, where native-resolution improvements also emerge. We also validate BlendQuadNorm, which stays close to LayerNorm behavior and serves as a conservative default for periodic FNO settings. These results identify normalization as a previously overlooked source of resolution dependence in neural operators.

Bum Jun Kim, Makoto Kawano, Yusuke Iwasawa +1

The University of Tokyo, Japan

PDE Operator Learning Neural Operators

May 4, 2026cs.LG

Quantitative Sobolev Approximation Bounds for Neural Operators with Empirical Validation on Burgers Equation

Neural operators have emerged as a powerful tool for learning mappings between infinite-dimensional function spaces. However, their approximation properties in Sobolev norms remain poorly quantified, even though these norms control both function values and derivatives and are the natural metrics for PDE well-posedness, stability, and generalization. We develop a functional-analytic framework for operator learning in Sobolev spaces and connect it to the numerical behavior of Fourier Neural Operators (FNOs) on a prototypical PDE. First, for a continuous nonlinear operator

\mathcal{G}: H^{s}(D)\to H^{t}(D')

with

s > d/2

and inputs restricted to a compact subset of

H^{s}(D)

, we prove that

\mathcal{G}

can be uniformly approximated in

H^{t}

-norm by a neural operator with

\mathcal{O}(\varepsilon^{-d/s})

trainable parameters. This yields an explicit complexity--error relation of the form

\|\mathcal{G}-\mathcal{G}_θ\|_{H^{t}} \lesssim C N^{-s/d}

. We then study the one-dimensional viscous Burgers solution operator

\mathcal{G}: u_{0}\mapsto u(\cdot,1)

on a bounded

H^{1}

-ball and train FNOs with an

H^{1}

-loss. Across a sweep of model sizes, we obtain test

H^{1}

-errors down to

\mathcal{O}(10^{-7})

and relative errors of order

10^{-3}

, with predictions accurately matching both solutions and spatial derivatives on held-out data. A log-log plot of Sobolev error versus parameter count exhibits an approximate power law

\|\mathcal{G}-\mathcal{G}_θ\|_{H^{1}} \approx C N^{-α}

with empirical exponent

α\approx 1.4

, and long-horizon training reveals optimization instabilities in large FNOs, providing quantitative evidence that Sobolev-space approximation theory meaningfully predicts neural-operator scaling behavior.

Nicole Hao

Department of Mathematics, Cornell University

Fourier Neural Operator PDE Solving PDE Operator Learning Neural Network Approximation Theory

May 4, 2026cs.LG

Isotropic Fourier Neural Operators

Fourier Neural Operators are deep learning models that learn mappings between function spaces and can be used to learn and solve partial differential equations (PDEs), in some cases significantly faster than traditional PDE solvers. Within the model are Fourier layers, which apply linear transformations directly to the Fourier modes, with parameters depending on the wave numbers. However, most physical systems are isotropic, with the results being independent of the coordinate system chosen, but the linear transformations do not necessarily respect these symmetries. We propose a modification to the linear transformations that ensures spatial symmetries are respected, called the Isotropic Fourier Neural Operator, which both improves model performance and reduces the number of parameters by up to a factor of 16 in 2D and 96 in 3D.

Michael F. Staddon

Independent Researcher

Fourier Neural Operator Neural PDE Solvers Equivariant Neural Networks PDE Operator Learning Neural Operators

May 1, 2026cs.CE

HyCOP: Hybrid Composition Operators for Interpretable Learning of PDEs

We introduce HyCOP, a modular framework that learns parametric PDE solution operators by composing simple modules (advection, diffusion, learned closures, boundary handling) in a query-conditioned way. Rather than learning a monolithic map, HyCOP learns a policy over short programs - which module to apply and for how long - conditioned on regime features and state statistics. Modules may be numerical sub-solvers or learned components, enabling hybrid surrogates evaluated at arbitrary query times without autoregressive rollout. Across diverse PDE benchmarks, HyCOP produces interpretable programs, delivers order-of-magnitude OOD improvements over monolithic neural operators, and supports modular transfer through dictionary updates (e.g., boundary swaps, residual enrichment). Our theory characterizes expressivity and gives an error decomposition that separates composition error from module error and doubles as a process-level diagnostic.

Jinpai Zhao, Nishant Panda, Yen Ting Lin +3

Oden Institute University of Texas at Austin Austin, TX, USA · Information Sciences, CAI-3 Los Alamos National Laboratory Los Alamos, NM 87545, USA · Norwegian University of Life Sciences Ås, Norway / Simula Research Lab. Oslo, Norway

Neural Surrogate Modeling Neural Network Interpretability PDE Operator Learning Operator Learning Modular Neural Networks

May 1, 2026cs.LG

Learning the Helmholtz equation operator with DeepONet for non-parametric 2D geometries

This paper deals with solving the 2D Helmholtz equation on non-parametric domains, leveraging a physics-informed neural operator network based on the DeepONet framework. We consider a 2D square domain with an inclusion of arbitrary boundary geometry at its center. This inclusion acts as a scatterer for an incoming harmonic wave. The aim is to learn the operator linking the geometry of the scatterer to the resulting scattered field. A signed distance function to the boundary of the inner inclusion, evaluated at several points in the domain, is used to encode its geometry. It serves as input for the branch part of the DeepONet architecture, while local information is used as input for the trunk part. This approach enables the encoding of arbitrary geometries, whether they are parameterized or not. The evaluation of the model on unseen geometries is compared with its finite element method (FEM) equivalent to test its generalization capabilities. The trained network weights implicitly embed the local physics and their interaction with the domain geometry. If the training space sufficiently covers the target evaluation space, the model can generalize accordingly. Furthermore, it can be refined to extend to another region of interest without retraining from scratch. This framework also avoids the need to remesh the domain for each geometry. The proposed approach delivers a computationally lighter surrogate model than FEM alternatives and avoids relying on FEM-generated training data.

Rodolphe Barlogis, Ferhat Tamssaouet, Quentin Falcoz +1

PROMES-CNRS, Université Perpignan Via Domitia (UPVD), Perpignan, France · LAAS-CNRS, Université de Toulouse (UT), Toulouse, France

PDE Solving PDE Operator Learning Deep Operator Networks Physics-Informed Neural Operators

May 1, 2026cs.LG

PILIR: Physics-Informed Local Implicit Representation

Physics-Informed Neural Networks have become a powerful mesh-free method for solving partial differential equations, but their performance is often limited by spectral bias. Specifically, in standard MLPs used in PINNs, the global parameter coupling causes the model to prioritize learning low-frequency components, resulting in slow convergence for high-frequency details. To overcome this limitation, we introduce the Physics-Informed Local Implicit Representation (PILIR). Our approach separates the global physical domain into a discrete latent feature space and a continuous generative decoder. By using a learnable grid to encode explicit spatial locality, PILIR can capture high-frequency details locally, preventing dilution by global patterns. A generative neural operator then synthesizes these local latent features into continuous physical fields, allowing accurate reconstruction of fine-scale structures. Experiments on a range of challenging PDEs show that PILIR effectively mitigates spectral bias, thereby boosting the convergence of high-frequency details and achieving superior accuracy compared to state-of-the-art methods.

Jianfeng Li, Feng Wang, Ke Tang

School of Computer Science, Wuhan University · School of Computer Science and Technology, Southern University of Science and Technology

Neural PDE Solvers Implicit Neural Representations Spectral Bias PDE Operator Learning Physics-Informed ML Physics-Informed Neural Operators

Apr 30, 2026eess.IV

RETO: A Rotary-Enhanced Transformer Operator for High-Fidelity Prediction of Automotive Aerodynamics

Rapid aerodynamic evaluation is crucial for modern vehicle design, yet existing neural operators struggle to capture intricate spatial correlations. We propose the rotary-enhanced transformer operator (RETO), a novel neural solver featuring a dual-stage spatial awareness mechanism: sinusoidal-cosine encodings for global referencing and rotary positional encodings (RoPE) for relative displacements. RoPE encodes spatial relations via unitary rotations, enforcing translation invariance and enhancing local gradient resolution. RETO is validated on ShapeNet and the high-fidelity DrivAerML benchmark. On ShapeNet, RETO achieves a relative

L_2

error of 0.063, outperforming RegDGCNN at 0.125 and representing a 16% improvement over the Transolver baseline, which yields an error of 0.075. These performance gains are further amplified on the DrivAerML dataset, where RETO achieves relative

L_2

errors of 0.089 for surface pressure and 0.097 for velocity. In comparison, Transolver results in errors of 0.116 and 0.121 for the same metrics, indicating that RETO achieves precision enhancements of 23% and 19%, respectively. For comprehensive comparison, the surface pressure and velocity errors for AB-UBT are 0.102 and 0.124, while RegDGCNN yields 0.235 and 0.312, respectively. Information-theoretical analysis shows that the entropy peak of RETO at 0.35 is significantly lower than that of Transolver at 0.75 under

10^4

resolution, indicating a focused attentional mechanism capable of preserving localized gradients against global diffusion.

Bojun Zhang, Huiyu Yang, Yunpeng Wang +4

Department of Mechanics and Aerospace Engineering, Southern University of Science and Technology, Shenzhen, 518055, China · Shenzhen Key Laboratory of Complex Aerospace Flows, Southern University of Science and Technology,Shenzhen 518055, China · Ningbo Key Laboratory of Advanced Manufacturing Simulation, Eastern Institute of Technology, Ningbo, 315200, China +1

Rotary Positional Embeddings PDE Operator Learning Neural Operators Computational Fluid Dynamics

Apr 28, 2026cs.LG

Shearlet Neural Operators for Anisotropic-Shock-Dominated and Multi-scale parametric partial differential equations

Neural operators have emerged as powerful data-driven surrogates for learning solution operators of parametric partial differential equations (PDEs). However, widely used Fourier Neural Operators (FNOs) rely on global Fourier representations, which can be inefficient for resolving anisotropic structures, sharp gradients, and spatially localized discontinuities that arise in shock-dominated and multiscale regimes. To address these limitations, we introduce the Shearlet Neural Operator (SNO), a neural operator architecture that replaces the Fourier transform with a shearlet-based representation. Shearlets offer directional, multiscale, and spatially localized atoms with near-optimal sparse approximation of anisotropic features, providing an inductive bias aligned with PDE solutions containing edges, fronts, and shocks. SNO learns in the shearlet domain and reconstructs predictions via the inverse transform, retaining efficient spectral computation while improving locality and directional selectivity. Across seven benchmark PDE families, including strongly anisotropic advection, anisotropic diffusion, and nonlinear conservation laws with straight, curved, interacting, spiral, and polygonal shock structures, SNO consistently improves predictive accuracy and feature fidelity over FNO baselines, with the largest gains observed in anisotropic and discontinuity-dominated settings.

Fabio Pereira dos Santos, Julio de Castro Vargas Fernandes, Adriano Mauricio de Almeida Cortes

Coordination of Mathematical and Computational Methods National Laboratory of Scientific Computing Av. Getúlio Vargas, 333 - Quitandinha, Petrópolis - RJ, 25651-075 · Systems and Computational Engineering Program, COPPE/UFRJ, Federal University of Rio de Janeiro, P.O. Box 68542, Rio de Janeiro, RJ, 21941-909, Brasil

PDE Solving PDE Operator Learning Neural Operators

Apr 27, 2026math.NA

Adaptive-Distribution Randomized Neural Networks for PDEs: A Low-Dimensional Distribution-Learning Framework

Randomized neural networks (RaNNs) are attractive for partial differential equations (PDEs) because they replace expensive end-to-end training with a linear least-squares solve over randomized hidden features. Their practical performance, however, depends strongly on the sampling distribution of the hidden-layer parameters, which is usually chosen heuristically and problem by problem. This distribution sensitivity is a central bottleneck in randomized neural PDE solvers. In this work, we propose Adaptive-Distribution Randomized Neural Networks (AD-RaNN), a framework that promotes randomized feature generation from a fixed heuristic choice to a low-dimensional adaptive optimization problem. Instead of training all hidden weights and biases, AD-RaNN parameterizes the hidden-feature sampling distribution by a low-dimensional vector p and optimizes only p, thereby preserving the least-squares structure of RaNNs while reducing manual distribution tuning. The method uses a two-stage strategy: ridge-regularized reduced training for stable distribution-parameter optimization, followed by an unregularized least-squares refit for final solution recovery. We develop two adaptive mechanisms, PDE-Driven Adaptive Distribution (PDAD) and Data-Driven Adaptive Distribution (DDAD), and deploy them in space-time solvers, discrete-time solvers, and operator-learning models. We also incorporate an adaptive layer-growth enhancement for localized structures. For the reduced optimization problem, we establish well-posedness of the reduced objectives, consistency of ridge-regularized minimizers, an efficient gradient formula, and a practical lower-bound estimate for the ridge parameter. Numerical experiments on benchmark problems show that AD-RaNN provides an effective distribution-level adaptation mechanism, reduces reliance on hand-crafted hidden-feature distributions, and achieves strong empirical accuracy.

You Yang, Fei Wang

School of Mathematics and Statistics, Xi’an Jiaotong University, Xi’an, Shaanxi 710049, China. · School of Mathematics and Statistics & State Key Laboratory of Multiphase Flow in Power Engineering, Xi’an Jiaotong University, Xi’an, Shaanxi 710049, China.

Neural PDE Solvers PDE Operator Learning

Apr 22, 2026cs.LG

Droplet-LNO: Physics-Informed Laplace Neural Operators for Accurate Prediction of Droplet Spreading Dynamics on Complex Surfaces

Spreading of liquid droplets on solid substrates constitutes a classic multiphysics problem with widespread applications ranging from inkjet printing, spray cooling, to biomedical microfluidic systems. Yet, accurate computational fluid dynamic (CFD) simulations are prohibitively expensive, taking more than 18 to 24 hours for each transient computation. In this paper, Physics-Informed Laplace Operator Neural Network (PI-LNO) is introduced, representing a novel architecture where the Laplace integral transform function serves as a learned physics-informed functional basis. Extensive comparative benchmark studies were performed against five other state-of-the-art approaches: UNet, UNet with attention modules (UNet-AM), DeepONet, Physics-Informed UNet (PI-UNet), and Laplace Neural Operator (LNO). Through complex Laplace transforms, PI-LNO natively models the exponential transient dynamics of the spreading process. A TensorFlow-based PI-LNO is trained on multi-surface CFD data spanning contact angles

θ_s ε[20,160]

, employing a physics-regularized composite loss combining data fidelity (MSE, MAE, RMSE) with Navier-Stokes, Cahn-Hilliard, and causality constraints.

Ganesh Sahadeo Meshram, Partha Pratim Chakrabarti, Suman Chakraborty

Department of Mechanical Engineering, IIT Kharagpur, Kharagpur, India · Department of Computer Science & Engineering, IIT Kharagpur, Kharagpur, India

PDE Operator Learning Physics-Informed Neural Operators Neural Operators Computational Fluid Dynamics

Apr 20, 2026cs.LG

Neural Shape Operator Surrogates -- Expression Rate Bounds

We prove error bounds for operator surrogates of solution operators for partial differential and boundary integral equations on families of domains which are diffeomorphic to one common reference (or latent) domain

D_{ref}

. The pullback of the PDE to

D_{ref}

via affine-parametric shape encoding produces a collection of holomorphic parametric PDEs on

D_{ref}

. Sufficient conditions for (uniformly with respect to the parameter) well-posedness are given, implying existence, uniqueness and stability of parametric solution families on

D_{ref}

. We illustrate the abstract hypotheses by reviewing recent holomorphy results for a suite of elliptic and parabolic PDEs. Quantified parametric holomorphy implies existence of finite-parametric, discrete approximations of the parametric solution families with convergence rates in terms of the number

N

of parameters. We obtain constructive proofs of existence of Neural and Spectral Operator surrogates for the shape-to-solution maps with error bounds and convergence rate guarantees uniform on the collection of admissible shapes. We admit principal-component shape encoders and frame decoders. Our results support in particular the (empirically reported) ability of neural operators to realize data-to-solution maps for elliptic and parabolic PDEs and BIEs that generalize across parametric families of shapes.

Helmut Harbrecht, Christoph Schwab

PDE Operator Learning Neural Network Approximation Theory Operator Learning Neural Operators

Apr 18, 2026math.NA

Singularity Formation: Synergy in Theoretical, Numerical and Machine Learning Approaches

This thesis develops numerical and theoretical approaches for understanding and analyzing singularity formation in Partial Differential Equations (PDEs). The singularity formation in the Navier-Stokes Equation (NSE) is famously challenging as one of the seven Clay Prize problems. Unlike simpler equations such as the Nonlinear Heat (NLH) or Keller-Segel (KS) equations, where formal asymptotics near blowup are better understood, the intrinsic complexity of NSE makes quantitative analytical treatment difficult, if not impossible, without numerical guidance. Building on numerical insights, we introduce a robust analytical framework to simplify and systematize pen-and-paper proofs for simpler singular PDEs. We present a novel approach based on enforcing vanishing modulation conditions for perturbations around approximate blowup profiles, complemented by singularly weighted energy estimates. We demonstrate the efficacy of our method on PDEs with complicated asymptotics, such as NLH and the Complex Ginzburg-Landau (CGL) equation, and address the open problem of singularity formation in the 3D KS equation with logistic damping. We develop and refine numerical approaches that facilitate deeper insights into singularity formation. We demonstrate that machine learning methods significantly enhance our capability to identify and characterize potential blowup solutions with high precision. We improve on existing Physics-Informed Neural Network (PINN) and Neural Operator (NO) frameworks. Moreover, we present a novel machine learning paradigm, the Kolmogorov-Arnold Network (KAN) architecture, whose interpretability and excellent scaling properties are achieved through learnable nonlinearities.

Yixuan Wang

CALIFORNIA INSTITUTE OF TECHNOLOGY Pasadena, California · [math.NA]

PDE Solving Kolmogorov-Arnold Networks PDE Operator Learning Physics-Informed ML

Apr 17, 2026cs.LG

Late Fusion Neural Operators for Extrapolation Across Parameter Space in Partial Differential Equations

Developing neural operators that accurately predict the behavior of systems governed by partial differential equations (PDEs) across unseen parameter regimes is crucial for robust generalization in scientific and engineering applications. In practical applications, variations in physical parameters induce distribution shifts between training and prediction regimes, making extrapolation a central challenge. As a result, the way parameters are incorporated into neural operator models plays a key role in their ability to generalize, particularly when state and parameter representations are entangled. In this work, we introduce the Late Fusion Neural Operator, an architecture that disentangles learning state dynamics from parameter effects, improving predictive performance both within and beyond the training distribution. Our approach combines neural operators for learning latent state representations with sparse regression to incorporate parameter information in a structured manner. Across four benchmark PDEs including advection, Burgers, and both 1D and 2D reaction-diffusion equations, the proposed method consistently outperforms Fourier Neural Operator and CAPE-FNO. Late Fusion Neural Operators achieve consistently the best performance in all experiments, with an average RMSE reduction of 72.9% in-domain and 71.8% out-domain compared to the second-best method. These results demonstrate strong generalization across both in-domain and out-domain parameter regimes.

Eva van Tegelen, Taniya Kapoor, George A. K. van Voorn +2

Artificial Intelligence Group, Wageningen University and Research, Netherlands · Mathematical and Statistical Methods, Wageningen University and Research, Netherlands

Surrogate Modeling PDE Surrogate Modeling OOD Generalization PDE Operator Learning Neural Operators

Mar 18, 2026cs.LG

Translation Invariance of Neural Operators for the FitzHugh-Nagumo Model

Neural operators (NOs) are powerful deep learning frameworks designed to learn solution operators of partial differential equations. This study evaluates the ability of NOs' to capture the stiff spatio-temporal dynamics of the FitzHugh-Nagumo model. A key contribution of this study is the assessment of the translation invariance using a novel training strategy. Models are trained using an applied current with varying spatial locations and intensities at a fixed time, while the test set presents a challenging out-of-distribution scenario where the current is translated in both time and space. This approach significantly reduces dataset generation costs. We benchmark seven NO architectures: Convolutional Neural Operators (CNOs), Deep Operator Networks (DeepONets), DeepONets with CNN encoders, Proper Orthogonal Decomposition DeepONets, Fourier Neural Operators (FNOs), Tucker Tensorized FNOs, and Local Neural Operators. We evaluated these models based on their accuracy, efficiency, and inference speed. These results demonstrate that CNOs generalize well to translated test dynamics, whereas other architectures do not generalize well. On the training set, all architectures achieve comparable accuracy, with FNOs achieving the highest precision. However, this higher accuracy comes at an elevated computational cost. Meanwhile, DeepONets and their variants exhibit superior training and inference efficiency. These findings highlight the capabilities and limitations of NOs in modeling complex ionic dynamics and provide a comprehensive benchmark for scenarios involving translated dynamics.

Luca Pellegrini

Department of Mathematics, University of Pavia, Pavia, Italy · Euler Institute, Faculty of Informatics, Universit`a della Svizzera italiana, Lugano, Switzerland

Fourier Neural Operator PDE Operator Learning Deep Operator Networks Neural Operators

Mar 4, 2026cs.LG

Jacobian-Adaptive Weighting for Stability: Enhancing Long-term Rollout of Neural Partial Differential Equation Solvers via Spatially-Adaptive Regularization

Data-driven surrogate models can significantly accelerate the simulation of continuous dynamical systems, yet the step-wise accumulation of errors during autoregressive time-stepping often leads to spectral blow-up and unphysical divergence. Existing global regularization techniques can enforce contractive dynamics but uniformly damp high-frequency features, causing over-smoothing; meanwhile, long-horizon trajectory optimization methods are severely constrained by memory bottlenecks. This paper proposes Jacobian-Adaptive Weighting for Stability (JAWS), which reformulates operator learning as a Maximum A Posteriori (MAP) estimation problem with spatially heteroscedastic uncertainty, enabling the regularization strength to adapt automatically based on local physical complexity: enforcing contraction in smooth regions to suppress noise while relaxing constraints near singular features such as shocks to preserve gradient information. Experiments demonstrate that JAWS serves as an effective spectral pre-conditioner for trajectory optimization, allowing short-horizon, memory-efficient training to match the accuracy of long-horizon baselines. Validations on the 1D viscous Burgers' equation and 2D flow past a cylinder (

\text{Re}=400

out-of-distribution generalization) confirm the method's advantages in long-term stability, preservation of physical conservation properties, and computational efficiency. This significant reduction in memory usage makes the method particularly well-suited for stable and efficient long-term simulation of large-scale flow fields in practical engineering applications.

Fengxiang Nie, Yasuhiro Suzuki

University of Hiroshima

Neural PDE Solvers Algorithmic Stability PDE Operator Learning Computational Fluid Dynamics

Feb 17, 2026physics.flu-dyn

Fluids You Can Trust: Property-Preserving Operator Learning for Incompressible Flows

We present a novel property-preserving kernel-based operator learning method for incompressible flows governed by the incompressible Navier--Stokes equations. Traditional numerical solvers incur significant computational costs to respect incompressibility. Operator learning offers efficient surrogate models, but current neural operators fail to exactly enforce physical properties such as incompressibility, periodicity, and turbulence. Our kernel method maps input functions to expansion coefficients of output functions in a property-preserving kernel basis, ensuring that predicted velocity fields \emph{analytically} and \emph{simultaneously} preserve the aforementioned physical properties. We present universal approximation results and worst-case a priori convergence rates for our framework; empirically, the observed convergence rates exceed the pessimistic predictions across most benchmarks, motivating a formulation of more optimistic convergence rates. Another central contribution of this work is a novel computational framework revolving around streaming construction of kernel Gramians and recursive Schur-complement algorithms to solve the large block linear systems arising from property-preserving kernel methods. This kernel-based framework makes operator learning practical at scales up to

10{,}000

training functions sampled at up to

10{,}000

spatial locations. We evaluate the method on challenging 2D and 3D, laminar and turbulent, incompressible flow problems. Our method achieves up to six orders of magnitude lower relative

\ell_2

errors upon generalization and trains up to five orders of magnitude faster compared to neural operators. Moreover, while our method enforces incompressibility analytically, neural operators exhibit large divergence errors. Our results show that our method provides an accurate and efficient surrogate for incompressible flows.

Ramansh Sharma, Matthew Lowery, Houman Owhadi +1

Kahlert School of Computing University of Utah UT, USA · Department of Computing and Mathematical Sciences California Institute of Technology CA, USA

PDE Operator Learning Operator Learning Computational Fluid Dynamics

Feb 4, 2026cs.LG

Transolver-3: Scaling Up Transformer Solvers to Industrial-Scale Geometries

Deep learning has emerged as a transformative tool for the neural surrogate modeling of partial differential equations (PDEs), known as neural PDE solvers. However, scaling these solvers to industrial-scale geometries with over

10^8

cells remains a fundamental challenge due to the prohibitive memory complexity of processing high-resolution meshes. We present Transolver-3, a new member of the Transolver family as a highly scalable framework designed for high-fidelity physics simulations. To bridge the gap between limited GPU capacity and the resolution requirements of complex engineering tasks, we introduce two key architectural optimizations: faster slice and deslice by exploiting matrix multiplication associative property and geometry slice tiling to partition the computation of physical states. Combined with an amortized training strategy by learning on random subsets of original high-resolution meshes and a physical state caching technique during inference, Transolver-3 enables high-fidelity field prediction on industrial-scale meshes. Extensive experiments demonstrate that Transolver-3 can handle meshes with over 160 million cells, achieving impressive performance across three challenging simulation benchmarks, including aircraft and automotive design tasks. Code is available at https://github.com/thuml/Transolver-3.

Hang Zhou, Haixu Wu, Haonan Shangguan +4

School of Software, BNRist, Tsinghua University, China.

Efficient Neural Network Inference Neural PDE Solvers PDE Surrogate Modeling PDE Operator Learning Neural Operators

Jan 23, 2026cs.LG

SFO: Learning PDE Operators via Spectral Filtering

Partial differential equations (PDEs) govern complex systems, yet neural operators often struggle to efficiently capture the long-range, nonlocal interactions inherent in their solution maps. We introduce Spectral Filtering Operator (SFO), a neural operator that parameterizes integral kernels using the Universal Spectral Basis (USB), a fixed, global orthonormal basis derived from the eigenmodes of the Hilbert matrix in spectral filtering theory. Motivated by our theoretical finding that the discrete Green's functions of shift-invariant PDE discretizations exhibit spatial Linear Dynamical System (LDS) structure, we prove that these kernels admit compact approximations in the USB. By learning only the spectral coefficients of rapidly decaying eigenvalues, SFO achieves a highly efficient representation. Across six benchmarks, including reaction-diffusion, fluid dynamics, and 3D electromagnetics, SFO achieves state-of-the-art accuracy, reducing error by up to 40% relative to strong baselines while using substantially fewer parameters.

Noam Koren, Rafael Moschopoulos, Kira Radinsky +1

Department of Computer Science Technion - Israel Institute of Technology · Department of Computer Science Princeton University

PDE Surrogate Modeling PDE Operator Learning Spectral Methods Neural Operators

Dec 29, 2025cs.LG

PGOT: A Physics-Geometry Operator Transformer for Complex PDEs

While Transformers have demonstrated remarkable potential in modeling Partial Differential Equations (PDEs), modeling large-scale unstructured meshes with complex geometries remains a significant challenge. Existing efficient architectures often employ feature dimensionality reduction strategies, which inadvertently induces Geometric Aliasing, resulting in the loss of critical physical boundary information. To address this, we propose the Physics-Geometry Operator Transformer (PGOT), designed to reconstruct physical feature learning through explicit geometry awareness. Specifically, we propose Spectrum-Preserving Geometric Attention (SpecGeo-Attention). Utilizing a ``physics slicing-geometry injection" mechanism, this module incorporates multi-scale geometric encodings to explicitly preserve multi-scale geometric features while maintaining linear computational complexity

O(N)

. Furthermore, PGOT dynamically routes computations to low-order linear paths for smooth regions and high-order non-linear paths for shock waves and discontinuities based on spatial coordinates, enabling spatially adaptive and high-precision physical field modeling. PGOT achieves consistent state-of-the-art performance across four standard benchmarks and excels in large-scale industrial tasks including airfoil and car designs.

Zhuo Zhang, Xi Yang, Ying Miao +5

National University of Defense Technology · Peking University · National University of Singapore +2

Neural PDE Solvers PDE Surrogate Modeling Geometric Representation Learning PDE Operator Learning Neural Operators

Dec 9, 2025cs.LG

Spectral Embedding via Chebyshev Bases for Robust DeepONet Approximation

Deep Operator Networks (DeepONets) have emerged as a powerful framework for data-driven operator learning, providing flexible surrogates for nonlinear mappings arising in partial differential equations (PDEs). However, the standard trunk network, which operates directly on raw spatial or spatiotemporal coordinates through fully connected layers, often struggles to represent sharp gradients, boundary layers, and other non-periodic solution structures on bounded domains. To address these limitations, we introduce the Spectral-Embedded Deep Operator Network (SEDONet), a novel DeepONet architecture in which the trunk is driven by a fixed Chebyshev spectral dictionary instead of coordinate inputs. This non-periodic spectral embedding provides a principled inductive bias for bounded domains, enabling the learned operator to capture fine-scale features that are difficult for Fourier-based or MLP-only trunks to represent. SEDONet is evaluated on the 2-D Poisson equation, 1-D Burgers' equation, 1-D advection-diffusion equation, Allen-Cahn equation, Lorenz-96 chaotic system, and Darcy flow, covering elliptic, hyperbolic, parabolic, chaotic, and multiscale problems. Across all benchmarks, SEDONet consistently achieves the lowest or statistically comparable relative

L^2

errors among DeepONet, FEDONet, and SEDONet, with improvements of up to 54% over the baseline DeepONet and consistent gains over Fourier-embedded variants on bounded, non-periodic problems. Energy spectrum analyses further demonstrate that SEDONet more accurately preserves intermediate- and high-frequency solution structures. The proposed framework provides a simple, parameter-neutral modification to DeepONets, offering a robust and computationally efficient spectral approach for surrogate modeling of nonlinear operators in scientific computing.

Muhammad Abid, Omer San

Department of Mechanical and Aerospace Engineering, University of Tennessee, Knoxville, TN 37996, USA.

PDE Surrogate Modeling PDE Operator Learning Spectral Methods Deep Operator Networks

Oct 21, 2025cs.LG

Solver-Integrated Adversarial Attacking and Training of Neural Operators

Neural operators are widely used as fast surrogates for numerical PDE solvers, mapping input functions to solution functions. However, their generalizability and robustness are not yet clearly defined in the operator-learning setting, which differs from traditional adversarial robustness definitions. This paper studies the generalizability and robustness of a learned neural operator from a solver-integrated perspective, addressing the challenge that the output of a learned operator and a numerical solver tends to change in tandem under input perturbation. First, we formalize the definition of generalization and robustness through a model-solver error operator, identifying fixed-input model-solver loss as generalization metric, and norm-bounded adversarial attack loss increase and Jacobian-error function norm as robustness metric. Second, we identify the solver-integrated adversarial attack as appropriate for PDE operator learning and show why model-only or fixed-ground-truth attacks can be insufficient when the solver output also changes with the input. Third, we develop solver-integrated adversarial training methods for neural operators. Experiments on representative PDE benchmarks show that this solver-integrated adversarial training clearly improves both generalizability and robustness. Deeper solver integration yields more effective attacks, more informative samples, and more efficient training than less integrated alternatives. These results provide a general framework for robust operator training and automatic sample selection without heavy manual intervention. More broadly, the formulation applies to adversarial regression whenever a ground-truth oracle can evaluate, and ideally differentiate, the true input-output map; PDE operator learning is one such case.

Yifei Sun

Adversarial Training Adversarial Attacks Neural Network Robustness Adversarial Robustness PDE Operator Learning Neural Operators

Oct 1, 2025cs.LG

RheOFormer: A generative transformer model for simulation of complex fluids and flows

The ability to model mechanics of soft materials under flowing conditions is key in designing and engineering processes and materials with targeted properties. This generally requires solution of internal stress tensor, related to the deformation tensor through nonlinear and history-dependent constitutive models. Traditional numerical methods for non-Newtonian fluid dynamics often suffer from prohibitive computational demands and poor scalability to new problem instances. Developments in data-driven methods have mitigated some limitations but still require retraining across varied physical conditions. In this work, we introduce Rheological Operator Transformer (RheOFormer), a generative operator learning method leveraging self-attention to efficiently learn different spatial interactions and features of complex fluid flows. We benchmark RheOFormer across a range of different viscometric and non-viscometric flows with different types of viscoelastic and elastoviscoplastic mechanics in complex domains against ground truth solutions. Our results demonstrate that RheOFormer can accurately learn both scalar and tensorial nonlinear mechanics of different complex fluids and predict the spatio-temporal evolution of their flows, even when trained on limited datasets. Its strong generalization capabilities and computational efficiency establish RheOFormer as a robust neural surrogate for accelerating predictive complex fluid simulations, advancing data-driven experimentation, and enabling real-time process optimization across a wide range of applications.

Maedeh Saberi, Amir Barati Farimani, Safa Jamali

Department of Mechanical and Industrial Engineering, Northeastern University, Boston, MA 02115, USA · Department of Mechanical Engineering, Carnegie Mellon University, Pittsburgh, PA 15213, USA

Neural Surrogate Modeling Transformer PDE Operator Learning Computational Fluid Dynamics

Jul 9, 2025cs.LG

Discretization-independent operator learning for partial differential equations

We develop a new and general encode-approximate-reconstruct operator learning model that leverages learned neural representations of bases for input and output function distributions. We introduce the concepts of numerical operator learning and discretization independence, which clarify the relationship between theoretical formulations and practical realizations of operator learning models. Our model is discretization-independent, making it particularly effective for multiresolution learning. We establish theoretical approximation guarantees, demonstrating uniform universal approximation under strong assumptions on the input functions and statistical approximation under weaker conditions. To our knowledge, this is the first comprehensive study that investigates how discretization independence enables robust and efficient multiresolution operator learning. We validate our method through extensive numerical experiments involving both local and nonlocal PDEs, including time-independent and time-dependent problems. The results show that multiresolution training significantly improves accuracy and computational efficiency. Moreover, multiresolution training further enhances empirical discretization independence.

Jacob Hauck, Yanzhi Zhang

Department of Mathematics and Statistics, Missouri University of Science and Technology, Rolla, MO 65409

PDE Operator Learning Neural Network Approximation Theory Operator Learning Neural Operators

Jun 23, 2025math.NA

DPG loss functions for learning parameter-to-solution maps by neural networks

We develop, analyze, and experimentally explore residual-based loss functions for machine learning of parameter-to-solution maps in the context of parameter-dependent families of partial differential equations (PDEs). Our primary concern is on rigorous accuracy certification to enhance the prediction capability of the resulting deep neural network reduced models. This is achieved by the use of variationally correct loss functions. Through one specific example of an elliptic PDE, details for establishing the variational correctness of a loss function from an ultraweak Discontinuous Petrov Galerkin (DPG) discretization are worked out. Despite the focus on the example, the proposed concepts apply to a much wider scope of problems, namely problems for which stable DPG formulations are available. The issue of high-contrast diffusion fields and ensuing difficulties with degrading ellipticity are discussed. Both numerical results and theoretical arguments illustrate that for high-contrast diffusion parameters the proposed DPG loss functions deliver much more robust performance than simpler least-squares losses.

Pablo Cortés Castillo, Wolfgang Dahmen, Jay Gopalakrishnan

PDE Solving PDE Surrogate Modeling Robust Loss Functions PDE Operator Learning Physics-Informed ML

Mar 7, 2025cs.CE

From Theory to Application: A Practical Introduction to Neural Operators in Scientific Computing

This review examines neural operator architectures for learning solution operators of parametric partial differential equations (PDEs), with an emphasis on conceptual clarity and practical implementation. The work analyzes key models, including DeepONet, PCANet, and the Fourier Neural Operator, highlighting their underlying representations, computational structures, and comparative performance. These architectures are demonstrated on three canonical PDE problems: the Poisson equation, a linear elasticity problem, and a hyperelasticity problem. To make the presentation self-contained, key foundational topics are introduced, including finite-dimensional representations of function spaces, singular-value decomposition, and sampling from infinite-dimensional function spaces. Beyond forward modeling, the review discusses the use of neural operators as surrogate models within a Bayesian inverse-problem framework, including prior specification, forward-map approximation, and posterior computation. The performance of the three neural-operator architectures is evaluated on in-distribution samples, out-of-distribution samples, and Bayesian inference tasks. The review also discusses challenges related to prediction accuracy and generalization, outlining emerging strategies such as residual-based error correction and multi-level training. The review concludes by positioning neural operators within broader scientific-computing workflows and by identifying directions for reliable, scalable operator learning.

Prashant K. Jha

Department of Mechanical Engineering, South Dakota School of Mines and Technology, Rapid City, SD 57701, USA.

Neural Surrogate Modeling Fourier Neural Operator PDE Operator Learning Deep Operator Networks Neural Operators

Feb 1, 2025cs.LG

Active Learning with Bayesian Multi-Fidelity Laplace Neural Operators for Oscillatory Parametric PDEs

Surrogate models of parametric dynamical systems are essential for many-query and real-time predictions in engineering applications such as design optimization and digital twins. However, generating high-fidelity (HF) training data over a broad range of parameters and operating conditions remains computationally expensive. To address this challenge, we propose a Bayesian multi-fidelity Laplace neural operator (MF-LNO) for uncertainty-aware active learning of oscillatory parametric PDEs. Specifically, the proposed Bayesian MF-LNO iteratively calibrates the discrepancy between low- and high-fidelity data, where predictive uncertainty guides the adaptive acquisition of informative HF trajectories. Such predictive uncertainty is quantified via replica-exchange stochastic gradient Langevin dynamics (reSGLD), whose broad posterior exploration enables uncertainty to serve as an error indicator for adaptive HF sample acquisition. Numerical experiments on the Lorenz system, Duffing oscillator, and beam dynamics demonstrate that uncertainty-guided HF sample acquisition consistently outperforms random sampling, while the proposed Bayesian MF-LNO achieves higher prediction accuracy than MF-DeepONet with predictive uncertainty quantification. These results demonstrate that Bayesian multi-fidelity LNOs, combined with uncertainty-guided active learning, provide a data-efficient framework for operator learning in engineering dynamical systems.

Bongseok Kim, Haoyang Zheng, Michael Penwarden +1

School of Mechanical Engineering, Purdue University, 585 Purdue Mall, West Lafayette, 47907, IN, USA. · Department of Mathematics, Purdue University, 150 North University Street, West Lafayette, 47907, IN, USA.

Dynamical Systems Surrogate Modeling Uncertainty Quantification Active Learning PDE Operator Learning Neural Operators

Dec 15, 2021math.NA

Exponential Convergence of Deep Operator Networks for Elliptic Partial Differential Equations

We construct and analyze approximation rates of deep operator networks (ONets) between infinite-dimensional spaces that emulate with an exponential rate of convergence the coefficient-to-solution map of elliptic second-order partial differential equations. In particular, we consider problems set in

d

-dimensional periodic domains,

d=1, 2, \dots

, and with analytic right-hand sides and coefficients. Our analysis covers linear, elliptic second order divergence-form PDEs as, e.g., diffusion-reaction problems, parametric diffusion equations, and elliptic systems such as linear isotropic elastostatics in heterogeneous materials. We leverage the exponential convergence of spectral collocation methods for boundary value problems whose solutions are analytic. In the present periodic and analytic setting, this follows from classical elliptic regularity. Within the ONet branch and trunk construction of [Chen and Chen, 1993] and of [Lu et al., 2021], we show the existence of deep ONets which emulate the coefficient-to-solution map to a desired accuracy in the

H^1

norm, uniformly over the coefficient set. We prove that the neural networks in the ONet have size

\mathcal{O}(\left|\log(\varepsilon)\right|^κ)

, where

\varepsilon>0

is the approximation accuracy, for some

κ>0

depending on the physical space dimension.

Carlo Marcati, Christoph Schwab

Seminar for Applied Mathematics, ETH Zurich, Switzerland

PDE Solving PDE Operator Learning Neural Network Approximation Theory Deep Operator Networks