Neural Operators

Follow this topic Ask about this topic

Neural operators learn maps from input fields, parameters, or geometries to output fields, often approximating the solution map of a partial differential equation. DeepONet uses branch and trunk networks, Fourier Neural Operators learn interactions in spectral space, and graph and transformer operators handle irregular meshes and complex geometries; physics-informed losses and structure-preserving designs add physical constraints. These models can make repeated simulation, forecasting, and inverse design much faster than running a numerical solver for every case. A central challenge is making predictions reliable across new resolutions, geometries, and physical regimes while preserving stability over long rollouts and matching classical solvers when high accuracy is required.

Latest papers

313 papers52 papers in the last four weeks, up 24% on the four weeks before. 0.3% of all new papers.

Mar 16, 2026

Physics-Informed Neural Systems for the Simulation of EUV Electromagnetic Wave Diffraction from a Lithography Mask

Physics-informed neural networks (PINNs) and neural operators (NOs) for solving the problem of diffraction of Extreme Ultraviolet (EUV) electromagnetic waves from contemporary lithography masks are presented. A novel hybrid Waveguide Neural Operator (WGNO) is introduced, based on a waveguide method with its most computationally expensive components replaced by a neural network. To evaluate performance, the accuracy and inference time of PINNs and NOs are compared against modern numerical solvers for a series of problems with known exact solutions. The emphasis is placed on investigation of solution accuracy by considered artificial neural systems for 13.5 nm and 11.2 nm wavelengths. Numerical experiments on realistic 2D and 3D masks demonstrate that PINNs and neural operators achieve competitive accuracy and significantly reduced prediction times, with the proposed WGNO architecture reaching state-of-the-art performance. The presented neural operator has pronounced generalizing properties, meaning that for unseen problem parameters it delivers a solution accuracy close to that for parameters seen in the training dataset. These results provide a highly efficient solution for accelerating the design and optimization workflows of next-generation lithography masks.

Vasiliy A. Es'kin, Egor V. Ivanov

Department of Radiophysics, University of Nizhny Novgorod 23 Gagarin Ave., Nizhny Novgorod 603022, Russia

Neural PDE Solvers Neural Operators Physics-Informed ML Semiconductor Manufacturing

Mar 16, 2026

Gauge-Equivariant Intrinsic Neural Operators for Geometry-Consistent Learning of Elliptic PDE Maps

Learning solution operators of partial differential equations (PDEs) from data has emerged as a promising route to fast surrogate models in multi-query scientific workflows. However, for geometric PDEs whose inputs and outputs transform under changes of local frame (gauge), many existing operator-learning architectures remain representation-dependent, brittle under metric perturbations, and sensitive to discretization changes. We propose Gauge-Equivariant Intrinsic Neural Operators (GINO), a class of neural operators that parameterize elliptic solution maps primarily through intrinsic spectral multipliers acting on geometry-dependent spectra, coupled with gauge-equivariant nonlinearities. This design decouples geometry from learnable functional dependence and enforces consistency under frame transformations. We validate GINO on controlled problems on the flat torus (

\mathbb{T}^2

), where ground-truth resolvent operators and regularized Helmholtz--Hodge decompositions admit closed-form Fourier representations, enabling theory-aligned diagnostics. Across experiments E1--E6, GINO achieves low operator-approximation error, near machine-precision gauge equivariance, robustness to structured metric perturbations, strong cross-resolution generalization with small commutation error under restriction/prolongation, and structure-preserving performance on a regularized exact/coexact decomposition task. Ablations further link the smoothness of the learned spectral multiplier to stability under geometric perturbations. These results suggest that enforcing intrinsic structure and gauge equivariance yields operator surrogates that are more geometry-consistent and discretization-robust for elliptic PDEs on form-valued fields.

Pengcheng Cheng

School of Mathematics, Jilin University, Changchun, 130012, China

PDE Operator Learning Neural Operators PDE Solving Equivariant Neural Networks

Mar 15, 2026

Windowed Fourier Propagator: A Frequency-Local Neural Operator for Wave Equations in Inhomogeneous Media

Wave equations are fundamental to describing a vast array of physical phenomena, yet their simulation in inhomogeneous media poses a computational challenge due to the highly oscillatory nature of the solutions. To overcome the high costs of traditional solvers, we propose the Windowed Fourier Propagator (WFP), a novel neural operator that efficiently learns the solution operator. The WFP's design is rooted in the physical principle of frequency locality, where wave energy scatters primarily to adjacent frequencies. By learning a set of compact, localized propagators, each mapping an input frequency to a small window of outputs, our method avoids the complexity of dense interaction models and achieves computational efficiency. Another key feature is the explicit preservation of superposition, which enables remarkable generalization from simple training data (e.g., plane waves) to arbitrary, complex wave states. We demonstrate that the WFP provides an explainable, efficient and accurate framework for data-driven wave modeling in complex media.

Yiyang Cai, Zixuan Qiu, Yunlu Shu +4

School of Mathematical Sciences, Fudan University, Shanghai, 200433, China · Shanghai Center for Mathematical Sciences, Fudan University, Shanghai, 200438, China · Center for Applied Mathematics, Fudan University, Shanghai, 200438, China

PDE Operator Learning Neural Operators Neural Surrogate Modeling Fourier Neural Operator

Mar 13, 2026

NOIR: Neural Operator mapping for Implicit Representations

This paper presents NOIR, a framework that reframes core medical imaging tasks as operator learning between continuous function spaces, challenging the prevailing paradigm of discrete grid-based deep learning. Instead of operating on fixed pixel or voxel grids, NOIR embeds discrete medical signals into shared Implicit Neural Representations and learns a Neural Operator that maps between their latent modulations, enabling resolution-independent function-to-function transformations. We evaluate NOIR across multiple 2D and 3D downstream tasks, including segmentation, shape completion, image-to-image translation, and image synthesis, on several public datasets such as Shenzhen, OASIS-4, SkullBreak, fastMRI, as well as an in-house clinical dataset. It achieves competitive performance at native resolution while demonstrating strong robustness to unseen discretizations, and empirically satisfies key theoretical properties of neural operators. The project page is available here: https://github.com/Sidaty1/NOIR-io.

Sidaty El Hadramy, Nazim Haouchine, Michael Wehrli +1

Department of Biomedical Engineering, University of Basel, Allschwil, Switzerland · Harvard Medical School, Brigham and Women’s Hospital, Boston, United States

Neural Operators Medical Image Synthesis Implicit Neural Representations Medical Image Segmentation Image-to-Image Translation

Mar 12, 2026

Topological DeepONets and a generalization of the Chen-Chen operator approximation theorem

Deep Operator Networks (DeepONets) provide a branch-trunk neural architecture for approximating nonlinear operators acting between function spaces. In the classical operator approximation framework, the input is a function

u\in C(K_1)

defined on a compact set

K_1

(typically a compact subset of a Banach space), and the operator maps

u

to an output function

G(u)\in C(K_2)

defined on a compact Euclidean domain

K_2\subset\mathbb{R}^d

. In this paper, we develop a topological extension in which the operator input lies in an arbitrary Hausdorff locally convex space

X

. We construct topological feedforward neural networks on

X

using continuous linear functionals from the dual space

X^*

and introduce topological DeepONets whose branch component acts on

X

through such linear measurements, while the trunk component acts on the Euclidean output domain. Our main theorem shows that continuous operators

G:V\to C(K;\mathbb{R}^m)

, where

V\subset X

and

K\subset\mathbb{R}^d

are compact, can be uniformly approximated by such topological DeepONets. This extends the classical Chen-Chen operator approximation theorem from spaces of continuous functions to locally convex spaces and yields a branch-trunk approximation theorem beyond the Banach-space setting.

Vugar Ismailov

Neural Network Approximation Theory Deep Operator Networks Neural Operators

Mar 12, 2026

Inverse Neural Operator for ODE Parameter Optimization

We propose the Inverse Neural Operator (INO), a two-stage framework for recovering hidden ODE parameters from sparse, partial observations. In Stage 1, a Conditional Fourier Neural Operator (C-FNO) with cross-attention learns a differentiable surrogate that reconstructs full ODE trajectories from arbitrary sparse inputs, suppressing high-frequency artifacts via spectral regularization. In Stage 2, an Amortized Drifting Model (ADM) learns a kernel-weighted velocity field in parameter space, transporting random parameter initializations toward the ground truth without backpropagating through the surrogate, avoiding the Jacobian instabilities that afflict gradient-based inversion in stiff regimes. Experiments on a real-world stiff atmospheric chemistry benchmark (POLLU, 25 parameters) and a synthetic Gene Regulatory Network (GRN, 40 parameters) show that INO outperforms gradient-based and amortized baselines in parameter recovery accuracy while requiring only 0.23s inference time, a 487x speedup over iterative gradient descent.

Zhi-Song Liu, Wenqing Peng, Helmi Toropainen +5

Department of Computational Engineering, Lappeenranta-Lahti University of Technology, Lahti, Finland · Atmospheric Modelling Center Lahti (AMC-Lahti), Finland · University of Helsinki, Helsinki, Finland +3

Parameter Estimation Ordinary Differential Equations Neural Operators Learning to Optimize Neural Surrogate Modeling

Mar 10, 2026

A neural operator for predicting vibration frequency response curves from limited data

In the design of engineered components, rigorous vibration testing is essential for performance validation and identification of resonant frequencies and amplitudes encountered during operation. Performing this evaluation numerically via machine learning has great potential to accelerate design iteration and make testing workflows more efficient. However, dynamical systems are conventionally difficult to solve via machine learning methods without using physics-based regularizing loss functions. To properly perform this forecasting task, a structure that has an inspectable physical obedience can be devised without the use of regularizing terms from first principles. The method employed in this work is a neural operator integrated with an implicit numerical scheme. This architecture enables operators to learn of the underlying state-space dynamics from limited data, allowing generalization to untested driving frequencies and initial conditions. This network can infer the system's global frequency response by training on a small set of input conditions. As a foundational proof of concept, this investigation verifies the machine learning algorithm with a linear, single-degree-of-freedom system, demonstrating implicit obedience of dynamics. This approach demonstrates 99.87% accuracy in predicting the Frequency Response Curve (FRC), forecasting the frequency and amplitude of linear resonance training on 7% of the bandwidth of the solution. By training machine learning models to internalize physics information rather than trajectory, better generalization accuracy can be realized, vastly improving the timeframe for vibration studies on engineered components.

D. Bluedorn, A. Badawy, B. E. Saunders +2

Department of Mechanical and Aerospace Engineering, New Mexico State University, Las Cruces, NM 88003, USA. · Knight Foundation School of Computing and Information Science, Florida International University, Miami, FL 33199, USA. · Sandia National Laboratories, Albuquerque, NM 87123, USA.

Neural Surrogate Modeling Neural Operators Structural Mechanics System Identification

Mar 9, 2026

Wiener Chaos Expansion based Neural Operator for Singular Stochastic Partial Differential Equations

In this paper, we explore how our recently developed Wiener Chaos Expansion (WCE)-based neural operator (NO) can be applied to singular stochastic partial differential equations, e.g., the dynamic

\boldsymbolΦ^4_2

model simulated in the recent works. Unlike the previous WCE-NO which solves SPDEs by simply inserting Wick-Hermite features into the backbone NO model, we leverage feature-wise linear modulation (FiLM) to appropriately capture the dependency between the solution of singular SPDE and its smooth remainder. The resulting WCE-FiLM-NO shows excellent performance on

\boldsymbolΦ^4_2

, as measured by relative

L_2

loss, out-of-distribution

L_2

loss, and autocorrelation score; all without the help of renormalisation factor. In addition, we also show the potential of simulating

\boldsymbolΦ^4_3

data, which is more aligned with real scientific practice in statistical quantum field theory. To the best of our knowledge, this is among the first works to develop an efficient data-driven surrogate for the dynamical

\boldsymbolΦ^4_3

model.

Dai Shi, Luke Thompson, Andi Han +3

University of Cambridge · University of Sydney · Chinese Academy of Science

Stochastic PDEs PDE Surrogate Modeling Neural Operators

Mar 9, 2026

Tau-BNO: Brain Neural Operator for Tau Transport Model

Mechanistic modeling provides a biophysically grounded framework for studying the spread of pathological tau protein in tauopathies like Alzheimer's disease. Existing approaches typically model tau propagation as a diffusive process on the brain's structural connectome, reproducing macroscopic patterns but neglecting microscale cellular transport and reaction mechanisms. The Network Transport Model (NTM) was introduced to fill this gap, explaining how region-level progression of tau emerges from microscale biophysical processes. However, the NTM faces a common challenge for complex models defined by large systems of partial differential equations: the inability to perform parameter inference and mechanistic discovery due to high computational burden and slow model simulations. To overcome this barrier, we propose Tau-BNO, a Brain Neural Operator surrogate framework for rapidly approximating NTM dynamics that captures both intra-regional reaction kinetics and inter-regional network transport. Tau-BNO combines a function operator that encodes kinetic parameters with a query operator that preserves initial state information, while approximating anisotropic transport through a spectral kernel that retains directionality. Empirical evaluations demonstrate high predictive accuracy (

R^2\approx

0.98) across diverse biophysical regimes and an 89% performance improvement over state-of-the-art sequence models like Transformers and Mamba, which lack inherent structural priors. By reducing simulation time from hours to seconds, we show that the surrogate model is capable of producing new insights and generating new hypotheses. This framework is readily extensible to a broader class of connectome-based biophysical models, showcasing the transformative value of deep learning surrogates to accelerate analysis of large-scale, computationally intensive dynamical systems.

Nuutti Barron, Heng Rao, Urmi Saha +6

Department of Radiology and Biomedical Imaging, University of California San Francisco, USA · College of Computer Science and Engineering, Northeastern University, China · Department of Computer Science, Wake Forest University, USA +1

Neural Surrogate Modeling PDE Surrogate Modeling Neural Operators Deep Operator Networks

Mar 4, 2026

Extending Neural Operators: Robust Handling of Functions Beyond the Training Set

We develop a rigorous framework for extending neural operators to handle out-of-distribution input functions. We leverage kernel approximation techniques and provide theory for characterizing the input-output function spaces in terms of Reproducing Kernel Hilbert Spaces (RKHSs). We provide theorems on the requirements for reliable extensions and their predicted approximation accuracy. We also establish formal relationships between specific kernel choices and their corresponding Sobolev Native Spaces. This connection further allows the extended neural operators to reliably capture not only function values but also their derivatives. Our methods are empirically validated through the solution of elliptic partial differential equations (PDEs) involving operators on manifolds having point-cloud representations and handling geometric contributions. We report results on key factors impacting the accuracy and computational performance of the extension approaches.

Blaine Quackenbush, Paul J. Atzberger

Department of Mathematics, University of California Santa Barbara (UCSB). · Department of Mathematics, Department of Mechanical Engineering, University of California Santa Barbara (UCSB)

Neural Operators OOD Generalization PDE Operator Learning Neural Network Approximation Theory

Mar 3, 2026

Shape Derivative-Informed Neural Operators with Application to Risk-Averse Shape Optimization

Shape optimization under uncertainty (OUU) is computationally intensive for classical PDE-based methods due to the high cost of repeated sampling-based risk evaluation across many uncertainty realizations and varying geometries, while standard neural surrogates often fail to provide accurate and efficient sensitivities for optimization. We introduce Shape-DINO, a derivative-informed neural operator framework for learning PDE solution operators on families of varying geometries, with a particular focus on accelerating PDE-constrained shape OUU. Shape-DINOs encode geometric variability through diffeomorphic mappings to a fixed reference domain and employ a derivative-informed operator learning objective that jointly learns the PDE solution and its Fréchet derivatives with respect to design variables and uncertain parameters, enabling accurate state predictions and reliable gradients for large-scale OUU. We establish a priori error bounds linking surrogate accuracy to optimization error and prove universal approximation results for multi-input reduced basis neural operators in suitable

C^1

norms. We demonstrate efficiency and scalability on three representative shape OUU problems, including boundary design for a Poisson equation and shape design governed by steady-state Navier-Stokes exterior flows in two and three dimensions. Across these examples, Shape-DINOs produce more reliable optimization results than operator surrogates trained without derivative information. In our examples, Shape-DINOs achieve 3-8 orders-of-magnitude speedups in state and gradient evaluations. Counting training data generation, Shape-DINOs reduce necessary PDE solves by 1-2 orders-of-magnitude compared to a strictly PDE-based approach for a single OUU problem. Moreover, Shape-DINO construction costs can be amortized across many objectives and risk measures, enabling large-scale shape OUU for complex systems.

Xindi Gong, Dingcheng Luo, Thomas O'Leary-Roseberry +2

Oden Institute for Computational Engineering and Sciences, The University of Texas at Austin, Austin, Texas, USA · School of Mathematical Sciences, Queensland University of Technology, Brisbane, Australia · Centre for Data Science, Queensland University of Technology, Brisbane, Australia +3

PDE Operator Learning Neural Operators Uncertainty Quantification Constrained Optimization

Mar 3, 2026

From Complex Dynamics to DynFormer: Rethinking Transformers for PDEs

Partial differential equations (PDEs) are fundamental for modeling complex physical systems, yet classical numerical solvers face prohibitive computational costs in high-dimensional and multi-scale regimes. While Transformer-based neural operators have emerged as powerful data-driven alternatives, they conventionally treat all discretized spatial points as uniform, independent tokens. This monolithic approach ignores the intrinsic scale separation of physical fields, applying computationally prohibitive global attention that redundantly mixes smooth large-scale dynamics with high-frequency fluctuations. Rethinking Transformers through the lens of complex dynamics, we propose DynFormer, a novel dynamics-informed neural operator. Rather than applying a uniform attention mechanism across all scales, DynFormer explicitly assigns specialized network modules to distinct physical scales. It leverages a Spectral Embedding to isolate low-frequency modes, enabling a Kronecker-structured attention mechanism to efficiently capture large-scale global interactions with reduced complexity. Concurrently, we introduce a Local-Global-Mixing transformation. This module utilizes nonlinear multiplicative frequency mixing to implicitly reconstruct the small-scale, fast-varying turbulent cascades that are slaved to the macroscopic state, without incurring the cost of global attention. Integrating these modules into a hybrid evolutionary architecture ensures robust long-term temporal stability. Extensive memory-aligned evaluations across four PDE benchmarks demonstrate that DynFormer achieves up to a 95% reduction in relative error compared to state-of-the-art baselines, while significantly reducing GPU memory consumption. Our results establish that embedding first-principles physical dynamics into Transformer architectures yields a highly scalable, theoretically grounded blueprint for PDE surrogate modeling.

Pengyu Lai, Yixiao Chen, Dewu Yang +3

School of Aeronautics and Astronautics, Shanghai Jiao Tong University, Shanghai, 200240, China

PDE Surrogate Modeling Neural Operators Physics-Informed Neural Operators Transformer Attention Attention Mechanisms

Mar 2, 2026

Neural Operator-Grounded Continuous Tensor Function Representation and Its Applications

Recently, continuous tensor functions have attracted increasing attention, because they can unifiedly represent data both on mesh grids and beyond mesh grids. However, since mode-

n

product is essentially discrete and linear, the potential of current continuous tensor function representations is still locked. To break this bottleneck, we suggest neural operator-grounded mode-

n

operators as a continuous and nonlinear alternative of discrete and linear mode-

n

product. Instead of mapping the discrete core tensor to the discrete target tensor, proposed mode-

n

operator directly maps the continuous core tensor function to the continuous target tensor function, which provides a genuine continuous representation of real-world data and can ameliorate discretization artifacts. Empowering with continuous and nonlinear mode-

n

operators, we propose a neural operator-grounded continuous tensor function representation (abbreviated as NO-CTR), which can more faithfully represent complex real-world data compared with classic discrete tensor representations and continuous tensor function representations. Theoretically, we also prove that any continuous tensor function can be approximated by NO-CTR. To examine the capability of NO-CTR, we suggest an NO-CTR-based multi-dimensional data completion model. Extensive experiments across various data on regular mesh grids (multi-spectral images and color videos), on mesh girds with different resolutions (Sentinel-2 images) and beyond mesh grids (point clouds) demonstrate the superiority of NO-CTR.

Ruoyang Su, Xi-Le Zhao, Sheng Liu +3

School of Mathematical Sciences, University of Electronic Science and Technology of China, Chengdu 611731, China · School of Mathematics and Statistics, Xi’an Jiaotong University, Xi’an 710049, China · Department of Mathematics, Hong Kong Baptist University, Hong Kong, China

Neural Operators Tensor Completion

Mar 2, 2026

One Operator to Rule Them All? On Boundary-Indexed Operator Families in Neural PDE Solvers

Neural PDE solvers are often described as learning solution operators that map problem data to PDE solutions. In this work, we argue that this interpretation is generally incorrect when boundary conditions vary. We show that standard neural operator training implicitly learns a boundary-indexed family of operators, rather than a single boundary-agnostic operator, with the learned mapping fundamentally conditioned on the boundary-condition distribution seen during training. We formalize this perspective by framing operator learning as conditional risk minimization over boundary conditions, which leads to a non-identifiability result outside the support of the training boundary distribution. As a consequence, generalization in forcing terms or resolution does not imply generalization across boundary conditions. We support our theoretical analysis with controlled experiments on the Poisson equation, demonstrating sharp degradation under boundary-condition shifts, cross-distribution failures between distinct boundary ensembles, and convergence to conditional expectations when boundary information is removed. Our results clarify a core limitation of current neural PDE solvers and highlight the need for explicit boundary-aware modeling in the pursuit of foundation models for PDEs.

Lennon J. Shikhman

College of Computing, Georgia Institute of Technology · Department of Mathematics and Systems Engineering, Florida Institute of Technology

PDE Operator Learning Neural Operators Neural Network Generalization

Mar 1, 2026

Operator Learning Using Weak Supervision from Walk-on-Spheres

Training neural PDE solvers is often bottlenecked by expensive data generation or unstable physics-informed neural network (PINN) involving challenging optimization landscapes due to higher-order derivatives. To tackle this issue, we propose an alternative approach using Monte Carlo approaches to estimate the solution to the PDE as a stochastic process for weak supervision during training. Leveraging the Walk-on-Spheres method, we introduce a learning scheme called Walk-on-Spheres Neural Operator (WoS-NO) which uses weak supervision from WoS to train any given neural operator. We propose to amortize the cost of Monte Carlo walks across the distribution of PDE instances using stochastic representations from the WoS algorithm to generate cheap, noisy, estimates of the PDE solution during training. This is formulated into a data-free physics-informed objective where a neural operator is trained to regress against these weak supervisions, allowing the operator to learn a generalized solution map for an entire family of PDEs. This strategy does not require expensive pre-computed datasets, avoids computing higher-order derivatives for loss functions that are memory-intensive and unstable, and demonstrates zero-shot generalization to novel PDE parameters and domains. Experiments show that for the same number of training steps, our method exhibits up to 8.75

\times

improvement in

L_2

-error compared to standard physics-informed training schemes, up to 6.31

\times

improvement in training speed, and reductions of up to 2.97

\times

in GPU memory consumption. We present the code at https://github.com/neuraloperator/WoS-NO

Hrishikesh Viswanath, Hong Chul Nam, Xi Deng +3

Purdue University, USA · ETH Zurich, Switzerland · California Institute of Technology, USA +1

PDE Operator Learning Physics-Informed ML Neural Operators Weakly Supervised Learning

Mar 1, 2026

A level-wise training scheme for learning neural multigrid smoothers with application to integral equations

Convolution-type integral equations commonly occur in signal processing and image processing. Discretizing these equations yields large and ill-conditioned linear systems. While the classic multigrid method is effective for solving linear systems derived from partial differential equations (PDE) problems, it fails to solve integral equations because its smoothers, which are implemented as conventional relaxation methods, are ineffective in reducing high-frequency components in the errors. We propose a novel neural multigrid scheme where learned neural operators replace classical smoothers. Unlike classical smoothers, these operators are trained offline. Once trained, the neural smoothers generalize to new right-hand-side vectors without retraining, making it an efficient solver. We design level-wise loss functions incorporating spectral filtering to emulate the multigrid frequency decomposition principle, ensuring each operator focuses on solving distinct high-frequency spectral bands. Although we focus on integral equations, the framework is generalizable to all kinds of problems, including PDE problems. Our experiments demonstrate superior efficiency over classical solvers and robust convergence across varying problem sizes and regularization weights.

Lingfeng Li, Yin King Chu, Raymond Chan +1

Hetao Institute of Mathematics and Interdisciplinary Sciences, Shenzhen, Guangdong, China. · Department of Mathematics, Hong Kong University of Science and Technology, Clear Water Bay, Hong Kong, China. · Department of Data Science, Department of Operations and Risk Management,Lingnan University, Tuen Mun, Hong Kong, China. +1

Neural Operators Neural PDE Solvers

Mar 1, 2026

Random Features for Operator-Valued Kernels: Bridging Kernel Methods and Neural Operators

In this work, we investigate the generalization properties of random feature methods. Our analysis extends prior results for Tikhonov regularization to a broad class of spectral regularization techniques and further generalizes the setting to operator-valued kernels. This unified framework enables a rigorous theoretical analysis of neural operators and neural networks through the lens of the Neural Tangent Kernel (NTK). In particular, it allows us to establish optimal learning rates and provides a good understanding of how many neurons are required to achieve a given accuracy. Furthermore, we establish minimax rates in the well-specified case and also in the misspecified case, where the target is not contained in the reproducing kernel Hilbert space. These results sharpen and complete earlier findings for specific kernel algorithms.

Mike Nguyen, Nicole Mücke

Technical University of Braunschweig

Random Feature Methods Kernel Methods Neural Network Generalization Neural Operators Spectral Regularization

Feb 27, 2026

A Boundary Integral-based Neural Operator for Mesh Deformation

This paper presents an efficient mesh deformation method based on boundary integration and neural operators, formulating the problem as a linear elasticity boundary value problem (BVP). To overcome the high computational cost of traditional finite element methods and the limitations of existing neural operators in handling Dirichlet boundary conditions for vector fields, we introduce a direct boundary integral representation using a Dirichlet-type Green's tensor. This formulation expresses the internal displacement field solely as a function of boundary displacements, eliminating the need to solve for unknown tractions. Building on this, we design a Boundary-Integral-based Neural Operator (BINO) that learns the geometry- and material-aware Green's traction kernel. A key technical advantage of our framework is the mathematical decoupling of the physical integration process from the geometric representation via geometric descriptors. While this study primarily demonstrates robust generalization across diverse boundary conditions, the architecture inherently possesses potential for cross-geometry adaptation. Numerical experiments, including large deformations of flexible beams and rigid-body motions of NACA airfoils, confirm the model's high accuracy and strict adherence to the principles of linearity and superposition. The results demonstrate that the proposed framework ensures mesh quality and computational efficiency, providing a reliable new paradigm for parametric mesh generation and shape optimization in engineering.

Zhengyu Wu, Jun Liu, Wei Wang

School of Electronics and Information Engineering, Hangzhou Dianzi University, Hangzhou, China

Neural Operators PDE Operator Learning PDE Solving Deformable Object Simulation

Feb 26, 2026

Neural Operators Can Discover Functional Clusters

Operator learning is reshaping scientific computing by amortizing inference across infinite families of problems. While neural operators (NOs) are increasingly well understood for regression, far less is known for classification and its unsupervised analogue: clustering. We prove that sample-based neural operators can learn any finite collection of classes in an infinite-dimensional reproducing kernel Hilbert space, even when the classes are neither convex nor connected, under mild kernel sampling assumptions. Our universal clustering theorem shows that any

K

closed classes can be approximated to arbitrary precision by NO-parameterized classes in the upper Kuratowski topology on closed sets, a notion that can be interpreted as disallowing false-positive misclassifications. Building on this, we develop an NO-powered clustering pipeline for functional data and apply it to unlabeled families of ordinary differential equation (ODE) trajectories. Discretized trajectories are lifted by a fixed pre-trained encoder into a continuous feature map and mapped to soft assignments by a lightweight trainable head. Experiments on diverse synthetic ODE benchmarks show that the resulting practical SNO recovers latent dynamical structure in regimes where classical methods fail, providing evidence consistent with our universal clustering theory.

Yicen Li, Jose Antonio Lara Benitez, Ruiyang Hong +3

McMaster University and the Vector Institute, Main St., Hamilton, L8S 4L8, Ontario, Canada · Rice University, Main St., Houston, 77005, Texas, United States

Clustering Unsupervised Clustering Neural Operators Scientific ML

Feb 25, 2026

NESTOR: A Nested MOE-based Neural Operator for Large-Scale PDE Pre-Training

Neural operators have emerged as an efficient paradigm for solving PDEs, overcoming the limitations of traditional numerical methods and significantly improving computational efficiency. However, due to the diversity and complexity of PDE systems, existing neural operators typically rely on a single network architecture, which limits their capacity to fully capture heterogeneous features and complex system dependencies. This constraint poses a bottleneck for large-scale PDE pre-training based on neural operators. To address these challenges, we propose a large-scale PDE pre-trained neural operator based on a nested Mixture-of-Experts (MoE) framework. In particular, the image-level MoE is designed to capture global dependencies, while the token-level Sub-MoE focuses on local dependencies. Our model can selectively activate the most suitable expert networks for a given input, thereby enhancing generalization and transferability. We conduct large-scale pre-training on twelve PDE datasets from diverse sources and successfully transfer the model to downstream tasks. Extensive experiments demonstrate the effectiveness of our approach.

Dengdi Sun, Xiaoya Zhou, Xiao Wang +4

School of Artificial Intelligence, Anhui University, Hefei, China · School of Computer Science and Technology, Anhui University, Hefei, China

Neural Operators PDE Surrogate Modeling Mixture-of-Experts Models

Feb 25, 2026

The Error of Deep Operator Networks Is the Sum of Its Parts: Branch-Trunk and Mode Error Decompositions

Operator learning has the potential to strongly impact scientific computing by learning solution operators for differential equations, potentially accelerating multi-query tasks such as design optimization and uncertainty quantification by orders of magnitude. Despite proven universal approximation properties, deep operator networks (DeepONets) often exhibit limited accuracy and generalization in practice, which hinders their adoption. Understanding these limitations is therefore crucial for further advancing the approach. This work analyzes performance limitations of the classical DeepONet architecture. It is shown that the approximation error is dominated by the branch network when the internal dimension is sufficiently large, and that the learned trunk basis can often be replaced by classical basis functions without a significant impact on performance. To investigate this further, a modified DeepONet is constructed in which the trunk network is replaced by the left singular vectors of the training solution matrix. This modification yields several key insights. First, for examples involving the KdV and Burgers equations, a spectral bias in the branch network is observed, with coefficients of dominant, low-frequency modes learned more effectively. Second, through the interplay of the singular-value weighting and the optimizer's neglect of small modes, the branch error is dominated by modes with large and intermediate singular values. Third, using a shared branch network for all mode coefficients, as in the standard architecture, improves generalization of small modes compared to a stacked architecture in which coefficients are computed separately. Finally, detrimental coupling between modes in parameter space is identified, which weakens with increasing network width.

Alexander Heinlein, Johannes Taraz

Delft Institute of Applied Mathematics, Delft University of Technology, Delft, 2628CD, The Netherlands

Deep Operator Networks Neural Operators Neural Network Generalization Neural Network Training Dynamics

Feb 25, 2026

From Basis to Basis: Gaussian Particle Representation for Interpretable PDE Operators

Learning PDE dynamics for fluids increasingly relies on neural operators and Transformer-based models, yet these approaches often lack interpretability and struggle with localized, high-frequency structures while incurring quadratic cost in spatial samples. We propose representing fields with a Gaussian basis, where learned atoms carry explicit geometry (centers, anisotropic scales, weights) and form a compact, mesh-agnostic, directly visualizable state. Building on this representation, we introduce a Gaussian Particle Operator that acts in modal space: learned Gaussian modal windows perform a Petrov-Galerkin measurement, and PG Gaussian Attention enables global cross-scale coupling. This basis-to-basis design is resolution-agnostic and achieves near-linear complexity in N for a fixed modal budget, supporting irregular geometries and seamless 2D-to-3D extension. On standard PDE benchmarks and real datasets, our method attains state-of-the-art competitive accuracy while providing intrinsic interpretability.

Zhihao Li, Yu Feng, Zhilu Lai +1

The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · The Hong Kong University of Science and Technology, Hong Kong SAR, China

PDE Operator Learning Neural Operators Computational Fluid Dynamics

Feb 24, 2026

Structure-Aware Epistemic Uncertainty Quantification for Neural Operator PDE Surrogates

Neural operators (NOs) provide fast, resolution-invariant surrogates for mapping input fields to PDE solution fields, but their predictions can exhibit significant epistemic uncertainty due to finite data, imperfect optimization, and distribution shift. For practical deployment in scientific computing, uncertainty quantification (UQ) must be both computationally efficient and spatially faithful, i.e., uncertainty bands should align with the localized residual structures that matter for downstream risk management. We propose a structure-aware epistemic UQ scheme that exploits the modular anatomy common to modern NOs (lifting-propagation-recovering). Instead of applying unstructured weight perturbations (e.g., naive dropout) across the entire network, we restrict Monte Carlo sampling to a module-aligned subspace by injecting stochasticity only into the lifting module, and treat the learned solver dynamics (propagation and recovery) as deterministic. We instantiate this principle with two lightweight lifting-level perturbations, including channel-wise multiplicative feature dropout and a Gaussian feature perturbation with matched variance, followed by standard calibration to construct uncertainty bands. Experiments on challenging PDE benchmarks (including discontinuous-coefficient Darcy flow and geometry-shifted 3D car CFD surrogates) demonstrate that the proposed structure-aware design yields more reliable coverage, tighter bands, and improved residual-uncertainty alignment compared with common baselines, while remaining practical in runtime.

Haoze Song, Zhihao Li, Mengyi Deng +4

The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · The Hong Kong University of Science and Technology, Hong Kong SAR, China

Uncertainty Quantification Epistemic Uncertainty Neural Operators PDE Surrogate Modeling Uncertainty Calibration

Feb 21, 2026

SGNO: Spectral Generator Neural Operators for Stable Long Horizon PDE Rollouts

Autoregressive neural PDE surrogates predict future states by repeatedly applying a learned one-step operator. This is a simple and widely used method, but small one-step errors can accumulate during long rollouts. The resulting drift often appears as spectral amplitude distortion, phase misalignment, and nonlinear mode-interaction error. These effects are especially important for time-dependent PDEs with clear Fourier structure. We introduce the Spectral Generator Neural Operator (SGNO), a structured autoregressive neural operator for long-horizon PDE forecasting. SGNO organizes each learned one-step map as a structured spectral evolution update. A real-valued nonpositive diagonal generator provides a gain-controlled spectral backbone, while a learned correction pathway with complex-valued spectral mixing completes the residual evolution. This design gives the autoregressive step an evolution-like structure while retaining the flexibility needed for dissipative, dispersive, transport-dominated, and nonlinear PDEs. SGNO is designed for periodic linear and semilinear evolution PDEs with Fourier multiplier linear dynamics. Across ten mechanism-matched APEBench tasks spanning this regime, SGNO consistently outperforms strong single-step autoregressive baselines in long-horizon rollout accuracy, reducing GMean100 by a median of 74.8% relative to the strongest available non-SGNO baseline, with per-task reductions ranging from 13.6% to 92.9%. The gains are strongest on dispersive and transport-dominated tasks, as well as tasks involving nonlinear closure and mode coupling. Spectral diagnostics show lower spectral energy error and improved rollout-level phase fidelity. Ablations show that the constrained generator, the structured update, and the learned correction pathway each contribute to performance. The code is available at https://github.com/cruiseresearchgroup/SGNO.

Jiayi Li, Penghao Jiang, Hira Saleem +3

PDE Surrogate Modeling Neural Operators Spatiotemporal Forecasting

Feb 18, 2026

Geometric Neural Operators via Lie Group-Constrained Latent Dynamics

Neural operators offer an effective framework for learning solutions of partial differential equations for many physical systems in a resolution-invariant and data-driven manner. Existing neural operators, however, often suffer from instability in multi-layer iteration and long-horizon rollout, which stems from the unconstrained Euclidean latent space updates that violate the geometric and conservation laws. To address this challenge, we propose to constrain manifolds with low-rank Lie algebra parameterization that performs group action updates on the latent representation. Our method, termed Manifold Constraining based on Lie group (MCL), acts as an efficient plug-and-play module that enforces geometric inductive bias to existing neural operators. Extensive experiments on various partial differential equations, such as 1-D Burgers and 2-D Navier-Stokes, over a wide range of parameters and steps demonstrate that our method effectively lowers the relative prediction error by 30-50% at the cost of 2.26% of parameter increase. The results show that our approach provides a scalable solution for improving long-term prediction fidelity by addressing the principled geometric constraints absent in the neural operator updates.

Jiaquan Zhang, Fachrina Dewi Puspitasari, Songbo Zhang +7

School of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu, China · Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, China · Institute of Software Chinese Academy of Sciences, Beijing, China +2

Neural PDE Solvers PDE Operator Learning Neural Operators Lie Group Methods

Feb 17, 2026

Flowers: A Warp Drive for Neural PDE Solvers

We introduce Flowers, a neural architecture for learning PDE solution operators built entirely from multihead warps. Aside from pointwise channel mixing and a multiscale scaffold, Flowers use no Fourier multipliers, no dot-product attention, and no convolutional mixing. Each head predicts a displacement field and warps the mixed input features. Motivated by physics and computational efficiency, displacements are predicted pointwise, without any spatial aggregation, and nonlocality enters only through sparse sampling at source coordinates, one per head. Stacking warps in multiscale residual blocks yields Flowers, which implement adaptive, global interactions at linear cost. We theoretically motivate this design through three complementary lenses: flow maps for conservation laws, waves in inhomogeneous media, and a kinetic-theoretic continuum limit. Flowers achieve excellent performance on a broad suite of 2D and 3D time-dependent PDE benchmarks, particularly flows and waves. A compact 17M-parameter model consistently outperforms Fourier, convolution, and attention-based baselines of similar size, while a 150M-parameter variant improves over recent transformer-based foundation models with much more parameters, data, and training compute.

Till Muser, Alexandra Spitzer, Matti Lassas +2

Department of Mathematics and Computer Science, University of Basel, Basel, Switzerland · Department of Mathematics and Statistics, University of Helsinki, Helsinki, Finland · Simons Chair in Computational and Applied Mathematics and Earth Science, Rice University, Houston, TX, USA

Neural PDE Solvers PDE Surrogate Modeling PDE Operator Learning Neural Operators

Feb 17, 2026

Neural-POD: A Plug-and-Play Neural Operator Framework for Infinite-Dimensional Functional Nonlinear Proper Orthogonal Decomposition

AI for science (AI4Science) models often suffer from discretization: learned representations remain tied to the training grid, limiting transfer across resolutions, solvers and applications. We introduce Neural Proper Orthogonal Decomposition (Neural-POD), a plug-and-play neural operator that learns nonlinear, orthogonal basis functions directly in function space and can be integrated in both projection-based reduced order models and operator-learning frameworks such as DeepONet. Neural-POD replaces SVD-derived, resolution-dependent linear modes with continuous, resolution-invariant bases learned via sequential residual minimization, analogous to Gram-Schmidt orthogonalization. The framework supports training under task-specific norms (e.g.,

L^2

,

L^1

), improves out-of-distribution generalization to unseen parameter regimes, and captures nonlinear structure in complex systems. Because the learned bases are interpretable and reusable, Neural-POD serves as a general representation module for AI4Science workflows. We demonstrate Neural-POD on Burgers' and Navier-Stokes equations.

Changhong Mou, Binghang Lu, Guang Lin

Department of Mathematics and Statistics, Utah State University, 3900 Old Main Hill, Logan, UT, 84322, USA. · School of Electrical and Computer Engineering, Purdue University, 610 Purdue Mall, West Lafayette, IN, 47907, USA. · Department of Mathematics, Purdue University, 610 Purdue Mall, West Lafayette, IN, 47907, USA. +1

Proper Orthogonal Decomposition Orthogonal Representation Learning Reduced-Order Modeling Neural Operators

Feb 16, 2026

Learning Data-Efficient and Generalizable Neural Operators via Fundamental Physics Knowledge

Recent advances in scientific machine learning (SciML) have enabled neural operators (NOs) to serve as powerful surrogates for modeling the dynamic evolution of physical systems governed by partial differential equations (PDEs). While existing approaches focus primarily on learning simulations from the target PDE, they often overlook more fundamental physical principles underlying these equations. Inspired by how numerical solvers are compatible with simulations of different settings of PDEs, we propose a multiphysics training framework that jointly learns from both the original PDEs and their simplified basic forms. Our framework enhances data efficiency, reduces predictive errors, and improves out-of-distribution (OOD) generalization, particularly in scenarios involving shifts of physical parameters and synthetic-to-real transfer. Our method is architecture-agnostic and demonstrates consistent improvements in normalized root mean square error (nRMSE) across a wide range of 1D/2D/3D PDE problems. Through extensive experiments, we show that explicit incorporation of fundamental physics knowledge significantly strengthens the generalization ability of neural operators. We will release models and codes at https://sites.google.com/view/sciml-fundemental-pde.

Siying Ma, Mehrdad M. Zadeh, Mauricio Soroco +3

Simon Fraser University · Georgia Institute of Technology

PDE Surrogate Modeling Neural Operators Physics-Informed Neural Operators Data-Efficient Learning OOD Generalization

Feb 4, 2026

Transolver-3: Scaling Up Transformer Solvers to Industrial-Scale Geometries

Deep learning has emerged as a transformative tool for the neural surrogate modeling of partial differential equations (PDEs), known as neural PDE solvers. However, scaling these solvers to industrial-scale geometries with over

10^8

cells remains a fundamental challenge due to the prohibitive memory complexity of processing high-resolution meshes. We present Transolver-3, a new member of the Transolver family as a highly scalable framework designed for high-fidelity physics simulations. To bridge the gap between limited GPU capacity and the resolution requirements of complex engineering tasks, we introduce two key architectural optimizations: faster slice and deslice by exploiting matrix multiplication associative property and geometry slice tiling to partition the computation of physical states. Combined with an amortized training strategy by learning on random subsets of original high-resolution meshes and a physical state caching technique during inference, Transolver-3 enables high-fidelity field prediction on industrial-scale meshes. Extensive experiments demonstrate that Transolver-3 can handle meshes with over 160 million cells, achieving impressive performance across three challenging simulation benchmarks, including aircraft and automotive design tasks. Code is available at https://github.com/thuml/Transolver-3.

Hang Zhou, Haixu Wu, Haonan Shangguan +4

School of Software, BNRist, Tsinghua University, China.

Efficient Neural Network Inference Neural PDE Solvers Transformer Neural Operator PDE Surrogate Modeling PDE Operator Learning Neural Operators

Jan 23, 2026

SFO: Learning PDE Operators via Spectral Filtering

Partial differential equations (PDEs) govern complex systems, yet neural operators often struggle to efficiently capture the long-range, nonlocal interactions inherent in their solution maps. We introduce Spectral Filtering Operator (SFO), a neural operator that parameterizes integral kernels using the Universal Spectral Basis (USB), a fixed, global orthonormal basis derived from the eigenmodes of the Hilbert matrix in spectral filtering theory. Motivated by our theoretical finding that the discrete Green's functions of shift-invariant PDE discretizations exhibit spatial Linear Dynamical System (LDS) structure, we prove that these kernels admit compact approximations in the USB. By learning only the spectral coefficients of rapidly decaying eigenvalues, SFO achieves a highly efficient representation. Across six benchmarks, including reaction-diffusion, fluid dynamics, and 3D electromagnetics, SFO achieves state-of-the-art accuracy, reducing error by up to 40% relative to strong baselines while using substantially fewer parameters.

Noam Koren, Rafael Moschopoulos, Kira Radinsky +1

Department of Computer Science Technion - Israel Institute of Technology · Department of Computer Science Princeton University

PDE Surrogate Modeling PDE Operator Learning Spectral Methods Neural Operators