Neural Operators

Ask about this topic

Overview

Neural operators learn maps from input fields, parameters, or geometries to output fields, often approximating the solution map of a partial differential equation. DeepONet uses branch and trunk networks, Fourier Neural Operators learn interactions in spectral space, and graph and transformer operators handle irregular meshes and complex geometries; physics-informed losses and structure-preserving designs add physical constraints. These models can make repeated simulation, forecasting, and inverse design much faster than running a numerical solver for every case. A central challenge is making predictions reliable across new resolutions, geometries, and physical regimes while preserving stability over long rollouts and matching classical solvers when high accuracy is required.

Latest papers

CardsList

313 papers52 papers in the last four weeks, up 24% on the four weeks before. 0.3% of all new papers.

Jan 21, 2026

Learning and extrapolating scale-invariant processes

Machine Learning (ML) has deeply changed some fields recently, like Language and Vision and we may expect it to be relevant also to the analysis of of complex systems. Here we want to tackle the question of how and to which extent can one regress scale-free processes, i.e. processes displaying power law behavior, like earthquakes or avalanches? We are interested in predicting the large ones, i.e. rare events in the training set which therefore require extrapolation capabilities of the model. For this we consider two paradigmatic problems that are statistically self-similar. The first one is a 2-dimensional fractional Gaussian field obeying linear dynamics, self-similar by construction and amenable to exact analysis. The second one is the Abelian sandpile model, exhibiting self-organized criticality. The emerging paradigm of Geometric Deep Learning shows that including known symmetries into the model's architecture is key to success. Here one may hope to extrapolate only by leveraging scale invariance. This is however a peculiar symmetry, as it involves possibly non-trivial coarse-graining operations and anomalous scaling. We perform experiments on various existing architectures like U-net, Riesz network (scale invariant by construction), or our own proposals: a wavelet-decomposition based Graph Neural Network (with discrete scale symmetry), a Fourier embedding layer and a Fourier-Mellin Neural Operator. Based on these experiments and a complete characterization of the linear case, we identify the main issues relative to spectral biases and coarse-grained representations, and discuss how to alleviate them with the relevant inductive biases.

Anaclara Alvez-Canepa, Cyril Furtlehner, François P. Landes

Laboratoire Interdisciplinaire des Sciences du Numérique (LISN) · Université Paris-Saclay and · INRIA-Saclay

Invariant Representation LearningNeural Operators
AskPDF
Dec 29, 2025

PGOT: A Physics-Geometry Operator Transformer for Complex PDEs

While Transformers have demonstrated remarkable potential in modeling Partial Differential Equations (PDEs), modeling large-scale unstructured meshes with complex geometries remains a significant challenge. Existing efficient architectures often employ feature dimensionality reduction strategies, which inadvertently induces Geometric Aliasing, resulting in the loss of critical physical boundary information. To address this, we propose the Physics-Geometry Operator Transformer (PGOT), designed to reconstruct physical feature learning through explicit geometry awareness. Specifically, we propose Spectrum-Preserving Geometric Attention (SpecGeo-Attention). Utilizing a ``physics slicing-geometry injection" mechanism, this module incorporates multi-scale geometric encodings to explicitly preserve multi-scale geometric features while maintaining linear computational complexity O(N)O(N). Furthermore, PGOT dynamically routes computations to low-order linear paths for smooth regions and high-order non-linear paths for shock waves and discontinuities based on spatial coordinates, enabling spatially adaptive and high-precision physical field modeling. PGOT achieves consistent state-of-the-art performance across four standard benchmarks and excels in large-scale industrial tasks including airfoil and car designs.

Zhuo Zhang, Xi Yang, Ying Miao +5

National University of Defense Technology · Peking University · National University of Singapore +2

Neural PDE SolversPDE Surrogate ModelingGeometric Representation LearningPDE Operator LearningNeural Operators
AskPDF
Dec 23, 2025

GeoTransolver: Learning Physics on Irregular Domains Using Multi-scale Geometry Aware Physics Attention Transformer

We present GeoTransolver, a multiscale geometry-aware physics attention transformer for Computer Aided Engineering (CAE). GeoTransolver extends the Transolver backbone with GALE (Geometry-Aware Latent Embeddings) attention, which pairs physics-aware self-attention on learned state slices with cross-attention to a shared geometry and global context computed via multi-scale ball queries (inspired by Domino) and reused in every block. Implemented and released in NVIDIA PhysicsNeMo, GeoTransolver persistently projects geometry and global parameters, into physical state spaces to anchor computations to domain structure and operating regimes. We benchmark on DrivAerML, SHIFT-SUV, and SHIFT-Wing against Domino, Transolver (PhysicsNeMo implementation), and literature-reported AB-UPT, evaluating drag/lift R2 and relative L1 errors on field variables. As an additional nonlinear structural mechanics application, we also report Transolver and GeoTransolver results on bumper-beam and full-vehicle Body-in-White (BIW) crash-dynamics benchmarks, evaluating relative L2 trajectory error and probe-level kinematic MSE. GeoTransolver delivers improved accuracy, robustness to geometry and regime shifts, and favorable data efficiency; we include DrivAerML ablations and qualitative contour and design-trend results, advancing operator learning for high-fidelity surrogates on complex, irregular, non-linear domains.

Corey Adams, Rishikesh Ranade, Ram Cherukuri +1

NVIDIA Corporation, Santa Clara, US

Neural Surrogate ModelingStructural MechanicsTransformer AttentionNeural OperatorsComputational Fluid Dynamics
AskPDF
Dec 4, 2025

Efficient Generative Transformer Operators For Million-Point PDEs

We introduce ECHO, a transformer-operator framework for generating million-point PDE trajectories. While existing neural operators (NOs) have shown promise for solving partial differential equations, they remain limited in practice due to poor scalability on dense grids, error accumulation during dynamic unrolling, and task-specific design. ECHO addresses these challenges through three key innovations. (i) It employs a hierarchical convolutional encode-decode architecture that achieves a 100 ×\times spatio-temporal compression while preserving fidelity on mesh points. (ii) It incorporates a training and adaptation strategy that enables high-resolution PDE solution generation from sparse input grids. (iii) It adopts a generative modeling paradigm that learns complete trajectory segments, mitigating long-horizon error drift. The training strategy decouples representation learning from downstream task supervision, allowing the model to tackle multiple tasks such as trajectory generation, forward and inverse problems, and interpolation. The generative model further supports both conditional and unconditional generation. We demonstrate state-of-the-art performance on million-point simulations across diverse PDE systems featuring complex geometries, high-frequency dynamics, and long-term horizons.

Armand Kassaï Koupaï, Lise Le Boudec, Patrick Gallinari

Sorbonne Université, CNRS, ISIR, 75005 Paris, France · Criteo AI Lab, Paris, France

PDE Surrogate ModelingTransformer Neural OperatorNeural OperatorsPDE Inverse Problems
AskPDF
Nov 25, 2025

CarBench: A Comprehensive Benchmark for Neural Surrogates on High-Fidelity 3D Car Aerodynamics

Benchmarking has been the cornerstone of progress in computer vision, natural language processing, and the broader deep learning domain, driving algorithmic innovation through standardized datasets and reproducible evaluation protocols. The growing availability of large-scale Computational Fluid Dynamics (CFD) datasets has opened new opportunities for applying machine learning to aerodynamic and engineering design. Yet, despite this progress, there exists no standardized benchmark for large-scale numerical simulations in engineering design. In this work, we introduce CarBench, the first comprehensive benchmark dedicated to large-scale 3D car aerodynamics, performing a large-scale evaluation of state-of-the-art models on DrivAerNet++, the largest public dataset for automotive aerodynamics, containing over 8,000 high-fidelity car simulations. We assess eleven architectures spanning neural operator methods (e.g., Fourier Neural Operator), geometric deep learning (PointNet, RegDGCNN, PointMAE, PointTransformer), transformer-based neural solvers (Transolver, Transolver++, AB-UPT), and implicit field networks (TripNet). Beyond standard interpolation tasks, we perform cross-category experiments in which transformer-based solvers trained on a single car archetype are evaluated on unseen categories. Our analysis covers predictive accuracy, physical consistency, computational efficiency, and statistical uncertainty. To accelerate progress in data-driven engineering, we open-source the benchmark framework, including training pipelines, uncertainty estimation routines based on bootstrap resampling, and pretrained model weights, establishing the first reproducible foundation for large-scale learning from high-fidelity CFD simulations, available at https://github.com/Mohamedelrefaie/CarBench.

Mohamed Elrefaie, Dule Shu, Matt Klenk +1

Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA, USA · Schwarzman College of Computing, Massachusetts Institute of Technology, Cambridge, MA, USA · Future Product Innovation, Toyota Research Institute, Los Altos, CA, USA

Automotive AerodynamicsNeural Surrogate ModelingBenchmark DesignComputational Fluid DynamicsNeural Operators
AskPDF
Oct 21, 2025

Solver-Integrated Adversarial Attacking and Training of Neural Operators

Neural operators are widely used as fast surrogates for numerical PDE solvers, mapping input functions to solution functions. However, their generalizability and robustness are not yet clearly defined in the operator-learning setting, which differs from traditional adversarial robustness definitions. This paper studies the generalizability and robustness of a learned neural operator from a solver-integrated perspective, addressing the challenge that the output of a learned operator and a numerical solver tends to change in tandem under input perturbation. First, we formalize the definition of generalization and robustness through a model-solver error operator, identifying fixed-input model-solver loss as generalization metric, and norm-bounded adversarial attack loss increase and Jacobian-error function norm as robustness metric. Second, we identify the solver-integrated adversarial attack as appropriate for PDE operator learning and show why model-only or fixed-ground-truth attacks can be insufficient when the solver output also changes with the input. Third, we develop solver-integrated adversarial training methods for neural operators. Experiments on representative PDE benchmarks show that this solver-integrated adversarial training clearly improves both generalizability and robustness. Deeper solver integration yields more effective attacks, more informative samples, and more efficient training than less integrated alternatives. These results provide a general framework for robust operator training and automatic sample selection without heavy manual intervention. More broadly, the formulation applies to adversarial regression whenever a ground-truth oracle can evaluate, and ideally differentiate, the true input-output map; PDE operator learning is one such case.

Yifei Sun

Adversarial TrainingAdversarial AttacksNeural Network RobustnessAdversarial RobustnessPDE Operator LearningNeural Operators
AskPDF
Aug 20, 2025

Neural Operators Which Certifiably Simultaneously Solve Infinitely Many Convex Programs using Logarithmic Rank and Linear Complexity

Neural operators (NOs) are a class of deep learning models designed to simultaneously solve infinitely many related problems by casting them into an infinite-dimensional space, whereon these NOs operate. Such operator learning problems arise naturally in mathematical finance, for instance, when learning families of optimal portfolios or hedging strategies. Two significant gaps, however, remain between theory and practice: 1) most universal approximation guarantees do not describe practically implementable NOs or are pure existence statements; and 2) worst-case parameter bounds from universal approximation theorems suggest that constructed NOs may require an unrealistically large number of parameters to solve most operator learning problems, which stands in direct opposition to a slew of experimental evidence. This paper closes these gaps by exhibiting a short and simple training algorithm to construct a specific class of NOs, proximal equilibrium operators (PEOs), using (realistic) finite-dimensional deep equilibrium layers to solve families of convex optimization problems over a separable Hilbert space XX. Here, the inputs are smooth, convex loss functions on XX, and the outputs are the associated (approximate) solutions to the optimization problems defined by each input loss. Our main result is a certificate showing that, when the input losses lie in a suitable infinite-dimensional set, our PEOs trained with our algorithm do in fact uniformly approximate the corresponding solutions to arbitrary precision, with rank and width growing only logarithmically and the depth growing linearly in the reciprocal of the approximation error. We then validate both our theoretical results and the trainability of PEOs on three applications: non-linear PDEs, stochastic optimal control, and portfolio optimization and hedging problems in mathematical finance.

Anastasis Kratsios, Ariel Neufeld, Philipp Schmocker

Department of Mathematics, McMaster University and the Vector Institute, Canada · Division of Mathematical Sciences, Nanyang Technological University, Singapore

Neural OperatorsOperator LearningNeural Network Approximation TheoryProximal Optimization
AskPDF
Jul 29, 2025

PVD-ONet: A Multi-scale Neural Operator Method for Singularly Perturbed Boundary Layer Problems

Physics-informed neural networks and physics-informed DeepONet excel at solving partial differential equations but often struggle with singularly perturbed problems. To address this limitation, we propose two novel data-free frameworks: the Prandtl--Van Dyke neural network(PVD-Net) and its operator-learning extension, the Prandtl--Van Dyke Deep Operator Network(PVD-ONet). Each framework provides leading-order and high-order versions to meet different modeling requirements. The leading-order PVD-Net integrates Prandtl's matching condition for improved stability, while the high-order PVD-Net employs Van Dyke's matching principle to capture fine-scale boundary layer structures with higher accuracy. Extending PVD-Net to operator learning, PVD-ONet enables efficient prediction for families of boundary layer problems without retraining. Numerical experiments on representative one- and two-dimensional problems demonstrate that the proposed methods outperform existing baselines. Beyond forward prediction, the framework also supports inverse problems by inferring the scaling exponent governing boundary layer thickness from sparse observations, highlighting its potential for practical applications.

Tiantian Sun, Jian Zu

School of Mathematics and Statistics, Center for Mathematics and Interdisciplinary Sciences, Northeast Normal University, Changchun 130024, P.R. China

Neural OperatorsDeep Operator Networks
AskPDF
Jul 9, 2025

Discretization-independent operator learning for partial differential equations

We develop a new and general encode-approximate-reconstruct operator learning model that leverages learned neural representations of bases for input and output function distributions. We introduce the concepts of numerical operator learning and discretization independence, which clarify the relationship between theoretical formulations and practical realizations of operator learning models. Our model is discretization-independent, making it particularly effective for multiresolution learning. We establish theoretical approximation guarantees, demonstrating uniform universal approximation under strong assumptions on the input functions and statistical approximation under weaker conditions. To our knowledge, this is the first comprehensive study that investigates how discretization independence enables robust and efficient multiresolution operator learning. We validate our method through extensive numerical experiments involving both local and nonlocal PDEs, including time-independent and time-dependent problems. The results show that multiresolution training significantly improves accuracy and computational efficiency. Moreover, multiresolution training further enhances empirical discretization independence.

Jacob Hauck, Yanzhi Zhang

Department of Mathematics and Statistics, Missouri University of Science and Technology, Rolla, MO 65409

PDE Operator LearningNeural Network Approximation TheoryOperator LearningNeural Operators
AskPDF
May 24, 2025

Spatiotemporal Reconstruction of Global Radiation from Sparse Sensor Sequences

Much of the physical world is inaccessible to direct measurement and can be reached only through the sparse traces it leaves in other observable variables. Here we introduce the Temporal Radiation Operator Network (TRON), a spatiotemporal neural operator that learns proxy-to-field mappings, reconstructing otherwise unobserved physical fields from sparse, indirect sensor histories. For cosmic-radiation mapping, TRON reconstructs sea-level effective dose at 65,341 locations from histories of 12 neutron monitors, achieving 0.096% mean relative L2L_2 error against EXPACS-generated reference fields in a chronologically held-out year; batched GPU inference evaluates a global field in approximately ∼\sim1ms, compared with ∼\sim57.6s for EXPACS on one CPU thread. On measured weather networks, TRON reconstructs temperature at held-out stations from thermometer or barometer histories, outperforming interpolation, calendar-based and hypsometric baselines. These results establish proxy-to-field reconstruction as a general route to recovering spatially distributed physical information beyond the direct reach of sparse sensor networks.

Kazuma Kobayashi, Tapas Tripura, Jay Phil Yoo +5

Nuclear, Plasma and Radiological Engineering, The Grainger College of Engineering, University of Illinois Urbana-Champaign, IL, USA. · Department of Applied Mechanics, Indian Institute of Technology Delhi, New Delhi, India. · Civil and Urban Engineering Department, New York University Abu Dhabi, Abu Dhabi, UAE. +3

Neural OperatorsInverse Problems
AskPDF
Mar 7, 2025

From Theory to Application: A Practical Introduction to Neural Operators in Scientific Computing

This review examines neural operator architectures for learning solution operators of parametric partial differential equations (PDEs), with an emphasis on conceptual clarity and practical implementation. The work analyzes key models, including DeepONet, PCANet, and the Fourier Neural Operator, highlighting their underlying representations, computational structures, and comparative performance. These architectures are demonstrated on three canonical PDE problems: the Poisson equation, a linear elasticity problem, and a hyperelasticity problem. To make the presentation self-contained, key foundational topics are introduced, including finite-dimensional representations of function spaces, singular-value decomposition, and sampling from infinite-dimensional function spaces. Beyond forward modeling, the review discusses the use of neural operators as surrogate models within a Bayesian inverse-problem framework, including prior specification, forward-map approximation, and posterior computation. The performance of the three neural-operator architectures is evaluated on in-distribution samples, out-of-distribution samples, and Bayesian inference tasks. The review also discusses challenges related to prediction accuracy and generalization, outlining emerging strategies such as residual-based error correction and multi-level training. The review concludes by positioning neural operators within broader scientific-computing workflows and by identifying directions for reliable, scalable operator learning.

Prashant K. Jha

Department of Mechanical Engineering, South Dakota School of Mines and Technology, Rapid City, SD 57701, USA.

Neural Surrogate ModelingFourier Neural OperatorPDE Operator LearningDeep Operator NetworksNeural Operators
AskPDF
Feb 1, 2025

Active Learning with Bayesian Multi-Fidelity Laplace Neural Operators for Oscillatory Parametric PDEs

Surrogate models of parametric dynamical systems are essential for many-query and real-time predictions in engineering applications such as design optimization and digital twins. However, generating high-fidelity (HF) training data over a broad range of parameters and operating conditions remains computationally expensive. To address this challenge, we propose a Bayesian multi-fidelity Laplace neural operator (MF-LNO) for uncertainty-aware active learning of oscillatory parametric PDEs. Specifically, the proposed Bayesian MF-LNO iteratively calibrates the discrepancy between low- and high-fidelity data, where predictive uncertainty guides the adaptive acquisition of informative HF trajectories. Such predictive uncertainty is quantified via replica-exchange stochastic gradient Langevin dynamics (reSGLD), whose broad posterior exploration enables uncertainty to serve as an error indicator for adaptive HF sample acquisition. Numerical experiments on the Lorenz system, Duffing oscillator, and beam dynamics demonstrate that uncertainty-guided HF sample acquisition consistently outperforms random sampling, while the proposed Bayesian MF-LNO achieves higher prediction accuracy than MF-DeepONet with predictive uncertainty quantification. These results demonstrate that Bayesian multi-fidelity LNOs, combined with uncertainty-guided active learning, provide a data-efficient framework for operator learning in engineering dynamical systems.

Bongseok Kim, Haoyang Zheng, Michael Penwarden +1

School of Mechanical Engineering, Purdue University, 585 Purdue Mall, West Lafayette, 47907, IN, USA. · Department of Mathematics, Purdue University, 150 North University Street, West Lafayette, 47907, IN, USA.

Dynamical SystemsSurrogate ModelingUncertainty QuantificationActive LearningPDE Operator LearningNeural Operators
AskPDF
Oct 18, 2024

Polynomial Scaling is Possible For Neural Operator Approximations of Structured Families of BSDEs

Neural operator (NO) architectures learn nonlinear maps between infinite-dimensional function spaces and are widely used to accelerate simulation and enable data-driven model discovery. While universality results ensure expressivity, they do not address complexity: for broad operator classes described only through regularity (e.g.\ uniform continuity or CrC^r-regularity), information-theoretic lower bounds imply that minimax-optimal NO approximation rates scale exponentially in the reciprocal accuracy 1/ε1/\varepsilon. This has shifted the focus of NO theory toward identifying additional problem-specific structure, beyond regularity, under which suitably tailored NO architectures can leverage to unlock polynomial scaling in 1/ε1/\varepsilon. We exhibit the first polynomial-scaling regime for NO approximations of solution operators in stochastic analysis; by identifying structured families of non-Markovian BSDEs with randomized terminal condition parameterized by the Sobolev-regular terminal condition and by Sobolev-regular additive nonlinear perturbations of the generator. We prove that their solution operator can be approximated (uniformly over the family) by a tailored NO whose number of trainable parameters grows polynomially in 1/ε1/\varepsilon. We unlock this polynomial scaling regime by informing the NO's inductive bias by factoring out the singular part of the associated semilinear elliptic PDE Green's function and by incorporating the Doléans--Dade exponential of the BSDE's common non-Markovian factor into the NO's decoding layers. As a byproduct, we extend polynomial-scaling guarantees from families of linear elliptic PDEs on regular domains to the semilinear setting.

Takashi Furuya, Anastasis Kratsios

Doshisha University, Japan · Department of Mathematics McMaster University and Vector Institute, Canada

Stochastic Differential EquationsPDE SolvingNeural Network Approximation TheoryNeural Operators
AskPDF