Overview

Neural surrogate modeling replaces repeated calls to expensive simulators with learned approximations of physical fields, system dynamics, or quantities of interest, making tasks such as design search, control, and Bayesian inference more practical. Common approaches include neural operators for PDE solution maps, graph neural networks for meshes and networks, reduced-order latent models, and physics-informed or hybrid models that combine learned components with governing equations. Researchers also use multi-fidelity training, active learning, and probabilistic models to reduce data needs and estimate uncertainty. A central challenge is that good one-step or in-distribution accuracy does not guarantee stable long rollouts, reliable extrapolation, physical consistency, or improved downstream decisions; classical solvers remain important when stringent accuracy and verification are required.

Latest papers

472 papers44 papers in the last four weeks, down 30% on the four weeks before. 0.3% of all new papers.

Dec 30, 2025

Micro-Macro Tensor Neural Surrogates for Uncertainty Quantification in Collisional Plasma

Plasma kinetic equations exhibit pronounced sensitivity to microscopic perturbations in model parameters and data, making reliable and efficient uncertainty quantification (UQ) essential for predictive simulations. However, the cost of uncertainty sampling, the high-dimensional phase space, and multiscale stiffness pose severe challenges to both computational efficiency and error control in traditional numerical methods. These aspects are further emphasized in presence of collisions where the high-dimensional nonlocal collision integrations and conservation properties pose severe constraints. To overcome this, we present a variance-reduced Monte Carlo framework for UQ in the Vlasov--Poisson--Landau (VPL) system, in which neural network surrogates replace the multiple costly evaluations of the Landau collision term. The method couples a high-fidelity, asymptotic-preserving VPL solver with inexpensive, strongly correlated surrogates based on the Vlasov--Poisson--Fokker--Planck (VPFP) and Euler--Poisson (EP) equations. For the surrogate models, we introduce a generalization of the separable physics-informed neural network (SPINN), developing a class of tensor neural networks based on an anisotropic micro-macro decomposition, to reduce velocity-moment costs, model complexity, and the curse of dimensionality. To further increase correlation with VPL, we calibrate the VPFP model and design an asymptotic-preserving SPINN whose small- and large-Knudsen limits recover the EP and VP systems, respectively. Numerical experiments show substantial variance reduction over standard Monte Carlo, accurate statistics with far fewer high-fidelity samples, and lower wall-clock time, while maintaining robustness to stochastic dimension.
Dec 23, 2025

GeoTransolver: Learning Physics on Irregular Domains Using Multi-scale Geometry Aware Physics Attention Transformer

We present GeoTransolver, a multiscale geometry-aware physics attention transformer for Computer Aided Engineering (CAE). GeoTransolver extends the Transolver backbone with GALE (Geometry-Aware Latent Embeddings) attention, which pairs physics-aware self-attention on learned state slices with cross-attention to a shared geometry and global context computed via multi-scale ball queries (inspired by Domino) and reused in every block. Implemented and released in NVIDIA PhysicsNeMo, GeoTransolver persistently projects geometry and global parameters, into physical state spaces to anchor computations to domain structure and operating regimes. We benchmark on DrivAerML, SHIFT-SUV, and SHIFT-Wing against Domino, Transolver (PhysicsNeMo implementation), and literature-reported AB-UPT, evaluating drag/lift R2 and relative L1 errors on field variables. As an additional nonlinear structural mechanics application, we also report Transolver and GeoTransolver results on bumper-beam and full-vehicle Body-in-White (BIW) crash-dynamics benchmarks, evaluating relative L2 trajectory error and probe-level kinematic MSE. GeoTransolver delivers improved accuracy, robustness to geometry and regime shifts, and favorable data efficiency; we include DrivAerML ablations and qualitative contour and design-trend results, advancing operator learning for high-fidelity surrogates on complex, irregular, non-linear domains.
Dec 23, 2025

Optimality-Informed Neural Networks for Solving Parametric Optimization Problems

Many engineering tasks require solving families of nonlinear constrained optimization problems, parametrized in setting-specific variables. This is computationally demanding, particularly if solutions have to be computed across strongly varying parameter values, e.g., in real-time control or for model-based design. Thus, we propose to learn the mapping from parameters to the primal optimal solutions and to their corresponding duals using neural networks, giving a dense estimation in contrast to gridded approaches. Our approach, Optimality-Informed Neural Networks (OptINNs), combines (i) a KKT-residual loss that penalizes violations of the first-order optimality conditions under standard constraint qualification assumptions, and (ii) problem-specific output activations that enforce simple inequality constraints (e.g., box-type/positivity) by construction. This design reduces data requirements, allows the prediction of dual variables, and improves feasibility and closeness to optimality compared to penalty-only training. Taking quadratic penalties as a baseline, as previously proposed for this problem class in the literature, our method simplifies hyperparameter tuning and attains tighter adherence to optimality conditions. We evaluate OptINNs on different nonlinear optimization problems ranging from low to high dimensions and compare them with equally sized neural networks trained using the quadratic-penalty method. On small problems, OptINNs match a quadratic-penalty baseline in primal accuracy while additionally predicting dual variables with low error. On larger problems, OptINNs achieve lower constraint violations, lower primal error, and lower errors in the predicted costs. These results suggest that embedding feasibility and optimality into the network architecture and loss can make learning-based surrogates more accurate, feasible, and data-efficient for parametric optimization.
Dec 23, 2025

NeuralCrop: Combining physics and machine learning for improved crop modelling

Global gridded crop models (GGCMs) are crucial for projecting climate impacts on agricultural productivity and assessing associated risks to food security. However, state-of-the-art GGCMs are subject to substantial uncertainties arising from reduced representations of complex processes. Projected increases in the frequency and severity of climate extremes require more reliable agricultural yield simulations. Recent machine learning approaches trained on observational data provide alternatives to GGCMs, but remain under-constrained due to sparse observations and show poor out-of-distribution generalization. Here we introduce NeuralCrop, a differentiable hybrid GGCM that combines the strengths of an advanced process-based GGCM (LPJmL) with data-driven components. NeuralCrop is first trained to reproduce its underlying fully process-based version, before it is fine-tuned on observational flux data of physiological variables. Evaluation against held-out flux-tower observations from European wheat and US maize sites shows 32.6-45.2% reductions in mean normalized root-mean-square error for daily carbon and water fluxes. Although fine-tuned only on physiological variables, NeuralCrop improves temporal correlations with independent yield records across 71.4% of the cultivated area for European wheat and 84.0% for the US Corn Belt during 1981-2019. It performs highly competitively compared to GGCMI Phase~3 models. NeuralCrop reproduces yield anomalies more accurately across a range of soil-moisture conditions and especially for extreme drought conditions. For large-scale, long-term simulations, NeuralCrop runs nearly two orders of magnitude faster than the CPU-based LPJmL baseline model, mainly thanks to GPU acceleration. Our results highlight end-to-end hybrid crop modelling as a promising avenue for improving crop simulations and informing food security assessments under climate change.
Dec 11, 2025

Tracking large chemical reaction networks and rare events by neural networks

Chemical reaction networks are widely used to model stochastic dynamics in biophysical processes. Their exact probabilistic description via the chemical master equation becomes intractable for high-dimensional systems due to the exponential growth of the state space. This difficulty is further exacerbated when metastable transitions and rare events dominate the dynamics. Here, we present an advanced neural-network approach that effectively tackles large reaction networks and rare-event dynamics within a unified framework. Building on autoregressive representations of the full joint probability distribution, we achieve efficient training through second-order optimization methods, including natural gradient descent and the time-dependent variational principle. We leverage enhanced-sampling strategies that enable accurate resolution of low-probability regions governing rare transitions. We demonstrate the method on a multistable genetic toggle switch and a mitogen-activated protein kinase (MAPK) cascade consisting of 16 species and 35 reactions with dense coupling. We further quantify rare events in spatially extended Schlögl models in both one- and two-dimensional lattices, reaching system sizes with extremely large state spaces up to 3×10153\times10^{15}. These results highlight the potential of neural modeling as a flexible and scalable framework for studying high-dimensional chemical reaction networks and rare events.
Dec 1, 2025

Fourier Neural Operators Explained: A Practical Perspective

Partial differential equations (PDEs) govern a wide variety of dynamical processes in science and engineering, yet obtaining their numerical solutions often requires high-resolution discretizations and repeated evaluations of complex operators, leading to substantial computational costs. Neural operators have recently emerged as a powerful framework for learning mappings between function spaces directly from data, enabling efficient surrogate models for PDE systems. Among these architectures, the Fourier Neural Operator (FNO) has become the most influential and widely adopted due to its elegant spectral formulation, which captures global correlations through learnable transformations in Fourier space while remaining invariant to discretization and resolution. Despite their success, the practical use of FNOs is often hindered by an incomplete understanding among practitioners of their theoretical foundations, practical constraints, and implementation details, which can lead to their incorrect or unreliable application. This work presents a comprehensive and practice-oriented guide to FNOs, unifying their mathematical principles with implementation strategies. We provide an intuitive exposition of the concepts of operator theory and signal processing that underlie the FNO, detail its spectral parameterization and the computational design of all its components, and address common misunderstandings encountered in the literature. The exposition is closely integrated with the NeuralOperator 2.0.0 library, offering modular state-of-the-art implementations that faithfully reflect the theory. By connecting rigorous foundations with practical insight, this guide aims to establish a clear and reliable framework for applying FNOs effectively across diverse scientific and engineering fields.
Nov 25, 2025

CarBench: A Comprehensive Benchmark for Neural Surrogates on High-Fidelity 3D Car Aerodynamics

Benchmarking has been the cornerstone of progress in computer vision, natural language processing, and the broader deep learning domain, driving algorithmic innovation through standardized datasets and reproducible evaluation protocols. The growing availability of large-scale Computational Fluid Dynamics (CFD) datasets has opened new opportunities for applying machine learning to aerodynamic and engineering design. Yet, despite this progress, there exists no standardized benchmark for large-scale numerical simulations in engineering design. In this work, we introduce CarBench, the first comprehensive benchmark dedicated to large-scale 3D car aerodynamics, performing a large-scale evaluation of state-of-the-art models on DrivAerNet++, the largest public dataset for automotive aerodynamics, containing over 8,000 high-fidelity car simulations. We assess eleven architectures spanning neural operator methods (e.g., Fourier Neural Operator), geometric deep learning (PointNet, RegDGCNN, PointMAE, PointTransformer), transformer-based neural solvers (Transolver, Transolver++, AB-UPT), and implicit field networks (TripNet). Beyond standard interpolation tasks, we perform cross-category experiments in which transformer-based solvers trained on a single car archetype are evaluated on unseen categories. Our analysis covers predictive accuracy, physical consistency, computational efficiency, and statistical uncertainty. To accelerate progress in data-driven engineering, we open-source the benchmark framework, including training pipelines, uncertainty estimation routines based on bootstrap resampling, and pretrained model weights, establishing the first reproducible foundation for large-scale learning from high-fidelity CFD simulations, available at https://github.com/Mohamedelrefaie/CarBench.
Nov 22, 2025

Diffusion-based Surrogate Model for Time-varying Underwater Acoustic Channels

Accurate modeling of time-varying underwater acoustic channels is essential for the design, evaluation, and deployment of reliable underwater communication systems. Conventional physics models require detailed environmental knowledge, while stochastic replay methods are constrained by the limited diversity of measured channels and often fail to generalize to unseen scenarios, reducing their practical applicability. To address these challenges, we propose StableUASim, a pre-trained conditional latent diffusion surrogate model that captures the stochastic dynamics of underwater acoustic communication channels. Leveraging generative modeling, StableUASim produces diverse and statistically realistic channel realizations, while supporting conditional generation from specific measurement samples. Pre-training enables rapid adaptation to new environments using minimal additional data, and the autoencoder latent representation facilitates efficient channel analysis and compression. Experimental results demonstrate that StableUASim accurately reproduces key channel characteristics and communication performance, providing a scalable, data-efficient, and physically consistent surrogate model for both system design and machine learning-driven underwater applications. In this work, we focus on sparse underwater channels, and expanding the diversity of the pre-training dataset is expected to further improve generalization to a broader range of underwater propagation environments.
Nov 19, 2025

GLOBE: Accurate Surrogates for Boundary-Driven PDEs via Domain-Inspired Architectures and Equivariance

We introduce GLOBE, a nonlinear neural surrogate for boundary-driven, homogeneous, weakly nonlinear PDEs that draws inductive bias from boundary-element methods and equivariant ML, representing solutions as superpositions of learnable Green's-function-like kernels evaluated from boundary faces to targets. The architecture is translation-, rotation-, and parity-equivariant; discretization-invariant; and units-invariant via rigorous nondimensionalization. On AirFRANS (steady incompressible RANS over NACA airfoils), a compact ~117k-parameter GLOBE model has lower mean-squared error than published values for the strongest reported ML surrogates on both the interpolation and low-data splits, in single-run comparisons under the released AirFRANS normalization. The model evaluates fields at arbitrary points, handles non-watertight meshes, and generalizes under Reynolds number and angle-of-attack distribution shifts. We further introduce a hierarchical Barnes-Hut acceleration that exploits the kernel's far-field decay to reduce evaluation cost from quadratic to near-linear in the number of boundary elements, with a tunable opening angle that recovers exact dense evaluation in the zero-angle limit. Quantitative validation focuses on 2D AirFRANS; on 3D DrivAerML cars, GLOBE predicts surface and volume fields on the full meshes, and we compare it with a matched Transolver baseline. Together, these results show that boundary-element-inspired inductive biases can substantially improve the accuracy of ML surrogates for boundary-driven PDEs while scaling to large 3D meshes.
Nov 19, 2025

SURFing to the Fundamental Limit of Jet Tagging

Beyond the practical goal of improving search and measurement sensitivity through better jet tagging algorithms, there is a deeper question: what are their upper performance limits? Generative surrogate models with learned likelihood functions offer a new approach to this problem, provided the surrogate correctly captures the underlying data distribution. In this work, we introduce the SUrrogate ReFerence (SURF) method, a new approach to validating generative models. This framework enables exact Neyman-Pearson tests by training the target model on samples from another tractable surrogate, which is itself trained on real data. We argue that the EPiC-FM generative model is a valid surrogate reference for JetClass jets and apply SURF to show that modern jet taggers may already be operating close to the true statistical limit. By contrast, we find that autoregressive GPT models unphysically exaggerate top vs. QCD separation power encoded in the surrogate reference, implying that they are giving a misleading picture of the fundamental limit.
Nov 8, 2025

Hard-constraint physics-residual networks for hydrogen crossover prediction and high-pressure extrapolation in PEM water electrolysis

Hydrogen crossover is a critical safety and efficiency constraint in high-pressure polymer electrolyte membrane water electrolysis (PEMWE), but accurate prediction remains difficult because data are limited, transport physics are strongly coupled, and industrial operation requires reliable extrapolation beyond observed conditions. This study develops a hard-constraint physics-residual network (PR-Net) for hydrogen crossover prediction in PEMWE and compares it with a purely data-driven neural network (NN) and a soft-constraint physics-informed neural network (PINN). PR-Net embeds Henry's, Fick's, and Faraday's laws as a deterministic backbone and learns only a residual correction for unmodelled nonlinear effects. The benchmark includes 184 observations from eight peer-reviewed sources across six membrane types, covering 1-200 bar, 25−85°C25-85°C, and 0.05−5.0Acm−20.05-5.0 A cm^{-2}. PR-Net achieves R2=99.57±0.16R^2 = 99.57 \pm 0.16%, with 9-fold lower prediction variability than NN and PINN. In pressure-axis extrapolation, PR-Net attains R2=94.02±0.92R^2 = 94.02 \pm 0.92% at 200 bar, 2.5 times beyond the training pressure range, compared with 68.06±5.5268.06 \pm 5.52% for PINN and 58.00±8.6058.00 \pm 8.60% for NN (p < 0.001). Residual analysis indicates that the learned correction captures part of the high-pressure gas-phase non-ideality and recovers a transport-regime transition near 0.23Acm−20.23 A cm^{-2} between Fickian diffusion-dominated and Faradaic production-dominated transport. With a computation time of 1.08±0.34ms1.08 \pm 0.34 ms on low-power embedded hardware, PR-Net provides a practical framework for real-time crossover monitoring, adaptive process control, and safer high-pressure green-hydrogen operation.
Oct 9, 2025

Climate Surrogates for Scalable Multi-Agent Reinforcement Learning: A Case Study with CICERO-SCM

Climate policy analysis requires models that capture multi-gas climate effects, but such models are too slow to embed in reinforcement learning loops at scale. In collaboration with the European Environment Agency, we develop a multi-agent reinforcement learning (MARL) framework that integrates a higher-fidelity climate surrogate as the environment transition, enabling regional agents to learn policies under multi-gas dynamics. We train a recurrent surrogate on 20,00020{,}000 multi-gas emission pathways to emulate CICERO-SCM. The surrogate achieves near-simulator accuracy (global-mean temperature RMSE ≈ ⁣4 ⁣× ⁣10−4 K\approx\!4\!\times \!10^{-4}\,\mathrm{K}) with ∼ ⁣1000×\sim\!1000\times faster one-step inference and yields > ⁣100×>\!100\times end-to-end MARL training speed-up. We show policy agreement with the simulator in tractable settings and propose a replay- and rank-consistency test (Kendall's ττ) for assessing policy fidelity when simulator-in-the-loop training is infeasible. This enables large-scale multi-agent policy experiments while retaining high-fidelity multi-gas climate response.
Oct 1, 2025

RheOFormer: A generative transformer model for simulation of complex fluids and flows

The ability to model mechanics of soft materials under flowing conditions is key in designing and engineering processes and materials with targeted properties. This generally requires solution of internal stress tensor, related to the deformation tensor through nonlinear and history-dependent constitutive models. Traditional numerical methods for non-Newtonian fluid dynamics often suffer from prohibitive computational demands and poor scalability to new problem instances. Developments in data-driven methods have mitigated some limitations but still require retraining across varied physical conditions. In this work, we introduce Rheological Operator Transformer (RheOFormer), a generative operator learning method leveraging self-attention to efficiently learn different spatial interactions and features of complex fluid flows. We benchmark RheOFormer across a range of different viscometric and non-viscometric flows with different types of viscoelastic and elastoviscoplastic mechanics in complex domains against ground truth solutions. Our results demonstrate that RheOFormer can accurately learn both scalar and tensorial nonlinear mechanics of different complex fluids and predict the spatio-temporal evolution of their flows, even when trained on limited datasets. Its strong generalization capabilities and computational efficiency establish RheOFormer as a robust neural surrogate for accelerating predictive complex fluid simulations, advancing data-driven experimentation, and enabling real-time process optimization across a wide range of applications.
Sep 19, 2025

ArchesClimate: Probabilistic Decadal Ensemble Generation With Flow Matching

Internal variability is a dominant contributor to the uncertainty of predictions at the interannual to decadal timescale. A typical approach to separating the internal variability from forced climate responses is to generate large ensembles of simulations under different initial conditions. Due to the complexity of Earth System Models, generating these large ensembles is computationally expensive. In this work, we present ArchesClimate, a deep learning-based climate model emulator designed to reduce the cost of exploring internal variability at timescales ranging from monthly to decadal. ArchesClimate is trained on decadal hindcasts of the IPSL-CM6A-LR climate model. We train a flow matching model following ArchesWeatherGen, which we adapt to predict near-term climate. Once trained, the model generates states at a one-month lead time from the states of the two preceding months, and can be used to auto-regressively emulate climate model simulations. We show that for up to 10 years, these generations are stable and physically consistent. We also show that for several important climate variables, ArchesClimate generates simulations that are interchangeable with the IPSL model. This work suggests that climate model emulators could reduce the cost of generating large ensembles with climate models.
Aug 22, 2025

RF-Informed Graph Neural Networks for Accurate and Data-Efficient Circuit Performance Prediction

Accurately predicting the performance of active radio frequency (RF) circuits is essential for modern wireless systems but remains challenging due to highly nonlinear behavior and the high computational cost of traditional simulation tools. Existing machine learning (ML) surrogates often require large datasets to generalize across various topologies or are not accurate on held-out circuits. This work presents a lightweight, data-efficient, and topology-aware graph neural network (GNN) framework for predicting key performance metrics of active RF circuit classes, such as low-noise amplifiers (LNAs), mixers, voltage-controlled oscillators (VCOs), power amplifiers (PAs), and voltage amplifiers (VAs). The proposed framework employs RFIC domain-informed feature indexing to enable cross-topology adaptability by cheap encoding of functional device semantics (e.g., differential pair and varactor transistors) and efficient knowledge transfer. The surrogate model represents circuits using device-terminal graph abstractions to preserve fine-grained connectivity and transistor-level symmetry. The final model is generalized to a wide variety of classes by being trained in parallel. Experimental results demonstrate accurate modeling of multimodal and heavy-tailed RF performance distributions, achieving an average mean relative error (MRE) of 2.71% across nineteen topologies, an improvement of 3.3x and 20x faster in training over prior art, and the generalization to held-out topologies is improved by ~26.2x. Furthermore, this work shows ~36x training data efficiency compared to state-of-the-art, demonstrating its effectiveness for scalable and deployment-ready RF design automation.
Aug 3, 2025

Robust Surrogate Modeling for Explainable Graph Neural Networks

Graph Neural Networks (GNNs) have achieved remarkable performance in a wide range of graph-related learning tasks. However, explaining their predictions remains a challenging problem, especially due to the mismatch between the graphs used during training and those encountered during explanation. Most existing methods optimize soft edge masks on weighted graphs to highlight important substructures, but these graphs differ from the unweighted graphs on which GNNs are trained. This distributional shift leads to unreliable gradients and degraded explanation quality, especially when generating small, sparse subgraphs. To address this issue, we propose explaining a robust surrogate whose explanations provide a faithful proxy for those of the original model. Specifically, we train this surrogate model through an iterative re-weighting strategy that alternates between identifying explanation subgraphs and adapting the model to the resulting weighted graph distribution, supporting stable explanation refinement while preserving the validity of the soft-mask relaxation. We evaluate our method on multiple benchmark datasets using different GNN backbones and explanation methods. Experimental results show that our method consistently improves explanation quality and can be flexibly integrated with different architectures.
Jul 1, 2025

A Neural Operator based on Dynamic Mode Decomposition

The scientific computation methods development in conjunction with artificial intelligence technologies remains a hot research topic. Finding a balance between lightweight and accurate computations is a solid foundation for this direction. The study presents a neural operator based on the dynamic mode decomposition algorithm (DMD), mapping functional spaces, which combines DMD and deep learning (DL) for spatiotemporal processes efficient modeling. Solving PDEs for various initial and boundary conditions requires significant computational resources. The method suggested automatically extracts key modes and system dynamics using them to construct predictions, reducing computational costs compared to traditional numerical methods. The approach has demonstrated its efficiency through comparative analysis of performance with closest analogues DeepONet and FNO in the heat equation, Laplaces equation, and Burgers equation solutions approximation, where it achieves high reconstruction accuracy.
May 19, 2025

AneumoBench: A Source-Linked Benchmark for Synthetic-Geometry Transfer in Aneurysm CFD

Scientific machine learning uses simulation data to train surrogate models for fast physical-field prediction across geometries. Local shape editing can expand limited geometry collections, but whether its variants improve prediction on unseen geometries, and how to allocate them across sources, require controlled evaluation. We introduce AneumoBench, a dataset and benchmark linking 401 source aneurysm geometries to 9,693 locally edited descendant records, with computational fluid dynamics (CFD) fields computed on both. It contains 80,752 steady velocity-pressure cases across eight inlet conditions and 9,715 transient sequences of velocity, pressure, and wall shear stress (WSS). Each sequence contains 100 frames sampled at 0.01-s intervals from a 1-s cardiac cycle. Mesh, point, and voxel interfaces support steady field prediction and WSS forecasting from four observed frames. With family-disjoint splits, we compare source-only training, descendant training, and descendant pretraining followed by source fine-tuning across nine architectures on 79 held-out sources. Under the reported schedules, two-stage training lowers steady-field and reset-window WSS errors relative to source-only training. With the number of sampled fields and training updates fixed within each comparison, GraphSAGE benefits from descendant training and from distributing a fixed number of descendants across more sources. For WSS, reset-window gains do not consistently persist through 96-step rollout, and lower trajectory error need not improve cycle-level shear metrics or hotspot localization. These data and protocols enable researchers to compare descendant selection and training strategies on the same unseen source geometries.
Apr 17, 2025

Transferrable Surrogates in Expressive Neural Architecture Search Spaces

Neural architecture search (NAS) faces a challenge in balancing the exploration of expressive, broad search spaces that enable architectural innovation with the need for efficient evaluation of architectures to effectively search such spaces. We investigate surrogate model training for improving search in highly expressive NAS search spaces based on context-free grammars. We show that i) surrogate models trained either using zero-cost-proxy metrics and neural graph features (GRAF) or by fine-tuning an off-the-shelf LM have high predictive power for the performance of architectures both within and across datasets, ii) these surrogates can be used to filter out bad architectures when searching on novel datasets, thereby significantly speeding up search and achieving better final performances, and iii) the surrogates can be further used directly as the search objective for huge speed-ups.
Mar 7, 2025

From Theory to Application: A Practical Introduction to Neural Operators in Scientific Computing

This review examines neural operator architectures for learning solution operators of parametric partial differential equations (PDEs), with an emphasis on conceptual clarity and practical implementation. The work analyzes key models, including DeepONet, PCANet, and the Fourier Neural Operator, highlighting their underlying representations, computational structures, and comparative performance. These architectures are demonstrated on three canonical PDE problems: the Poisson equation, a linear elasticity problem, and a hyperelasticity problem. To make the presentation self-contained, key foundational topics are introduced, including finite-dimensional representations of function spaces, singular-value decomposition, and sampling from infinite-dimensional function spaces. Beyond forward modeling, the review discusses the use of neural operators as surrogate models within a Bayesian inverse-problem framework, including prior specification, forward-map approximation, and posterior computation. The performance of the three neural-operator architectures is evaluated on in-distribution samples, out-of-distribution samples, and Bayesian inference tasks. The review also discusses challenges related to prediction accuracy and generalization, outlining emerging strategies such as residual-based error correction and multi-level training. The review concludes by positioning neural operators within broader scientific-computing workflows and by identifying directions for reliable, scalable operator learning.
Nov 14, 2024

Neural Graph Simulator for Complex Systems

Simulating complex systems governed by coupled ordinary differential equations (ODEs) typically requires explicit governing equations and incurs high computational cost from iterative numerical solvers. We propose the Neural Graph Simulator (NGS), a general-purpose data-driven framework that learns multivariate continuous-time dynamics directly from noisy and incomplete observations, without prior knowledge of the equations. Unlike neural ODE-based approaches that depend on repeated solver evaluations, NGS approximates the state transition integral using elapsed time intervals as direct inputs. This eliminates numerical integration while naturally supporting irregular sampling and heterogeneous time scales. Through its graph network architecture, NGS generalizes to unseen graph structures and larger system sizes, as demonstrated on independently sampled Erdős--Rényi graphs. We evaluate NGS on comprehensive benchmarks covering linear diffusion, nonlinear and chaotic dynamics, systems with stiffness-like behavior, and real-world spatiotemporal datasets. NGS combines competitive accuracy across benchmarks with substantial speedups over numerical solvers in computationally demanding settings. Notably, NGS exhibits strong extrapolation to unseen graph sizes and longer time horizons, and supports non-uniform time steps. These results establish NGS as an efficient and scalable alternative to solver-dependent neural simulators.
Oct 29, 2024

A-UTE: Advection Informed, Uncertainty Aware Temperature Emulator

Physics-based Earth system models (ESMs) are essential for attributing climate change and generating scenario projections, yet their reliance on high-resolution numerical integration makes multi-decadal experiments expensive. In parallel, deep learning has delivered strong gains in short-range weather forecasting; however, auto-regressive roll-outs can accumulate error and become unstable when extended to decade-scale climate emulation. We introduce A-UTE: Advection Informed, Uncertainty Aware Temperature Emulator, aimed at stable multi-year emulation across heterogeneous climate models and grid resolutions. A-UTE is trained on various physics-based models at varying spatial resolutions to emulate temperature fields over a 10-year horizon. A-UTE formulates climate emulation as a forward-time stochastic dynamical system. We propose an auto-regressive ODE-SDE surrogate in which transport dynamics are constrained by an advection consistent ODE component, while a learned neural SDE term models coarse-grained variability and cross-model discrepancy at monthly resolution. We train A-UTE under negative log-likelihood objective for principled uncertainty estimates and probabilistic evaluation. Experiments across 20 climate models show that A-UTE improves long roll-out stability and accuracy relative to relevant baselines, advancing data-driven climate emulation with explicit physical structure and uncertainty-aware predictions.