The Physics Is the Network: Fast, Training-Free Neural Emulation
Authors: Luka Todorovic, Kaushal Gianchandani, Sai Ravela
Organizations: Department of Mathematics and Department of Statistics, University of Michigan, Ann Arbor, MI · NASA Postdoctoral Program Fellow, NASA Goddard Institute for Space Studies, New York, NY · Department of Earth, Atmospheric, and Planetary Sciences, Massachusetts Institute of Technology, Cambridge, MA
Neural emulation need not require learning a substitute for known physics. We investigate PolyNets that couple a continuous-time neural representation of polynomial dynamics with a simple recurrent network for discrete-time integration. When the governing coefficients are known, the resulting tensor graph requires no training and preserves the prescribed numerical scheme while enabling efficient hardware execution and parallel simulation. Using the one-dimensional Korteweg-de Vries equation, we compare PolyNet with CPU and GPU finite-difference Runge-Kutta solvers and three trained neural models: a state-map emulator, a physics-informed predictor, and a learned-tendency integrator. Analytical soliton solutions provide a reference for amplitude and phase errors across grid resolutions, 10^6 dispersively scaled time steps, ensembles of up to 10^4 trajectories, and varying initial amplitudes. PolyNet achieves accuracy comparable to the numerical solvers, including at amplitudes outside the learned models' training conditions. It combines this fidelity with high GPU throughput, approximately constant over the tested resolution range, and favorable ensemble scaling.
Figures & tables
Figure 1: PolyNet graph for the semidiscrete KdV tendency, FN(U)=−6U⊙D1U−D3U . Linear derivative branches and a componentwise product implement the prescribed physics. The four-stage recurrent RK4 time integrator is not shown.
Figure 2: Training time for the three learned models and setup time for CPU finite-difference RK4, GPU finite-difference RK4, and PolyNet versus grid size N . The unsuccessful NPDE configuration at N=100 is omitted. Shading around log–log fits includes an imposed minimum display width and should not be interpreted as a confidence interval.
Figure 3: Time-step throughput versus grid size N . Each timed block advances 1000 steps; throughput in steps/ms is 1000/Tmedian , where Tmedian is the median block duration in milliseconds. Shading around log–log fits includes an imposed minimum display width and is not a confidence interval.
Figure 4: Maximum sampled absolute (a) phase and (b) amplitude errors against the analytical solitary-wave reference versus grid size N , over 106 steps at the prescribed resolution-dependent dispersive time step. Unsuccessful GPU finite-difference RK4 configurations at N=100 and N=200 are omitted.
Figure 5: Median batch-step throughput (steps/ms) versus the number of independent soliton trajectories at N=400 . Each batch step advances the entire ensemble; aggregate trajectory-step throughput is the plotted value multiplied by ensemble size.
Figure 6: Maximum sampled absolute (a) phase and (b) amplitude errors versus initial soliton height h , for N=400 and 106 integration steps. The neural models were trained at h=3 using analytical soliton targets; other heights were held out. Periodic phase separation is bounded by L/2=20 , explaining the saturated phase-error curves.
Neural PDE solvers use trainable models to approximate solutions or solution operators for partial differential equations, often with physical constraints.
Simulation is central to modern engineering and science, but the cost of numerical solvers for partial differential equations (PDEs) remains a bottleneck whenever fast or many-query evaluations are required. Neural emulators trained on solver-generated data promise significant speedups, yet they are usually framed as opaque alternatives to the very methods that produce their training signal. This thesis argues the two paradigms are more alike than different: neural architectures mirror classical discretizations, their errors are amenable to the same spectral analysis, and insight flows profitably in both directions. We approach the relationship by disentangling the multiple roles a solver plays in the emulator learning pipeline. Mode-wise Fourier analysis then provides a common language in which solver errors, architectural inductive biases, and training objectives can all be read off simultaneously. Taken together, this allows synthesizing three contributions. (1) APEBench, a comprehensive benchmarking suite for autoregressive neural emulators of PDEs that uses fast differentiable pseudo-spectral solvers in JAX. (2) Progressively Refined Differentiable Physics, an investigation of the effect of unconverged solvers on surrogate training. (3) Neural Emulator Superiority, an analysis of the influence of numerical errors and architectural inductive biases.
Felix Koehler
Technische Universität München TUM School of Computation, Information and Technology
Amortizing physics-informed neural networks (PINNs) across related PDEs requires describing each equation to a reusable solver. Coefficient vectors encode numerical parameters in predefined slots, leaving operator and cross-field assignments implicit. We make these relationships explicit in an operator graph, with nodes for fields, derivatives, terms, and residuals and coefficients retained as term attributes. A graph hypernetwork generates diagonal codes that initialize a meta-trained factorized PINN for each target equation. Meta-training and target-specific adaptation use governing equations and prescribed conditions without solution labels. We compare coefficient-vector, DeepSets-based term-set, and graph conditioning by solution accuracy within a fixed adaptation budget. In scalar convection-diffusion-reaction problems, both term-based descriptors improve high-reaction accuracy, with similar performance. In two-field Fisher-KPP, meta-training sees uncoupled and one-way systems; after 3,000 adaptation steps on unseen two-way coupling, the graph's mean final error is 35.7% below the term set and 67.7% below the coefficient vector. In a fixed-structure capacitively coupled plasma model, the coefficient vector performs best. These results support extending coefficient conditioning with explicit equation relationships for physics-based solver adaptation.
Neural surrogate models offer fast approximate mappings from PDE parameters to solutions, but they typically treat solving as a purely statistical task: once trained, they struggle to correct their own constraint violations and extrapolate beyond the training distribution. Recent hybrid methods promote physical correctness by targeting the PDE residual via gradient descent or Gauss--Newton steps, but inherit the compute cost and instability of the underlying classical optimizers. We show, theoretically and empirically, that numerically minimizing the PDE residual can be an unreliable proxy for reconstruction accuracy in ill-conditioned systems, explaining why these methods often do not make accurate predictions despite achieving low residuals. We propose error-conditioned Neural Solvers (ENS), built on a different principle: rather than an optimization target, the PDE residual field is passed as a direct input to the network at each iteration, enabling it to read the spatial structure of its own errors and learn an update policy to iteratively correct its predictions. Across four PDE families, ENS attains the highest prediction accuracy in the large majority of settings, with gains reaching 10× on turbulent Kolmogorov flow, while avoiding the expensive compute cost of hybrid methods. ENS's learned correction policy generalizes under distribution shift, including zero-shot parameter changes and cross-equation transfer, where its relative advantage is largest in the ill-conditioned regimes where residual minimization is least reliable. Project website: https://neuralsolver.github.io/.
Haina Jiang, Liam Wang, Peng-Chen Chen +4
University of Michigan · KAIST AI · Los Alamos National Laboratory