PINNeval: A Comprehensive Evaluation Standard for Physics-Informed Neural Networks
Organizations: Joint Lab Artificial Intelligence and Data Science, Osnabrück University, Osnabrück, Germany
Abstract
The Physics-Informed Neural Network (PINN) literature has produced a rapidly expanding catalogue of methods. We divide a PINN training pipeline systematically into four components and observe that many existing benchmarks study these components in isolation. These components are domain-importance adaptation, loss balancing, neural network architecture, and optimization scheme. Moreover, the studies set the surrounding and not considered components of the pipeline to a minimal default that is no longer representative of current best practice approaches. Conclusions drawn under such conditions may not transfer to competitive pipelines. In this work, we propose a new evaluation protocol that (1) treats the four components as an integrated system; (2) crucially requires competitive implementations of the components not under investigation; (3) specifies the metrics and reporting format; and (4) is accompanied by structured documentation that ensures the traceability of evaluations across studies. Based on this comprehensive protocol, we release PINNpipe, an innovative JAX package containing 18 different methods from the current literature as well as nine efficiently implemented benchmark problems. Using PINNpipe, we illustrate our evaluation protocol in relation to established benchmark studies and report results that challenge common assumptions.
Figures & tables
Appendix figures & tables29 assets
Supplementary material from the paper’s appendix.
Appendix
| Symbol | Description | Default |
|---|---|---|
| learning rate | ||
| EMA decay for first moment | ||
| EMA decay for , , | ||
| numerical stability constant | ||
| eigenbasis update period |
| Symbol | Description | Value |
|---|---|---|
| Initial inverse Hessian | ||
| Armijo constant (all stages) | ||
| Curvature tolerance, line-search stage 1 | ||
| Curvature tolerance, line-search stage 2 | ||
| Curvature tolerance, line-search stage 3 | ||
| Maximum trials per line-search stage | each |
| Wave | Burgers | AC | KdV | Poisson | Heat | KS | GL | RT | |
|---|---|---|---|---|---|---|---|---|---|
| Architecture | |||||||||
| Activation | Tanh | Tanh | Tanh | Tanh | Tanh | Tanh | Tanh | Swish | Tanh |
| Fourier Features Scale | |||||||||
| Hard constraint | |||||||||
| Axis | |||||||||
| Periodicity |
| Wave | Burgers | AC | KdV | Poisson | Heat | KS | GL | RT | |
| Learning rate schedule | |||||||||
| Initial learning rate | |||||||||
| Decay rate | |||||||||
| Decay steps | |||||||||
| Warmup steps | |||||||||
| Training | |||||||||
| Wave | Burgers | AC | KdV | Poisson | Heat | KS | GL | RT | |
| Learning rate schedule | |||||||||
| Initial learning rate | |||||||||
| Decay rate | |||||||||
| Decay steps | |||||||||
| Warmup steps | |||||||||
| One-stage Training | |||||||||
| Benchmark | Reference solution / solver | Verification | Est. accuracy |
|---|---|---|---|
| Poisson | exact, float64 | exact by construction | |
| Heat | exact, float64 | exact by construction | |
| Wave | exact, float64 | exact by construction | |
| KdV | Chebfun spin , ETDRK4, 511 modes, | reproduced to ; / refinement | |
| KS | Chebfun spin , 511 modes, | reproduced to ; / refinement | |
| GL | Chebfun spin2 , , | reproduced to ; / refinement |
| Benchmark | Random-R | vRBA | LAS | R3 | RAD |
|---|---|---|---|---|---|
| Wave | |||||
| KdV | |||||
| RT | |||||
| Heat | |||||
| AC | |||||
| Burgers |
| Benchmark | Random-R | vRBA | LAS | R3 | RAD |
|---|---|---|---|---|---|
| Wave | 10.53 | ||||
| KdV | 10.10 | ||||
| RT | 29.80 | ||||
| Heat | 5.43 | ||||
| AC | 11.24 | ||||
| Burgers | 6.26 |
| Benchmark | GradNorm-based | SoftAdapt | ReLoBRaLo | Fixed | GradNorm-MTL |
|---|---|---|---|---|---|
| Wave | |||||
| AC | |||||
| KS | |||||
| Poisson | |||||
| Burgers | |||||
| GL |
| Benchmark | GradNorm-based | SoftAdapt | ReLoBRaLo | Fixed | GradNorm_MTL |
|---|---|---|---|---|---|
| Wave | 10.95 | ||||
| AC | 9.62 | ||||
| KS | 116.20 | ||||
| Poisson | 4.29 | ||||
| Burgers | 6.60 | ||||
| GL | 27.06 |
| Relative error | Runtime [min] | |||||
|---|---|---|---|---|---|---|
| Benchmark | MLP | Modified MLP | PirateNet | MLP | Modified MLP | PirateNet |
| Wave | ||||||
| KdV | ||||||
| RT | ||||||
| Heat | ||||||
| AC | ||||||
| Relative error | Runtime [min] | |||||
|---|---|---|---|---|---|---|
| Benchmark | MLP | Modified MLP | PirateNet | MLP | Modified MLP | PirateNet |
| Wave | ||||||
| KdV | ||||||
| RT | ||||||
| Heat | ||||||
| AC | ||||||
| Relative error | Runtime [min] | |||||
|---|---|---|---|---|---|---|
| Benchmark | MLP | Modified MLP | PirateNet | MLP | Modified MLP | PirateNet |
| Wave | ||||||
| KdV | ||||||
| RT | ||||||
| Heat | ||||||
| AC | ||||||
| Benchmark | PirateNet, SOAP float32 (P1) † | PirateNet, SOAP float64 | MLP, SOAP float32 | MLP, SOAP float64 | MLP, SOAP SSBroyden2 float64 (P2) † |
|---|---|---|---|---|---|
| Wave | |||||
| KdV | |||||
| RT | |||||
| Heat | |||||
| AC | – a | ||||
| Burgers |
| Benchmark | PirateNet, SOAP float32 (P1) † | PirateNet, SOAP float64 | MLP, SOAP float32 | MLP, SOAP float64 | MLP, SOAP SSBroyden2 float64 (P2) † |
|---|---|---|---|---|---|
| Wave | |||||
| KdV | |||||
| RT | |||||
| Heat | |||||
| AC | – a | ||||
| Burgers |
| Benchmark | PirateNet-SOAP | SOAP-SSBroyden | |
|---|---|---|---|
| SOAP phase | SSBroyden Phase | ||
| Burgers | GiB | GiB | GiB |
| Korteweg–de Vries | GiB | GiB | GiB |
| Allen–Cahn | GiB | GiB | GiB |
| Kuramoto–Sivashinsky | GiB | GiB | GiB |
| Wave | GiB | GiB | GiB |
Physics-Informed ML
Physics-informed machine learning combines data-driven models with physical laws to solve, predict, and infer scientific systems.
See all 906 papers