Improving the accuracy of physics-informed neural networks via last-layer retraining

Saad Qadeer; Panos Stinis

math.NAMar 4, 2026

Improving the accuracy of physics-informed neural networks via last-layer retraining

Authors: Saad Qadeer, Panos Stinis

Organizations: Advanced Computing, Mathematics and Data Division, Pacific Northwest National Laboratory, Richland, WA

Abstract

Physics-informed neural networks (PINNs) are a versatile tool in the burgeoning field of scientific machine learning for solving partial differential equations (PDEs). However, determining suitable training strategies for them is not obvious, with the result that they typically yield moderately accurate solutions. In this article, we propose a method for improving the accuracy of PINNs by coupling them with a post-processing step that seeks the best approximation in a function space associated with the network. We find that our method yields errors four to five orders of magnitude lower than those of the parent PINNs across architectures and dimensions. Moreover, we can reuse the basis functions for the linear space in more complex settings, such as time-dependent and nonlinear problems, allowing for transfer learning. Our approach also provides a residual-based metric that allows us to optimally choose the number of basis functions employed.

Figures & tables

Panel of Figure 1 — Figure 1: Poisson equation on the one dimensional domain $\Omega_{1}$ . The errors $\lVert u_{r}-u\rVert$ for different values of $r$ , measured in $L^{\infty}$ (blue) and $L^{2}$ (red), for problem (i), across PINN architectures. We also show the PINN errors $\lVert u^{\theta}_{\text{NN}}-u\rVert$ in $L^{\infty}$ (yellow) and $L^{2}$ (purple) as the horizontal curves. The residuals $\lVert e_{r}\rVert$ in in $L^{\infty}$ (green) and $L^{2}$ (cyan) are also shown; note the close similarity in the behaviour of the error and residual curves.

Panel of Figure 2 — Figure 2: Poisson equation in the square box $\Omega_{2}$ . The errors $\lVert u_{r}-u\rVert$ for different values of $r$ , measured in $L^{\infty}$ (blue) and $L^{2}$ (red), for problem (ii), across PINN architectures. We also show the PINN errors $\lVert u^{\theta}_{\text{NN}}-u\rVert$ in $L^{\infty}$ (yellow) and $L^{2}$ (purple) as the horizontal curves. The residuals $\lVert e_{r}\rVert$ in in $L^{\infty}$ (green) and $L^{2}$ (cyan) are also shown; note the close similarity in the behaviour of the error and residual curves.

Panel of Figure 3 — Figure 3: Poisson equation on the L-shaped domain $\Omega_{3}$ . The errors $\lVert u_{r}-u\rVert$ for different values of $r$ , measured in $L^{\infty}$ (blue) and $L^{2}$ (red), for problem (iii), across PINN architectures. We also show the PINN errors $\lVert u^{\theta}_{\text{NN}}-u\rVert$ in $L^{\infty}$ (yellow) and $L^{2}$ (purple) as the horizontal curves. The residuals $\lVert e_{r}\rVert$ in in $L^{\infty}$ (green) and $L^{2}$ (cyan) are also shown; note the close similarity in the behaviour of the error and residual curves.

Panel of Figure 4 — Figure 4: The heat equation on $\Omega_{1}=[-1,1]$ ; we show the errors $E_{r,p}$ for different values of $r$ , measured with $p=\infty$ (blue) and $p=2$ (red). We also show present the $L^{\infty}$ (yellow) and $L^{2}$ (purple) norms of the residuals.

Panel of Figure 5 — Figure 5: The heat equation on the square box $\Omega_{2}$ ; the errors $E_{r,p}$ are shown for different values of $r$ , measured with $p=\infty$ (blue) and $p=2$ (red), as well as the averaged $L^{\infty}$ (yellow) and $L^{2}$ (purple) norms of the residuals.

Panel of Figure 6 — Figure 6: The heat equation in the L-shaped domain $\Omega_{3}$ . We present the errors $E_{r,p}$ for different values of $r$ , measured with $p=\infty$ (blue) and $p=2$ (red), along with the averaged $L^{\infty}$ (yellow) and $L^{2}$ (purple) norms of the residuals.

Panel of Figure 7 — Figure 7: The steady viscous Burgers’ equation ( 10 ) on the 1D interval $\Omega_{1}$ . The errors for different values of $r$ are shown, measured in $L^{\infty}$ (blue) and $L^{2}$ (red), along with the $L^{\infty}$ (yellow) and $L^{2}$ (purple) norms of the residuals.

Panel of Figure 8 — Figure 8: The Poisson–Boltzmann equation ( 11 ) on the 1D interval $\Omega_{1}$ . The errors for different values of $r$ are shown, measured in $L^{\infty}$ (blue) and $L^{2}$ (red), along with the $L^{\infty}$ (yellow) and $L^{2}$ (purple) norms of the residuals.

Explore similar work

Scale-PINN: Learning Efficient Physics-Informed Neural Networks Through Sequential Correction
Feb 23, 2026Pao-Hsiung Chiu, Jian Cheng Wong, Chin Chun Ooi +3Neural PDE Solvers Physics-Informed ML
Institute of High Performance Computing, A*STAR, Singapore. · Centre for Frontier AI Research, A*STAR, Singapore. · School of Mechanical Engineering, Tianjin University, China. +1
Linearized PINN with pretrained nonlinear layers
Sep 14, 2026Wenhao Chen, Alexandre M. TartakovskyEfficient Neural Network Inference PDE Solving
Civil and Environmental Engineering Department University of Illinois Urbana-Champaign Urbana, IL 61801, USA · Pacific Northwest National Laboratory Richland, WA 99352, USA
Curriculum Learning of Physics-Informed Neural Networks based on Spatial Correlation
May 14, 2026Xujia Chen, Xinyue Hu, Letian Chen +2PDE Solving Curriculum Learning
Department of Automation, Tsinghua University, Beijing 100084, P.R. China

Feb 23, 2026J/K move · Enter open · S save

Scale-PINN: Learning Efficient Physics-Informed Neural Networks Through Sequential Correction

Pao-Hsiung Chiu, Jian Cheng Wong, Chin Chun Ooi, Chang Wei+2

Institute of High Performance Computing, A*STAR, Singapore. · Centre for Frontier AI Research, A*STAR, Singapore. · School of Mechanical Engineering, Tianjin University, China.+1

Neural PDE Solvers Physics-Informed ML Neural Network Optimization

Open paper AskPDF arXiv

Physics-informed neural networks (PINNs) have emerged as a promising mesh-free paradigm for solving partial differential equations, yet adoption in science and engineering is limited by slow training and modest accuracy relative to modern numerical solvers. We introduce the Sequential Correction Algorithm for Learning Efficient PINN (Scale-PINN), a learning strategy that bridges modern physics-informed learning with numerical algorithms. Scale-PINN incorporates the iterative residual-correction principle, a cornerstone of numerical solvers, directly into the loss formulation, marking a paradigm shift in how PINN losses can be conceived and constructed. This integration enables Scale-PINN to achieve unprecedented convergence speed across PDE problems from different physics domain, including reducing training time on a challenging fluid-dynamics problem for state-of-the-art PINN from hours to sub-2 minutes while maintaining superior accuracy, and enabling application to representative problems in aerodynamics and urban science. By uniting the rigor of numerical methods with the flexibility of deep learning, Scale-PINN marks a significant leap toward the practical adoption of PINNs in science and engineering through scalable, physics-informed learning. Codes are available at https://github.com/chiuph/SCALE-PINN.

Abstract

Explore similar work

Scale-PINN: Learning Efficient Physics-Informed Neural Networks Through Sequential Correction

Linearized PINN with pretrained nonlinear layers

Curriculum Learning of Physics-Informed Neural Networks based on Spatial Correlation