Improving the accuracy of physics-informed neural networks via last-layer retraining

Saad Qadeer; Panos Stinis

math.NAMar 4, 2026

Improving the accuracy of physics-informed neural networks via last-layer retraining

Authors: Saad Qadeer, Panos Stinis

Organizations: Advanced Computing, Mathematics and Data Division, Pacific Northwest National Laboratory, Richland, WA

Abstract

Physics-informed neural networks (PINNs) are a versatile tool in the burgeoning field of scientific machine learning for solving partial differential equations (PDEs). However, determining suitable training strategies for them is not obvious, with the result that they typically yield moderately accurate solutions. In this article, we propose a method for improving the accuracy of PINNs by coupling them with a post-processing step that seeks the best approximation in a function space associated with the network. We find that our method yields errors four to five orders of magnitude lower than those of the parent PINNs across architectures and dimensions. Moreover, we can reuse the basis functions for the linear space in more complex settings, such as time-dependent and nonlinear problems, allowing for transfer learning. Our approach also provides a residual-based metric that allows us to optimally choose the number of basis functions employed.

Figures & tables

Panel of Figure 1 — Figure 1: Poisson equation on the one dimensional domain $\Omega_{1}$ . The errors $\lVert u_{r}-u\rVert$ for different values of $r$ , measured in $L^{\infty}$ (blue) and $L^{2}$ (red), for problem (i), across PINN architectures. We also show the PINN errors $\lVert u^{\theta}_{\text{NN}}-u\rVert$ in $L^{\infty}$ (yellow) and $L^{2}$ (purple) as the horizontal curves. The residuals $\lVert e_{r}\rVert$ in in $L^{\infty}$ (green) and $L^{2}$ (cyan) are also shown; note the close similarity in the behaviour of the error and residual curves.

Panel of Figure 2 — Figure 2: Poisson equation in the square box $\Omega_{2}$ . The errors $\lVert u_{r}-u\rVert$ for different values of $r$ , measured in $L^{\infty}$ (blue) and $L^{2}$ (red), for problem (ii), across PINN architectures. We also show the PINN errors $\lVert u^{\theta}_{\text{NN}}-u\rVert$ in $L^{\infty}$ (yellow) and $L^{2}$ (purple) as the horizontal curves. The residuals $\lVert e_{r}\rVert$ in in $L^{\infty}$ (green) and $L^{2}$ (cyan) are also shown; note the close similarity in the behaviour of the error and residual curves.

Panel of Figure 3 — Figure 3: Poisson equation on the L-shaped domain $\Omega_{3}$ . The errors $\lVert u_{r}-u\rVert$ for different values of $r$ , measured in $L^{\infty}$ (blue) and $L^{2}$ (red), for problem (iii), across PINN architectures. We also show the PINN errors $\lVert u^{\theta}_{\text{NN}}-u\rVert$ in $L^{\infty}$ (yellow) and $L^{2}$ (purple) as the horizontal curves. The residuals $\lVert e_{r}\rVert$ in in $L^{\infty}$ (green) and $L^{2}$ (cyan) are also shown; note the close similarity in the behaviour of the error and residual curves.

Panel of Figure 4 — Figure 4: The heat equation on $\Omega_{1}=[-1,1]$ ; we show the errors $E_{r,p}$ for different values of $r$ , measured with $p=\infty$ (blue) and $p=2$ (red). We also show present the $L^{\infty}$ (yellow) and $L^{2}$ (purple) norms of the residuals.

Panel of Figure 5 — Figure 5: The heat equation on the square box $\Omega_{2}$ ; the errors $E_{r,p}$ are shown for different values of $r$ , measured with $p=\infty$ (blue) and $p=2$ (red), as well as the averaged $L^{\infty}$ (yellow) and $L^{2}$ (purple) norms of the residuals.

Panel of Figure 6 — Figure 6: The heat equation in the L-shaped domain $\Omega_{3}$ . We present the errors $E_{r,p}$ for different values of $r$ , measured with $p=\infty$ (blue) and $p=2$ (red), along with the averaged $L^{\infty}$ (yellow) and $L^{2}$ (purple) norms of the residuals.

Panel of Figure 7 — Figure 7: The steady viscous Burgers’ equation ( 10 ) on the 1D interval $\Omega_{1}$ . The errors for different values of $r$ are shown, measured in $L^{\infty}$ (blue) and $L^{2}$ (red), along with the $L^{\infty}$ (yellow) and $L^{2}$ (purple) norms of the residuals.

Panel of Figure 8 — Figure 8: The Poisson–Boltzmann equation ( 11 ) on the 1D interval $\Omega_{1}$ . The errors for different values of $r$ are shown, measured in $L^{\infty}$ (blue) and $L^{2}$ (red), along with the $L^{\infty}$ (yellow) and $L^{2}$ (purple) norms of the residuals.

Explore similar work

Feb 23, 2026

Scale-PINN: Learning Efficient Physics-Informed Neural Networks Through Sequential Correction

Physics-informed neural networks (PINNs) have emerged as a promising mesh-free paradigm for solving partial differential equations, yet adoption in science and engineering is limited by slow training and modest accuracy relative to modern numerical solvers. We introduce the Sequential Correction Algorithm for Learning Efficient PINN (Scale-PINN), a learning strategy that bridges modern physics-informed learning with numerical algorithms. Scale-PINN incorporates the iterative residual-correction principle, a cornerstone of numerical solvers, directly into the loss formulation, marking a paradigm shift in how PINN losses can be conceived and constructed. This integration enables Scale-PINN to achieve unprecedented convergence speed across PDE problems from different physics domain, including reducing training time on a challenging fluid-dynamics problem for state-of-the-art PINN from hours to sub-2 minutes while maintaining superior accuracy, and enabling application to representative problems in aerodynamics and urban science. By uniting the rigor of numerical methods with the flexibility of deep learning, Scale-PINN marks a significant leap toward the practical adoption of PINNs in science and engineering through scalable, physics-informed learning. Codes are available at https://github.com/chiuph/SCALE-PINN.

Pao-Hsiung Chiu, Jian Cheng Wong, Chin Chun Ooi +3

Institute of High Performance Computing, A*STAR, Singapore. · Centre for Frontier AI Research, A*STAR, Singapore. · School of Mechanical Engineering, Tianjin University, China. +1

Neural PDE Solvers Physics-Informed ML Neural Network Optimization

Ask PDF

Sep 14, 2026

Linearized PINN with pretrained nonlinear layers

We propose a linearized Physics-Informed Neural Network (lPINN), a reduced-order neural basis method for forward and inverse differential equations. In an offline stage, lPINN learns operator-compatible continuous neural basis functions from an ensemble of numerical solutions. The basis functions are differentiable through automatic differentiation and are pretrained using solution data together with either derivative information or physics residuals. For each new problem instance, the basis functions are frozen and the solution is obtained by minimizing the governing-equation residual together with applicable initial, boundary, regularization, and observational terms. Unlike surrogate and operator-learning methods, the training data define the trial space offline, while the instance-specific solution is computed online by enforcing the governing physics. Relative to vanilla PINNs, lPINN pretrains the nonlinear hidden-layer representation offline and performs online inference only in the final linear layer. We evaluate lPINN on forward and inverse problems for the advection-diffusion equation, Burgers' equation, and the nonlinear pendulum equation. Compared with vanilla PINNs, lPINN achieves lower solution and parameter errors while reducing online inference times by approximately one to more than three orders of magnitude, with the largest gains generally observed for limited residual or measurement data. Cross-resolution experiments show that the learned continuous representation can be evaluated on finer meshes without retraining and with nearly unchanged accuracy.

Wenhao Chen, Alexandre M. Tartakovsky

Civil and Environmental Engineering Department University of Illinois Urbana-Champaign Urbana, IL 61801, USA · Pacific Northwest National Laboratory Richland, WA 99352, USA

Efficient Neural Network Inference PDE Solving Reduced-Order Modeling Physics-Informed ML PDE Inverse Problems

Ask PDF

May 14, 2026

Curriculum Learning of Physics-Informed Neural Networks based on Spatial Correlation

Physics-Informed Neural Networks (PINNs) combine deep learning with physical constraints for solving partial differential equations (PDEs), and are widely applied in fluid mechanics, heat transfer, and solid mechanics. However, PINN training still suffers from high-dimensional non-convex loss landscapes, imbalanced multiobjective constraints, and ineffective information propagation. Existing curriculum learning and causality-guided strategies improve training stability, but mainly focus on temporal or parametric progression, lacking explicit treatment of spatial information propagation and inter-region consistency. Moreover, they are not directly applicable to boundary value problems (BVPs) with strong spatial coupling. To address this issue, we propose a spatially correlated curriculum learning framework for PINNs. To the best of our knowledge, this is the first work to address PINN training difficulties from the perspective of spatial coupling among subregions. First, spatial causal weights guide information from near-boundary regions inward, reducing optimization failures and spurious convergence. Second, a low-frequency information bridge enforces pseudo-label-based consistency across spatially separated regions, suppressing global low-frequency drift. Third, a region-adaptive reweighting strategy adjusts subregion losses to reduce local residuals and recover high-frequency details. Experiments on PDE benchmarks show that, under comparable computational cost, the proposed method alleviates training failures and improves solution accuracy. The code is available at https://github.com/pigofmomo/CurriculumLearningPINN.

Xujia Chen, Xinyue Hu, Letian Chen +2

Department of Automation, Tsinghua University, Beijing 100084, P.R. China

PDE Solving Curriculum Learning Physics-Informed ML

Ask PDF