Scientific machine learning has opened new avenues for solving parameterized partial differential equations (PDEs), enabling models to learn a family of PDEs and generalize to unseen instances. In this context, data-driven operator learning methods typically require large training datasets, while physics-informed neural networks (PINNs) suffer from difficult optimization and limited generalization, especially for nonlinear PDEs. We propose Newton-PINet, a physics-informed network enhanced by Newton linearization, offering an effective meta-learning framework for nonlinear PDEs. Newton-PINet (i) employs a physics-informed multilayer network with skip connections, where the output-layer weights are solved by least squares; (ii) adopts a two-stage learning strategy that first leverages gradient-based training to learn robust representations from the available training tasks, and then performs gradient-free fine-tuning on the output layer for fast task-specific generalization; and (iii) incorporates a Newton linearization method to speed up the least-squares iteration for nonlinear PDE problems. On a challenging nonlinear reaction-diffusion benchmark, Newton-PINet achieves up to three orders of magnitude lower relative error than recent neural solvers, while using 16x fewer training tasks and over an order of magnitude less training time (about 5 minutes versus several hours). This work advances the meta-learning of PINNs toward data-efficient, fast, and generalizable physics solvers. The code and datasets are publicly available at: https://github.com/fycAIwj/Newton-PINet.
Figures & tables
Figure 1: Newton-PINet model framework. Stage 1 (pre-training) meta-learns the task-agnostic parameters Θ={w~,λpde,λreg} by gradient descent (outer loop), while the output-layer weights w of each training task are updated by gradient-free Tikhonov-regularized least squares (inner loop); Stage 2 (inference) freezes Θ and adapts only w to new tasks. The bottom panel compares Picard and Newton linearization; the latter accelerates the convergence of nonlinear iterations for solving nonlinear PDEs.
Figure 2
Nonlinear term
Newton-linearized expression
(um)k+1
m(uk)m−1uk+1+(1−m)(uk)m
(umux)k+1
m(uk)m−1uxkuk+1+(uk)muxk+1−m(uk)muxk
[sinh(u)]k+1
cosh(uk)uk+1+sinh(uk)−cosh(uk)uk
[exp(u)]k+1
exp(uk)uk+1+exp(uk)−exp(uk)uk
[uln(u)]k+1
(ln(uk)+1)uk+1+ukln(uk)−(ln(uk)+1)uk
Table 1: Newton-linearized expressions of several nonlinear terms derived by our work. m denotes the exponent. k and k+1 refer to the current and next nonlinear iterations, respectively.
Problem
Meta-learning epoch
Meta-learning time cost
Inference time per test task
Test MSE of models
DNN
PINet
Newton-PINet
Burgers’
1000
286s
0.035s
1.06e-4
1.10e-4
1.79e-6
Generalized KdV
1000
453s
0.359s
3.74e-2
1.49e-2
1.94e-3
Kuramoto-Sivashinsky
3000
1014s
0.552s
5.03e-1
106
3.45e-2
Convection-diffusion- reaction
1000
203s
0.099s
5.27e-1
4.65e-3
4.32e-5
Klein-Gordon
1000
105s
0.065s
1.04
1.88e-7
2.36e-7
Table 2: Summary of model performance on nonlinear PDEs in this study. The reported meta-learning and fine-tuning time correspond to the Newton-PINet. “NaN” entries under PINet indicate numerical overflow during computation. See Appendix D.1 for detailed descriptions of all PDE problems and meta-learning configurations (summarized in Table 8 ). See Appendix D.2 for details on data generation, error metrics, and computational setup.
Figure 2: Burgers’ problem. (a) Meta-learning convergence of Newton-PINet vs. PINet under different nonlinear iterations (N) (lines and shaded areas: the median convergence path and interquartile ranges of 5 random-seed runs). (b) Boxplot of error distributions across all test tasks and predictions by Newton-PINet (N=6).
Unsupervised
Supervised
Hybrid
PO-DeepONet
Newton-PINet
DeepONet
T-DeepONet-TF
PINO
Newton-PINet
No. training tasks
1000
50
1000
1000
1000
50
Test relative L2 error
1.38e-2
3.51e-3
1.17e-2
2.08e-3
3.80e-3
5.46e-4
Training time
9.25h
291s
2800s
3041s
1200s
133s
Table 3: Comparison of training cost and performance at γ=0.01 on Burgers’ problem.
Figure 3: Convection-diffusion-reaction (CDR) problem. (a) Meta-learning convergence of Newton-PINet vs. PINet under different nonlinear iterations (N). (b,c) Test task MSE and runtime aggregated from 5 runs. In (b), the two circled regions correspond to Newton-PINet (N=4) and PINet (N=12), respectively. (d) Error distribution boxplot across all test tasks for different models; the prediction fields correspond to Newton-PINet (N=4).
Model
No. training tasks
Test relative MSE
Training time
Inference time
PI-neural-solver ( Boudec et al., 2025 )
800
2.91e-4
4h30m
0.284s
Newton-PINet (Ours)
50
1.72e-7
301s
0.080s
Table 4: Model comparison on a 1D nonlinear reaction-diffusion problem.
Appendix figures & tables38 assets
Supplementary material from the paper’s appendix.
Appendix
Newton
Picard
Iter.
∥Aw−b∥
EOC
MSE
∥Aw−b∥
EOC
MSE
1
1.27×10−2
–
4.91×10−2
8.64×10−3
–
2.65×10−2
2
5.62×10−2
–
3.28×10−3
6.30×10−3
–
4.48×10−3
3
7.26×10−3
1.98
2.71×10−5
6.01×10−3
1.26
5.15×10−4
4
2.98×10−3
1.73
4.69×10−7
4.54×10−3
1.26
5.20×10−5
5
2.98×10−3
1.85
4.59×10−7
4.08×10−3
1.16
1.50×10−5
Appendix
Table 5: Convergence process comparison between Newton and Picard iterations
Newton
Picard
Initial guess
∥u0−ugt∥2
∥Aw−b∥
EOC
MSE
∥Aw−b∥
EOC
MSE
u(t)=ut=0
28.59
2.98×10−3
1.85
4.59×10−7
4.08×10−3
1.16
1.50×10−5
Zero field
48.93
2.98×10−3
1.63
4.62×10−7
4.60×10−3
1.26
5.19×10−5
Constant field
94.73
1.33
0.53
1.53×10−1
4.07×10−1
0.87
6.01×10−2
Appendix
Table 6: Impact of initial guess on the convergence
Newton
Picard
Scale
λreg
EOC
MSE
EOC
MSE
1×
7.58×10−4
1.85
4.59×10−7
1.16
1.50×10−5
10×
7.58×10−3
1.80
8.90×10−7
1.16
1.45×10−5
102×
7.58×10−2
1.55
1.27×10−5
1.15
3.22×10−5
103×
7.58×10−1
0.93
3.19×10−4
1.08
4.52×10−4
104×
7.58
0.85
6.64×10−3
0.96
5.99×10−3
Appendix
Table 7: Impact of λreg on convergence order (EOC) and MSE
Problem
Mesh size (test)
Dim. npde+nic+nbc ; No. A rows (test)
Dim. w ; No. A columns
No. train/ test tasks
Nonlinear iteration
No. time blocks
Burgers’ (1D + time)
51×129
6810
512
16/34
6
1
Generalized KdV (1D + time)
101×257
4402
1024
10/35
6
6
Kuramoto-Sivashinsky (1D + time)
63×129
649
1024
100/50
4
15
Convection-diffusion- reaction (1D + time)
101×129
3404
512
40/160
4
4
Klein-Gordon (1D + time)
32×32 (128×128)
1148 (16892)
512
16/64
4
1
Hyperbolic heat (1D + time)
32×32 (128×128)
1148 (16892)
900
16/64
4
1
Appendix
Table 8: Summary of meta-learning parameter configurations on nonlinear PDEs in this study. For time-dependent PDEs, the mesh size is reported as t×x , whereas for time-independent PDEs it is reported as x×y(×z) . “Mesh size” and “No. A rows” refer to settings in the meta-learning stage unless otherwise marked with “(test)”, which denotes the inference stage. “Dim.” denotes the dimension, and “No.” stands for “number of”. “–” indicates that the time-blocking strategy is not applicable for time-independent problems.
Figure 4: MSE distributions across all test tasks for different models on the generalized KdV and K-S problems.
Figure 5: Prediction results of Newton-PINet for the generalized KdV problem.
Figure 6: Prediction results of Newton-PINet for the K-S problem.
Figure 7: Prediction result of Newton-PINet for the 2D lid-driven cavity problem. Velocity magnitude is computed as V=u2+v2 .
Figure 8: Prediction results of Newton-PINet for the CDR problem.
Figure 9: Prediction results of Newton-PINet for the other 5 nonlinear PDE problems.
Figure 10: Prediction results of Newton-PINet for the 3D parametric diffusion-reaction problem.
Dirichlet BC
Mixed BC
Inference time (s)
0.067
0.071
MSE
(1.42±4.54)×10−5
(1.49±4.48)×10−5
Relative L2
(5.11±11.30)×10−3
(5.47±11.30)×10−3
Appendix
Table 9: Model generalization to unseen boundary condition (BC) for the 1D+time hyperbolic heat problem.
Metric
Value
Inference time (s)
0.074
MSE
(1.50±3.23)×10−5
Relative L2
(6.12±11.40)×10−3
Appendix
Table 10: Generalization to unseen irregular geometries and parameters for the 1D+time hyperbolic heat problem.
Figure 11: Prediction results of Newton-PINet for the hyperbolic heat problem across varying domain shapes and PDE parameters, with meta-training performed on regular geometry.
Metric
Value
Inference time (s)
0.104
MSE
(5.56±7.39)×10−6
Relative L2
(9.21±4.09)×10−3
Appendix
Table 11: Generalization to unseen irregular geometries and parameters for the 3D parametric diffusion-reaction problem.
Figure 12: Prediction results of Newton-PINet for the 3D parametric diffusion-reaction equation across varying domain shapes and PDE parameters, with meta-training performed on regular geometry.
Model
No. training tasks
Test relative L2 error (training time)
γ=0.1
γ=0.01
γ=0.001
γ=0.0001
Unsupervised
PO-DeepONet ( Wang et al., 2021 )
1000
–
1.38e-2
2.16e-1
2.48e-1
(9.25h)
–
–
Newton-PINet (Ours)
50
1.33e-3
3.51e-3
1.37e-1
2.01e-1
(291s)
(291s)
(291s)
(291s)
Supervised
DeepONet ( Wei et al., 2025b )
1000
–
1.17e-2
2.30e-1
2.88e-1
Appendix
Table 12: Model comparison on the 1D Burgers’ problem. The initial conditions for generating all the training and test data are drawn from GRF∼N(0,625(−Δ+25I)−4) . The lowest errors are highlighted in bold. “–” denotes results not reported in the references.
Model
No. training tasks
Test relative MSE
Training time
Inference time
Unsupervised
PINNs+L-BFGS
800
6.13e-1
–
369s
PINNs-multi-opt
7.57e-1
–
16.5s
PPINNs
3.94e-1
4h15m
0.291s
P 2 INNs
5.69e-1
11h
0.676s
PO-DeepONet
4.10e-1
3h30m
0.438s
Hybrid
PI-DeepONet
800
7.90e-2
3h30m
0.443s
Appendix
Table 13: Model comparison on the 1D nonlinear reaction-diffusion problem. Baseline results are from Boudec et al. (2025) . Time is reported in hours (h), minutes (m), and seconds (s). Best and second-best are bold and underlined, respectively.
Figure 13: Prediction results of Newton-PINet for the 1D nonlinear reaction-diffusion benchmark problem.
Model
No. training tasks
Test relative MSE
Training time
Inference time
PI-neural-solver (A6000-GPU)
800
2.91e-4
4h30m
0.284s
Newton-PINet (A6000-GPU)
50
1.72e-7
301s
0.080s
Newton-PINet (V100-GPU)
50
1.71e-7
119s
0.084s
Appendix
Table 14: Comparison of model performance under different GPU configurations.
Meta-learning PINNs
Nonlinear benchmark problems (number of test tasks)
Heat (64)
Allen-Cahn (32)
Diffusion -reaction (64)
Parametric diffusion -reaction (100)
Unsupervised
Random
Error 1
0.0052
0.015
0.011
0.0022
Time 2
156
496
1073
612
MAML
Error 1
0.0045
–
–
–
Time 2
188
–
–
–
Center
Error 1
0.0045
0.012
0.0095
0.0018
Appendix
Table 15: Model comparison on nonlinear benchmark problems. Baselines are reported from Penwarden et al. (2023) ; Wong et al. (2026c) . “–” denotes results not reported in the references.
Model
Network architecture
Activation
Initialization
Optimizer / LR
Special components
DeepONet
Branch: 4 layers × 100 units
tanh
He uniform
Adam, Cosine decay
Learnable global bias
Trunk: 4 layers × 100 units
10−3→10−6
PO-DeepONet
Branch: 4 layers × 100 units
tanh
Glorot normal
Adam, Cosine decay
AD-based physics residual
Trunk: 4 layers × 100 units
10−3→10−6
MT-DeepONet
Branch: [100, 100] → 100 (latent)
ReLU
Glorot normal
Adam, Cosine decay
Layer normalization
Trunk: [100, 100, 100] → 100 (latent)
Leaky ReLU
10−3→10−6
No activation on final layer
Appendix
Table 16: Summary of baseline model architectures and hyperparameter configurations.
Problem
Model
No. model parameters
No. training / test tasks
Training time (s)
Test MSE
Generalized KdV
DeepONet
87101
10 / 35
642
8.24e-2
PO-DeepONet
87101
4437
5.01e-1
MT-DeepONet
77400
588
9.13e-2
FNO
2376321
639
6.12e-2
PINO
2376321
657
9.63e-2
Tucker-FNO
1067137
954
9.12e-2
Appendix
Table 17: Model comparison on the newly introduced benchmarks. Best and second-best results are highlighted in bold and underlined, respectively.
Method
γ (test tasks of Burgers’ problem)
0.001
0.003
0.004
0.006
…
0.047
Meta-learning
λreg
7.57e-4
MSE
3.30e-5
2.07e-6
1.44e-6
4.59e-7
…
7.54e-8
Per-task optimization
λreg
2.07e-5
4.34e-4
4.81e-4
3.26e-4
…
9.79e-5
MSE
3.21e-5
2.01e-6
1.44e-6
4.69e-7
…
7.07e-8
Appendix
Table 18: Sensitivity of the meta-learned λreg to task adaptation in test inference. For meta-learning, λreg is learned during meta-training and kept fixed during testing. For Per-task optimization, λreg is initialized from the meta-learned model parameters and then optimized for each test task through gradient-based updates.
Problem
Metrics
Initial guess for the nonlinear iteration ( uguess )
Ground truth
All ut=0 (ours)
All zeros
All ones
Normal random
Burgers’
Distance
0
26.4
45.9
93.2
93.6
MSE
1.24e-6
1.24e-6
1.24e-6
1.11e-04
4.84e-3
Relative L2
1.04e-3
1.04e-3
1.04e-3
9.61e-03
4.96e-2
K-S
Distance
0
3.3
13.3
23.9
24.4
MSE
2.84e-2
2.83e-2
2.91e-2
2.91e-2
3.08e-2
Appendix
Table 19: Effect of different initial guesses for the nonlinear iterations on test errors.
Figure 14: Convergence comparison of single-task generalization starting from the true field versus u0 as initial guesses.
Do=1
2
3
Number of A columns
512
1024
1536
Inference time (s)
0.25
0.31
0.40
Appendix
Table 20: Inference time versus Do , with Nn=128 and L=5 fixed.
Nn=64
128
256
512
Number of A columns
768
1536
3072
6144
Inference time (s)
0.29
0.40
0.76
2.11
Appendix
Table 21: Inference time versus Nn , with Do=3 and L=5 fixed.
Metrics
Number of blocks
1
2
4
6
10
15
20
Test MSE
3.56e-3
4.46e-3
9.57e-6
2.93e-6
9.07e-7
1.05e-6
5.78e-7
Test relative L2
2.23e-1
3.96e-2
3.56e-3
1.80e-3
1.13e-3
1.12e-3
1.08e-3
Inference time per task (s)
3.74e-2
6.82e-2
1.01e-1
4.04e-1
6.77e-1
1.26
1.66
Appendix
Table 22: Ablation study for different time-blocking strategies.
Errors
Weight parameter for initial condition ( λic )
1
1.5
2
5
10
Test MSE
9.57e-6
6.56e-6
5.75e-6
5.77e-6
6.46e-6
Test relative L2
3.56e-3
2.89e-3
2.67e-3
2.67e-3
2.80e-3
Appendix
Table 23: Effect of initial condition (IC) weight in the Tikhonov regularization on test errors.
Figure 15: Mitigation of drift at block boundaries via adjustment of the initial condition (IC) weight in the Tikhonov regularization (4-time-block for CDR inference).
Problem
Dirichlet
Neumann
Mixed
Time 1
MSE
Rel. L2
Time 1
MSE
Rel. L2
Time 1
MSE
Rel. L2
Heat
0.067
1.69e-6
1.76e-3
0.071
1.80e-5
5.84e-3
0.086
5.35e-5
1.04e-2
Klein-Gordon
0.064
2.36e-7
3.86e-4
0.084
2.34e-6
1.16e-3
0.082
4.05e-6
1.13e-3
Appendix
Table 24: Model performance under different boundary conditions.
Problem
Inference time per test task (s)
Test MSE
Test Relative L2
2D LDC
0.40
2.30e-4
6.04e-2
Appendix
Table 25: Model performance for 2D lid-driven cavity (LDC) under discontinuous boundary conditions.
Problem
Skip connections
Network depth
1 layer
2 layers
3 layers
4 layers
Burgers’
w/o
1.04e-2
3.75e-4
4.77e-5
5.02e-5
± 1.22e-2
± 1.50e-3
± 1.20e-4
± 1.05e-4
w/
1.04e-2
1.43e-5
1.93e-6
1.79e-6
± 1.22e-2
± 7.21e-5
± 9.24e-6
± 7.98e-6
CDR
w/o
4.70e-3
7.51e-5
6.80e-4
2.12e-3
Appendix
Table 26: Test relative L2 errors of model with (w/) and without (w/o) skip connections under different network depths.
Problem
Skip connections
Number of nodes in each layer
256 256 256 256
128 128 128 128
64 64 64 64
32 32 32 32
16 16 16 16
16 32 64 128
128 64 32 16
Burgers’
w/o
9.13e-6
5.02e-5
1.01e-3
1.24e-3
1.77e-3
4.87e-4
1.14e-3
± 1.87e-5
± 1.05e-4
± 2.06e-3
± 2.12e-3
± 2.38e-3
± 8.03e-4
± 1.95e-3
w/
2.19e-6
1.79e-6
4.12e-6
8.14e-5
2.03e-3
3.17e-4
5.41e-6
± 7.43e-6
± 7.98e-6
± 1.44e-5
± 1.11e-4
± 2.78e-3
± 5.66e-4
± 1.66e-5
CDR
w/o
3.88e-4
2.12e-3
4.80e-3
5.98e-2
1.05e-1
1.34e-3
1.24e-1
Appendix
Table 27: Test relative L2 errors with (w/) and without (w/o) skip connections under different network widths (from top to bottom, corresponding to layer 1 ∼ 4).
Problem
Skip connections
Mesh resolution ( t,x )
11 × 29
13 × 33
17 × 43
26 × 65
51 × 129
Burgers’
w/o
4.17e-5
7.19e-6
2.23e-5
1.27e-5
5.02e-5
± 4.57e-5
± 1.52e-5
± 2.30e-5
± 1.74e-5
± 1.05e-4
w/
2.57e-4
5.54e-6
2.80e-5
1.49e-6
1.79e-6
± 7.24e-4
± 1.62e-5
± 4.61e-6
± 7.00e-6
± 7.98e-6
CDR
11 × 29
21 × 49
41 × 69
71 × 99
101 × 129
Appendix
Table 28: Test relative L2 errors with (w/) and without (w/o) skip connections under different mesh resolutions
Problem
Activation function
Network depth
1 layer
2 layers
3 layers
4 layers
Burgers’
tanh
1.07e-2
1.30e-4
6.39e-4
7.13e-5
± 1.30e-2
± 2.42e-4
± 2.54e-3
± 1.60e-4
Gaussian
1.14e-2
4.64e-3
8.27e-6
3.28e-6
± 1.31e-2
± 8.95e-3
± 3.06e-5
± 1.26e-5
sin
1.04e-2
1.43e-5
1.93e-6
1.79e-6
Appendix
Table 29: Test relative L2 errors of models with different activation functions under varying network depths
Problem
Weight initialization
Initial learning rate
0.0005
0.005
0.05
Burgers’
Xavier
7.57e-5
2.83e-6
2.09e-3
± 1.62e-4
± 9.33e-6
± 5.54e-3
He
1.57e-5
1.79e-6
2.69e-3
± 7.89e-5
± 7.98e-6
± 4.93e-3
CDR
Xavier
2.62e-2
5.79e-4
1.11e-3
Appendix
Table 30: Test relative L2 errors under different weight initialization methods and initial learning rates
We propose a linearized Physics-Informed Neural Network (lPINN), a reduced-order neural basis method for forward and inverse differential equations. In an offline stage, lPINN learns operator-compatible continuous neural basis functions from an ensemble of numerical solutions. The basis functions are differentiable through automatic differentiation and are pretrained using solution data together with either derivative information or physics residuals. For each new problem instance, the basis functions are frozen and the solution is obtained by minimizing the governing-equation residual together with applicable initial, boundary, regularization, and observational terms. Unlike surrogate and operator-learning methods, the training data define the trial space offline, while the instance-specific solution is computed online by enforcing the governing physics. Relative to vanilla PINNs, lPINN pretrains the nonlinear hidden-layer representation offline and performs online inference only in the final linear layer. We evaluate lPINN on forward and inverse problems for the advection-diffusion equation, Burgers' equation, and the nonlinear pendulum equation. Compared with vanilla PINNs, lPINN achieves lower solution and parameter errors while reducing online inference times by approximately one to more than three orders of magnitude, with the largest gains generally observed for limited residual or measurement data. Cross-resolution experiments show that the learned continuous representation can be evaluated on finer meshes without retraining and with nearly unchanged accuracy.
Wenhao Chen, Alexandre M. Tartakovsky
Civil and Environmental Engineering Department University of Illinois Urbana-Champaign Urbana, IL 61801, USA · Pacific Northwest National Laboratory Richland, WA 99352, USA
Physics-informed neural networks (PINNs) have emerged as a promising mesh-free paradigm for solving partial differential equations, yet adoption in science and engineering is limited by slow training and modest accuracy relative to modern numerical solvers. We introduce the Sequential Correction Algorithm for Learning Efficient PINN (Scale-PINN), a learning strategy that bridges modern physics-informed learning with numerical algorithms. Scale-PINN incorporates the iterative residual-correction principle, a cornerstone of numerical solvers, directly into the loss formulation, marking a paradigm shift in how PINN losses can be conceived and constructed. This integration enables Scale-PINN to achieve unprecedented convergence speed across PDE problems from different physics domain, including reducing training time on a challenging fluid-dynamics problem for state-of-the-art PINN from hours to sub-2 minutes while maintaining superior accuracy, and enabling application to representative problems in aerodynamics and urban science. By uniting the rigor of numerical methods with the flexibility of deep learning, Scale-PINN marks a significant leap toward the practical adoption of PINNs in science and engineering through scalable, physics-informed learning. Codes are available at https://github.com/chiuph/SCALE-PINN.
Pao-Hsiung Chiu, Jian Cheng Wong, Chin Chun Ooi +3
Institute of High Performance Computing, A*STAR, Singapore. · Centre for Frontier AI Research, A*STAR, Singapore. · School of Mechanical Engineering, Tianjin University, China. +1
Physics-informed neural networks (PINNs) are a versatile tool in the burgeoning field of scientific machine learning for solving partial differential equations (PDEs). However, determining suitable training strategies for them is not obvious, with the result that they typically yield moderately accurate solutions. In this article, we propose a method for improving the accuracy of PINNs by coupling them with a post-processing step that seeks the best approximation in a function space associated with the network. We find that our method yields errors four to five orders of magnitude lower than those of the parent PINNs across architectures and dimensions. Moreover, we can reuse the basis functions for the linear space in more complex settings, such as time-dependent and nonlinear problems, allowing for transfer learning. Our approach also provides a residual-based metric that allows us to optimally choose the number of basis functions employed.
Saad Qadeer, Panos Stinis
Advanced Computing, Mathematics and Data Division, Pacific Northwest National Laboratory, Richland, WA