Scientific machine learning has opened new avenues for solving parameterized partial differential equations (PDEs), enabling models to learn a family of PDEs and generalize to unseen instances. In this context, data-driven operator learning methods typically require large training datasets, while physics-informed neural networks (PINNs) suffer from difficult optimization and limited generalization, especially for nonlinear PDEs. We propose Newton-PINet, a physics-informed network enhanced by Newton linearization, offering an effective meta-learning framework for nonlinear PDEs. Newton-PINet (i) employs a physics-informed multilayer network with skip connections, where the output-layer weights are solved by least squares; (ii) adopts a two-stage learning strategy that first leverages gradient-based training to learn robust representations from the available training tasks, and then performs gradient-free fine-tuning on the output layer for fast task-specific generalization; and (iii) incorporates a Newton linearization method to speed up the least-squares iteration for nonlinear PDE problems. On a challenging nonlinear reaction-diffusion benchmark, Newton-PINet achieves up to three orders of magnitude lower relative error than recent neural solvers, while using 16x fewer training tasks and over an order of magnitude less training time (about 5 minutes versus several hours). This work advances the meta-learning of PINNs toward data-efficient, fast, and generalizable physics solvers. The code and datasets are publicly available at: https://github.com/fycAIwj/Newton-PINet.
Figures & tables
Figure 1: Newton-PINet model framework. Stage 1 (pre-training) meta-learns the task-agnostic parameters Θ={w~,λpde,λreg} by gradient descent (outer loop), while the output-layer weights w of each training task are updated by gradient-free Tikhonov-regularized least squares (inner loop); Stage 2 (inference) freezes Θ and adapts only w to new tasks. The bottom panel compares Picard and Newton linearization; the latter accelerates the convergence of nonlinear iterations for solving nonlinear PDEs.
Figure 2
Nonlinear term
Newton-linearized expression
(um)k+1
m(uk)m−1uk+1+(1−m)(uk)m
(umux)k+1
m(uk)m−1uxkuk+1+(uk)muxk+1−m(uk)muxk
[sinh(u)]k+1
cosh(uk)uk+1+sinh(uk)−cosh(uk)uk
[exp(u)]k+1
exp(uk)uk+1+exp(uk)−exp(uk)uk
[uln(u)]k+1
(ln(uk)+1)uk+1+ukln(uk)−(ln(uk)+1)uk
Table 1: Newton-linearized expressions of several nonlinear terms derived by our work. m denotes the exponent. k and k+1 refer to the current and next nonlinear iterations, respectively.
Problem
Meta-learning epoch
Meta-learning time cost
Inference time per test task
Test MSE of models
DNN
PINet
Newton-PINet
Burgers’
1000
286s
0.035s
1.06e-4
1.10e-4
1.79e-6
Generalized KdV
1000
453s
0.359s
3.74e-2
1.49e-2
1.94e-3
Kuramoto-Sivashinsky
3000
1014s
0.552s
5.03e-1
106
3.45e-2
Convection-diffusion- reaction
1000
203s
0.099s
5.27e-1
4.65e-3
4.32e-5
Klein-Gordon
1000
105s
0.065s
1.04
1.88e-7
2.36e-7
Table 2: Summary of model performance on nonlinear PDEs in this study. The reported meta-learning and fine-tuning time correspond to the Newton-PINet. “NaN” entries under PINet indicate numerical overflow during computation. See Appendix D.1 for detailed descriptions of all PDE problems and meta-learning configurations (summarized in Table 8 ). See Appendix D.2 for details on data generation, error metrics, and computational setup.
Figure 2: Burgers’ problem. (a) Meta-learning convergence of Newton-PINet vs. PINet under different nonlinear iterations (N) (lines and shaded areas: the median convergence path and interquartile ranges of 5 random-seed runs). (b) Boxplot of error distributions across all test tasks and predictions by Newton-PINet (N=6).
Unsupervised
Supervised
Hybrid
PO-DeepONet
Newton-PINet
DeepONet
T-DeepONet-TF
PINO
Newton-PINet
No. training tasks
1000
50
1000
1000
1000
50
Test relative L2 error
1.38e-2
3.51e-3
1.17e-2
2.08e-3
3.80e-3
5.46e-4
Training time
9.25h
291s
2800s
3041s
1200s
133s
Table 3: Comparison of training cost and performance at γ=0.01 on Burgers’ problem.
Figure 3: Convection-diffusion-reaction (CDR) problem. (a) Meta-learning convergence of Newton-PINet vs. PINet under different nonlinear iterations (N). (b,c) Test task MSE and runtime aggregated from 5 runs. In (b), the two circled regions correspond to Newton-PINet (N=4) and PINet (N=12), respectively. (d) Error distribution boxplot across all test tasks for different models; the prediction fields correspond to Newton-PINet (N=4).
Model
No. training tasks
Test relative MSE
Training time
Inference time
PI-neural-solver ( Boudec et al., 2025 )
800
2.91e-4
4h30m
0.284s
Newton-PINet (Ours)
50
1.72e-7
301s
0.080s
Table 4: Model comparison on a 1D nonlinear reaction-diffusion problem.
Appendix figures & tables38 assets
Supplementary material from the paper’s appendix.
Appendix
Newton
Picard
Iter.
∥Aw−b∥
EOC
MSE
∥Aw−b∥
EOC
MSE
1
1.27×10−2
–
4.91×10−2
8.64×10−3
–
2.65×10−2
2
5.62×10−2
–
3.28×10−3
6.30×10−3
–
4.48×10−3
3
7.26×10−3
1.98
2.71×10−5
6.01×10−3
1.26
5.15×10−4
4
2.98×10−3
1.73
4.69×10−7
4.54×10−3
1.26
5.20×10−5
5
2.98×10−3
1.85
4.59×10−7
4.08×10−3
1.16
1.50×10−5
Appendix
Table 5: Convergence process comparison between Newton and Picard iterations
Newton
Picard
Initial guess
∥u0−ugt∥2
∥Aw−b∥
EOC
MSE
∥Aw−b∥
EOC
MSE
u(t)=ut=0
28.59
2.98×10−3
1.85
4.59×10−7
4.08×10−3
1.16
1.50×10−5
Zero field
48.93
2.98×10−3
1.63
4.62×10−7
4.60×10−3
1.26
5.19×10−5
Constant field
94.73
1.33
0.53
1.53×10−1
4.07×10−1
0.87
6.01×10−2
Appendix
Table 6: Impact of initial guess on the convergence
Newton
Picard
Scale
λreg
EOC
MSE
EOC
MSE
1×
7.58×10−4
1.85
4.59×10−7
1.16
1.50×10−5
10×
7.58×10−3
1.80
8.90×10−7
1.16
1.45×10−5
102×
7.58×10−2
1.55
1.27×10−5
1.15
3.22×10−5
103×
7.58×10−1
0.93
3.19×10−4
1.08
4.52×10−4
104×
7.58
0.85
6.64×10−3
0.96
5.99×10−3
Appendix
Table 7: Impact of λreg on convergence order (EOC) and MSE
Problem
Mesh size (test)
Dim. npde+nic+nbc ; No. A rows (test)
Dim. w ; No. A columns
No. train/ test tasks
Nonlinear iteration
No. time blocks
Burgers’ (1D + time)
51×129
6810
512
16/34
6
1
Generalized KdV (1D + time)
101×257
4402
1024
10/35
6
6
Kuramoto-Sivashinsky (1D + time)
63×129
649
1024
100/50
4
15
Convection-diffusion- reaction (1D + time)
101×129
3404
512
40/160
4
4
Klein-Gordon (1D + time)
32×32 (128×128)
1148 (16892)
512
16/64
4
1
Hyperbolic heat (1D + time)
32×32 (128×128)
1148 (16892)
900
16/64
4
1
Appendix
Table 8: Summary of meta-learning parameter configurations on nonlinear PDEs in this study. For time-dependent PDEs, the mesh size is reported as t×x , whereas for time-independent PDEs it is reported as x×y(×z) . “Mesh size” and “No. A rows” refer to settings in the meta-learning stage unless otherwise marked with “(test)”, which denotes the inference stage. “Dim.” denotes the dimension, and “No.” stands for “number of”. “–” indicates that the time-blocking strategy is not applicable for time-independent problems.
Figure 4: MSE distributions across all test tasks for different models on the generalized KdV and K-S problems.
Figure 5: Prediction results of Newton-PINet for the generalized KdV problem.
Figure 6: Prediction results of Newton-PINet for the K-S problem.
Figure 7: Prediction result of Newton-PINet for the 2D lid-driven cavity problem. Velocity magnitude is computed as V=u2+v2 .
Figure 8: Prediction results of Newton-PINet for the CDR problem.
Figure 9: Prediction results of Newton-PINet for the other 5 nonlinear PDE problems.
Figure 10: Prediction results of Newton-PINet for the 3D parametric diffusion-reaction problem.
Dirichlet BC
Mixed BC
Inference time (s)
0.067
0.071
MSE
(1.42±4.54)×10−5
(1.49±4.48)×10−5
Relative L2
(5.11±11.30)×10−3
(5.47±11.30)×10−3
Appendix
Table 9: Model generalization to unseen boundary condition (BC) for the 1D+time hyperbolic heat problem.
Metric
Value
Inference time (s)
0.074
MSE
(1.50±3.23)×10−5
Relative L2
(6.12±11.40)×10−3
Appendix
Table 10: Generalization to unseen irregular geometries and parameters for the 1D+time hyperbolic heat problem.
Figure 11: Prediction results of Newton-PINet for the hyperbolic heat problem across varying domain shapes and PDE parameters, with meta-training performed on regular geometry.
Metric
Value
Inference time (s)
0.104
MSE
(5.56±7.39)×10−6
Relative L2
(9.21±4.09)×10−3
Appendix
Table 11: Generalization to unseen irregular geometries and parameters for the 3D parametric diffusion-reaction problem.
Figure 12: Prediction results of Newton-PINet for the 3D parametric diffusion-reaction equation across varying domain shapes and PDE parameters, with meta-training performed on regular geometry.
Model
No. training tasks
Test relative L2 error (training time)
γ=0.1
γ=0.01
γ=0.001
γ=0.0001
Unsupervised
PO-DeepONet ( Wang et al., 2021 )
1000
–
1.38e-2
2.16e-1
2.48e-1
(9.25h)
–
–
Newton-PINet (Ours)
50
1.33e-3
3.51e-3
1.37e-1
2.01e-1
(291s)
(291s)
(291s)
(291s)
Supervised
DeepONet ( Wei et al., 2025b )
1000
–
1.17e-2
2.30e-1
2.88e-1
Appendix
Table 12: Model comparison on the 1D Burgers’ problem. The initial conditions for generating all the training and test data are drawn from GRF∼N(0,625(−Δ+25I)−4) . The lowest errors are highlighted in bold. “–” denotes results not reported in the references.
Model
No. training tasks
Test relative MSE
Training time
Inference time
Unsupervised
PINNs+L-BFGS
800
6.13e-1
–
369s
PINNs-multi-opt
7.57e-1
–
16.5s
PPINNs
3.94e-1
4h15m
0.291s
P 2 INNs
5.69e-1
11h
0.676s
PO-DeepONet
4.10e-1
3h30m
0.438s
Hybrid
PI-DeepONet
800
7.90e-2
3h30m
0.443s
Appendix
Table 13: Model comparison on the 1D nonlinear reaction-diffusion problem. Baseline results are from Boudec et al. (2025) . Time is reported in hours (h), minutes (m), and seconds (s). Best and second-best are bold and underlined, respectively.
Figure 13: Prediction results of Newton-PINet for the 1D nonlinear reaction-diffusion benchmark problem.
Model
No. training tasks
Test relative MSE
Training time
Inference time
PI-neural-solver (A6000-GPU)
800
2.91e-4
4h30m
0.284s
Newton-PINet (A6000-GPU)
50
1.72e-7
301s
0.080s
Newton-PINet (V100-GPU)
50
1.71e-7
119s
0.084s
Appendix
Table 14: Comparison of model performance under different GPU configurations.
Meta-learning PINNs
Nonlinear benchmark problems (number of test tasks)
Heat (64)
Allen-Cahn (32)
Diffusion -reaction (64)
Parametric diffusion -reaction (100)
Unsupervised
Random
Error 1
0.0052
0.015
0.011
0.0022
Time 2
156
496
1073
612
MAML
Error 1
0.0045
–
–
–
Time 2
188
–
–
–
Center
Error 1
0.0045
0.012
0.0095
0.0018
Appendix
Table 15: Model comparison on nonlinear benchmark problems. Baselines are reported from Penwarden et al. (2023) ; Wong et al. (2026c) . “–” denotes results not reported in the references.
Model
Network architecture
Activation
Initialization
Optimizer / LR
Special components
DeepONet
Branch: 4 layers × 100 units
tanh
He uniform
Adam, Cosine decay
Learnable global bias
Trunk: 4 layers × 100 units
10−3→10−6
PO-DeepONet
Branch: 4 layers × 100 units
tanh
Glorot normal
Adam, Cosine decay
AD-based physics residual
Trunk: 4 layers × 100 units
10−3→10−6
MT-DeepONet
Branch: [100, 100] → 100 (latent)
ReLU
Glorot normal
Adam, Cosine decay
Layer normalization
Trunk: [100, 100, 100] → 100 (latent)
Leaky ReLU
10−3→10−6
No activation on final layer
Appendix
Table 16: Summary of baseline model architectures and hyperparameter configurations.
Problem
Model
No. model parameters
No. training / test tasks
Training time (s)
Test MSE
Generalized KdV
DeepONet
87101
10 / 35
642
8.24e-2
PO-DeepONet
87101
4437
5.01e-1
MT-DeepONet
77400
588
9.13e-2
FNO
2376321
639
6.12e-2
PINO
2376321
657
9.63e-2
Tucker-FNO
1067137
954
9.12e-2
Appendix
Table 17: Model comparison on the newly introduced benchmarks. Best and second-best results are highlighted in bold and underlined, respectively.
Method
γ (test tasks of Burgers’ problem)
0.001
0.003
0.004
0.006
…
0.047
Meta-learning
λreg
7.57e-4
MSE
3.30e-5
2.07e-6
1.44e-6
4.59e-7
…
7.54e-8
Per-task optimization
λreg
2.07e-5
4.34e-4
4.81e-4
3.26e-4
…
9.79e-5
MSE
3.21e-5
2.01e-6
1.44e-6
4.69e-7
…
7.07e-8
Appendix
Table 18: Sensitivity of the meta-learned λreg to task adaptation in test inference. For meta-learning, λreg is learned during meta-training and kept fixed during testing. For Per-task optimization, λreg is initialized from the meta-learned model parameters and then optimized for each test task through gradient-based updates.
Problem
Metrics
Initial guess for the nonlinear iteration ( uguess )
Ground truth
All ut=0 (ours)
All zeros
All ones
Normal random
Burgers’
Distance
0
26.4
45.9
93.2
93.6
MSE
1.24e-6
1.24e-6
1.24e-6
1.11e-04
4.84e-3
Relative L2
1.04e-3
1.04e-3
1.04e-3
9.61e-03
4.96e-2
K-S
Distance
0
3.3
13.3
23.9
24.4
MSE
2.84e-2
2.83e-2
2.91e-2
2.91e-2
3.08e-2
Appendix
Table 19: Effect of different initial guesses for the nonlinear iterations on test errors.
Figure 14: Convergence comparison of single-task generalization starting from the true field versus u0 as initial guesses.
Do=1
2
3
Number of A columns
512
1024
1536
Inference time (s)
0.25
0.31
0.40
Appendix
Table 20: Inference time versus Do , with Nn=128 and L=5 fixed.
Nn=64
128
256
512
Number of A columns
768
1536
3072
6144
Inference time (s)
0.29
0.40
0.76
2.11
Appendix
Table 21: Inference time versus Nn , with Do=3 and L=5 fixed.
Metrics
Number of blocks
1
2
4
6
10
15
20
Test MSE
3.56e-3
4.46e-3
9.57e-6
2.93e-6
9.07e-7
1.05e-6
5.78e-7
Test relative L2
2.23e-1
3.96e-2
3.56e-3
1.80e-3
1.13e-3
1.12e-3
1.08e-3
Inference time per task (s)
3.74e-2
6.82e-2
1.01e-1
4.04e-1
6.77e-1
1.26
1.66
Appendix
Table 22: Ablation study for different time-blocking strategies.
Errors
Weight parameter for initial condition ( λic )
1
1.5
2
5
10
Test MSE
9.57e-6
6.56e-6
5.75e-6
5.77e-6
6.46e-6
Test relative L2
3.56e-3
2.89e-3
2.67e-3
2.67e-3
2.80e-3
Appendix
Table 23: Effect of initial condition (IC) weight in the Tikhonov regularization on test errors.
Figure 15: Mitigation of drift at block boundaries via adjustment of the initial condition (IC) weight in the Tikhonov regularization (4-time-block for CDR inference).
Problem
Dirichlet
Neumann
Mixed
Time 1
MSE
Rel. L2
Time 1
MSE
Rel. L2
Time 1
MSE
Rel. L2
Heat
0.067
1.69e-6
1.76e-3
0.071
1.80e-5
5.84e-3
0.086
5.35e-5
1.04e-2
Klein-Gordon
0.064
2.36e-7
3.86e-4
0.084
2.34e-6
1.16e-3
0.082
4.05e-6
1.13e-3
Appendix
Table 24: Model performance under different boundary conditions.
Problem
Inference time per test task (s)
Test MSE
Test Relative L2
2D LDC
0.40
2.30e-4
6.04e-2
Appendix
Table 25: Model performance for 2D lid-driven cavity (LDC) under discontinuous boundary conditions.
Problem
Skip connections
Network depth
1 layer
2 layers
3 layers
4 layers
Burgers’
w/o
1.04e-2
3.75e-4
4.77e-5
5.02e-5
± 1.22e-2
± 1.50e-3
± 1.20e-4
± 1.05e-4
w/
1.04e-2
1.43e-5
1.93e-6
1.79e-6
± 1.22e-2
± 7.21e-5
± 9.24e-6
± 7.98e-6
CDR
w/o
4.70e-3
7.51e-5
6.80e-4
2.12e-3
Appendix
Table 26: Test relative L2 errors of model with (w/) and without (w/o) skip connections under different network depths.
Problem
Skip connections
Number of nodes in each layer
256 256 256 256
128 128 128 128
64 64 64 64
32 32 32 32
16 16 16 16
16 32 64 128
128 64 32 16
Burgers’
w/o
9.13e-6
5.02e-5
1.01e-3
1.24e-3
1.77e-3
4.87e-4
1.14e-3
± 1.87e-5
± 1.05e-4
± 2.06e-3
± 2.12e-3
± 2.38e-3
± 8.03e-4
± 1.95e-3
w/
2.19e-6
1.79e-6
4.12e-6
8.14e-5
2.03e-3
3.17e-4
5.41e-6
± 7.43e-6
± 7.98e-6
± 1.44e-5
± 1.11e-4
± 2.78e-3
± 5.66e-4
± 1.66e-5
CDR
w/o
3.88e-4
2.12e-3
4.80e-3
5.98e-2
1.05e-1
1.34e-3
1.24e-1
Appendix
Table 27: Test relative L2 errors with (w/) and without (w/o) skip connections under different network widths (from top to bottom, corresponding to layer 1 ∼ 4).
Problem
Skip connections
Mesh resolution ( t,x )
11 × 29
13 × 33
17 × 43
26 × 65
51 × 129
Burgers’
w/o
4.17e-5
7.19e-6
2.23e-5
1.27e-5
5.02e-5
± 4.57e-5
± 1.52e-5
± 2.30e-5
± 1.74e-5
± 1.05e-4
w/
2.57e-4
5.54e-6
2.80e-5
1.49e-6
1.79e-6
± 7.24e-4
± 1.62e-5
± 4.61e-6
± 7.00e-6
± 7.98e-6
CDR
11 × 29
21 × 49
41 × 69
71 × 99
101 × 129
Appendix
Table 28: Test relative L2 errors with (w/) and without (w/o) skip connections under different mesh resolutions
Problem
Activation function
Network depth
1 layer
2 layers
3 layers
4 layers
Burgers’
tanh
1.07e-2
1.30e-4
6.39e-4
7.13e-5
± 1.30e-2
± 2.42e-4
± 2.54e-3
± 1.60e-4
Gaussian
1.14e-2
4.64e-3
8.27e-6
3.28e-6
± 1.31e-2
± 8.95e-3
± 3.06e-5
± 1.26e-5
sin
1.04e-2
1.43e-5
1.93e-6
1.79e-6
Appendix
Table 29: Test relative L2 errors of models with different activation functions under varying network depths
Problem
Weight initialization
Initial learning rate
0.0005
0.005
0.05
Burgers’
Xavier
7.57e-5
2.83e-6
2.09e-3
± 1.62e-4
± 9.33e-6
± 5.54e-3
He
1.57e-5
1.79e-6
2.69e-3
± 7.89e-5
± 7.98e-6
± 4.93e-3
CDR
Xavier
2.62e-2
5.79e-4
1.11e-3
Appendix
Table 30: Test relative L2 errors under different weight initialization methods and initial learning rates
Civil and Environmental Engineering Department University of Illinois Urbana-Champaign Urbana, IL 61801, USA · Pacific Northwest National Laboratory Richland, WA 99352, USA
Institute of High Performance Computing, A*STAR, Singapore. · Centre for Frontier AI Research, A*STAR, Singapore. · School of Mechanical Engineering, Tianjin University, China. +1