Physics-informed neural networks (PINNs) provide a learning-based framework for solving partial differential equations (PDEs), yet their training behavior can change substantially throughout optimization. Residual distributions, gradient interactions, regional learning difficulty, and model-capacity requirements may evolve over time, while the network architecture and major training mechanisms are typically determined before training. We propose PINNMorph, an online PINN adaptation framework based on large language model (LLM)-guided policy evolution. PINNMorph maintains a population of state-conditioned adaptation policies that map execution diagnostics to controlled interventions over topology modification, additive representation augmentation, objective balancing, gradient handling, adaptive sampling, and optimizer-phase control. At each intervention opportunity, candidate programs are instantiated from the current policy population, selected according to the observed training state, and applied directly to the PINN under training. The resulting model inherits its existing parameters and training state and continues optimization along the same trajectory. Execution outcomes are subsequently used to evaluate interventions and evolve the policy population. Unlike pre-training architecture search or fixed adaptation rules, PINNMorph jointly adapts the current PINN and the policies governing its interventions using feedback from actual training. Experiments on 13 PDE benchmarks show that PINNMorph achieves lower solution errors than SA-PINN, ConFIG, RoPINN, HARMONIC, and PINNsAgent across all evaluated problems. Ablation studies further examine the effects of online adaptation, state-conditioned intervention selection, and execution-feedback-driven policy evolution.
Figures & tables
Figure 1: Overview of PINNMorph. At each adaptation point, PINNMorph diagnoses the current training state, uses a policy population to propose candidate actions, selects an executable action based on validity and state-conditioned scoring, and applies it online to the PINN under training. The resulting execution feedback is audited against the estimated no-intervention trajectory and used to evolve the policy population for subsequent adaptations.
PDE
SA-PINN
ConFIG
RoPINN
HARMONIC
PINNsAgent
PINNMorph
KS 1D
6.40E+0 ± 4.63E-2
1.47E+1 ± 4.60E-1
6.37E+0 ± 5.01E-1
5.55E+1 ± 5.88E+0
1.12E+0 ± 3.27E-2
1.05E+0 ± 3.44E-1
Burgers 1D
4.77E-4 ± 3.08E-5
2.47E-4 ± 1.27E-4
6.31E-4 ± 6.35E-5
1.32E-4 ± 2.13E-5
7.24E-5 ± 1.49E-5
6.48E-5 ± 3.29E-5
NS 2D CG
7.10E-3 ± 3.11E-4
2.46E-2 ± 7.81E-3
9.91E-3 ± 5.27E-3
5.34E-3 ± 4.84E-4
4.44E-3 ± 1.30E-3
1.47E-3 ± 4.13E-4
Heat 2D CG
3.41E-3 ± 6.07E-5
1.60E-2 ± 3.96E-3
4.92E-3 ± 8.93E-4
2.87E-2 ± 4.96E-4
3.61E-3 ± 8.71E-4
1.81E-3 ± 4.48E-4
Heat 2D MS
3.03E-4 ± 2.25E-5
3.18E-4 ± 2.99E-5
3.95E-4 ± 8.06E-6
4.20E-4 ± 6.47E-5
3.13E-4 ± 2.59E-5
2.34E-4 ± 2.21E-5
Heat 2D VC
1.45E-2 ± 5.17E-3
3.52E-2 ± 4.91E-3
4.70E-1 ± 4.86E-3
9.95E-2 ± 3.86E-2
5.62E-3 ± 3.33E-3
1.82E-3 ± 2.54E-4
Table 1: Solution MSE on 13 PDE benchmarks. Results are reported as mean ± standard deviation over five independent runs. Lower is better.
Variant
Online Adaptation
State-Conditioned Selection
Policy Evolution
Feedback Evolution
No Morph
×
×
×
×
Random Selection
✓
×
✓
✓
Random Evolution
✓
✓
✓
×
PINNMorph
✓
✓
✓
✓
Table 2: Component configurations used in the ablation study.
PDE
No Morph
Random Selection
Random Evolution
PINNMorph
KS 1D
5.21E+0 ± 1.76E+0
6.24E+0 ± 1.87E+0
5.03E+0 ± 5.74E-1
1.05E+0 ± 3.44E-1
Burgers 1D
6.82E-4 ± 5.05E-4
3.62E-4 ± 1.35E-4
7.92E-4 ± 4.96E-4
6.48E-5 ± 3.29E-5
NS 2D CG
8.72E-3 ± 3.38E-3
9.46E-2 ± 1.10E-2
4.39E-3 ± 1.86E-3
1.47E-3 ± 4.13E-4
Heat 2D CG
5.17E-3 ± 6.43E-4
1.95E-2 ± 4.83E-3
4.01E-2 ± 5.46E-3
1.81E-3 ± 4.48E-4
Heat 2D MS
4.26E-3 ± 3.91E-4
3.89E-3 ± 2.72E-4
4.43E-3 ± 8.00E-5
2.34E-4 ± 2.21E-5
Heat 2D VC
3.43E-2 ± 1.67E-2
4.20E-2 ± 1.46E-2
4.94E-3 ± 2.07E-3
1.82E-3 ± 2.54E-4
Table 3: Component ablation results. Results are reported as MSE (mean ± standard deviation) over five independent runs. Lower is better.
Figure 2: Representative training trajectories of PINNMorph. Training loss and the Decision and Audit physics scores are tracked throughout optimization, with the two physics scores computed on the fixed decision and audit sets, respectively.
Training state
Diagnostic signal
Action
Before → After
Localized error
Shock-localized residual
Localized Expert
1.01×10−2→6.32×10−4
Representation limit
Non-local residual
Fourier Branch
3.71×10−3→2.59×10−4
Spatial imbalance
Residual imbalance
Adaptive Sampling
3.08×10−2→7.63×10−3
Gradient conflict
Inter-equation conflict
Topology Split
1.36×10−3→6.53×10−4
Spatial imbalance
High-D residual imbalance
Adaptive Sampling
2.26×10−3→8.08×10−4
Optimization plateau
Stable gradients
Correction Branch
4.86×10−3→1.46×10−3
Table 4: Representative online adaptations of PINNMorph and their effects under different training states.
Variant
LLM Calls
LLM Tokens
Wall Time (s)
Median MSE Ratio
No Morph
0
0
1250
12.65 ×
Random Selection
30
6.52×104
1273
10.77 ×
Random Evolution
33
1.06×105
1286
10.30 ×
PINNMorph
38
4.40×105
1308
1.00 ×
Table 5: Computational cost and median MSE ratios of PINNMorph and its ablated variants. All variants use the same 20,000 -step optimization budget. Median MSE ratios are computed relative to PINNMorph.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Group
Mechanism
Target and execution rule
Parameters
Topology
Widen
Replicate units in selected hidden layers and divide outgoing weights by replication count (Net2Wider map).
Width fraction; layers.
Topology
Regrow
Expand selected hidden layers through the same function-preserving duplication map.
Width fraction; layers.
Topology
Add layer
Insert a new hidden layer after a selected layer; initialize its weights and continue the live model.
Layer index.
Topology
Prune
Remove low-utility hidden units while respecting a minimum width.
Width fraction; minimum width; layers.
Topology
Redistribute
Remove donor units and add capacity elsewhere in the hidden stack.
Move fraction; minimum width; layers.
Topology
Local reinitialization
Reinitialize a bounded fraction of units in selected hidden layers.
Reset fraction; layers.
Appendix
Table 6: Executable online mechanisms. “Zero output” means that the added module leaves the current function unchanged at insertion. Numeric arguments are optional; the executor derives defaults from the validated strength and diagnostic state.
Neural PDE solvers use trainable models to approximate solutions or solution operators for partial differential equations, often with physical constraints.
Physics-informed neural networks (PINNs) require coordinated choices across representation, architecture, sampling, constraints, loss construction, and optimization, yet effective configurations can vary substantially across partial differential equations (PDEs). Existing automated design methods search over these choices, but training outcomes are still used primarily to rank candidates rather than to inform subsequent designs. We develop PINNForge, an execution-grounded large language model (LLM)-driven evolutionary framework that treats observed PINN training behavior as a cross-generation design signal. PINNForge initializes diverse configurations from PDE-related prior knowledge, executes them, retains globally strong configurations, and feeds their execution evidence and accumulated run-level experience back to the LLM. The LLM then revises and recombines coupled design components or explores new combinations within a generate--execute--evaluate--evolve loop. Across 25 PDE benchmarks, PINNForge achieves the lowest mean MSE on 24 tasks among RoPINN, PINNsFormer, PINNsAgent, and PINNForge. Removing knowledge guidance, execution feedback, or evolutionary search increases the median task-wise MSE ratio to 3.74×, 12.10×, and 10.10×, respectively, relative to full PINNForge.
Mingyang Yu, Xu Yang, Jun Zhang +2
Nankai University · East China Normal University · Hanyang University
Amortizing physics-informed neural networks (PINNs) across related PDEs requires describing each equation to a reusable solver. Coefficient vectors encode numerical parameters in predefined slots, leaving operator and cross-field assignments implicit. We make these relationships explicit in an operator graph, with nodes for fields, derivatives, terms, and residuals and coefficients retained as term attributes. A graph hypernetwork generates diagonal codes that initialize a meta-trained factorized PINN for each target equation. Meta-training and target-specific adaptation use governing equations and prescribed conditions without solution labels. We compare coefficient-vector, DeepSets-based term-set, and graph conditioning by solution accuracy within a fixed adaptation budget. In scalar convection-diffusion-reaction problems, both term-based descriptors improve high-reaction accuracy, with similar performance. In two-field Fisher-KPP, meta-training sees uncoupled and one-way systems; after 3,000 adaptation steps on unseen two-way coupling, the graph's mean final error is 35.7% below the term set and 67.7% below the coefficient vector. In a fixed-structure capacitively coupled plasma model, the coefficient vector performs best. These results support extending coefficient conditioning with explicit equation relationships for physics-based solver adaptation.
Physics-informed neural networks (PINNs) are a versatile tool in the burgeoning field of scientific machine learning for solving partial differential equations (PDEs). However, determining suitable training strategies for them is not obvious, with the result that they typically yield moderately accurate solutions. In this article, we propose a method for improving the accuracy of PINNs by coupling them with a post-processing step that seeks the best approximation in a function space associated with the network. We find that our method yields errors four to five orders of magnitude lower than those of the parent PINNs across architectures and dimensions. Moreover, we can reuse the basis functions for the linear space in more complex settings, such as time-dependent and nonlinear problems, allowing for transfer learning. Our approach also provides a residual-based metric that allows us to optimally choose the number of basis functions employed.
Saad Qadeer, Panos Stinis
Advanced Computing, Mathematics and Data Division, Pacific Northwest National Laboratory, Richland, WA