PINNForge: Execution-Grounded Evolutionary Design of Physics-Informed Neural Networks

Mingyang Yu; Xu Yang; Jun Zhang; Jing Xu; Keqian Li

cs.AISep 19, 2026

PINNForge: Execution-Grounded Evolutionary Design of Physics-Informed Neural Networks

Authors: Mingyang Yu, Xu Yang, Jun Zhang, Jing Xu, Keqian Li

Organizations: Nankai University · East China Normal University · Hanyang University

Abstract

Physics-informed neural networks (PINNs) require coordinated choices across representation, architecture, sampling, constraints, loss construction, and optimization, yet effective configurations can vary substantially across partial differential equations (PDEs). Existing automated design methods search over these choices, but training outcomes are still used primarily to rank candidates rather than to inform subsequent designs. We develop PINNForge, an execution-grounded large language model (LLM)-driven evolutionary framework that treats observed PINN training behavior as a cross-generation design signal. PINNForge initializes diverse configurations from PDE-related prior knowledge, executes them, retains globally strong configurations, and feeds their execution evidence and accumulated run-level experience back to the LLM. The LLM then revises and recombines coupled design components or explores new combinations within a generate--execute--evaluate--evolve loop. Across 25 PDE benchmarks, PINNForge achieves the lowest mean MSE on 24 tasks among RoPINN, PINNsFormer, PINNsAgent, and PINNForge. Removing knowledge guidance, execution feedback, or evolutionary search increases the median task-wise MSE ratio to 3.74 $\times$ , 12.10 $\times$ , and 10.10 $\times$ , respectively, relative to full PINNForge.

Figures & tables

Figure 1: Overview of the PINNForge framework.

PDE	RoPINN	PINNsFormer	PINNsAgent	PINNForge
burgers_1d	7.31E-04 $\pm$ 6.13E-05	1.08E-03 $\pm$ 8.21E-06	6.52E-05 $\pm$ 1.55E-05	6.50E-05 $\pm$ 4.42E-07
burgers_2d	2.40E-01 $\pm$ 3.12E-02	2.65E-01 $\pm$ 6.83E-02	1.98E-01 $\pm$ 1.66E-02	1.95E-01 $\pm$ 1.31E-02
gray_scott_2d	8.64E-03 $\pm$ 4.53E-05	1.57E-02 $\pm$ 1.32E-03	4.36E-03 $\pm$ 3.08E-05	3.60E-03 $\pm$ 2.65E-04
heat_2d_complex_geometry	4.00E-04 $\pm$ 9.10E-05	2.78E-03 $\pm$ 5.77E-05	1.84E-03 $\pm$ 9.95E-04	4.45E-04 $\pm$ 9.72E-05
heat_2d_long_time	2.56E+00 $\pm$ 4.53E-01	3.83E+00 $\pm$ 1.64E-01	4.32E+00 $\pm$ 2.78E+00	1.13E+00 $\pm$ 4.21E-02
heat_2d_multiscale	2.75E-05 $\pm$ 7.52E-06	5.57E-05 $\pm$ 9.93E-06	3.51E-05 $\pm$ 2.20E-05	1.19E-05 $\pm$ 2.16E-06

Table 1: Mean

\pm

standard deviation of MSE over 10 independent runs on 25 PDEs, computed on the fixed reference evaluation points. Lower is better.

Variant	Knowl. guidance	Exec. feedback	Parent evolution	Median ratio
w/o Knowledge Guidance	$\times$	$\checkmark$	$\checkmark$	$3.74\times$
w/o Execution Feedback to LLM	$\checkmark$	$\times$	$\checkmark$	$12.10\times$
w/o Evolutionary Search	$\checkmark$	$\checkmark$	$\times$	$10.10\times$
PINNForge	$\checkmark$	$\checkmark$	$\checkmark$	$1.00\times$

Table 2: Ablation summary over 12 representative PDEs. The last column reports the median task-wise MSE ratio relative to the full PINNForge; lower is better.

Method	Selection protocol	Budget
RoPINN	Best of 5 independently initialized 20K-step candidates	100K
PINNsFormer	Best of 5 independently initialized 20K-step candidates	100K
PINNsAgent	Best of 5 independently initialized 20K-step candidates	100K
PINNForge	53 $\times$ 1K LF + Top-3 $\times$ 10K HF	$\leq$ 83K

Table 3: PINN optimizer-step budget used to obtain the selected result for one PDE.

Figure 2: Search trajectories on four representative PDEs. At each generation, the candidate with the lowest low-fidelity MSE is selected, and its MSE, training loss, and PDE residual are shown. Stars mark the corresponding results after independent high-fidelity retraining of the final Global Top-3.

PDE	G0	Best LF	Final HF	G0 $\rightarrow$ HF	LF $\rightarrow$ HF
burgers_1d	3.64E-03	2.05E-04	6.52E-05	$55.81\times$	$3.15\times$
heat_2d_multiscale	3.02E-02	9.14E-05	1.01E-05	$2{,}986.95\times$	$9.05\times$
poisson_5d	3.96E-04	1.56E-07	5.65E-08	$7{,}008.58\times$	$2.77\times$
navier_stokes_2d_classic	1.86E-03	1.61E-05	2.35E-06	$792.89\times$	$6.85\times$

Table 4: Representative single-run trajectories. G0 is the best Generation-0 evaluation MSE, LF the best low-fidelity MSE, and HF the best MSE after high-fidelity retraining.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Design dimension	Representative executable choices
Representation	Coordinate transforms, Fourier or periodic features, and multiscale representations supported by the implementation.
Architecture	Network depth and width, activation functions, residual connections, and other supported topology-related choices.
Sampling	Collocation-point allocation, sampling distributions, and adaptive or localized resampling mechanisms.
Constraints	Soft constraints, exact constraint transforms when available, and supported periodic or interface handling.
Loss	PDE-residual construction, loss weighting, normalization, causal weighting, and other supported physical-loss handling.
Optimization	Optimizer choice, learning rate, gradient handling, scheduling, and staged optimization.

Appendix

Table 7

Context	Generation 0	Generation $g\geq 1$	Purpose
Target PDE description	$\checkmark$	$\checkmark$	Provides the governing equation, computational domain, IC/BC, and problem characteristics
Prior knowledge $K_{\mathcal{P}}$	$\checkmark$	$\checkmark$	Provides PINN design rationale relevant to the current PDE
Allowed search space $\Omega$	$\checkmark$	$\checkmark$	Restricts the network, sampling, constraint, loss, and optimization components that may be generated
Training / resource budget	$\checkmark$	$\checkmark$	Restricts parameter count, sampling scale, and training budget
Role / search-strategy condition	$\checkmark$	$\checkmark$	Specifies the design focus of the current candidate
Global Top-3 AlgorithmSpec	–	$\checkmark$	Provides validated high-performing parent configurations

Appendix

Table 8

Generation-0 role	Primary design emphasis
Robust reference	Construct a conservative reference design that prioritizes broadly applicable mechanisms and stable training.
Constraint / causality	Emphasize physical-constraint enforcement and, when relevant, temporal or causal structure.
Localized sampling	Emphasize local structure, collocation-point placement, and adaptive or localized refinement of difficult regions.
Representation	Emphasize network input or feature representation and its interaction with the solver architecture.
Residual / loss	Emphasize PDE-residual construction, physical-loss formulation, and loss-handling choices.
Optimization	Emphasize optimizer selection, scheduling, gradient behavior, and numerical training stability.

Appendix

Table 9

Evolution strategy	Primary evolution direction
Refine	Use the current strongest candidate as the primary parent, preserve effective existing mechanisms, and make limited adjustments targeting its main shortcomings.
Physics-Guided	Adjust sampling-, constraint-, and loss-related designs primarily according to the observed PDE residual and initial/boundary-condition errors.
Architecture-Guided	Adjust network representation, architecture, and related training configurations according to representation capacity, model capacity, and optimization behavior.
Synthesis	Combine complementary, well-performing design mechanisms from multiple Global Top-3 parents into a new complete configuration.
Novelty	Explore design combinations not yet covered by the current high-performing candidates or the existing search trajectory.

Appendix

Table 10

Variant	Prior Knowledge	Run-scoped Posterior	Execution Feedback to LLM	Parent-based Evolution
w/o Knowledge Guidance	$\times$	$\times$	$\checkmark$	$\checkmark$
w/o Execution Feedback to LLM	$\checkmark$	$\times$	$\times$	$\checkmark$
w/o Evolutionary Search	$\checkmark$	$\checkmark$	$\checkmark$	$\times$
PINNForge	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$

Appendix

Table 11

Component	Parent Configuration	Execution Evidence	Design Decision	Child Configuration
Architecture	Modified MLP, $6\times 128$ ; no residual connections	Completed the full 1,000-step training without NaN or solution collapse	Preserve. The architecture was numerically stable in the observed run	Keep Modified MLP, $6\times 128$
Constraint	Exact hard transform burgers_1d_initial_dirichlet_exact	IC error $=2.25\times 10^{-8}$ , BC error $=8.74\times 10^{-8}$	Preserve. Do not reallocate search emphasis to constraints that are already well satisfied	Keep the exact hard transform
Sampling	10,000 Sobol interior points + RAD; add 500 each time; cap at 15,000; residual exponent 2.5	Persistent residual hotspot at $x\in[-0.24,0.24]$ , $t\in[0.15,0.39]$	Modify. Strengthen local refinement while preserving global Sobol coverage	Add 600 each time; raise the cap to 16,000; keep the remaining RAD settings unchanged
Optimization	Adam 800 steps, lr $1.0\times 10^{-3}$ , one-cycle; L-BFGS 200 steps, history 100	Optimization remains numerically stable, but clear residual error remains at the end of training	Explore. Treat optimization changes as accompanying exploration rather than misclassifying a stable optimizer as a failure	Increase Adam lr to $1.2\times 10^{-3}$ , switch to step decay; L-BFGS history 120

Appendix

Table 12

Metric	Parent G6-C4	Child G7-C4
Solution MSE	$5.71\times 10^{-4}$	$2.21\times 10^{-4}$
PDE residual	$6.05\times 10^{-2}$	$2.14\times 10^{-2}$
IC error	$2.25\times 10^{-8}$	$1.25\times 10^{-8}$
BC error	$8.74\times 10^{-8}$	$2.53\times 10^{-8}$

Appendix

Table 13

Generation	Candidate	Parent(s)	Evolution Mode	Main Inherited / Modified Components
0	G0-C2	–	Independent Initialization	4 $\times$ 64 modified MLP, tanh , soft constraints, Sobol + RAD, Adam + L-BFGS
1	G1-C3	G0-C2, G0-C6, G0-C4	Multi-Parent Synthesis	Combine RAD sampling, a deeper network topology, and optimization control; expand to a 6 $\times$ 96 modified MLP
2	G2-C1	G1-C3	Single-Parent Refinement	Preserve the 6 $\times$ 96 network; increase the RAD exponent and interior-point budget
3	G3-C4	G2-C1, G2-C4	Multi-Parent Synthesis	Introduce an exact hard IC/BC transform; use fixed weighting; increase the RAD cap
4	G4-C4	G3-C4, G3-C2	Multi-Parent Synthesis	Preserve hard constraints and Sobol + RAD; widen the network to 6 $\times$ 128; add one-cycle Adam and L-BFGS recovery
5	G5-C4	G4-C4, G4-C3	Multi-Parent Synthesis	Adjust the Adam schedule, RAD capacity, and L-BFGS history

Appendix

Table 14

Setting	Value
Generation-0 candidates	8
Candidates per later generation	5
Min. / max. completed generations	4 / 10
Parent population	Global Top-3
Low-fidelity training	1K steps
High-fidelity training	10K steps

Appendix

Table 15

Generation-0 direction	Temperature
Robust reference	0.20
Constraint / causality	0.35
Localized sampling	0.40
Representation	0.45
Residual / loss	0.45
Optimization	0.40

Appendix

Table 16

Evolution strategy	Normal	Escape
Refine	0.20	0.35
Physics-Guided	0.36	0.51
Architecture-Guided	0.42	0.57
Synthesis	0.47	0.62
Novelty	0.58	0.73

Appendix

Table 17

Component	Configuration
GPU	NVIDIA GeForce RTX 4090
NVIDIA Driver	550.163.01
CUDA	12.4
Python	3.12
PyTorch	2.6.0+cu124
Precision	FP32

Appendix

Table 18

PDE Problem	RoPINN	PINNsFormer	PINNsAgent	PINNForge	Rank	PINNacle Best
burgers_1d	7.31E-04 $\pm$ 6.13E-05	1.08E-03 $\pm$ 8.21E-06	6.52E-05 $\pm$ 1.55E-05	6.50E-05 $\pm$ 4.42E-07	1	7.90E-05
burgers_2d	2.40E-01 $\pm$ 3.12E-02	2.65E-01 $\pm$ 6.83E-02	1.98E-01 $\pm$ 1.66E-02	1.95E-01 $\pm$ 1.31E-02	1	1.09E-01
gray_scott_2d	8.64E-03 $\pm$ 4.53E-05	1.57E-02 $\pm$ 1.32E-03	4.36E-03 $\pm$ 3.08E-05	3.60E-03 $\pm$ 2.65E-04	1	4.32E-03
heat_2d_complex_geometry	4.00E-04 $\pm$ 9.10E-05	2.78E-03 $\pm$ 5.77E-05	1.84E-03 $\pm$ 9.95E-04	4.45E-04 $\pm$ 9.72E-05	2	8.53E-04
heat_2d_long_time	2.56E+00 $\pm$ 4.53E-01	3.83E+00 $\pm$ 1.64E-01	4.32E+00 $\pm$ 2.78E+00	1.13E+00 $\pm$ 4.21E-02	1	1.13E+00
heat_2d_multiscale	2.75E-05 $\pm$ 7.52E-06	5.57E-05 $\pm$ 9.93E-06	3.51E-05 $\pm$ 2.20E-05	1.19E-05 $\pm$ 2.16E-06	1	5.27E-05

Appendix

Table 13: Complete MSE results on 25 PDEs over ten independent runs. Lower is better.

PDE	Characteristic	w/o Knowledge Guidance	w/o Execution Feedback to LLM	w/o Evolutionary Search	PINNForge
burgers_1d	Nonlinear / shock	7.42E-05 $\pm$ 6.31E-05	6.32E-04 $\pm$ 7.82E-05	2.78E-04 $\pm$ 3.24E-05	6.50E-05 $\pm$ 4.42E-07
burgers_2d	2D nonlinear transport	2.44E-01 $\pm$ 2.86E-02	6.71E-01 $\pm$ 8.37E-02	1.67E+00 $\pm$ 2.31E-01	1.95E-01 $\pm$ 1.31E-02
gray_scott_2d	Coupled reaction–diffusion	5.29E-03 $\pm$ 4.91E-04	5.36E-03 $\pm$ 5.62E-03	6.23E-03 $\pm$ 6.87E-04	3.60E-03 $\pm$ 2.65E-04
heat_2d_multiscale	Multiscale dynamics	3.81E-05 $\pm$ 4.28E-06	2.08E-04 $\pm$ 2.74E-04	1.30E-03 $\pm$ 1.66E-03	1.19E-05 $\pm$ 2.16E-06
heat_2d_varying_coefficient	Variable coefficient	2.18E-03 $\pm$ 2.03E-03	2.57E-03 $\pm$ 2.71E-04	4.15E-03 $\pm$ 4.86E-04	1.59E-03 $\pm$ 4.77E-04
heat_2d_complex_geometry	Complex geometry	1.90E-03 $\pm$ 2.41E-03	1.59E+00 $\pm$ 2.14E-01	1.97E-02 $\pm$ 2.77E-03	4.45E-04 $\pm$ 9.72E-05

Appendix

Table 14: Complete component-ablation results over 12 representative PDEs. Results are mean

\pm

standard deviation. Lower MSE is better.

Method	Selection Protocol	Budget
RoPINN	Best of 5 independently initialized 20K candidates	100K
PINNsFormer	Best of 5 independently initialized 20K candidates	100K
PINNsAgent	Best of 5 independently initialized 20K candidates	100K
PINNForge	53 $\times$ 1K LF + Top-3 $\times$ 10K HF	$\leq$ 83K

Appendix

Table 15: Optimizer-step budget for obtaining the selected result on one PDE.

PDE	Generation-0 Best MSE	Best LF MSE	Final HF MSE	G0 $\rightarrow$ Best LF	G0 $\rightarrow$ Final HF
burgers_1d	3.64E-03	2.05E-04 (G9)	6.52E-05	17.76 $\times$	55.81 $\times$
burgers_2d	2.49E-01	2.13E-01 (G5)	1.78E-01	1.17 $\times$	1.39 $\times$
gray_scott_2d	4.37E-03	4.03E-03 (G9)	4.07E-03	1.08 $\times$	1.07 $\times$
heat_2d_complex_geometry	1.28E-02	3.26E-03 (G8)	4.63E-04	3.92 $\times$	27.64 $\times$
heat_2d_long_time	1.13E+00	1.10E+00 (G5)	1.08E+00	1.03 $\times$	1.05 $\times$
heat_2d_multiscale	3.02E-02	9.14E-05 (G9)	1.01E-05	330.42 $\times$	2,986.95 $\times$

Appendix

Table 22

Physics-Informed ML

Physics-informed machine learning combines data-driven models with physical laws to solve, predict, and infer scientific systems.

See all 830 papers

Evolutionary Algorithm-Guided LLMs for Physics-Informed Neural Network Design
Jul 17, 2026Xu Yang, Mingyang Yu, Jing Xu +1Neural PDE Solvers Evolutionary Optimization
Shanghai Institute of Intelligent Education, East China Normal University, Shanghai 200062, China · College of Artificial Intelligence, Nankai University, Tianjin 300350, China
EvoPINN: Agentic Discovery of Executable Algorithms for Physics-Informed Neural Networks
Jul 29, 2026Peng Yin, Kai Li, Yifan Zhang +1Automated Algorithm Discovery LLM-Based Program Synthesis
School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences, Beijing, China · Institute of Automation, Chinese Academy of Sciences, Beijing, China
PINNMorph: Evolving Online Adaptation Policies for Physics-Informed Neural Networks
Sep 26, 2026Xu Yang, Mingyang Yu, Jun Zhang +2Neural PDE Solvers Physics-Informed ML
East China Normal University · Nankai University · Hanyang University

Jul 17, 2026J/K move · Enter open · S save

Evolutionary Algorithm-Guided LLMs for Physics-Informed Neural Network Design

Xu Yang, Mingyang Yu, Jing Xu, Keqian Li

Shanghai Institute of Intelligent Education, East China Normal University, Shanghai 200062, China · College of Artificial Intelligence, Nankai University, Tianjin 300350, China

Neural PDE Solvers Evolutionary Optimization Large Language Model-Guided Optimization Physics-Informed ML

Open paper AskPDF arXiv

Physics-informed neural networks (PINNs) are unusually sensitive to interacting choices of architecture, activation, loss weighting, collocation, optimization, and constraint enforcement. Large language models (LLMs) can propose these choices, but independent recommendations do not accumulate experience from previously trained PINNs. We propose a closed-loop evolutionary algorithm that guides an LLM to generate complete, executable PINN configurations across generations, using measured training outcomes to determine subsequent search decisions. The algorithm maintains an evaluated population and lineage, applies parent-conditioned mutation and crossover, preserves elite and diverse solutions, rejects effective duplicates, and converts parent-relative successes and failures into the next-generation context supplied to the LLM. Every proposed configuration is executed directly under an exact optimizer-step budget. On a one-dimensional multiscale wave equation, two independent ten-generation runs trained 60 PINNs for 600,000 optimizer steps. In both runs, the best configuration appeared in the final generation, with best mean-squared error reduced by 2.97% and 95.38% relative to the initial population. The stronger run validated residual connections and increased depth on separate branches, combined them in a later generation, and then refined width and collocation density. It also revealed that low solution error can coexist with a high PDE residual. These results demonstrate the feasibility of evolutionary-algorithm-guided LLMs for PINN design on a controlled PDE while motivating broader, physics-aware evaluation.

Abstract

Physics-Informed ML

Explore similar work

Evolutionary Algorithm-Guided LLMs for Physics-Informed Neural Network Design

EvoPINN: Agentic Discovery of Executable Algorithms for Physics-Informed Neural Networks

PINNMorph: Evolving Online Adaptation Policies for Physics-Informed Neural Networks