cs.AISep 29, 2026

PE-EK-PINN: Physics Embedding with Evolving Kernel for Scalable Physics-Informed Neural Networks

Authors: Huiwen Zhang, Feng Ye, Chu Ma

Organizations: Department of Electrical & Computer Engineering Madison, WI, 53706, USA

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Physics-Informed Neural Networks (PINNs) embed governing equations into deep learning, but enforce them only through loss residuals, leaving highly oscillatory wave behavior to be discovered by optimization. As a result, methods that achieve relative L2L_2 errors below 10−310^{-3} on standard manufactured Helmholtz benchmarks can fail on practical radiation problems involving singular excitations, absorbing boundaries, and wave fields spanning tens of wavelengths. Architectural physics embedding addresses this limitation by factorizing the field into analytically derived oscillatory kernels and learnable envelopes. However, the kernel dictionary must be manually constructed and scales with the number of elementary units, growing exponentially with the depth of hierarchically structured systems such as antenna arrays and metasurfaces. We propose PE-EK-PINN (Physics Embedded with Evolving Kernels), which treats physics kernels as reusable learned representations rather than fixed analytical inputs. A converged subsystem field is frozen and promoted to an evolved kernel, whose transformed copies are reused to represent higher-level configurations without deriving new governing equations. The resulting hierarchy makes the peak number of active kernels independent of system size and reduces cumulative training cost from O(N)O(N) to O(log⁡N)O(\log N). Experiments on dipole arrays, composite line-source geometries, and cross arrays demonstrate the dramatic training cost reduction, while achieving a reduced or comparable relative L2L_2 error. One notable example is PE-EK-PINN solves a 256256-dipole array more than 30 times faster than direct PE-PINN.

Figures & tables

ScenarioMethodRef.Rel. L2L_{2}
Manufactured Solution (separable, ∼\sim 4 wavelengths)PINNRaissi et al. (2019)0.97570
gPINNYu et al. (2022)0.32550
AHD-PINNDashtbayaz et al. (2024)0.19030
SPINNCho et al. (2023)0.08090
SPINN (m)Cho et al. (2023)0.05950
RoPINNWu et al. (2024)0.33380
Table 1: Relative L2L_{2} errors on two Helmholtz settings. The manufactured-solution block follows the protocol of prior work: Ω=[−1,1]3\Omega=[-1,1]^{3} , u=sin⁡(4πx1)sin⁡(4πx2)sin⁡(3πx3)u=\sin(4\pi x_{1})\sin(4\pi x_{2})\sin(3\pi x_{3}) , with the corresponding source term obtained by substituting uu into the Helmholtz equation, and the number of collocation points Nc=323N_{c}=32^{3} . The dipole block is a 2.42.4 GHz radiation problem with a singular excitation and absorbing truncation, with both methods run under identical settings. First-block baselines are as reported in Duan et al. (2025) ; all remaining values are from our own runs using the authors’ recommended settings.
Figure 1: Overview of PE-EK-PINN structure.
ScenarioBase config.PE-EK-PINN constructionTransformation
Dipole arraySingle dipole1→2×2→4×4→8×8→16×161\rightarrow 2{\times}2\rightarrow 4{\times}4\rightarrow 8{\times}8\rightarrow 16{\times}16Translation
Composite line geometry2λ2\lambda line2λ2\lambda line →\rightarrow {Cross, 5-point star}Translation + rotation
Cross array (Section B.2 )0.5λ0.5\lambda line0.5λ0.5\lambda line →\rightarrow Cross →2×2→4×4\rightarrow 2{\times}2\rightarrow 4{\times}4Translation + rotation
Table 2: Overview of the evaluated source configurations.
MethodNdipolesN_{\mathrm{dipoles}}NkernelsN_{\mathrm{kernels}}Stage T.Cum. T.MSERel. L2L_{2}
PE-PINN11200:19:47N/A9.48×10−69.48\times 10^{-6}2.14×10−22.14\times 10^{-2}
PE-EK-PINN2×22\times 2400:26:5100:46:382.49×10−52.49\times 10^{-5}1.55×10−21.55\times 10^{-2}
PE-EK-PINN4×44\times 4400:26:5301:13:311.86×10−41.86\times 10^{-4}1.62×10−21.62\times 10^{-2}
PE-EK-PINN8×88\times 8400:27:2901:41:003.78×10−33.78\times 10^{-3}2.71×10−22.71\times 10^{-2}
PE-EK-PINN16×1616\times 16400:29:3502:10:353.20×10−13.20\times 10^{-1}9.43×10−29.43\times 10^{-2}
PE-PINN2×22\times 2801:07:48N/A5.83×10−55.83\times 10^{-5}2.37×10−22.37\times 10^{-2}
Table 3: Accuracy and training time for the dipole-array experiments. PE-PINN results for the 2×22\times 2 and 4×44\times 4 configurations serve as an ablation baseline. Cumulative time includes all preceding cascade stages starting from the single-dipole model.
Figure 2: Visualized results of the 16×1616\times 16 dipole-array and 5-point star finite-line experiments. Please refer to Appendix B.3 for the complete visualized results.
MethodConfig.KernelsStage T.Cum. T.MSERel. L2L_{2}
PE-PINNLine1001:37:08N/A8.85×10−58.85\times 10^{-5}4.12×10−24.12\times 10^{-2}
PE-EK-PINNTwo-line cross200:16:2901:53:371.18×10−41.18\times 10^{-4}3.34×10−23.34\times 10^{-2}
PE-PINNTwo-line cross2003:13:19N/A1.87×10−31.87\times 10^{-3}1.33×10−11.33\times 10^{-1}
PE-PINNLine wide1002:21:57N/A2.07×10−42.07\times 10^{-4}7.56×10−27.56\times 10^{-2}
PE-EK-PINN5-point star500:39:1803:01:141.24×10−31.24\times 10^{-3}6.32×10−26.32\times 10^{-2}
Table 4: Accuracy and training time for the finite-line-source experiments. The wider-domain line kernel is trained on Ω=[−3.6,3.6]2\Omega=[-3.6,3.6]^{2} to cover the coordinate range reached after rotation.
Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

MethodConfig.KernelsStage T.Cum. T.MSERel. L2L_{2}
PE-PINN0.5λ0.5\lambda line400:42:5900:42:596.38×10−56.38\times 10^{-5}6.39×10−26.39\times 10^{-2}
PE-EK-PINNCross200:16:2600:59:251.07×10−41.07\times 10^{-4}4.18×10−24.18\times 10^{-2}
PE-PINNCross801:00:50N/A5.30×10−45.30\times 10^{-4}9.33×10−29.33\times 10^{-2}
PE-EK-PINN2×22\times 2400:31:4401:31:098.00×10−48.00\times 10^{-4}5.07×10−25.07\times 10^{-2}
PE-PINN2×22\times 23204:05:07N/A3.20×10−33.20\times 10^{-3}1.01×10−11.01\times 10^{-1}
PE-EK-PINN4×44\times 4400:32:2402:03:337.57×10−37.57\times 10^{-3}5.90×10−25.90\times 10^{-2}
Appendix
Table 5: Accuracy and training time for the hierarchical cross-array experiments. PE-PINN baselines use primitive point-source kernels only.
Figure 3: Visualized results of the dipole-array experiments.
Figure 4: Training-time scaling for direct PE-PINN and PE-EK-PINN. Stage time denotes the training cost of the current cascade level, while cumulative time includes all preceding cascade stages. Direct PE-PINN times for the 8×88\times 8 and 16×1616\times 16 arrays are extrapolated based on the observed near-linear scaling with active-kernel count.
Figure 5: Visualized results of the composite line-source experiments.
Figure 6: Visualized results the cross-array experiments.
Figure 7: Training-time scaling for the hierarchical cross-array experiment. Stage time denotes the training cost of the current cascade level, while cumulative time includes all preceding stages from the elementary 0.5λ0.5\lambda line-source model. The direct PE-PINN time for the 4×44\times 4 cross array is extrapolated based on the observed near-linear scaling with active kernel count.

Explore similar work

CardsList
Aug 6, 2026cs.LG

Alternating Levenberg-Marquardt Training of Physics-Informed Neural Networks with Fourier-Enhanced Features

Physics-informed neural networks (PINNs) often fail to accurately resolve partial differential equations (PDEs) with high-frequency or multi-scale solutions, as well as strongly nonlinear problems. Two factors underlie this difficulty: spectral bias, the tendency of neural networks to underfit high-frequency features; and representation-coefficient coupling, the entanglement of representation learning and coefficient fitting within a single nonconvex optimization objective. In this work, we propose the Fourier-enhanced alternating Levenberg--Marquardt PINN (FALM-PINN), an optimization framework that decouples representation learning from coefficient fitting. The upper-level problem learns a Fourier-enhanced basis that enriches the latent space with high-frequency components, while the lower-level problem resolves the coupling by fitting the projection coefficients on this basis, solving a nonlinear least-squares problem with the Levenberg--Marquardt algorithm. The framework applies to general nonlinear and coupled PDE systems, and reduces to a single-step convex optimization problem for linear PDEs. We prove global convergence of the alternating training scheme in both cases. Numerical examples on multiple challenging high-frequency and nonlinear PDEs show that FALM-PINN achieves relative L2L^2 errors up to two orders of magnitude lower than state-of-the-art baselines.

Yulun Wu, Matthieu Barreau, Miguel Aguiar +1

Division of Decision and Control Systems, School of Electrical Engineering and Computer Science, KTH Royal Institute of Technology, Stockholm, 100 44, Sweden

Parametric Physics-Informed Neural NetworkPartial Differential EquationsNewtonFourierNeural Network
AskPDF
Jun 18, 2026cs.LG

Evolutionary Two-Stage Hyperparameter Optimization Strategies for Physics-Informed Neural Networks

Physics-Informed Neural Networks (PINNs) solve Partial Differential Equations (PDEs) by embedding physical laws into neural network training. However, their performance suffers from unstable convergence, training plateaus, and strong sensitivity to architectural and optimization hyperparameters due to the highly non-convex and multi-term structure of the physics-informed loss. In this setting, the outer-loop hyperparameter search is a noisy and black-box optimization problem over heterogeneous parameters, where classical local or gradient-based strategies are easily trapped in suboptimal regions. Evolutionary algorithms, with their population-based exploration and ability to handle mixed, non-differentiable search spaces, provide a more robust mechanism for discovering promising configurations. We propose and investigate a two-stage approach based on evolutionary algorithms that combines exploration and exploitation parts of PINNs training to improve solution accuracy and robustness under fixed computational budgets. In the first stage, we perform low-fidelity training runs with truncated epochs to rapidly screen candidate configurations, treating hyperparameter selection as a black-box outer-loop problem. In the second stage, only the most promising candidates are fully trained with standard gradient-based optimizers to refine the solution. Evaluated on three popular problems, namely Advection, Klein-Gordon and Helmholtz equations, our method consistently outperforms standard training and achieves significantly lower mean error within constrained computational resources.

Fedor Buzaev, Dmitry Efremenko, Egor Bugaev +4

HSE University · 1HSE University · AXXX +1

Parametric Physics-Informed Neural NetworkGeneral Grid Search FrameworkNeural Network OptimizationNeural NetworkGenetic AlgorithmsConvergence
AskPDF
Jul 2, 2026cs.LG

An Optimisation Framework for the Well-Conditioned Training of Physics-Informed Neural Networks

Physics-informed neural networks (PINNs) have emerged as a promising route to solve partial differential equations, yet they have struggled to reach the precision of classical solvers. The obstacle is increasingly understood to be one of optimisation, owing to the severely ill-conditioned loss landscape. We present DSGNAR\textbf{DSGNAR}: Doubly-Sketched Gauss-Newton with Adaptive Ratio, a scalable second-order optimisation framework that confronts this ill-conditioning and, in doing so, obtains unprecedented accuracy and speed. DSGNAR\textbf{DSGNAR} couples a doubly-sketched Gauss-Newton model with a novel strategy that carefully controls both regularisation and step length. Across a suite of problems spanning nonlinear, chaotic, multi-scale, high-dimensional, and Navier-Stokes, the framework greatly improves on the state of the art: able to attain relative ℓ2\ell_2 errors as low as 3×10−163\times10^{-16} in double precision, improve contemporary results by five orders of magnitude on the canonical Burgers' equation, and as much as eight orders on a high-dimensional Poisson problem, while remaining markedly faster. We further show that, in single precision, solutions at the limit of round-off error can be obtained very quickly: Burgers' equation to ℓ2rel=4.75×10−7\ell_2^{\text{rel}} = 4.75 \times 10^{-7} in under ten seconds. The framework is also robust to the choice of architecture, arithmetic precision, and initial hyperparameters. The code is available at https://www.github.com/wephy/physics-informed-neural-networks

Joseph Webb, Sadok Jerad, Coralia Cartis

Mathematical Institute, University of Oxford, Oxford, United Kingdom

Parametric Physics-Informed Neural NetworkNewtonNeural NetworkSolversPrompt Optimization
AskPDF