cs.LGSep 19, 2026

LPINNs: First-Layer Gated Localization for Physics-Informed Neural Networks

Authors: Lakshay Chawla, Hardik Jain

Organizations: Indian Institute of Technology Jodhpur

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Physics-informed neural networks (PINNs) use one shared representation over the computational domain, which can become difficult to optimize on long domains and for high-order operators. We study a minimal alternative: multiply the first hidden activation of an otherwise unchanged dense PINN by input-dependent localization functions, giving first-layer units receptive fields without partitioning the domain or adding interface losses. We screen 13 families of localization functions, in up to three parameterizations each, on a nonlinear harmonic oscillator (HO), a heat equation on a long spatial interval, and a manufactured four-dimensional (4D) fourth-order problem, with ten paired seeds throughout. Three configurations give large reductions in solution error at matched budgets: (i) Fixed Gaussian localization functions on the 2π2π HO domain cut mean solution RMSE from 4.8369×10−14.8369\times10^{-1} to 8.83×10−38.83\times10^{-3} at 3k epochs. (ii) The inverse-quadratic family with learnable centers and widths cuts it from 2.896×10−12.896\times10^{-1} to 3.06×10−23.06\times10^{-2} on the 8π8π heat domain at 10k epochs. (iii) Fixed bump localization functions cut it from 1.75947×1011.75947\times10^{1} to 2.260×10−12.260\times10^{-1} on the 4π4π 4D domain at 10k epochs. Every paired seed improves in these three comparisons. The screen also shows that the mechanism is not a free win: on HO only 2 of 13 families beat the baseline, and 10 of the remaining 11 are 9 to 23 times worse; on 4D four families are non-finite and five are more than three orders of magnitude worse than the baseline. The inverse-quadratic family is the only one that beats the baseline on all three equations. Overall, these results show that first-layer localization can provide measurable improvements to baseline PINNs on long-domain and high-order problems.

Figures & tables

ProblemDomain valuesEpoch budgets
HOt∈[0,L], L∈{π,2π,3π,4π}t\in[0,L],\ L\in\{\pi,2\pi,3\pi,4\pi\}11 k, 33 k, 55 k, 1010 k, 5050 k, 100100 k
Heatx∈[0,L], t∈[0,1]x\in[0,L],\ t\in[0,1] L∈{π,2π,4π,8π}L\in\{\pi,2\pi,4\pi,8\pi\}11 k, 33 k, 55 k, 1010 k, 5050 k
4Dx∈[0,L]4, L∈{π,2π,4π}x\in[0,L]^{4},\ L\in\{\pi,2\pi,4\pi\}11 k, 33 k, 55 k, 1010 k
Table 1: Domain sizes and epoch budgets used in the localization campaign.
Localization FunctionEpochsFinal Residual LossResidual RMSESolution RMSESolution MAE
Baseline3k5.010×10−25.010\times 10^{-2}1.1995×10−11.1995\times 10^{-1}4.8369×10−14.8369\times 10^{-1}4.0788×10−14.0788\times 10^{-1}
Gaussian3k1.85×10−31.85\times 10^{-3}2.639×10−22.639\times 10^{-2}8.83×10−38.83\times 10^{-3}7.06×10−37.06\times 10^{-3}
Baseline1k6.230×10−26.230\times 10^{-2}1.7438×10−11.7438\times 10^{-1}6.7583×10−16.7583\times 10^{-1}5.5689×10−15.5689\times 10^{-1}
Gaussian1k1.1420×10−11.1420\times 10^{-1}3.0155×10−13.0155\times 10^{-1}2.14721×1002.14721\times 10^{0}1.79705×1001.79705\times 10^{0}
Table 2: HO at L=2πL=2\pi with 64×6464\times 64 networks and fixed Gaussian localization functions. Means over ten paired seeds. All metric entries are shown in scientific notation. Bold marks the localized solution RMSE and MAE at the larger budget.
Figure 1: HO at L=2πL=2\pi and 3k epochs: recovered trajectories with the numerical reference (left), residual-loss histories (center), and absolute residuals (right). Thin curves are the ten paired seeds and thick curves are their means, with loss averaged in log⁡10\log_{10} space.
Localization FunctionEpochsFinal Residual LossResidual RMSESolution RMSESolution MAE
Baseline3k6.433×10−16.433\times 10^{-1}7.073×10−17.073\times 10^{-1}2.909×10−12.909\times 10^{-1}2.328×10−12.328\times 10^{-1}
Inverse-quadratic3k1.208×10−11.208\times 10^{-1}3.353×10−13.353\times 10^{-1}1.011×10−11.011\times 10^{-1}7.17×10−27.17\times 10^{-2}
Baseline10k5.212×10−15.212\times 10^{-1}7.047×10−17.047\times 10^{-1}2.896×10−12.896\times 10^{-1}2.319×10−12.319\times 10^{-1}
Inverse-quadratic10k2.51×10−22.51\times 10^{-2}1.499×10−11.499\times 10^{-1}3.06×10−23.06\times 10^{-2}2.24×10−22.24\times 10^{-2}
Table 3: Heat at L=8πL=8\pi with 64×6464\times 64 networks and inverse-quadratic localization functions with learnable centers and widths. Means over ten paired seeds. All metric entries are shown in scientific notation. Bold marks the localized solution RMSE and MAE at the larger budget.
Figure 2: Heat equation at L=8πL=8\pi and 10k epochs for both models: solution at t=0.5t=0.5 (left), residual-loss histories (center), and absolute residuals at t=0.5t=0.5 (right). Thin curves are the ten paired seeds and thick curves are their means, with loss averaged in log⁡10\log_{10} space.
Localization FunctionEpochsFinal Residual LossResidual RMSESolution RMSESolution MAE
Baseline3k2.6200×10−32.6200\times 10^{-3}5.3494×10−25.3494\times 10^{-2}1.420850×1021.420850\times 10^{2}1.111457×1021.111457\times 10^{2}
Bump3k2.5200×10−32.5200\times 10^{-3}1.114×10−31.114\times 10^{-3}9.221×10−19.221\times 10^{-1}5.287×10−15.287\times 10^{-1}
Baseline10k5.34×10−55.34\times 10^{-5}7.320×10−37.320\times 10^{-3}1.75947×1011.75947\times 10^{1}1.40520×1011.40520\times 10^{1}
Bump10k4.72×10−54.72\times 10^{-5}3.44×10−43.44\times 10^{-4}2.260×10−12.260\times 10^{-1}1.296×10−11.296\times 10^{-1}
Table 4: 4D at L=4πL=4\pi with 32×3232\times 32 networks and fixed bump localization functions. Means over ten paired seeds. All metric entries are shown in scientific notation; the exponents are given inside the cells. Bold marks the localized solution RMSE and MAE at the larger budget.
Figure 3: 4D problem at L=4πL=4\pi and 10k epochs for both models: normalized solution on the diagonal slice (left), empirical distribution of relative error (center), and residual-loss histories (right). Thin curves are the ten paired seeds and thick curves are their means, with loss averaged in log⁡10\log_{10} space.
Localization FunctionHOHeat4D
Baseline9.946×10−19.946\times 10^{-1}8.386×10−28.386\times 10^{-2}3.126×1023.126\times 10^{2}
Gaussian8.402×10−58.402\times 10^{-5} (F)7.956×10−27.956\times 10^{-2} (F)n/a
Activated1.788×1011.788\times 10^{1} (S)7.502×10−27.502\times 10^{-2} (S)1.777×1071.777\times 10^{7} (F)
Super-Gaussian1.541×1011.541\times 10^{1} (S)7.470×10−27.470\times 10^{-2} (M)1.621×1071.621\times 10^{7} (S)
Ricker1.148×1011.148\times 10^{1} (F)8.339×10−28.339\times 10^{-2} (F)n/a
Boxcar9.1409.140 (S)7.237×10−27.237\times 10^{-2} (S)1.346×1071.346\times 10^{7} (S)
Table 5: Screen of localization-function families at the primary settings: HO at 2π2\pi /3k, Heat at 8π8\pi /10k, and 4D at 4π4\pi /10k. Entries are mean solution MSE over ten seeds; lower is better. F, S, and M denote fixed, learnable-width, and learnable-center/width parameterizations; each cell gives the lowest finite parameterization for that family, with the winning variant in parentheses. n/a means that no tested parameterization produced a finite solution metric for that problem. Bold marks the lowest valid entry in each problem column. Columns have different solution scales and should be compared only within a column.
More in this topic

Physics-Informed ML

Physics-informed machine learning combines data-driven models with physical laws to solve, predict, and infer scientific systems.

See all 843 papers

Explore similar work

CardsList
Sep 14, 2026

Linearized PINN with pretrained nonlinear layers

We propose a linearized Physics-Informed Neural Network (lPINN), a reduced-order neural basis method for forward and inverse differential equations. In an offline stage, lPINN learns operator-compatible continuous neural basis functions from an ensemble of numerical solutions. The basis functions are differentiable through automatic differentiation and are pretrained using solution data together with either derivative information or physics residuals. For each new problem instance, the basis functions are frozen and the solution is obtained by minimizing the governing-equation residual together with applicable initial, boundary, regularization, and observational terms. Unlike surrogate and operator-learning methods, the training data define the trial space offline, while the instance-specific solution is computed online by enforcing the governing physics. Relative to vanilla PINNs, lPINN pretrains the nonlinear hidden-layer representation offline and performs online inference only in the final linear layer. We evaluate lPINN on forward and inverse problems for the advection-diffusion equation, Burgers' equation, and the nonlinear pendulum equation. Compared with vanilla PINNs, lPINN achieves lower solution and parameter errors while reducing online inference times by approximately one to more than three orders of magnitude, with the largest gains generally observed for limited residual or measurement data. Cross-resolution experiments show that the learned continuous representation can be evaluated on finer meshes without retraining and with nearly unchanged accuracy.

Wenhao Chen, Alexandre M. Tartakovsky

Civil and Environmental Engineering Department University of Illinois Urbana-Champaign Urbana, IL 61801, USA · Pacific Northwest National Laboratory Richland, WA 99352, USA

Efficient Neural Network InferencePDE SolvingReduced-Order ModelingPhysics-Informed MLPDE Inverse Problems
AskPDF
Mar 4, 2026

Improving the accuracy of physics-informed neural networks via last-layer retraining

Physics-informed neural networks (PINNs) are a versatile tool in the burgeoning field of scientific machine learning for solving partial differential equations (PDEs). However, determining suitable training strategies for them is not obvious, with the result that they typically yield moderately accurate solutions. In this article, we propose a method for improving the accuracy of PINNs by coupling them with a post-processing step that seeks the best approximation in a function space associated with the network. We find that our method yields errors four to five orders of magnitude lower than those of the parent PINNs across architectures and dimensions. Moreover, we can reuse the basis functions for the linear space in more complex settings, such as time-dependent and nonlinear problems, allowing for transfer learning. Our approach also provides a residual-based metric that allows us to optimally choose the number of basis functions employed.

Saad Qadeer, Panos Stinis

Advanced Computing, Mathematics and Data Division, Pacific Northwest National Laboratory, Richland, WA

Neural PDE SolversPhysics-Informed MLFine-Tuning
AskPDF
Jul 15, 2026

LIGO-PINN: Learned Initialization via Gated Optimization to Alleviate Convergence Failures in Physics Informed Neural Networks

Physics-informed neural networks (PINNs) have had a broad research impact in modeling domains governed by partial differential equations (PDE). However, PINNs have been shown to perform poorly, sometimes even converging to trivial solutions, in challenging PDE domains, or when generalizing to unseen but related PDE domains. Previously proposed solutions detail hyperparameter tuning to reduce loss imbalance between data-driven and physics guided losses, curriculum learning based training strategies, or dynamic re-sampling of hard collocation points. These methods face certain pitfalls: hyperparameter tuning is expensive, designing a training curriculum is ambiguous in multi-parameter PDE settings, and dynamic resampling still fails in complex PDE settings. Complementary to this line of thinking, we believe the initial PINN network weights also play a crucial role in the emergence of catastrophic failures during training, yet the effect of PINN weight initialization has been surprisingly under-investigated. To this end, we propose a framework for Learned Initialization via Gated Layerwise Optimization (LIGO-PINN) to overcome PINN convergence failures. Through rigorous evaluation on 1D and 2D PDE domains, including a challenging 2D fluid dynamics setting, we demonstrate that our methodology outperforms state-of-the-art methods designed to alleviate PINN failures, achieving a 91.5% average performance improvement across six baselines and 81% over the strongest baseline. We also verify that LIGO-PINN generalizes to 3D unstructured domains. Finally, we analyze training dynamics across all three PDE domains to explain both LIGO-PINN's improvement and the convergence failure of traditional PINNs. Code: https://github.com/scailab/ligo-pinn Keywords: Machine Learning, Physics-Informed Neural Networks, Deep Learning, PDE Modeling

Nilay Anurag, Shital Adhikari, Taniya Kapoor +1

Department of Computer Science, Stevens Institute of Technology · Artificial Intelligence Group, Wageningen University & Research

Neural PDE SolversNeural Network OptimizationPhysics-Informed ML
AskPDF