cs.LGAug 26, 2026

Physics-Informed Foresight Pruning for Sparse PINN Solvers of Nonlinear PDEs

Authors: Ahmad Ishaque Karimi, Uvini Balasuriya Mudiyanselage, Kookjin Lee

Organizations: School of Computing and Augmented Intelligence Arizona State University Tempe, USA

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Physics-informed neural networks (PINNs) often rely on over-parameterized models to optimize coupled solution and differential-residual objectives, leaving unclear how much capacity is necessary and what pruning should preserve. We study foresight pruning at initialization for sparse PirateNet PDE solvers. Standard neural tangent kernel spectrum-aware pruning (NTK-SAP) aims to preserve output-side training dynamics but may overlook parameters whose main influence arises through derivatives in the governing equations. We introduce physics-informed spectrum-aware pruning (PI-SAP), which assigns saliency using sensitivity of the PDE residual. Experiments on the Gray-Scott equations, complex Ginzburg-Landau equation, Burgers' equation, and linear convection equation show that PI-SAP more consistently preserves Gray-Scott residual fidelity and is competitive under aggressive sparsity. However, no criterion is uniformly optimal across equations or sparsity levels. Small-batch PINN-NTK diagnostics further show that residual fidelity, solution accuracy, and kernel conditioning are distinct objectives, motivating pruning methods that explicitly balance solution-side and residual-side training dynamics during optimization.

Figures & tables

Fig. 1: PirateNet backbone and pruning scope used in this study. The upper view summarizes the coordinate embedding, auxiliary feature streams, adaptive blocks, and output projection, while the lower view details one adaptive block. Blue components contain kernel parameters eligible for global pruning by NTK-SAP or PI-SAP. Adaptive gates αℓ\alpha_{\ell} , biases, and physics-informed output coefficients remain dense. The architecture follows the PirateNets design of Wang et al. [ 7 ] ; the pruning-scope representation is specific to our implementation.
PruneNTK err.PI err.Δ\DeltaNTK res.PI res.
10%1.1439×10−21.1439{\times}10^{-2}7.5343×10−37.5343{\times}10^{-3}-34.1%1.2996×10−51.2996{\times}10^{-5}1.1403×10−51.1403{\times}10^{-5}
30%1.7896×10−21.7896{\times}10^{-2}1.7555×10−21.7555{\times}10^{-2}-1.9%1.5345×10−51.5345{\times}10^{-5}1.5200×10−51.5200{\times}10^{-5}
50%4.3067×10−24.3067{\times}10^{-2}4.4727×10−24.4727{\times}10^{-2}+3.9%2.6953×10−52.6953{\times}10^{-5}2.1316×10−52.1316{\times}10^{-5}
70%2.7291×10−12.7291{\times}10^{-1}1.9430×10−11.9430{\times}10^{-1}-28.8%5.1482×10−55.1482{\times}10^{-5}4.8413×10−54.8413{\times}10^{-5}
90%4.3493×10−14.3493{\times}10^{-1}3.7866×10−13.7866{\times}10^{-1}-12.9%3.8921×10−43.8921{\times}10^{-4}3.8713×10−43.8713{\times}10^{-4}
TABLE I: Results for the Gray–Scott equations in the stripe-forming regime. Δ\Delta denotes the PI-SAP mean-error change relative to NTK-SAP; negative values indicate improvement.
Methoduu high-freq.vv high-freq.
Dense PirateNet0.02270.0277
NTK-SAP 70%0.72380.7749
PI-SAP 70%0.35910.4132
TABLE II: Terminal high-frequency errors for the Gray–Scott equations at 70% pruning.
Methoduu err.vv err.rur_{u} lossrvr_{v} loss
Dense0.02640.04892.262×10−42.262{\times}10^{-4}8.219×10−58.219{\times}10^{-5}
NTK-SAP0.17860.32291.973×10−41.973{\times}10^{-4}6.971×10−56.971{\times}10^{-5}
Cond. NTK-SAP0.19660.35162.589×10−42.589{\times}10^{-4}9.279×10−59.279{\times}10^{-5}
PI-SAP0.20060.35786.667×10−46.667{\times}10^{-4}2.086×10−42.086{\times}10^{-4}
PINN-block SAP0.22490.40211.605×10−41.605{\times}10^{-4}5.671×10−55.671{\times}10^{-5}
TABLE III: Diagnostic extensions for the Gray–Scott equations at 70% pruning and 600k total steps.
PruneNTK err.PI err.BetterNTK res.PI res.
10%2.6062×10−22.6062{\times}10^{-2}2.5384×10−22.5384{\times}10^{-2}PI1.0831×10−51.0831{\times}10^{-5}1.1401×10−51.1401{\times}10^{-5}
30%2.8769×10−22.8769{\times}10^{-2}3.4257×10−23.4257{\times}10^{-2}NTK1.4901×10−51.4901{\times}10^{-5}1.4704×10−51.4704{\times}10^{-5}
50%4.6634×10−24.6634{\times}10^{-2}5.4244×10−25.4244{\times}10^{-2}NTK2.7359×10−52.7359{\times}10^{-5}2.6487×10−52.6487{\times}10^{-5}
70%6.5615×10−26.5615{\times}10^{-2}5.8078×10−25.8078{\times}10^{-2}PI7.9350×10−57.9350{\times}10^{-5}8.2189×10−58.2189{\times}10^{-5}
90%1.8836×10−11.8836{\times}10^{-1}1.7434×10−11.7434{\times}10^{-1}PI4.7924×10−34.7924{\times}10^{-3}3.7924×10−33.7924{\times}10^{-3}
TABLE IV: Ginzburg–Landau summary. Dense mean error is 2.8017×10−22.8017{\times}10^{-2} and dense residual is 1.4557×10−51.4557{\times}10^{-5} .
WidthDenseBest prunedMethod
1281.0849×10−21.0849{\times}10^{-2}3.7532×10−33.7532{\times}10^{-3}PI 30%
2563.1485×10−23.1485{\times}10^{-2}5.4160×10−35.4160{\times}10^{-3}PI 30%
5123.0051×10−13.0051{\times}10^{-1}2.5324×10−32.5324{\times}10^{-3}NTK 90%
TABLE V: Best pruned results for Burgers’ equation across network widths.
β\betaWidthDenseBest prunedMethod
11281.2495×10−21.2495{\times}10^{-2}5.2080×10−35.2080{\times}10^{-3}PI 30%
 2561.1271×10−21.1271{\times}10^{-2}5.0310×10−35.0310{\times}10^{-3}PI 70%
 5121.3225×10−11.3225{\times}10^{-1}7.2160×10−37.2160{\times}10^{-3}NTK 90%
51281.6948×10−21.6948{\times}10^{-2}6.8240×10−36.8240{\times}10^{-3}NTK 50%
 2569.2240×10−39.2240{\times}10^{-3}9.4590×10−39.4590{\times}10^{-3}PI 30%
 5129.6970×10−39.6970{\times}10^{-3}7.4770×10−37.4770{\times}10^{-3}PI 70%
TABLE VI: Best pruned results for the convection equation across transport parameter β\beta and network width.
More in this topic

Neural PDE Solvers

Neural PDE solvers use trainable models to approximate solutions or solution operators for partial differential equations, often with physical constraints.

See all 269 papers

Explore similar work

CardsList

  1. SS-ESOAP: Self-Scaled Adaptive Preconditioning for Physics-Informed Learning

    Aug 29, 2026Guangyuan Wang, Mads Toftrup, Sebastian Loeschcke +2Neural Network OptimizationPhysics-Informed ML

    McGill University · California Institute of Technology · Aarhus University +1

  2. PINNs Failure Modes are Overfitting

    May 29, 2026Nigel T. Andersen, Takashi MatsubaraNeural Network GeneralizationNeural PDE Solvers

    Graduate School of Information Science and Technology Hokkaido University Sapporo, Japan · RIKEN Center for Advanced Intelligence Project (AIP) Tokyo, Japan

  3. Linearized PINN with pretrained nonlinear layers

    Sep 14, 2026Wenhao Chen, Alexandre M. TartakovskyEfficient Neural Network InferencePDE Solving

    Civil and Environmental Engineering Department University of Illinois Urbana-Champaign Urbana, IL 61801, USA · Pacific Northwest National Laboratory Richland, WA 99352, USA