cs.LGMay 22, 2026

Coupling-Robust Accuracy in Multiphysics Physics Informed Neural Networks via Kronecker-Preconditioned Optimization

Authors: Youngjae ParkJaemin KimJunghwa Hong

Organizations: Dept. of Control and Instrumentation Engineering, Korea University, Sejong, South Korea · BK21 FOUR Smart Mobility Education and Research Team, Korea University, Sejong, South Korea

Abstract

Physics-informed neural networks (PINNs) for coupled multiphysics systems suffer systematic accuracy degradation as inter-equation coupling strengthens. We provide a theoretical explanation through neural tangent kernel (NTK) analysis: for linearly coupled systems, the standard NTK's spectral radius grows as Ω(γ2)Ω(γ^2) with coupling strength γγ, shrinking the stable learning rate, while block-diagonal Gauss--Newton (GN) preconditioning yields a preconditioned NTK KP=JH+JK_P = JH^{+}J^\top whose spectral radius is bounded by SS (number of networks), independent of γγ. Adam's diagonal preconditioning destroys this projector structure -- inflating λmaxλ_{\max} far above SS for any coupling type -- and its residual-dynamics kernel grows as Θ(γ)Θ(γ), placing its stable learning rate strictly between gradient descent and GN. For one-way coupling the limitation is class-wide: no diagonal preconditioner, fixed or adaptive, halves the driving residual in fewer than Ω(γ)Ω(γ) iterations (Ω(γ2)Ω(γ^2) if fixed), whereas block-diagonal GN requires O(1)O(1). We verify Ω(γ2)Ω(γ^2) growth across linearly coupled benchmarks and confirm λmax(KP)=Sλ_{\max}(K_P) = S in all three 1D systems, including nonlinearly coupled NP+P. Combining the Kronecker-preconditioned optimizer SOAP with inverse-gradient-norm loss balancing (SOAP+GradNorm) yields coupling-robust accuracy: across 222 experiments spanning three 1D systems and a 2D electroosmotic flow benchmark, SOAP+GradNorm maintains final-epoch L2L_2 accuracy across coupling strengths, with 2.3×\leq 2.3\times degradation in nonlinear NP+P while Adam+GradNorm fails (L2>0.1L_2 > 0.1). SOAP+GradNorm further scales to a 2D, 6-PDE electroosmotic flow at EDL-resolved conditions down to ε=0.01\varepsilon = 0.01 -- a regime all prior PINN electrokinetics studies have avoided -- where Adam+GradNorm fails entirely (L2>0.3L_2 > 0.3).

Explore similar work

Jul 2, 2026cs.LG

An Optimisation Framework for the Well-Conditioned Training of Physics-Informed Neural Networks

Physics-informed neural networks (PINNs) have emerged as a promising route to solve partial differential equations, yet they have struggled to reach the precision of classical solvers. The obstacle is increasingly understood to be one of optimisation, owing to the severely ill-conditioned loss landscape. We present DSGNAR\textbf{DSGNAR}: Doubly-Sketched Gauss-Newton with Adaptive Ratio, a scalable second-order optimisation framework that confronts this ill-conditioning and, in doing so, obtains unprecedented accuracy and speed. DSGNAR\textbf{DSGNAR} couples a doubly-sketched Gauss-Newton model with a novel strategy that carefully controls both regularisation and step length. Across a suite of problems spanning nonlinear, chaotic, multi-scale, high-dimensional, and Navier-Stokes, the framework greatly improves on the state of the art: able to attain relative 2\ell_2 errors as low as 3×10163\times10^{-16} in double precision, improve contemporary results by five orders of magnitude on the canonical Burgers' equation, and as much as eight orders on a high-dimensional Poisson problem, while remaining markedly faster. We further show that, in single precision, solutions at the limit of round-off error can be obtained very quickly: Burgers' equation to 2rel=4.75×107\ell_2^{\text{rel}} = 4.75 \times 10^{-7} in under ten seconds. The framework is also robust to the choice of architecture, arithmetic precision, and initial hyperparameters. The code is available at https://www.github.com/wephy/physics-informed-neural-networks
Joseph Webb, Sadok Jerad, Coralia Cartis
Apr 16, 2026cs.LG

Lightweight Geometric Adaptation for Training Physics-Informed Neural Networks

Physics-Informed Neural Networks (PINNs) often suffer from slow convergence, training instability, and reduced accuracy on challenging partial differential equations due to the anisotropic and rapidly varying geometry of their loss landscapes. We propose a lightweight curvature-aware optimization framework that augments existing first-order optimizers with an adaptive predictive correction based on secant information. Consecutive gradient differences are used as a cheap proxy for local geometric change, together with a step-normalized secant curvature indicator to control the correction strength. The framework is plug-and-play, computationally efficient, and broadly compatible with existing optimizers, without explicitly forming second-order matrices. Experiments on diverse PDE benchmarks show consistent improvements in convergence speed, training stability, and solution accuracy over standard optimizers and strong baselines, including on the high-dimensional heat equation, Gray--Scott system, Belousov--Zhabotinsky system, and 2D Kuramoto--Sivashinsky system.
Kang An, Chenhao Si, Shiqian Ma +1
Jul 27, 2026cs.LG

Variational Boosting for Physics-Informed Neural Networks

Physics-Informed Neural Networks (PINNs) solve differential equations by minimizing the residual of a nonlinear operator over a neural parameterization of the solution. However, monolithic PINNs often suffer from ill-conditioning, spectral bias, and optimization instability. We introduce a variational boosting framework in which solutions are constructed additively in function space. Each stage trains a weak learner whose converged correction satisfies a local orthogonality condition, equivalent to a projected functional gradient descent step onto the tangent space of the network's function manifold. Because each correction network is deliberately small, the restricted minimization admits full Newton or conjugate gradient updates, which are typically infeasible in large PINNs. The resulting method separates global nonlinear refinement into a sequence of well-conditioned subproblems while preserving the full variational structure of the operator. This framework provides a geometric interpretation of multi-stage PINNs as projected functional gradient descent and enables stable second-order optimization for nonlinear differential equations.
Pavlos Protopapas, Kaylee Vo