Eikonal Regularisation in Physics-Informed Neural Networks for Three-Dimensional Level-Set Advection: Transferability of Two-Dimensional Design Principles

Physics-informed neural networks (PINNs) have emerged as a promising route to solve partial differential equations, yet they have struggled to reach the precision of classical solvers. The obstacle is increasingly understood to be one of optimisation, owing to the severely ill-conditioned loss landscape. We present $\textbf{DSGNAR}$: Doubly-Sketched Gauss-Newton with Adaptive Ratio, a scalable second-order optimisation framework that confronts this ill-conditioning and, in doing so, obtains unprecedented accuracy and speed. $\textbf{DSGNAR}$ couples a doubly-sketched Gauss-Newton model with a novel strategy that carefully controls both regularisation and step length. Across a suite of problems spanning nonlinear, chaotic, multi-scale, high-dimensional, and Navier-Stokes, the framework greatly improves on the state of the art: able to attain relative $\ell_2$ errors as low as $3\times10^{-16}$ in double precision, improve contemporary results by five orders of magnitude on the canonical Burgers' equation, and as much as eight orders on a high-dimensional Poisson problem, while remaining markedly faster. We further show that, in single precision, solutions at the limit of round-off error can be obtained very quickly: Burgers' equation to $\ell_2^{\text{rel}} = 4.75 \times 10^{-7}$ in under ten seconds. The framework is also robust to the choice of architecture, arithmetic precision, and initial hyperparameters. The code is available at https://www.github.com/wephy/physics-informed-neural-networks

Physics-informed neural networks (PINNs) for coupled multiphysics systems suffer systematic accuracy degradation as inter-equation coupling strengthens. We provide a theoretical explanation through neural tangent kernel (NTK) analysis: for linearly coupled systems, the standard NTK's spectral radius grows as $Ω(γ^2)$ with coupling strength $γ$, shrinking the stable learning rate, while block-diagonal Gauss--Newton (GN) preconditioning yields a preconditioned NTK $K_P = JH^{+}J^\top$ whose spectral radius is bounded by $S$ (number of networks), independent of $γ$. Adam's diagonal preconditioning destroys this projector structure -- inflating $λ_{\max}$ far above $S$ for any coupling type -- and its residual-dynamics kernel grows as $Θ(γ)$, placing its stable learning rate strictly between gradient descent and GN. For one-way coupling the limitation is class-wide: no diagonal preconditioner, fixed or adaptive, halves the driving residual in fewer than $Ω(γ)$ iterations ($Ω(γ^2)$ if fixed), whereas block-diagonal GN requires $O(1)$. We verify $Ω(γ^2)$ growth across linearly coupled benchmarks and confirm $λ_{\max}(K_P) = S$ in all three 1D systems, including nonlinearly coupled NP+P. Combining the Kronecker-preconditioned optimizer SOAP with inverse-gradient-norm loss balancing (SOAP+GradNorm) yields coupling-robust accuracy: across 222 experiments spanning three 1D systems and a 2D electroosmotic flow benchmark, SOAP+GradNorm maintains final-epoch $L_2$ accuracy across coupling strengths, with $\leq 2.3\times$ degradation in nonlinear NP+P while Adam+GradNorm fails ($L_2 > 0.1$). SOAP+GradNorm further scales to a 2D, 6-PDE electroosmotic flow at EDL-resolved conditions down to $\varepsilon = 0.01$ -- a regime all prior PINN electrokinetics studies have avoided -- where Adam+GradNorm fails entirely ($L_2 > 0.3$).

Neural operators evaluate parametric partial differential equations cheaply but degrade sharply outside their training distribution. Physics-informed neural networks avoid dependence on labeled data, yet their optimization can be basin-fragile: when the governing residual admits multiple solutions, a PINN trained from scratch may converge to a physically incorrect state despite achieving a small residual. We show that these failure modes can be addressed jointly: an imperfect NO provides the structural prior needed to place a PINN in the correct solution basin, while the PDE residual refines the solution beyond the operator's accuracy. We introduce a three-stage framework that freezes the spatial basis of a physics-informed NO, extrapolates its solution branch to an out-of-distribution parameter using a polynomial continuation prior, and distills the resulting field into a fresh PINN. The NO need not be accurate at the target; it transfers solution-branch information, while PDE residual minimization in the PINN governs convergence. We evaluate the framework on three nonlinear PDEs: 1D viscous Burgers, 2D steady Allen-Cahn near a pitchfork bifurcation, and 2D steady lid-driven cavity flow. For Allen-Cahn, where the trivial solution satisfies the PDE residual exactly, a standard PINN collapses to the trivial zero branch, whereas distillation from the crude extrapolated operator recovers the non-trivial branch that matches the finite-difference reference. For the lid-driven cavity, extrapolating to a Reynolds number of Re = 3200 accelerates convergence to the correct physical state, achieving competitive accuracy using fewer parameters and optimization steps than recent literature baselines. These results establish a simple principle: an NO need not accurately predict the solution to be useful; it only needs to identify the correct basin from which PINN optimization can recover it.