math.NANov 9, 2026

Block-Sparse Gauss-Newton Optimization of Finite Basis Physics-Informed Neural Networks: Sparse Automatic Differentiation Approach

Authors: Denis Korolev, Adrian Hill, Jeeeun Lee

Organizations: Weierstrass Institute for Applied Analysis and Stochastics, Anton-Wilhelm-Amo-Str. 39, Berlin, 10117, Germany · BIFOLD – Berlin Institute for the Foundations of Learning and Data, Franklinstr. 28/29, Berlin, 10587, Germany · Machine Learning Group, Technical University of Berlin, Franklinstr. 28/29, Berlin, 10587, Germany · KAIST InnoCORE PRISM-AI Center, Korea Advanced Institute of Science and Technology (KAIST), 291 Daehak-ro, Yuseong-gu, Daejeon, 34141, Republic of Korea

Abstract

In this work, we develop an efficient Gauss-Newton optimization approach for finite basis physics-informed neural networks (FBPINNs). The key ingredient is the overlapping domain decomposition architecture of FBPINNs, which localizes the influence of each subnetwork and induces block sparsity in both the residual Jacobian and the corresponding Gauss-Newton matrix. We characterize this structure and exploit it through sparse automatic differentiation, using graph coloring to partition structurally orthogonal Jacobian columns or rows. This reduces differentiation costs and memory requirements while preserving all structurally nonzero derivative information. The resulting compressed Jacobian representations allow us to assemble the Gauss-Newton system directly in block-sparse form, enabling the use of sparse direct solvers and preconditioned iterative methods. In this way, the otherwise expensive second-order optimization problem is reduced to scalable sparse linear algebra. We demonstrate the approach on multiscale PDEs, where accurate approximation requires rich neural network parametrizations and large numbers of collocation points. Numerical experiments show that our block-sparse Gauss-Newton method attains high accuracy and substantially accelerates optimization relative to first-order baselines while remaining computationally tractable for large problem sizes.

Figures & tables

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Tensor-Train Compressed Separable PINNs: A Curvature-Aware Optimization Framework for Parametric PDEs in High Dimensions

    Sep 28, 2026Denis Korolev, Martin EigelNeural PDE SolversPhysics-Informed ML

  2. An Optimisation Framework for the Well-Conditioned Training of Physics-Informed Neural Networks

    Jul 2, 2026Joseph Webb, Sadok Jerad, Coralia CartisGauss-Newton OptimizationPDE Solving

  3. SS-ESOAP: Self-Scaled Adaptive Preconditioning for Physics-Informed Learning

    Aug 29, 2026Guangyuan Wang, Mads Toftrup, Sebastian Loeschcke +2Neural Network OptimizationPhysics-Informed ML