A PAC-Bayesian View of Generalisation for Physics-Informed Machine Learning

Thien V. Nguyen; Amaury Habrard; Benjamin Guedj

cs.LGMay 25, 2026

A PAC-Bayesian View of Generalisation for Physics-Informed Machine Learning

Authors: Thien V. Nguyen, Amaury Habrard, Benjamin Guedj

Abstract

Physics-informed machine learning (PIML) integrates mechanistic knowledge, typically in the form of partial differential equations (PDE), into data-driven models. Despite strong empirical performance, its statistical generalisation properties remain poorly understood, particularly in the regression setting with unbounded losses. Existing analyses rely on approximation or stability arguments and do not fully capture how physical structure influences generalisation from finite data. In this work, we develop a PAC-Bayesian framework for PIML that provides high-probability generalisation guarantees in the presence of unbounded losses. We adopt a multi-task perspective that jointly treats data fidelity, PDE residuals, initial and boundary conditions, avoiding the looseness induced by standard union-bound approaches. Our analysis leverages the structure of physics-informed objectives to derive novel bounds where the complexity scales with input-gradient norms of the losses, revealing a direct link between physical regularity and generalisation. We instantiate this framework under Sobolev and Poincaré-type assumptions, yielding two classes of bounds that trade off statistical complexity and smoothness in different regimes. Building on these results, we propose a self-bounding-aware learning algorithm that directly optimises tractable surrogates of the derived bounds, along with a practical procedure to estimate the associated constants in realistic settings. Empirical evaluations on standard PDE benchmarks demonstrate that our bounds are non-vacuous, significantly tighter than union-bound baselines, and can be effectively minimised during training. Overall, our results provide a principled statistical foundation for the generalisation of physics-informed models.

Explore similar work

Jul 29, 2026stat.ML

PIKS: Universal Physics-Informed Kernel Methods

Physics-informed machine learning incorporates physical principles --often expressed via differential operators-- into data-driven models. While physics-informed neural networks (PINNs) dominate empirical applications, the complexity of neural network architectures and optimization landscapes hinders the development of a corresponding learning theory. In turn, kernel methods offer an appealing alternative with closed-form solutions and analytical tractability, yet existing guarantees primarily cover the well-specified setting where the target belongs to the native Reproducing Kernel Hilbert Space (RKHS). This imposes unrealistic regularity assumptions that physical targets often fail to satisfy. In this paper, we introduce and analyze Physics-Informed Kernel methodS (PIKS). We establish the universal consistency of PIKS for linear differential constraints, proving that for universal kernels (such as Gaussian or Matérn), the estimator asymptotically learns the target while satisfying physical constraints. We further derive finite-sample bounds under suitable source conditions. Our analysis is based on extending classical operator-theoretic analysis of kernel methods to physics-informed machine learning. Numerical experiments demonstrate that PIKS can be competitive with PINNs and traditional finite element methods.

Joachim Bona-Pellissier, Giacomo Meanti, Matteo Santacesaria +1

Physics-Informed Machine Learning Kernel Method Kernel Hilbert Spaces Inequality Constraints Universality

Ask PDF

Jun 18, 2025cs.LG

Interpretability and Generalization Bounds for Learning Spatial Physics

While there are many applications of ML to scientific problems that look promising, visuals can be deceiving. Using numerical analysis techniques, we rigorously quantify the accuracy, convergence rates, and generalization bounds of certain ML models applied to linear differential equations for parameter discovery or solution finding. Beyond the quantity and discretization of data, we identify that the function space of the data is critical to the generalization of the model. A similar lack of generalization is empirically demonstrated for commonly used models, including physics-specific techniques. Counterintuitively, we find that different classes of models can exhibit opposing generalization behaviors. Based on our theoretical analysis, we also introduce a new mechanistic interpretability lens on scientific models whereby Green's function representations can be extracted from the weights of black-box models. Our results inform a new cross-validation technique for measuring generalization in physical systems, which can serve as a benchmark.

Alejandro Francisco Queiruga, Theo Gutman-Solo, Shuai Jiang

Physics-Informed Machine Learning Generalization Bounds Ordinary Differential Equations Model Interpretability Mechanistic Interpretability Scientific Discovery

Ask PDF

Sep 16, 2026stat.ML

Fast Learning Rates for Physics-Informed Kernel Methods

In physics-informed machine learning, a target function

u^*

is learned from noisy value observations

y_i=u^*(x_i)+ \varepsilon_i

, together with differential information, given either by noisy observations

d_j=(Du^*)(z_j)+ξ_j

or by a known physical constraint

Du^*=v

. We consider the setting where

D

is a linear differential operator and analyze a physics-informed kernel estimator

\hat u

combining

n

value observations and

m

differential observations. In this context, we ask how much can differential information improve predictions, and how does this improvement depend quantitatively on

n

m

, and

D

. We prove finite-sample bounds, supported by numerical simulations, revealing a two-regime structure for the prediction error. When

m

is limited, the rate depends jointly on

n

and

m

; when

m

exceeds a problem-dependent threshold, the rate saturates and matches the oracle rate obtained when the perfect constraint

D \hat u = Du^*

is imposed. Examples are discussed for Sobolev spaces which are reproducing kernel Hilbert spaces and include partial Laplacian constraints on the torus and gradient observations on bounded domains. These examples illustrate the range of possible learning rate improvements --- from the standard nonparametric

n^{-1/4}

to the parametric rate

n^{-1/2}

. Finally, we derive physically consistent rates in a stronger norm that jointly controls the errors in

\hat u

and

D\hat u

Luc Brogat-Motte, Joachim Bona-Pellissier, Giacomo Meanti +1

Physics-Informed Machine Learning Kernel Method Learning Rates Gradient Concentration

Ask PDF