Physics-informed random neural architectures on compact boundaryless multiply connected manifolds for random-operator models in machine learning and uncertainty quantification

Christian Soize

stat.MLOct 10, 2026

Physics-informed random neural architectures on compact boundaryless multiply connected manifolds for random-operator models in machine learning and uncertainty quantification

Authors: Christian Soize

Organizations: Université Gustave Eiffel, MSME UMR 8208, 5 bd Descartes, 77454 Marne-la-Vallée, France

Abstract

We introduce stochastic physics-informed neural networks (SPINNs) for stochastic boundary-value problems (SBVPs) with random operators. The method enforces physics through a structural probabilistic coupling: a common Gaussian stochastic germ drives both the SBVP random operator and a manifold-based stochastic neural architecture. The stochastic neural network is non-feedforward and intrinsically random. A latent Gaussian field and an inhomogeneous Poisson point process on a compact, boundaryless, multiply connected manifold generate random neuron locations, geodesically local sparse connectivity, and a sparse random weight matrix, all governed by a low-dimensional hyperparameter. For deterministic inputs, the network output is random and can represent non-Gaussian random operators. Training is performed in observation space: SBVP simulations produce random observables, while the SNN defines a predictive conditional probability density function. The supervised objective combines data fidelity, through negative log-likelihood, with a cross-entropy alignment term relative to the SBVP-induced conditional distribution. The resulting target is a convex mixture of observation-induced and SBVP-induced distributions, with a dimensionless parameter controlling robustness to model--data mismatch. The shared stochastic germ yields common-random-numbers Monte Carlo estimators and reduced-variance gradients. The method uses only evaluations of the SBVP solution mapping, without differentiating or modifying it, and avoids PDE-residual or boundary-penalty losses. A stochastic elliptic problem on a periodic cylindrical surface illustrates the approach in SBVP-consistent and SBVP-misspecified regimes. A generalized-posterior formulation is also outlined for hyperparameter inference and posterior averaging without per-query reoptimization.

Figures & tables

Panel of Figure 1 — Figure 1: SBVP: (a) Structured mesh $\accentset{\rule{9.65991pt}{0.3pt}}{{\mathscr{S}}}^{\,h}_{c}$ approximating $\accentset{\rule{9.65991pt}{0.3pt}}{{\mathscr{S}}}{\!{}_{c}}$ . (b) Graph of the eigenvalues $\mu_{1},\ldots,\mu_{m}$ of the covariance matrix $[C_{U_{c}}]$ , plotted on a $\log_{10}$ scale. (c) A realization of the random field $\{U_{c}^{h}({\textbf{x}}),{\textbf{x}}\in\accentset{\rule{9.65991pt}{0.3pt}}{{\mathscr{S}}}^{\,h}_{c}\}$ .

Panel of Figure 2 — Figure 2: SNN. (a) Structured mesh ${\mathscr{S}}_{h}$ approximating ${\mathscr{S}}$ . (b) $\log_{10}$ -scale plot of the dominant eigenvalues $\lambda_{1},\ldots,\lambda_{m}$ of $[C_{{\textbf{U}}}]$ for prescribed $(\tau_{0},h_{1},h_{2},n_{h})$ and one realization of $({\bm{\beta}}^{(1)},{\bm{\beta}}^{(2)})$ . (c) Corresponding realization of $\{U^{h}_{\bm{\theta}}({\textbf{x}}),\,{\textbf{x}}\in{\mathscr{S}}_{h}\}$ .

Panel of Figure 3 — Figure 3: SNN Case 1: (a) realization of the random neuron locations on ${\mathscr{S}}_{h}$ , superimposed on the associated realization of the latent random field. (b) Realization of the sparse connectivity corresponding to the edge-mask matrix, but drawn along the geodesics.

Panel of Figure 4 — Figure 4: Case 1: $\log_{10}$ -scale plots of the graphs of functions ${\mathcal{J}}^{\hbox{\pppppcarac SNN}}$ , $\widehat{\rm{CE}}$ , and ${\mathcal{J}}^{\hbox{\pppppcarac SPINN}}(\cdot,;\alpha_{0})$ on the grid points $j\in\{1,\ldots,324\}$ .

Panel of Figure 5 — Figure 5: Case 1: Plots of the functions $k\mapsto\beta_{k,n_{h},{\hbox{{\ppppcarac trial}}}}^{\,(1),{\,\hbox{{\ppppcarac opt}}}}$ and $k\mapsto\beta_{k,n_{h},{\hbox{{\ppppcarac trial}}}}^{\,(2),{\,\hbox{{\ppppcarac opt}}}}$ on $\{1,\ldots,32\}$ , and of $i\mapsto\beta_{i,{\hbox{{\ppppcarac trial}}}}^{\,\hbox{{\ppppcarac opt}}}$ on $\{1,\ldots,80\}$ .

Panel of Figure 6 — Figure 6: Case 1: Plots of the functions $n\mapsto\log_{10}({\mathcal{J}}^{\hbox{\pppppcarac SPINN}}(n))$ , $k\mapsto{\bm{\beta}}_{k,n_{h}}^{\,(1),{\,\hbox{{\ppppcarac opt}}}}$ , and $k\mapsto{\bm{\beta}}_{k,n_{h}}^{\,(2),{\,\hbox{{\ppppcarac opt}}}}$ on $\{1,\ldots,32\}$ .

Eqs.	( 3.2 )	( 3.1 )	( 3.3 )	( 3.5 )	( 3.5 )	( 3.6 )	( 3.9 )	( 3.9 )	( 3.10 )
	$\scriptstyle{\hbox{\ppcarac NLL}}_{\hbox{\ppppcarac SNN}}^{\,\hbox{{\pppcarac opt}}}$	$\scriptstyle{\hbox{\ppcarac NLL}}_{\hbox{\ppppcarac SNN}}^{\hbox{{\pppcarac test}}}$	$\scriptstyle\hbox{{\ppcarac Cr}}_{o}$	$\scriptstyle\hbox{{\ppcarac Cr}}_{{\bm{\alpha}}}$	$\scriptstyle\hbox{{\ppcarac Cr}}_{{\bm{\beta}}}$	$\scriptstyle\hbox{{\ppcarac Cr}}_{{\hbox{\ppppcarac IOC}}}$	$\scriptstyle{\hbox{{\ppcarac CRPS}}}^{\hbox{\pppppcarac SNN}}_{\hbox{{\ppppcarac test}}}$	$\scriptstyle{\hbox{{\ppcarac CRPS}}}^{\hbox{{\ppppcarac train}}}_{\hbox{{\ppppcarac test}}}$	$\scriptstyle{\hbox{{\ppcarac CRPS}}}^{\hbox{{\ppppcarac scaled}}}_{\hbox{{\ppppcarac test}}}$
TRIAL	7839.4	1521.5	0.03046	0.05574	0.05459	0.2350	0.043313	0.040216	1.0770
PGD	7852.6	1519.8	0.03338	0.05479	0.05274	0.2273	0.042751	0.040216	1.0630

Table 1: Case 1: Evaluation-criteria values of the reference configuration (

N=200

n_{h}=32

, and

n_{\hbox{{\ppppcarac sim}}}=100

$\scriptstyle k$	$\scriptstyle\hbox{{\pcarac CI}}^{\hbox{{\ppppcarac train}}}_{k}({\mathbb{x}}^{6}_{\hbox{{\ppppcarac test}}})$	$\scriptstyle\hbox{{\pcarac CI}}^{\hbox{\pppppcarac SNN}}_{k}({\mathbb{x}}^{6}_{\hbox{{\ppppcarac test}}})$	$\scriptstyle\hbox{{\pcarac CI}}^{\hbox{{\ppppcarac train}}}_{k}({\mathbb{x}}^{24}_{\hbox{{\ppppcarac test}}})$	$\scriptstyle\hbox{{\pcarac CI}}^{\hbox{\pppppcarac SNN}}_{k}({\mathbb{x}}^{24}_{\hbox{{\ppppcarac test}}})$
14	$[-0.8472,-0.5951]$	$[-0.8097,-0.5537]$	$[-0.7526,-0.5485]$	$[-0.8241,-0.5810]$
35	$[1.0775,1.4817]$	$[1.0125,1.4362]$	$[1.0061,1.3164]$	$[1.0106,1.4174]$
41	$[1.3910,1.8954]$	$[1.2778,1.8351]$	$[1.2755,1.6860]$	$[1.2772,1.8219]$

Table 2: Case 1: For

k=14

35

, and

41

, conditional confidence intervals

\hbox{{\pcarac CI}}^{\hbox{{\ppppcarac train}}}_{k}({\mathbb{x}}^{i}_{\hbox{{\ppppcarac test}}})

and

\hbox{{\pcarac CI}}^{\hbox{\pppppcarac SNN}}_{k}({\mathbb{x}}^{i}_{\hbox{{\ppppcarac test}}})

for inputs

i\in\{6,24\}

from the test dataset, with

N=200

n_{h}=32

, and

n_{\hbox{{\ppppcarac sim}}}=100

Panel of Figure 7 — Figure 7: Case 1: Plots of the probability density functions $y\mapsto p_{YY_{k}|{\mathbb{X}}}^{\hbox{\pppppcarac SNN}}(y|{\mathbb{x}}_{\hbox{{\ppppcarac test}}}^{6};{\bm{\theta}}^{\,\hbox{{\ppppcarac opt}}})$ (thin black line) and $y\mapsto p_{k}^{\hbox{{\ppppcarac train}}}(y|{\mathbb{x}}_{\hbox{{\ppppcarac test}}}^{6})$ (thick blue line)

Panel of Figure 8 — Figure 8: Case 1: Plots of the probability density functions $y\mapsto p_{YY_{k}|{\mathbb{X}}}^{\hbox{\pppppcarac SNN}}(y|{\mathbb{x}}_{\hbox{{\ppppcarac test}}}^{24};{\bm{\theta}}^{\,\hbox{{\ppppcarac opt}}})$ (thin black line) and $y\mapsto p_{k}^{\hbox{{\ppppcarac train}}}(y|{\mathbb{x}}_{\hbox{{\ppppcarac test}}}^{24})$ (thick blue line)

Panel of Figure 9 — Figure 9: Case 1: Convergence analysis with respect to $n_{h}$ , $n_{\hbox{{\ppppcarac sim}}}$ , and $N$ , with $n_{h}^{\hbox{{\ppppcarac rc}}}=32$ , $n_{\hbox{{\ppppcarac sim}}}^{\hbox{{\ppppcarac rc}}}=100$ , and $N^{\hbox{{\ppppcarac rc}}}=200$ . Plots of the following functions:(a) $n_{h}\mapsto{\hbox{{\ppcarac CRPS}}}^{\hbox{{\ppppcarac scaled}}}_{\hbox{{\ppppcarac test}}}(n_{h},n_{\hbox{{\ppppcarac sim}}}^{\hbox{{\ppppcarac rc}}},N^{\hbox{{\ppppcarac rc}}})$ , (b) $n_{\hbox{{\ppppcarac sim}}}\mapsto{\hbox{\ppcarac NLL}}_{\hbox{\ppppcarac SNN}}^{\,\hbox{{\pppcarac opt}}}(n_{h}^{\hbox{{\ppppcarac rc}}},n_{\hbox{{\ppppcarac sim}}},N^{\hbox{{\ppppcarac rc}}})$ , and (c) $N\mapsto{\hbox{{\ppcarac CRPS}}}^{\hbox{{\ppppcarac scaled}}}_{\hbox{{\ppppcarac test}}}(n_{h}^{\hbox{{\ppppcarac rc}}},n_{\hbox{{\ppppcarac sim}}}^{\hbox{{\ppppcarac rc}}},N)$ .

Eqs.	( 4.12 )	( 4.24 )	( 4.33 )	( 6.7 )
$N$	$n_{\hbox{{\pppcarac int}}}$	$n_{\hbox{{\pppcarac inc}}}$	$n_{b}$	$\alpha_{0}$
130	48	3426	32	1.0717
200	118	8067	80	1.0458
250	168	12625	114	1.0718
300	218	13156	148	1.0688

Table 3: Case 1:Neuron-architecture parameters and

\alpha_{0}

values for the alignment term, as a function of

N\in\{130,200,250,300\}

, for

n_{h}=n_{h}^{\hbox{{\ppppcarac rc}}}=32

and

n_{\hbox{{\ppppcarac sim}}}=n_{\hbox{{\ppppcarac sim}}}^{\hbox{{\ppppcarac rc}}}=100

Eqs.	( 3.2 )	( 3.1 )	( 3.3 )	( 3.5 )	( 3.5 )	( 3.6 )	( 3.9 )	( 3.9 )	( 3.10 )
$N$	$\scriptstyle{\hbox{\ppcarac NLL}}_{\hbox{\ppppcarac SNN}}^{\,\hbox{{\pppcarac opt}}}$	$\scriptstyle{\hbox{\ppcarac NLL}}_{\hbox{\ppppcarac SNN}}^{\hbox{{\pppcarac test}}}$	$\scriptstyle\hbox{{\ppcarac Cr}}_{o}$	$\scriptstyle\hbox{{\ppcarac Cr}}_{{\bm{\alpha}}}$	$\scriptstyle\hbox{{\ppcarac Cr}}_{{\bm{\beta}}}$	$\scriptstyle\hbox{{\ppcarac Cr}}_{{\hbox{\ppppcarac IOC}}}$	$\scriptstyle{\hbox{{\ppcarac CRPS}}}^{\hbox{\pppppcarac SNN}}_{\hbox{{\ppppcarac test}}}$	$\scriptstyle{\hbox{{\ppcarac CRPS}}}^{\hbox{{\ppppcarac train}}}_{\hbox{{\ppppcarac test}}}$	$\scriptstyle{\hbox{{\ppcarac CRPS}}}^{\hbox{{\ppppcarac scaled}}}_{\hbox{{\ppppcarac test}}}$
130	7765.0	1481.9	0.04796	0.05168	0.04783	0.2214	0.042364	0.040216	1.0534
200	7852.6	1519.8	0.03338	0.05479	0.05274	0.2273	0.042751	0.040216	1.0630
250	8002.6	1495.8	0.07003	0.05281	0.04728	0.2290	0.041656	0.040216	1.0358
300	7931.6	1492.2	0.06308	0.05292	0.04879	0.2242	0.040939	0.040216	1.0180

Table 4: Case 1: Evaluation-criteria values for the PGD-Adam optimal solution as a function of

N\in\{130,200,250,300\}

, for

n_{h}=n_{h}^{\hbox{{\ppppcarac rc}}}=32

and

n_{\hbox{{\ppppcarac sim}}}=n_{\hbox{{\ppppcarac sim}}}^{\hbox{{\ppppcarac rc}}}=100

Panel of Figure 10 — Figure 10: Case 2: $\log_{10}$ -scale plots of the graphs of functions ${\mathcal{J}}^{\hbox{\pppppcarac SNN}}$ , $\widehat{\rm{CE}}$ , and ${\mathcal{J}}^{\hbox{\pppppcarac SPINN}}(\cdot,;\alpha_{0})$ on the grid points $j\in\{1,\ldots,648\}$ .

Panel of Figure 11 — Figure 11: Case 2: Plots of the functions $n\mapsto\log_{10}({\mathcal{J}}^{\hbox{\pppppcarac SPINN}}(n))$ , $k\mapsto{\bm{\beta}}_{k,n_{h}}^{\,(1),{\,\hbox{{\ppppcarac opt}}}}$ , and $k\mapsto{\bm{\beta}}_{k,n_{h}}^{\,(2),{\,\hbox{{\ppppcarac opt}}}}$ on $\{1,\ldots,32\}$ .

Eqs.	( 3.2 )	( 3.1 )	( 3.3 )	( 3.5 )	( 3.5 )	( 3.6 )	( 3.9 )	( 3.9 )	( 3.10 )
	$\scriptstyle{\hbox{\ppcarac NLL}}_{\hbox{\ppppcarac SNN}}^{\,\hbox{{\pppcarac opt}}}$	$\scriptstyle{\hbox{\ppcarac NLL}}_{\hbox{\ppppcarac SNN}}^{\hbox{{\pppcarac test}}}$	$\scriptstyle\hbox{{\ppcarac Cr}}_{o}$	$\scriptstyle\hbox{{\ppcarac Cr}}_{{\bm{\alpha}}}$	$\scriptstyle\hbox{{\ppcarac Cr}}_{{\bm{\beta}}}$	$\scriptstyle\hbox{{\ppcarac Cr}}_{{\hbox{\ppppcarac IOC}}}$	$\scriptstyle{\hbox{{\ppcarac CRPS}}}^{\hbox{\pppppcarac SNN}}_{\hbox{{\ppppcarac test}}}$	$\scriptstyle{\hbox{{\ppcarac CRPS}}}^{\hbox{{\ppppcarac train}}}_{\hbox{{\ppppcarac test}}}$	$\scriptstyle{\hbox{{\ppcarac CRPS}}}^{\hbox{{\ppppcarac scaled}}}_{\hbox{{\ppppcarac test}}}$
TRIAL	8332.1	2189.9	0.24217	0.08323	0.07075	0.2703	0.05872	0.04139	1.4185
PGD	8554.8	1700.5	0.00615	0.08557	0.07109	0.2657	0.05579	0.04149	1.3478

Table 5: Case 2: Evaluation-criteria values of the reference configuration (

N=200

n_{h}=32

, and

n_{\hbox{{\ppppcarac sim}}}=100

$\scriptstyle k$	$\scriptstyle\hbox{{\pcarac CI}}^{\hbox{{\ppppcarac train}}}_{k}({\mathbb{x}}^{6}_{\hbox{{\ppppcarac test}}})$	$\scriptstyle\hbox{{\pcarac CI}}^{\hbox{\pppppcarac SNN}}_{k}({\mathbb{x}}^{6}_{\hbox{{\ppppcarac test}}})$	$\scriptstyle\hbox{{\pcarac CI}}^{\hbox{{\ppppcarac train}}}_{k}({\mathbb{x}}^{24}_{\hbox{{\ppppcarac test}}})$	$\scriptstyle\hbox{{\pcarac CI}}^{\hbox{\pppppcarac SNN}}_{k}({\mathbb{x}}^{24}_{\hbox{{\ppppcarac test}}})$
14	$[-0.7260,-0.4726]$	$[-0.7333,-0.4249]$	$[-0.6375,-0.4233]$	$[-0.7347,-0.4264]$
35	$[0.8761,1.2625]$	$[0.7741,1.2307]$	$[0.8054,1.0856]$	$[0.7620,1.2102]$
41	$[1.1512,1.6467]$	$[1.0590,1.5940]$	$[1.0288,1.4066]$	$[0.9874,1.5006]$

Table 6: Case 2: For

k=14

35

, and

41

, conditional confidence intervals

\hbox{{\pcarac CI}}^{\hbox{{\ppppcarac train}}}_{k}({\mathbb{x}}^{i}_{\hbox{{\ppppcarac test}}})

and

\hbox{{\pcarac CI}}^{\hbox{\pppppcarac SNN}}_{k}({\mathbb{x}}^{i}_{\hbox{{\ppppcarac test}}})

for inputs

i\in\{6,24\}

from the test dataset, with

N=200

n_{h}=32

, and

n_{\hbox{{\ppppcarac sim}}}=100

Panel of Figure 12 — Figure 12: Case 2: Plots of the probability density functions $y\mapsto p_{YY_{k}|{\mathbb{X}}}^{\hbox{\pppppcarac SNN}}(y|{\mathbb{x}}_{\hbox{{\ppppcarac test}}}^{6};{\bm{\theta}}^{\,\hbox{{\ppppcarac opt}}})$ (thin black line) and $y\mapsto p_{k}^{\hbox{{\ppppcarac train}}}(y|{\mathbb{x}}_{\hbox{{\ppppcarac test}}}^{6})$ (thick blue line), which should not be expected to closely align.

Panel of Figure 13 — Figure 13: Case 2: Plots of the probability density functions $y\mapsto p_{YY_{k}|{\mathbb{X}}}^{\hbox{\pppppcarac SNN}}(y|{\mathbb{x}}_{\hbox{{\ppppcarac test}}}^{24};{\bm{\theta}}^{\,\hbox{{\ppppcarac opt}}})$ (thin black line) and $y\mapsto p_{k}^{\hbox{{\ppppcarac train}}}(y|{\mathbb{x}}_{\hbox{{\ppppcarac test}}}^{24})$ (thick blue line), which should not be expected to closely align.

Eqs.	( 4.12 )	( 4.24 )	( 4.33 )	( 6.7 )
$N$	$n_{\hbox{{\pppcarac int}}}$	$n_{\hbox{{\pppcarac inc}}}$	$n_{b}$	$\alpha_{0}$
130	48	4267	32	0.3247
200	118	8053	80	0.3543
250	168	12607	114	0.4372
300	218	18120	148	0.3621

Table 7: Case 2: Neuron-architecture parameters and

\alpha_{0}

values for the alignment term, as a function of

N\in\{130,200,250,300\}

, for

n_{h}=32

and

n_{\hbox{{\ppppcarac sim}}}=100

Eqs.	( 3.2 )	( 3.1 )	( 3.3 )	( 3.5 )	( 3.5 )	( 3.6 )	( 3.9 )	( 3.9 )	( 3.10 )
$N$	$\scriptstyle{\hbox{\ppcarac NLL}}_{\hbox{\ppppcarac SNN}}^{\,\hbox{{\pppcarac opt}}}$	$\scriptstyle{\hbox{\ppcarac NLL}}_{\hbox{\ppppcarac SNN}}^{\hbox{{\pppcarac test}}}$	$\scriptstyle\hbox{{\ppcarac Cr}}_{o}$	$\scriptstyle\hbox{{\ppcarac Cr}}_{{\bm{\alpha}}}$	$\scriptstyle\hbox{{\ppcarac Cr}}_{{\bm{\beta}}}$	$\scriptstyle\hbox{{\ppcarac Cr}}_{{\hbox{\ppppcarac IOC}}}$	$\scriptstyle{\hbox{{\ppcarac CRPS}}}^{\hbox{\pppppcarac SNN}}_{\hbox{{\ppppcarac test}}}$	$\scriptstyle{\hbox{{\ppcarac CRPS}}}^{\hbox{{\ppppcarac train}}}_{\hbox{{\ppppcarac test}}}$	$\scriptstyle{\hbox{{\ppcarac CRPS}}}^{\hbox{{\ppppcarac scaled}}}_{\hbox{{\ppppcarac test}}}$
130	8155.2	1653.7	0.01372	0.08581	0.07546	0.2705	0.05332	0.04139	1.2881
200	8554.8	1700.0	0.00615	0.08557	0.07109	0.2657	0.05579	0.04139	1.3478
250	7923.0	1570.2	0.00918	0.08086	0.06821	0.2593	0.04967	0.04139	1.2000
300	8984.9	1752.8	0.02524	0.09526	0.07900	0.2805	0.05970	0.04139	1.4423

Table 8: Case 2: Evaluation-criteria values for the PGD-Adam optimal solution as a function of

N\in\{130,200,250,300\}

, for

n_{h}=32

and

n_{\hbox{{\ppppcarac sim}}}=100

Physics-Informed ML

Physics-informed machine learning combines data-driven models with physical laws to solve, predict, and infer scientific systems.

See all 836 papers

Oct 2, 2026

S $^{2}$ -PINN: Stochastic Separable Physics-Informed Neural Networks

Uncertainty quantification (UQ) for random partial differential equations (PDEs) is ubiquitous in computational science and engineering. However, classical spectral solvers for this class of problems face the curse of dimensionality, and existing neural solvers often ignore the stochastic structure that makes moments and calibration tractable. We introduce a stochastic separable physics-informed neural network, dubbed S

^{2}

-PINN, that represents the solution

u(t,\mathbf{x},\mathbf{Z})

of a random PDE with a learnable Gaussian spatial dictionary, Fourier temporal features, and a generalized polynomial chaos (gPC) stochastic basis, coupled by a low-rank Canonical Polyadic (CP) tensor decomposition core. The method is trained with a hybrid strong-form and gPC-projected residual loss. Our theoretical analysis establishes that the separable class is dense in

L^2

under mild conditions, and the projected residual corresponds exactly to a stochastic Galerkin constraint. Furthermore, we show that mini-batch projection coefficients are logarithmically dependent on the number of gPC modes, and that the orthogonality penalty controls the conditioning of the learned spatial dictionary. Using four manufactured random PDE benchmarks, we show that S

^{2}

-PINN outperforms nine baselines in terms of mean and variance accuracy, as well as calibration, while using significantly fewer parameters. Further evaluations on non-manufactured Poisson and Darcy problems, a stochastic Navier--Stokes problem, a diffusion scaling study of higher random dimensions, and two stochastic inverse problems reveal the generalization capabilities of the proposed structure. Together, these results support stochastic separability as an effective design principle for physics-informed neural UQ. The code for the experiments can be found in https://github.com/DMax1314/s2pinn

Zhendong Li, Akwum Onwunta

Industrial and Systems Engineering Dept. Lehigh University Bethlehem, PA 18015, USA

Stochastic PDEs Neural PDE Solvers Uncertainty Quantification Physics-Informed ML

Ask PDF

Aug 29, 2025

Convergence of Stochastic Gradient Methods for Wide Two-Layer Physics-Informed Neural Networks for the Poisson Equation

Physics informed neural networks (PINNs) represent a very popular class of neural solvers for partial differential equations. In practice, one often employs stochastic gradient descent type algorithms to train the neural network. Therefore, the convergence guarantee of stochastic gradient descent is of fundamental importance. In this work, we establish the linear convergence of stochastic gradient descent / flow in training over-parameterized two layer PINNs with a general class of activation functions for solving one model second-order elliptic problem, i.e., the Poisson equation, in the sense of high probability. These results extend the existing result [20] in which gradient descent was analyzed. The challenge of the analysis lies in handling the dynamic randomness introduced by stochastic optimization methods. The key of the analysis lies in ensuring the positive definiteness of suitable Gram matrices during the training. The analysis sheds insight into the dynamics of the optimization process, and provides guarantees on physics informed neural networks trained by stochastic algorithms.

Bangti Jin, Longjun Wu

Department of Mathematics, The Chinese University of Hong Kong, Shatin, N.T., Hong Kong, P.R. China

Neural PDE Solvers PDE Solving Stochastic Optimization Convergence Physics-Informed ML Stochastic Gradient Descent

Ask PDF

Sep 14, 2026

Physics Informed Random Feature Neural Networks for Solving PDEs

Machine learning-based partial differential equations (PDEs) solvers have attracted significant attention in recent years. Most progress in this area has been driven by deep neural networks such as physics-informed neural networks (PINNs) and kernel method (such as physics-informed Gaussian Processes). We introduce a physics-informed random feature method for countering part of the spectral bias which PINN-based solvers are facing for a certain class of PDEs. Random feature method was originally proposed to approximate large-scale kernel machines and can be viewed as a specialized randomized neural network. Compared to other state-of-the-art PINN-based solvers which require a large number of collocation points, our proposed method reduces the computational complexity. In this paper, we develop a rigorous approximation error analysis and derive high-probability error bounds on the

H^1

norm. We provide extensive numerical tests for verifying our theoretical guarantees on error decay rates, as well as several comparison tests to showcase our claimed capability for combating spectral bias in these deep learning based methods.

Chi-An Chen, Chunyang Liao, Ming Zhong

Department of Applied Mathematics, Mathematics, Illinois Institute of Technology, Chicago, IL, 60616 · Department of Mathematical Sciences, University of Arkansas, Fayetteville, AR, 72701 · Department of Mathematics, University of Houston, Houston, TX, 77204

PDE Solving Random Feature Methods Neural Network Approximation Theory Physics-Informed ML Partial Differential Equations

Ask PDF

Abstract

Physics-Informed ML

Explore similar work

S2^{2}2-PINN: Stochastic Separable Physics-Informed Neural Networks

Convergence of Stochastic Gradient Methods for Wide Two-Layer Physics-Informed Neural Networks for the Poisson Equation

Physics Informed Random Feature Neural Networks for Solving PDEs

S $^{2}$ -PINN: Stochastic Separable Physics-Informed Neural Networks