Uncovering and Fixing Collider Bias in Bayesian PINNs

Michael Obermayr; Robert Peharz

cs.LGOct 8, 2026

Uncovering and Fixing Collider Bias in Bayesian PINNs

Authors: Michael Obermayr, Robert Peharz

Organizations: TU Graz

Abstract

Bayesian physics-informed neural networks (B-PINNs) are a popular framework for parameter and state inference from sparse or noisy observations. They are commonly formulated via a collider structure, in which physical and trajectory parameters are assumed to be a priori independent and become coupled through virtual likelihoods on differential-equation residuals that enforce physical consistency. We show that this modeling choice can induce severe systematic bias in the posterior over physical parameters: even when the prior is favorably centered on the ground-truth parameters, the resulting posterior can drift away and concentrate far from them. As a remedy, we advocate a hierarchical chain model in which physics generates trajectories, which in turn generate observations. The chain model does not suffer from this posterior bias, but it poses a harder, so-called doubly intractable, inference problem due to a physics-dependent normalization constant. This challenge can be resolved by discretizing the underlying stochastic dynamics, after which the chain posterior can be sampled exactly with particle MCMC. We identify two distinct mechanisms characterizing the collider bias, derive analytical approximations of their magnitudes, and establish diagnostic criteria for predicting when standard B-PINNs remain reliable. Experiments confirm the predicted bias and show that the chain formulation successfully avoids it.

Figures & tables

Panel of Figure 1 — Figure 1: Modeling choices for physics-informed Bayesian inference. (a) Collider (B-PINN): Physics parameters $\lambda$ and trajectory parameters $\theta$ are a priori independent, interacting only when conditioned on equation residuals $r$ . (b) Chain (ours): Physics $\lambda$ directly generates trajectories $\theta$ , which in turn generate data $y$ .

Figure 2: Posterior marginals for the damped linear oscillator. System governed by

\ddot{x}=-\frac{c}{m}\dot{x}-\frac{k}{m}x+\sigma_{r}\xi(t)

with white noise

\xi(t)

, mass

m

, damping

c

and spring constant

k

, ground-truth parameters

\{m,c,k\}=\{1.0,1.2,4.0\}

, residual scale fixed at

\sigma_{r}=0.3

, and 40 noisy observations (

\sigma_{\text{obs}}=0.02

). Posteriors are sampled via NUTS (24,000 samples) for B-PINN and PMMH [ 2 ] (14,000 samples) for our proposed chain model. Although all three physical parameters are inferred jointly, only the ratios

c/m

and

k/m

are identifiable; we therefore display their corresponding marginal posteriors. Despite initialization at the ground truth and identical priors centered at the ground truth, only the chain model accurately recovers the true values. In contrast, B-PINN exhibits a systematic collider bias, underestimating dissipation (

c/m

) and stiffness (

k/m

). The bias is reproducible over resampled datasets. Full experimental details are provided in Section 5 .

Acceleration residuals ( $A_{\lambda}=I_{d}$ ): metric bias inactive
	Oscillator		Double Pendulum		Relativistic Particle
Model (latent)	$\log\alpha$	$c/m$	$\log\alpha$	$m_{2}/m_{1}$	$\log\alpha$	$c$
Chain (state space)	$-0.01$	0.99	$-0.03$	0.99	$-0.02$	0.89
Collider (state space)	$-0.01$	0.99	$-0.03$	0.99	$-0.02$	0.89
Chain (Fourier MLP)	$-0.00$	1.00	—	—	—	—
Collider B-PINN (Fourier MLP)	$-0.00$	0.61	$-0.03^{\dagger}$	0.94	$+0.00$	1.78

Table 1: The chain model recovers the truth in every system; the collider fails where Section 3 predicts.

\log\alpha

: posterior mean of the log scale factor along the weakly identified drift-invariant direction (Eq. ( 37 ); correct:

\approx 0

). Other columns: posterior mean of an identified quantity divided by its true value (correct:

1.00

). Bold : off by more than a factor of two.

\dagger

: chains did not converge or stalled; median of the draws, for both columns of the system.

\ddagger

: exact Kalman likelihood, since PMMH cannot reach this posterior. All parameters in Appendix A.3.3 .

Figure 3: Longer records amplify the dissipation bias (oscillator, acceleration residuals). Posterior mean of

\log(c/m)

minus its true value, with

\pm 1

posterior standard deviation. The chain stays near the truth on both latents, and its posterior contracts as

\mathcal{T}

grows; on the state-space latent the collider coincides with it (open rings). The B-PINN drifts further from the truth while its posterior width stays constant, so its error grows from

1.55

2.67

standard deviations.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Mechanism	Physical Origin	Scaling	Null Condition
Residual Metric	State-space map $A_{\lambda}$ depends on $\lambda$	$\Theta(n_{\mathrm{eff}})$ or $\Theta(\mathcal{T})$	$A_{\lambda}$ invariant to $\lambda$
Dissipation	Phase-space volume contraction	$\Theta(\mathcal{T})$	Markovian latent, or $\sum_{k}\lvert\operatorname{Re}s_{k}\rvert$ invariant to $\lambda$

Appendix

Table 2: Taxonomy of structural biases in collider B-PINNs. Scaling denotes the asymptotic rate of the collider bias

\Delta=\log(p_{\mathrm{collider}}/p_{\mathrm{chain}})

. Null conditions indicate when

\Delta

is invariant to inferred parameters;

\{s_{k}\}

are eigenvalues of

Df_{\lambda}

	Oscillator	Double pendulum	Relativistic particle
ground truth $\lambda$	$(m,c,k)=(1,\,1.2,\,4)$	$(m_{1},m_{2},\ell_{1},\ell_{2})=(1,\,0.5,\,1,\,0.8)$	$(m_{0},q,c)=(1,\,1,\,1)$
residual scale $\sigma_{r}$ (fixed)	$0.30$	$0.02$	$0.02$
horizon $\mathcal{T}$	$24$	$12$	$12$
observations $K$	$40$	$20$	$20$
observation noise $\sigma_{\mathrm{obs}}$	$0.02$	$0.02$ rad	$0.005$
Euler–Maruyama steps $T$	$1248$	$1235$	$12160$

Appendix

Table 3: Per-system settings. The MLP surrogate maps a Fourier-feature embedding of dimension

33

through two hidden layers of width

24

to the

d

output coordinates.

Nd

is the number of residual components, i.e., collocation points times dimension.

residual	leg	$m$ (1.00)	$c$ (1.20)	$k$ (4.00)	$c/m$ (1.20)	$k/m$ (4.00)
force residual	Chain (state space)	$1.015\pm 0.14$	$1.209\pm 0.20$	$3.981\pm 0.51$	$1.209\pm 0.24$	$3.948\pm 0.43$
	Collider (state space)	$9.3{\cdot}10^{-6}$ $\ddagger$	$3.7{\cdot}10^{-4}$ $\ddagger$	$0.03387$ $\ddagger$	$39.74$ $\ddagger$	$3656$ $\ddagger$
	Collider (B-PINN)	$0.2391\pm 0.050$	$0.5663\pm 0.13$	$1.832\pm 0.25$	$2.462\pm 0.73$	$7.935\pm 1.7$
acceleration residual	Chain (state space)	$1.035\pm 0.22$	$1.219\pm 0.28$	$4.030\pm 0.79$	$1.191\pm 0.20$	$3.927\pm 0.41$
	Collider (state space)	$1.035\pm 0.22$	$1.219\pm 0.28$	$4.030\pm 0.79$	$1.191\pm 0.20$	$3.927\pm 0.41$
	Collider (B-PINN)	$1.314\pm 0.28$	$0.9419\pm 0.24$	$4.115\pm 0.81$	$0.7311\pm 0.18$	$3.162\pm 0.35$

Appendix

Table 4: Oscillator. The three inference legs under both residual metrics, on one dataset and one prior, with

\sigma_{r}

held at its true value. Posterior mean

\pm

standard deviation over 14000 draws (state space) and 24000 draws (B-PINN).

\dagger

: a walk at the Euler map’s stability boundary, not a posterior; its median is quoted;

\ddagger

: chains that have not merged (

\hat{R}>1.05

), likewise a median rather than a posterior.

residual	leg	$m_{1}$ (1.00)	$m_{2}$ (0.500)	$\ell_{1}$ (1.00)	$\ell_{2}$ (0.800)	$m_{1}{+}m_{2}$ (1.50)	$m_{2}/m_{1}$ (0.500)
force residual	Chain (state space)	$1.078\pm 0.26$	$0.5367\pm 0.13$	$1.011\pm 0.011$	$0.7890\pm 0.010$	$1.615\pm 0.39$	$0.4976\pm 0.012$
	Collider (state space)	$0.1713$ $\ddagger$	$0.02538$ $\ddagger$	$1.386$ $\ddagger$	$0.07589$ $\ddagger$	$0.1967$ $\ddagger$	$0.1481$ $\ddagger$
	Collider (B-PINN)	$0.08769\pm 0.020$	$0.02269\pm 0.0054$	$0.1441\pm 0.019$	$0.2137\pm 0.034$	$0.1104\pm 0.022$	$0.2709\pm 0.085$
acceleration residual	Chain (state space)	$1.021\pm 0.30$	$0.5062\pm 0.15$	$1.011\pm 0.012$	$0.7926\pm 0.011$	$1.527\pm 0.45$	$0.4958\pm 0.011$
	Collider (state space)	$1.021\pm 0.30$	$0.5062\pm 0.15$	$1.011\pm 0.012$	$0.7926\pm 0.011$	$1.527\pm 0.45$	$0.4958\pm 0.011$
	Collider (B-PINN)	$1.004$ $\ddagger$	$0.4715$ $\ddagger$	$1.023$ $\ddagger$	$0.7808$ $\ddagger$	$1.476$ $\ddagger$	$0.4699$ $\ddagger$

Appendix

Table 5: Double pendulum. The three inference legs under both residual metrics, on one dataset and one prior, with

\sigma_{r}

held at its true value. Posterior mean

\pm

standard deviation over 14000 draws (state space) and 24000 draws (B-PINN).

\dagger

: a walk at the Euler map’s stability boundary, not a posterior; its median is quoted;

\ddagger

: chains that have not merged (

\hat{R}>1.05

), likewise a median rather than a posterior.

residual	leg	$m_{0}$ (1.00)	$q$ (1.00)	$c$ (1.00)	$q/m_{0}$ (1.00)
force residual	Chain (state space)	$0.8233\pm 0.15$	$0.8681\pm 0.15$	$0.8939\pm 0.12$	$1.057\pm 0.055$
	Collider (state space)	$2.1{\cdot}10^{-5}$ $\ddagger$	$0.00383$ $\ddagger$	$4133$ $\ddagger$	$182.8$ $\ddagger$
	Collider (B-PINN)	$0.1630\pm 0.036$	$0.1485\pm 0.032$	$1.608\pm 0.27$	$0.9127\pm 0.036$
acceleration residual	Chain (state space)	$0.9941\pm 0.29$	$1.049\pm 0.31$	$0.8881\pm 0.087$	$1.056\pm 0.046$
	Collider (state space)	$0.9941\pm 0.29$	$1.049\pm 0.31$	$0.8881\pm 0.087$	$1.056\pm 0.046$
	Collider (B-PINN)	$1.101\pm 0.33$	$0.9872\pm 0.29$	$1.783\pm 0.25$	$0.8969\pm 0.015$

Appendix

Table 6: Relativistic particle. The three inference legs under both residual metrics, on one dataset and one prior, with

\sigma_{r}

held at its true value. Posterior mean

\pm

standard deviation over 14000 draws (state space) and 24000 draws (B-PINN).

\dagger

: a walk at the Euler map’s stability boundary, not a posterior; its median is quoted;

\ddagger

: chains that have not merged (

\hat{R}>1.05

), likewise a median rather than a posterior.

Sampled posteriors, ten datasets
	$\log(c/m)$			$\log(k/m)$
Model (latent)	displ.	sd	$90\%$ cov.	displ.	sd	$90\%$ cov.
Chain (state space)	$+0.02\pm 0.05$	$0.17$	$10/10$	$+0.02\pm 0.02$	$0.11$	$10/10$
Collider B-PINN (MLP)	$\mathbf{-0.42\pm 0.03}$	$0.24$	$\mathbf{4/10}$	$\mathbf{-0.12\pm 0.03}$	$0.11$	$\mathbf{7/10}$
Exact posteriors on a linear surrogate, ten datasets
Chain (state space)	$-0.03\pm 0.05$	$0.17$	$8/10$	$+0.02\pm 0.04$	$0.11$	$9/10$

Appendix

Table 7: Replication of the oscillator configuration over datasets (acceleration residuals). Top: the neural B-PINN and our chain model, sampled as in the main text, on ten datasets. Bottom: exactly computed posteriors on a linear surrogate, on ten further datasets. displ. : mean over datasets of the posterior mean of the log parameter minus the log truth,

\pm

its standard error. sd : mean posterior standard deviation.

90\%

cov. : number of datasets whose central

90\%

credible interval contains the truth.

$\sigma_{r}$		Chain (state space)		Chain (linear surrogate)		Collider (linear surrogate)
data	model	displ.	$90\%$ cov.	displ.	$90\%$ cov.	displ.	$90\%$ cov.
$0.05$	$0.05$	$-0.01\pm 0.02$	$9/10$	$-0.03\pm 0.02$	$9/10$	$\mathbf{-0.07\pm 0.02}$	$\mathbf{7/10}$
$0.1$	$0.1$	$-0.01\pm 0.03$	$9/10$	$-0.02\pm 0.03$	$9/10$	$\mathbf{-0.12\pm 0.03}$	$\mathbf{7/10}$
$0.3$	$0.3$	$-0.03\pm 0.05$	$8/10$	$-0.05\pm 0.05$	$9/10$	$\mathbf{-0.46\pm 0.04}$	$\mathbf{1/10}$
$1.0$	$1.0$	$-0.04\pm 0.07$	$8/10$	$-0.16\pm 0.07$	$9/10$	$\mathbf{-0.74\pm 0.03}$	$\mathbf{0/10}$
$0$	$0.3$	$\mathbf{+0.40\pm 0.01}$	$\mathbf{0/10}$	$\mathbf{+0.37\pm 0.01}$	$\mathbf{1/10}$	$-0.15\pm 0.01$	$10/10$

Appendix

Table 8: Sensitivity to the residual scale (oscillator, acceleration residuals, exact posteriors, ten datasets). Displacement of

\log(c/m)

from the truth (mean over datasets

\pm

standard error) and number of datasets whose central

90\%

credible interval contains the truth. Top:

\sigma_{r}

matched to the data. Bottom:

\sigma_{r}

misspecified.

Configuration	Active terms	Direction	Collider	Chain (MLP)	Removed
force residuals	metric, dissipation	$\log m$	$-1.453$	$+0.497$	$65.8\%$
acceleration residuals	dissipation	$\log(c/m)$	$-0.526$	$-0.013$	$97.5\%$

Appendix

Table 9: Restoring the normalizer on the neural surrogate (oscillator). Displacements from the truth in log space (posterior mean of the log parameter minus the log truth) for the Collider B-PINN and for Chain (Fourier MLP), and the fraction of the displacement removed by the correction,

1-|\text{chain}|/|\text{collider}|

. Active terms are the terms of Eq. ( 39 ) that depend on the inferred parameters in that configuration. The last configuration is the

\mathcal{T}=24

run of the horizon study in Fig. 3 .

Figure 4: Exact closed form of the residual metric bias against measurements on the oscillator (Eq. ( 22 )). Closed form in grey, the collider on the Markovian state-space latent in dark red, the chain in blue. On the state-space latent,

\Delta(\lambda)

along the drift-invariant direction matches

-T\log\alpha

to within

2.8\cdot 10^{-12}

nats for each number of time steps shown (

T=312

1{,}248

2{,}496

;

T=624

, omitted for clarity, agrees equally), whereas the chain’s profile stays flat.

System	Method	Runs	Location	$\hat{R}$	ESS	Width	Cost
Oscillator	PF + PMMH (ours)	$1$	$1.015\pm 0.140$	$1.00$	$604$	$1.00$	$\mathbf{0.81}$
force residuals	NSVI	$8$	$0.78$ – $1.43$	—	—	$0.15$ – $0.28$	—
	NPSGLD	$10$	$1.00$ – $1.12$ $\dagger$	$1.3$ – $3.0$	$3$ – $32$	$0.03$ – $1.25$	$56$ – $2385$
Oscillator	PF + PMMH (ours)	$1$	$1.035\pm 0.220$	$1.00$	$875$	$1.00$	$\mathbf{0.51}$
accel. residuals	NSVI	$2$	$1.28$	—	—	$1.36$ – $1.38$	—
	NPSGLD	$3$	$1.00$ – $1.12$ $\dagger$	$1.1$ – $1.5$	$6$ – $86$	$1.09$ – $1.90$	$21$ – $190$

Appendix

Table 10: Nested inference compared with our particle filter with PMMH. Ranges are over the hyperparameter sweep of each method. Location is the drift-invariant scale (

m

for the oscillator,

m_{1}{+}m_{2}

for the pendulum), reported as posterior mean

\pm

standard deviation where

\hat{R}\leq 1.05

and as a daggered median otherwise. Width is the posterior standard deviation divided by that of our method;

1.00

is correct, and

0.03

means thirty-fold overconfidence. Cost is in seconds per effective sample. A run counts as diverged if its location is off by more than a factor of

100

or its spread exceeds

100

reference standard deviations.

Explore similar work

Functional-prior-based approaches to Bayesian PDE-constrained inversion using physics-informed neural networks
May 8, 2026Ryoichiro Agata, Tomohisa OkazakiBayesian Inverse Problems Physics-Informed ML
Japan Agency for Marine-Earth Science and Technology, Japan · Disaster Prevention Research Institute, Kyoto University, Japan · RIKEN Center for Advanced Intelligence Project, Japan
Silent Failures in Physics-Informed Neural Networks: Parameter Poisoning and the Limits of Loss-Based Validation
Jun 23, 2026David McShannon, Nicholas DietrichModel Misspecification Physics-Informed ML
Learning Physics from an Imperfect Ancestor
Sep 21, 2026S. Mohammad Mousavi, Teeratorn Kadeethum, Nikolaos Bouklas +1PDE Surrogate Modeling OOD Generalization
Sibley School of Mechanical and Aerospace Engineering, Cornell University · AI Lab, Siemens Energy · Department of Civil and Systems Engineering, Johns Hopkins University

May 8, 2026, physics.geo-phJ/K move · Enter open · S save

Functional-prior-based approaches to Bayesian PDE-constrained inversion using physics-informed neural networks

Ryoichiro Agata, Tomohisa Okazaki

Japan Agency for Marine-Earth Science and Technology, Japan · Disaster Prevention Research Institute, Kyoto University, Japan · RIKEN Center for Advanced Intelligence Project, Japan

Bayesian Inverse Problems Physics-Informed ML Variational Inference PDE Inverse Problems

Open paper AskPDF arXiv

Physics-informed neural networks (PINNs) provide a mesh-free framework for solving PDE-constrained inverse problems, but their extension to Bayesian inversion still faces a fundamental difficulty: prior distributions are typically defined in the weight space of neural networks, whereas physically meaningful prior assumptions are more naturally expressed in function space. In this study, we introduce a unified framework, termed functional-prior-based approaches to Bayesian PDE-constrained inversion using physics-informed neural networks (fpBPINN), to incorporate functional priors into Bayesian PINN-based inversion. We consider two complementary approaches. The first is a functional-prior-informed Bayesian PINN (FPI-BPINN), in which a neural network weight prior is learned to be consistent with a prescribed functional prior, and Bayesian inference is subsequently performed in weight space. The second is function-space particle-based variational inference for PINNs (fParVI-PINN), which performs Bayesian estimation using ParVI directly in function space. We also show that random Fourier features (RFF) play an important role in representing Gaussian functional priors with neural networks and in improving posterior approximation. We applied the proposed approaches to one-dimensional seismic traveltime tomography and two-dimensional Darcy-flow permeability inversion. These numerical experiments showed that both approaches accurately estimated posterior distributions, highlighting the significance of introducing physically interpretable functional priors into Bayesian PINN-based inverse problems. We also identified the contrasting advantages of FPI-BPINN and fParVI-PINN, namely flexibility and accuracy, respectively.

Abstract

Explore similar work

Functional-prior-based approaches to Bayesian PDE-constrained inversion using physics-informed neural networks

Silent Failures in Physics-Informed Neural Networks: Parameter Poisoning and the Limits of Loss-Based Validation

Learning Physics from an Imperfect Ancestor