Robust Experimental Design via Generalised Bayesian Inference

Yasir Zubayr Barlas; Sabina J. Sloman; Samuel Kaski

stat.MLNov 10, 2025

Robust Experimental Design via Generalised Bayesian Inference

Authors: Yasir Zubayr Barlas, Sabina J. Sloman, Samuel Kaski

Organizations: The University of Manchester · Aalto University

Abstract

Bayesian optimal experimental design is a principled framework for conducting experiments that leverages Bayesian inference to quantify how much information one can expect to gain from selecting a certain design. However, accurate Bayesian inference relies on the assumption that one's statistical model of the data-generating process is correctly specified. If this assumption is violated, Bayesian methods can lead to poor inference and estimates of information gain. Generalised Bayesian (or Gibbs) inference is a more robust probabilistic inference framework that replaces the likelihood in the Bayesian update by a suitable loss function. In this work, we present Generalised Bayesian Optimal Experimental Design (GBOED), an extension of generalised Bayesian inference to the experimental design setting which achieves robustness in both design and inference. Using a generalisation of Bayesian decision theory, we derive a new acquisition function, the Gibbs expected information gain. Our empirical results demonstrate that GBOED can enhance robustness to outliers and incorrect assumptions about the outcome noise distribution.

Figures & tables

Figure 1: Designs selected by BOED ( top row ) and GBOED ( bottom row ) in a 2D location finding example. Designs that cluster around the objects ( red crosses ) are most informative in determining the objects’ locations. Left column : Well-specified scenario. BOED and GBOED select designs that correctly cluster around and identify the objects. Right column : Misspecified scenario. BOED is unable to explore well, failing to locate both objects. GBOED explores better and clusters around both objects, correctly locating them.

Figure 2: Performance across four experimental design problems (columns) and three model specification scenarios (rows); mean MMD/RMSE per experiment conducted and standard error shaded. Smaller

y

-axis values are better. Methods compared: BOED ( blue ), GBOED with different loss functions ( red / pink ), BEIG + GBI ( green ), Random + GBI ( gold ), and R-IDeA ( cyan ). For GBI methods, markers distinguish the loss function used. GBOED with our proposed weighted score matching parametrisation (GBOED-Exp-Decay; red ) often achieves low MMD where applicable.

Appendix figures & tables25 assets

Supplementary material from the paper’s appendix.

Appendix

Well-Specified
Rate	RMSE	MMD	NLL
Importance Weight	1.6024 (0.0402)	0.0768 (0.0079)	1.6288 (0.0371)
No Importance Weight	$1.6437\,(0.0428)$	$0.0965\,(0.0099)$	$1.6954\,(0.0434)$
Asymmetric Outliers
Importance Weight	2.4716 (0.1016)	0.4803 (0.0314)	3.4828 (0.2064)
No Importance Weight	$2.4991\,(0.0998)$	$0.4886\,(0.0312)$	$3.5836\,(0.2010)$

Appendix

Table 2: Comparison of using an importance weight as in Theorem 4.5 , or avoiding it as in Overstall et al. (2026) . RMSE, MMD, and NLL are recorded with mean (and standard error) over 3 different models, each replicated under 30 random seeds

(3\times 30=90\text{ replications in total})

. Top : Well-specified setting. Middle : Observations corrupted with Asymmetric Outliers. Bottom : True error distribution is Laplacian.

Panel of Figure 3 — Figure 3: Comparison of different values of the rate $b$ in the exponential decay method for selecting $c$ in $r_{\mathrm{IMQ}}$ under different choices of $q_{1}$ and $q_{2}$ .

Well-Specified
Rate	RMSE	MMD	NLL
$b=0.04$	1.6024 (0.0402)	0.0768 (0.0079)	1.6288 (0.0371)
$b=0.06$	$1.6086\,(0.0415)$	$0.0793\,(0.0086)$	$1.6377\,(0.0393)$
$b=0.08$	$1.6191\,(0.0428)$	$0.0837\,(0.0090)$	$1.6517\,(0.0410)$
$b=0.10$	$1.6418\,(0.0475)$	$0.0931\,(0.0112)$	$1.6891\,(0.0503)$
$b=0.12$	$1.6954\,(0.0632)$	$0.1114\,(0.0170)$	$1.8088\,(0.0997)$

Appendix

Table 6: Comparison of different rates

b

from the exponential decay method under both well-specified and misspecified scenarios in the linear regression problem, with

\omega=1

and

(q_{1},q_{2})=(9,1)

. RMSE, MMD, and NLL are recorded with mean (and standard error) over 3 different models, each replicated under 30 random seeds

(3\times 30=90\text{ replications in total})

Figure 4: Comparison between variational inference (VI) and MCMC to compute posteriors when using BOED on the (well-specified) 2D location finding problem. VI fails to reduce MMD, whereas MCMC reduces it towards zero.

Parameter	Value
$K$	2
$\alpha$	1
$b$	0.1
$m$	0.0001
$\sigma$	0.5

Appendix

Table 7: Assumed location finding parameters.

\sigma

can differ in the real-world environment depending on the misspecification present.

Method	Deployment Time (s)
BOED	$344.3563\,(0.5001)$
Power-Like $\omega=0.8$	$363.7655\,(1.4246)$
Laplante et al. (2025)	$459.8116\,(20.3834)$
Exp-Decay $b=0.30$	$431.6239\,(1.4035)$
Random + Exp-Decay	$328.0011\,(0.8702)$
BEIG + Exp-Decay	$379.6731\,(0.5902)$

Appendix

Table 8: Deployment times in seconds for the pharmacokinetics problem under the well-specified setting, with

\omega=0.8

if not stated. Mean (and standard error) over 100 replications.

Method	Deployment Time (s)
BOED	$7387.5378\,(26.0662)$
Power-Like $\omega=0.6$	$8046.8724\,(36.0040)$
Random + Power-Like	4825.8437 (33.2226)
BEIG + Power-Like	$8095.9237\,(57.6731)$

Appendix

Table 9: Deployment times in seconds for the constant elasticity of substitution problem under the Asymmetric Outliers setting, with

\omega=0.6

if not stated. Mean (and standard error) over 50 replications.

Method	Deployment Time (s)
BOED	42711.3084 (2231.0811)
Power-Like $\omega=0.4$	$53347.7212\,(3465.5066)$
Laplante et al. (2025)	$68970.6800\,(4554.4166)$
Exp-Decay $b=0.06$	$49611.9172\,(3111.6843)$
Random + Exp-Decay	$49371.2993\,(6091.7013)$
BEIG + Exp-Decay	$57400.5504\,(4024.3568)$

Appendix

Table 10: Deployment times in seconds for the 2D location finding problem under the well-specified setting, with

\omega=0.4

if not stated. Mean (and standard error) over 20 replications.

Well-Specified
Method	RMSE	MMD	NLL
BOED	$1.7383\,(0.0476)$	$0.1340\,(0.0119)$	$1.8032\,(0.0499)$
R-IDeA	$1.7470\,(0.0448)$	$0.1275\,(0.0101)$	$1.7798\,(0.0443)$
Power-Like $\omega=0.8$	$1.8665\,(0.0577)$	$0.1889\,(0.0157)$	$1.9885\,(0.0669)$
Unweighted-SM	$1.6090\,(0.0406)$	$0.0807\,(0.0083)$	$1.6423\,(0.0383)$
Laplante et al. (2025)	$6.9500\,(0.4326)$	$0.9378\,(0.0652)$	$31.9929\,(2.3881)$

Appendix

Table 11: Comparison of methods under both well-specified and misspecified scenarios in the linear regression problem, with

\omega=1

if not stated.

(q_{1},q_{2})=(9,1)

and

b=0.04

for the exponential decay method. RMSE, MMD, and NLL are recorded with mean (and standard error) over 3 different models, each replicated under 30 random seeds

(3\times 30=90\text{ replications in total})

Figure 5: Methods compared on the well-specified scenario and the two misspecified scenarios for the Bayesian linear regression problem. RMSE, MMD, and NLL from left to right. Top : Well-specified scenario. Middle : Asymmetric Outliers scenario. Bottom : Misspecified Error Distribution (Laplacian errors) scenario.

Well-Specified
Method	RMSE	MMD	NLL
BOED	$1.4027\,(0.0097)$	0.1112 (0.0055)	$1.3728\,(0.0184)$
R-IDeA	0.2441 (0.0038)	$0.1124\,(0.0098)$	-0.2322 (0.1063)
Power-Like $\omega=0.2$	$1.9750\,(0.0277)$	$0.2636\,(0.0096)$	$2.0029\,(0.0487)$
Laplante et al. (2025)	$2.5063\,(0.0141)$	$0.3378\,(0.0053)$	$2.0432\,(0.0211)$
Exp-Decay $b=0.30$	$2.6353\,(0.0153)$	$0.3475\,(0.0061)$	$2.0093\,(0.0207)$

Appendix

Table 12: Comparison of methods under both well-specified and misspecified scenarios in the pharmacokinetics problem, with

\omega=0.2

if not stated.

(q_{1},q_{2})=(0.8,0.2)

and

b=0.30

for the exponential decay method. RMSE, MMD, and NLL are reported with mean (and standard error) over 100 replications.

Figure 6: Methods compared on the well-specified scenario and the two misspecified scenarios for the pharmacokinetics problem. NLL for R-IDeA is too large to plot for reasonable comparison of the methods in the misspecified scenarios, and so is not visible in the plots. RMSE, MMD, and NLL from left to right. Top : Well-specified scenario. Middle : Asymmetric Outliers scenario. Bottom : Misspecified Error Distribution scenario.

Well-Specified
Method	RMSE	NLL
BOED	$0.1440\,(0.0125)$	$1.1130\,(0.4905)$
Power-Like $\omega=0.6$	$0.1389\,(0.0120)$	$0.7708\,(0.5125)$
Random + Power-Like	0.1307 (0.0107)	-0.0398 (0.3373)
BEIG + Power-Like	$0.1691\,(0.0153)$	$3.0154\,(1.0611)$
Asymmetric Outliers

Appendix

Table 13: Comparison of methods under both well-specified and misspecified scenarios in the constant elasticity of substitution problem, with

\omega=0.6

if not stated. RMSE and NLL are recorded with mean (and standard error) over 50 replications.

Figure 7: Methods compared on the well-specified scenario and the two misspecified scenarios for the constant elasticity of substitution problem. RMSE and NLL from left to right. Top : Well-specified scenario. Middle : Asymmetric Outliers scenario. Bottom : Misspecified Error Distribution scenario.

Figure 8: Methods compared on the well-specified scenario and the two misspecified scenarios for the 2D location finding problem. RMSE, MMD, and NLL from left to right. Top : Well-specified scenario. Middle : Asymmetric Outliers scenario. Bottom : Misspecified Error Distribution scenario.

Figure 9: Performance across three experimental design problems (columns) and three model specification scenarios (rows); mean MMD/RMSE per experiment conducted and standard error shaded. Smaller

y

Well-Specified
Method	RMSE	MMD	NLL
BOED	1.7383 (0.0476)	$0.1340\,(0.0119)$	$1.8032\,(0.0499)$
R-IDeA	$1.7470\,(0.0448)$	0.1275 (0.0101)	1.7798 (0.0443)
Power-Like $\omega=0.4$	$2.4107\,(0.0934)$	$0.4035\,(0.0279)$	$2.8378\,(0.1307)$
Laplante et al. (2025)	$11.4690\,(0.4346)$	$1.2685\,(0.0160)$	$52.8847\,(3.4198)$
Exp-Decay $b=0.04$	$6.7598\,(0.4049)$	$0.9062\,(0.0533)$	$22.8541\,(1.7444)$

Appendix

Table 16: Comparison of methods under both well-specified and misspecified scenarios in the linear regression problem, with

\omega=0.4

if not stated.

(q_{1},q_{2})=(9,1)

and

b=0.04

for the exponential decay method. RMSE, MMD, and NLL are recorded with mean (and standard error) over 3 different models, each replicated under 30 random seeds

(3\times 30=90\text{ replications in total})

Well-Specified
Method	RMSE	MMD	NLL
BOED	1.7383 (0.0476)	$0.1340\,(0.0119)$	$1.8032\,(0.0499)$
R-IDeA	$1.7470\,(0.0448)$	0.1275 (0.0101)	1.7798 (0.0443)
Power-Like $\omega=0.8$	$1.8665\,(0.0577)$	$0.1889\,(0.0157)$	$1.9885\,(0.0669)$
Laplante et al. (2025)	$7.4977\,(0.4345)$	$1.0056\,(0.0570)$	$34.7885\,(2.4839)$
Exp-Decay $b=0.04$	$1.9185\,(0.0807)$	$0.2006\,(0.0216)$	$2.1103\,(0.1275)$

Appendix

Table 17: Comparison of methods under both well-specified and misspecified scenarios in the linear regression problem, with

\omega=0.8

if not stated.

(q_{1},q_{2})=(9,1)

and

b=0.04

for the exponential decay method. RMSE, MMD, and NLL are recorded with mean (and standard error) over 3 different models, each replicated under 30 random seeds

(3\times 30=90\text{ replications in total})

Asymmetric Outliers
Method	RMSE	MMD	NLL
BOED	$1.9233\,(0.0480)$	$0.5155\,(0.0346)$	$4.3711\,(0.4065)$
R-IDeA	1.5297 (0.0624)	$1.3524\,(0.0298)$	$114.3451\,(11.5569)$
Power-Like $\omega=0.6$	$1.9235\,(0.0480)$	0.4999 (0.0336)	3.8985 (0.3503)
Laplante et al. (2025)	$6.1839\,(0.6953)$	$0.7486\,(0.0576)$	$10.8593\,(1.1219)$
Exp-Decay $b=0.30$	$4.6345\,(0.4344)$	$0.7935\,(0.0537)$	$9.8364\,(0.8213)$

Appendix

Table 18: Comparison of methods under misspecified scenarios in the pharmacokinetics problem, with

\omega=0.6

if not stated.

(q_{1},q_{2})=(0.8,0.2)

and

b=0.30

for the exponential decay method. RMSE, MMD, and NLL are reported with mean (and standard error) over 100 replications.

Well-Specified
Method	RMSE	MMD	NLL
BOED	$1.4027\,(0.0097)$	$0.1112\,(0.0055)$	$1.3728\,(0.0184)$
R-IDeA	0.2441 (0.0038)	$0.1124\,(0.0098)$	-0.2322 (0.1063)
Power-Like $\omega=0.8$	$1.3777\,(0.0085)$	0.0963 (0.0053)	$1.3128\,(0.0163)$
Laplante et al. (2025)	$1.5445\,(0.0224)$	$0.2203\,(0.0138)$	$1.8882\,(0.0750)$
Exp-Decay $b=0.30$	$1.4560\,(0.0188)$	$0.1668\,(0.0109)$	$1.6234\,(0.0566)$

Appendix

Table 19: Comparison of methods under both well-specified and misspecified scenarios in the pharmacokinetics problem, with

\omega=0.8

if not stated.

(q_{1},q_{2})=(0.8,0.2)

and

b=0.30

for the exponential decay method. RMSE, MMD, and NLL are reported with mean (and standard error) over 100 replications.

Well-Specified
Method	RMSE	NLL
BOED	$0.1440\,(0.0125)$	$1.1130\,(0.4905)$
Power-Like $\omega=0.2$	0.1245 (0.0107)	$-0.2648\,(0.3617)$
Random + Power-Like $\omega=0.2$	$0.1322\,(0.0100)$	$-0.3875\,(0.3657)$
BEIG + Power-Like $\omega=0.2$	$0.1302\,(0.0127)$	$0.2129\,(0.4470)$
Power-Like $\omega=0.8$	$0.1317\,(0.0117)$	$0.5707\,(0.5557)$

Appendix

Table 20: Comparison of methods under both well-specified and misspecified scenarios in the constant elasticity of substitution problem. Power likelihoods are compared with learning rates

\omega=0.2

and

\omega=0.8

. RMSE and NLL are recorded with mean (and standard error) over 50 replications.

Figure 10: Comparison of different learning rates for computing the Gibbs EIG using the negative log-likelihood loss, in the context of the Bayesian linear regression problem. Marked crosses are the 10 designs with the greatest Gibbs EIG for a particular curve. Bottom : The Gibbs EIG at

\omega=0.1

. EIG values are lower than the other two curves, and designs slightly further away from the extremes have greater EIG. Middle : The Gibbs EIG at

\omega=0.5

. Less smooth EIG curve than the curve above, with some indication of designs with greater EIG values being somewhat away from the extremes. Top : The BEIG (

\omega=1

). Greatest EIG values are at the most extreme ends of the design space, and the EIG curve is much smoother than the other two.

Figure 11: Comparison of different downweighting rates

c

on the Gibbs EIG under different priors, for the Bayesian linear regression problem.

\omega=1

across all curves. Left : poor prior (unit Gaussian, larger variance). Right : good prior (places high density on the data-generating parameter value, smaller variance). Smaller

c

values tend to result in queries slightly away from the extremes, with more noise compared to the other curves. Large

c

values have GEIG curves close to the unweighted score matching loss.

Figure 12: Comparison of different loss functions for computing the Gibbs EIG, under the pharmacokinetics problem. Marked crosses are the 5 designs with the greatest (Gibbs) EIG for a particular curve. The Gibbs EIG can vary greatly depending on the loss function used.

Figure 13: Comparison of methods in the (maximum) Gibbs EIG acquired during experimentation for the 2D location finding problem. Mean (and standard error) over 20 replications. Left : Well-specified scenario. Right : Asymmetric Outliers scenario.

\omega=0.4

if not stated.

Generalized Bayesian Inference

See all 8 papers

Goal-driven Bayesian Optimal Experimental Design for Robust Decision-Making Under Model Uncertainty
May 25, 2026Jinwoo Go, Xiaoning Qian, Byung-Jun YoonBayesian Experimental Design Decision-Making under Uncertainty
Computing & Data Sciences, Brookhaven National Laboratory, Upton, NY · Department of Electrical & Computer Engineering, Texas A&M University, College Station, TX · Department of Computer Science & Engineering, Texas A&M University, College Station, TX
Maximin Robust Bayesian Experimental Design
Mar 14, 2026Hany Abdulsamad, Sahel Iqbal, Christian A. Naesseth +2Bayesian Experimental Design Mutual Information
UvA–Bosch Delta Lab University of Amsterdam · Department of Statistics University of Oxford · School of Mathematics University of Edinburgh +1
Beyond Expected Information Gain: Stable Bayesian Optimal Experimental Design with Integral Probability Metrics and Plug-and-Play Extensions
Apr 23, 2026Di Wu, Ling Liang, Haizhao YangBayesian Experimental Design Integral Probability Metrics
Department of Mathematics The University of Tennessee, Knoxville TN 37916, USA · Department of Mathematics and Department of Computer Science University of Maryland, College Park MD 20742, USA

May 25, 2026J/K move · Enter open · S save

Goal-driven Bayesian Optimal Experimental Design for Robust Decision-Making Under Model Uncertainty

Jinwoo Go, Xiaoning Qian, Byung-Jun Yoon

Computing & Data Sciences, Brookhaven National Laboratory, Upton, NY · Department of Electrical & Computer Engineering, Texas A&M University, College Station, TX · Department of Computer Science & Engineering, Texas A&M University, College Station, TX

Bayesian Experimental Design Decision-Making under Uncertainty

Open paper AskPDF arXiv

Bayesian optimal experimental design (BOED) selects experiments to maximize information gain about model parameters. However, in decision-critical settings, reducing parameter uncertainty does not necessarily improve downstream decisions, as only specific parameter directions relevant to the objective truly matter. We propose GoBOED, a goal-driven BOED framework that directly optimizes experimental designs for a specified decision-making objective. GoBOED combines an amortized variational posterior surrogate with a differentiable convex decision layer, enabling gradient-based design optimization that is fully decision-focused. We theoretically show that GoBOED gradients are insensitive to parameter directions irrelevant to the decision objective, providing a formal justification for why goal-driven design achieves equivalent decision quality over a wider set of experimental designs than information-gain maximization. Empirically, across source localization, epidemic management, and pharmacokinetic control, GoBOED identifies designs that better align with downstream decision objectives and reveals that near-optimal design windows are substantially wider than those predicted by goal-agnostic BOED approaches.

Abstract

Generalized Bayesian Inference

Explore similar work

Goal-driven Bayesian Optimal Experimental Design for Robust Decision-Making Under Model Uncertainty

Maximin Robust Bayesian Experimental Design

Beyond Expected Information Gain: Stable Bayesian Optimal Experimental Design with Integral Probability Metrics and Plug-and-Play Extensions