Maximin Robust Bayesian Experimental Design

Hany Abdulsamad; Sahel Iqbal; Christian A. Naesseth; Takuo Matsubara; Adrien Corenflos

stat.MLMar 14, 2026

Maximin Robust Bayesian Experimental Design

Authors: Hany Abdulsamad, Sahel Iqbal, Christian A. Naesseth, Takuo Matsubara, Adrien Corenflos

Organizations: UvA–Bosch Delta Lab University of Amsterdam · Department of Statistics University of Oxford · School of Mathematics University of Edinburgh · Department of Statistics University of Warwick

Abstract

We address the brittleness of Bayesian experimental design under model misspecification by formulating the problem as a max--min game between the experimenter and an adversarial nature subject to information-theoretic constraints. We demonstrate that this approach yields a robust objective governed by Sibson's $α$ -mutual information (MI), which identifies the $α$ -tilted posterior as the robust belief update and establishes the Rényi divergence as the appropriate measure of conditional information gain. To mitigate the bias and variance of nested Monte Carlo estimators needed to estimate Sibson's $α$ -MI, we adopt a PAC-Bayes framework to search over stochastic design policies, yielding rigorous high-probability lower bounds on the robust expected information gain that explicitly control finite-sample error.

Figures & tables

Panel of Figure 1 — Figure 1: Comparison of nominal and robust formulations for linear regression and A/B testing. Left: histograms of realized information gains under nominal ( $\alpha=1.0$ ) and robust ( $\alpha=0.056)$ assumptions. Right: expected vs. actual coverage for varying values of $\alpha$ . Nominal posteriors are clearly overconfident; calibration improves for values $\alpha<1$ . Data obtained from $10^{4}$ experiments.

	Linear Regression		A/B Testing
$\alpha$	Random	Optimal	Random	Optimal
0.056	$4.62\times 10^{-1}$	$\mathbf{2.47\times 10^{-1}}$	$1.01\times 10^{-1}$	$\mathbf{7.49\times 10^{-2}}$
0.115	$4.78\times 10^{-1}$	$\mathbf{2.50\times 10^{-1}}$	$9.03\times 10^{-2}$	$\mathbf{7.13\times 10^{-2}}$
0.237	$5.22\times 10^{-1}$	$\mathbf{2.53\times 10^{-1}}$	$8.98\times 10^{-2}$	$\mathbf{7.31\times 10^{-2}}$
0.487	$5.91\times 10^{-1}$	$\mathbf{2.56\times 10^{-1}}$	$9.77\times 10^{-2}$	$\mathbf{8.88\times 10^{-2}}$
1.0	$6.82\times 10^{-1}$	$\mathbf{2.57\times 10^{-1}}$	$1.09\times 10^{-1}$	$\mathbf{1.06\times 10^{-1}}$

Table 1: Posterior-mean RMSE for linear regression and A/B testing. Comparing the robust posterior under random and optimal designs for varying

\alpha

. Robust designs that optimize Sibson’s MI lead to smaller mean squared errors. Data obtained over

10^{4}

experiments.

Table 3

Panel of Figure 2 — Figure 2: Empirical distributions of regret (left) and design optimality (right) for linear regression (top) and A/B testing (bottom). Comparing naive optimization against our PAC-Bayes policy. Naive optimization has higher regret and variability. Its designs are suboptimal, reflected in lower similarity scores relative to the theoretical optimal design. Data from $1024$ trials.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

$N$	Mean	$P_{10}$	$P_{90}$	Mean	$P_{10}$	$P_{90}$	Mean	$P_{10}$	$P_{90}$	Mean	$P_{10}$	$P_{90}$	Mean	$P_{10}$	$P_{90}$	Mean	$P_{10}$	$P_{90}$	Mean	$P_{10}$	$P_{90}$
	$\lambda=10^{2}$			$\lambda=10^{3}$			$\lambda=10^{4}$			$\lambda=10^{5}$			$\lambda=10^{6}$			$\lambda=10^{7}$			$\lambda=10^{8}$
64	0.062	0.038	0.090	0.020	0.012	0.029	0.008	0.007	0.010	$\mathbf{0.011}$	$\mathbf{0.008}$	$\mathbf{0.014}$	$\mathbf{0.026}$	$\mathbf{0.010}$	$\mathbf{0.043}$	$\mathbf{0.023}$	$\mathbf{0.019}$	$\mathbf{0.028}$	0.019	0.010	0.029
128	0.081	0.044	0.123	0.037	0.016	0.062	0.037	0.025	0.052	$\mathbf{0.024}$	$\mathbf{0.012}$	$\mathbf{0.037}$	$\mathbf{0.006}$	$\mathbf{0.005}$	$\mathbf{0.007}$	$\mathbf{0.013}$	$\mathbf{0.009}$	$\mathbf{0.018}$	0.023	0.009	0.041
256	0.094	0.050	0.145	0.019	0.013	0.028	0.020	0.012	0.028	$\mathbf{0.021}$	$\mathbf{0.012}$	$\mathbf{0.031}$	$\mathbf{0.019}$	$\mathbf{0.010}$	$\mathbf{0.030}$	$\mathbf{0.022}$	$\mathbf{0.010}$	$\mathbf{0.035}$	0.018	0.008	0.033

Appendix

Table 4: Sensitivity of PAC-Bayes policies to the choice of

\lambda

on the linear regression problem. We sweep over

\lambda

and

N

and report the mean relative regret with

10

th and

90

th percentiles.

Panel of Figure 4 — Figure 4: Robust expected information gain as a function of $\alpha\in(0,1)$ for an A/B testing problem with 25 participants. The optimal allocation, highlighted in dark gray, shifts as $\alpha$ varies.

$\alpha$	Sibson	GEIG	Mean	$P_{10}$	$P_{90}$	Mean	$P_{10}$	$P_{90}$
	Design		Sibson RMSE			GEIG RMSE
0.011	0	30	$\mathbf{0.120}$	0.034	0.208	0.132	0.036	0.231
0.022	0	33	$\mathbf{0.098}$	0.030	0.167	0.112	0.033	0.193
0.041	0	35	$\mathbf{0.081}$	0.025	0.143	0.093	0.028	0.158
0.077	0	37	$\mathbf{0.072}$	0.022	0.141	0.079	0.025	0.146
0.147	0	39	$\mathbf{0.071}$	0.020	0.144	0.076	0.024	0.145

Appendix

Table 5: Comparison of Sibson’s and GEIG’s optimal designs and RMSE on an A/B testing problem with

N_{x}=100

. The two objectives lead to different design choices. We report the optimal allocation, mean RMSE, the

10

th and

90

th percentiles for each method. Evaluation over

10^{4}

trials.

Bayesian Experimental Design

Bayesian experimental design uses probabilistic models to choose measurements that improve learning or support better decisions.

See all 63 papers

Robust Experimental Design via Generalised Bayesian Inference
Nov 10, 2025Yasir Zubayr Barlas, Sabina J. Sloman, Samuel KaskiGeneralized Bayesian Inference Bayesian Experimental Design
The University of Manchester · Aalto University
Robust Bayesian Decision Making under Adversarial Uncertainty
Jul 9, 2026Haripriya Harikumar, Sammie Katt, Yasir Zubayr Barlas +1Adversarial Robustness Bayesian Decision Theory
Department of Computer Science, The University of Manchester, UK · ELLIS Institute Finland · Department of Computer Science, Aalto University, Espoo, Finland
Beyond Expected Information Gain: Stable Bayesian Optimal Experimental Design with Integral Probability Metrics and Plug-and-Play Extensions
Apr 23, 2026Di Wu, Ling Liang, Haizhao YangBayesian Experimental Design Integral Probability Metrics
Department of Mathematics The University of Tennessee, Knoxville TN 37916, USA · Department of Mathematics and Department of Computer Science University of Maryland, College Park MD 20742, USA

Nov 10, 2025J/K move · Enter open · S save

Robust Experimental Design via Generalised Bayesian Inference

Yasir Zubayr Barlas, Sabina J. Sloman, Samuel Kaski

The University of Manchester · Aalto University

Generalized Bayesian Inference Bayesian Experimental Design Bayesian Inference

Open paper AskPDF arXiv

Bayesian optimal experimental design is a principled framework for conducting experiments that leverages Bayesian inference to quantify how much information one can expect to gain from selecting a certain design. However, accurate Bayesian inference relies on the assumption that one's statistical model of the data-generating process is correctly specified. If this assumption is violated, Bayesian methods can lead to poor inference and estimates of information gain. Generalised Bayesian (or Gibbs) inference is a more robust probabilistic inference framework that replaces the likelihood in the Bayesian update by a suitable loss function. In this work, we present Generalised Bayesian Optimal Experimental Design (GBOED), an extension of generalised Bayesian inference to the experimental design setting which achieves robustness in both design and inference. Using a generalisation of Bayesian decision theory, we derive a new acquisition function, the Gibbs expected information gain. Our empirical results demonstrate that GBOED can enhance robustness to outliers and incorrect assumptions about the outcome noise distribution.

Abstract

Bayesian Experimental Design

Explore similar work

Robust Experimental Design via Generalised Bayesian Inference

Robust Bayesian Decision Making under Adversarial Uncertainty

Beyond Expected Information Gain: Stable Bayesian Optimal Experimental Design with Integral Probability Metrics and Plug-and-Play Extensions