Bayesian Experimental Design

Ask about this topic

Overview

Bayesian experimental design studies how to choose experiments or measurements sequentially when observations are costly and uncertainty matters. Common approaches select designs by expected information gain or posterior uncertainty, often using Gaussian processes, Monte Carlo estimates, or amortized neural policies; Bayesian optimization and active learning extend these ideas to expensive black-box problems. Applications range from scientific parameter estimation and sensor placement to clinical questioning, reliability testing, and adaptive data collection. A central tension is that the most informative experiment may not improve a downstream decision, while model error, constraints, and computational cost can make standard information-gain criteria unreliable or impractical.

Latest papers

CardsList

63 papers10 papers in the last four weeks, against 2 the four weeks before. 0.1% of all new papers.

Jun 8, 2026

Evidence Acquisition and Stopping via Lattice Distillation

Sequential evidence acquisition requires deciding what to observe and when to stop from partial information. We introduce a complete-record view of policy supervision: each fully observed training instance defines a lattice of evidence subsets, with a stop/query decision at every node. For a modest number of evidence groups, this structure can be enumerated even when individual groups contain rich representations. Our method, Subset Lattice Distillation (SLD), scores subsets with a shared masked predictor, constructs multi-step decision targets by cost-sensitive backward planning, and distills them into an observed-history controller. Uniformly resampled masks train the student across the subset space, while combinatorial planning remains offline. Separating subset scores from acquisition prices allows the predictor to be reused when the policy is replanned for new costs. Our analysis identifies action-margin conditions under which privileged supervision preserves optimal online decisions. On retrospective influenza monitoring, SLD achieves higher phase F1 at lower acquisition cost than both the closest-cost adaptive baselines and the fixed full-evidence model. A geographic and seasonal holdout, price perturbations, and ablation studies provide further evaluation.

Yixuan Zhang, Yang Song, Jiayi Zhang +4

Section of Health Data Science and AI, Department of Public Health, University of Copenhagen, Copenhagen, Denmark · Section of Health Data Science & AI, Department of Public Health, University of Copenhagen, Copenhagen, Denmark · Rutgers University, New Brunswick, NJ, USA +1

Epidemic ForecastingSelective PredictionBayesian InferenceCost-Aware InferenceBayesian Experimental Design
AskPDF
Jun 1, 2026

ShaplEIG: Bayesian Experimental Design for Shapley Value Estimation

Shapley values are a principled attribution measure widely used in interpretable machine learning, but their exact computation scales exponentially with the number of players, motivating a wide range of approximation methods based on value function evaluations of sampled coalitions. This raises the question of whether approximation accuracy can be improved by adaptively selecting coalitions for evaluation based on previous evaluations. This is particularly relevant in settings where the value function is costly and the number of evaluations is severely limited, such as retraining-based feature importance, data valuation, and hyperparameter importance. For this purpose, we propose ShaplEIG, a Bayesian experimental design approach that approximates the expensive value function using a Gaussian process surrogate and adaptively selects coalitions based on their expected information gain about the Shapley values. By the linearity of the Shapley values in the value function, we show that the expected information gain is available in closed form. Furthermore, we propose an efficient computation scheme that reduces the complexity from exponential to polynomial in the number of players via elementary symmetric polynomials. In extensive experiments across diverse costly applications, our method consistently improves sample efficiency in the low-budget regime over state-of-the-art baselines.

David Rundel, Fabian Fumagalli, Maximilian Muschalik +2

Department of Statistics, LMU Munich, Munich, Germany · Munich Center for Machine Learning, Munich, Germany · Institute for Informatics, LMU Munich, Munich, Germany +2

Shapley Value AttributionAdaptive SamplingBayesian Experimental Design
AskPDF
May 31, 2026

CA-BED: Conversation-Aware Bayesian Experimental Design

Large Language Models (LLMs) excel at static reasoning tasks, yet their performance often degrades in interactive scenarios where information must be actively acquired through questioning. A key challenge lies in selecting questions that reduce uncertainty while incorporating responses that may be ambiguous or only partially informative. To address this, we propose Conversation-Aware Bayesian Experimental Design (CA-BED), an inference-time probabilistic dialog planning framework that integrates Bayesian Experimental Design with LLM-based likelihood estimation to optimize question selection over multiple conversational turns. CA-BED maintains a belief distribution over hypotheses, anticipates possible answers, and propagates expected information gain through a simulated conversation tree. Across two structured entity-deduction benchmarks, CA-BED yields an average 21.8% improvement in success rates over direct prompting, with comparable gains relative to alternative information-seeking methods. It achieves these gains with an average increase of only 1.8 conversational turns compared to direct prompting.

Daniel Arnould, Rashad Aziz, Zixuan Kang +5

Active Information GatheringBayesian Experimental DesignConversational Agents
AskPDF
May 26, 2026

When Should an AI Scientist Stop? Verifiable Experiment Steering and Refusal for Autonomous Discovery

We present CARTOGRAPH, a verification layer for AI scientists that couples unresolved-subspace experiment steering (select), explicit ambiguity closure (resolve), and residual-based library inadequacy detection (refuse). Under a local linear-Gaussian bridge, raw unresolved projection is the isotropic unresolved Fisher-information trace, while CARTOGRAPH-A is the exact unresolved A-optimal rule; closed-form EIG and Box-Hill arise as local comparators rather than global equivalents. Across five testbeds, CARTOGRAPH-A beats raw projection 129W/0T/15L at d = 8 (p < 10^-21) in a replicated structured cascade. More distinctively, the framework tentatively identifies three out-of-library pharmacokinetic mechanisms and then revokes those identifications as residuals expose structural misfit, while one perturbed in-library control stays identified throughout. In low-dimensional pharmacokinetic and filtered EPA settings, near-ties against disagreement are predicted by theory and observed. Finally, in a retrospective audit of 40 positive claims from the published A-Lab autonomous materials system, the refuse guard flags all 4 claims later marked inconclusive under manual reanalysis while passing 32/36 confirmed claims. Code is available at https://github.com/ai4science-boed/cartograph.git

Neel Tushar Shah, Manglam Kartik

Indian Institute of Technology Bombay, Mumbai, India.

Model MisspecificationAutonomous Scientific DiscoveryBayesian Experimental DesignAutonomous Experimentation
AskPDF
May 26, 2026

Constrained Bayesian Experimental Design via Online Planning

Bayesian experimental design (BED) is a principled framework for data-efficient design of sequential experiments. However, existing BED methods are unable to adapt to dynamic constraints inherent in real-world tasks due to budget limitations, varying costs, or physical constraints that restrict how designs evolve over time. In this paper, we introduce a novel approach to BED that enables constrained optimization of experimental designs by combining offline pre-training of an amortized policy and a posterior network with online multi-step lookahead planning using scenario trees. We empirically demonstrate that our method yields substantially more informative design sequences than existing methods across a range of constrained BED tasks, while incurring only a modest additional computational overhead.

Yujia Guo, Daolang Huang, Xinyu Zhang +3

ELLIS Institute Finland · Department of Computer Science, Aalto University, Finland · Department of Computer Science, University of Manchester, UK

Bayesian Experimental DesignSequential Experimental Design
AskPDF
May 25, 2026

Goal-driven Bayesian Optimal Experimental Design for Robust Decision-Making Under Model Uncertainty

Bayesian optimal experimental design (BOED) selects experiments to maximize information gain about model parameters. However, in decision-critical settings, reducing parameter uncertainty does not necessarily improve downstream decisions, as only specific parameter directions relevant to the objective truly matter. We propose GoBOED, a goal-driven BOED framework that directly optimizes experimental designs for a specified decision-making objective. GoBOED combines an amortized variational posterior surrogate with a differentiable convex decision layer, enabling gradient-based design optimization that is fully decision-focused. We theoretically show that GoBOED gradients are insensitive to parameter directions irrelevant to the decision objective, providing a formal justification for why goal-driven design achieves equivalent decision quality over a wider set of experimental designs than information-gain maximization. Empirically, across source localization, epidemic management, and pharmacokinetic control, GoBOED identifies designs that better align with downstream decision objectives and reveals that near-optimal design windows are substantially wider than those predicted by goal-agnostic BOED approaches.

Jinwoo Go, Xiaoning Qian, Byung-Jun Yoon

Computing & Data Sciences, Brookhaven National Laboratory, Upton, NY · Department of Electrical & Computer Engineering, Texas A&M University, College Station, TX · Department of Computer Science & Engineering, Texas A&M University, College Station, TX

Bayesian Experimental DesignDecision-Making under Uncertainty
AskPDF
May 20, 2026

Exploration and Experimentation with a Misspecified Simulator

Randomized experiments are the standard for measuring causal effects that guide decisions, but are costly. Simulators calibrated to historical data offer a cheaper alternative, but their policies can become suboptimal under drift, a change in dynamics after calibration. We study if, when, and how to collect data for recalibrating the simulator. We consider a system with multiple units running in parallel and managed by a single operator. The real-world system is a partially observed process with a latent variable, while the simulator models it as a fully observed Markovian process, and thus is misspecified. Two data collection strategies exploit the current policy: (i) never recalibrating and (ii) recalibrating from on-policy data (passive adaptation). Two others collect off-policy data by trying actions outside the current policy: (iii) continuous exploration across all units and (iv) randomized experimentation on a sample of units over a predefined window. We investigate these strategies theoretically and in a vending-machine simulation. Experiments can improve on passive adaptation when drift is invisible in on-policy data or recalibration lags behind continuing drift. Experimentation is preferable to exploration when trying a worse action is expensive and the dynamics may not have changed.

Harsh Parikh, Gabriel Levin-Konigsberg, Sohrab Andaz +3

Duke University, Durham, USA · Yale University, New Haven, USA

Sim-to-Real TransferRL ExplorationBayesian Experimental Design
AskPDF
May 12, 2026

Elicitation-Augmented Bayesian Optimization

Human-in-the-loop Bayesian optimization (HITL BO) methods utilize human expertise to improve the sample-efficiency of BO. Most HITL BO methods assume that a domain expert can quantify their knowledge, for instance by pinpointing query locations or specifying their prior beliefs about the location of the maximum as a probability distribution. However, since human expertise is often tacit and cannot be explicitly quantified, we consider a setting where domain knowledge of an expert is elicited via pairwise comparisons of designs. We interpret the expert's pairwise judgements as noisy evidence about the values of the observable objective function and develop a principled method for combining the information obtained via direct observations and pairwise queries. Specifically, we derive a cost-aware value-of-information acquisition function that balances direct observations against pairwise queries. The proposed method approaches the convex hull of the trajectories of the individual information sources: when pairwise queries are cheap it substantially improves sample-efficiency over observation-only BO, and when pairwise queries are costly or noisy, it recovers the performance of standard BO by relying on direct observations alone.

Alvar Haltia, Ville Hyvönen, Samuel Kaski

Aalto University · University of Helsinki · Aalto University, University of Manchester

Bayesian OptimizationHuman-in-the-Loop EvaluationBayesian Experimental Design
AskPDF
May 11, 2026

Many Needles in a Haystack: Active Hit Discovery for Perturbation Experiments

High-throughput gene perturbation experiments can test several genetic interventions in parallel, yet experimental budgets remain limited. A central goal is hit discovery: identifying as many perturbations as possible whose phenotypic effect exceeds a predefined threshold. Pure exploration strategies are statistically inefficient, wasting budget on low-value regions. Bayesian optimization methods offer a principled alternative but target a single global optimum, over-exploiting dominant modes while neglecting other high-value regions. We formalize hit discovery as a sequential experimental design problem and propose Probability-of-Hit, an acquisition function that directly targets threshold exceedance by ranking candidates according to their posterior probability of being a hit. We prove asymptotic optimality of this approach and demonstrate strong empirical performance on both synthetic benchmarks and real biological immunology datasets, including up to 6.4% improvement over baselines on the Schmidt IL-2 dataset.

Andrea Rubbi, Arpit Merchant, Samuel Ogden +4

Department of Computer Science and Technology, University of Cambridge, Cambridge, UK · Cambridge Center for AI in Medicine, University of Cambridge, Cambridge, UK · MediaTek Research, Cambridge, UK +1

Bayesian OptimizationBayesian Experimental Design
AskPDF
May 7, 2026

DARTS: Targeting Prognostic Covariates in Budget-Constrained Sequential Experiments

Randomized controlled trials typically assume that prognostic covariates are known and available at no cost. In practice, obtaining high-dimensional pretreatment data is costly, forcing a trade-off between covariate-adaptive precision and a measurement budget. We introduce Dynamic Adaptive Rerandomization via Thompson Sampling (DARTS), which treats covariate acquisition as a sequential optimization problem embedded within a design-based causal inference task. A budgeted combinatorial Thompson sampler learns which covariates are most prognostic across successive batches; selected covariates then drive rerandomization and regression adjustment to reduce batch-level average treatment effect variance. Our primary theoretical contribution is a decoupling result: adaptive covariate selection based on past batches preserves batch-level randomization validity, and the cumulative inverse-variance weighted estimator achieves at least nominal asymptotic coverage. We further derive a Bayes risk bound for the acquisition layer that matches the minimax lower bound up to logarithmic factors. Empirically, DARTS systematically concentrates the budget on informative features, significantly closing the efficiency gap to oracle designs while maintaining strict inferential validity.

Kateryna Husar, Alexander Volfovsky

Department of Statistical Science Duke University Durham, NC 27705

Thompson SamplingCausal Effect EstimationFeature SelectionBayesian Experimental Design
AskPDF
May 2, 2026

The Perceptual Bandwidth Bottleneck in Vision-Language Models: Active Visual Reasoning via Sequential Experimental Design

Visual perception in modern Vision-Language Models (VLMs) is constrained by a perceptual bandwidth bottleneck: a broad field of view preserves global context but sacrifices the fine-grained details required for complex reasoning. We argue that high-resolution visual reasoning is therefore not only semantic reasoning but also task-relevant evidence acquisition under limited perceptual bandwidth. Inspired by active vision and information foraging, we formalise this process as sequential Bayesian optimal experimental design (S-BOED), where an agent decides which visual evidence to acquire before answering. Since exact Bayesian inference is intractable in continuous gigapixel spaces, we derive a tractable coverage--resolution objective as a proxy for task-relevant information gain. We instantiate this framework with FOVEA, a training-free procedure that refines VLM crop proposals through evidence-oriented probing. Experiments on high-resolution benchmarks show consistent gains over direct and ReAct-style baselines, with particularly strong improvements in search-dominated remote-sensing settings.

Anjie Liu, Ziqin Gong, Yan Song +6

The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · University College London, London, United Kingdom · ShanghaiTech University, Shanghai, China +2

VLM ReasoningVisually Grounded ReasoningBayesian Experimental DesignActive Visual PerceptionFine-Grained Visual Perception
AskPDF
May 1, 2026

Adaptive Querying with AI Persona Priors

We study adaptive querying for learning user-dependent quantities of interest, such as responses to held-out items and psychometric indicators, within tight query budgets. Classical Bayesian design and computerized adaptive testing typically rely on restrictive parametric assumptions or expensive posterior approximations, limiting their use in heterogeneous, high-dimensional, and cold-start settings. We introduce a persona-induced latent variable model that represents a user's state through membership in a finite dictionary of AI personas, each offering response distributions produced by a large language model. This yields expressive priors with closed-form posterior updates and efficient finite-mixture predictions, enabling scalable Bayesian design for sequential item selection. Experiments on synthetic data and WorldValuesBench demonstrate that persona-based posteriors deliver accurate probabilistic predictions and an interpretable adaptive elicitation pipeline.

Kaizheng Wang, Yuhang Wu, Assaf Zeevi

Department of Industrial Engineering and Operations Research and Data Science Institute, Columbia University. · Decision, Risk, and Operations Division, Columbia Business School.

Latent Variable ModelsBayesian Experimental DesignUser Preference Modeling
AskPDF
Apr 26, 2026

Accelerating Quantum Materials Characterization: Hybrid Active Learning for Autonomous Spin Wave Spectroscopy

Autonomous neutron spectroscopy must solve three distinct tasks: detection (where is the signal?), inference (which Hamiltonian governs it?), and refinement (what are the parameters?). No single controller solves all three equally well. We present TAS-AI, a hybrid agnostic-to-physics-informed framework for autonomous triple-axis spin-wave spectroscopy that separates these tasks explicitly. In blind reconstruction benchmarks, model-agnostic methods such as random sampling, coarse grids, and Gaussian-process mappers reach a global error threshold more reliably and with fewer measurements than physics-informed planning, supporting the claim that discovery and inference are distinct tasks requiring distinct controllers. Once signal structure is localized, the physics-informed stage performs in-loop Hamiltonian discrimination and parameter refinement: in a controlled square-lattice test between nearest-neighbor-only and J1-J2 Hamiltonians, TAS-AI reaches a decisive AIC-derived evidence ratio (>100) in fewer than 10 measurements, while motion-aware scheduling cuts wall-clock time by 32% at a fixed measurement budget. We also identify a failure mode of posterior-weighted design, algorithmic myopia, in which the planner over-refines the current leading model while under-sampling low-intensity falsification probes. A constrained falsification channel sharply reduces time spent committed to the wrong model and accelerates correct model selection without modifying the Bayesian inference engine. In controlled two-model ablations, both a deterministic top-two max-disagreement rule and an LLM-based audit committee achieve this gain under identical constraints. We demonstrate the full workflow in silico using a high-fidelity digital twin and provide an open-source Python implementation.

William Ratcliff

NIST Center for Neutron Research, National Institute of Standards and Technology, Gaithersburg, MD 20899, USA

Active LearningMaterials ScienceBayesian Experimental DesignAutonomous Experimentation
AskPDF
Apr 24, 2026

Spend Less, Fit Better: Budget-Efficient Scaling Law Fitting via Active Experiment Selection

Scaling laws are used to plan multi-million-dollar training runs, but fitting those laws can itself cost millions. In modern large-scale workflows, assembling a sufficiently informative set of pilot experiments is already a major budget-allocation problem rather than a routine preprocessing step. We formulate scaling-law fitting as budget-aware sequential experimental design: given a finite pool of runnable experiments with heterogeneous costs, choose which runs to execute so as to maximize extrapolation accuracy in a high-cost target region. We then propose an uncertainty-aware method for sequentially allocating experimental budget toward the runs most useful for target-region extrapolation. Across a diverse benchmark of scaling-law tasks, our method consistently outperforms classical design-based baselines, and often approaches the performance of fitting on the full experimental set while using only about 10% of the total training budget. Our code is available at https://github.com/PlanarG/active-sl.

Sijie Li, Shanda Li, Haowei Lin +3

Peking University · Carnegie Mellon University · Datadog

Active Experimental DesignScaling LawsBayesian Experimental DesignSequential Experimental Design
AskPDF
Apr 23, 2026

Beyond Expected Information Gain: Stable Bayesian Optimal Experimental Design with Integral Probability Metrics and Plug-and-Play Extensions

Bayesian Optimal Experimental Design (BOED) provides a rigorous framework for decision-making tasks in which data acquisition is often the critical bottleneck, especially in resource-constrained settings. Traditionally, BOED typically selects designs by maximizing expected information gain (EIG), commonly defined through the Kullback-Leibler (KL) divergence. However, classical evaluation of EIG often involves challenging nested expectations, and even advanced variational methods leave the underlying log-density-ratio objective unchanged. As a result, support mismatch, tail underestimation, and rare-event sensitivity remain intrinsic concerns for KL-based BOED. To address these fundamental bottlenecks, we introduce an IPM-based BOED framework that replaces density-based divergences with integral probability metrics (IPMs), including the Wasserstein distance, Maximum Mean Discrepancy, and Energy Distance, resulting in a highly flexible plug-and-play BOED framework. We establish theoretical guarantees showing that IPM-based utilities provide stronger geometry-aware stability under surrogate-model error and prior misspecification than classical EIG-based utilities. We also validate the proposed framework empirically, demonstrating that IPM-based designs yield highly concentrated credible sets. Furthermore, by extending the same sample-based BOED template in a plug-and-play manner to geometry-aware discrepancies beyond the IPM class, illustrated by a neural optimal transport estimator, we achieve accurate optimal designs in high-dimensional settings where conventional nested Monte Carlo estimators and advanced variational methods fail.

Di Wu, Ling Liang, Haizhao Yang

Department of Mathematics The University of Tennessee, Knoxville TN 37916, USA · Department of Mathematics and Department of Computer Science University of Maryland, College Park MD 20742, USA

Bayesian Experimental DesignIntegral Probability Metrics
AskPDF
Apr 1, 2026

Efficient and Principled Scientific Discovery through Bayesian Optimization: A Tutorial

Traditional scientific discovery relies on an iterative hypothesise-experiment-refine cycle that has driven progress for centuries, but its intuitive, ad-hoc implementation often wastes resources, yields inefficient designs, and misses critical insights. This tutorial presents Bayesian Optimisation (BO), a principled probability-driven framework that formalises and automates this core scientific cycle. BO uses surrogate models (e.g., Gaussian processes) to model empirical observations as evolving hypotheses, and acquisition functions to guide experiment selection, balancing exploitation of known knowledge and exploration of uncharted domains to eliminate guesswork and manual trial-and-error. We first frame scientific discovery as an optimisation problem, then unpack BO's core components, end-to-end workflows, and real-world efficacy via case studies in catalysis, materials science, organic synthesis, and molecule discovery. We also cover critical technical extensions for scientific applications, including batched experimentation, heteroscedasticity, contextual optimisation, and human-in-the-loop integration. Tailored for a broad audience, this tutorial bridges AI advances in BO with practical natural science applications, offering tiered content to empower cross-disciplinary researchers to design more efficient experiments and accelerate principled scientific discovery.

Zhongwei Yu, Rasul Tutunov, Alexandre Max Maraval +13

The Hong Kong University of Science and Technology (Guangzhou) · Huawei Noah’s Ark Lab · University of Science and Technology of China +5

Bayesian OptimizationScientific DiscoveryBayesian Experimental DesignAutonomous Experimentation
AskPDF
Mar 27, 2026

Conditional Neural Bayes Ratio Estimation for Experimental Design Optimisation

For frontier experiments operating at the edge of detectability, instrument design directly determines the probability of discovery. We introduce Conditional Neural Bayes Ratio Estimation (cNBRE), which extends neural Bayes ratio estimation by conditioning on design parameters, enabling a single trained network to estimate Bayes factors across a continuous design space. Applied to 21-cm radio cosmology with simulations representative of the REACH experiment, the amortised nature of cNBRE enables systematic design space exploration that would be intractable with traditional point-wise methods, while recovering established physical relationships. The analysis demonstrates a ~20 percentage point variation in detection probability with antenna orientation for a single night of observation, a design decision that would be trivial to implement if determined prior to antenna construction. This framework enables efficient, globally-informed experimental design optimisation for a wide range of scientific applications.

S. A. K. Leeney, T. Gessey-Jones, W. J. Handley +3

Cavendish Astrophysics, University of Cambridge, Cambridge CB3 0HE, UK. · Kavli Institute for Cosmology, University of Cambridge, Cambridge CB3 0HA, UK. · PhysicsX, London, UK. +1

Bayesian Experimental DesignSimulation-Based InferenceDensity Ratio Estimation
AskPDF
Mar 27, 2026

Curvature-aware Expected Free Energy as an Acquisition Function for Bayesian Optimization

We propose an Expected Free Energy-based acquisition function for Bayesian optimization to solve the joint learning and optimization problem, i.e., optimize and learn the underlying function simultaneously. We show that, under specific assumptions, Expected Free Energy reduces to Upper Confidence Bound, Lower Confidence Bound, and Expected Information Gain. We prove that Expected Free Energy has unbiased convergence guarantees for concave functions. Using the results from these derivations, we introduce a curvature-aware update law for Expected Free Energy and show its proof of concept using a system identification problem on a Van der Pol oscillator. On a two-dimensional benchmark with an oscillatory landscape, our adaptive Expected Free Energy acquisition achieves competitive performance in both regret and mean squared error, unlike the typical acquisition functions that perform well in only one metric.

Ajith Anil Meera, Wouter Kouw

Faculty of Electrical Engineering, TU Eindhoven, The Netherlands

Free Energy CalculationBayesian OptimizationBayesian Experimental DesignExpected Free Energy
AskPDF
Mar 22, 2026

ALMAB-DC: Active Learning, Multi-Armed Bandits, and Distributed Computing for Sequential Experimental Design and Black-Box Optimization

Sequential experimental design under expensive, gradient-free objectives is a central challenge in computational statistics: evaluation budgets are tightly constrained and information must be extracted efficiently from each observation. We propose ALMAB-DC, a GP-based sequential design framework combining active learning, multi-armed bandits (MAB), and distributed asynchronous computing for expensive black-box experimentation. A Gaussian process surrogate with uncertainty-aware acquisition identifies informative query points; a UCB or Thompson-sampling bandit controller allocates evaluations across parallel workers; and an asynchronous scheduler handles heterogeneous runtimes. We present cumulative regret bounds for the bandit components and characterize parallel scalability via Amdahl's Law. We validate ALMAB-DC on five benchmarks. On the two statistical experimental-design tasks, ALMAB-DC achieves lower simple regret than Equal Spacing, Random, and D-optimal designs in dose--response optimization, and in adaptive spatial field estimation matches the Greedy Max-Variance benchmark while outperforming Latin Hypercube Sampling; at K=4K=4 the distributed setting reaches target performance in one-quarter of sequential wall-clock rounds. On three ML/engineering tasks (CIFAR-10 HPO, CFD drag minimization, MuJoCo RL), ALMAB-DC achieves 93.4% CIFAR-10 accuracy (outperforming BOHB by 1.7,pp and Optuna by 1.1,pp), reduces airfoil drag to CD=0.059C_D = 0.059 (36.9% below Grid Search), and improves RL return by 50% over Grid Search. All advantages over non-ALMAB baselines are statistically significant under Bonferroni-corrected Mann--Whitney UU tests. Distributed execution achieves 7.5×7.5\times speedup at K=16K = 16 agents, consistent with Amdahl's Law.

Foo Hui-Mean, Yuan-chin I Chang

Institute of Statistical Science, Academia Sinica, Taipei, Taiwan

Sequential Experimental DesignActive Experimental DesignBayesian OptimizationBayesian Experimental DesignActive LearningMulti-Armed Bandits
AskPDF
Mar 17, 2026

Deep Adaptive Model-Based Design of Experiments

Model-based design of experiments (MBDOE) is essential for efficient parameter estimation in nonlinear dynamical systems. However, conventional adaptive MBDOE requires costly posterior inference and design optimization between each experimental step, precluding real-time applications. We address this by combining Deep Adaptive Design (DAD), which amortizes sequential design into a neural network policy trained offline, with differentiable mechanistic models. For dynamical systems with known governing equations but uncertain parameters, we extend sequential contrastive training objectives to handle nuisance parameters and propose a transformer-based policy architecture that respects the temporal structure of dynamical systems. We demonstrate the approach on four systems of increasing complexity: a fed-batch bioreactor with Monod kinetics, a Haldane bioreactor with uncertain substrate inhibition, a two-compartment pharmacokinetic model with nuisance clearance parameters, and a DC motor for real-time deployment.

Arno Strouwen, Sebastian Micluţa-Câmpeanu

Strouwen Statistics, Hasselt, Belgium · Biosystems Department, KU Leuven, Belgium · JuliaHub, Massachusetts, USA +1

Bayesian Experimental DesignNonlinear System Identification
AskPDF
Mar 14, 2026

Maximin Robust Bayesian Experimental Design

We address the brittleness of Bayesian experimental design under model misspecification by formulating the problem as a max--min game between the experimenter and an adversarial nature subject to information-theoretic constraints. We demonstrate that this approach yields a robust objective governed by Sibson's αα-mutual information (MI), which identifies the αα-tilted posterior as the robust belief update and establishes the Rényi divergence as the appropriate measure of conditional information gain. To mitigate the bias and variance of nested Monte Carlo estimators needed to estimate Sibson's αα-MI, we adopt a PAC-Bayes framework to search over stochastic design policies, yielding rigorous high-probability lower bounds on the robust expected information gain that explicitly control finite-sample error.

Hany Abdulsamad, Sahel Iqbal, Christian A. Naesseth +2

UvA–Bosch Delta Lab University of Amsterdam · Department of Statistics University of Oxford · School of Mathematics University of Edinburgh +1

Bayesian Experimental DesignMutual Information
AskPDF
Mar 12, 2026

Optimal Experimental Design for Reliable Learning of History-Dependent Constitutive Laws

History-dependent constitutive models serve as macroscopic closures for the aggregated effects of micromechanics. Their parameters are typically learned from experimental data. With a limited experimental budget, eliciting the full range of responses needed to characterize the constitutive relation can be difficult. As a result, the data can be well explained by a range of parameter choices, leading to parameter estimates that are uncertain or unreliable. To address this issue, we propose a Bayesian optimal experimental design framework to quantify, interpret, and maximize the utility of experimental designs for reliable learning of history-dependent constitutive models. In this framework, the design utility is defined as the expected reduction in parametric uncertainty or the expected information gain. This enables in silico design optimization using simulated data and reduces the cost of physical experiments for reliable parameter identification. We introduce two approximations that make this framework practical for advanced material testing with expensive forward models and high-dimensional data: (i) a Gaussian approximation of the expected information gain, and (ii) a surrogate approximation of the Fisher information matrix. The former enables efficient design optimization and interpretation, while the latter extends this approach to batched design optimization by amortizing the cost of repeated utility evaluations. Our numerical studies of uniaxial tests for viscoelastic solids show that optimized specimen geometries and loading paths yield image and force data that significantly improve parameter identifiability relative to random designs, especially for parameters associated with memory effects.

Kaushik Bhattacharya, Lianghao Cao, Andrew Stuart

California Institute of Technology, 1200 E. California Blvd., Pasadena, CA 91125, USA.

Bayesian Experimental DesignModel CalibrationParameter IdentifiabilityMaterials Science
AskPDF
Mar 12, 2026

Wasserstein Gradient Flows for Batch Bayesian Optimal Experimental Design

Bayesian optimal experimental design (BOED) provides a powerful, decision-theoretic framework for selecting experiments so as to maximise the expected utility of the data to be collected. In practice, however, its applicability can be limited by the difficulty of optimising the chosen utility. The expected information gain (EIG), for example, is often high-dimensional and strongly non-convex. This challenge is particularly acute in the batch setting, where multiple experiments are to be designed simultaneously. In this paper, we introduce a new approach to batch EIG-based BOED via a probabilistic lifting of the original optimisation problem to the space of probability measures. In particular, we propose to optimise an entropic regularisation of the expected utility over the space of design measures. Under mild conditions, we show that this objective admits a unique minimiser, which can be explicitly characterised in the form of a Gibbs distribution. The resulting design law can be used directly as a randomised batch-design policy, or as a computational relaxation from which a deterministic batch is extracted. To obtain scalable approximations when the batch size is large, we then consider two tractable restrictions of the full batch distribution: a mean-field family, and an i.i.d. product family. For the i.i.d. objective, and formally for its mean-field extension, we derive the corresponding Wasserstein gradient flow, characterise its long-time behaviour, and obtain particle-based algorithms via space-time discretisations. We also introduce doubly stochastic variants that combine interacting particle updates with Monte Carlo estimators of the EIG gradient. Finally, we illustrate the performance of the proposed methods in several numerical experiments, demonstrating their ability to explore multimodal optimisation landscapes and obtain high-utility batches in challenging examples.

Louis Sharrock

Department of Statistical Science, University College London.

Bayesian Experimental DesignWasserstein Gradient Flows
AskPDF
Mar 4, 2026

Observationally Informed Adaptive Causal Experimental Design

Randomized Controlled Trials (RCTs) represent the gold standard for causal inference yet remain a scarce resource. While large-scale observational data is often available, it is utilized only for retrospective fusion, and remains discarded in prospective trial design due to bias concerns. We argue this "tabula rasa" data acquisition strategy is fundamentally inefficient. In this work, we propose Active Residual Learning, a new paradigm that leverages the observational model as a foundational prior. This approach shifts the experimental focus from learning target causal quantities from scratch to efficiently estimating the residuals required to correct observational bias. To operationalize this, we introduce the R-Design framework. Theoretically, we establish two key advantages: (1) a structural efficiency gap, proving that estimating smooth residual contrasts admits strictly faster convergence rates than reconstructing full outcomes; and (2) information efficiency, where we quantify the redundancy in standard parameter-based acquisition (e.g., BALD), demonstrating that such baselines waste budget on task-irrelevant nuisance uncertainty. We propose R-EPIG (Residual Expected Predictive Information Gain), a unified criterion that directly targets the causal estimand, minimizing residual uncertainty for estimation or clarifying decision boundaries for policy. Experiments on synthetic and semi-synthetic benchmarks demonstrate that R-Design significantly outperforms baselines, confirming that repairing a biased model is far more efficient than learning one from scratch.

Erdun Gao, Liang Zhang, Jake Fawkes +5

Australian Institute for Machine Learning, Adelaide University · Hong Kong University of Science and Technology (Guangzhou) · University College London +3

Active Experimental DesignCausal InferenceCausal Effect EstimationAdaptive SamplingBayesian Experimental DesignResidual Learning
AskPDF
Mar 2, 2026

SEED-SET: Scalable Evolving Experimental Design for System-level Ethical Testing

As autonomous systems such as drones, become increasingly deployed in high-stakes, human-centric domains, it is critical to evaluate the ethical alignment since failure to do so imposes imminent danger to human lives, and long term bias in decision-making. Automated ethical benchmarking of these systems is understudied due to the lack of ubiquitous, well-defined metrics for evaluation, and stakeholder-specific subjectivity, which cannot be modeled analytically. To address these challenges, we propose SEED-SET, a Bayesian experimental design framework that incorporates domain-specific objective evaluations, and subjective value judgments from stakeholders. SEED-SET models both evaluation types separately with hierarchical Gaussian Processes, and uses a novel acquisition strategy to propose interesting test candidates based on learnt qualitative preferences and objectives that align with the stakeholder preferences. We validate our approach for ethical benchmarking of autonomous agents on two applications and find our method to perform the best. Our method provides an interpretable and efficient trade-off between exploration and exploitation, by generating up to 2×2\times optimal test candidates compared to baselines, with 1.25×1.25\times improvement in coverage of high dimensional search spaces.

Anjali Parashar, Yingke Li, Eric Yang Yu +4

Laboratory for Information & Decision Systems (LIDS), MIT · Saab Inc.

AI Safety EvaluationBenchmark DesignBayesian Experimental DesignAgent Benchmarks
AskPDF
Feb 2, 2026

Correcting Boundary Bias and Observation Independence in Bayesian Experimental Design

In many experimental settings, active learning can improve sample efficiency by sequentially selecting where to measure, which is particularly valuable when experiments are expensive. Gaussian processes with variance-based acquisition criteria are widely used for this purpose, but have two limitations. First, they are observation-independent: their posterior variance depends only on where samples are acquired, not on what is measured, impairing their sensitivity to the structure of the acquired data. Second, they inflate the variance near boundaries, leading to excessive sampling at the edges of the space compared to the interior. These limitations undermine the gains in sampling efficiency expected from sequential acquisition. We address both limitations. We derive a reconstruction-driven design density and use the posterior mean to build a training-free warp that places more measurements where the target function varies rapidly. A geometric equalizer separately corrects boundary bias. Across sixteen synthetic and two real-data benchmarks, the geometric equalizer consistently improves function reconstruction by correcting boundary bias, while the reconstruction warp provides further gains by concentrating measurements where the posterior mean varies rapidly.

Sanna Jarl, Jens Sjölund, Jonathan J. S. Scragg +1

Department of Material Science & Engineering Uppsala University RISE · Department of Information Technology Uppsala University · RISE

Active LearningAdaptive SamplingBayesian Experimental DesignGaussian Process RegressionActive Experimental Design
AskPDF
Jan 16, 2026

A Probabilistic Approach to Trajectory-Based Optimal Experimental Design

We present a novel probabilistic approach for optimal experimental path design. In this approach a discrete path optimization problem is defined on a static navigation mesh, and trajectories are modeled as random variables governed by a parametric Markov policy. The discrete path optimization problem is then replaced with an equivalent stochastic optimization problem over the policy parameters, resulting in an optimal probability model that samples estimates of the optimal discrete path. This approach enables exploration of the utility function's distribution tail and treats the utility function of the design as a black box, making it applicable to linear and nonlinear inverse problems and beyond experimental design. Numerical verification and analysis are carried out by using a parameter identification problem widely used in model-based optimal experimental design, namely a two-dimensional time-dependent advection diffusion problem in which the initial condition is the inference target. Experiments use both coarse and fine navigation meshes, with either a single moving sensor or a group of seven coordinated sensors, and the proposed approach is evaluated under D-, A-, and E-optimality criteria.

Ahmed Attia

Mathematics and Computer Science Division, Argonne National Lab, USA

Stochastic OptimizationTrajectory OptimizationBayesian Experimental Design
AskPDF
Dec 28, 2025

JADAI: Jointly Amortizing Adaptive Design and Bayesian Inference

We consider problems of parameter estimation where design variables can be actively optimized to maximize information gain. To this end, we introduce JADAI, a framework that jointly amortizes Bayesian adaptive design and inference by training a policy, a history network, and an inference network end-to-end. The networks minimize a generic loss that aggregates incremental reductions in posterior error along experimental sequences. Inference networks are instantiated with diffusion-based posterior estimators that can approximate high-dimensional and multimodal posteriors at every experimental step. Across standard adaptive design benchmarks, JADAI achieves superior or competitive performance.

Niels Bracher, Lars Kühmichel, Desi R. Ivanova +3

1Rensselaer Polytechnic Institute, USA · 2TU Dortmund University, Germany · University of Oxford, UK.

Parameter EstimationAmortized InferenceBayesian InferenceBayesian Experimental Design
AskPDF
Nov 10, 2025

Robust Experimental Design via Generalised Bayesian Inference

Bayesian optimal experimental design is a principled framework for conducting experiments that leverages Bayesian inference to quantify how much information one can expect to gain from selecting a certain design. However, accurate Bayesian inference relies on the assumption that one's statistical model of the data-generating process is correctly specified. If this assumption is violated, Bayesian methods can lead to poor inference and estimates of information gain. Generalised Bayesian (or Gibbs) inference is a more robust probabilistic inference framework that replaces the likelihood in the Bayesian update by a suitable loss function. In this work, we present Generalised Bayesian Optimal Experimental Design (GBOED), an extension of generalised Bayesian inference to the experimental design setting which achieves robustness in both design and inference. Using a generalisation of Bayesian decision theory, we derive a new acquisition function, the Gibbs expected information gain. Our empirical results demonstrate that GBOED can enhance robustness to outliers and incorrect assumptions about the outcome noise distribution.

Yasir Zubayr Barlas, Sabina J. Sloman, Samuel Kaski

The University of Manchester · Aalto University

Generalized Bayesian InferenceBayesian Experimental DesignBayesian Inference
AskPDF
Oct 1, 2025

Approximation of differential entropy in Bayesian optimal experimental design

Bayesian optimal experimental design provides a principled framework for selecting experimental configurations that maximize obtained information. In this work, we consider settings where the likelihood entropy is independent of the design or can be evaluated explicitly, which applies, e.g. to additive noise models with design-independent or explicitly computable noise entropy. This reduces the problem to maximum entropy estimation, where the nested integral defining the expected information gain collapses into a single integral. Our study is motivated by large-scale inference problems, such as inverse problems, where the computational cost is dominated by expensive likelihood evaluations. Under additive Gaussian noise model, we propose a computational approach in which the evidence density is approximated by a Monte Carlo or quasi-Monte Carlo surrogate, while the differential entropy is evaluated using standard methods without additional likelihood evaluations. We prove that this strategy achieves convergence rates that are comparable to, or better than, state-of-the-art methods for full expected information gain estimation, particularly when the cost of entropy evaluation is negligible. Our approach relies only on mild smoothness of the forward map and avoids stronger technical assumptions required in earlier work. We also present numerical experiments, which confirm some of our theoretical findings.

Chuntao Chen, Tapio Helin, Nuutti Hyvönen +1

Bayesian Experimental DesignValue of InformationBayesian Inverse Problems
AskPDF