cs.LGApr 1, 2026
SaveEfficient and Principled Scientific Discovery through Bayesian Optimization: A Tutorial
Organizations: The Hong Kong University of Science and Technology (Guangzhou) · Huawei Noah’s Ark Lab · University of Science and Technology of China · Tsinghua University ·
Abstract
Traditional scientific discovery relies on an iterative hypothesise-experiment-refine cycle that has driven progress for centuries, but its intuitive, ad-hoc implementation often wastes resources, yields inefficient designs, and misses critical insights. This tutorial presents Bayesian Optimisation (BO), a principled probability-driven framework that formalises and automates this core scientific cycle. BO uses surrogate models (e.g., Gaussian processes) to model empirical observations as evolving hypotheses, and acquisition functions to guide experiment selection, balancing exploitation of known knowledge and exploration of uncharted domains to eliminate guesswork and manual trial-and-error. We first frame scientific discovery as an optimisation problem, then unpack BO's core components, end-to-end workflows, and real-world efficacy via case studies in catalysis, materials science, organic synthesis, and molecule discovery. We also cover critical technical extensions for scientific applications, including batched experimentation, heteroscedasticity, contextual optimisation, and human-in-the-loop integration. Tailored for a broad audience, this tutorial bridges AI advances in BO with practical natural science applications, offering tiered content to empower cross-disciplinary researchers to design more efficient experiments and accelerate principled scientific discovery.
Figures & tables
Figure 1: A four-panel comic illustrates the conceptual journey of translating human scientific intuition into a formalised, autonomous discovery framework for identifying a high-performance “ever-light” catalyst. All characters, dialogues, and events depicted in this narrative are entirely fictional and are used solely for illustrative purposes. The main characters are Professor Aris (senior researcher), Leo (PhD student), and Sarah (research engineer). The four scenes: (1) The Wall of Infinite Choices highlights the complexity of the chemical search space and the tension between logical certainty and probabilistic exploration; (2) The High Stakes of “Just One More” exposes the “exploitation trap” of settling for local optima under resource constraints versus unguided exploration; (3) The “Ghost” in the Machine poses the question of translating uncertainty into actionable instructions for autonomous experimentation; (4) The Vision of the Autonomous Lab resolves this tension via Bayesian optimization, where surrogate models quantify uncertainty, acquisition functions balance exploration/exploitation, and autonomous labs navigate the search space efficiently.
Figure 2: The illustration of core paradigms of scientific discovery with historical representatives. (i) Demonstrative discovery derives certain knowledge from self-evident first principles through formal logical deduction, , with the inferential process itself independent of empirical observation. (ii) Inductive discovery constructs general laws by systematically accumulating empirical data via controlled experiments, inducing patterns from an evidence base. (iii) Hypothetico-deductive discovery proposes tentative hypotheses, derives testable predictions, validates them against empirical data, and refines or rejects the hypotheses through consistency checks. (iv) Bayesian hypothetico-deductive discovery is an emergent modern methodology that quantifies degrees of belief across a hypothesis space and iteratively updates these beliefs via Bayesian inference; the resulting posterior provides both a summary of robust knowledge and guidance for decision-making.
Figure 3: The AntBO workflow for automated antibody design (Figure 1 of [ 46 ] ). (A) Constraints on the search space: Examples of biologically unsatisfiable CDRH3 sequences (e.g., excessive repetition or invalid charge) that the optimiser must learn to avoid. (B) The Bayesian Optimisation loop: (Step 1) Starting from an initial dataset of sequences and their energy/affinity scores; (Step 2) Fitting a Gaussian Process ( ) to model the objective and its uncertainty; (Step 3) Maximising an acquisition function ( ) within a local trust region to propose a candidate that balances performance and biological plausibility; (Step 4) Evaluating the candidate via a black-box oracle (e.g., the Absolut! simulator) to generate new data and update the model.
Figure 4: Workflow of the spatially adaptive active learning strategy. Stage I minimises overpotential via active learning, while a CVAE defines a low-overpotential subspace. Stage II then maximises stability within this subspace to identify highly active and durable catalysts. PES, EI, and KG denote predictive entropy search, expected improvement, and knowledge gradient, respectively. Func.: function.
Figure 5: Scientific discovery as sequential model-based optimisation: iteratively updating a model of the unknown mechanism based on experimental data, and using this model to guide the selection of future experiments.
| Case Name | Section | Total Trials | Performance Measure | Best Solution Performance | |
|---|---|---|---|---|---|
| (Maximise / Minimise) | Random Search | BO | |||
| Photocatalytic HER Catalyst | 5.1 | 200 | Regret (minimise) | 21.8 | 9.7 |
| HEA Nanozyme Formulation | 5.2 | 200 | Regret (minimise) | 244 | 155 |
| OER Electrocatalyst Design | 5.3 | 200 | Overpotential (mV, minimise) | 245 | 219 |
| Buchwald-Hartwig Reaction | 5.4 | 200 | Regret (minimise) | 47 | 9.9 |
| Molecular QED Optimisation | 5.5 | 100 | QED Score (maximise) | 0.881 | 0.918 |
Table 1: Preview of final optimisation performance across scientific discovery case studies
Figure 6: Illustration of surrogate modelling in black-box optimisation. The red curve shows the true objective function, and red dots indicate observed data points that are slightly perturbed from the true objectives. The blue curve represents the surrogate model’s predicted mean, while the shaded region denotes the 95% confidence interval, quantifying the model’s uncertainty about the objective.
Figure 7: ReLU & Maximising each of these functions implies encouraging their internal expressions to be positive. In the case of the Heaviside function, this corresponds to driving the argument above zero to trigger a non-zero response. For the ReLU function, the goal is to make the argument as large and positive as possible to maximise the acquisition value.
Figure 8: Visualization of the Gaussian Process (GP) surrogate model and acquisition functions used in Bayesian Optimization. The true objective function, , is shown as a dashed black line. Observed data points are marked as red circles, where the Gaussian noise satisfies . The GP surrogate model is depicted as the blue curve (mean prediction), with the shaded region representing the 95% confidence interval ( ). The acquisition functions—Upper Confidence Bound (UCB), Expected Improvement (EI), and Probability of Improvement (PI)—are plotted in green, blue, and red, respectively, to illustrate their behaviour in guiding the selection of the next evaluation point. We do not plot Thompson Sampling as it is randomly drawn from the posterior surrogate.
Figure 9: Bayesian Optimisation Loop: At round , a surrogate model (e.g., a Gaussian process) is fitted to the observed data points ( ). The next point is selected by optimising an acquisition function derived from the model posterior. The new observation is then evaluated and added to the dataset, iteratively refining the model. The solid brown line depicts the mean prediction of the model, while the dashed lines illustrate the associated variance at each input point .
Figure 10: The optimisation curves (the best input found after each iteration) for the problems with a mathematical design space. We compare HEBO with random search and standard BO with LCB acquisition. For reproducibility, each curve averages runs across 16 random seeds, and the shaded area displays the standard error across seeds.
Figure 11: Optimisation curves for molecular QED maximisation using descriptor-based GP-UCB and random search. Each curve shows the mean best QED score found across 16 random seeds; shaded regions indicate standard deviation. BO consistently outperforms random search, demonstrating the effectiveness of integrating BO with domain-specific scientific tools.
More in this topic
Bayesian Optimization
Bayesian optimization uses probabilistic models and sequential experiments to find strong solutions when evaluating a black-box objective is costly.
See all 243 papers