Efficient and Principled Scientific Discovery through Bayesian Optimization: A Tutorial
Authors: Zhongwei Yu, Rasul Tutunov, Alexandre Max Maraval, Zikai Xie, Zhenzhi Tan, Jiankang Wang, Bin Cao, Zijing Li, Liangliang Xu, Qi Yang, Jun Jiang, Sanzhong Luo, Zhenxiao Guo, Tongyi Zhang, Haitham Bou-Ammar, Jun Wang
Organizations: The Hong Kong University of Science and Technology (Guangzhou) · Huawei Noah’s Ark Lab · University of Science and Technology of China · Tsinghua University · The University of Hong Kong · Haihe Laboratory of Sustainable Chemical Transformations · Shanghai University · University College London
Traditional scientific discovery relies on an iterative hypothesise-experiment-refine cycle that has driven progress for centuries, but its intuitive, ad-hoc implementation often wastes resources, yields inefficient designs, and misses critical insights. This tutorial presents Bayesian Optimisation (BO), a principled probability-driven framework that formalises and automates this core scientific cycle. BO uses surrogate models (e.g., Gaussian processes) to model empirical observations as evolving hypotheses, and acquisition functions to guide experiment selection, balancing exploitation of known knowledge and exploration of uncharted domains to eliminate guesswork and manual trial-and-error. We first frame scientific discovery as an optimisation problem, then unpack BO's core components, end-to-end workflows, and real-world efficacy via case studies in catalysis, materials science, organic synthesis, and molecule discovery. We also cover critical technical extensions for scientific applications, including batched experimentation, heteroscedasticity, contextual optimisation, and human-in-the-loop integration. Tailored for a broad audience, this tutorial bridges AI advances in BO with practical natural science applications, offering tiered content to empower cross-disciplinary researchers to design more efficient experiments and accelerate principled scientific discovery.
Figures & tables
Figure 1: A four-panel comic illustrates the conceptual journey of translating human scientific intuition into a formalised, autonomous discovery framework for identifying a high-performance “ever-light” catalyst. All characters, dialogues, and events depicted in this narrative are entirely fictional and are used solely for illustrative purposes. The main characters are Professor Aris (senior researcher), Leo (PhD student), and Sarah (research engineer). The four scenes: (1) The Wall of Infinite Choices highlights the complexity of the chemical search space and the tension between logical certainty and probabilistic exploration; (2) The High Stakes of “Just One More” exposes the “exploitation trap” of settling for local optima under resource constraints versus unguided exploration; (3) The “Ghost” in the Machine poses the question of translating uncertainty into actionable instructions for autonomous experimentation; (4) The Vision of the Autonomous Lab resolves this tension via Bayesian optimization, where surrogate models quantify uncertainty, acquisition functions balance exploration/exploitation, and autonomous labs navigate the search space efficiently.
Figure 2: The illustration of core paradigms of scientific discovery with historical representatives. (i) Demonstrative discovery derives certain knowledge from self-evident first principles through formal logical deduction, , with the inferential process itself independent of empirical observation. (ii) Inductive discovery constructs general laws by systematically accumulating empirical data via controlled experiments, inducing patterns from an evidence base. (iii) Hypothetico-deductive discovery proposes tentative hypotheses, derives testable predictions, validates them against empirical data, and refines or rejects the hypotheses through consistency checks. (iv) Bayesian hypothetico-deductive discovery is an emergent modern methodology that quantifies degrees of belief across a hypothesis space and iteratively updates these beliefs via Bayesian inference; the resulting posterior provides both a summary of robust knowledge and guidance for decision-making.
Figure 3: The AntBO workflow for automated antibody design (Figure 1 of [ 46 ] ). (A) Constraints on the search space: Examples of biologically unsatisfiable CDRH3 sequences (e.g., excessive repetition or invalid charge) that the optimiser must learn to avoid. (B) The Bayesian Optimisation loop: (Step 1) Starting from an initial dataset of sequences and their energy/affinity scores; (Step 2) Fitting a Gaussian Process ( GPθ ) to model the objective and its uncertainty; (Step 3) Maximising an acquisition function ( αθ ) within a local trust region to propose a candidate x∗ that balances performance and biological plausibility; (Step 4) Evaluating the candidate via a black-box oracle (e.g., the Absolut! simulator) to generate new data and update the model.
Figure 4: Workflow of the spatially adaptive active learning strategy. Stage I minimises overpotential via active learning, while a CVAE defines a low-overpotential subspace. Stage II then maximises stability within this subspace to identify highly active and durable catalysts. PES, EI, and KG denote predictive entropy search, expected improvement, and knowledge gradient, respectively. Func.: function.
Figure 5: Scientific discovery as sequential model-based optimisation: iteratively updating a model of the unknown mechanism based on experimental data, and using this model to guide the selection of future experiments.
Case Name
Section
Total Trials
Performance Measure
Best Solution Performance
(Maximise / Minimise)
Random Search
BO
Photocatalytic HER Catalyst
5.1
200
Regret (minimise)
21.8
9.7
HEA Nanozyme Formulation
5.2
200
Regret (minimise)
244
155
OER Electrocatalyst Design
5.3
200
Overpotential (mV, minimise)
245
219
Buchwald-Hartwig Reaction
5.4
200
Regret (minimise)
47
9.9
Molecular QED Optimisation
5.5
100
QED Score (maximise)
0.881
0.918
Table 1: Preview of final optimisation performance across scientific discovery case studies
Figure 6: Illustration of surrogate modelling in black-box optimisation. The red curve shows the true objective function, and red dots indicate observed data points that are slightly perturbed from the true objectives. The blue curve represents the surrogate model’s predicted mean, while the shaded region denotes the 95% confidence interval, quantifying the model’s uncertainty about the objective.
Figure 7: ReLU (u) & I(u) Maximising each of these functions implies encouraging their internal expressions to be positive. In the case of the Heaviside function, this corresponds to driving the argument above zero to trigger a non-zero response. For the ReLU function, the goal is to make the argument as large and positive as possible to maximise the acquisition value.
Figure 8: Visualization of the Gaussian Process (GP) surrogate model and acquisition functions used in Bayesian Optimization. The true objective function, f(x)=sin(2πx) , is shown as a dashed black line. Observed data points are marked as red circles, where the Gaussian noise satisfies N(0,0.12) . The GP surrogate model is depicted as the blue curve (mean prediction), with the shaded region representing the 95% confidence interval ( μ±2σ ). The acquisition functions—Upper Confidence Bound (UCB), Expected Improvement (EI), and Probability of Improvement (PI)—are plotted in green, blue, and red, respectively, to illustrate their behaviour in guiding the selection of the next evaluation point. We do not plot Thompson Sampling as it is randomly drawn from the posterior surrogate.
Figure 9: Bayesian Optimisation Loop: At round N , a surrogate model (e.g., a Gaussian process) is fitted to the observed data points ( {xt,yt}t=1N ). The next point xN+1 is selected by optimising an acquisition function derived from the model posterior. The new observation yN+1 is then evaluated and added to the dataset, iteratively refining the model. The solid brown line depicts the mean prediction of the model, while the dashed lines illustrate the associated variance at each input point x .
Figure 10: The optimisation curves (the best input found after each iteration) for the problems with a mathematical design space. We compare HEBO with random search and standard BO with LCB acquisition. For reproducibility, each curve averages runs across 16 random seeds, and the shaded area displays the standard error across seeds.
Figure 11: Optimisation curves for molecular QED maximisation using descriptor-based GP-UCB and random search. Each curve shows the mean best QED score found across 16 random seeds; shaded regions indicate standard deviation. BO consistently outperforms random search, demonstrating the effectiveness of integrating BO with domain-specific scientific tools.
The high cost and data scarcity in scientific exploration have motivated the use of large language models (LLMs) as knowledge-driven components in Bayesian optimization (BO). However, existing approaches typically embed LLMs directly into the sampling or surrogate modeling pipeline, without fully leveraging their significantly lower evaluation cost compared to real-world experiments. To address this limitation, we propose LLM-Accelerated Bayesian Optimization (LABO), a framework that combines LLM predictions with experimental observations within a single BO loop. LABO employs a gating criterion to dynamically balance the reliance on LLM predictions versus actual experiments. By leveraging inexpensive LLM evaluations to broadly explore the search space and reserving costly real experiments only for regions with high uncertainty, LABO achieves more sample-efficient optimization. We provide a theoretical analysis with a cumulative regret bound that formalizes this efficiency gain. Empirical results across diverse scientific tasks demonstrate that LABO consistently outperforms existing methods under identical experimental budgets. Our results suggest that LABO offers a practical and theoretically grounded approach for integrating LLMs into scientific discovery workflows.
Zhuo Chen, Xinzhe Yuan, Jianshu Zhang +8
equal contribution · Shanghai Artificial Intelligence Laboratory, Shanghai, China · School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, China +7
When applying Bayesian optimization (BO) to scientific workflow, a major yet often overlooked source of uncertainty is the task itself -- namely, what to optimize and how to evaluate it -- which can evolve as evidence accumulates. We introduce Generate-Select-Refine (GSR), a open-ended BO framework that alternates between task generation and task optimization. Starting from a user-provided seed task, GSR generates new tasks in a coarse-to-fine manner while a task-acquisition function schedules optimization. Asymptotically, it concentrates evaluations on the best task, incurring only logarithmic regret overhead relative to single-task BO. We apply GSR to new product development, chemical synthesis scaling, algorithm analysis, and patent repurposing, where it outperforms existing LLM-based optimizers.
Masaki Adachi, Yuta Suzuki, Juliusz Ziomek
Lattice Lab · Toyota Motor Corporation · Machine Learning Research Group +1
Mixed-variable optimization in scientific discovery often requires exploring large combinatorial spaces under costly experimental evaluations, making sample?efficient Bayesian optimization (BO) essential. However, existing BO methods typ?ically rely on structural descriptors to define similarity among categorical variables, implicitly assuming that structural similarity reflects similarity in task performance. In scientific optimization, this assumption is often violated, leading to surrogate representations that are poorly aligned with the optimization objective. To address this limitation, we propose TagBO (Task-aware Graph Bayesian Optimization), a method that integrates LLM-derived relational priors with descriptor-based struc?tural information for mixed-variable Bayesian optimization. Rather than directly using LLM outputs as predictive scores, TagBO treats LLM outputs as a potentially noisy source of task-aware relational priors and extracts coarse ordinal relational signals from LLM reasoning to construct task-aware similarity structures for scien?tific optimization. TagBO then performs constrained Gaussian process Bayesian optimization over the resulting joint discrete-continuous representation space. Ex?periments on diverse chemistry and materials formulation tasks demonstrate that TagBO consistently improves sample efficiency under fixed evaluation budgets across LLMs of varying capabilities.
Xinzhe Yuan, Zimu Zeng, Zhuo Chen +6
Harbin Institute of Technology · Shanghai Artificial Intelligence Laboratory · Fudan University +4