Informing Acquisition Functions via Foundation Models for Molecular Discovery
Organizations: Department of Computer Science, University of Toronto · Data Science Institute · Vector Institute · NVIDIA Research Seattle Robotics Lab (SRL) ·
Abstract
Bayesian Optimization (BO) is a key methodology for accelerating molecular discovery by estimating the mapping from molecules to their properties while seeking the optimal candidate. Typically, BO iteratively updates a probabilistic surrogate model of this mapping and optimizes acquisition functions derived from the model to guide molecule selection. However, its performance is limited in low-data regimes with insufficient prior knowledge and vast candidate spaces. Large language models (LLMs) and chemistry foundation models offer rich priors to enhance BO, but high-dimensional features, costly in-context learning, and the computational burden of deep Bayesian surrogates hinder their full utilization. To address these challenges, we propose a likelihood-free BO method that bypasses explicit surrogate modeling and directly leverages priors from general LLMs and chemistry-specific foundation models to inform acquisition functions. Our method also learns a tree-structured partition of the molecular search space with local acquisition functions, enabling efficient candidate selection via Monte Carlo Tree Search. By further incorporating coarse-grained LLM-based clustering, it substantially improves scalability to large candidate sets by restricting acquisition function evaluations to clusters with statistically higher property values. We show through extensive experiments and ablations that the proposed method substantially improves scalability, robustness, and sample efficiency in LLM-guided BO for molecular discovery.
Figures & tables
Appendix figures & tables47 assets
Supplementary material from the paper’s appendix.
Appendix
| Model | PP | DC HunAcc | DC Spearman | DC PairAcc | PC HunAcc | PC Spearman | PC PairAcc |
|---|---|---|---|---|---|---|---|
| GPT-4o | |||||||
| Gemini-2.5-pro | |||||||
| Qwen3-Plus |
| Category | Parameter | Value |
|---|---|---|
| General | 0.5 | |
| 0.5 | ||
| 0.005, 0.01 | ||
| tree_depth | 3 | |
| p_val | 0, 0.01, 0.05 | |
| threshold | 0.5 |
| Category | Parameter | Value |
|---|---|---|
| PEFT | batch_size | 16, 32 |
| Head Parameters: | ||
| n_epochs | 50 | |
| learning rate | 1e-2 | |
| batch_size | 256 | |
| leaf_sample_size | 2 |
| Dataset | LLMAT | GP | Laplace | Random |
|---|---|---|---|---|
| Redoxmer | 92.3 0.9 | 85.4 2.5 | 75.2 3.1 | 30.9 6.4 |
| Solvation | 83.9 2.8 | 85.3 1.1 | 76.1 1.5 | 44.5 4.8 |
| Kinase | 77.1 4.7 | 78.8 1.8 | 67.9 4.4 | 47.4 6.6 |
| Laser | 74.3 3.4 | 47.8 4.8 | 63.9 3.8 | 30.2 5.6 |
| Photovoltaics | 68.6 6.4 | 55.0 6.6 | 59.3 6.3 | 41.1 6.3 |
| Photoswitches | 78.5 3.0 | 82.4 3.5 | 59.2 5.3 | 34.9 7.5 |
| Dataset | Size | T5-Chem | Fingerprints | MolFormer | T5-Base | GPT2-Med | GPT2-Large | LLaMA-2-7B |
|---|---|---|---|---|---|---|---|---|
| Redoxmer | 1407 | 0.940 | 0.931 | 0.919 | 0.920 | 0.892 | 0.882 | 0.915 |
| Solvation | 1407 | 0.856 | 0.833 | 0.845 | 0.847 | 0.834 | 0.841 | 0.843 |
| Kinase | 10449 | 0.757 | 0.680 | 0.736 | 0.708 | 0.689 | 0.697 | 0.744 |
| Laser | 10000 | 0.574 | 0.598 | 0.548 | 0.514 | 0.566 | 0.570 | 0.629 |
| PCE | 10000 | 0.608 | 0.595 | 0.577 | 0.581 | 0.549 | 0.578 | 0.607 |
| Photoswitch | 392 | 0.441 | 0.516 | 0.352 | 0.320 | 0.303 | 0.261 | 0.335 |
| Dataset | GP worst | LLMAT worst | Laplace worst |
|---|---|---|---|
| Redoxmer | 70.15 2.94 (GPT2-Med) | 82.09 2.29 (LLaMA-2-7B) | 72.86 2.76 (GPT2-Med) |
| Solvation | 56.85 3.37 (GPT2-Med) | 82.72 3.29 (GIN) | 66.36 1.38 (LLaMA-2-7B) |
| Kinase | 55.53 4.21 (Fingerprints) | 74.64 3.28 (MolFormer) | 50.83 4.07 (Fingerprints) |
| Laser | 42.74 3.10 (GPT2-Med) | 56.78 4.96 (Fingerprints) | 57.84 2.72 (Fingerprints) |
| Photovoltaics | 53.90 4.90 (GPT2-Med) | 69.44 3.36 (LLaMA-2-7B) | 57.47 3.06 (Fingerprints) |
| Photoswitches | 51.50 6.13 (GPT2-Med) | 68.02 6.22 (LLaMA-2-7B) | 63.12 4.10 (GIN) |
| Dataset | LLMAT | GP | Laplace |
|---|---|---|---|
| Redoxmer | LLM clearly better | LLM slightly better | Comparable |
| Solvation | LLM slightly better | LLM slightly better | LLM slightly better |
| Kinase | LLM better | LLM better | LLM better |
| Laser | LLM clearly better | LLM better | LLM clearly better |
| Photovoltaics | LLM better | LLM slightly better | LLM better |
| Photoswitches | FP clearly better | FP clearly better | LLM slightly better |
| Algorithm | Morgan FP | Best FM | Best FM beats FP | Average FM | Average FM beats FP | ||
|---|---|---|---|---|---|---|---|
| LLMAT | 74.8 | 84.4 | +9.7 | 5/6 | 77.4 | +2.6 | 4/6 |
| GP | 72.0 | 78.7 | +6.7 | 5/6 | 68.2 | 3.8 | 1/6 |
| Laplace | 65.3 | 81.2 | +15.9 | 6/6 | 72.3 | +7.0 | 3/6 |
| Dataset | LLMAT | Decoupled | Gain |
|---|---|---|---|
| Redoxmer | 94.6 2.4 | 89.6 4.4 | +5.0 |
| Solvation | 92.0 3.2 | 86.2 4.0 | +5.8 |
| Kinase | 88.7 3.7 | 88.2 3.9 | +0.4 |
| Laser | 82.5 4.8 | 83.5 2.9 | 1.0 |
| Photovoltaics | 88.8 3.3 | 88.2 4.5 | +0.6 |
| Photoswitches | 88.8 6.0 | 86.7 5.8 | +2.0 |
| Config | Gap AUC |
|---|---|
| LLMAT w/ LLM Clustering | 82.36 1.49 |
| LLMAT w/ K-means Clustering | 80.24 1.63 |
| LLMAT Base (w/o Clustering) | 80.53 1.58 |
| LLMAT w/ Greedy Selection (UCT Greedy) | 80.29 1.62 |
| LLMAT Base w/o Meta ( ) | 79.75 1.76 |
| LLMAT Base w/o MCTS (tree depth ) | 78.39 1.89 |
| Depth | Method | Avg. # samples | Final loss | Avg. | Avg. |
|---|---|---|---|---|---|
| 0 | meta | 25.4 | 0.1871 0.0187 | 7.9 | 17.5 |
| 0 | w/o meta | 23.2 | 0.1439 0.0155 | 8.5 | 14.7 |
| 1 | meta | 11.5 | 0.4121 0.0267 | 3.4 | 8.1 |
| 1 | w/o meta | 11.4 | 0.4269 0.0290 | 3.5 | 7.9 |
| 2 | meta | 7.6 | 0.2772 0.0330 | 2.3 | 5.4 |
| 2 | w/o meta | 7.4 | 0.3221 0.0461 | 2.2 | 5.2 |
| Depth | Gap AUC ( ) | GAP@25 | GAP@50 | GAP@100 | Regret@100 ( ) | Time (s) |
|---|---|---|---|---|---|---|
| 0 | 78.12 1.77 | 0.7341 0.0334 | 0.8671 0.0256 | 0.9748 0.0055 | 7.2938 1.9920 | 43.9 4.2 |
| 1 | 79.60 1.57 | 0.7813 0.0291 | 0.9103 0.0163 | 0.9554 0.0125 | 6.1819 1.9737 | 69.4 6.7 |
| 2 | 78.00 1.73 | 0.7473 0.0334 | 0.8831 0.0234 | 0.9630 0.0098 | 6.9169 2.0112 | 105.7 10.1 |
| 3 | 80.08 1.53 | 0.7652 0.0318 | 0.9056 0.0198 | 0.9725 0.0073 | 5.8139 1.7825 | 131.6 13.5 |
| 4 | 79.91 1.67 | 0.7701 0.0314 | 0.9129 0.0183 | 0.9628 0.0107 | 6.2854 2.1980 | 171.6 18.5 |
| Dataset | GPT-4o | Gemini | Qwen3 | Latency | AUC-GAP (pp) | |
|---|---|---|---|---|---|---|
| Redox-Mer | 1,407 | $0.103 | $0.076 | $0.018 | 3.3 min | 0.937 |
| Solvation | 1,407 | $0.102 | $0.075 | $0.018 | 3.3 min | +0.089 |
| Kinase | 10,449 | $0.647 | $0.482 | $0.111 | 22.1 min | +5.773 |
| Laser | 10,000 | $1.042 | $0.671 | $0.203 | 20.8 min | +1.149 |
| PCE | 10,000 | $0.670 | $0.485 | $0.118 | 20.8 min | +4.833 |
| Photoswitch | 392 | $0.023 | $0.018 | $0.004 | 0.8 min | 0.101 |
| Pool size | LLMAT | GP | Laplace | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Train | Predict | Total | Train | Predict | Total | Train | Predict | Total | |
| 392 | 0.680 | 0.058 | 0.816 | 1.521 | 0.014 | 1.548 | 1.323 | 0.181 | 1.516 |
| 1,407 | 0.590 | 0.097 | 0.773 | 1.424 | 0.035 | 1.485 | 1.542 | 0.597 | 2.167 |
| 10,000 | 0.925 | 0.673 | 1.713 | 2.091 | 0.272 | 2.616 | 2.836 | 4.513 | 7.557 |
| 10,449 | 0.828 | 0.701 | 1.636 | 2.350 | 0.318 | 2.944 | 3.733 | 6.764 | 10.801 |
Bayesian Optimization
Bayesian optimization uses probabilistic models and sequential experiments to find strong solutions when evaluating a black-box objective is costly.
See all 271 papers