cs.LGOct 10, 2026

TagBO: LLM-Driven Task-Aware Graph Bayesian Optimization for Scientific Discovery

Authors: Xinzhe Yuan, Zimu Zeng, Zhuo Chen, Jianshu Zhang, Jinzong Dong, Jingyi Chen, Nanyang Ye, Huan Xiong, 

Organizations: Harbin Institute of Technology · Shanghai Artificial Intelligence Laboratory · Fudan University · Shenzhen Loop Area Institute · 

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Mixed-variable optimization in scientific discovery often requires exploring large combinatorial spaces under costly experimental evaluations, making sample?efficient Bayesian optimization (BO) essential. However, existing BO methods typ?ically rely on structural descriptors to define similarity among categorical variables, implicitly assuming that structural similarity reflects similarity in task performance. In scientific optimization, this assumption is often violated, leading to surrogate representations that are poorly aligned with the optimization objective. To address this limitation, we propose TagBO (Task-aware Graph Bayesian Optimization), a method that integrates LLM-derived relational priors with descriptor-based struc?tural information for mixed-variable Bayesian optimization. Rather than directly using LLM outputs as predictive scores, TagBO treats LLM outputs as a potentially noisy source of task-aware relational priors and extracts coarse ordinal relational signals from LLM reasoning to construct task-aware similarity structures for scien?tific optimization. TagBO then performs constrained Gaussian process Bayesian optimization over the resulting joint discrete-continuous representation space. Ex?periments on diverse chemistry and materials formulation tasks demonstrate that TagBO consistently improves sample efficiency under fixed evaluation budgets across LLMs of varying capabilities.

Figures & tables

Figure 1: Overview of TagBO on Suzuki coupling reaction optimization. For each categorical variable, TagBO constructs an LLM-derived task-aware relational graph from semantic rankings under the target task, alongside a structural graph derived from physicochemical descriptors. The two signals are fused to form a task-aware optimization space, which is combined with continuous variables and used for Bayesian optimization.
Figure 2: Optimization trajectories on six tasks. The y-axis reports the best objective value found so far, normalized by the oracle optimum, and the x-axis reports the iteration number. TagBO is compared against One-hot Feature BO, Property Feature BO, COMBO, BOPRO, and LGBO.
Figure 3: Comparison of the average number of iterations required to reach 95% of the oracle optimum on six benchmark tasks. Lower is better.
Figure 4: Ablation analysis on the Suzuki dataset. (A) Comparison with two reduced variants using only LLM-derived scoring features or only physicochemical property features. (B) Optimization trajectories by different LLMs, where curves report the best objective value found so far, normalized by the oracle optimum. (C) Mean Step@95% under different LLMs. ∗T*^{T} denote Thinking variants.
Figure 5: 3D UMAP visualization of feature spaces. Each point represents a reaction candidate; the horizontal axes show the UMAP embedding, and the vertical axis encodes the reaction yield.
Figure 6: Real-world formulation optimization. (A) Thermal conductivity and the best value found so far over 34 physical evaluations. (B) Final measurements relative to their qualification thresholds; values above one satisfy the requirements.
Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Input/OutputNameRange / CountDescriptionType
InputsDB_block22Donor-bridge molecular building blockCategorical
 A_block100Acceptor molecular building blockCategorical
OutputPredictedSO[0,+∞)[0,+\infty)DFT-predicted spectral overlap scoreContinuous
Appendix
Table 1: Input and Output variables for Organic Photovoltaics task.
Input/OutputNameRange / CountDescriptionType
Inputsligand12Ligand selected from 12 categorical optionsCategorical
 base8Base selected from 8 categorical optionsCategorical
 solvent4Solvent selected from 4 categorical optionsCategorical
Outputyield[0,100][0,100]Reaction yield to be maximizedContinuous
Appendix
Table 2: Input and Output variables for Suzuki–Miyaura Reaction task.
Input/OutputNameRange / CountDescriptionType
Inputsfrag_a14Molecular fragment selected for the first fragment positionCategorical
 frag_b13Molecular fragment selected for the second fragment positionCategorical
 frag_c19Molecular fragment selected for the third fragment positionCategorical
Outputspeak_score[0,1][0,1]Photophysical peak score to be maximizedContinuous
Appendix
Table 3: Input and Output variables for Organic Laser Molecules task.
Input/OutputNameRange / CountDescriptionType
Inputsorganic16Organic componentCategorical
 cation3Metal cationCategorical
 anion4Halide anionCategorical
Outputhse_gap[0,+∞)[0,+\infty)HSE06-computed band gapContinuous
Appendix
Table 4: Input and Output variables for Perovskite Bandgap task.
Input/OutputNameRange / CountDescriptionType
Inputsr1_label2Substituent group selected for the R1 positionCategorical
 r3_label8Substituent group selected for the R3 positionCategorical
 r4_label8Substituent group selected for the R4 positionCategorical
 r5_label11Substituent group selected for the R5 positionCategorical
Outputsabs_lam_diff[0,+∞)[0,+\infty)Absolute wavelength difference to be minimizedContinuous
 ered(−∞,+∞)(-\infty,+\infty)Reduction potential to be minimizedContinuous
Appendix
Table 5: Input and Output variables for Redox Flow Battery task.
Input/OutputNameRange / CountDescriptionType
Inputsorganic17Organic ammonium componentCategorical
 organic_molarity[0, 10]Molarity of the organic componentContinuous
 solvent3SolventCategorical
 solvent_molarity[0, 15]Molarity of the solvent componentContinuous
 inorganic_molarity[0, 5]Molarity of the inorganic precursorContinuous
 acid_molarity[0, 25]Molarity of the acid additiveContinuous
Appendix
Table 6: Input and Output variables for Crystallization Process task.
Input/OutputNameRange / CountDescriptionType
Inputsfiller_129First fillerCategorical
 filler_229Second fillerCategorical
 filler_329Third fillerCategorical
 filler_429Fourth fillerCategorical
 filler_529Fifth fillerCategorical
 filler_629Sixth fillerCategorical
Appendix
Table 7: Input and Output variables for the Real-world Formulation Experiment.
Figure 7: Ablation study of TagBO across six datasets. We compare the full TagBO framework with two reduced variants using only LLM-derived scoring features or only physicochemical property features.
Figure 8: Optimization performance of LLM variants for feature extraction on the Suzuki–Miyaura reaction dataset. Left : the optimization trajectories, where curves denote the best objective value found so far, normalized by the oracle optimum. Right : the mean Step@95%, where bar height indicates the average number of iterations required to reach 95% of the optimum, and error bars reflect variation across seeds. ∗T*^{T} denotes Thinking variants.
Figure 9: Optimization performance of repeated Intern-S1-241B Thinking variants for feature extraction on the Suzuki–Miyaura reaction dataset. Left : the optimization trajectories, where curves denote the best objective value found so far, normalized by the oracle optimum. Right : the mean Step@95%, where bar height indicates the average number of iterations required to reach 95% of the optimum, and error bars reflect variation across seeds. ∗T*^{T} denote Thinking variants
More in this topic

Bayesian Optimization

Bayesian optimization uses probabilistic models and sequential experiments to find strong solutions when evaluating a black-box objective is costly.

See all 235 papers

Explore similar work

CardsList
May 21, 2026

LABO: LLM-Accelerated Bayesian Optimization through Broad Exploration and Selective Experimentation

The high cost and data scarcity in scientific exploration have motivated the use of large language models (LLMs) as knowledge-driven components in Bayesian optimization (BO). However, existing approaches typically embed LLMs directly into the sampling or surrogate modeling pipeline, without fully leveraging their significantly lower evaluation cost compared to real-world experiments. To address this limitation, we propose LLM-Accelerated Bayesian Optimization (LABO), a framework that combines LLM predictions with experimental observations within a single BO loop. LABO employs a gating criterion to dynamically balance the reliance on LLM predictions versus actual experiments. By leveraging inexpensive LLM evaluations to broadly explore the search space and reserving costly real experiments only for regions with high uncertainty, LABO achieves more sample-efficient optimization. We provide a theoretical analysis with a cumulative regret bound that formalizes this efficiency gain. Empirical results across diverse scientific tasks demonstrate that LABO consistently outperforms existing methods under identical experimental budgets. Our results suggest that LABO offers a practical and theoretically grounded approach for integrating LLMs into scientific discovery workflows.

Zhuo Chen, Xinzhe Yuan, Jianshu Zhang +8

equal contribution · Shanghai Artificial Intelligence Laboratory, Shanghai, China · School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, China +7

AI for ScienceBayesian OptimizationLarge Language Model-Guided OptimizationLarge Language Model-Guided Scientific Discovery
AskPDF
May 18, 2026

Unleashing LLMs in Bayesian Optimization: Preference-Guided Framework for Scientific Discovery

Scientific discovery is increasingly constrained by costly experiments and limited resources, underscoring the need for efficient optimization in AI for science. Bayesian Optimization (BO), though widely adopted for balancing exploration and exploitation, often exhibits slow cold-start performance and poor scalability in high-dimensional settings, limiting its applicability in real-world scientific problems. To overcome these challenges, we propose LLM-Guided Bayesian Optimization (LGBO), the first LLM preference-guided BO framework that continuously integrates the semantic reasoning of large language models (LLMs) into the optimization loop. Unlike prior works that use LLMs only for warm-start initialization or candidate generation, LGBO introduces a region-lifted preference mechanism that embeds LLM-driven preferences into every iteration, shifting the surrogate mean in a stable and controllable way. Theoretically, we prove that LGBO does not perform significantly worse than standard BO in the worst case, while achieving significantly faster convergence when preferences align with the objective. Empirically, LGBO consistently outperforms existing methods across diverse dry benchmarks in physics, chemistry, biology, and materials science. Most notably, in a new wet-lab optimization of Fe-Cr battery electrolytes, LGBO attains 90% of the best observed value within 6 iterations, whereas standard BO and existing LLM-augmented baselines require more than 10. Together, these results suggest that LGBO offers a promising direction for integrating LLMs into scientific optimization workflows.

Xinzhe Yuan, Zhuo Chen, Jianshu Zhang +4

Shanghai Artificial Intelligence Laboratory, Shanghai, China · Shanghai Jiao Tong University, Shanghai, China · Harbin Institute of Technology, Harbin, China

Bayesian OptimizationLarge Language Model-Guided OptimizationSafe Bayesian OptimizationLarge Language Model-Guided Scientific Discovery
AskPDF
May 8, 2026

Open-Ended Task Discovery via Bayesian Optimization

When applying Bayesian optimization (BO) to scientific workflow, a major yet often overlooked source of uncertainty is the task itself -- namely, what to optimize and how to evaluate it -- which can evolve as evidence accumulates. We introduce Generate-Select-Refine (GSR), a open-ended BO framework that alternates between task generation and task optimization. Starting from a user-provided seed task, GSR generates new tasks in a coarse-to-fine manner while a task-acquisition function schedules optimization. Asymptotically, it concentrates evaluations on the best task, incurring only logarithmic regret overhead relative to single-task BO. We apply GSR to new product development, chemical synthesis scaling, algorithm analysis, and patent repurposing, where it outperforms existing LLM-based optimizers.

Masaki Adachi, Yuta Suzuki, Juliusz Ziomek

Lattice Lab · Toyota Motor Corporation · Machine Learning Research Group +1

Bayesian Optimization
AskPDF