TagBO: LLM-Driven Task-Aware Graph Bayesian Optimization for Scientific Discovery

Xinzhe Yuan; Zimu Zeng; Zhuo Chen; Jianshu Zhang; Jinzong Dong; Jingyi Chen; Nanyang Ye; Huan Xiong

cs.LGOct 10, 2026

TagBO: LLM-Driven Task-Aware Graph Bayesian Optimization for Scientific Discovery

Authors: Xinzhe Yuan, Zimu Zeng, Zhuo Chen, Jianshu Zhang, Jinzong Dong, Jingyi Chen, Nanyang Ye, Huan Xiong,

Organizations: Harbin Institute of Technology · Shanghai Artificial Intelligence Laboratory · Fudan University · Shenzhen Loop Area Institute ·

Abstract

Mixed-variable optimization in scientific discovery often requires exploring large combinatorial spaces under costly experimental evaluations, making sample?efficient Bayesian optimization (BO) essential. However, existing BO methods typ?ically rely on structural descriptors to define similarity among categorical variables, implicitly assuming that structural similarity reflects similarity in task performance. In scientific optimization, this assumption is often violated, leading to surrogate representations that are poorly aligned with the optimization objective. To address this limitation, we propose TagBO (Task-aware Graph Bayesian Optimization), a method that integrates LLM-derived relational priors with descriptor-based struc?tural information for mixed-variable Bayesian optimization. Rather than directly using LLM outputs as predictive scores, TagBO treats LLM outputs as a potentially noisy source of task-aware relational priors and extracts coarse ordinal relational signals from LLM reasoning to construct task-aware similarity structures for scien?tific optimization. TagBO then performs constrained Gaussian process Bayesian optimization over the resulting joint discrete-continuous representation space. Ex?periments on diverse chemistry and materials formulation tasks demonstrate that TagBO consistently improves sample efficiency under fixed evaluation budgets across LLMs of varying capabilities.

Figures & tables

Figure 1: Overview of TagBO on Suzuki coupling reaction optimization. For each categorical variable, TagBO constructs an LLM-derived task-aware relational graph from semantic rankings under the target task, alongside a structural graph derived from physicochemical descriptors. The two signals are fused to form a task-aware optimization space, which is combined with continuous variables and used for Bayesian optimization.

Figure 2: Optimization trajectories on six tasks. The y-axis reports the best objective value found so far, normalized by the oracle optimum, and the x-axis reports the iteration number. TagBO is compared against One-hot Feature BO, Property Feature BO, COMBO, BOPRO, and LGBO.

Figure 3: Comparison of the average number of iterations required to reach 95% of the oracle optimum on six benchmark tasks. Lower is better.

Figure 4: Ablation analysis on the Suzuki dataset. (A) Comparison with two reduced variants using only LLM-derived scoring features or only physicochemical property features. (B) Optimization trajectories by different LLMs, where curves report the best objective value found so far, normalized by the oracle optimum. (C) Mean Step@95% under different LLMs. $*^{T}$ denote Thinking variants.

Figure 5: 3D UMAP visualization of feature spaces. Each point represents a reaction candidate; the horizontal axes show the UMAP embedding, and the vertical axis encodes the reaction yield.

Figure 6: Real-world formulation optimization. (A) Thermal conductivity and the best value found so far over 34 physical evaluations. (B) Final measurements relative to their qualification thresholds; values above one satisfy the requirements.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Input/Output	Name	Range / Count	Description	Type
Inputs	DB_block	22	Donor-bridge molecular building block	Categorical
	A_block	100	Acceptor molecular building block	Categorical
Output	PredictedSO	$[0,+\infty)$	DFT-predicted spectral overlap score	Continuous

Appendix

Table 1: Input and Output variables for Organic Photovoltaics task.

Input/Output	Name	Range / Count	Description	Type
Inputs	ligand	12	Ligand selected from 12 categorical options	Categorical
	base	8	Base selected from 8 categorical options	Categorical
	solvent	4	Solvent selected from 4 categorical options	Categorical
Output	yield	$[0,100]$	Reaction yield to be maximized	Continuous

Appendix

Table 2: Input and Output variables for Suzuki–Miyaura Reaction task.

Input/Output	Name	Range / Count	Description	Type
Inputs	frag_a	14	Molecular fragment selected for the first fragment position	Categorical
	frag_b	13	Molecular fragment selected for the second fragment position	Categorical
	frag_c	19	Molecular fragment selected for the third fragment position	Categorical
Outputs	peak_score	$[0,1]$	Photophysical peak score to be maximized	Continuous

Appendix

Table 3: Input and Output variables for Organic Laser Molecules task.

Input/Output	Name	Range / Count	Description	Type
Inputs	organic	16	Organic component	Categorical
	cation	3	Metal cation	Categorical
	anion	4	Halide anion	Categorical
Output	hse_gap	$[0,+\infty)$	HSE06-computed band gap	Continuous

Appendix

Table 4: Input and Output variables for Perovskite Bandgap task.

Input/Output	Name	Range / Count	Description	Type
Inputs	r1_label	2	Substituent group selected for the R1 position	Categorical
	r3_label	8	Substituent group selected for the R3 position	Categorical
	r4_label	8	Substituent group selected for the R4 position	Categorical
	r5_label	11	Substituent group selected for the R5 position	Categorical
Outputs	abs_lam_diff	$[0,+\infty)$	Absolute wavelength difference to be minimized	Continuous
	ered	$(-\infty,+\infty)$	Reduction potential to be minimized	Continuous

Appendix

Table 5: Input and Output variables for Redox Flow Battery task.

Input/Output	Name	Range / Count	Description	Type
Inputs	organic	17	Organic ammonium component	Categorical
	organic_molarity	[0, 10]	Molarity of the organic component	Continuous
	solvent	3	Solvent	Categorical
	solvent_molarity	[0, 15]	Molarity of the solvent component	Continuous
	inorganic_molarity	[0, 5]	Molarity of the inorganic precursor	Continuous
	acid_molarity	[0, 25]	Molarity of the acid additive	Continuous

Appendix

Table 6: Input and Output variables for Crystallization Process task.

Input/Output	Name	Range / Count	Description	Type
Inputs	filler_1	29	First filler	Categorical
	filler_2	29	Second filler	Categorical
	filler_3	29	Third filler	Categorical
	filler_4	29	Fourth filler	Categorical
	filler_5	29	Fifth filler	Categorical
	filler_6	29	Sixth filler	Categorical

Appendix

Table 7: Input and Output variables for the Real-world Formulation Experiment.

Figure 7: Ablation study of TagBO across six datasets. We compare the full TagBO framework with two reduced variants using only LLM-derived scoring features or only physicochemical property features.

Figure 8: Optimization performance of LLM variants for feature extraction on the Suzuki–Miyaura reaction dataset. Left : the optimization trajectories, where curves denote the best objective value found so far, normalized by the oracle optimum. Right : the mean Step@95%, where bar height indicates the average number of iterations required to reach 95% of the optimum, and error bars reflect variation across seeds. $*^{T}$ denotes Thinking variants.

Figure 9: Optimization performance of repeated Intern-S1-241B Thinking variants for feature extraction on the Suzuki–Miyaura reaction dataset. Left : the optimization trajectories, where curves denote the best objective value found so far, normalized by the oracle optimum. Right : the mean Step@95%, where bar height indicates the average number of iterations required to reach 95% of the optimum, and error bars reflect variation across seeds. $*^{T}$ denote Thinking variants

Bayesian Optimization

Bayesian optimization uses probabilistic models and sequential experiments to find strong solutions when evaluating a black-box objective is costly.

See all 235 papers

LABO: LLM-Accelerated Bayesian Optimization through Broad Exploration and Selective Experimentation
May 21, 2026Zhuo Chen, Xinzhe Yuan, Jianshu Zhang +8AI for Science Bayesian Optimization
equal contribution · Shanghai Artificial Intelligence Laboratory, Shanghai, China · School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, China +7
Unleashing LLMs in Bayesian Optimization: Preference-Guided Framework for Scientific Discovery
May 18, 2026Xinzhe Yuan, Zhuo Chen, Jianshu Zhang +4Bayesian Optimization Large Language Model-Guided Optimization
Shanghai Artificial Intelligence Laboratory, Shanghai, China · Shanghai Jiao Tong University, Shanghai, China · Harbin Institute of Technology, Harbin, China
Open-Ended Task Discovery via Bayesian Optimization
May 8, 2026Masaki Adachi, Yuta Suzuki, Juliusz ZiomekBayesian Optimization
Lattice Lab · Toyota Motor Corporation · Machine Learning Research Group +1

May 21, 2026J/K move · Enter open · S save

LABO: LLM-Accelerated Bayesian Optimization through Broad Exploration and Selective Experimentation

Zhuo Chen, Xinzhe Yuan, Jianshu Zhang, Jinzong Dong+7

equal contribution · Shanghai Artificial Intelligence Laboratory, Shanghai, China · School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, China+7

AI for Science Bayesian Optimization Large Language Model-Guided Optimization Large Language Model-Guided Scientific Discovery

Open paper AskPDF arXiv

The high cost and data scarcity in scientific exploration have motivated the use of large language models (LLMs) as knowledge-driven components in Bayesian optimization (BO). However, existing approaches typically embed LLMs directly into the sampling or surrogate modeling pipeline, without fully leveraging their significantly lower evaluation cost compared to real-world experiments. To address this limitation, we propose LLM-Accelerated Bayesian Optimization (LABO), a framework that combines LLM predictions with experimental observations within a single BO loop. LABO employs a gating criterion to dynamically balance the reliance on LLM predictions versus actual experiments. By leveraging inexpensive LLM evaluations to broadly explore the search space and reserving costly real experiments only for regions with high uncertainty, LABO achieves more sample-efficient optimization. We provide a theoretical analysis with a cumulative regret bound that formalizes this efficiency gain. Empirical results across diverse scientific tasks demonstrate that LABO consistently outperforms existing methods under identical experimental budgets. Our results suggest that LABO offers a practical and theoretically grounded approach for integrating LLMs into scientific discovery workflows.

Abstract

Bayesian Optimization

Explore similar work

LABO: LLM-Accelerated Bayesian Optimization through Broad Exploration and Selective Experimentation

Unleashing LLMs in Bayesian Optimization: Preference-Guided Framework for Scientific Discovery

Open-Ended Task Discovery via Bayesian Optimization