LLINBO: Trustworthy LLM-in-the-Loop Bayesian Optimization
Organizations: University of Michigan, Ann Arbor
Abstract
Bayesian optimization (BO) is a sequential decision-making tool widely used for optimizing expensive black-box functions. Recently, Large Language Models (LLMs) have shown remarkable adaptability in low-data regimes, making them promising tools for black-box optimization by leveraging contextual knowledge to propose high-quality query points. However, LLMs lack explicit surrogate modeling and calibrated uncertainty, and their internal mechanisms remain opaque. This structural opacity can result in unnecessary experiments, which are costly in BO and may prevent the search from approaching the optimum. To address this, we propose LLINBO: LLM-in-the-Loop BO, a general framework that wraps existing LLM-based BO approaches, deciding when and how to act on their suggestions and when to revert to classical BO. We introduce three mechanisms for this interaction, each accompanied by a no-harm theoretical guarantee that ensures sublinear cumulative regret for common kernels while requiring no performance assumptions on the LLM. Beyond these theoretical guarantees, we show through simulation studies that LLINBO makes existing LLM-based BO more reliable while still exploiting the LLM's few-shot learning ability to outperform classical BO. We further demonstrate its practical use on a real-world, expensive 3D printing experiment. Code is available at https://github.com/UMDataScienceLab/LLM-in-the-Loop-BO.
Figures & tables
| Method | Rastrigin-2 | Robot RF-4 |
|---|---|---|
| BO | ||
| LLAMBO | ||
| LLAMBO-light | ||
| LLINBO-Transient | ||
| LLINBO-Justify | ||
| LLINBO-Constrained |
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
| Feature | LLINBO | HAIC-BO | F-BO | LLM-based BO |
|---|---|---|---|---|
| Minimal assumption on external info | ✓ | ✗ | ✓ | ✓/✗ |
| Theoretical guarantees | ✓ | ✓/✗ | ✓ | ✗ |
| Handles early LLM strength, later weakness | ✓ | ✗ | ✗ | ✓/✗ |
| Preserves BO structure | ✓ | ✓ | ✓ | ✗ |
| Dynamic reliance adjustment | ✓ | ✗ | ✗ | ✗ |
| Method | Fully informed | Partially informed | Minimally informed |
|---|---|---|---|
| LLINBO-Transient | |||
| LLINBO-Justify | |||
| LLINBO-Constrained |
| Simulation functions | Description Card [ Function Patterns ] |
|---|---|
| Levy-2 | highly multimodal but with a unique global maximum. |
| Rastrigin-2 | which is highly multimodal, non-convex function with a large number of regularly spaced local minima. |
| Branin-2 | smooth, multimodal benchmark with three global maxima |
| Bukin-2 | steep, narrow, and highly non-convex landscape with a sharp valley and a unique global maximum |
| Hartmann-4 | 4-dimensional, non-convex, multi-modal and is composed of weighted, anisotropic Gaussian-like bumps centered at different points, making it highly non-separable and challenging to optimize. |
| Ackley-6 | 6-dimensional, non-convex, and multi-modal. The function exhibits a nearly flat outer region and a large hole at the center, resulting in many local optima surrounding a single global optimum. It is highly symmetric and separable in nature, but optimization is still challenging due to the numerous local maxima. |
| Phases | Prompts |
|---|---|
| Warmstarting LLAMBO LLAMBO-light | You are assisting me with maximizing a black-box function. The function is Description Card [ Function Patterns ]. Suggest Description Card [ Dimensionality ] promising starting points in the range . Return the points strictly in JSON format as a list of Description Card [ Dimensionality ]-dimensional vectors. Do not include any explanations, labels, formatting, or extra text. The response must be strictly valid JSON. |
| Candidate sampling LLAMBO | The following are past evaluations of a black-box function. The function is Description Card [ Function Patterns ]. Data Card The allowable ranges for x is [0, 1]^ Description Card [ Dimensionality ]. Recommend a new x that can achieve the function value of Target Score . Return only a single Description Card [ Dimensionality ]-dimensional numerical vector with the highest possible precision. Do not include any explanations, labels, formatting, or extra text. The response must be strictly valid JSON. |
| Surrogate modeling LLAMBO | The following are past evaluations of a black-box function, which is Description Card [ Function Patterns ]. Data Card The allowable ranges for x is [0, 1]^ Description Card [ Dimensionality ]. Predict the function value at x = . Return only a single numerical value. Do not include any explanations, labels, formatting, or extra text. The response must be strictly a valid floating-point number. |
| Candidate generation LLAMBO-light | The following are past evaluations of a black-box function, which is Description Card [ Function Patterns ]. Data Card The allowable ranges for x is [0, 1]^ Description Card [ Dimensionality ]. Based on the past data, recommend the next point to evaluate that balances exploration and exploitation: - Exploration means selecting a point in an unexplored or less-sampled region that is far from the previously evaluated points. - Exploitation means selecting a point close to the previously high-performing evaluations. The goal is to eventually find the global maximum. Return only a single Description Card [ Dimensionality ]-dimensional numerical vector with high precision. The response must be valid JSON with no explanations, labels, or extra formatting. Do not include any explanations, labels, formatting, or extra text. |
| Phases | Prompts |
|---|---|
| Warmstarting LLAMBO LLAMBO-light | You are assisting with automated hyperparameter tuning using Description Card [ Model Patterns ] for a regression task. Description Card [ Data Patterns ] Model performance is evaluated using mean square error (MSE). The dataset contains samples and continuous features. The following hyperparameters are tunable: Description Card [ Controllable Hyperparameters ]. Please suggest Description Card [ Dimensionality ] diverse yet effective configurations to initiate a Bayesian Optimization process for hyperparameter tuning. Format your response strictly as a JSON array of Description Card [ Dimensionality ]-dimensional numerical vectors (lists). Do not include explanations, comments, or any extra text outside the JSON. |
| Candidate sampling LLAMBO | The following are examples of the performance of a Description Card [ Model Patterns ] measured in mean square error and the corresponding model hyperparameter configurations. Data Card The model is evaluated on a regression task. Description Card [ Data Patterns ] The allowable ranges for the hyperparameters are: Description Card [ Controllable Hyperparameters ]. Recommend a configuration that can achieve the target mean square error of Target Score , and each dimension must be strictly within the allowable range specified above. Return only a single Description Card [ Dimensionality ]-dimensional numerical vector with the highest possible precision. The response needs to be a list and must be strictly valid JSON. Do not include any explanations, labels, formatting, or extra text like json . |
| Surrogate modeling LLAMBO | The following are examples of the performance of a Description Card [ Model Patterns ] measured in mean square error and the corresponding model hyperparameter configurations. Data Card The model is evaluated on a regression task. Description Card [ Data Patterns ] The dataset contains samples and features and all of the features are continuous. Predict the mean square error when the model hyperparameter configuration is set to be . Do not include any explanations, labels, formatting, or extra text. The response must be strictly a valid floating-point number. |
| Phases | Prompts |
| Candidate generation LLAMBO-light | The following are examples of the performance of a Description Card [ Model Patterns ] measured in mean square error and the corresponding model hyperparameter configurations. Data Card The model is evaluated on a regression task. Description Card [ Data Patterns ] The dataset contains samples and features and all of the features are continuous. The allowable ranges for the hyperparameters are: Description Card [ Controllable Hyperparameters ]. Your goal is to recommend the next setting to evaluate that balances exploration and exploitation: - Exploration favors regions that are less-sampled or farther from existing evaluations. - Exploitation favors regions near previously low mean square error. To encourage exploration, avoid suggesting values too close to past evaluations. You are on iteration out of . The ultimate objective is to find the global minimum prediction mean square error. The ideal prediction mean square error is 0. Return only a single Description Card [ Dimensionality ]-dimensional numerical vector with the highest possible precision. Do not include any explanations, labels, formatting, or extra text like json . The response must be strictly valid JSON. |
| Phases | Prompts |
|---|---|
| Warmstarting LLAMBO LLAMBO-light | You are assisting with process planning for 3D printing a simple part using Overture PETG filament on an Ender 3 Pro in a room-temperature environment (around 22°C). The objective is to reduce stringing as much as possible, using knowledge of PETG printing behavior. Parameter Description Card After each print, stringing is measured via an image-based algorithm, returning a percentage between 0 and 100%. You must now propose 2 promising combinations of Nozzle Temperature and Z Hop that are likely to minimize stringing, based on your understanding of PETG behavior (e.g., lower nozzle temps reduce oozing, moderate Z hop avoids drag). Format your answer strictly as a valid JSON list of 5-dimensional vectors. Each vector should be: [Nozzle Temperature (°C), Z Hop Height (mm), Coasting Volume (mm³), Retraction Distance (mm), Outer Wall Wipe Distance (mm)]. Do not include any explanations, labels, formatting, or extra text. |
| Candidate sampling LLAMBO | The following are past evaluations of the stringing percentage and their corresponding Nozzle Temperature and Z Hop values: Data Card Parameter Description Card Recommend a new [Nozzle Temperature (°C), Z Hop Height (mm), Coasting Volume (mm³), Retraction Distance (mm), Outer Wall Wipe Distance (mm)] that can achieve the stringing percentage of Target Score . Instructions: Return only one 5D vector: ‘[Nozzle Temperature (°C), Z Hop Height (mm), Coasting Volume (mm³), Retraction Distance (mm), Outer Wall Wipe Distance (mm)]‘. Ensure the values respect the allowed ranges and increments. Respond with strictly valid JSON format. Do not include any explanations, comments, or extra text. |
| Surrogate modeling LLAMBO | The following are past evaluations of the stringing percentage and the corresponding Nozzle Temperature and Z Hop. Data Card Parameter Description Card Predict the stringing percentage at [Nozzle Temperature, Z Hop Height, Coasting Volume, Retraction Distance, Outer Wall Wipe Distance] = . The stringing percentage needs to be a single value between 0 and 100. Return only a single numerical value. Do not include any explanations, labels, formatting, percentage symbol, or extra text. |
| Phases | Prompts |
| Candidate generation LLAMBO-light | The following are past evaluations of the stringing percentage and their corresponding Nozzle Temperature and Z Hop values: Data Card Parameter Description Card Your goal is to recommend the next setting to evaluate that balances exploration and exploitation: Exploration favors regions that are less-sampled or farther from existing evaluations. Exploitation favors regions near previously low stringing percentages. You are on iteration out of 10. The ultimate objective is to find the global minimum stringing percentage. The ideal stringing percentage is 0%. Instructions: Return only one 5-dimensional vector: [Nozzle Temperature (°C), Z Hop Height (mm), Coasting Volume (mm³), Retraction Distance (mm), Outer Wall Wipe Distance (mm)]. Ensure the values respect the allowed ranges and increments. Respond with strictly valid JSON format. Do not include any explanations and comments. |
Bayesian Optimization
Bayesian optimization uses probabilistic models and sequential experiments to find strong solutions when evaluating a black-box objective is costly.
See all 268 papers