BayMOTH: Bayesian optiMizatiOn with meTa-lookahead -- a simple approacH

Rahman Ejaz; Varchas Gopalaswamy; Ricardo Luna; Aarne Lees; Vineet Gundecha; Christopher Kanan; Soumyendu Sarkar; Riccardo Betti

cs.LGApr 13, 2026

BayMOTH: Bayesian optiMizatiOn with meTa-lookahead -- a simple approacH

Authors: Rahman Ejaz, Varchas Gopalaswamy, Ricardo Luna, Aarne Lees, Vineet Gundecha, Christopher Kanan, Soumyendu Sarkar, Riccardo Betti

Organizations: Laboratory for Laser Energetics, Rochester, NY, USA · Department of Mechanical Engineering, University of Rochester, Rochester, NY, USA · Hewlett Packard Labs, Santa Clara, CA, USA · Department of Computer Science, University of Rochester, Rochester, NY, USA ·

Abstract

Bayesian optimization (BO) has for sequential optimization of expensive black-box functions demonstrated practicality and effectiveness in many real-world settings. Meta-Bayesian optimization (meta-BO) focuses on improving the sample efficiency of BO by making use of information from related tasks. Although meta-BO is sample-efficient when task structure transfers, poor alignment between meta-training and test tasks can cause suboptimal queries to be suggested during online optimization. To this end, we propose a simple meta-BO algorithm that utilizes related-task information when determined useful, falling back to lookahead otherwise, within a unified framework. We demonstrate competitiveness of our method with existing approaches on function optimization tasks, while retaining strong performance in low task-relatedness regimes where test tasks share limited structure with the meta-training set.

Figures & tables

Figure 2: Optimization trajectories across relatedness settings (Sets 1–4) for synthetic ICF experiments. Sample budget is restricted to 10, reflecting the realistic sample budget for ICF.

Figure 3: Simple regret trajectories on HPO tasks. The combined plot includes HPO-B tasks such as glmnet (ID:5860) and rpart.preproc (ID:4796), along with the HPOBench XGBoost results. Curves show performance versus sample steps, with mean trajectories and variability over randomly initialized starting locations.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

\mathcal{S}_{\chi,t}\leftarrow\mathrm{NCC}\!\left(\mu_{t},\tilde{\mu}_{\chi,t}\right)

Appendix

Algorithm 1 BayMOTH

Figure 4: Set 1 (highest NCC in set

\approx

0.85) trajectory and source-task usage across

\gamma

values.

Figure 5: Set 2 (highest NCC in set

\approx

0.60) trajectory and source-task usage across

\gamma

values.

Figure 6: Set 3 (highest NCC in set

\approx

0.82) trajectory and source-task usage across

\gamma

values.

Figure 7: Set 4 (highest NCC in set

\approx

0.10) trajectory and source-task usage across

\gamma

values.

Figure 8: Scan on the number of samples for the Monte Carlo estimator, using set 1 of the synthetic ICF dataset. Shown is $<>$ line and $\pm\sigma$ shaded region over 30 randomly initialized starting locations.

Figure 9: Ablation on the greedy balancing parameter $\alpha$ . Shown is $<>$ line and $\pm\sigma$ shaded region over 30 randomly initialized starting locations.

Method	Cost profile
BayMOTH	Offline source task GP meta-training (cheap $\approx$ 40s); primary computational cost is paid online during proposal generation.
MetaBO	Main cost is offline PPO training; approximately 4 hours for 2000 PPO iterations.
NAP	Main cost is offline PPO training; approximately 11 hours for 2000 PPO iterations.

Appendix

Table 1: High-level computational cost profile of BayMOTH and offline-trained baselines.

BayMOTH stage	Frequency	Share of proposal time	Avg per call	Meaning
Source-task GP training	once	N/A	40.00 s	One-time offline training.
Comparison GP update	once after observation	N/A	0.234 s	Online overhead outside the proposer.
Proposal optimization	once per suggestion	100.00%	44.84 s	Full online proposal cost.
BayMOTH acquisition evaluation	inside L-BFGS-B	98.66%	145.85 ms	Nearly all proposal-time computation is spent here.
Horizon-step simulation	once per acquisition evaluation	97.99%	144.85 ms	Main bottleneck.
Future EI (inside horizon)	$M$ times per acquisition evaluation	72.17%	21.34 ms	Repeated future-EI computation dominates the inner loop.

Appendix

Table 2: Profiling-based breakdown of BayMOTH proposal-time cost on ranger9 (

d=10

Panel of Figure 13 — (a) Set 1: High relatedness

Panel of Figure 14 — (c) Set 3: Moderate relatedness

Figure 12: Source task pool sensitivity studies on test task performance (Set 1 test task). Shown is

<>

line and

\pm\sigma

shaded region over 100 randomly initialized starting locations.

Figure 13: Source task pool sensitivity studies on test task performance (Set 4 test task). Shown is

<>

line and

\pm\sigma

shaded region over 100 randomly initialized starting locations.

BayMOTH
$\Lambda_{0}$ $\mathcal{GP}$ kernel	RBF
$\Lambda_{0}$ $\mathcal{GP}$ kernel length-scale	0.05*4
$\Lambda_{1}$ $\mathcal{GP}$ kernel	RBF
$\Lambda_{1}$ $\mathcal{GP}$ kernel length-scale	0.05
$\mathcal{GP}_{\chi}$ kernel	RBF
$\mathcal{GP}_{\chi}$ kernel length-scale	0.05

Appendix

Table 3: BayMOTH configuration.

2-OPT
$\Lambda_{0}$ $\mathcal{GP}$ kernel	RBF
$\Lambda_{0}$ $\mathcal{GP}$ kernel length-scale	0.05*4
$\Lambda_{1}$ $\mathcal{GP}$ kernel	RBF
$\Lambda_{1}$ $\mathcal{GP}$ kernel length-scale	0.05
Estimator number of samples $M$	5
Input normalization	$[0,1]^{d}$

Appendix

Table 4: 2-OPT configuration.

Bayesian Optimization

Bayesian optimization uses probabilistic models and sequential experiments to find strong solutions when evaluating a black-box objective is costly.

See all 240 papers

Fast Bayesian Optimization Through Linear Models
Oct 21, 2026Colin Doumont, Henry Moss, Philipp HennigBayesian Optimization Surrogate Modeling
University of Tübingen, Tübingen AI Center · Lancaster University
Amortized Acquisition Optimization for Bayesian Optimization with Variational Mutual Information
Jan 13, 2026Farhad MirkarimiBayesian Optimization Black-Box Optimization
Open-Ended Task Discovery via Bayesian Optimization
May 8, 2026Masaki Adachi, Yuta Suzuki, Juliusz ZiomekBayesian Optimization
Lattice Lab · Toyota Motor Corporation · Machine Learning Research Group +1

Oct 21, 2026J/K move · Enter open · S save

Fast Bayesian Optimization Through Linear Models

Colin Doumont, Henry Moss, Philipp Hennig

University of Tübingen, Tübingen AI Center · Lancaster University

Bayesian Optimization Surrogate Modeling

Open paper AskPDF arXiv

Bayesian optimization (BO) is typically used only for problems where the underlying function is deemed "expensive" (e.g. in time or money), with the speed of the BO algorithm being of little importance. In the real world, however, problems span a continuous range of values, going from cheap to expensive. In this paper, we present BO algorithms through the lens of a speed-performance Pareto frontier: we compare BO against faster algorithms like CMA-ES, and highlight how certain regions of the Pareto frontier are currently empty. As a solution, we propose a new set of fast BO methods that exploit the linear structure in existing BO algorithms, for both continuous and combinatorial inputs (from ~10 to 6000+ dimensions). Our methods achieve performance similar to state-of-the-art BO, while being more than

10^2\times

10^3\times

faster, opening the field of BO to an entirely new class of problems.

Abstract

Bayesian Optimization

Explore similar work

Fast Bayesian Optimization Through Linear Models

Amortized Acquisition Optimization for Bayesian Optimization with Variational Mutual Information

Open-Ended Task Discovery via Bayesian Optimization