MF-SCBO : Multi-fidelity Scalable Constrained Bayesian Optimization

Lucas Palazzolo; Mickaël Binois; Laëtitia Giraldi

cs.LGSep 24, 2026

MF-SCBO : Multi-fidelity Scalable Constrained Bayesian Optimization

Authors: Lucas Palazzolo, Mickaël Binois, Laëtitia Giraldi

Organizations: Université Côte d’Azur, Inria, Calisto team, Sophia Antipolis, France · Université Côte d’Azur, Inria, Acumes team, CNRS, LJAD, Sophia Antipolis, France

Abstract

Many real-world optimization problems rely on expensive simulations or experiments, making the efficient use of available data essential. Multi-fidelity optimization of high-dimensional black-box functions subject to black-box constraints is increasingly relevant as the cost of objective evaluations continues to rise in applications such as machine learning, engineering, and control. To our knowledge, no existing method simultaneously addresses high-dimensionality, black-box constraints, an arbitrary number of fidelity levels, and non-nested sampling. In this work, we extend the Scalable Constrained Bayesian Optimization method to the multi-fidelity setting, resulting in the MF-SCBO method. The proposed approach is evaluated on standard benchmark functions as well as challenging problems. The experimental results demonstrate that MF-SCBO generally achieves better convergence than both the single-fidelity SCBO and the other multi-fidelity method considered in this high-dimensional and constrained settings.

Figures & tables

\boldsymbol{x_{\text{next}}}\in\argmax_{\boldsymbol{x}\in\Omega}a(\boldsymbol{x})

Algorithm 1 Bayesian Optimization

Figure 1: Multi-fidelity Gaussian process regression on a toy function. Multi-fidelity GP (red) constructed from a small number of high-fidelity data points and a larger set of low-fidelity data points (red). The nested case is considered, where the high-fidelity samples are a subset of the low-fidelity points. The standard GP built using only high-fidelity data is displayed in blue. The

95\%

confidence intervals predicted by the GPs are shown as shaded regions.

Figure 2: MF-SCBO representation for two fidelity levels (green for the low fidelity and blue for the high fidelity): (1) Initial points are taken from

\Omega

(represented by crosses for the low fidelity and circle for the high fidelity), and the trust region is defined (grid with central black circle). (2) The different Gaussian process models are fitted. (3a)

r

random points (represented by triangles) are sampled in the trust region. (3b) A batch of

q

realizations is computed and selected via Thompson sampling on these

r

designs. For each realization, the

x

axis corresponds to the

\boldsymbol{x}

points in

\Omega

and is discretized by the

\{\boldsymbol{x}_{\text{discr},i}\}_{i=1}^{r}

(the triangles) chosen previously. The

y

axis shows the realizations. For simplicity, a single graph is shown for the objective and the constraints. For each realization, we keep the best point (completely filled triangle). (3c) For the

q

best points

\{\boldsymbol{x}_{\text{next},i}\}_{i=1}^{q}

, the fidelity level is choose by computing ( 16 ). (4) The real value of the objectives and the constraints are computed and added to the previous observations. (5) The trust region is readjusted by modifying its center and radius.

Figure 3: Convergence curves for the synthetic experiments: Hartmann6D, Borehole8D, Ackley20D, Rosenbrock40D, and Rosenbrock100D. Blue corresponds to the MF-SCBO- predicted method, green to MF-SCBO- evaluated , black to SCBO, and red to MFMFES. The first row shows the best value as a function of high-fidelity evaluations, the second row shows the best value versus total cost, and the third row displays the number of points used at each fidelity level $s$ for each iteration over the total cost.

Instance		Hart.6D	Borehole8D	Ackley20D	Rosenb.40D	Rosenb.100D
$\begin{array}[]{c}\text{Parameters}\end{array}$	$\begin{array}[]{c}\text{dim}\\ S+1\\ \eta\\ \text{batch size}\\ \#\mathcal{X}_{0}\end{array}$	$\begin{array}[]{c}6\\ 3\\ (1,10,20)\\ 8\\ (12,8,4)\end{array}$	$\begin{array}[]{c}8\\ 2\\ (1,20)\\ 8\\ (26,6)\end{array}$	$\begin{array}[]{c}20\\ 2\\ (1,20)\\ 8\\ (72,8)\end{array}$	$\begin{array}[]{c}40\\ 3\\ (1,10,20)\\ 8\\ (104,40,16)\end{array}$	$\begin{array}[]{c}100\\ 3\\ (1,10,20)\\ 8\\ (200,40,20)\end{array}$
$\begin{array}[]{c}\text{Results}\end{array}$	$\begin{array}[]{c}\text{MFSCBO-{eval.}}\\ \text{MFSCBO-{pred.}}\\ \text{SCBO}\\ \text{MFMEF}\end{array}$	$\begin{array}[]{c}\mathbf{-3.25}(\pm 0.05)\\ -3.23(\pm 0.06)\\ -3.18(\pm 0.13)\\ -3.20(\pm 0.11)\end{array}$	$\begin{array}[]{c}7.94(\pm 0.03)\\ \mathbf{8.07}(\pm 0.11)\\ 7.95(\pm 0.03)\\ 7.83(\pm 0.01)\end{array}$	$\begin{array}[]{c}2.62(\pm 0.20)\\ \mathbf{0.50}(\pm 0.21)\\ 9.40(\pm 1.07)\\ -\end{array}$	$\begin{array}[]{c}4.81(\pm 0.32)\!\times\!10^{3}\\ 5.97(\pm 0.62)\!\times\!10^{3}\\ \mathbf{2.52}(\pm 0.21)\!\times\!10^{3}\\ -\end{array}$	$\begin{array}[]{c}2.11(\pm 0.06)\!\times\!10^{4}\\ 2.39(\pm 0.14)\!\times\!10^{4}\\ \mathbf{1.75}(\pm 0.07)\!\times\!10^{4}\\ -\end{array}$

Table 1: Parameters and optimization results for the synthetic functions: Hartmann6D, Borehole8D, Ackley20D, Rosenbrock40D, and Rosenbrock100. The table lists the problem dimension, number of fidelities, associated cost, batch size, and number of initial points for each fidelity. The mean of the best value after

4000

objective function evaluations is reported for each method, along with the

1.96\times

standard deviation. The best value across methods is highlighted in bold.

Figure 4: Convergence curves for the realistic experiments: Solar2, Solar7 and Airfoil. Blue corresponds to the MF-SCBO- predicted method, green to MF-SCBO- evaluated , black to SCBO, and red to MFMFES. The first row shows the best value as a function of high-fidelity evaluations, the second row shows the best value versus total cost, and the third row displays the number of points used at each fidelity level $s$ for each iteration over the total cost.

Instance		Solar2	Solar7	Airfoil
$\begin{array}[]{c}\text{Parameters}\end{array}$	$\begin{array}[]{c}\text{dim}\\ S+1\\ \eta\\ \text{batch size}\\ \#\mathcal{X}_{0}\end{array}$	$\begin{array}[]{c}12\\ 3\\ (1,50)\\ 8\\ (42,6)\end{array}$	$\begin{array}[]{c}6\\ 3\\ (1,50)\\ 8\\ (20,4)\end{array}$	$\begin{array}[]{c}10\\ 2\\ (1,20)\\ 8\\ (36,4)\end{array}$
$\begin{array}[]{c}\text{Results}\end{array}$	$\begin{array}[]{c}\text{MFSCBO-{eval.}}\\ \text{MFSCBO-{pred.}}\\ \text{SCBO}\\ \text{MFMEF}\end{array}$	$\begin{array}[]{c}\mathbf{-3.46}(\pm 0.82)\!\times\!10^{6}\\ -2.34(\pm 1.89)\!\times\!10^{6}\\ -2.59(\pm 1.97)\!\times\!10^{6}\\ -\end{array}$	$\begin{array}[]{c}\mathbf{-4.86}(\pm 0.023)\!\times\!10^{3}\\ -3.77(\pm 0.944)\!\times\!10^{3}\\ -4.80(\pm 0.178)\!\times\!10^{3}\\ 2.77(\pm 1.98)\!\times\!10^{9}\end{array}$	$\begin{array}[]{c}\mathbf{-1.84}(\pm 0.03)\!\times\!10^{2}\\ -1.75(\pm 0.06)\!\times\!10^{2}\\ -1.81(\pm 0.05)\!\times\!10^{2}\\ -\end{array}$

Table 2: Parameters and optimization results for the realistic functions: Solar2, Solar7 and Airfoil. The table lists the problem dimension, number of fidelities, associated cost, batch size, and number of initial points for each fidelity. The mean of the best value after

4000

objective function evaluations is reported for each method, along with the

1.96\times

standard deviation. The best value across methods is highlighted in bold.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Two dimensional Ackley function at two different fidelity levels, $0$ for the low fidelity function and $1$ for the high fidelity function.

Figure 6: Two dimensional Rosenbrok function at three different fidelity levels, from $0$ (lowest) to $2$ (highest).

Instance	# of variables			# of constraints		# of stoch. outpus
	cont.	discr.	fixed	simu.	a priori	(obj. or constr.)
solar2	$12$	$2$	$(x_{6},\;x_{11})=(2650,\;36)$	$7$	$5$	$4$
solar7	$6$	$1$	$x_{4}=40$	$4$	$2$	$3$

Appendix

Table 3: Characteristics of the solar2 and solar7 instances, including the number of variables (continuous and discrete), the fixed values of the discrete variables, the number of constraints (simulated and a priori), and the number of stochastic outputs.

Figure 7: Illustration of the PARSEC airfoil parameterization.

Coefficient	Bound	Description
$x_{1}$	$[0.005,0.06]$	Leading-edge radius
$x_{2}$	$[0.25,0.5]$	Upper crest position in horizontal coordinates
$x_{3}$	$[0.05,0.15]$	Upper crest position in vertical coordinates
$x_{4}$	$[-1,-0.4]$	Upper crest curvature
$x_{5}$	$[0.35,0.5]$	Lower crest position in horizontal coordinates
$x_{6}$	$[-0.12,-0.04]$	Lower crest position in vertical coordinates

Appendix

Table 4: Description of the shape variables for the PARSEC parameterization of airfoils. The trailing-edge thickness,

x_{9}

, is not optimized and is fixed to

0

Index	Reynolds	Mach	Angle of attack (deg.)	Weights $\omega_{i}$
$1$	$6.3\times 10^{6}$	$0.5$	$2$	$0.4$
$2$	$6.3\times 10^{6}$	$0.55$	$2.2$	$0.2$
$3$	$6.3\times 10^{6}$	$0.5$	$2.5$	$0.2$
$4$	$6.3\times 10^{6}$	$0.55$	$2$	$0.2$

Appendix

Table 5: Operating conditions used in the multi-point design strategy, together with the associated weights

\omega_{i}

Figure 8: Convergence curves for different number of fidelity level for Hartmann6D and Rosenbrock100D. Green, blue and red correspond respectively to

2

3

4

number of fidelity levels. Circles correspond to the evaluated method and squares to the predicted method.

Figure 9: Convergence curves for different batch size for Hartmann6D, Borehole8D, Ackley20D and Rosenbrock100D. Green, blue and red correspond respectively to $4$ , $8$ et $16$ batch sizes. Circles correspond to the evaluated method and squares to the predicted method.

Figure 10: Convergence curves for different costs for Hartmann6D, Borehole8D, Ackley20D and Rosenbrock100D. Green and blue correspond respectively to $(1,20)$ and $(1,50)$ costs for two fidelities and $(1,10,20)$ , $(1,20,50)$ for three fidelities. Circles correspond to the evaluated method and squares to the predicted method.

Explore similar work

Out-Of-The-Loop Multi-Fidelity Bayesian Optimization
Aug 4, 2026Gustavo Sutter, Hao Wang, Luis Ricardez-Sandoval +2Multi-Fidelity Bayesian Optimization
Cheriton School of Computer Science, University of Waterloo, Waterloo, ON, Canada · Vector Institute, Toronto, ON, Canada · Department of Chemical Engineering, University of Waterloo, Waterloo, ON, Canada +2
Reduced-Space Multi-Fidelity Bayesian Optimization of Process Simulation Models
Sep 15, 2026Niki Triantafyllou, Andrea Bernardi, Maria M. PapathanasiouMulti-Fidelity Multi-Objective Bayesian Optimization Algorithms
Sargent Centre for Process Systems Engineering, Department of Chemical Engineering, Imperial College London, UK
Local Constrained Bayesian Optimization
Mar 9, 2026Jing Jingzhe, Fan Zheyi, Szu Hui Ng +1Bayesian Optimization Trust Region
State Key Laboratory of Mathematical Sciences, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing, China · School of Mathematical Sciences, University of Chinese Academy of Sciences, Beijing, China · Department of Industrial Systems Engineering & Management, National University of Singapore, Singapore.

Aug 4, 2026, cs.LGJ/K move · Enter open · S save

Out-Of-The-Loop Multi-Fidelity Bayesian Optimization

Gustavo Sutter, Hao Wang, Luis Ricardez-Sandoval, Pascal Poupart+1

Cheriton School of Computer Science, University of Waterloo, Waterloo, ON, Canada · Vector Institute, Toronto, ON, Canada · Department of Chemical Engineering, University of Waterloo, Waterloo, ON, Canada+2

Multi-Fidelity Bayesian Optimization Molecular Optimization

Open paper AskPDF arXiv

Black-box optimization is a ubiquitous problem in science and engineering, often dealing with expensive objective functions with cheaper lower-fidelity proxies available. Multi-fidelity Bayesian optimization (MF-BO) is a principled approach to this problem, leveraging correlations across different fidelities when querying the objective. However, for many important MF-BO tasks, the true highest-fidelity function is prohibitively expensive to be part of the optimization loop. Nevertheless, practitioners often have gold standard data (observations of the highest-fidelity function) obtained from previous experiments that might provide information for the current task. For instance, in molecular optimization, chemists often pick the top-

k

candidate molecules using various computer simulations, and later reveal their true objective function values. In this work, we demonstrate the suboptimality of standard MF-BO algorithms in the real-world scenarios above, even under ideal assumptions. Next, we mitigate this problem by incorporating historical high-fidelity data accompanied by task descriptors---which can be explicitly given or extracted from unstructured metadata. We demonstrate the effectiveness of our methods on synthetic functions, as well as real-world problems in chemistry and hyperparameter optimization.

Abstract

Explore similar work

Out-Of-The-Loop Multi-Fidelity Bayesian Optimization

Reduced-Space Multi-Fidelity Bayesian Optimization of Process Simulation Models

Local Constrained Bayesian Optimization