Converge Then Diversify: Decoupling Convergence and Diversity in Multi-Objective Bayesian Optimisation

Chao Jiang; Yueling Huang; Miqing Li

cs.AISep 11, 2026

Converge Then Diversify: Decoupling Convergence and Diversity in Multi-Objective Bayesian Optimisation

Authors: Chao Jiang, Yueling Huang, Miqing Li

Organizations: School of Computer Science University of Birmingham

Abstract

Multi-objective Bayesian optimisation (MOBO) is a sample-efficient approach for optimising expensive black-box functions with multiple objectives. In MOBO, the goal is to adequately approximate the Pareto front; that is, to obtain a high-quality solution set with 1) good convergence (closeness to the Pareto front) and 2) good diversity (spread across the Pareto front). Existing MOBO methods typically aim to accomplish these two tasks simultaneously, i.e., driving the search towards the Pareto front while maintaining a diverse set of nondominated solutions, such that the solutions, ideally, can gradually approach the entire front. When sufficient search budgets are available, this approach is effective. However, considering both convergence and diversity throughout the search is not easy and requires careful design. Under very tight budgets, there may not be enough solutions generated to be able to simultaneously approach the entire Pareto front. To address this issue, this paper proposes a converge-then-diversify (CTD) approach that decouples convergence and diversity into two stages. In the first stage, CTD focuses on convergence, aiming to quickly drive the search toward a single point on the Pareto front. In the second stage, CTD focuses on diversity, aiming to spread solutions across the front. We present two simple instantiations of CTD by using widely adopted acquisition functions in the area. Experimental results show that, across all 446 pairwise comparisons, CTD statistically outperforms state-of-the-art methods in 72.9% of the cases, performs equivalently in 21.1%, and is statistically worse in only 6.1%, with the advantage being particularly evident in settings with very tight evaluation budgets or in high-dimensional problems.

Figures & tables

Panel of Figure 1 — Figure 1: Illustration of the search processes of the conventional MOBO approach and the proposed approach. (a) The conventional approach that moves towards the Pareto front while simultaneously spreading nondominated solutions. (b) The proposed convergence-then-diversity strategy that first moves directly towards the Pareto front and then spreads along it.

Method	DTLZ1 ( $m=2$ )		DTLZ2 ( $m=2$ )		DTLZ3 ( $m=2$ )		DTLZ4 ( $m=2$ )		DTLZ5 ( $m=2$ )		DTLZ6 ( $m=2$ )		DTLZ7 ( $m=2$ )
	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std
Sobol	0.00e+00	0.00e+00 ( $\sim$ )	1.23e-02	1.82e-02 ( $+$ )	0.00e+00	0.00e+00 ( $\sim$ )	1.34e-02	2.21e-02 ( $+$ )	1.20e-02	1.33e-02 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )
ParEGO	0.00e+00	0.00e+00 ( $\sim$ )	2.97e-01	7.15e-02 ( $+$ )	1.94e+02	5.61e+02 ( $\sim$ )	2.84e-01	6.52e-02 ( $+$ )	2.39e-01	1.06e-01 ( $+$ )	2.67e-01	1.34e-01 ( $\sim$ )	3.86e-01	2.38e-01 ( $-$ )
TS-TCH	0.00e+00	0.00e+00 ( $\sim$ )	3.68e-02	3.99e-02 ( $+$ )	0.00e+00	0.00e+00 ( $\sim$ )	3.47e-02	3.43e-02 ( $+$ )	4.32e-02	4.18e-02 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )
PSL	7.92e+00	2.82e+01 ( $\sim$ )	8.61e-02	5.44e-02 ( $+$ )	2.66e+02	7.12e+02 ( $\sim$ )	9.88e-02	3.64e-02 ( $+$ )	8.61e-02	5.44e-02 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	4.64e-01	1.71e-01 ( $-$ )
JES	0.00e+00	0.00e+00 ( $\sim$ )	2.43e-01	6.00e-02 ( $+$ )	5.37e+02	6.34e+02 ( $\sim$ )	2.65e-01	5.99e-02 ( $+$ )	2.74e-01	4.40e-02 ( $+$ )	2.22e-01	7.26e-02 ( $+$ )	3.72e-01	1.86e-01 ( $-$ )

Table 1: The HV results (mean and standard deviation) of the ten methods on the 20 benchmark and real-world problems. The method with the best mean HV is highlighted in bold. The symbols “

-

”, “

\sim

” and “

+

” indicate that our method CTD is statistically worse than, equivalent to, and better than the competitor, respectively.

Figure 2: Mean hypervolume trajectories of the ten methods on the representative DTLZ2 problem (

m=2

d=11

) over 30 independent runs. The shaded regions indicate one standard deviation around the mean.

Panel of Figure 3 — Figure 3: Solutions obtained by the proposed CTD and the nine peer methods on DTLZ2 ( $m=2$ , $d=11$ ) under budgets of 40 and 100 function evaluations. All methods start from the same $2(d+1)$ initial points generated by a scrambled Sobol sequence. The black line denotes the Pareto front, and brighter colours indicate solutions obtained at later evaluations.

Method	DTLZ1 ( $m=2$ )		DTLZ2 ( $m=2$ )		DTLZ3 ( $m=2$ )		DTLZ4 ( $m=2$ )		DTLZ5 ( $m=2$ )		DTLZ6 ( $m=2$ )		DTLZ7 ( $m=2$ )
	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std
Sobol	0.00e+00	0.00e+00 ( $\sim$ )	1.54e-02	1.39e-02 ( $+$ )	0.00e+00	0.00e+00 ( $\sim$ )	1.37e-02	1.66e-02 ( $+$ )	1.17e-02	1.61e-02 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $\sim$ )
NParEGO	0.00e+00	0.00e+00 ( $\sim$ )	6.94e-02	4.58e-02 ( $\sim$ )	3.98e+02	4.96e+02 ( $\sim$ )	5.68e-02	6.14e-02 ( $\sim$ )	7.16e-02	6.93e-02 ( $\sim$ )	1.36e-01	9.29e-02 ( $\sim$ )	1.39e-02	7.63e-02 ( $\sim$ )
TS-TCH	0.00e+00	0.00e+00 ( $\sim$ )	2.76e-03	4.60e-03 ( $+$ )	0.00e+00	0.00e+00 ( $\sim$ )	2.92e-03	8.28e-03 ( $+$ )	5.82e-03	1.47e-02 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $\sim$ )
JES	0.00e+00	0.00e+00 ( $\sim$ )	5.95e-02	5.53e-02 ( $\sim$ )	3.24e+02	7.44e+02 ( $\sim$ )	3.93e-02	4.22e-02 ( $\sim$ )	4.20e-02	4.51e-02 ( $\sim$ )	1.12e-01	9.06e-02 ( $\sim$ )	0.00e+00	0.00e+00 ( $\sim$ )
NEHVI	0.00e+00	0.00e+00 ( $\sim$ )	2.38e-02	4.35e-02 ( $+$ )	4.40e+02	6.41e+02 ( $\sim$ )	2.82e-02	4.75e-02 ( $+$ )	3.20e-02	5.52e-02 ( $+$ )	1.71e-01	6.73e-02 ( $\sim$ )	1.50e-02	7.13e-02 ( $\sim$ )

Table 2: The HV results (mean and standard deviation) of the eight methods on the 20 benchmark and real-world problems under noisy settings. The method with the best mean HV is highlighted in bold. The symbols “

-

”, “

\sim

” and “

+

” indicate that the proposed CTD is statistically worse than, equivalent to, and better than the competitor, respectively.

Figure 4: Solutions obtained by the proposed CTD and the nine peer methods on DTLZ2 (

m=2

d=50

) under 200 evaluations. All methods start from the same

2(d+1)

initial points generated by a scrambled Sobol sequence. The black line denotes the Pareto front, and brighter colours indicate solutions obtained at later evaluations.

Method	DTLZ1 ( $m=2$ )		DTLZ2 ( $m=2$ )		DTLZ3 ( $m=2$ )		DTLZ4 ( $m=2$ )		DTLZ5 ( $m=2$ )		DTLZ6 ( $m=2$ )		DTLZ7 ( $m=2$ )
	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std
CTD ${(10^{-1})}$	0.00e+00	0.00e+00 ( $\sim$ )	1.93e-01	1.07e-01 ( $+$ )	5.42e+02	7.71e+02 ( $\sim$ )	2.21e-01	8.52e-02 ( $+$ )	2.20e-01	9.21e-02 ( $+$ )	3.26e-01	4.89e-02 ( $\sim$ )	1.73e-01	2.61e-02 ( $\sim$ )
CTD ${(10^{-2})}$	1.92e+00	1.05e+01 ( $\sim$ )	3.17e-01	5.62e-02 ( $\sim$ )	4.93e+02	1.06e+03 ( $\sim$ )	3.16e-01	4.19e-02 ( $+$ )	3.17e-01	4.51e-02 ( $+$ )	3.07e-01	5.16e-02 ( $\sim$ )	1.71e-01	2.43e-02 ( $\sim$ )
CTD ${(10^{-3})}$	1.87e-01	8.49e-01 ( $\sim$ )	3.44e-01	2.32e-02 ( $\sim$ )	6.01e+02	1.38e+03 ( $\sim$ )	3.55e-01	1.46e-02 ( $\sim$ )	3.40e-01	2.22e-02 ( $\sim$ )	3.00e-01	5.33e-02 ( $\sim$ )	1.70e-01	2.37e-02 ( $\sim$ )
CTD ${(10^{-5})}$	5.18e-01	2.84e+00 ( $\sim$ )	3.33e-01	3.64e-02 ( $\sim$ )	7.00e+02	9.85e+02 ( $\sim$ )	3.46e-01	2.06e-02 ( $\sim$ )	3.43e-01	2.33e-02 ( $\sim$ )	3.17e-01	6.28e-02 ( $\sim$ )	1.75e-01	2.73e-02 ( $\sim$ )
CTD ${(10^{-6})}$	0.00e+00	0.00e+00 ( $\sim$ )	2.90e-01	4.78e-02 ( $+$ )	6.96e+02	8.26e+02 ( $\sim$ )	3.02e-01	3.33e-02 ( $+$ )	2.90e-01	5.75e-02 ( $+$ )	3.15e-01	5.20e-02 ( $\sim$ )	1.62e-01	1.20e-02 ( $\sim$ )

Table 3: The HV results (mean and standard deviation) of our method with six different threshold on the 20 benchmark and real-world problems. The method with the best mean HV is highlighted in bold. The symbols “

-

”, “

\sim

” and “

+

” indicate that our method CTD is statistically worse than, equivalent to, and better than the competitor, respectively.

Method	DTLZ1 ( $m=2$ )		DTLZ2 ( $m=2$ )		DTLZ3 ( $m=2$ )		DTLZ4 ( $m=2$ )		DTLZ5 ( $m=2$ )		DTLZ6 ( $m=2$ )		DTLZ7 ( $m=2$ )
	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std
CTD tch	0.00e+00	0.00e+00 ( $\sim$ )	3.55e-01	1.22e-02 ( $\sim$ )	5.07e+02	6.87e+02 ( $\sim$ )	3.56e-01	1.20e-02 ( $\sim$ )	3.51e-01	1.52e-02 ( $\sim$ )	3.53e-01	6.73e-02 ( $-$ )	3.75e-01	2.16e-01 ( $-$ )
CTD	6.68e+00	2.43e+01	3.46e-01	2.09e-02	4.17e+02	8.64e+02	3.52e-01	1.46e-02	3.49e-01	1.95e-02	3.19e-01	4.51e-02	1.71e-01	2.43e-02
Method	DTLZ1 ( $m=3$ )		DTLZ2 ( $m=3$ )		DTLZ3 ( $m=3$ )		DTLZ4 ( $m=3$ )		DTLZ5 ( $m=3$ )		DTLZ6 ( $m=3$ )		DTLZ7 ( $m=3$ )
	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std
CTD tch	4.30e+03	7.83e+03 ( $\sim$ )	4.68e-01	5.62e-02 ( $-$ )	1.60e+06	8.57e+05 ( $\sim$ )	4.78e-01	5.39e-02 ( $-$ )	1.05e-01	1.43e-02 ( $\sim$ )	9.86e-02	2.38e-02 ( $\sim$ )	4.58e-01	2.68e-01 ( $\sim$ )

Table 4: The HV results (mean and standard deviation) of the two methods on the 20 benchmark and real-world problems on 30 independent runs. The method with the best mean HV is highlighted in bold. The symbols “

-

”, “

\sim

” and “

+

” indicate that CTD tch is statistically better than, equivalent to, and worse than EHVI-based CTD, respectively.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Method	DTLZ1 ( $m=2$ )		DTLZ2 ( $m=2$ )		DTLZ3 ( $m=2$ )		DTLZ4 ( $m=2$ )		DTLZ5 ( $m=2$ )		DTLZ6 ( $m=2$ )		DTLZ7 ( $m=2$ )
	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std
Sobol	5.13e-02	2.81e-01 ( $\sim$ )	2.45e-02	2.33e-02 ( $+$ )	3.26e-01	1.79e+00 ( $\sim$ )	2.47e-02	2.69e-02 ( $+$ )	1.76e-02	1.61e-02 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )
ParEGO	2.56e+00	8.00e+00 ( $\sim$ )	4.05e-01	2.23e-03 ( $+$ )	3.91e+02	6.99e+02 ( $\sim$ )	4.04e-01	2.04e-03 ( $+$ )	4.03e-01	2.73e-03 ( $+$ )	3.01e-01	1.13e-01 ( $\sim$ )	6.32e-01	2.12e-01 ( $-$ )
TS-TCH	0.00e+00	0.00e+00 ( $\sim$ )	1.49e-01	2.42e-02 ( $+$ )	0.00e+00	0.00e+00 ( $\sim$ )	1.51e-01	2.30e-02 ( $+$ )	1.51e-01	2.23e-02 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )
PSL	9.15e+01	1.44e+02 ( $-$ )	3.94e-01	1.10e-02 ( $+$ )	1.05e+03	1.62e+03 ( $\sim$ )	1.17e-01	2.07e-02 ( $+$ )	3.94e-01	1.10e-02 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	4.91e-01	1.63e-01 ( $-$ )
JES	0.00e+00	0.00e+00 ( $\sim$ )	3.99e-01	2.77e-03 ( $+$ )	9.96e+02	8.75e+02 ( $\sim$ )	3.99e-01	3.57e-03 ( $+$ )	4.00e-01	2.76e-03 ( $+$ )	2.51e-01	7.30e-02 ( $+$ )	6.45e-01	1.86e-01 ( $-$ )

Appendix

Table 5: The HV results (mean and standard deviation) of the ten methods on the 20 benchmark and real-world problems under 200 evaluations. The method with the best mean HV is highlighted in bold. The symbols “

-

”, “

\sim

” and “

+

” indicate that our method CTD is statistically worse than, equivalent to, and better than the competitor, respectively.

Method	DTLZ1 ( $m=2$ )		DTLZ2 ( $m=2$ )		DTLZ3 ( $m=2$ )		DTLZ4 ( $m=2$ )		DTLZ5 ( $m=2$ )		DTLZ6 ( $m=2$ )		DTLZ7 ( $m=2$ )
	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std
Sobol	3.20e-01	1.75e+00 ( $\sim$ )	2.08e-02	1.60e-02 ( $+$ )	1.10e+02	6.03e+02 ( $+$ )	2.34e-02	2.02e-02 ( $+$ )	2.18e-02	2.17e-02 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $\sim$ )
ParEGO	0.00e+00	0.00e+00 ( $\sim$ )	1.42e-01	4.65e-02 ( $\sim$ )	6.00e+02	7.25e+02 ( $\sim$ )	1.39e-01	7.34e-02 ( $\sim$ )	1.40e-01	8.26e-02 ( $\sim$ )	2.27e-01	7.34e-02 ( $\sim$ )	1.04e-01	1.55e-01 ( $\sim$ )
TS-TCH	0.00e+00	0.00e+00 ( $\sim$ )	5.39e-03	1.06e-02 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	8.57e-03	1.40e-02 ( $+$ )	1.35e-02	2.17e-02 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $\sim$ )
JES	0.00e+00	0.00e+00 ( $\sim$ )	1.43e-01	6.86e-02 ( $\sim$ )	8.24e+02	9.69e+02 ( $\sim$ )	1.06e-01	5.03e-02 ( $\sim$ )	9.97e-02	6.62e-02 ( $\sim$ )	2.17e-01	6.15e-02 ( $\sim$ )	1.37e-02	3.91e-02 ( $\sim$ )
EHVI	0.00e+00	0.00e+00 ( $\sim$ )	6.86e-02	8.47e-02 ( $+$ )	8.92e+02	8.95e+02 ( $\sim$ )	5.31e-02	5.55e-02 ( $+$ )	8.97e-02	7.69e-02 ( $\sim$ )	1.94e-01	5.54e-02 ( $\sim$ )	8.76e-02	1.61e-01 ( $\sim$ )

Appendix

Table 6: The HV results (mean and standard deviation) of the eight methods on the 20 benchmark and real-world problems under noisy settings and 200 evaluations. The method with the best mean HV is highlighted in bold. The symbols “

-

”, “

\sim

” and “

+

” indicate that the proposed CTD is statistically worse than, equivalent to, and better than the competitor, respectively.

Method	DTLZ1 ( $m=2$ )		DTLZ2 ( $m=2$ )		DTLZ3 ( $m=2$ )		DTLZ4 ( $m=2$ )		DTLZ5 ( $m=2$ )		DTLZ6 ( $m=2$ )		DTLZ7 ( $m=2$ )
	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std
Sobol	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )
ParEGO	1.34e+05	2.41e+04 ( $+$ )	6.48e-02	9.40e-02 ( $+$ )	1.91e+05	5.93e+04 ( $+$ )	2.08e-02	4.07e-02 ( $+$ )	2.92e-01	3.36e-01 ( $+$ )	8.85e-02	5.30e-02 ( $+$ )	4.01e-01	2.11e-01 ( $-$ )
TS-TCH	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )
PSL	8.86e+04	2.98e+04 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	3.99e+04	4.16e+04 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	3.50e-01	2.02e-01 ( $\sim$ )
JES	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )

Appendix

Table 7: The HV results (mean and standard deviation) of the ten methods on 14 problems with

d=50

. The method with the best mean HV is highlighted in bold. The symbols “

-

”, “

\sim

” and “

+

” indicate that the proposed CTD is statistically worse than, equivalent to, and better than the competitor, respectively.

	Sobol	ParEGO	TS-TCH	PSL	JES	EHVI	MORBO	MOBO-OSD	qPOTS
CTD (ours)	0/0/14	2/0/12	0/0/14	1/1/12	0/0/14	0/6/8	0/0/14	4/1/9	0/0/14

Appendix

Table 8: The summary of statistical results for the noisy experiments in Table 7 . Here, the left, median, and right numbers are the counts of test problems where the CTD was statistically worse, equivalent, or better to the peer method, respectively.

Method	DTLZ1 ( $m=2$ )		DTLZ2 ( $m=2$ )		DTLZ3 ( $m=2$ )		DTLZ4 ( $m=2$ )		DTLZ5 ( $m=2$ )		DTLZ6 ( $m=2$ )		DTLZ7 ( $m=2$ )
	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std
CTD ${(10^{-1})}$	0.00e+00	0.00e+00 ( $\sim$ )	4.08e-01	1.49e-03 ( $\sim$ )	9.70e+02	1.01e+03 ( $\sim$ )	4.08e-01	1.19e-03 ( $+$ )	4.08e-01	1.36e-03 ( $+$ )	3.32e-01	3.53e-02 ( $\sim$ )	1.86e-01	3.22e-02 ( $\sim$ )
CTD ${(10^{-2})}$	1.92e+00	1.05e+01 ( $\sim$ )	4.09e-01	1.45e-03 ( $\sim$ )	9.23e+02	1.30e+03 ( $\sim$ )	4.09e-01	1.08e-03 ( $\sim$ )	4.09e-01	1.53e-03 ( $\sim$ )	3.14e-01	4.42e-02 ( $\sim$ )	1.79e-01	3.03e-02 ( $\sim$ )
CTD ${(10^{-3})}$	1.84e+00	6.59e+00 ( $\sim$ )	4.09e-01	1.32e-03 ( $\sim$ )	1.17e+03	1.60e+03 ( $\sim$ )	4.10e-01	1.49e-03 ( $\sim$ )	4.09e-01	1.05e-03 ( $\sim$ )	3.08e-01	5.12e-02 ( $\sim$ )	1.81e-01	3.07e-02 ( $\sim$ )
CTD ${(10^{-5})}$	5.13e+00	1.36e+01 ( $\sim$ )	4.09e-01	1.04e-03 ( $\sim$ )	9.84e+02	1.13e+03 ( $\sim$ )	4.10e-01	1.43e-03 ( $\sim$ )	4.10e-01	1.41e-03 ( $\sim$ )	3.31e-01	3.64e-02 ( $\sim$ )	1.83e-01	3.09e-02 ( $\sim$ )
CTD ${(10^{-6})}$	6.05e+00	2.74e+01 ( $\sim$ )	4.08e-01	1.10e-03 ( $\sim$ )	1.13e+03	1.14e+03 ( $\sim$ )	4.09e-01	1.46e-03 ( $+$ )	4.09e-01	1.61e-03 ( $+$ )	3.23e-01	4.11e-02 ( $\sim$ )	1.64e-01	1.66e-02 ( $\sim$ )

Appendix

Table 9: The HV results (mean and standard deviation) of our method with six different threshold on the 20 benchmark and real-world problems under 200 evaluations. The method with the best mean HV is highlighted in bold. The symbols “

-

”, “

\sim

” and “

+

” indicate that our method CTD is statistically worse than, equivalent to, and better than the competitor, respectively.

Method	DTLZ1 ( $m=2$ )		DTLZ2 ( $m=2$ )		DTLZ3 ( $m=2$ )		DTLZ4 ( $m=2$ )		DTLZ5 ( $m=2$ )		DTLZ6 ( $m=2$ )		DTLZ7 ( $m=2$ )
	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std
CTD tch	0.00e+00	0.00e+00 ( $\sim$ )	6.38e-02	5.88e-02 ( $\sim$ )	4.43e+02	8.41e+02 ( $\sim$ )	4.19e-02	4.82e-02 ( $\sim$ )	4.88e-02	5.12e-02 ( $\sim$ )	1.02e-01	6.21e-02 ( $\sim$ )	5.90e-02	1.15e-01 ( $-$ )
CTD	8.72e-03	4.78e-02	7.59e-02	5.95e-02	3.69e+02	6.54e+02	5.68e-02	4.74e-02	6.70e-02	4.69e-02	1.33e-01	7.48e-02	1.48e-02	5.13e-02
Method	DTLZ1 ( $m=3$ )		DTLZ2 ( $m=3$ )		DTLZ3 ( $m=3$ )		DTLZ4 ( $m=3$ )		DTLZ5 ( $m=3$ )		DTLZ6 ( $m=3$ )		DTLZ7 ( $m=3$ )
	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std
CTD tch	6.61e+03	6.16e+03 ( $\sim$ )	5.20e-02	5.54e-02 ( $+$ )	1.68e+06	6.09e+05 ( $\sim$ )	5.40e-02	5.44e-02 ( $+$ )	5.21e-03	8.78e-03 ( $+$ )	2.18e-02	2.95e-02 ( $+$ )	0.00e+00	0.00e+00 ( $\sim$ )

Appendix

Table 10: The HV results (mean and standard deviation) of the eight methods on the 20 benchmark and real-world problems under noisy settings. The method with the best mean HV is highlighted in bold. The symbols “

-

”, “

\sim

” and “

+

” indicate that CTD tch is statistically better than, equivalent to, and worse than EHVI-based CTD, respectively.

Method	DTLZ1 ( $m=2$ )		DTLZ2 ( $m=2$ )		DTLZ3 ( $m=2$ )		DTLZ4 ( $m=2$ )		DTLZ5 ( $m=2$ )		DTLZ6 ( $m=2$ )		DTLZ7 ( $m=2$ )
	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std
CTD tch	1.53e+05	1.97e+04 ( $+$ )	3.16e-01	1.68e-01 ( $-$ )	2.71e+05	7.69e+04 ( $\sim$ )	1.69e-01	1.09e-01 ( $\sim$ )	1.12e+00	2.39e-01 ( $-$ )	1.64e-01	1.00e-01 ( $\sim$ )	3.63e-01	1.83e-01 ( $-$ )
CTD	1.94e+05	2.83e+04	1.84e-01	1.59e-01	2.97e+05	7.55e+04	1.29e-01	1.17e-01	7.42e-01	4.11e-01	1.56e-01	9.46e-02	2.11e-01	2.45e-02
Method	DTLZ1 ( $m=3$ )		DTLZ2 ( $m=3$ )		DTLZ3 ( $m=3$ )		DTLZ4 ( $m=3$ )		DTLZ5 ( $m=3$ )		DTLZ6 ( $m=3$ )		DTLZ7 ( $m=3$ )
	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std
CTD tch	1.10e+08	1.95e+07 ( $+$ )	1.72e+01	2.47e+00 ( $-$ )	4.61e+08	1.09e+08 ( $\sim$ )	7.68e+00	1.61e+00 ( $-$ )	1.72e+01	2.01e+00 ( $-$ )	1.64e-02	2.91e-02 ( $+$ )	6.34e-01	2.80e-01 ( $-$ )

Appendix

Table 11: The HV results (mean and standard deviation) of the two methods on 14 problems with

d=50

. The method with the best mean HV is highlighted in bold. The symbols “

-

”, “

\sim

” and “

+

” indicate that the method is statistically better than, equivalent to, and worse than CTD, respectively.

	CTD tch
CTD	7/4/3

Appendix

Table 12: The summary of statistical results for the high-dimensional settings in Table 11 . Here, the left, median, and right numbers are the counts of test problems where the CTD was statistically worse, equivalent, or better to CTD tch , respectively.

Explore similar work

Multi-Objective Bayesian Optimization via Adaptive \varepsilon-Constraints Decomposition
Apr 17, 2026Yaohong Yang, Sammie Katt, Samuel KaskiBayesian Optimization Multi-Objective Optimization
Department of Computer Science, Aalto University, Espoo, Finland · ELLIS Institute Finland · Department of Computer Science, University of Manchester, Manchester, United Kingdom
Do We Really Need to Approach the Entire Pareto Front in Many-Objective Bayesian Optimisation?
Apr 10, 2026Chao Jiang, Jingyu Huang, Miqing LiMulti-Objective Optimization Bayesian Optimization
School of Computer Science, University of Birmingham · School of Mathematics, University of Birmingham
Learning What to Evaluate: Correlation-Aware Decoupling for Multiobjective Bayesian Optimization
Sep 26, 2026Ashwin Renganathan, Peter BachmanBayesian Optimization Multi-Objective Optimization
Aerospace Engineering Pennsylvania State University University Park, PA 16802 · Institute of Computational and Data Sciences (ICDS) Pennsylvania State University University Park, PA 16802

Apr 17, 2026J/K move · Enter open · S save

Multi-Objective Bayesian Optimization via Adaptive \varepsilon-Constraints Decomposition

Yaohong Yang, Sammie Katt, Samuel Kaski

Department of Computer Science, Aalto University, Espoo, Finland · ELLIS Institute Finland · Department of Computer Science, University of Manchester, Manchester, United Kingdom

Bayesian Optimization Multi-Objective Optimization

Open paper AskPDF arXiv

Multi-objective Bayesian optimization (MOBO) provides a principled framework for optimizing multiple expensive black-box functions. However, existing MOBO methods often struggle with coverage, scalability, and handling constraints and preferences. In this work we propose STAGE-BO, Sequential Targeting Adaptive Gap-Filling

\varepsilon

-Constraint Bayesian Optimization: by analyzing the coverage of the surrogate Pareto front, our method identifies the Pareto front point with the largest uncovered gap, and uses its coordinates to define adaptive constraints in

\varepsilon

-constraint method, which transforms the problem into a sequence of inequality-constrained subproblems, efficiently solved via constrained expected improvement acquisition. Our approach provides uniform Pareto coverage without hypervolume computation and naturally handles constraints and preferences. Experiments on synthetic and real-world benchmarks demonstrate superior coverage and competitive hypervolume performance against state-of-the-art baselines. Our code implementation can be found at https://github.com/YangYaohong1/STAGE-BO.

Abstract

Explore similar work

Multi-Objective Bayesian Optimization via Adaptive \varepsilon-Constraints Decomposition

Do We Really Need to Approach the Entire Pareto Front in Many-Objective Bayesian Optimisation?

Learning What to Evaluate: Correlation-Aware Decoupling for Multiobjective Bayesian Optimization