Do We Really Need to Approach the Entire Pareto Front in Many-Objective Bayesian Optimisation?

Chao Jiang; Jingyu Huang; Miqing Li

cs.AIApr 10, 2026

Do We Really Need to Approach the Entire Pareto Front in Many-Objective Bayesian Optimisation?

Authors: Chao Jiang, Jingyu Huang, Miqing Li

Organizations: School of Computer Science, University of Birmingham · School of Mathematics, University of Birmingham

Abstract

Many-objective optimisation, a subset of multi-objective optimisation, involves optimisation problems with more than three objectives. As the number of objectives increases, the number of solutions needed to adequately represent the entire Pareto front typically grows substantially. This makes it challenging, if not infeasible, to design a search algorithm capable of effectively exploring the entire Pareto front. This difficulty is particularly acute in the Bayesian optimisation paradigm, where sample efficiency is critical and only a limited number of solutions (often a few hundred) are evaluated. Moreover, after the optimisation process, the decision-maker eventually selects just one solution for deployment, regardless of how many high-quality, diverse solutions are available. In light of this, we argue an idea that under a very limited evaluation budget, it may be more useful to focus on finding a single solution of the highest possible quality for the decision-maker, rather than aiming to approximate the entire Pareto front as existing many-/multi-objective Bayesian optimisation methods typically do. Bearing this idea in mind, this paper proposes a single point-based multi-objective search framework (SPMO) that aims to improve the quality of solutions along a direction that leads to a good tradeoff between objectives. Within SPMO, we present a simple acquisition function, called expected single-point improvement (ESPI), working under both noiseless and noisy scenarios. We show that ESPI can be optimised effectively with gradient-based methods via the sample average approximation (SAA) approach and theoretically prove its convergence guarantees under the SAA. We also empirically demonstrate that the proposed SPMO is computationally tractable and outperforms state-of-the-arts on a wide range of benchmark and real-world problems.

Figures & tables

Figure 1: An illustration of our idea in a bi-objective case, in comparison with existing methods that aim to search for the entire Pareto front. The red points represent (nondominated) solutions that existing methods may obtain, and the blue point represents what our method aims for. It can be seen that the red points are much more diversified, hence, as a whole, having a better hypervolume (HV) value ( Zitzler and Thiele, 1999 ) . However, the blue point has better convergence (i.e., closer to the Pareto front) than any single red point, which may be more likely to be preferred by the decision-maker.

Method	DTLZ1	DTLZ2	Inverted DTLZ1	Inverted DTLZ2	Convex DTLZ2	Scaled DTLZ2	Car side impact	Sum up
	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	$+$ / $\sim$ / $-$
Sobol	3.7e+0 (3.0e–1) +	2.4e–1 (4.6e–2) +	4.8e+0 (3.1e–1) +	6.0e–1 (5.7e–2) +	-3.1e–1 (2.5e–1) +	2.3e–1 (5.1e–2) +	-1.9e–1 (2.5e–2) +	7/ 0/ 0
ParEGO	3.4e+0 (1.9e–1) +	6.7e–2 (8.1e–2) +	3.2e+0 (5.4e–1) +	2.5e–1 (1.3e–2) +	-1.7e+0 (2.0e–1) +	1.3e–1 (8.9e–2) +	-3.3e–1 (6.0e–3) +	7/ 0/ 0
TS-TCH	3.9e+0 (2.2e–1) +	2.0e–1 (4.1e–2) +	4.8e+0 (3.2e–1) +	4.6e–1 (1.2e–2) +	-7.1e–1 (2.1e–1) +	2.6e–1 (3.8e–2) +	-2.7e–1 (1.5e–2) +	7/ 0/ 0
EHVI	3.5e+0 (9.6e–2) +	9.3e–3 (3.5e–3) +	4.0e+0 (4.4e–1) +	2.3e–1 (5.5e–3) +	-1.4e+0 (2.1e–1) +	3.1e–1 (6.0e–2) +	-3.3e–1 (1.0e–2) +	7/ 0/ 0
C-EHVI	3.6e+0 (1.4e–1) +	3.3e–3 (3.8e–3) +	3.9e+0 (4.6e–1) +	2.5e–1 (1.8e–2) +	-3.5e–1 (2.9e–1) +	7.6e–2 (8.0e–2) +	-3.3e–1 (1.0e–2) +	7/ 0/ 0

Table 1: Results of the distance-based metric (log distance) obtained by the SPMO and the six peer methods on the benchmark problems with 5 objectives and the car side impact design problem on 30 runs. The method with the best mean is highlighted in bold. The symbols “

+

”, “

\sim

” and “

-

” indicate that the method is statistically worse than, equivalent to and better than our SPMO, respectively.

Figure 2: Trajectories of the distance-based metric (log distance) obtained by the seven methods on the benchmark problems with 5 objectives and the car side impact design problem. Each coloured line represents the mean metric value on 30 independent runs (after the initial Sobol samples, represented by the dashed grey line).

Method	DTLZ1	DTLZ2	Inverted DTLZ1	Inverted DTLZ2	Convex DTLZ2	Scaled DTLZ2	Car side impact	Sum up
	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	$+$ / $\sim$ / $-$
Sobol	8.7e+12 (3.7e+11) +	3.5e–2 (1.5e–2) +	5.1e+12 (1.1e+12) +	8.7e–4 (1.5e–3) +	3.8e–1 (1.7e–1) +	3.8e–2 (2.1e–2) +	2.6e–1 (1.8e–2) +	7/ 0/ 0
ParEGO	9.1e+12 (1.8e+11) +	1.3e–1 (5.6e–2) +	8.8e+12 (7.0e+11) +	4.0e–2 (2.9e–3) +	1.1e+0 (6.2e–2) +	8.7e–2 (5.3e–2) +	3.6e–1 (2.6e–3) +	7/ 0/ 0
TS-TCH	8.4e+12 (3.8e+11) +	3.2e–2 (1.3e–2) +	4.9e+12 (1.2e+12) +	7.1e–3 (1.1e–3) +	6.5e–1 (1.4e–1) +	2.8e–2 (1.5e–2) +	3.1e–1 (9.3e–3) +	7/ 0/ 0
EHVI	9.0e+12 (9.5e+10) +	1.9e–1 ( 7.2e–3 ) ∼	7.5e+12 (9.7e+11) +	4.5e–2 (1.4e–3) +	1.0e+0 (9.7e–2) +	1.5e–2 (1.4e–2) +	3.6e–1 (6.8e–3) +	6/ 1/ 0
C-EHVI	9.0e+12 (1.0e+11) +	1.6e–1 (1.9e–2) +	7.7e+12 (7.8e+11) +	4.1e–2 (4.1e–3) +	4.3e–1 (2.1e–1) +	1.2e–1 (5.1e–2) +	3.6e–1 (4.4e–3) ∼	6/ 1/ 0

Table 2: The HV of the best solution (in terms of its HV value) obtained by SPMO and the peer methods on the benchmark problems with 5 objectives and the car side impact design problem on 30 runs. The method with the best mean is highlighted in bold. The symbols “

+

”, “

\sim

” and “

-

” indicate that the method is statistically worse than, equivalent to and better than our SPMO, respectively.

Method	DTLZ1	DTLZ2	Inverted DTLZ1	Inverted DTLZ2	Convex DTLZ2	Scaled DTLZ2	Car side impact	Sum up
	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	$+$ / $\sim$ / $-$
Sobol	1.0e+13 (3.1e+10) +	7.9e–2 (2.5e–2) +	5.6e+12 (8.7e+11) +	9.3e–4 (1.5e–3) +	5.8e–1 (2.1e–1) +	8.6e–2 (3.1e–2) +	5.2e–1 (1.0e–2) +	7/ 0/ 0
ParEGO	1.0e+13 (3.8e+10) -	3.6e–1 (2.1e–1) +	9.1e+12 (6.2e+11) +	1.4e–1 (1.0e–2) -	1.5e+0 ( 2.8e–2 ) -	2.2e–1 (1.6e–1) +	7.4e–1 (1.4e–2) -	3/ 0/ 4
TS-TCH	1.0e+13 (2.6e+10) +	5.9e–2 (2.7e–2) +	5.4e+12 (1.0e+12) +	2.0e–2 (3.6e–3) +	1.0e+0 (1.5e–1) +	5.1e–2 (2.7e–2) +	6.5e–1 (1.1e–2) -	6/ 0/ 1
EHVI	1.0e+13 ( 3.3e+8 ) -	8.3e–1 ( 6.2e–2 ) -	8.2e+12 (6.6e+11) +	2.1e–1 ( 1.9e–3 ) -	1.5e+0 (6.9e–2) +	2.0e–2 (1.7e–2) +	7.4e–1 (9.6e–3) -	3/ 0/ 4
C-EHVI	1.0e+13 (1.6e+11) ∼	3.6e–1 (8.5e–2) +	8.0e+12 (6.9e+11) +	7.2e–2 (9.2e–3) -	5.8e–1 (2.7e–1) +	2.1e–1 (1.1e–1) +	5.9e–1 ( 3.0e–2) -	4/ 1/ 2

Table 3: The HV of all the solutions obtained by the seven methods on the seven benchmark (with five objectives) and real-world problems on 30 independent runs. The method with the best mean HV is highlighted in bold. The symbols “

+

”, “

\sim

”, and “

-

” indicate that a method is statistically worse than, equivalent to, and better than SPMO, respectively.

Figure 3: Spider chart of the best solution (in terms of its HV) obtained by the seven methods on the inverted DTLZ1 problem with 5 objectives in a typical run. Each axis in the spider chart represents one objective. Here, the objective values are multiplied by

-1

in this minimisation problem, such that a solution with a larger area indicates better quality.

Figure 4: Left: Violin plots of the HV values of the best solution (with respect to its HV) obtained by all the methods on the car cab design problem in 30 independent runs. Right: The objective values (normalised and multiplied by

-1

) of the best solution (with the highest HV) obtained by each method on the cab design problem in a typical run.

Appendix figures & tables37 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Illustration of the proposed expected single-point improvement (ESPI) in a bi-objective space. The figure shows the utopian point (black star), nondominated points in the current dataset

\mathcal{D}^{n}

(red dots), and a new candidate point whose true objective values are not yet observed (blue dot). The dashed arc represents the current best Euclidean distance from the utopian point to the nondominated set, denoted as

g^{*}

(red dotted line). The blue dotted line represents the distance from the new candidate point to the utopian point

\bm{z}^{*}

, which may improve upon the current best distance

g^{*}

. When a new point lies within the shaded blue region and is closer to the utopian point, the improvement

I_{SP}(\cdot)

is higher.

Problem	Reference point	Suggested by
DTLZ1	$(400.0,...,400.0)\in\mathbb{R}^{m}$	Balandat et al. [2020] , Chugh [2020]
DTLZ2	$(1.1,\dots,1.1)\in\mathbb{R}^{m}$	Balandat et al. [2020] , Daulton et al. [2020] , Ishibuchi et al. [2018]
Inverted DTLZ1	$(400.0,\dots,400.0)\in\mathbb{R}^{m}$	Chugh [2020] , Ishibuchi et al. [2018]
Inverted DTLZ2	$(1.1,\dots,1.1)\in\mathbb{R}^{m}$	Ishibuchi et al. [2018]
Convex DTLZ2	$(1.1,\dots,1.1)\in\mathbb{R}^{m}$	Ishibuchi et al. [2018]
Scaled DTLZ2	$(1.12^{0},\dots,1.12^{m})\in\mathbb{R}^{m}$	Ishibuchi et al. [2018]

Appendix

Table 4: Reference points of the six benchmark problems used for hypervolume computation in EHVI and NEHVI, as well as for performance evaluation. Note that the referents points of the two real-world problems, i.e., car side impact design and car cab design, are set to

(1.1,...,1.1)\in\mathbb{R}^{m}

in the normalised objective space following the practice in Tanabe and Ishibuchi [2020] the utopian and nadir points are available at https://github.com/ryojitanabe/reproblems/tree/master/ideal_nadir_points ).

Method	DTLZ1 (3)	DTLZ2 (3)	Inverted DTLZ1 (3)	Inverted DTLZ2 (3)	Convex DTLZ2 (3)	Scaled DTLZ2 (3)	Sum up
	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	$+$ / $\sim$ / $-$
Sobol	3.8e+0 (3.3e–1) +	2.3e–1 (5.1e–2) +	4.4e+0 (4.0e–1) +	5.7e–2 (5.3e–2) +	-7.7e–2 (1.7e–1) +	2.2e–1 (4.8e–2) +	6/ 0/ 0
ParEGO	3.7e+0 (2.0e–1) +	4.7e–3 (3.1e–3) +	3.5e+0 (5.7e–1) +	-3.0e–1 (7.7e–3) +	-9.3e–1 (1.2e–1) +	5.2e–3 (4.7e–3) +	6/ 0/ 0
TS-TCH	3.9e+0 (2.6e–1) +	1.4e–1 (7.1e–2) +	4.3e+0 (3.6e–1) +	-1.1e–1 (2.4e–2) +	-3.4e–1 (1.3e–1) +	1.5e–1 (3.6e–2) +	6/ 0/ 0
EHVI	3.6e+0 (1.4e–1) +	4.6e–3 (2.5e–3) +	4.0e+0 (2.1e–1) +	-3.0e–1 (4.3e–3) +	-9.1e–1 (1.1e–1) +	1.8e–2 (6.0e–2) +	6/ 0/ 0
C-EHVI	3.6e+0 (1.2e–1) +	1.6e–3 (1.7e–3) +	4.0e+0 (2.6e–1) +	-2.8e–1 (2.9e–2) +	-4.5e–1 (2.4e–1) +	3.2e–3 (5.5e–3) +	6/ 0/ 0

Appendix

Table 5: Results of the distance-based metric (log distance) obtained by the SPMO and the peer methods on the noiseless problems with 3 objectives ( top ) and 10 objectives ( bottom ) on 30 independent runs. The method with the best mean is highlighted in bold. The symbols “

+

”, “

\sim

” and “

-

” indicate that the method is statistically worse than, equivalent to and better than our SPMO, respectively.

Method	DTLZ1 (3)	DTLZ2 (3)	Inverted DTLZ1 (3)	Inverted DTLZ2 (3)	Convex DTLZ2 (3)	Scaled DTLZ2 (3)	Sum up
	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	$+$ / $\sim$ / $-$
Sobol	5.3e+7 (3.3e+6) +	3.3e–2 (2.1e–2) +	4.4e+7 (5.7e+6) +	1.1e–1 (2.5e–2) +	1.9e–1 (1.1e–1) +	3.7e–2 (1.8e–2) +	6/ 0/ 0
ParEGO	5.6e+7 (1.5e+6) +	1.6e–1 (7.0e–3) +	5.5e+7 (4.2e+6) +	3.1e–1 (3.7e–3) +	7.2e–1 (5.1e–2) +	1.6e–1 ( 9.5e–3 ) -	5/ 0/ 1
TS-TCH	5.3e+7 (3.1e+6) +	5.0e–2 (3.1e–2) +	4.6e+7 (5.2e+6) +	2.0e–1 (1.2e–2) +	3.7e–1 (8.4e–2) +	4.8e–2 (2.5e–2) +	6/ 0/ 0
EHVI	5.7e+7 (7.7e+5) +	1.6e–1 (4.6e–3) +	5.1e+7 (2.1e+6) +	3.1e–1 (2.2e–3) +	6.8e–1 (4.6e–2) +	1.3e–1 (3.4e–2) -	5/ 0/ 1
C-EHVI	5.7e+7 (8.3e+5) +	1.3e–1 (1.3e–2) +	5.1e+7 (2.4e+6) +	3.0e–1 (1.4e–2) +	4.5e–1 (1.4e–1) +	1.4e–1 (1.7e–2) -	5/ 0/ 1

Appendix

Table 6: The HV of the best solution (in terms of its HV value) obtained by the proposed SPMO and the peer methods on the noiseless problems with 3 objectives ( top ) and 10 objectives ( bottom ) on 30 independent runs. The method with the best mean is highlighted in bold. The symbols “

+

”, “

\sim

” and “

-

” indicate that the method is statistically worse than, equivalent to and better than our SPMO, respectively.

Method	DTLZ1 (3)	DTLZ2 (3)	Inverted DTLZ1 (3)	Inverted DTLZ2 (3)	Convex DTLZ2 (3)	Scaled DTLZ2 (3)	Sum up
	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	$+$ / $\sim$ / $-$
Sobol	6.3e+7 (2.8e+5) ∼	4.9e–2 (2.7e–2) +	5.2e+7 (2.9e+6) +	2.1e–1 (2.6e–2) +	2.5e–1 (1.3e–1) +	6.2e–2 (2.5e–2) +	5/ 1/ 0
ParEGO	6.3e+7 (1.1e+6) -	5.4e–1 (5.2e–2) -	5.8e+7 (2.6e+6) +	6.7e–1 (1.1e–2) -	1.0e+0 (9.0e–2) ∼	5.4e–1 ( 7.5e–2 ) -	1/ 1/ 4
TS-TCH	6.2e+7 (7.8e+5) ∼	7.5e–2 (4.2e–2) +	5.2e+7 (3.3e+6) +	4.1e–1 (1.2e–2) -	5.4e–1 (1.1e–1) +	8.8e–2 (4.6e–2) +	4/ 1/ 1
EHVI	6.4e+7 (1.0e+5) -	6.4e–1 ( 2.2e–2 ) -	5.8e+7 (2.4e+6) +	7.0e–1 ( 3.9e–3 ) -	1.0e+0 ( 6.9e–2 ) ∼	2.6e–1 (7.7e–2) -	1/ 1/ 4
C-EHVI	6.3e+7 (6.8e+5) -	3.3e–1 (5.4e–2) +	5.7e+7 (3.2e+6) +	3.2e–1 (3.5e–2) +	5.7e–1 (2.0e–1) +	3.3e–1 (6.5e–2) -	4/ 0/ 2

Appendix

Table 7: The HV of all the solutions obtained by the proposed SPMO and the peer methods on the noiseless problems with 3 objectives ( top ) and 10 objectives ( bottom ) on 30 independent runs, respectively. The method with the best mean is highlighted in bold. The symbols “

+

”, “

\sim

”, and “

-

” indicate that a method is statistically worse than, equivalent to, and better than SPMO, respectively.

Panel of Figure 6 — Figure 6: Violin plots of the distance-based metric (log distance) obtained by the proposed SPMO and the peer methods on the noiseless problems with 3 and 10 objectives. Each violin represents the distribution of the distance-based metric obtained by a method over 30 independent runs.

Panel of Figure 7 — Figure 7: Violin plots of the HV of the best solution (in terms of its HV value) obtained by the proposed SPMO and the peer methods on the noiseless problems with 3 and 10 objectives. Each violin represents the distribution of HV values obtained by a method over 30 independent runs.

Panel of Figure 8 — Figure 8: Violin plots of the HV of all evaluated solutions obtained by the proposed SPMO and the peer methods on the noiseless problems with 3 objectives ( top ) and 10 objectives ( bottom ), respectively. Each violin represents the distribution of HV values obtained by a method over 30 independent runs.

Panel of Figure 9 — Figure 9: Trajectories of the distance metric obtained by the SPMO and the peer methods on the noiseless problems with 3 and 10 objectives. Each coloured line represents the mean distance of the closest solution to the utopian point on 30 independent runs (after the initial Sobol samples, represented by the dashed grey line).

Method	DTLZ3	DTLZ4	DTLZ5	DTLZ6	DTLZ7	Sum up
	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	$+$ / $\sim$ / $-$
Sobol	6.3e+0 (1.6e–1) +	2.7e–1 (6.0e–2) +	2.5e–1 (4.1e–2) +	2.2e+0 (2.1e–2) +	3.1e+0 (5.0e–2) +	5/ 0/ 0
ParEGO	5.5e+0 (1.1e–1) ∼	2.3e–1 (7.1e–2) +	4.0e–2 (3.1e–2) +	4.3e–2 (1.6e–1) +	2.2e+0 (8.0e–2) +	4/ 1/ 0
TS-TCH	6.3e+0 (1.6e–1) +	2.6e–1 (5.0e–2) +	2.3e–1 (5.9e–2) +	2.2e+0 (1.6e–2) +	3.1e+0 (4.8e–2) +	5/ 0/ 0
EHVI	5.4e+0 ( 4.6e–2 ) -	8.7e–2 (9.4e–2) +	3.0e–1 (6.0e–2) +	-9.4e–7 (8.4e–7) +	1.7e+0 (8.5e–2) +	4/ 0/ 1
JES	5.5e+0 (5.0e–2) ∼	2.3e–1 (8.1e–2) +	5.2e–2 (5.8e–2) +	2.1e–2 (1.1e–1) +	2.2e+0 (5.8e–2) +	4/ 1/ 0

Appendix

Table 8: Results of the distance-based metric (log distance) obtained by obtained by the SPMO and the peer methods on DTLZ3–DTLZ7 with 5 objectives on 30 independent runs. The method with the best mean is highlighted in bold. The symbols “

+

”, “

\sim

” and “

-

” indicate that the method is statistically worse than, equivalent to and better than our SPMO, respectively.

Method	DTLZ3	DTLZ4	DTLZ5	DTLZ6	DTLZ7	Sum up
	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	$+$ / $\sim$ / $-$
Sobol	9.2e+19 (1.1e+18) +	2.2e–2 (1.4e–2) +	8.4e+4 (9.5e+2) ∼	8.7e+3 (8.2e+2) +	-0.0e+0 (0.0e+0) +	4/ 1/ 0
ParEGO	9.7e+19 (5.2e+17) ∼	2.5e–2 (3.0e–2) +	8.9e+4 ( 1.4e+3 ) -	8.8e+4 (3.8e+3) -	2.8e+5 (2.7e+4) +	2/ 1/ 2
TS-TCH	9.3e+19 (1.0e+18) +	1.3e–2 (1.1e–2) +	8.6e+4 (1.1e+3) -	9.3e+3 (9.9e+2) +	-0.0e+0 (0.0e+0) +	4/ 0/ 1
EHVI	9.8e+19 ( 1.1e+17 ) -	1.3e–1 (5.6e–2) +	8.3e+4 (1.6e+3) +	9.0e+4 ( 1.3e–2 ) -	3.8e+5 (2.2e+4) ∼	2/ 1/ 2
JES	9.8e+19 (1.4e+17) -	3.6e–2 (3.5e–2) +	8.9e+4 (1.5e+3) -	8.8e+4 (3.5e+3) -	2.8e+5 (2.0e+4) +	2/ 0/ 3

Appendix

Table 9: The HV of the best solution (in terms of its HV value) obtained by SPMO and the peer methods on the DTLZ3–DTLZ7 with 5 objectives on 30 independent runs. The method with the best mean is highlighted in bold. The symbols “

+

”, “

\sim

” and “

-

” indicate that the method is statistically worse than, equivalent to and better than our SPMO, respectively.

Method	DTLZ3	DTLZ4	DTLZ5	DTLZ6	DTLZ7	Sum up
	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	$+$ / $\sim$ / $-$
Sobol	3.9e+20 (1.7e+20) ∼	4.9e–2 (2.4e–2) +	2.1e+5 (1.1e+5) ∼	1.4e+5 (1.7e+4) +	-0.0e+0 (0.0e+0) +	3/ 2/ 0
ParEGO	2.1e+20 (2.1e+20) ∼	5.3e–2 (7.4e–2) +	2.4e+5 (8.7e+4) ∼	3.9e+5 (1.6e+5) -	9.1e+5 (4.1e+5) +	2/ 2/ 1
TS-TCH	5.0e+20 ( 3.0e+20 ) -	2.1e–2 (1.8e–2) +	1.8e+5 (8.2e+4) +	1.8e+5 (4.2e+4) +	-0.0e+0 (0.0e+0) +	4/ 0/ 1
EHVI	1.7e+20 (1.5e+20) ∼	9.3e–1 (7.1e–1) ∼	3.5e+5 (1.8e+5) -	4.2e+5 ( 1.2e+5 ) -	1.9e+6 (5.3e+5) ∼	0/ 3/ 2
JES	2.7e+20 (2.7e+20) ∼	8.8e–2 (1.3e–1) +	3.6e+5 ( 3.5e+5 ) ∼	4.0e+5 (1.6e+5) -	1.0e+6 (4.3e+5) +	2/ 2/ 1

Appendix

Table 10: The HV of all evaluated solutions obtained by SPMO and the peer methods on DTLZ3–DTLZ7 with 5 objectives on 30 independent runs. The method with the best mean is highlighted in bold. The symbols “

+

”, “

\sim

” and “

-

” indicate that the method is statistically worse than, equivalent to and better than our SPMO, respectively.

Method	DTLZ1	DTLZ2	Inverted DTLZ1	Inverted DTLZ2	Convex DTLZ2	Scaled DTLZ2	Sum up
	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	$+$ / $\sim$ / $-$
Sobol	3.7e+0 (4.2e–1) +	1.9e–1 (5.6e–2) +	4.7e+0 (3.4e–1) +	6.1e–1 (6.1e–2) +	-2.9e–1 (2.2e–1) +	2.2e–1 (4.4e–2) +	6/ 0/ 0
NParEGO	3.6e+0 (3.1e–1) +	1.0e–1 (8.2e–2) +	3.1e+0 (3.8e–1) +	1.8e–1 (5.0e–2) +	-1.3e+0 (2.7e–1) +	5.2e–2 (5.9e–2) +	6/ 0/ 0
TS-TCH	3.8e+0 (3.5e–1) +	2.0e–1 (4.2e–2) +	4.9e+0 (3.2e–1) +	4.6e–1 (5.5e–2) +	-4.7e–1 (2.3e–1) +	2.1e–1 (5.0e–2) +	6/ 0/ 0
NEHVI	3.5e+0 (1.4e–1) +	-1.1e–1 (8.1e–2) +	4.0e+0 (5.8e–1) +	1.5e–1 (3.3e–2) +	-1.3e+0 (3.0e–1) +	2.9e–1 (9.5e–2) +	6/ 0/ 0
JES	3.4e+0 (1.2e–1) +	1.3e–1 (1.2e–1) +	4.5e+0 (1.1e–1) +	1.9e–1 (6.5e–2) +	-6.9e–1 (3.5e–1) +	1.0e–1 (8.7e–2) +	6/ 0/ 0

Appendix

Table 11: Results of the distance-based metric (log distance) obtained by the SPMO and the peer methods on the noisy problems with 5 objectives on 30 independent runs. The method with the best mean is highlighted in bold. The symbols “

+

”, “

\sim

” and “

-

” indicate that the method is statistically worse than, equivalent to and better than our SPMO, respectively.

Method	DTLZ1	DTLZ2	Inverted DTLZ1	Inverted DTLZ2	Convex DTLZ2	Scaled DTLZ2	Sum up
	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	$+$ / $\sim$ / $-$
Sobol	8.6e+12 (5.2e+11) +	5.1e–2 (3.0e–2) +	5.4e+12 (1.2e+12) +	8.2e–4 (1.5e–3) +	4.0e–1 (1.8e–1) +	3.8e–2 (1.9e–2) +	6/ 0/ 0
NParEGO	9.1e+12 (2.5e+11) +	1.0e–1 (4.9e–2) +	9.0e+12 (4.1e+11) +	5.8e–2 (1.6e–2) +	1.2e+0 (2.0e–1) +	1.3e–1 (4.0e–2) +	6/ 0/ 0
TS-TCH	8.5e+12 (5.1e+11) +	5.0e–2 (1.4e–2) +	4.8e+12 (1.2e+12) +	8.1e–3 (5.4e–3) +	5.3e–1 (1.7e–1) +	3.1e–2 (1.6e–2) +	6/ 0/ 0
NEHVI	9.1e+12 (1.5e+11) +	2.8e–1 (7.6e–2) +	7.4e+12 (1.2e+12) +	6.7e–2 (1.0e–2) +	1.2e+0 (2.1e–1) +	1.2e–2 (1.0e–2) +	6/ 0/ 0
JES	9.0e+12 (1.3e+11) +	8.7e–2 (7.2e–2) +	6.1e+12 (3.2e+11) +	5.8e–2 (2.0e–2) +	7.8e–1 (3.1e–1) +	1.0e–1 (5.7e–2) +	6/ 0/ 0

Appendix

Table 12: The HV of the best solution (in terms of its HV value) obtained by SPMO and the peer methods on the noisy problems with 5 objectives on 30 independent runs. The method with the best mean is highlighted in bold. The symbols “

+

”, “

\sim

” and “

-

” indicate that the method is statistically worse than, equivalent to and better than our SPMO, respectively.

Method	DTLZ1	DTLZ2	Inverted DTLZ1	Inverted DTLZ2	Convex DTLZ2	Scaled DTLZ2	Sum up
	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	$+$ / $\sim$ / $-$
Sobol	4.5e+13 (1.4e+13) +	2.1e–1 (8.4e–2) +	8.2e+12 (4.7e+12) +	8.5e–4 (1.5e–3) +	9.3e–1 (5.1e–1) +	1.6e–1 (5.7e–2) +	6/ 0/ 0
NParEGO	5.8e+13 (2.3e+13) ∼	4.3e–1 (3.9e–1) +	1.7e+13 (9.6e+12) +	5.1e–1 (1.3e–1) +	1.1e+1 (4.8e+0) +	7.7e–1 (6.5e–1) +	5/ 1/ 0
TS-TCH	5.1e+13 (1.3e+13) ∼	1.9e–1 (8.3e–2) +	6.9e+12 (2.6e+12) +	3.2e–2 (1.8e–2) +	2.2e+0 (9.8e–1) +	9.0e–2 (6.5e–2) +	5/ 1/ 0
NEHVI	7.2e+13 (2.1e+13) ∼	3.1e+0 ( 8.9e–1 ) -	1.5e+13 (1.7e+13) +	7.7e–1 (2.0e–1) +	9.5e+0 (3.8e+0) +	1.5e–2 (1.3e–2) +	4/ 1/ 1
JES	7.5e+13 ( 2.3e+13 ) ∼	4.3e–1 (5.0e–1) +	3.5e+13 (2.0e+13) ∼	5.7e–1 (1.5e–1) +	4.6e+0 (2.8e+0) +	4.3e–1 (3.3e–1) +	4/ 2/ 0

Appendix

Table 13: The HV of all the solutions obtained by the proposed SPMO and the peer methods on the noisy problems with five objectives on 30 independent runs. The method with the best mean is highlighted in bold. The symbols “

+

”, “

\sim

”, and “

-

” indicate that a method is statistically worse than, equivalent to, and better than SPMO, respectively.

Figure 10: Violin plots of the distance-based metric (log distance) obtained by the six methods on the noisy problems with five objectives. Each violin represents the distribution of the distance-based metric obtained by a method over 30 independent runs.

Figure 11: Violin plots of the HV of the best solution (in terms of its HV value) obtained by the six methods on the noisy problems with five objectives. Each violin represents the distribution of maximum HV values obtained by a method over 30 independent runs.

Figure 12: Violin plots of the HV of all evaluated solutions obtained by the six methods on the noisy problems with five objectives. Each violin represents the distribution of maximum HV values obtained by a method over 30 independent runs.

Method	DTLZ1	DTLZ2	Inverted DTLZ1	Inverted DTLZ2	Convex DTLZ2	Scaled DTLZ2	Sum up
	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	$+$ / $\sim$ / $-$
Sobol	3.7e+0 (3.0e–1) +	2.4e–1 (4.6e–2) +	4.8e+0 (3.1e–1) +	6.0e–1 (5.7e–2) +	-3.1e–1 (2.5e–1) +	2.3e–1 (5.1e–2) +	6/ 0/ 0
ParEGO	3.4e+0 (1.9e–1) +	6.7e–2 (8.1e–2) +	3.2e+0 (5.4e–1) +	2.5e–1 (1.3e–2) +	-1.7e+0 (2.0e–1) +	1.3e–1 (8.9e–2) +	6/ 0/ 0
TS-TCH	3.9e+0 (2.2e–1) +	2.0e–1 (4.1e–2) +	4.8e+0 (3.2e–1) +	4.6e–1 (1.2e–2) +	-7.1e–1 (2.1e–1) +	2.6e–1 (3.8e–2) +	6/ 0/ 0
EHVI	3.5e+0 (9.6e–2) +	9.3e–3 (3.5e–3) +	4.0e+0 (4.4e–1) +	2.3e–1 (5.5e–3) +	-1.4e+0 (2.1e–1) +	3.1e–1 (6.0e–2) +	6/ 0/ 0
JES	3.4e+0 (1.3e–1) +	1.1e–1 (8.0e–2) +	4.5e+0 (1.7e–1) +	2.6e–1 (2.0e–2) +	-1.0e+0 (4.5e–1) +	8.8e–2 (9.3e–2) +	6/ 0/ 0

Appendix

Table 14: Results of the distance-based metric (log distance) obtained by the SPMO and the five peer methods with a batch size

q=5

on the problems with 5 objectives on 30 independent runs. The method with the best mean is highlighted in bold. The symbols “

+

”, “

\sim

” and “

-

” indicate that the method is statistically worse than, equivalent to and better than our SPMO, respectively.

Method	DTLZ1	DTLZ2	Inverted DTLZ1	Inverted DTLZ2	Convex DTLZ2	Scaled DTLZ2	Sum up
	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	$+$ / $\sim$ / $-$
Sobol	8.7e+12 (3.7e+11) +	3.5e–2 (1.5e–2) +	5.1e+12 (1.1e+12) +	8.7e–4 (1.5e–3) +	3.8e–1 (1.7e–1) +	3.8e–2 (2.1e–2) +	6/ 0/ 0
ParEGO	9.1e+12 (1.8e+11) +	1.3e–1 (5.6e–2) +	8.8e+12 (7.0e+11) +	4.0e–2 (2.9e–3) +	1.1e+0 (6.2e–2) +	8.7e–2 (5.3e–2) +	6/ 0/ 0
TS-TCH	8.4e+12 (3.8e+11) +	3.2e–2 (1.3e–2) +	4.9e+12 (1.2e+12) +	7.1e–3 (1.1e–3) +	6.5e–1 (1.4e–1) +	2.8e–2 (1.5e–2) +	6/ 0/ 0
EHVI	9.0e+12 (9.5e+10) +	1.9e–1 ( 7.2e–3 ) ∼	7.5e+12 (9.7e+11) +	4.5e–2 (1.4e–3) +	1.0e+0 (9.7e–2) +	1.5e–2 (1.4e–2) +	5/ 1/ 0
JES	9.0e+12 (1.2e+11) +	1.0e–1 (3.9e–2) +	6.2e+12 (5.0e+11) +	3.9e–2 (4.5e–3) +	8.5e–1 (2.4e–1) +	1.1e–1 (5.5e–2) +	6/ 0/ 0

Appendix

Table 15: The HV of the best solution (in terms of its HV value) obtained by SPMO and the five peer methods with a batch size

q=5

on the problems with 5 objectives on 30 independent runs. The method with the best mean is highlighted in bold. The symbols “

+

”, “

\sim

” and “

-

” indicate that the method is statistically worse than, equivalent to and better than our SPMO, respectively.

Method	DTLZ1	DTLZ2	Inverted DTLZ1	Inverted DTLZ2	Convex DTLZ2	Scaled DTLZ2	Sum up
	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	$+$ / $\sim$ / $-$
Sobol	4.4e+13 (1.4e+13) ∼	1.4e–1 (5.6e–2) +	7.7e+12 (2.9e+12) +	9.5e–4 (1.6e–3) +	9.5e–1 (4.6e–1) +	1.6e–1 (6.6e–2) +	5/ 1/ 0
ParEGO	2.9e+13 (1.2e+13) ∼	6.5e–1 (5.3e–1) +	1.5e+13 (7.8e+12) +	5.0e–1 (9.6e–2) ∼	9.6e+0 (4.8e+0) +	4.2e–1 (3.9e–1) +	4/ 2/ 0
TS-TCH	4.5e+13 ( 1.3e+13 ) ∼	8.9e–2 (5.3e–2) +	6.8e+12 (1.8e+12) +	5.0e–2 (1.7e–2) +	3.2e+0 (1.7e+0) +	7.4e–2 (5.2e–2) +	5/ 1/ 0
EHVI	2.9e+13 (1.8e+13) ∼	2.1e+0 ( 5.9e–1 ) -	1.9e+13 (1.7e+13) +	6.1e–1 ( 1.5e–1 ) ∼	7.4e+0 (2.2e+0) +	2.0e–2 (1.7e–2) +	3/ 2/ 1
JES	3.1e+13 (2.8e+13) ∼	6.9e–1 (5.2e–1) +	7.4e+13 ( 9.1e+13 ) -	4.7e–1 (1.1e–1) ∼	5.7e+0 (4.6e+0) +	7.5e–1 (6.3e–1) +	3/ 2/ 1

Appendix

Table 16: The HV of all the solutions obtained by the six methods on the problems with five objectives on 30 independent runs. The method with the best mean is highlighted in bold. The symbols “

+

”, “

\sim

”, and “

-

” indicate that a method is statistically worse than, equivalent to, and better than SPMO, respectively.

Figure 13: Violin plots of the distance-based metric (log distance) obtained by the proposed SPMO and the peer methods on the problems with five objectives. Each violin represents the distribution of the distance-based metric obtained by a method over 30 independent runs.

Figure 14: Violin plots of the HV of the best solution (in terms of its HV value) obtained by the six methods with a batch size

q=5

on the problems with five objectives. Each violin represents the distribution of maximum HV values obtained by a method over 30 independent runs.

Figure 15: Violin plots of the HV of all evaluated solutions obtained by the six methods on the problems with five objectives. Each violin represents the distribution of maximum HV values obtained by a method over 30 independent runs.

Method	DTLZ1	DTLZ2	Inverted DTLZ1	Inverted DTLZ2	Convex DTLZ2	Scaled DTLZ2	Sum up
	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	$+$ / $\sim$ / $-$
SPMO_0.01	3.1e+0 (4.9e–1) ∼	6.2e–4 (1.4e–3) -	2.8e+0 ( 4.2e–1 ) ∼	2.1e–1 (3.6e–5) ∼	-2.1e+0 (1.6e–2) ∼	6.4e–5 (3.9e–5) -	0/ 4/ 2
SPMO_0.1	2.9e+0 (3.6e–1) -	3.0e–4 ( 2.3e–4 ) -	3.0e+0 (5.3e–1) ∼	2.1e–1 (4.6e–5) ∼	-2.1e+0 (2.6e–2) ∼	5.3e–5 ( 2.4e–5 ) -	0/ 3/ 3
SPMO_1.0	2.8e+0 ( 6.1e–1 ) ∼	3.3e–4 (2.0e–4) -	2.9e+0 (5.0e–1) ∼	2.1e–1 ( 3.4e–5 ) ∼	-2.1e+0 (1.6e–2) ∼	5.7e–5 (2.4e–5) -	0/ 4/ 2
SPMO	3.1e+0 (3.0e–1)	9.0e–4 (8.3e–4)	2.9e+0 (4.9e–1)	2.1e–1 (5.0e–5)	-2.1e+0 ( 7.7e–3 )	1.7e–4 (1.2e–4)

Appendix

Table 17: Results of the distance-based metric (log distance) obtained by the SPMO with four different utopian points on the problems with five objectives on 30 independent runs. The method with the best mean is highlighted in bold. The symbols “

+

”, “

\sim

” and “

-

” indicate that the method is statistically worse than, equivalent to and better than SPMO (current), respectively.

Method	DTLZ1	DTLZ2	Inverted DTLZ1	Inverted DTLZ2	Convex DTLZ2	Scaled DTLZ2	Sum up
	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	$+$ / $\sim$ / $-$
SPMO_0.01	9.4e+12 (3.7e+11) ∼	1.9e–1 ( 1.0e–2 ) ∼	9.3e+12 ( 3.8e+11 ) ∼	4.9e–2 (9.0e–6) ∼	1.3e+0 ( 7.8e–3 ) ∼	1.6e–1 (1.4e–2) ∼	0/ 6/ 0
SPMO_0.1	9.5e+12 (2.7e+11) -	1.9e–1 (1.6e–2) ∼	9.0e+12 (5.6e+11) ∼	4.9e–2 (1.1e–5) ∼	1.2e+0 (1.1e–2) ∼	1.6e–1 (1.0e–2) ∼	0/ 5/ 1
SPMO_1.0	9.5e+12 ( 3.8e+11 ) -	1.9e–1 (1.3e–2) ∼	9.1e+12 (4.2e+11) ∼	4.9e–2 ( 8.5e–6 ) ∼	1.3e+0 (7.8e–3) ∼	1.6e–1 (1.4e–2) ∼	0/ 5/ 1
SPMO	9.3e+12 (2.8e+11)	1.9e–1 (1.4e–2)	9.2e+12 (4.8e+11)	4.9e–2 (1.2e–5)	1.3e+0 (5.0e–3)	1.6e–1 ( 1.6e–2 )

Appendix

Table 18: The HV of the best solution (in terms of its HV value) obtained by the proposed SPMO with four different utopian points on the problems with five objectives on 30 independent runs. The method exhibiting the best mean is highlighted in bold. The symbols “

+

”, “

\sim

”, and “

-

” denote that a method is statistically worse than, equivalent to, or better than SPMO (current), respectively.

Method	DTLZ1	DTLZ2	Inverted DTLZ1	Inverted DTLZ2	Convex DTLZ2	Scaled DTLZ2	Sum up
	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	$+$ / $\sim$ / $-$
SPMO_0.01	3.1e+13 (1.7e+13) ∼	2.7e+0 (1.2e+0) -	3.5e+13 (3.7e+13) ∼	6.5e–1 (7.0e–1) ∼	2.5e+1 (1.8e+1) ∼	2.2e+0 ( 1.5e+0 ) ∼	0/ 5/ 1
SPMO_0.1	3.5e+13 (1.5e+13) ∼	2.7e+0 (1.2e+0) -	2.6e+13 (1.7e+13) ∼	6.1e–1 (4.5e–1) ∼	2.7e+1 (1.7e+1) ∼	1.8e+0 (1.0e+0) ∼	0/ 5/ 1
SPMO_1.0	3.3e+13 (1.4e+13) ∼	3.0e+0 ( 1.6e+0 ) -	3.2e+13 (1.9e+13) ∼	7.1e–1 ( 6.3e–1 ) ∼	2.9e+1 ( 2.2e+1 ) ∼	2.1e+0 (1.5e+0) ∼	0/ 5/ 1
SPMO	3.7e+13 ( 2.2e+13 )	1.5e+0 (7.0e–1)	3.7e+13 ( 3.5e+13 )	5.9e–1 (3.3e–1)	2.6e+1 (1.7e+1)	1.9e+0 (1.6e+0)

Appendix

Table 19: The HV of all the solutions obtained by the SPMO with four different utopian points on the problems with five objectives on 30 independent runs. The method with the best mean is highlighted in bold. The symbols “

+

”, “

\sim

”, and “

-

” indicate that a method is statistically worse than, equivalent to, and better than SPMO (current), respectively.

Figure 16: Violin plots of the distance-based metric (log distance) obtained by the four methods on the problems with five objectives. Each violin represents the distribution of the distance-based metric obtained by a method over 30 independent runs.

Figure 17: Violin plots of the HV of the best solution (in terms of its HV value) obtained by the four methods on the problems with five objectives. Each violin represents the distribution of maximum HV values obtained by a method over 30 independent runs.

Figure 18: Violin plots of the HV of all evaluated solutions obtained by the four methods the problems with five objectives. Each violin represents the distribution of maximum HV values obtained by a method over 30 independent runs.

Method	DTLZ1	DTLZ2	Inverted DTLZ1	Inverted DTLZ2	Convex DTLZ2	Scaled DTLZ2	Sum up
	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	$+$ / $\sim$ / $-$
SPMO Tch	3.5e+0 (3.2e–1) +	3.5e–5 ( 4.3e–5 ) -	2.9e+0 (6.5e–1) ∼	3.1e–1 (3.3e–2) +	-1.8e+0 (1.7e–1) +	2.9e–5 ( 1.9e–5 ) -	3/ 1/ 2
SPMO ws	3.6e+0 (1.3e–1) +	2.1e–4 (1.5e–4) -	3.1e+0 (5.4e–1) ∼	2.2e–1 (2.7e–3) +	-1.2e+0 (2.7e–1) +	3.2e–4 (3.4e–4) +	4/ 1/ 1
SPMO	3.1e+0 ( 3.0e–1 )	9.0e–4 (8.3e–4)	2.9e+0 ( 4.9e–1 )	2.1e–1 ( 5.0e–5 )	-2.1e+0 ( 7.7e–3 )	1.7e–4 (1.2e–4)

Appendix

Table 20: Results of the distance-based metric (log distance) obtained by the SPMO using three different single-point metrics on the problems with five objectives on 30 independent runs. The method with the best mean is highlighted in bold. The symbols “

+

”, “

\sim

” and “

-

” indicate that the method is statistically worse than, equivalent to and better than our SPMO (i.e., SPMO dist ), respectively.

Method	DTLZ1	DTLZ2	Inverted DTLZ1	Inverted DTLZ2	Convex DTLZ2	Scaled DTLZ2	Sum up
	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	$+$ / $\sim$ / $-$
SPMO Tch	9.1e+12 (3.0e+11) +	1.6e–1 (1.9e–2) +	9.1e+12 (5.7e+11) ∼	2.7e–2 (5.9e–3) +	1.2e+0 (4.2e–2) +	1.6e–1 (2.2e–2) ∼	4/ 2/ 0
SPMO ws	9.1e+12 (1.3e+11) +	1.7e–1 (1.4e–2) +	9.0e+12 (6.5e+11) ∼	4.8e–2 (6.7e–4) +	9.7e–1 (1.2e–1) +	1.7e–1 ( 1.5e–2 ) ∼	4/ 2/ 0
SPMO	9.3e+12 ( 2.8e+11 )	1.9e–1 ( 1.4e–2 )	9.2e+12 ( 4.8e+11 )	4.9e–2 ( 1.2e–5 )	1.3e+0 ( 5.0e–3 )	1.6e–1 (1.6e–2)

Appendix

Table 21: The HV of the best solution (in terms of its HV value) obtained by the proposed SPMO using three different single-point metrics on the problems with five objectives on 30 independent runs. The method exhibiting the best mean is highlighted in bold. The symbols “

+

”, “

\sim

”, and “

-

” denote that a method is statistically worse than, equivalent to, or better than SPMO (i.e., SPMO dist ), respectively.

Method	DTLZ1	DTLZ2	Inverted DTLZ1	Inverted DTLZ2	Convex DTLZ2	Scaled DTLZ2	Sum up
	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	Mean (Std)	$+$ / $\sim$ / $-$
SPMO Tch	1.5e+13 (6.2e+12) +	4.9e+0 ( 7.6e+0 ) -	2.1e+13 (1.5e+13) +	2.1e+0 ( 2.6e+0 ) -	1.3e+1 (7.4e+0) +	3.3e+0 ( 3.4e+0 ) -	3/ 0/ 3
SPMO ws	1.3e+13 (3.1e+12) +	1.6e+0 (7.5e–1) ∼	2.4e+13 (1.8e+13) +	7.4e–1 (7.5e–1) ∼	6.4e+0 (3.8e+0) +	1.2e+0 (3.5e–1) ∼	3/ 3/ 0
SPMO	3.7e+13 ( 2.2e+13 )	1.5e+0 (7.0e–1)	3.7e+13 ( 3.5e+13 )	5.9e–1 (3.3e–1)	2.6e+1 ( 1.7e+1 )	1.9e+0 (1.6e+0)

Appendix

Table 22: The HV of all the solutions obtained by the SPMO using three different single-point metrics on the problems with five objectives on 30 independent runs. The method with the best mean is highlighted in bold. The symbols “

+

”, “

\sim

”, and “

-

” indicate that a method is statistically worse than, equivalent to, and better than SPMO (i.e., SPMO dist ), respectively.

Figure 19: Violin plots of the distance-based metric (log distance) obtained by the SPMO using three different single-point metrics on the problems with five objectives. Each violin represents the distribution of the distance-based metric obtained by a method over 30 independent runs.

Figure 20: Violin plots of the HV of the best solution (in terms of its HV value) obtained by the SPMO using three different single-point metrics on the problems with 5 objectives. Each violin represents the distribution of the single-point HV obtained by a method over 30 independent runs.

Figure 21: Violin plots of the HV of all evaluated solutions obtained by the SPMO using three different single-point metrics on the problems with five objectives. Each violin represents the distribution of maximum HV values obtained by a method over 30 independent runs.

Device\Method	ParEGO	NParEGO	TS-TCH	EHVI	NEHVI	C-EHVI	JES	SPMO (ours)
CPU (3 obj)	3.46	3.49	12.29	4.23	6.15	10.23	9.24	2.40
GPU (3 obj)	5.19	3.94	14.74	3.05	5.73	9.48	14.02	2.24
CPU (5 obj)	2.59	2.14	28.32	36.03	97.98	29.89	25.16	2.58
GPU (5 obj)	9.02	10.33	22.29	10.32	63.90	23.89	25.80	5.31
CPU (10 obj)	8.76	5.97	87.44	1134.35	N/A	77.94	388.37	6.95
GPU (10 obj)	41.33	30.30	64.78	2426.04	N/A	86.45	676.31	24.05

Appendix

Table 23: Mean acquisition optimisation wall time in seconds based on the

2(d+1)

initial Sobol samples on DTLZ1 problems with

m=3,5,10

objectives, where

d=m+4

, over 30 runs. Experiments are conducted using a CPU (Intel Xeon CPU Platinum 8360Y @ 2.40 GHz) and a GPU (NVIDIA A100). Note that N/A means the wall time of NEHVI on DTLZ1 with 10 objectives exceeds 3 hours.

Explore similar work

Converge Then Diversify: Decoupling Convergence and Diversity in Multi-Objective Bayesian Optimisation
Sep 11, 2026Chao Jiang, Yueling Huang, Miqing LiMulti-Objective Optimization Bayesian Optimization
School of Computer Science University of Birmingham
Approximating Pareto Frontiers in Stochastic Multi-Objective Optimization via Hashing and Randomization
Apr 1, 2026Jinzhao Li, Nan Jiang, Yexiang XueMulti-Objective Optimization Pareto Optimization
Department of Computer Science, Purdue University, 610 Purdue Mall, West Lafayette, 47907, IN, United States. · Department of Computer Science, University of Texas at El Paso, 500 W University Ave, El Paso, 79968, TX, United States.
Multi-Objective Bayesian Optimization via Adaptive \varepsilon-Constraints Decomposition
Apr 17, 2026Yaohong Yang, Sammie Katt, Samuel KaskiBayesian Optimization Multi-Objective Optimization
Department of Computer Science, Aalto University, Espoo, Finland · ELLIS Institute Finland · Department of Computer Science, University of Manchester, Manchester, United Kingdom

Sep 11, 2026J/K move · Enter open · S save

Converge Then Diversify: Decoupling Convergence and Diversity in Multi-Objective Bayesian Optimisation

Chao Jiang, Yueling Huang, Miqing Li

School of Computer Science University of Birmingham

Multi-Objective Optimization Bayesian Optimization

Open paper AskPDF arXiv

Multi-objective Bayesian optimisation (MOBO) is a sample-efficient approach for optimising expensive black-box functions with multiple objectives. In MOBO, the goal is to adequately approximate the Pareto front; that is, to obtain a high-quality solution set with 1) good convergence (closeness to the Pareto front) and 2) good diversity (spread across the Pareto front). Existing MOBO methods typically aim to accomplish these two tasks simultaneously, i.e., driving the search towards the Pareto front while maintaining a diverse set of nondominated solutions, such that the solutions, ideally, can gradually approach the entire front. When sufficient search budgets are available, this approach is effective. However, considering both convergence and diversity throughout the search is not easy and requires careful design. Under very tight budgets, there may not be enough solutions generated to be able to simultaneously approach the entire Pareto front. To address this issue, this paper proposes a converge-then-diversify (CTD) approach that decouples convergence and diversity into two stages. In the first stage, CTD focuses on convergence, aiming to quickly drive the search toward a single point on the Pareto front. In the second stage, CTD focuses on diversity, aiming to spread solutions across the front. We present two simple instantiations of CTD by using widely adopted acquisition functions in the area. Experimental results show that, across all 446 pairwise comparisons, CTD statistically outperforms state-of-the-art methods in 72.9% of the cases, performs equivalently in 21.1%, and is statistically worse in only 6.1%, with the advantage being particularly evident in settings with very tight evaluation budgets or in high-dimensional problems.

Abstract

Explore similar work

Converge Then Diversify: Decoupling Convergence and Diversity in Multi-Objective Bayesian Optimisation

Approximating Pareto Frontiers in Stochastic Multi-Objective Optimization via Hashing and Randomization

Multi-Objective Bayesian Optimization via Adaptive \varepsilon-Constraints Decomposition