cs.AIApr 10, 2026

Do We Really Need to Approach the Entire Pareto Front in Many-Objective Bayesian Optimisation?

Authors: Chao Jiang, Jingyu Huang, Miqing Li

Organizations: School of Computer Science, University of Birmingham · School of Mathematics, University of Birmingham

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Many-objective optimisation, a subset of multi-objective optimisation, involves optimisation problems with more than three objectives. As the number of objectives increases, the number of solutions needed to adequately represent the entire Pareto front typically grows substantially. This makes it challenging, if not infeasible, to design a search algorithm capable of effectively exploring the entire Pareto front. This difficulty is particularly acute in the Bayesian optimisation paradigm, where sample efficiency is critical and only a limited number of solutions (often a few hundred) are evaluated. Moreover, after the optimisation process, the decision-maker eventually selects just one solution for deployment, regardless of how many high-quality, diverse solutions are available. In light of this, we argue an idea that under a very limited evaluation budget, it may be more useful to focus on finding a single solution of the highest possible quality for the decision-maker, rather than aiming to approximate the entire Pareto front as existing many-/multi-objective Bayesian optimisation methods typically do. Bearing this idea in mind, this paper proposes a single point-based multi-objective search framework (SPMO) that aims to improve the quality of solutions along a direction that leads to a good tradeoff between objectives. Within SPMO, we present a simple acquisition function, called expected single-point improvement (ESPI), working under both noiseless and noisy scenarios. We show that ESPI can be optimised effectively with gradient-based methods via the sample average approximation (SAA) approach and theoretically prove its convergence guarantees under the SAA. We also empirically demonstrate that the proposed SPMO is computationally tractable and outperforms state-of-the-arts on a wide range of benchmark and real-world problems.

Figures & tables

Figure 1: An illustration of our idea in a bi-objective case, in comparison with existing methods that aim to search for the entire Pareto front. The red points represent (nondominated) solutions that existing methods may obtain, and the blue point represents what our method aims for. It can be seen that the red points are much more diversified, hence, as a whole, having a better hypervolume (HV) value ( Zitzler and Thiele, 1999 ) . However, the blue point has better convergence (i.e., closer to the Pareto front) than any single red point, which may be more likely to be preferred by the decision-maker.
MethodDTLZ1DTLZ2Inverted DTLZ1Inverted DTLZ2Convex DTLZ2Scaled DTLZ2Car side impactSum up
 Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)++ / ∼\sim / −-
Sobol3.7e+0 (3.0e–1) +2.4e–1 (4.6e–2) +4.8e+0 (3.1e–1) +6.0e–1 (5.7e–2) +-3.1e–1 (2.5e–1) +2.3e–1 (5.1e–2) +-1.9e–1 (2.5e–2) +7/ 0/ 0
ParEGO3.4e+0 (1.9e–1) +6.7e–2 (8.1e–2) +3.2e+0 (5.4e–1) +2.5e–1 (1.3e–2) +-1.7e+0 (2.0e–1) +1.3e–1 (8.9e–2) +-3.3e–1 (6.0e–3) +7/ 0/ 0
TS-TCH3.9e+0 (2.2e–1) +2.0e–1 (4.1e–2) +4.8e+0 (3.2e–1) +4.6e–1 (1.2e–2) +-7.1e–1 (2.1e–1) +2.6e–1 (3.8e–2) +-2.7e–1 (1.5e–2) +7/ 0/ 0
EHVI3.5e+0 (9.6e–2) +9.3e–3 (3.5e–3) +4.0e+0 (4.4e–1) +2.3e–1 (5.5e–3) +-1.4e+0 (2.1e–1) +3.1e–1 (6.0e–2) +-3.3e–1 (1.0e–2) +7/ 0/ 0
C-EHVI3.6e+0 (1.4e–1) +3.3e–3 (3.8e–3) +3.9e+0 (4.6e–1) +2.5e–1 (1.8e–2) +-3.5e–1 (2.9e–1) +7.6e–2 (8.0e–2) +-3.3e–1 (1.0e–2) +7/ 0/ 0
Table 1: Results of the distance-based metric (log distance) obtained by the SPMO and the six peer methods on the benchmark problems with 5 objectives and the car side impact design problem on 30 runs. The method with the best mean is highlighted in bold. The symbols “ ++ ”, “ ∼\sim ” and “ −- ” indicate that the method is statistically worse than, equivalent to and better than our SPMO, respectively.
Figure 2: Trajectories of the distance-based metric (log distance) obtained by the seven methods on the benchmark problems with 5 objectives and the car side impact design problem. Each coloured line represents the mean metric value on 30 independent runs (after the initial Sobol samples, represented by the dashed grey line).
MethodDTLZ1DTLZ2Inverted DTLZ1Inverted DTLZ2Convex DTLZ2Scaled DTLZ2Car side impactSum up
 Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)++ / ∼\sim / −-
Sobol8.7e+12 (3.7e+11) +3.5e–2 (1.5e–2) +5.1e+12 (1.1e+12) +8.7e–4 (1.5e–3) +3.8e–1 (1.7e–1) +3.8e–2 (2.1e–2) +2.6e–1 (1.8e–2) +7/ 0/ 0
ParEGO9.1e+12 (1.8e+11) +1.3e–1 (5.6e–2) +8.8e+12 (7.0e+11) +4.0e–2 (2.9e–3) +1.1e+0 (6.2e–2) +8.7e–2 (5.3e–2) +3.6e–1 (2.6e–3) +7/ 0/ 0
TS-TCH8.4e+12 (3.8e+11) +3.2e–2 (1.3e–2) +4.9e+12 (1.2e+12) +7.1e–3 (1.1e–3) +6.5e–1 (1.4e–1) +2.8e–2 (1.5e–2) +3.1e–1 (9.3e–3) +7/ 0/ 0
EHVI9.0e+12 (9.5e+10) +1.9e–1 ( 7.2e–3 ) ∼7.5e+12 (9.7e+11) +4.5e–2 (1.4e–3) +1.0e+0 (9.7e–2) +1.5e–2 (1.4e–2) +3.6e–1 (6.8e–3) +6/ 1/ 0
C-EHVI9.0e+12 (1.0e+11) +1.6e–1 (1.9e–2) +7.7e+12 (7.8e+11) +4.1e–2 (4.1e–3) +4.3e–1 (2.1e–1) +1.2e–1 (5.1e–2) +3.6e–1 (4.4e–3) ∼6/ 1/ 0
Table 2: The HV of the best solution (in terms of its HV value) obtained by SPMO and the peer methods on the benchmark problems with 5 objectives and the car side impact design problem on 30 runs. The method with the best mean is highlighted in bold. The symbols “ ++ ”, “ ∼\sim ” and “ −- ” indicate that the method is statistically worse than, equivalent to and better than our SPMO, respectively.
MethodDTLZ1DTLZ2Inverted DTLZ1Inverted DTLZ2Convex DTLZ2Scaled DTLZ2Car side impactSum up
 Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)++ / ∼\sim / −-
Sobol1.0e+13 (3.1e+10) +7.9e–2 (2.5e–2) +5.6e+12 (8.7e+11) +9.3e–4 (1.5e–3) +5.8e–1 (2.1e–1) +8.6e–2 (3.1e–2) +5.2e–1 (1.0e–2) +7/ 0/ 0
ParEGO1.0e+13 (3.8e+10) -3.6e–1 (2.1e–1) +9.1e+12 (6.2e+11) +1.4e–1 (1.0e–2) -1.5e+0 ( 2.8e–2 ) -2.2e–1 (1.6e–1) +7.4e–1 (1.4e–2) -3/ 0/ 4
TS-TCH1.0e+13 (2.6e+10) +5.9e–2 (2.7e–2) +5.4e+12 (1.0e+12) +2.0e–2 (3.6e–3) +1.0e+0 (1.5e–1) +5.1e–2 (2.7e–2) +6.5e–1 (1.1e–2) -6/ 0/ 1
EHVI1.0e+13 ( 3.3e+8 ) -8.3e–1 ( 6.2e–2 ) -8.2e+12 (6.6e+11) +2.1e–1 ( 1.9e–3 ) -1.5e+0 (6.9e–2) +2.0e–2 (1.7e–2) +7.4e–1 (9.6e–3) -3/ 0/ 4
C-EHVI1.0e+13 (1.6e+11) ∼3.6e–1 (8.5e–2) +8.0e+12 (6.9e+11) +7.2e–2 (9.2e–3) -5.8e–1 (2.7e–1) +2.1e–1 (1.1e–1) +5.9e–1 ( 3.0e–2) -4/ 1/ 2
Table 3: The HV of all the solutions obtained by the seven methods on the seven benchmark (with five objectives) and real-world problems on 30 independent runs. The method with the best mean HV is highlighted in bold. The symbols “ ++ ”, “ ∼\sim ”, and “ −- ” indicate that a method is statistically worse than, equivalent to, and better than SPMO, respectively.
Figure 3: Spider chart of the best solution (in terms of its HV) obtained by the seven methods on the inverted DTLZ1 problem with 5 objectives in a typical run. Each axis in the spider chart represents one objective. Here, the objective values are multiplied by −1-1 in this minimisation problem, such that a solution with a larger area indicates better quality.
Figure 4: Left: Violin plots of the HV values of the best solution (with respect to its HV) obtained by all the methods on the car cab design problem in 30 independent runs. Right: The objective values (normalised and multiplied by −1-1 ) of the best solution (with the highest HV) obtained by each method on the cab design problem in a typical run.
Appendix figures & tables37 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Illustration of the proposed expected single-point improvement (ESPI) in a bi-objective space. The figure shows the utopian point (black star), nondominated points in the current dataset Dn\mathcal{D}^{n} (red dots), and a new candidate point whose true objective values are not yet observed (blue dot). The dashed arc represents the current best Euclidean distance from the utopian point to the nondominated set, denoted as g∗g^{*} (red dotted line). The blue dotted line represents the distance from the new candidate point to the utopian point z∗\bm{z}^{*} , which may improve upon the current best distance g∗g^{*} . When a new point lies within the shaded blue region and is closer to the utopian point, the improvement ISP(⋅)I_{SP}(\cdot) is higher.
ProblemReference pointSuggested by
DTLZ1(400.0,...,400.0)∈Rm(400.0,...,400.0)\in\mathbb{R}^{m}Balandat et al. [2020] , Chugh [2020]
DTLZ2(1.1,…,1.1)∈Rm(1.1,\dots,1.1)\in\mathbb{R}^{m}Balandat et al. [2020] , Daulton et al. [2020] , Ishibuchi et al. [2018]
Inverted DTLZ1(400.0,…,400.0)∈Rm(400.0,\dots,400.0)\in\mathbb{R}^{m}Chugh [2020] , Ishibuchi et al. [2018]
Inverted DTLZ2(1.1,…,1.1)∈Rm(1.1,\dots,1.1)\in\mathbb{R}^{m}Ishibuchi et al. [2018]
Convex DTLZ2(1.1,…,1.1)∈Rm(1.1,\dots,1.1)\in\mathbb{R}^{m}Ishibuchi et al. [2018]
Scaled DTLZ2(1.1∗20,…,1.1∗2m)∈Rm(1.1*2^{0},\dots,1.1*2^{m})\in\mathbb{R}^{m}Ishibuchi et al. [2018]
Appendix
Table 4: Reference points of the six benchmark problems used for hypervolume computation in EHVI and NEHVI, as well as for performance evaluation. Note that the referents points of the two real-world problems, i.e., car side impact design and car cab design, are set to (1.1,...,1.1)∈Rm(1.1,...,1.1)\in\mathbb{R}^{m} in the normalised objective space following the practice in Tanabe and Ishibuchi [2020] the utopian and nadir points are available at https://github.com/ryojitanabe/reproblems/tree/master/ideal_nadir_points ).
MethodDTLZ1 (3)DTLZ2 (3)Inverted DTLZ1 (3)Inverted DTLZ2 (3)Convex DTLZ2 (3)Scaled DTLZ2 (3)Sum up
 Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)++ / ∼\sim / −-
Sobol3.8e+0 (3.3e–1) +2.3e–1 (5.1e–2) +4.4e+0 (4.0e–1) +5.7e–2 (5.3e–2) +-7.7e–2 (1.7e–1) +2.2e–1 (4.8e–2) +6/ 0/ 0
ParEGO3.7e+0 (2.0e–1) +4.7e–3 (3.1e–3) +3.5e+0 (5.7e–1) +-3.0e–1 (7.7e–3) +-9.3e–1 (1.2e–1) +5.2e–3 (4.7e–3) +6/ 0/ 0
TS-TCH3.9e+0 (2.6e–1) +1.4e–1 (7.1e–2) +4.3e+0 (3.6e–1) +-1.1e–1 (2.4e–2) +-3.4e–1 (1.3e–1) +1.5e–1 (3.6e–2) +6/ 0/ 0
EHVI3.6e+0 (1.4e–1) +4.6e–3 (2.5e–3) +4.0e+0 (2.1e–1) +-3.0e–1 (4.3e–3) +-9.1e–1 (1.1e–1) +1.8e–2 (6.0e–2) +6/ 0/ 0
C-EHVI3.6e+0 (1.2e–1) +1.6e–3 (1.7e–3) +4.0e+0 (2.6e–1) +-2.8e–1 (2.9e–2) +-4.5e–1 (2.4e–1) +3.2e–3 (5.5e–3) +6/ 0/ 0
Appendix
Table 5: Results of the distance-based metric (log distance) obtained by the SPMO and the peer methods on the noiseless problems with 3 objectives ( top ) and 10 objectives ( bottom ) on 30 independent runs. The method with the best mean is highlighted in bold. The symbols “ ++ ”, “ ∼\sim ” and “ −- ” indicate that the method is statistically worse than, equivalent to and better than our SPMO, respectively.
MethodDTLZ1 (3)DTLZ2 (3)Inverted DTLZ1 (3)Inverted DTLZ2 (3)Convex DTLZ2 (3)Scaled DTLZ2 (3)Sum up
 Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)++ / ∼\sim / −-
Sobol5.3e+7 (3.3e+6) +3.3e–2 (2.1e–2) +4.4e+7 (5.7e+6) +1.1e–1 (2.5e–2) +1.9e–1 (1.1e–1) +3.7e–2 (1.8e–2) +6/ 0/ 0
ParEGO5.6e+7 (1.5e+6) +1.6e–1 (7.0e–3) +5.5e+7 (4.2e+6) +3.1e–1 (3.7e–3) +7.2e–1 (5.1e–2) +1.6e–1 ( 9.5e–3 ) -5/ 0/ 1
TS-TCH5.3e+7 (3.1e+6) +5.0e–2 (3.1e–2) +4.6e+7 (5.2e+6) +2.0e–1 (1.2e–2) +3.7e–1 (8.4e–2) +4.8e–2 (2.5e–2) +6/ 0/ 0
EHVI5.7e+7 (7.7e+5) +1.6e–1 (4.6e–3) +5.1e+7 (2.1e+6) +3.1e–1 (2.2e–3) +6.8e–1 (4.6e–2) +1.3e–1 (3.4e–2) -5/ 0/ 1
C-EHVI5.7e+7 (8.3e+5) +1.3e–1 (1.3e–2) +5.1e+7 (2.4e+6) +3.0e–1 (1.4e–2) +4.5e–1 (1.4e–1) +1.4e–1 (1.7e–2) -5/ 0/ 1
Appendix
Table 6: The HV of the best solution (in terms of its HV value) obtained by the proposed SPMO and the peer methods on the noiseless problems with 3 objectives ( top ) and 10 objectives ( bottom ) on 30 independent runs. The method with the best mean is highlighted in bold. The symbols “ ++ ”, “ ∼\sim ” and “ −- ” indicate that the method is statistically worse than, equivalent to and better than our SPMO, respectively.
MethodDTLZ1 (3)DTLZ2 (3)Inverted DTLZ1 (3)Inverted DTLZ2 (3)Convex DTLZ2 (3)Scaled DTLZ2 (3)Sum up
 Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)++ / ∼\sim / −-
Sobol6.3e+7 (2.8e+5) ∼4.9e–2 (2.7e–2) +5.2e+7 (2.9e+6) +2.1e–1 (2.6e–2) +2.5e–1 (1.3e–1) +6.2e–2 (2.5e–2) +5/ 1/ 0
ParEGO6.3e+7 (1.1e+6) -5.4e–1 (5.2e–2) -5.8e+7 (2.6e+6) +6.7e–1 (1.1e–2) -1.0e+0 (9.0e–2) ∼5.4e–1 ( 7.5e–2 ) -1/ 1/ 4
TS-TCH6.2e+7 (7.8e+5) ∼7.5e–2 (4.2e–2) +5.2e+7 (3.3e+6) +4.1e–1 (1.2e–2) -5.4e–1 (1.1e–1) +8.8e–2 (4.6e–2) +4/ 1/ 1
EHVI6.4e+7 (1.0e+5) -6.4e–1 ( 2.2e–2 ) -5.8e+7 (2.4e+6) +7.0e–1 ( 3.9e–3 ) -1.0e+0 ( 6.9e–2 ) ∼2.6e–1 (7.7e–2) -1/ 1/ 4
C-EHVI6.3e+7 (6.8e+5) -3.3e–1 (5.4e–2) +5.7e+7 (3.2e+6) +3.2e–1 (3.5e–2) +5.7e–1 (2.0e–1) +3.3e–1 (6.5e–2) -4/ 0/ 2
Appendix
Table 7: The HV of all the solutions obtained by the proposed SPMO and the peer methods on the noiseless problems with 3 objectives ( top ) and 10 objectives ( bottom ) on 30 independent runs, respectively. The method with the best mean is highlighted in bold. The symbols “ ++ ”, “ ∼\sim ”, and “ −- ” indicate that a method is statistically worse than, equivalent to, and better than SPMO, respectively.
Figure 6: Violin plots of the distance-based metric (log distance) obtained by the proposed SPMO and the peer methods on the noiseless problems with 3 and 10 objectives. Each violin represents the distribution of the distance-based metric obtained by a method over 30 independent runs.
Figure 7: Violin plots of the HV of the best solution (in terms of its HV value) obtained by the proposed SPMO and the peer methods on the noiseless problems with 3 and 10 objectives. Each violin represents the distribution of HV values obtained by a method over 30 independent runs.
Figure 8: Violin plots of the HV of all evaluated solutions obtained by the proposed SPMO and the peer methods on the noiseless problems with 3 objectives ( top ) and 10 objectives ( bottom ), respectively. Each violin represents the distribution of HV values obtained by a method over 30 independent runs.
Figure 9: Trajectories of the distance metric obtained by the SPMO and the peer methods on the noiseless problems with 3 and 10 objectives. Each coloured line represents the mean distance of the closest solution to the utopian point on 30 independent runs (after the initial Sobol samples, represented by the dashed grey line).
MethodDTLZ3DTLZ4DTLZ5DTLZ6DTLZ7Sum up
 Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)++ / ∼\sim / −-
Sobol6.3e+0 (1.6e–1) +2.7e–1 (6.0e–2) +2.5e–1 (4.1e–2) +2.2e+0 (2.1e–2) +3.1e+0 (5.0e–2) +5/ 0/ 0
ParEGO5.5e+0 (1.1e–1) ∼2.3e–1 (7.1e–2) +4.0e–2 (3.1e–2) +4.3e–2 (1.6e–1) +2.2e+0 (8.0e–2) +4/ 1/ 0
TS-TCH6.3e+0 (1.6e–1) +2.6e–1 (5.0e–2) +2.3e–1 (5.9e–2) +2.2e+0 (1.6e–2) +3.1e+0 (4.8e–2) +5/ 0/ 0
EHVI5.4e+0 ( 4.6e–2 ) -8.7e–2 (9.4e–2) +3.0e–1 (6.0e–2) +-9.4e–7 (8.4e–7) +1.7e+0 (8.5e–2) +4/ 0/ 1
JES5.5e+0 (5.0e–2) ∼2.3e–1 (8.1e–2) +5.2e–2 (5.8e–2) +2.1e–2 (1.1e–1) +2.2e+0 (5.8e–2) +4/ 1/ 0
Appendix
Table 8: Results of the distance-based metric (log distance) obtained by obtained by the SPMO and the peer methods on DTLZ3–DTLZ7 with 5 objectives on 30 independent runs. The method with the best mean is highlighted in bold. The symbols “ ++ ”, “ ∼\sim ” and “ −- ” indicate that the method is statistically worse than, equivalent to and better than our SPMO, respectively.
MethodDTLZ3DTLZ4DTLZ5DTLZ6DTLZ7Sum up
 Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)++ / ∼\sim / −-
Sobol9.2e+19 (1.1e+18) +2.2e–2 (1.4e–2) +8.4e+4 (9.5e+2) ∼8.7e+3 (8.2e+2) +-0.0e+0 (0.0e+0) +4/ 1/ 0
ParEGO9.7e+19 (5.2e+17) ∼2.5e–2 (3.0e–2) +8.9e+4 ( 1.4e+3 ) -8.8e+4 (3.8e+3) -2.8e+5 (2.7e+4) +2/ 1/ 2
TS-TCH9.3e+19 (1.0e+18) +1.3e–2 (1.1e–2) +8.6e+4 (1.1e+3) -9.3e+3 (9.9e+2) +-0.0e+0 (0.0e+0) +4/ 0/ 1
EHVI9.8e+19 ( 1.1e+17 ) -1.3e–1 (5.6e–2) +8.3e+4 (1.6e+3) +9.0e+4 ( 1.3e–2 ) -3.8e+5 (2.2e+4) ∼2/ 1/ 2
JES9.8e+19 (1.4e+17) -3.6e–2 (3.5e–2) +8.9e+4 (1.5e+3) -8.8e+4 (3.5e+3) -2.8e+5 (2.0e+4) +2/ 0/ 3
Appendix
Table 9: The HV of the best solution (in terms of its HV value) obtained by SPMO and the peer methods on the DTLZ3–DTLZ7 with 5 objectives on 30 independent runs. The method with the best mean is highlighted in bold. The symbols “ ++ ”, “ ∼\sim ” and “ −- ” indicate that the method is statistically worse than, equivalent to and better than our SPMO, respectively.
MethodDTLZ3DTLZ4DTLZ5DTLZ6DTLZ7Sum up
 Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)++ / ∼\sim / −-
Sobol3.9e+20 (1.7e+20) ∼4.9e–2 (2.4e–2) +2.1e+5 (1.1e+5) ∼1.4e+5 (1.7e+4) +-0.0e+0 (0.0e+0) +3/ 2/ 0
ParEGO2.1e+20 (2.1e+20) ∼5.3e–2 (7.4e–2) +2.4e+5 (8.7e+4) ∼3.9e+5 (1.6e+5) -9.1e+5 (4.1e+5) +2/ 2/ 1
TS-TCH5.0e+20 ( 3.0e+20 ) -2.1e–2 (1.8e–2) +1.8e+5 (8.2e+4) +1.8e+5 (4.2e+4) +-0.0e+0 (0.0e+0) +4/ 0/ 1
EHVI1.7e+20 (1.5e+20) ∼9.3e–1 (7.1e–1) ∼3.5e+5 (1.8e+5) -4.2e+5 ( 1.2e+5 ) -1.9e+6 (5.3e+5) ∼0/ 3/ 2
JES2.7e+20 (2.7e+20) ∼8.8e–2 (1.3e–1) +3.6e+5 ( 3.5e+5 ) ∼4.0e+5 (1.6e+5) -1.0e+6 (4.3e+5) +2/ 2/ 1
Appendix
Table 10: The HV of all evaluated solutions obtained by SPMO and the peer methods on DTLZ3–DTLZ7 with 5 objectives on 30 independent runs. The method with the best mean is highlighted in bold. The symbols “ ++ ”, “ ∼\sim ” and “ −- ” indicate that the method is statistically worse than, equivalent to and better than our SPMO, respectively.
MethodDTLZ1DTLZ2Inverted DTLZ1Inverted DTLZ2Convex DTLZ2Scaled DTLZ2Sum up
 Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)++ / ∼\sim / −-
Sobol3.7e+0 (4.2e–1) +1.9e–1 (5.6e–2) +4.7e+0 (3.4e–1) +6.1e–1 (6.1e–2) +-2.9e–1 (2.2e–1) +2.2e–1 (4.4e–2) +6/ 0/ 0
NParEGO3.6e+0 (3.1e–1) +1.0e–1 (8.2e–2) +3.1e+0 (3.8e–1) +1.8e–1 (5.0e–2) +-1.3e+0 (2.7e–1) +5.2e–2 (5.9e–2) +6/ 0/ 0
TS-TCH3.8e+0 (3.5e–1) +2.0e–1 (4.2e–2) +4.9e+0 (3.2e–1) +4.6e–1 (5.5e–2) +-4.7e–1 (2.3e–1) +2.1e–1 (5.0e–2) +6/ 0/ 0
NEHVI3.5e+0 (1.4e–1) +-1.1e–1 (8.1e–2) +4.0e+0 (5.8e–1) +1.5e–1 (3.3e–2) +-1.3e+0 (3.0e–1) +2.9e–1 (9.5e–2) +6/ 0/ 0
JES3.4e+0 (1.2e–1) +1.3e–1 (1.2e–1) +4.5e+0 (1.1e–1) +1.9e–1 (6.5e–2) +-6.9e–1 (3.5e–1) +1.0e–1 (8.7e–2) +6/ 0/ 0
Appendix
Table 11: Results of the distance-based metric (log distance) obtained by the SPMO and the peer methods on the noisy problems with 5 objectives on 30 independent runs. The method with the best mean is highlighted in bold. The symbols “ ++ ”, “ ∼\sim ” and “ −- ” indicate that the method is statistically worse than, equivalent to and better than our SPMO, respectively.
MethodDTLZ1DTLZ2Inverted DTLZ1Inverted DTLZ2Convex DTLZ2Scaled DTLZ2Sum up
 Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)++ / ∼\sim / −-
Sobol8.6e+12 (5.2e+11) +5.1e–2 (3.0e–2) +5.4e+12 (1.2e+12) +8.2e–4 (1.5e–3) +4.0e–1 (1.8e–1) +3.8e–2 (1.9e–2) +6/ 0/ 0
NParEGO9.1e+12 (2.5e+11) +1.0e–1 (4.9e–2) +9.0e+12 (4.1e+11) +5.8e–2 (1.6e–2) +1.2e+0 (2.0e–1) +1.3e–1 (4.0e–2) +6/ 0/ 0
TS-TCH8.5e+12 (5.1e+11) +5.0e–2 (1.4e–2) +4.8e+12 (1.2e+12) +8.1e–3 (5.4e–3) +5.3e–1 (1.7e–1) +3.1e–2 (1.6e–2) +6/ 0/ 0
NEHVI9.1e+12 (1.5e+11) +2.8e–1 (7.6e–2) +7.4e+12 (1.2e+12) +6.7e–2 (1.0e–2) +1.2e+0 (2.1e–1) +1.2e–2 (1.0e–2) +6/ 0/ 0
JES9.0e+12 (1.3e+11) +8.7e–2 (7.2e–2) +6.1e+12 (3.2e+11) +5.8e–2 (2.0e–2) +7.8e–1 (3.1e–1) +1.0e–1 (5.7e–2) +6/ 0/ 0
Appendix
Table 12: The HV of the best solution (in terms of its HV value) obtained by SPMO and the peer methods on the noisy problems with 5 objectives on 30 independent runs. The method with the best mean is highlighted in bold. The symbols “ ++ ”, “ ∼\sim ” and “ −- ” indicate that the method is statistically worse than, equivalent to and better than our SPMO, respectively.
MethodDTLZ1DTLZ2Inverted DTLZ1Inverted DTLZ2Convex DTLZ2Scaled DTLZ2Sum up
 Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)++ / ∼\sim / −-
Sobol4.5e+13 (1.4e+13) +2.1e–1 (8.4e–2) +8.2e+12 (4.7e+12) +8.5e–4 (1.5e–3) +9.3e–1 (5.1e–1) +1.6e–1 (5.7e–2) +6/ 0/ 0
NParEGO5.8e+13 (2.3e+13) ∼4.3e–1 (3.9e–1) +1.7e+13 (9.6e+12) +5.1e–1 (1.3e–1) +1.1e+1 (4.8e+0) +7.7e–1 (6.5e–1) +5/ 1/ 0
TS-TCH5.1e+13 (1.3e+13) ∼1.9e–1 (8.3e–2) +6.9e+12 (2.6e+12) +3.2e–2 (1.8e–2) +2.2e+0 (9.8e–1) +9.0e–2 (6.5e–2) +5/ 1/ 0
NEHVI7.2e+13 (2.1e+13) ∼3.1e+0 ( 8.9e–1 ) -1.5e+13 (1.7e+13) +7.7e–1 (2.0e–1) +9.5e+0 (3.8e+0) +1.5e–2 (1.3e–2) +4/ 1/ 1
JES7.5e+13 ( 2.3e+13 ) ∼4.3e–1 (5.0e–1) +3.5e+13 (2.0e+13) ∼5.7e–1 (1.5e–1) +4.6e+0 (2.8e+0) +4.3e–1 (3.3e–1) +4/ 2/ 0
Appendix
Table 13: The HV of all the solutions obtained by the proposed SPMO and the peer methods on the noisy problems with five objectives on 30 independent runs. The method with the best mean is highlighted in bold. The symbols “ ++ ”, “ ∼\sim ”, and “ −- ” indicate that a method is statistically worse than, equivalent to, and better than SPMO, respectively.
Figure 10: Violin plots of the distance-based metric (log distance) obtained by the six methods on the noisy problems with five objectives. Each violin represents the distribution of the distance-based metric obtained by a method over 30 independent runs.
Figure 11: Violin plots of the HV of the best solution (in terms of its HV value) obtained by the six methods on the noisy problems with five objectives. Each violin represents the distribution of maximum HV values obtained by a method over 30 independent runs.
Figure 12: Violin plots of the HV of all evaluated solutions obtained by the six methods on the noisy problems with five objectives. Each violin represents the distribution of maximum HV values obtained by a method over 30 independent runs.
MethodDTLZ1DTLZ2Inverted DTLZ1Inverted DTLZ2Convex DTLZ2Scaled DTLZ2Sum up
 Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)++ / ∼\sim / −-
Sobol3.7e+0 (3.0e–1) +2.4e–1 (4.6e–2) +4.8e+0 (3.1e–1) +6.0e–1 (5.7e–2) +-3.1e–1 (2.5e–1) +2.3e–1 (5.1e–2) +6/ 0/ 0
ParEGO3.4e+0 (1.9e–1) +6.7e–2 (8.1e–2) +3.2e+0 (5.4e–1) +2.5e–1 (1.3e–2) +-1.7e+0 (2.0e–1) +1.3e–1 (8.9e–2) +6/ 0/ 0
TS-TCH3.9e+0 (2.2e–1) +2.0e–1 (4.1e–2) +4.8e+0 (3.2e–1) +4.6e–1 (1.2e–2) +-7.1e–1 (2.1e–1) +2.6e–1 (3.8e–2) +6/ 0/ 0
EHVI3.5e+0 (9.6e–2) +9.3e–3 (3.5e–3) +4.0e+0 (4.4e–1) +2.3e–1 (5.5e–3) +-1.4e+0 (2.1e–1) +3.1e–1 (6.0e–2) +6/ 0/ 0
JES3.4e+0 (1.3e–1) +1.1e–1 (8.0e–2) +4.5e+0 (1.7e–1) +2.6e–1 (2.0e–2) +-1.0e+0 (4.5e–1) +8.8e–2 (9.3e–2) +6/ 0/ 0
Appendix
Table 14: Results of the distance-based metric (log distance) obtained by the SPMO and the five peer methods with a batch size q=5q=5 on the problems with 5 objectives on 30 independent runs. The method with the best mean is highlighted in bold. The symbols “ ++ ”, “ ∼\sim ” and “ −- ” indicate that the method is statistically worse than, equivalent to and better than our SPMO, respectively.
MethodDTLZ1DTLZ2Inverted DTLZ1Inverted DTLZ2Convex DTLZ2Scaled DTLZ2Sum up
 Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)++ / ∼\sim / −-
Sobol8.7e+12 (3.7e+11) +3.5e–2 (1.5e–2) +5.1e+12 (1.1e+12) +8.7e–4 (1.5e–3) +3.8e–1 (1.7e–1) +3.8e–2 (2.1e–2) +6/ 0/ 0
ParEGO9.1e+12 (1.8e+11) +1.3e–1 (5.6e–2) +8.8e+12 (7.0e+11) +4.0e–2 (2.9e–3) +1.1e+0 (6.2e–2) +8.7e–2 (5.3e–2) +6/ 0/ 0
TS-TCH8.4e+12 (3.8e+11) +3.2e–2 (1.3e–2) +4.9e+12 (1.2e+12) +7.1e–3 (1.1e–3) +6.5e–1 (1.4e–1) +2.8e–2 (1.5e–2) +6/ 0/ 0
EHVI9.0e+12 (9.5e+10) +1.9e–1 ( 7.2e–3 ) ∼7.5e+12 (9.7e+11) +4.5e–2 (1.4e–3) +1.0e+0 (9.7e–2) +1.5e–2 (1.4e–2) +5/ 1/ 0
JES9.0e+12 (1.2e+11) +1.0e–1 (3.9e–2) +6.2e+12 (5.0e+11) +3.9e–2 (4.5e–3) +8.5e–1 (2.4e–1) +1.1e–1 (5.5e–2) +6/ 0/ 0
Appendix
Table 15: The HV of the best solution (in terms of its HV value) obtained by SPMO and the five peer methods with a batch size q=5q=5 on the problems with 5 objectives on 30 independent runs. The method with the best mean is highlighted in bold. The symbols “ ++ ”, “ ∼\sim ” and “ −- ” indicate that the method is statistically worse than, equivalent to and better than our SPMO, respectively.
MethodDTLZ1DTLZ2Inverted DTLZ1Inverted DTLZ2Convex DTLZ2Scaled DTLZ2Sum up
 Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)++ / ∼\sim / −-
Sobol4.4e+13 (1.4e+13) ∼1.4e–1 (5.6e–2) +7.7e+12 (2.9e+12) +9.5e–4 (1.6e–3) +9.5e–1 (4.6e–1) +1.6e–1 (6.6e–2) +5/ 1/ 0
ParEGO2.9e+13 (1.2e+13) ∼6.5e–1 (5.3e–1) +1.5e+13 (7.8e+12) +5.0e–1 (9.6e–2) ∼9.6e+0 (4.8e+0) +4.2e–1 (3.9e–1) +4/ 2/ 0
TS-TCH4.5e+13 ( 1.3e+13 ) ∼8.9e–2 (5.3e–2) +6.8e+12 (1.8e+12) +5.0e–2 (1.7e–2) +3.2e+0 (1.7e+0) +7.4e–2 (5.2e–2) +5/ 1/ 0
EHVI2.9e+13 (1.8e+13) ∼2.1e+0 ( 5.9e–1 ) -1.9e+13 (1.7e+13) +6.1e–1 ( 1.5e–1 ) ∼7.4e+0 (2.2e+0) +2.0e–2 (1.7e–2) +3/ 2/ 1
JES3.1e+13 (2.8e+13) ∼6.9e–1 (5.2e–1) +7.4e+13 ( 9.1e+13 ) -4.7e–1 (1.1e–1) ∼5.7e+0 (4.6e+0) +7.5e–1 (6.3e–1) +3/ 2/ 1
Appendix
Table 16: The HV of all the solutions obtained by the six methods on the problems with five objectives on 30 independent runs. The method with the best mean is highlighted in bold. The symbols “ ++ ”, “ ∼\sim ”, and “ −- ” indicate that a method is statistically worse than, equivalent to, and better than SPMO, respectively.
Figure 13: Violin plots of the distance-based metric (log distance) obtained by the proposed SPMO and the peer methods on the problems with five objectives. Each violin represents the distribution of the distance-based metric obtained by a method over 30 independent runs.
Figure 14: Violin plots of the HV of the best solution (in terms of its HV value) obtained by the six methods with a batch size q=5q=5 on the problems with five objectives. Each violin represents the distribution of maximum HV values obtained by a method over 30 independent runs.
Figure 15: Violin plots of the HV of all evaluated solutions obtained by the six methods on the problems with five objectives. Each violin represents the distribution of maximum HV values obtained by a method over 30 independent runs.
MethodDTLZ1DTLZ2Inverted DTLZ1Inverted DTLZ2Convex DTLZ2Scaled DTLZ2Sum up
 Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)++ / ∼\sim / −-
SPMO_0.013.1e+0 (4.9e–1) ∼6.2e–4 (1.4e–3) -2.8e+0 ( 4.2e–1 ) ∼2.1e–1 (3.6e–5) ∼-2.1e+0 (1.6e–2) ∼6.4e–5 (3.9e–5) -0/ 4/ 2
SPMO_0.12.9e+0 (3.6e–1) -3.0e–4 ( 2.3e–4 ) -3.0e+0 (5.3e–1) ∼2.1e–1 (4.6e–5) ∼-2.1e+0 (2.6e–2) ∼5.3e–5 ( 2.4e–5 ) -0/ 3/ 3
SPMO_1.02.8e+0 ( 6.1e–1 ) ∼3.3e–4 (2.0e–4) -2.9e+0 (5.0e–1) ∼2.1e–1 ( 3.4e–5 ) ∼-2.1e+0 (1.6e–2) ∼5.7e–5 (2.4e–5) -0/ 4/ 2
SPMO3.1e+0 (3.0e–1)9.0e–4 (8.3e–4)2.9e+0 (4.9e–1)2.1e–1 (5.0e–5)-2.1e+0 ( 7.7e–3 )1.7e–4 (1.2e–4) 
Appendix
Table 17: Results of the distance-based metric (log distance) obtained by the SPMO with four different utopian points on the problems with five objectives on 30 independent runs. The method with the best mean is highlighted in bold. The symbols “ ++ ”, “ ∼\sim ” and “ −- ” indicate that the method is statistically worse than, equivalent to and better than SPMO (current), respectively.
MethodDTLZ1DTLZ2Inverted DTLZ1Inverted DTLZ2Convex DTLZ2Scaled DTLZ2Sum up
 Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)++ / ∼\sim / −-
SPMO_0.019.4e+12 (3.7e+11) ∼1.9e–1 ( 1.0e–2 ) ∼9.3e+12 ( 3.8e+11 ) ∼4.9e–2 (9.0e–6) ∼1.3e+0 ( 7.8e–3 ) ∼1.6e–1 (1.4e–2) ∼0/ 6/ 0
SPMO_0.19.5e+12 (2.7e+11) -1.9e–1 (1.6e–2) ∼9.0e+12 (5.6e+11) ∼4.9e–2 (1.1e–5) ∼1.2e+0 (1.1e–2) ∼1.6e–1 (1.0e–2) ∼0/ 5/ 1
SPMO_1.09.5e+12 ( 3.8e+11 ) -1.9e–1 (1.3e–2) ∼9.1e+12 (4.2e+11) ∼4.9e–2 ( 8.5e–6 ) ∼1.3e+0 (7.8e–3) ∼1.6e–1 (1.4e–2) ∼0/ 5/ 1
SPMO9.3e+12 (2.8e+11)1.9e–1 (1.4e–2)9.2e+12 (4.8e+11)4.9e–2 (1.2e–5)1.3e+0 (5.0e–3)1.6e–1 ( 1.6e–2 ) 
Appendix
Table 18: The HV of the best solution (in terms of its HV value) obtained by the proposed SPMO with four different utopian points on the problems with five objectives on 30 independent runs. The method exhibiting the best mean is highlighted in bold. The symbols “ ++ ”, “ ∼\sim ”, and “ −- ” denote that a method is statistically worse than, equivalent to, or better than SPMO (current), respectively.
MethodDTLZ1DTLZ2Inverted DTLZ1Inverted DTLZ2Convex DTLZ2Scaled DTLZ2Sum up
 Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)++ / ∼\sim / −-
SPMO_0.013.1e+13 (1.7e+13) ∼2.7e+0 (1.2e+0) -3.5e+13 (3.7e+13) ∼6.5e–1 (7.0e–1) ∼2.5e+1 (1.8e+1) ∼2.2e+0 ( 1.5e+0 ) ∼0/ 5/ 1
SPMO_0.13.5e+13 (1.5e+13) ∼2.7e+0 (1.2e+0) -2.6e+13 (1.7e+13) ∼6.1e–1 (4.5e–1) ∼2.7e+1 (1.7e+1) ∼1.8e+0 (1.0e+0) ∼0/ 5/ 1
SPMO_1.03.3e+13 (1.4e+13) ∼3.0e+0 ( 1.6e+0 ) -3.2e+13 (1.9e+13) ∼7.1e–1 ( 6.3e–1 ) ∼2.9e+1 ( 2.2e+1 ) ∼2.1e+0 (1.5e+0) ∼0/ 5/ 1
SPMO3.7e+13 ( 2.2e+13 )1.5e+0 (7.0e–1)3.7e+13 ( 3.5e+13 )5.9e–1 (3.3e–1)2.6e+1 (1.7e+1)1.9e+0 (1.6e+0) 
Appendix
Table 19: The HV of all the solutions obtained by the SPMO with four different utopian points on the problems with five objectives on 30 independent runs. The method with the best mean is highlighted in bold. The symbols “ ++ ”, “ ∼\sim ”, and “ −- ” indicate that a method is statistically worse than, equivalent to, and better than SPMO (current), respectively.
Figure 16: Violin plots of the distance-based metric (log distance) obtained by the four methods on the problems with five objectives. Each violin represents the distribution of the distance-based metric obtained by a method over 30 independent runs.
Figure 17: Violin plots of the HV of the best solution (in terms of its HV value) obtained by the four methods on the problems with five objectives. Each violin represents the distribution of maximum HV values obtained by a method over 30 independent runs.
Figure 18: Violin plots of the HV of all evaluated solutions obtained by the four methods the problems with five objectives. Each violin represents the distribution of maximum HV values obtained by a method over 30 independent runs.
MethodDTLZ1DTLZ2Inverted DTLZ1Inverted DTLZ2Convex DTLZ2Scaled DTLZ2Sum up
 Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)++ / ∼\sim / −-
SPMO Tch3.5e+0 (3.2e–1) +3.5e–5 ( 4.3e–5 ) -2.9e+0 (6.5e–1) ∼3.1e–1 (3.3e–2) +-1.8e+0 (1.7e–1) +2.9e–5 ( 1.9e–5 ) -3/ 1/ 2
SPMO ws3.6e+0 (1.3e–1) +2.1e–4 (1.5e–4) -3.1e+0 (5.4e–1) ∼2.2e–1 (2.7e–3) +-1.2e+0 (2.7e–1) +3.2e–4 (3.4e–4) +4/ 1/ 1
SPMO3.1e+0 ( 3.0e–1 )9.0e–4 (8.3e–4)2.9e+0 ( 4.9e–1 )2.1e–1 ( 5.0e–5 )-2.1e+0 ( 7.7e–3 )1.7e–4 (1.2e–4) 
Appendix
Table 20: Results of the distance-based metric (log distance) obtained by the SPMO using three different single-point metrics on the problems with five objectives on 30 independent runs. The method with the best mean is highlighted in bold. The symbols “ ++ ”, “ ∼\sim ” and “ −- ” indicate that the method is statistically worse than, equivalent to and better than our SPMO (i.e., SPMO dist ), respectively.
MethodDTLZ1DTLZ2Inverted DTLZ1Inverted DTLZ2Convex DTLZ2Scaled DTLZ2Sum up
 Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)++ / ∼\sim / −-
SPMO Tch9.1e+12 (3.0e+11) +1.6e–1 (1.9e–2) +9.1e+12 (5.7e+11) ∼2.7e–2 (5.9e–3) +1.2e+0 (4.2e–2) +1.6e–1 (2.2e–2) ∼4/ 2/ 0
SPMO ws9.1e+12 (1.3e+11) +1.7e–1 (1.4e–2) +9.0e+12 (6.5e+11) ∼4.8e–2 (6.7e–4) +9.7e–1 (1.2e–1) +1.7e–1 ( 1.5e–2 ) ∼4/ 2/ 0
SPMO9.3e+12 ( 2.8e+11 )1.9e–1 ( 1.4e–2 )9.2e+12 ( 4.8e+11 )4.9e–2 ( 1.2e–5 )1.3e+0 ( 5.0e–3 )1.6e–1 (1.6e–2) 
Appendix
Table 21: The HV of the best solution (in terms of its HV value) obtained by the proposed SPMO using three different single-point metrics on the problems with five objectives on 30 independent runs. The method exhibiting the best mean is highlighted in bold. The symbols “ ++ ”, “ ∼\sim ”, and “ −- ” denote that a method is statistically worse than, equivalent to, or better than SPMO (i.e., SPMO dist ), respectively.
MethodDTLZ1DTLZ2Inverted DTLZ1Inverted DTLZ2Convex DTLZ2Scaled DTLZ2Sum up
 Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)Mean (Std)++ / ∼\sim / −-
SPMO Tch1.5e+13 (6.2e+12) +4.9e+0 ( 7.6e+0 ) -2.1e+13 (1.5e+13) +2.1e+0 ( 2.6e+0 ) -1.3e+1 (7.4e+0) +3.3e+0 ( 3.4e+0 ) -3/ 0/ 3
SPMO ws1.3e+13 (3.1e+12) +1.6e+0 (7.5e–1) ∼2.4e+13 (1.8e+13) +7.4e–1 (7.5e–1) ∼6.4e+0 (3.8e+0) +1.2e+0 (3.5e–1) ∼3/ 3/ 0
SPMO3.7e+13 ( 2.2e+13 )1.5e+0 (7.0e–1)3.7e+13 ( 3.5e+13 )5.9e–1 (3.3e–1)2.6e+1 ( 1.7e+1 )1.9e+0 (1.6e+0) 
Appendix
Table 22: The HV of all the solutions obtained by the SPMO using three different single-point metrics on the problems with five objectives on 30 independent runs. The method with the best mean is highlighted in bold. The symbols “ ++ ”, “ ∼\sim ”, and “ −- ” indicate that a method is statistically worse than, equivalent to, and better than SPMO (i.e., SPMO dist ), respectively.
Figure 19: Violin plots of the distance-based metric (log distance) obtained by the SPMO using three different single-point metrics on the problems with five objectives. Each violin represents the distribution of the distance-based metric obtained by a method over 30 independent runs.
Figure 20: Violin plots of the HV of the best solution (in terms of its HV value) obtained by the SPMO using three different single-point metrics on the problems with 5 objectives. Each violin represents the distribution of the single-point HV obtained by a method over 30 independent runs.
Figure 21: Violin plots of the HV of all evaluated solutions obtained by the SPMO using three different single-point metrics on the problems with five objectives. Each violin represents the distribution of maximum HV values obtained by a method over 30 independent runs.
Device\MethodParEGONParEGOTS-TCHEHVINEHVIC-EHVIJESSPMO (ours)
CPU (3 obj)3.463.4912.294.236.1510.239.242.40
GPU (3 obj)5.193.9414.743.055.739.4814.022.24
CPU (5 obj)2.592.1428.3236.0397.9829.8925.162.58
GPU (5 obj)9.0210.3322.2910.3263.9023.8925.805.31
CPU (10 obj)8.765.9787.441134.35N/A77.94388.376.95
GPU (10 obj)41.3330.3064.782426.04N/A86.45676.3124.05
Appendix
Table 23: Mean acquisition optimisation wall time in seconds based on the 2(d+1)2(d+1) initial Sobol samples on DTLZ1 problems with m=3,5,10m=3,5,10 objectives, where d=m+4d=m+4 , over 30 runs. Experiments are conducted using a CPU (Intel Xeon CPU Platinum 8360Y @ 2.40 GHz) and a GPU (NVIDIA A100). Note that N/A means the wall time of NEHVI on DTLZ1 with 10 objectives exceeds 3 hours.

Explore similar work

CardsList
Sep 11, 2026

Converge Then Diversify: Decoupling Convergence and Diversity in Multi-Objective Bayesian Optimisation

Multi-objective Bayesian optimisation (MOBO) is a sample-efficient approach for optimising expensive black-box functions with multiple objectives. In MOBO, the goal is to adequately approximate the Pareto front; that is, to obtain a high-quality solution set with 1) good convergence (closeness to the Pareto front) and 2) good diversity (spread across the Pareto front). Existing MOBO methods typically aim to accomplish these two tasks simultaneously, i.e., driving the search towards the Pareto front while maintaining a diverse set of nondominated solutions, such that the solutions, ideally, can gradually approach the entire front. When sufficient search budgets are available, this approach is effective. However, considering both convergence and diversity throughout the search is not easy and requires careful design. Under very tight budgets, there may not be enough solutions generated to be able to simultaneously approach the entire Pareto front. To address this issue, this paper proposes a converge-then-diversify (CTD) approach that decouples convergence and diversity into two stages. In the first stage, CTD focuses on convergence, aiming to quickly drive the search toward a single point on the Pareto front. In the second stage, CTD focuses on diversity, aiming to spread solutions across the front. We present two simple instantiations of CTD by using widely adopted acquisition functions in the area. Experimental results show that, across all 446 pairwise comparisons, CTD statistically outperforms state-of-the-art methods in 72.9% of the cases, performs equivalently in 21.1%, and is statistically worse in only 6.1%, with the advantage being particularly evident in settings with very tight evaluation budgets or in high-dimensional problems.

Chao Jiang, Yueling Huang, Miqing Li

School of Computer Science University of Birmingham

Multi-Objective OptimizationBayesian Optimization
AskPDF
Apr 1, 2026

Approximating Pareto Frontiers in Stochastic Multi-Objective Optimization via Hashing and Randomization

Stochastic Multi-Objective Optimization (SMOO) is critical for decision-making trading off multiple potentially conflicting objectives in uncertain environments. SMOO aims at identifying the Pareto frontier, which contains all mutually non-dominating decisions. The problem is highly intractable due to the embedded probabilistic inference, such as computing the marginal, posterior probabilities, or expectations. Existing methods, such as scalarization, sample average approximation, and evolutionary algorithms, either offer arbitrarily loose approximations or may incur prohibitive computational costs. We propose XOR-SMOO, a novel algorithm that with probability 1−δ1-δ, obtains γγ-approximate Pareto frontiers (γ>1γ>1) for SMOO by querying an SAT oracle poly-log times in γγ and δδ. A γγ-approximate Pareto frontier is only below the true frontier by a fixed, multiplicative factor γγ. Thus, XOR-SMOO solves highly intractable SMOO problems (#P-hard) with only queries to SAT oracles while obtaining tight, constant factor approximation guarantees. Experiments on real-world road network strengthening and supply chain design problems demonstrate that XOR-SMOO outperforms several baselines in identifying Pareto frontiers that have higher objective values, better coverage of the optimal solutions, and the solutions found are more evenly distributed. Overall, XOR-SMOO significantly enhanced the practicality and reliability of SMOO solvers.

Jinzhao Li, Nan Jiang, Yexiang Xue

Department of Computer Science, Purdue University, 610 Purdue Mall, West Lafayette, 47907, IN, United States. · Department of Computer Science, University of Texas at El Paso, 500 W University Ave, El Paso, 79968, TX, United States.

Multi-Objective OptimizationPareto OptimizationApproximation Algorithms
AskPDF
Apr 17, 2026

Multi-Objective Bayesian Optimization via Adaptive \varepsilon-Constraints Decomposition

Multi-objective Bayesian optimization (MOBO) provides a principled framework for optimizing multiple expensive black-box functions. However, existing MOBO methods often struggle with coverage, scalability, and handling constraints and preferences. In this work we propose STAGE-BO, Sequential Targeting Adaptive Gap-Filling ε\varepsilon-Constraint Bayesian Optimization: by analyzing the coverage of the surrogate Pareto front, our method identifies the Pareto front point with the largest uncovered gap, and uses its coordinates to define adaptive constraints in ε\varepsilon-constraint method, which transforms the problem into a sequence of inequality-constrained subproblems, efficiently solved via constrained expected improvement acquisition. Our approach provides uniform Pareto coverage without hypervolume computation and naturally handles constraints and preferences. Experiments on synthetic and real-world benchmarks demonstrate superior coverage and competitive hypervolume performance against state-of-the-art baselines. Our code implementation can be found at https://github.com/YangYaohong1/STAGE-BO.

Yaohong Yang, Sammie Katt, Samuel Kaski

Department of Computer Science, Aalto University, Espoo, Finland · ELLIS Institute Finland · Department of Computer Science, University of Manchester, Manchester, United Kingdom

Bayesian OptimizationMulti-Objective Optimization
AskPDF