Converge Then Diversify: Decoupling Convergence and Diversity in Multi-Objective Bayesian Optimisation

Chao Jiang; Yueling Huang; Miqing Li

cs.AISep 11, 2026

Converge Then Diversify: Decoupling Convergence and Diversity in Multi-Objective Bayesian Optimisation

Authors: Chao Jiang, Yueling Huang, Miqing Li

Organizations: School of Computer Science University of Birmingham

Abstract

Multi-objective Bayesian optimisation (MOBO) is a sample-efficient approach for optimising expensive black-box functions with multiple objectives. In MOBO, the goal is to adequately approximate the Pareto front; that is, to obtain a high-quality solution set with 1) good convergence (closeness to the Pareto front) and 2) good diversity (spread across the Pareto front). Existing MOBO methods typically aim to accomplish these two tasks simultaneously, i.e., driving the search towards the Pareto front while maintaining a diverse set of nondominated solutions, such that the solutions, ideally, can gradually approach the entire front. When sufficient search budgets are available, this approach is effective. However, considering both convergence and diversity throughout the search is not easy and requires careful design. Under very tight budgets, there may not be enough solutions generated to be able to simultaneously approach the entire Pareto front. To address this issue, this paper proposes a converge-then-diversify (CTD) approach that decouples convergence and diversity into two stages. In the first stage, CTD focuses on convergence, aiming to quickly drive the search toward a single point on the Pareto front. In the second stage, CTD focuses on diversity, aiming to spread solutions across the front. We present two simple instantiations of CTD by using widely adopted acquisition functions in the area. Experimental results show that, across all 446 pairwise comparisons, CTD statistically outperforms state-of-the-art methods in 72.9% of the cases, performs equivalently in 21.1%, and is statistically worse in only 6.1%, with the advantage being particularly evident in settings with very tight evaluation budgets or in high-dimensional problems.

Figures & tables

Panel of Figure 1 — Figure 1: Illustration of the search processes of the conventional MOBO approach and the proposed approach. (a) The conventional approach that moves towards the Pareto front while simultaneously spreading nondominated solutions. (b) The proposed convergence-then-diversity strategy that first moves directly towards the Pareto front and then spreads along it.

Method	DTLZ1 ( $m=2$ )		DTLZ2 ( $m=2$ )		DTLZ3 ( $m=2$ )		DTLZ4 ( $m=2$ )		DTLZ5 ( $m=2$ )		DTLZ6 ( $m=2$ )		DTLZ7 ( $m=2$ )
	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std
Sobol	0.00e+00	0.00e+00 ( $\sim$ )	1.23e-02	1.82e-02 ( $+$ )	0.00e+00	0.00e+00 ( $\sim$ )	1.34e-02	2.21e-02 ( $+$ )	1.20e-02	1.33e-02 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )
ParEGO	0.00e+00	0.00e+00 ( $\sim$ )	2.97e-01	7.15e-02 ( $+$ )	1.94e+02	5.61e+02 ( $\sim$ )	2.84e-01	6.52e-02 ( $+$ )	2.39e-01	1.06e-01 ( $+$ )	2.67e-01	1.34e-01 ( $\sim$ )	3.86e-01	2.38e-01 ( $-$ )
TS-TCH	0.00e+00	0.00e+00 ( $\sim$ )	3.68e-02	3.99e-02 ( $+$ )	0.00e+00	0.00e+00 ( $\sim$ )	3.47e-02	3.43e-02 ( $+$ )	4.32e-02	4.18e-02 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )
PSL	7.92e+00	2.82e+01 ( $\sim$ )	8.61e-02	5.44e-02 ( $+$ )	2.66e+02	7.12e+02 ( $\sim$ )	9.88e-02	3.64e-02 ( $+$ )	8.61e-02	5.44e-02 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	4.64e-01	1.71e-01 ( $-$ )
JES	0.00e+00	0.00e+00 ( $\sim$ )	2.43e-01	6.00e-02 ( $+$ )	5.37e+02	6.34e+02 ( $\sim$ )	2.65e-01	5.99e-02 ( $+$ )	2.74e-01	4.40e-02 ( $+$ )	2.22e-01	7.26e-02 ( $+$ )	3.72e-01	1.86e-01 ( $-$ )

Table 1: The HV results (mean and standard deviation) of the ten methods on the 20 benchmark and real-world problems. The method with the best mean HV is highlighted in bold. The symbols “

-

”, “

\sim

” and “

+

” indicate that our method CTD is statistically worse than, equivalent to, and better than the competitor, respectively.

Figure 2: Mean hypervolume trajectories of the ten methods on the representative DTLZ2 problem (

m=2

d=11

) over 30 independent runs. The shaded regions indicate one standard deviation around the mean.

Panel of Figure 3 — Figure 3: Solutions obtained by the proposed CTD and the nine peer methods on DTLZ2 ( $m=2$ , $d=11$ ) under budgets of 40 and 100 function evaluations. All methods start from the same $2(d+1)$ initial points generated by a scrambled Sobol sequence. The black line denotes the Pareto front, and brighter colours indicate solutions obtained at later evaluations.

Method	DTLZ1 ( $m=2$ )		DTLZ2 ( $m=2$ )		DTLZ3 ( $m=2$ )		DTLZ4 ( $m=2$ )		DTLZ5 ( $m=2$ )		DTLZ6 ( $m=2$ )		DTLZ7 ( $m=2$ )
	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std
Sobol	0.00e+00	0.00e+00 ( $\sim$ )	1.54e-02	1.39e-02 ( $+$ )	0.00e+00	0.00e+00 ( $\sim$ )	1.37e-02	1.66e-02 ( $+$ )	1.17e-02	1.61e-02 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $\sim$ )
NParEGO	0.00e+00	0.00e+00 ( $\sim$ )	6.94e-02	4.58e-02 ( $\sim$ )	3.98e+02	4.96e+02 ( $\sim$ )	5.68e-02	6.14e-02 ( $\sim$ )	7.16e-02	6.93e-02 ( $\sim$ )	1.36e-01	9.29e-02 ( $\sim$ )	1.39e-02	7.63e-02 ( $\sim$ )
TS-TCH	0.00e+00	0.00e+00 ( $\sim$ )	2.76e-03	4.60e-03 ( $+$ )	0.00e+00	0.00e+00 ( $\sim$ )	2.92e-03	8.28e-03 ( $+$ )	5.82e-03	1.47e-02 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $\sim$ )
JES	0.00e+00	0.00e+00 ( $\sim$ )	5.95e-02	5.53e-02 ( $\sim$ )	3.24e+02	7.44e+02 ( $\sim$ )	3.93e-02	4.22e-02 ( $\sim$ )	4.20e-02	4.51e-02 ( $\sim$ )	1.12e-01	9.06e-02 ( $\sim$ )	0.00e+00	0.00e+00 ( $\sim$ )
NEHVI	0.00e+00	0.00e+00 ( $\sim$ )	2.38e-02	4.35e-02 ( $+$ )	4.40e+02	6.41e+02 ( $\sim$ )	2.82e-02	4.75e-02 ( $+$ )	3.20e-02	5.52e-02 ( $+$ )	1.71e-01	6.73e-02 ( $\sim$ )	1.50e-02	7.13e-02 ( $\sim$ )

Table 2: The HV results (mean and standard deviation) of the eight methods on the 20 benchmark and real-world problems under noisy settings. The method with the best mean HV is highlighted in bold. The symbols “

-

”, “

\sim

” and “

+

” indicate that the proposed CTD is statistically worse than, equivalent to, and better than the competitor, respectively.

Figure 4: Solutions obtained by the proposed CTD and the nine peer methods on DTLZ2 (

m=2

d=50

) under 200 evaluations. All methods start from the same

2(d+1)

initial points generated by a scrambled Sobol sequence. The black line denotes the Pareto front, and brighter colours indicate solutions obtained at later evaluations.

Method	DTLZ1 ( $m=2$ )		DTLZ2 ( $m=2$ )		DTLZ3 ( $m=2$ )		DTLZ4 ( $m=2$ )		DTLZ5 ( $m=2$ )		DTLZ6 ( $m=2$ )		DTLZ7 ( $m=2$ )
	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std
CTD ${(10^{-1})}$	0.00e+00	0.00e+00 ( $\sim$ )	1.93e-01	1.07e-01 ( $+$ )	5.42e+02	7.71e+02 ( $\sim$ )	2.21e-01	8.52e-02 ( $+$ )	2.20e-01	9.21e-02 ( $+$ )	3.26e-01	4.89e-02 ( $\sim$ )	1.73e-01	2.61e-02 ( $\sim$ )
CTD ${(10^{-2})}$	1.92e+00	1.05e+01 ( $\sim$ )	3.17e-01	5.62e-02 ( $\sim$ )	4.93e+02	1.06e+03 ( $\sim$ )	3.16e-01	4.19e-02 ( $+$ )	3.17e-01	4.51e-02 ( $+$ )	3.07e-01	5.16e-02 ( $\sim$ )	1.71e-01	2.43e-02 ( $\sim$ )
CTD ${(10^{-3})}$	1.87e-01	8.49e-01 ( $\sim$ )	3.44e-01	2.32e-02 ( $\sim$ )	6.01e+02	1.38e+03 ( $\sim$ )	3.55e-01	1.46e-02 ( $\sim$ )	3.40e-01	2.22e-02 ( $\sim$ )	3.00e-01	5.33e-02 ( $\sim$ )	1.70e-01	2.37e-02 ( $\sim$ )
CTD ${(10^{-5})}$	5.18e-01	2.84e+00 ( $\sim$ )	3.33e-01	3.64e-02 ( $\sim$ )	7.00e+02	9.85e+02 ( $\sim$ )	3.46e-01	2.06e-02 ( $\sim$ )	3.43e-01	2.33e-02 ( $\sim$ )	3.17e-01	6.28e-02 ( $\sim$ )	1.75e-01	2.73e-02 ( $\sim$ )
CTD ${(10^{-6})}$	0.00e+00	0.00e+00 ( $\sim$ )	2.90e-01	4.78e-02 ( $+$ )	6.96e+02	8.26e+02 ( $\sim$ )	3.02e-01	3.33e-02 ( $+$ )	2.90e-01	5.75e-02 ( $+$ )	3.15e-01	5.20e-02 ( $\sim$ )	1.62e-01	1.20e-02 ( $\sim$ )

Table 3: The HV results (mean and standard deviation) of our method with six different threshold on the 20 benchmark and real-world problems. The method with the best mean HV is highlighted in bold. The symbols “

-

”, “

\sim

” and “

+

” indicate that our method CTD is statistically worse than, equivalent to, and better than the competitor, respectively.

Method	DTLZ1 ( $m=2$ )		DTLZ2 ( $m=2$ )		DTLZ3 ( $m=2$ )		DTLZ4 ( $m=2$ )		DTLZ5 ( $m=2$ )		DTLZ6 ( $m=2$ )		DTLZ7 ( $m=2$ )
	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std
CTD tch	0.00e+00	0.00e+00 ( $\sim$ )	3.55e-01	1.22e-02 ( $\sim$ )	5.07e+02	6.87e+02 ( $\sim$ )	3.56e-01	1.20e-02 ( $\sim$ )	3.51e-01	1.52e-02 ( $\sim$ )	3.53e-01	6.73e-02 ( $-$ )	3.75e-01	2.16e-01 ( $-$ )
CTD	6.68e+00	2.43e+01	3.46e-01	2.09e-02	4.17e+02	8.64e+02	3.52e-01	1.46e-02	3.49e-01	1.95e-02	3.19e-01	4.51e-02	1.71e-01	2.43e-02
Method	DTLZ1 ( $m=3$ )		DTLZ2 ( $m=3$ )		DTLZ3 ( $m=3$ )		DTLZ4 ( $m=3$ )		DTLZ5 ( $m=3$ )		DTLZ6 ( $m=3$ )		DTLZ7 ( $m=3$ )
	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std
CTD tch	4.30e+03	7.83e+03 ( $\sim$ )	4.68e-01	5.62e-02 ( $-$ )	1.60e+06	8.57e+05 ( $\sim$ )	4.78e-01	5.39e-02 ( $-$ )	1.05e-01	1.43e-02 ( $\sim$ )	9.86e-02	2.38e-02 ( $\sim$ )	4.58e-01	2.68e-01 ( $\sim$ )

Table 4: The HV results (mean and standard deviation) of the two methods on the 20 benchmark and real-world problems on 30 independent runs. The method with the best mean HV is highlighted in bold. The symbols “

-

”, “

\sim

” and “

+

” indicate that CTD tch is statistically better than, equivalent to, and worse than EHVI-based CTD, respectively.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Method	DTLZ1 ( $m=2$ )		DTLZ2 ( $m=2$ )		DTLZ3 ( $m=2$ )		DTLZ4 ( $m=2$ )		DTLZ5 ( $m=2$ )		DTLZ6 ( $m=2$ )		DTLZ7 ( $m=2$ )
	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std
Sobol	5.13e-02	2.81e-01 ( $\sim$ )	2.45e-02	2.33e-02 ( $+$ )	3.26e-01	1.79e+00 ( $\sim$ )	2.47e-02	2.69e-02 ( $+$ )	1.76e-02	1.61e-02 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )
ParEGO	2.56e+00	8.00e+00 ( $\sim$ )	4.05e-01	2.23e-03 ( $+$ )	3.91e+02	6.99e+02 ( $\sim$ )	4.04e-01	2.04e-03 ( $+$ )	4.03e-01	2.73e-03 ( $+$ )	3.01e-01	1.13e-01 ( $\sim$ )	6.32e-01	2.12e-01 ( $-$ )
TS-TCH	0.00e+00	0.00e+00 ( $\sim$ )	1.49e-01	2.42e-02 ( $+$ )	0.00e+00	0.00e+00 ( $\sim$ )	1.51e-01	2.30e-02 ( $+$ )	1.51e-01	2.23e-02 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )
PSL	9.15e+01	1.44e+02 ( $-$ )	3.94e-01	1.10e-02 ( $+$ )	1.05e+03	1.62e+03 ( $\sim$ )	1.17e-01	2.07e-02 ( $+$ )	3.94e-01	1.10e-02 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	4.91e-01	1.63e-01 ( $-$ )
JES	0.00e+00	0.00e+00 ( $\sim$ )	3.99e-01	2.77e-03 ( $+$ )	9.96e+02	8.75e+02 ( $\sim$ )	3.99e-01	3.57e-03 ( $+$ )	4.00e-01	2.76e-03 ( $+$ )	2.51e-01	7.30e-02 ( $+$ )	6.45e-01	1.86e-01 ( $-$ )

Appendix

Table 5: The HV results (mean and standard deviation) of the ten methods on the 20 benchmark and real-world problems under 200 evaluations. The method with the best mean HV is highlighted in bold. The symbols “

-

”, “

\sim

” and “

+

” indicate that our method CTD is statistically worse than, equivalent to, and better than the competitor, respectively.

Method	DTLZ1 ( $m=2$ )		DTLZ2 ( $m=2$ )		DTLZ3 ( $m=2$ )		DTLZ4 ( $m=2$ )		DTLZ5 ( $m=2$ )		DTLZ6 ( $m=2$ )		DTLZ7 ( $m=2$ )
	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std
Sobol	3.20e-01	1.75e+00 ( $\sim$ )	2.08e-02	1.60e-02 ( $+$ )	1.10e+02	6.03e+02 ( $+$ )	2.34e-02	2.02e-02 ( $+$ )	2.18e-02	2.17e-02 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $\sim$ )
ParEGO	0.00e+00	0.00e+00 ( $\sim$ )	1.42e-01	4.65e-02 ( $\sim$ )	6.00e+02	7.25e+02 ( $\sim$ )	1.39e-01	7.34e-02 ( $\sim$ )	1.40e-01	8.26e-02 ( $\sim$ )	2.27e-01	7.34e-02 ( $\sim$ )	1.04e-01	1.55e-01 ( $\sim$ )
TS-TCH	0.00e+00	0.00e+00 ( $\sim$ )	5.39e-03	1.06e-02 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	8.57e-03	1.40e-02 ( $+$ )	1.35e-02	2.17e-02 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $\sim$ )
JES	0.00e+00	0.00e+00 ( $\sim$ )	1.43e-01	6.86e-02 ( $\sim$ )	8.24e+02	9.69e+02 ( $\sim$ )	1.06e-01	5.03e-02 ( $\sim$ )	9.97e-02	6.62e-02 ( $\sim$ )	2.17e-01	6.15e-02 ( $\sim$ )	1.37e-02	3.91e-02 ( $\sim$ )
EHVI	0.00e+00	0.00e+00 ( $\sim$ )	6.86e-02	8.47e-02 ( $+$ )	8.92e+02	8.95e+02 ( $\sim$ )	5.31e-02	5.55e-02 ( $+$ )	8.97e-02	7.69e-02 ( $\sim$ )	1.94e-01	5.54e-02 ( $\sim$ )	8.76e-02	1.61e-01 ( $\sim$ )

Appendix

Table 6: The HV results (mean and standard deviation) of the eight methods on the 20 benchmark and real-world problems under noisy settings and 200 evaluations. The method with the best mean HV is highlighted in bold. The symbols “

-

”, “

\sim

” and “

+

” indicate that the proposed CTD is statistically worse than, equivalent to, and better than the competitor, respectively.

Method	DTLZ1 ( $m=2$ )		DTLZ2 ( $m=2$ )		DTLZ3 ( $m=2$ )		DTLZ4 ( $m=2$ )		DTLZ5 ( $m=2$ )		DTLZ6 ( $m=2$ )		DTLZ7 ( $m=2$ )
	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std
Sobol	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )
ParEGO	1.34e+05	2.41e+04 ( $+$ )	6.48e-02	9.40e-02 ( $+$ )	1.91e+05	5.93e+04 ( $+$ )	2.08e-02	4.07e-02 ( $+$ )	2.92e-01	3.36e-01 ( $+$ )	8.85e-02	5.30e-02 ( $+$ )	4.01e-01	2.11e-01 ( $-$ )
TS-TCH	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )
PSL	8.86e+04	2.98e+04 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	3.99e+04	4.16e+04 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	3.50e-01	2.02e-01 ( $\sim$ )
JES	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )	0.00e+00	0.00e+00 ( $+$ )

Appendix

Table 7: The HV results (mean and standard deviation) of the ten methods on 14 problems with

d=50

. The method with the best mean HV is highlighted in bold. The symbols “

-

”, “

\sim

” and “

+

” indicate that the proposed CTD is statistically worse than, equivalent to, and better than the competitor, respectively.

	Sobol	ParEGO	TS-TCH	PSL	JES	EHVI	MORBO	MOBO-OSD	qPOTS
CTD (ours)	0/0/14	2/0/12	0/0/14	1/1/12	0/0/14	0/6/8	0/0/14	4/1/9	0/0/14

Appendix

Table 8: The summary of statistical results for the noisy experiments in Table 7 . Here, the left, median, and right numbers are the counts of test problems where the CTD was statistically worse, equivalent, or better to the peer method, respectively.

Method	DTLZ1 ( $m=2$ )		DTLZ2 ( $m=2$ )		DTLZ3 ( $m=2$ )		DTLZ4 ( $m=2$ )		DTLZ5 ( $m=2$ )		DTLZ6 ( $m=2$ )		DTLZ7 ( $m=2$ )
	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std
CTD ${(10^{-1})}$	0.00e+00	0.00e+00 ( $\sim$ )	4.08e-01	1.49e-03 ( $\sim$ )	9.70e+02	1.01e+03 ( $\sim$ )	4.08e-01	1.19e-03 ( $+$ )	4.08e-01	1.36e-03 ( $+$ )	3.32e-01	3.53e-02 ( $\sim$ )	1.86e-01	3.22e-02 ( $\sim$ )
CTD ${(10^{-2})}$	1.92e+00	1.05e+01 ( $\sim$ )	4.09e-01	1.45e-03 ( $\sim$ )	9.23e+02	1.30e+03 ( $\sim$ )	4.09e-01	1.08e-03 ( $\sim$ )	4.09e-01	1.53e-03 ( $\sim$ )	3.14e-01	4.42e-02 ( $\sim$ )	1.79e-01	3.03e-02 ( $\sim$ )
CTD ${(10^{-3})}$	1.84e+00	6.59e+00 ( $\sim$ )	4.09e-01	1.32e-03 ( $\sim$ )	1.17e+03	1.60e+03 ( $\sim$ )	4.10e-01	1.49e-03 ( $\sim$ )	4.09e-01	1.05e-03 ( $\sim$ )	3.08e-01	5.12e-02 ( $\sim$ )	1.81e-01	3.07e-02 ( $\sim$ )
CTD ${(10^{-5})}$	5.13e+00	1.36e+01 ( $\sim$ )	4.09e-01	1.04e-03 ( $\sim$ )	9.84e+02	1.13e+03 ( $\sim$ )	4.10e-01	1.43e-03 ( $\sim$ )	4.10e-01	1.41e-03 ( $\sim$ )	3.31e-01	3.64e-02 ( $\sim$ )	1.83e-01	3.09e-02 ( $\sim$ )
CTD ${(10^{-6})}$	6.05e+00	2.74e+01 ( $\sim$ )	4.08e-01	1.10e-03 ( $\sim$ )	1.13e+03	1.14e+03 ( $\sim$ )	4.09e-01	1.46e-03 ( $+$ )	4.09e-01	1.61e-03 ( $+$ )	3.23e-01	4.11e-02 ( $\sim$ )	1.64e-01	1.66e-02 ( $\sim$ )

Appendix

Table 9: The HV results (mean and standard deviation) of our method with six different threshold on the 20 benchmark and real-world problems under 200 evaluations. The method with the best mean HV is highlighted in bold. The symbols “

-

”, “

\sim

” and “

+

” indicate that our method CTD is statistically worse than, equivalent to, and better than the competitor, respectively.

Method	DTLZ1 ( $m=2$ )		DTLZ2 ( $m=2$ )		DTLZ3 ( $m=2$ )		DTLZ4 ( $m=2$ )		DTLZ5 ( $m=2$ )		DTLZ6 ( $m=2$ )		DTLZ7 ( $m=2$ )
	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std
CTD tch	0.00e+00	0.00e+00 ( $\sim$ )	6.38e-02	5.88e-02 ( $\sim$ )	4.43e+02	8.41e+02 ( $\sim$ )	4.19e-02	4.82e-02 ( $\sim$ )	4.88e-02	5.12e-02 ( $\sim$ )	1.02e-01	6.21e-02 ( $\sim$ )	5.90e-02	1.15e-01 ( $-$ )
CTD	8.72e-03	4.78e-02	7.59e-02	5.95e-02	3.69e+02	6.54e+02	5.68e-02	4.74e-02	6.70e-02	4.69e-02	1.33e-01	7.48e-02	1.48e-02	5.13e-02
Method	DTLZ1 ( $m=3$ )		DTLZ2 ( $m=3$ )		DTLZ3 ( $m=3$ )		DTLZ4 ( $m=3$ )		DTLZ5 ( $m=3$ )		DTLZ6 ( $m=3$ )		DTLZ7 ( $m=3$ )
	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std
CTD tch	6.61e+03	6.16e+03 ( $\sim$ )	5.20e-02	5.54e-02 ( $+$ )	1.68e+06	6.09e+05 ( $\sim$ )	5.40e-02	5.44e-02 ( $+$ )	5.21e-03	8.78e-03 ( $+$ )	2.18e-02	2.95e-02 ( $+$ )	0.00e+00	0.00e+00 ( $\sim$ )

Appendix

Table 10: The HV results (mean and standard deviation) of the eight methods on the 20 benchmark and real-world problems under noisy settings. The method with the best mean HV is highlighted in bold. The symbols “

-

”, “

\sim

” and “

+

” indicate that CTD tch is statistically better than, equivalent to, and worse than EHVI-based CTD, respectively.

Method	DTLZ1 ( $m=2$ )		DTLZ2 ( $m=2$ )		DTLZ3 ( $m=2$ )		DTLZ4 ( $m=2$ )		DTLZ5 ( $m=2$ )		DTLZ6 ( $m=2$ )		DTLZ7 ( $m=2$ )
	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std
CTD tch	1.53e+05	1.97e+04 ( $+$ )	3.16e-01	1.68e-01 ( $-$ )	2.71e+05	7.69e+04 ( $\sim$ )	1.69e-01	1.09e-01 ( $\sim$ )	1.12e+00	2.39e-01 ( $-$ )	1.64e-01	1.00e-01 ( $\sim$ )	3.63e-01	1.83e-01 ( $-$ )
CTD	1.94e+05	2.83e+04	1.84e-01	1.59e-01	2.97e+05	7.55e+04	1.29e-01	1.17e-01	7.42e-01	4.11e-01	1.56e-01	9.46e-02	2.11e-01	2.45e-02
Method	DTLZ1 ( $m=3$ )		DTLZ2 ( $m=3$ )		DTLZ3 ( $m=3$ )		DTLZ4 ( $m=3$ )		DTLZ5 ( $m=3$ )		DTLZ6 ( $m=3$ )		DTLZ7 ( $m=3$ )
	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std	Mean	Std
CTD tch	1.10e+08	1.95e+07 ( $+$ )	1.72e+01	2.47e+00 ( $-$ )	4.61e+08	1.09e+08 ( $\sim$ )	7.68e+00	1.61e+00 ( $-$ )	1.72e+01	2.01e+00 ( $-$ )	1.64e-02	2.91e-02 ( $+$ )	6.34e-01	2.80e-01 ( $-$ )

Appendix

Table 11: The HV results (mean and standard deviation) of the two methods on 14 problems with

d=50

. The method with the best mean HV is highlighted in bold. The symbols “

-

”, “

\sim

” and “

+

” indicate that the method is statistically better than, equivalent to, and worse than CTD, respectively.

	CTD tch
CTD	7/4/3

Appendix

Table 12: The summary of statistical results for the high-dimensional settings in Table 11 . Here, the left, median, and right numbers are the counts of test problems where the CTD was statistically worse, equivalent, or better to CTD tch , respectively.

Explore similar work

Apr 17, 2026

Multi-Objective Bayesian Optimization via Adaptive \varepsilon-Constraints Decomposition

Multi-objective Bayesian optimization (MOBO) provides a principled framework for optimizing multiple expensive black-box functions. However, existing MOBO methods often struggle with coverage, scalability, and handling constraints and preferences. In this work we propose STAGE-BO, Sequential Targeting Adaptive Gap-Filling

\varepsilon

-Constraint Bayesian Optimization: by analyzing the coverage of the surrogate Pareto front, our method identifies the Pareto front point with the largest uncovered gap, and uses its coordinates to define adaptive constraints in

\varepsilon

-constraint method, which transforms the problem into a sequence of inequality-constrained subproblems, efficiently solved via constrained expected improvement acquisition. Our approach provides uniform Pareto coverage without hypervolume computation and naturally handles constraints and preferences. Experiments on synthetic and real-world benchmarks demonstrate superior coverage and competitive hypervolume performance against state-of-the-art baselines. Our code implementation can be found at https://github.com/YangYaohong1/STAGE-BO.

Yaohong Yang, Sammie Katt, Samuel Kaski

Department of Computer Science, Aalto University, Espoo, Finland · ELLIS Institute Finland · Department of Computer Science, University of Manchester, Manchester, United Kingdom

Bayesian Optimization Multi-Objective Optimization

Ask PDF

Apr 10, 2026

Do We Really Need to Approach the Entire Pareto Front in Many-Objective Bayesian Optimisation?

Many-objective optimisation, a subset of multi-objective optimisation, involves optimisation problems with more than three objectives. As the number of objectives increases, the number of solutions needed to adequately represent the entire Pareto front typically grows substantially. This makes it challenging, if not infeasible, to design a search algorithm capable of effectively exploring the entire Pareto front. This difficulty is particularly acute in the Bayesian optimisation paradigm, where sample efficiency is critical and only a limited number of solutions (often a few hundred) are evaluated. Moreover, after the optimisation process, the decision-maker eventually selects just one solution for deployment, regardless of how many high-quality, diverse solutions are available. In light of this, we argue an idea that under a very limited evaluation budget, it may be more useful to focus on finding a single solution of the highest possible quality for the decision-maker, rather than aiming to approximate the entire Pareto front as existing many-/multi-objective Bayesian optimisation methods typically do. Bearing this idea in mind, this paper proposes a single point-based multi-objective search framework (SPMO) that aims to improve the quality of solutions along a direction that leads to a good tradeoff between objectives. Within SPMO, we present a simple acquisition function, called expected single-point improvement (ESPI), working under both noiseless and noisy scenarios. We show that ESPI can be optimised effectively with gradient-based methods via the sample average approximation (SAA) approach and theoretically prove its convergence guarantees under the SAA. We also empirically demonstrate that the proposed SPMO is computationally tractable and outperforms state-of-the-arts on a wide range of benchmark and real-world problems.

Chao Jiang, Jingyu Huang, Miqing Li

School of Computer Science, University of Birmingham · School of Mathematics, University of Birmingham

Multi-Objective Optimization Bayesian Optimization

Ask PDF

Sep 26, 2026

Learning What to Evaluate: Correlation-Aware Decoupling for Multiobjective Bayesian Optimization

Multiobjective Bayesian optimization (MOBO) with Gaussian process (GP) surrogates is a sample efficient approach to solving multiobjective optimization problems. In MOBO, a Bayesian decision theoretic acquisition function guides the adaptive selection of new candidate inputs, on which objectives and constraints are evaluated to update the surrogate model sequentially. Existing approaches maintain independent GP models for the objectives and constraints, with new observations evaluating all objectives and constraints in a coupled fashion. However, the objectives and constraints often contain inherent correlations which, if exploited, can enable

{decoupled}

evaluations where only a subset of them are evaluated at each round. We present a new approach that leverages a multitask GP model to jointly learn all objectives and constraints, and propose a total correlation metric that enables identifying an

{optimal}

subset of objectives and constraints to be evaluated at every round, even under uniform evaluation costs. Theoretically, we show that our acquisition policy is asymptotically consistent despite decoupling and that our proposed decoupled subset selection rule maximizes the expected posterior entropy reduction about unevaluated tasks under mild conditions. Empirically, we show that our approach outperforms coupled and decoupled baselines in the state of the art.

Ashwin Renganathan, Peter Bachman

Aerospace Engineering Pennsylvania State University University Park, PA 16802 · Institute of Computational and Data Sciences (ICDS) Pennsylvania State University University Park, PA 16802

Bayesian Optimization Multi-Objective Optimization Gaussian Processes

Ask PDF