Learning What to Evaluate: Correlation-Aware Decoupling for Multiobjective Bayesian Optimization

Ashwin Renganathan; Peter Bachman

stat.MLSep 26, 2026

Learning What to Evaluate: Correlation-Aware Decoupling for Multiobjective Bayesian Optimization

Authors: Ashwin Renganathan, Peter Bachman

Organizations: Aerospace Engineering Pennsylvania State University University Park, PA 16802 · Institute of Computational and Data Sciences (ICDS) Pennsylvania State University University Park, PA 16802

Abstract

Multiobjective Bayesian optimization (MOBO) with Gaussian process (GP) surrogates is a sample efficient approach to solving multiobjective optimization problems. In MOBO, a Bayesian decision theoretic acquisition function guides the adaptive selection of new candidate inputs, on which objectives and constraints are evaluated to update the surrogate model sequentially. Existing approaches maintain independent GP models for the objectives and constraints, with new observations evaluating all objectives and constraints in a coupled fashion. However, the objectives and constraints often contain inherent correlations which, if exploited, can enable ${decoupled}$ evaluations where only a subset of them are evaluated at each round. We present a new approach that leverages a multitask GP model to jointly learn all objectives and constraints, and propose a total correlation metric that enables identifying an ${optimal}$ subset of objectives and constraints to be evaluated at every round, even under uniform evaluation costs. Theoretically, we show that our acquisition policy is asymptotically consistent despite decoupling and that our proposed decoupled subset selection rule maximizes the expected posterior entropy reduction about unevaluated tasks under mild conditions. Empirically, we show that our approach outperforms coupled and decoupled baselines in the state of the art.

Figures & tables

Figure 1: The Branin and Currin test functions (left and middle left panels) and their total correlation $\mathrm{TC}(\mathbf{x})$ (middle right panel). Knowing the $\mathrm{TC}(\mathbf{x})$ structure allows us to decouple the two functions’ evaluations – $f_{1}$ is evaluated at $15$ points ( $\star$ , left) with high $\mathrm{TC}(\mathbf{x})$ and $f_{2}$ is evaluated on $30$ points ( $\bullet$ , middle left) with low $\mathrm{TC}(\mathbf{x})$ . Then, we fit two independent GPs using all $45$ points for both $f_{1}$ and $f_{2}$ ( $90$ evaluations total) and one multitask GP (MTGP) with only the $45$ decoupled evaluations. The right panel shows the MTGP’s reduction of posterior uncertainty in $f_{2}$ . Thus, appropriately chosen observation sites allow MTGP to transfer information between tasks, which can improve sample efficiency in MOBO.

Figure 2:

q\texttt{POTS-DOE}

decoupled evaluations (OSY)

Panel of Figure 3 — Figure 3: Synthetic experiments. The numbers in the labels follow $d/K/C$ dimensions. Solid lines are the mean and filled colors are the $\min/\max$ range over $10$ repetitions.

Panel of Figure 4 — Figure 4: Real-world experiments. Solid lines are the mean and filled colors are the $\min/\max$ range over $10$ repetitions.

Panel of Figure 5 — Figure 5: Ablation studies. Top row: ablation on $q$ (OSY problem); bottom row: ablation on $\tau$ .

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 6 — Figure 6: Synthetic experiments

Panel of Figure 7 — Figure 7: Real-world experiments.

Panel of Figure 8 — Figure 8: Ablation on batch size $q$ . Row 1: Branin-Currin, row-2: Car side-impact, row-3: Disc brake, row-4: OSY.

Panel of Figure 9 — Figure 9: Distribution of decoupled oracle evaluations (OSY)

Panel of Figure 10 — Figure 10: Distribution of decoupled oracle evaluations (Disc brake)

Test Case	Dim	Obj	Cons	Ref Point	Ablations:
BNH	2	2	2	$(-150,-50)$	–
BraninCurrin	2	2	0	$(-300,-20)$	$q=1,2,4,8$
C2DTLZ2	5	4	1	$(-10,-10,-10,-10)$	–
CarsideImpact	7	4	0	$(-50,-10,-20,-20)$	$q=1,2,4,8$
CONSTR	2	2	2	$(-10,-60)$	–
Constrained Branin-Currin	2	2	1	$(-300,-20)$	–

Appendix

Table 1: Summary of all experiments

Figure 11: Coupled single-task GP ablation on Disc brake, OSY, and Penicillin.

Figure 12: ICM versus LMC: cumulative runtime and hypervolume.

Figure 13: Exact versus greedy-plus-swap subset selection: cumulative selection time, hypervolume, and oracle evaluations.

Figure 14: Ablation on the decoupling threshold $\tau$ for Branin-Currin and OSY.

Figure 15: Runtime comparison on OSY.

Explore similar work

Adapting Nonstationary Multi-output Gaussian Processes to Bayesian Optimization
Sep 26, 2026Zikai XieBayesian Optimization Multi-Objective Optimization
State Key Laboratory of Precision and Intelligent Chemistry University of Science and Technology of China Hefei, Anhui 230000, China
Multi-Objective Bayesian Optimization via Adaptive \varepsilon-Constraints Decomposition
Apr 17, 2026Yaohong Yang, Sammie Katt, Samuel KaskiBayesian Optimization Multi-Objective Optimization
Department of Computer Science, Aalto University, Espoo, Finland · ELLIS Institute Finland · Department of Computer Science, University of Manchester, Manchester, United Kingdom
Converge Then Diversify: Decoupling Convergence and Diversity in Multi-Objective Bayesian Optimisation
Sep 11, 2026Chao Jiang, Yueling Huang, Miqing LiMulti-Objective Optimization Bayesian Optimization
School of Computer Science University of Birmingham

Sep 26, 2026J/K move · Enter open · S save

Adapting Nonstationary Multi-output Gaussian Processes to Bayesian Optimization

Zikai Xie

State Key Laboratory of Precision and Intelligent Chemistry University of Science and Technology of China Hefei, Anhui 230000, China

Bayesian Optimization Multi-Objective Optimization Gaussian Processes Gaussian Process Regression

Open paper AskPDF arXiv

Multi-objective Bayesian optimization (MOBO) commonly relies on independent Gaussian processes (GPs) with stationary kernels, limiting its ability to represent nonstationary structure and share information between objectives. However, expressive nonstationary GPs do not necessarily make reliable BO decisions. We study this mismatch for the multi-output low-rank nonstationary (MO-LRN) GP: strong training fit can coexist with large off-design errors and optimistic acquisition predictions. We introduce MOLRN-BO, which combines a regularized shared-spectral surrogate with objective-specific residuals, prequential mean correction and tempered covariance scaling, and Pareto-local qLogEHVI optimization with periodic global search. Experiments on 12 deterministic bi-objective benchmarks show that MOLRN-BO substantially improves upon the original MO-LRN and achieves the best average problem ranks for final normalized hypervolume and normalized inverted generational distance among nine evaluated algorithms. It also achieves the strongest adverse-tail performance while remaining competitive with the leading baselines in anytime optimization. Ablation studies further show that the shared spectral construction improves off-design prediction, the local--global decision policy improves optimization performance, and hierarchical calibration reduces systematic candidate bias. These results demonstrate that nonstationary multi-output surrogates can deliver strong and robust MOBO performance when their structure and use are explicitly adapted to the demands of sequential optimization.

Abstract

Explore similar work

Adapting Nonstationary Multi-output Gaussian Processes to Bayesian Optimization

Multi-Objective Bayesian Optimization via Adaptive \varepsilon-Constraints Decomposition

Converge Then Diversify: Decoupling Convergence and Diversity in Multi-Objective Bayesian Optimisation