Constrained optimization in high-dimensional black-box settings is difficult due to expensive evaluations, the lack of gradient information, and complex feasibility regions. In this work, we propose a Bayesian optimization method that combines a penalty formulation, a surrogate model, and a trust region strategy. The constrained problem is converted to an unconstrained form by penalizing constraint violations, which provides a unified modeling framework. A trust region restricts the search to a local region around the current best solution, which improves stability and efficiency in high dimensions. Within this region, we use the Expected Improvement acquisition function to select evaluation points by balancing improvement and uncertainty. The proposed Trust Region method integrates penalty-based constraint handling with local surrogate modeling. This combination enables efficient exploration of feasible regions while maintaining sample efficiency. We compare the proposed method with state-of-the-art methods on synthetic and real-world high-dimensional constrained optimization problems. The results show that the method identifies high-quality feasible solutions with fewer evaluations and maintains stable performance across different settings.
Figures & tables
Figure 1: Average objective values obtained over 30 repeated experiments on Ackley problem.
Figure 2: Average objective values obtained over 30 repeated experiments on L e^ vy problem.
Problems
TR- M EI
FuRBO
SCBO
Ackley
0.0085±0.0102
4.4120±0.3733
7.3662±1.0541
L e^ vy
0.0071±0.0002
4.8199±0.5856
5.4482±0.3754
Rastrigin
47.0507±1.1983
173.7190±40.0435
292.2801±36.344
Table 1: Best valid optimal solution with mean ± standard deviation.
Figure 3: Average objective values obtained over 30 repeated experiments on Rastrigin problem.
Problems
TR- M EI
FuRBO
SCBO
Stock
CVaR (rmin=1.45)
0.1743±0.0032
0.1743±0.0032
0.1935±0.0020
portfolio
CVaR (rmin=1.55)
0.2405±0.0027
0.2476±0.0017
0.2611±0.0020
Call option
CVaR (rmin=5.30)
0.2610±0.0033
0.2777±0.0017
0.3085±0.0018
portfolio
CVaR (rmin=5.40)
0.2970±0.0017
0.3159±0.0016
0.3340±0.0015
Table 2: Comparison of portfolio performance for different optimization methods.
Figure 4: Average objective values obtained over 30 repeated experiments for the stock portfolio problem with (a)rmin=1.45(left),(b)rmin=1.55(right) .
Figure 5: Average objective values obtained over 30 repeated experiments for the stock portfolio problem with (a)rmin=5.30(left),(b)rmin=5.40(right) .
Bayesian optimization (BO) for high-dimensional constrained problems remains a significant challenge due to the curse of dimensionality. We propose Local Constrained Bayesian Optimization (LCBO), a novel framework tailored for such settings. Unlike trust-region methods that are prone to premature shrinking when confronting tight or complex constraints, LCBO leverages the differentiable landscape of constraint-penalized surrogates to alternate between rapid local descent and uncertainty-driven exploration. Theoretically, we prove that LCBO achieves a convergence rate for the Karush-Kuhn-Tucker (KKT) residual that depends polynomially on the dimension d for common kernels under mild assumptions, offering a rigorous alternative to global BO where regret bounds typically scale exponentially. Extensive evaluations on high-dimensional benchmarks (up to 100D) demonstrate that LCBO consistently outperforms state-of-the-art baselines.
Jing Jingzhe, Fan Zheyi, Szu Hui Ng +1
State Key Laboratory of Mathematical Sciences, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing, China · School of Mathematical Sciences, University of Chinese Academy of Sciences, Beijing, China · Department of Industrial Systems Engineering & Management, National University of Singapore, Singapore.
Bayesian Optimisation (BO) under unknown constraints is particularly challenging when feasible regions are small. In such settings, existing methods that typically rely solely on evaluations of the true objective and constraints struggle to efficiently explore the design space. However, many real-world applications offer auxiliary data sources (e.g. surrogate models or simplified simulations) that can support early exploration. Despite this potential, their integration into constrained BO remains largely unexplored. We propose a general multi-source framework that extends constrained Max-value Entropy Search, capturing inter-source correlation while balancing evaluation cost and information gain. Experiments on both synthetic and physics-based benchmarks show that our method efficiently identifies feasible and optimal solutions, even when auxiliary data are only weakly correlated. The proposed approach consistently outperforms existing methods, particularly in early-stage exploration.
Hauke Maathuis, Roeland De Breuker, Saullo Castro +1
Discrete black-box optimization is often addressed using approaches such as Sequential Model-Based Optimization (SMBO), which aims to improve sample efficiency by fitting surrogate models that approximate a costly objective function over a discrete search space. In many real-world problems, the set of feasible inputs is often given by logical constraints known in advance. However, existing surrogate modeling techniques generally fail to capture the symbolic rules governing feasibility in discrete input spaces. In this paper, we propose a surrogate modeling approach based on tensor decomposition that captures the structure of discrete search spaces while directly integrating feasibility information. To implement this approach, we formulate surrogate model training as a constrained polynomial optimization problem and solve a relaxed formulation using a differentiable penalty term derived from T-norms. Our experiments on both synthetic and real-world benchmarks, including a pressure vessel design task, demonstrate that the proposed method improves sample efficiency by effectively guiding the search away from infeasible regions.
Keisuke Onoue, Ryosuke Kojima
Nara Institute of Science and Technology, Nara, Japan · Graduate School of Medicine, Kyoto University, Kyoto, Japan · RIKEN Center for Biosystems Dynamics Research, Japan