cs.LGMar 11, 2026

Bayesian Optimization of Partially Known Systems using Hybrid Models

Authors: Eike Cramer, Luis Kutschat, Oliver Stollenwerk, Joel A. Paulson, Alexander Mitsos

Organizations: University College London, Sargent Centre for Process Systems Engineering, Department of Chemical Engineering, London WC1E 7JE, United Kingdom · RWTH Aachen University, Process Systems Engineering (AVT.SVT), Aachen 52074, Germany · University of Wisconsin–Madison, Department of Chemical & Biological Engineering, Madison, WI 53706, USA · JARA-CSD, Jülich 52425, Germany · 

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Bayesian optimization (BO) has gained attention as an efficient algorithm for black-box optimization of expensive-to-evaluate systems, where the BO algorithm iteratively queries the system and suggests new trials based on a probabilistic model fitted to previous samples. Still, the standard BO loop may require a prohibitively large number of experiments to converge to the optimum, especially for high-dimensional and nonlinear systems. We present a hybrid model-based BO formulation that combines the iterative Bayesian learning of BO with partially known mechanistic physical models. Instead of learning a direct mapping from inputs to the objective, we write all known equations for a physics-based model and infer expressions for variables missing equations using a probabilistic model, in our case, a Gaussian process (GP). The final formulation then includes the GP as a constraint in the hybrid model, thereby allowing other physics-based nonlinear and implicit model constraints. This hybrid model formulation yields a constrained, nonlinear stochastic program, which we discretize using the sample-average approximation. In an in-silico optimization of a single-stage distillation, the hybrid BO model based on mass conservation laws yields significantly better designs than a standard BO loop. Furthermore, the hybrid model converges in as few as one iteration, depending on the initial samples, whereas, the standard BO does not converge within 25 for any of the seeds. Overall, the proposed hybrid BO scheme presents a promising optimization method for partially known systems, leveraging the strengths of both mechanistic modeling and data-driven optimization.

Figures & tables

Figure 1: Log-regret Regret=f′−f∗\mathrm{Regret}=f^{\prime}-f^{*} of BO for the illustrative example for random sampling, standard BO with EI, and hybrid-BO with SAA-EI. The shaded areas represent the 90% intervals of the regret convergence over 25 runs.
Figure 2: Progression of SAA-EI with hybrid acquisition problem ( 10 ). The top row shows the GP approximation of the unknown equation h(u)≈GP(u)h(u)\approx\mathcal{GP}(u) , the center row shows the approximation of the objective, and the bottom row shows the SAA-EI (Equation ( 9 )). The approximations show the 90% confidence intervals, where the approximation of the objective is displayed as the quantiles estimated from 25 discrete samples ξs\xi_{s} . Left: Iteration 1; center: Iteration 3; right: Iteration 6.
Figure 3: Sketch of the flash unit with fixed inlet stream composition and temperature TT and pressure pp as the decision variables.
VariableDescriptionUnit
FFFeedmol/s
DDDistillatemol/s
BBBottommol/s
z1z_{1}Mole fraction feed[-]
y1y_{1}Mole fraction distillate[-]
x1x_{1}Mole fraction bottom[-]
Table 1: Variables of the flash unit. Index 1 denotes the volatile component.
Figure 4: Log regret (see Equation ( 12 )) hybrid BO in comparison to standard BO with EI and random sampling. Confidence intervals from eight runs of the SAA-EI and 25 random and standard BO.
Figure 5: Comparison of the expected improvement (EI) acquisition functions in the Temperature - Pressure plane between the benchmark standard BO with EI (top) and the hybrid model BO using the SAA-EI function (bottom).
Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

SymbolDescriptionValue
y1,sety_{1,set}Purity specification of y1y_{1}0.660.66
pambp_{amb}Ambient pressure1.01325 bar1.01325\,\mathrm{bar}
FFFeed flow rate1.0 mol/s1.0\,\mathrm{mol/s}
DDTop stream flow rate0.3 mol/s0.3\,\mathrm{mol/s}
z1z_{1}Mole fraction of H 2 O in feed0.50.5
ζ1\zeta_{1}Cost factor purity violation of y1y_{1}100\,\mathrm{\}$
Appendix
Table 2: Parameters and fixed values for flash unit case study.
More in this topic

Bayesian Optimization

Bayesian optimization uses probabilistic models and sequential experiments to find strong solutions when evaluating a black-box objective is costly.

See all 243 papers

Explore similar work

CardsList
Apr 8, 2026

Bayesian Optimization for Mixed-Variable Problems in the Natural Sciences

Optimizing expensive black-box objectives over mixed search spaces is a common challenge across the natural sciences. Bayesian optimization (BO) offers sample-efficient strategies through probabilistic surrogate models and acquisition functions. However, its effectiveness diminishes in mixed or high-cardinality discrete spaces, where gradients are unavailable and optimizing the acquisition function becomes computationally demanding. In this work, we generalize the probabilistic reparameterization (PR) approach of Daulton et al. to handle non-equidistant discrete variables, enabling gradient-based optimization in fully mixed-variable settings with Gaussian process (GP) surrogates. With real-world scientific optimization tasks in mind, we conduct systematic benchmarks on synthetic and experimental objectives to obtain an optimized kernel formulations and demonstrate the robustness of our generalized PR method. We additionally show that, when combined with a modified BO workflow, our approach can efficiently optimize highly discontinuous and discretized objective landscapes. This work establishes a practical BO framework for addressing fully mixed optimization problems in the natural sciences, and is particularly well suited to autonomous laboratory settings where noise, discretization, and limited data are inherent.

Yuhao Zhang, Ti John, Matthias Stosiek +1

Department of Applied Physics, Aalto University, Puumiehenkuja 2, Espoo, 02320, Finland · School of Natural Sciences Physics Department, Technical University of Munich, James-Franck-Str. 1, Garching bei München, 85748, Germany · Atomistic Modeling Center, Munich Data Science Institute, Technical University of Munich, Walther-von-Dyck Str. 10, Garching bei München, 85748, Germany +1

Bayesian Optimization
AskPDF
Jun 7, 2026

Bayesian Optimization of a Multi-Product Chemical Reactor Using Composite Models and Partial Physics Knowledge

We study data-driven real-time economic optimization of a multi-product chemical reactor when no reliable first-principles model is available beyond a steady-state energy balance. Instead of learning the economic objective directly as a black-box function, we use a composite formulation in which Gaussian process (GP) models predict physically meaningful outputs, including product concentrations and reactor temperature, while profit is computed analytically from these predictions together with raw-material, product, and utility prices. This preserves the structure of the economic objective, makes it parametric in changing prices without needing retraining, and allows candidate operating points to be checked against the available energy balance through a physics residual. The GPs also provide predictive uncertainty, which is exploited in a Bayesian optimization (BO) framework both for data-efficient exploration and for conservative enforcement of the reactor temperature constraint through an upper confidence bound. The acquisition function additionally penalizes large energy-balance mismatch obtained by substituting the GP-predicted outputs and candidate inputs into the available steady-state energy balance. The approach is demonstrated on a benchmark simulation of a non-isothermal multi-product reactor. Relative to a trust-region safe BO implementation, the proposed method achieves better simulated economic performance within the available iteration budget. Relative to a purely data-driven BO approach that does not use the available physics information, it avoids reactor temperature constraint violations.

Liqiu Dong, Marta Zagórowska, Mehmet Mercangöz

Department of Chemical Engineering, Imperial College London, London, UK · DCSC, Delft University of Technology, Delft, Netherlands

Bayesian OptimizationSafe Bayesian OptimizationConstrained Optimization
AskPDF
Jun 24, 2026

Bridging Spherical Black-Box Optimizers

When gradient information is unavailable, black-box optimization (BBO) methods provide a practical alternative. While Evolution Strategies (ES), Consensus-Based Optimization (CBO), Optimization via Integration (OVI), and related methods have each been studied independently, their connections remain underexplored. We unify these approaches within a common theoretical framework, revealing that they differ primarily in two design choices: fitness aggregation (controlling sharpness preference) and consensus scope (controlling modality). Leveraging these insights, we introduce hybrid optimizers that interpolate between existing methods. Our ES-OVI hybrid allows explicit control over the preference for flat minima, enabling a trade-off between performance and robustness in continuous control tasks. Our CBO-OVI hybrids combine the higher-dimensional efficiency of parametric methods with the multimodal capabilities of particle-based approaches, achieving competitive results on language model merging under limited evaluation budgets. We validate our methods on standard BBO benchmarks and higher-dimensional locomotion tasks, demonstrating that the hybrid methods can outperform their constituent algorithms.

Johannes Ackermann, Stefano Peluchetti

The University of Tokyo · Sakana AI

Evolutionary OptimizationBlack-Box OptimizationEvolution Strategies
AskPDF