Machine Learning Algorithms for Improving Black Box Optimization Solvers
Organizations: Fakultät für Mathematik, Universität Wien Oskar-Morgenstern-Platz 1, A-1090 Wien, Austria · Department of Computer Science, Czech Technical University Karlovo Namesti 13, 121 35 Prague 2, Czech Republic
Abstract
Black-box optimization (BBO) addresses problems where objectives are accessible only through costly queries without gradients or explicit structure. Classical derivative-free methods---line search, direct search, and model-based solvers such as Bayesian optimization---form the backbone of BBO, yet often struggle in high-dimensional, noisy, or mixed-integer settings. Recent advances use machine learning (ML) and reinforcement learning (RL) to enhance BBO: ML provides expressive surrogates, adaptive updates, meta-learning-based algorithm selection, and generative models, while the RL--BBO interface includes dynamic operator configuration, robust policy search, and meta-optimization across tasks. This paper surveys these developments, covering representative algorithms such as the modular model-based optimization framework (mlrMBO), zeroth-order adaptive momentum methods (ZO-AdaMM), automated BBO (ABBO), distributed block-wise optimization (DiBB), partition-based Bayesian optimization (SPBOpt), the transformer-based optimizer (B2Opt), diffusion-model-based BBO, surrogate-assisted RL for differential evolution (Surr-RLDE), robust BBO (RBO), coordinate-ascent MORE with step size adaptation (CAS-MORE), log-barrier stochastic gradient descent (LB-SGD), policy improvement with black-box (PIBB), and offline Q-learning with Mamba backbones (Q-Mamba). We further review benchmark efforts such as the NeurIPS 2020 BBO Challenge and the MetaBox framework. A controlled numerical study further confirms that ML-based candidate screening can improve the evaluation-budget performance of direct search and differential evolution, while the benefit remains solver-dependent.
Figures & tables
| Algorithm / Method | Main Paper Section | Supplement Algorithm |
|---|---|---|
| SBO | Section 2.1 | Algorithm 1 |
| DS | Section 2.2 | Algorithm 2 |
| LS-dd | Section 2.2 | Algorithm 3 |
| LS+dd | Section 2.3 | Algorithm 4 |
| TR | Section 2.3 | Algorithm 1 |
| Generic-BBO-Workflow | Section 2.4 | Algorithm 5 |
| category | problem size | HPC? | software/references | ||
|---|---|---|---|---|---|
| small | medium | large | |||
| Polling-Based | NOMAD [ 12 ] , BFO [ 52 ] , NMSMAX [ 25 ] , PSM [ 11 ] , DSPFD [ 21 ] , DFLINT [ 41 ] , DFLBOX [ 40 ] , DFNDFL [ 19 ] , VRBBO [ 31 ] , SDBOX [ 42 ] , VRDFON [ 32 ] | ||||
| Surrogate-Based | pySOT [ 13 ] , Dakota [ 1 ] , SPLINE [ 24 ] , MISO [ 49 ] , GPyOpt [ 20 ] , BOHB [ 15 ] , Scikit-Optimize [ 8 ] , Spearmint [ 58 ] , Dragonfly [ 30 ] , SNOBFIT [ 28 ] | ||||
| Local-Approximation-Based | SSDFO [ 35 ] , FMINUNC [ 46 ] , BOBYQA [ 53 ] , BCDFO [ 22 ] , UOBYQA [ 54 ] , MATRS [ 34 ] | ||||
| Method | Core Idea | Domain | Refs. |
|---|---|---|---|
| mlrMBO | Modular SMBO with flexible surrogates | Mixed/Continuous | [ 3 ] |
| ZO-AdaMM | Zeroth-order Adam with adaptive moments | Continuous/Noisy | [ 5 ] |
| ABBO | Algorithm selection and chaining | General BBO | [ 47 ] |
| DiBB | Distributed block-wise optimization | High-dimensional | [ 10 ] |
| SPBOpt | Partition-based local BO for low budgets | Low-budget BBO | [ 57 ] |
| DFO-TR | Trust-region DFO applied to ML objectives | Continuous/Noisy | [ 18 ] |
| Method | Core Idea | Domain | Refs. |
|---|---|---|---|
| RBO | Robust regression for noisy gradient estimates | Noisy BBO/RL | [ 6 ] |
| LB-SGD | Log-barrier SGD with safe adaptive steps | Constrained BBO/RL | [ 61 ] |
| CAS-MORE | KL-decoupled mean/covariance and entropy control | Stochastic BBO/Episodic RL | [ 27 ] |
| Surr-RLDE | RL configures DE operators with surrogate-based policy training | MetaBBO | [ 45 ] |
| PI2 / PIBB | Cost-weighted policy updates | Evolutionary BBO/Policy search | [ 60 ] |
| Q-Mamba | Offline decomposed Q-learning and Mamba backbone | Large-action-space MetaBBO | [ 43 ] |
| Solver | Success (%) | Median rel. gap | ||
|---|---|---|---|---|
| DS | 47.1 | 0.086 | 0.357 | |
| ML-DS | 60.0 | 0.271 | 0.414 | |
| LSdd | 58.6 | 0.586 | 0.714 | |
| ML-LSdd | 58.6 | 0.186 | 0.471 | |
| DE | 24.3 | 0.000 | 0.014 | |
| ML-DE | 34.3 | 0.000 | 0.014 |
Black-Box Optimization
Black-box optimization finds high-quality solutions when objectives can only be queried, not directly analyzed or differentiated.
See all 193 papers