High-dimensional Bayesian optimisation (HDBO) is an increasingly crowded field with new methods proposed at every major conference, making it ever harder to answer a deceptively simple question: "Which method is currently state-of-the-art?" In this paper, we question the efficacy of benchmarking practices in the field, and find them to be systematically ineffective: multiple published works fail to benchmark against the strongest available baselines, or even include incorrect implementations of key methods. Without a shared reference implementation, such errors are difficult to detect during peer review. We therefore introduce BO-Arena, an actively maintained and extensible package providing canonical implementations of state-of-the-art algorithms, which allows researchers to develop new methods and benchmark them rigorously. Using BO-Arena, we find that what currently drives performance in HDBO is even simpler than previously thought. We use these insights to propose a new algorithm which is competitive with the state of the art on problems with many observations, without resorting to the complex GP approximations used in recent methods. Finally, we question whether there is currently sufficient evidence that any more complex method outperforms a simple Gaussian process baseline for HDBO.
Figures & tables
Figure 1: The lack of an actively maintained package, with validated implementations of algorithms, can lead to baselines that are weak, incorrect, or subsequently become outdated ( Cheon et al.,, 2025 ; Yu et al.,, 2026 ; Li et al.,, 2024 ) , with performance far below the current state-of-the-art “Vanilla BO” ( Hvarfner et al.,, 2024 ) . Our package, BO-Arena , aims to solve this problem.
Figure 2: Overview of the BO-Arena architecture. Researchers are able to implement new problems and BO algorithms by implementing the Problem (left) and AskTellAgent (right) interfaces respectively. The Arena (centre) orchestrates the BO loop between them. We provide concrete, verified implementations of current state-of-the-art BO algorithms, and easy access to a large suite of existing benchmark problems via the bencher package ( Papenmeier and Nardi,, 2025 ) .
Figure 3: Current leaderboard of state-of-the-art algorithms on a set of commonly used problems in recent HDBO literature. We believe that future HDBO papers should include these problems to prevent cherry-picking, and verify their baseline implementations for more valid conclusions. Obtaining these results is possible with a few lines of code in BO-Arena (see Section B.4 ), and the data needed to reproduce this plot is publicly available.
Figure 4: On common synthetic problems, the ranking of different algorithms can be inconsistent: on each of the above three problems, a different algorithm dominates by a significant margin. We believe that the community should enforce the presence of a core suite of problems in all HDBO papers (e.g. the more consistent leaderboard proposed in Figure 3 ), to prevent cherry-picking of problems.
Figure 5: Simply utilising the Vanilla BO algorithm and maintaining a sliding window of the last 100 observations (optionally enforcing that the current incumbent is included) is competitive with the state-of-the-art EULBO ( Maus et al.,, 2024 ) on high-throughput BO problems ( N≈20000 ), without having to resort to any complex GP approximation techniques.
Figure 6: Whilst RAASP sampling can be a key performance driver for non-differentiable acquisition functions (e.g. exact Thompson sampling used by TuRBO ), it makes no difference to performance when used with Vanilla BO (despite being used in the original implementation of Vanilla BO). Results on all six problems can be found in Section E.2 .
Figure 7: When combining TuRBO with LogEI and DSP, we obtain a modified version of Vanilla BO where trust regions have been added. On the problems shown, the addition of trust regions yields no performance improvements, achieving the same performance as standard Vanilla BO. Results on all six problems can be found in Section E.3 .
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
BO-Arena
COCO
HDBO on BBOB
poli
HPOBench
Bayesmark
HDBO algorithms
6
0
5
6
0
0
Verified vs. original papers
✓
–
×
×
–
–
Native implementations
✓
–
×
×
–
–
Continuous problems
48
24
24
0
40
28
Real-world problems
✓
×
×
✓
✓
✓
Max. continuous dimensionality
6392
640
60
–
2
3
Appendix
Table 1: Comparison of BO-Arena with existing packages for benchmarking BO algorithms (see below for references).
Figure 8: Extension of the results presented in Figure 1 to all benchmark problems.
Figure 9: Extension of the results presented in Figure 6 to all benchmark problems. We observe that whilst the use of RAASP sampling is often a performance driver for the non-differentiable acquisition function used in TuRBO (exact Thompson sampling), it makes no difference to performance when used with Vanilla BO.
Figure 10: Extension of the results presented in Figure 7 to all benchmark problems. A precise description of the set-up for each experiment can be found in Table 2 . We find that the addition of trust regions to Vanilla BO yields no performance improvement on any of the problems considered, and can even hurt performance.
High-dimensional Bayesian optimization (HDBO) seeks sample-efficient optimization when the number of variables is large relative to the evaluation budget. Recent LLM-based and agentic BO methods incorporate task knowledge and adapt search decisions during a run, but have primarily been evaluated on low- and moderate-dimensional problems. We ask whether this paradigm can transfer to the higher-dimensional regime. Our experiments show that these methods do not remain reliable in the high-dimensional regime, where the challenge is not only where to evaluate, but also which modeling assumption and search geometry to use when the objective's useful structure is unknown. We therefore introduce HERA, a Hypothesis- and Evidence-guided Research Agent that uses task context, optimization feedback, and structural diagnostics to revise search hypotheses, select and configure HDBO strategies, and determine their execution length. PRISM, its numerical optimization engine, generates and evaluates candidates sequentially within each search block, updating numerical models after each observation. HERA remains competitive with strong numerical HDBO baselines and outperforms the evaluated LLM-based and agentic methods on four metadata-free synthetic functions. Across eight real-world tasks, HERA achieves the best mean final objective among all evaluated systems on most benchmarks. Further analyses show that structural diagnostics change strategy use, metadata effects vary across tasks, and adaptive search blocks reduce inference cost.
Zhixuan Gao, Ke Xue, Rongxi Tan +2
State Key Laboratory of Novel Software Technology, Nanjing University · School of Artificial Intelligence, Nanjing University
Bayesian optimization (BO) is typically used only for problems where the underlying function is deemed "expensive" (e.g. in time or money), with the speed of the BO algorithm being of little importance. In the real world, however, problems span a continuous range of values, going from cheap to expensive. In this paper, we present BO algorithms through the lens of a speed-performance Pareto frontier: we compare BO against faster algorithms like CMA-ES, and highlight how certain regions of the Pareto frontier are currently empty. As a solution, we propose a new set of fast BO methods that exploit the linear structure in existing BO algorithms, for both continuous and combinatorial inputs (from ~10 to 6000+ dimensions). Our methods achieve performance similar to state-of-the-art BO, while being more than 102× to 103× faster, opening the field of BO to an entirely new class of problems.
Colin Doumont, Henry Moss, Philipp Hennig
University of Tübingen, Tübingen AI Center · Lancaster University
High-dimensional Bayesian optimization (BO) often fits Gaussian process (GP) surrogates from far fewer observations than input dimensions. Modern Vanilla BO can perform well in this regime with dimension-aware priors, initialization, and acquisition optimization, but it typically retains automatic relevance determination (ARD), fitting one lengthscale per input coordinate. We study this modeling choice and propose Iso-BO, a controlled modification that replaces the ARD GP with an isotropic GP using one shared lengthscale while keeping the surrounding BO pipeline matched. For radial kernels, we show that the marginal log likelihood (MLL) depends on the inverse-squared ARD lengthscales only through weighted pairwise distances among the observed inputs. The current design can therefore leave some ARD directions exactly invisible or only weakly constrained by the MLL. Iso-BO removes coordinatewise reweighting and fits a single shared scale instead. Lengthscale-fitting and predictive-density diagnostics show that this finite-data effect appears in practice, including when the data-generating process is anisotropic. Across GP-prior, synthetic, and real-world benchmarks, Iso-BO often improves over matched modern Vanilla BO and remains competitive with the included high-dimensional BO baselines under the tested budgets. Stress tests also show the expected boundary wherein sufficiently strong, learnable anisotropy can favor the more flexible ARD model.
Wei-Ting Tang, Madhav Muthyala, Joel A. Paulson
Department of Chemical and Biological Engineering University of Wisconsin–Madison Madison, WI 53706, USA