Information Theoretic Bayesian Optimization over the Probability Simplex

Federico Pavesi; Antonio Candelieri; Noémie Jaquier

cs.LGMar 10, 2026

Information Theoretic Bayesian Optimization over the Probability Simplex

Authors: Federico Pavesi, Antonio Candelieri, Noémie Jaquier

Organizations: Department of Robotics, Perception and Learning, KTH Royal Institute of Technology, Stockholm, Sweden · Department of Economics, Management and Statistics, University of Milan, Bicocca, Milan, Italy

Abstract

Bayesian optimization is a data-efficient technique that has been shown to be extremely powerful to optimize expensive, black-box, and possibly noisy objective functions. Many applications involve optimizing probabilities and mixtures which naturally belong to the probability simplex, a constrained non-Euclidean domain defined by non-negative entries summing to one. This paper introduces $α$ -GaBO, a novel family of Bayesian optimization algorithms over the probability simplex. Our approach is grounded in information geometry, a branch of Riemannian geometry which endows the simplex with a Riemannian metric and a class of connections. Based on information geometry theory, we construct Matérn kernels that reflect the geometry of the probability simplex, as well as a one-parameter family of geometric optimizers for the acquisition function. We validate our method on benchmark functions and on a variety of real-world applications including mixtures of components, mixtures of classifiers, and a robotic control task, showing its increased performance compared to constrained Euclidean approaches.

Figures & tables

Figure 1: $\alpha$ -GaBO leverages the sphere map $\varphi$ , which establishes an isometry between the probability simplex $\Delta^{d}$ and the positive orthant $\mathbb{S}^{d}_{\geq 0}$ of the sphere. BO on the simplex is performed via equivalent representations on the sphere.

	$\Delta^{2}$	$\Delta^{5}$	$\Delta^{10}$
Ackley $\log_{10}(F)$
Rosenbrock $\log_{10}(F)$
Griewank $\log_{10}(F)$
	$N$	$N$	$N$

Figure 2: Logarithm of the regret (median and quartiles) and distribution over the final recommendation for

\alpha_{0}

-GaBO ( ),

\alpha_{\text{-}1}

-GaBO ( ),

\mathbb{S}^{d}

-Eucl. BO ( ), and BORIS ( ) on benchmark functions.

Panel of Figure 3 — (a) Concrete, $\Delta^{6}\times[0,1]$ .

$d$	Method	Ackley			Rosenbrock			Griewank
		Median	IQR	$p$ (vs best)	Median	IQR	$p$ (vs best)	Median	IQR	$p$ (vs best)
2	$\mathbb{S}^{d}$ -Eucl. BO	$\bm{0.020}$	$0.987$	–	$0.002$	$0.008$	$1.0\times 10^{-4}$ ∗∗∗	$0.005$	$0.018$	$8.1\times 10^{-1}$
	BORIS	$0.023$	$1.793$	$9.6\times 10^{-1}$	$0.006$	$0.008$	$1.3\times 10^{-5}$ ∗∗∗	$0.011$	$0.025$	$1.8\times 10^{-3}$ ∗∗
	$\alpha_{0}$ -GaBO	$0.033$	$\bm{0.022}$	$3.7\times 10^{-1}$	$\bm{0.000}$	$\bm{0.000}$	–	$\bm{0.000}$	$0.007$	–
	$\alpha_{-1}$ -GaBO	$0.036$	$0.030$	$2.0\times 10^{-1}$	$0.000$	$0.001$	$2.2\times 10^{-3}$ ∗∗	$0.000$	$\bm{0.007}$	$5.0\times 10^{-2}$
5	$\mathbb{S}^{d}$ -Eucl. BO	$0.829$	$2.044$	$7.9\times 10^{-1}$	$3.179$	$5.209$	$8.4\times 10^{-2}$	$0.034$	$0.048$	$2.4\times 10^{-4}$ ∗∗∗

Table 1: Median and interquartile range of the final recommendation, and

p

-values of Mann-Whitney tests on benchmark functions. Statistical tests are against the best BO alternative per dimension ( ∗ :

p<\!0.05

, ∗∗ :

p<\!0.01

, ∗∗∗ :

p<\!0.001

Panel of Figure 4 — Figure 4: Regret (median and quartiles) and distribution over the final recommendation for $\alpha_{0}$ -GaBO ( ), $\alpha_{\text{-}1}$ -GaBO ( ), $\mathbb{S}^{d}$ -Eucl. BO ( ), and BORIS ( ) on mixture of classifiers for wall-following robot navigation dataset on $\Delta^{7}$ . All models outperform the best standalone classifier ( ).

Method	Concrete ( $d\!=\!7$ )			WF3 ( $d\!=\!3$ )			PCE10 ( $d\!=\!3$ )			Simple classifiers ( $d\!=\!7$ )			Robotics ( $d\!=\!6$ )
	Median	IQR	$p$	Median	IQR	$p$	Median	IQR	$p$	Median	IQR	$p$	Median	IQR	$p$
$\mathbb{S}^{d}$ -Eucl. BO	$\bm{104.098}$	$12.172$	–	$0.040$	$0.017$	$4.7e^{-1}$	$\bm{0.000}$	$0.024$	$9.8e^{-1}$	$\bm{0.369}$	$0.024$	–	$2.832$	$0.357$	$3.3e^{-3}$ ∗∗
BORIS	$68.372$	$13.874$	$1.4e^{-9}$ ∗∗∗	$\bm{0.029}$	$0.018$	–	$\bm{0.000}$	$0.026$	$4.9e^{-1}$	$0.380$	$\bm{0.020}$	$1.7e^{-2}$ ∗	$2.778$	$0.387$	$2.8e^{-3}$ ∗∗
$\alpha_{0}$ -GaBO	$88.025$	$\bm{4.129}$	$5.0e^{-7}$ ∗∗∗	$0.031$	$0.017$	$1.4e^{-1}$	$0.007$	$0.005$	$8.4e^{-4}$ ∗∗∗	$0.396$	$0.041$	$1.5e^{-4}$ ∗∗∗	$\bm{2.634}$	$0.225$	–
$\alpha_{-1}$ -GaBO	$65.760$	$24.594$	$1.4e^{-9}$ ∗∗∗	$0.032$	$\bm{0.015}$	$1.6e^{-1}$	$\bm{0.000}$	$\bm{0.004}$	–	$0.378$	$0.032$	$9.9e^{-2}$	$2.668$	$\bm{0.224}$	$2.9e^{-1}$

Table 2: Median and interquartile range of the final recommendation, and

p

-values of Mann-Whitney tests on real-world applications. Statistical tests are against the best BO alternative per dimension ( ∗ :

p<\!0.05

, ∗∗ :

p<\!0.01

, ∗∗∗ :

p<\!0.001

). Note that the objective function is maximized for concrete and minimized for the other applications.

Panel of Figure 5 — Figure 5: Left: Regret (median and quartiles) and distribution over the final recommendation for $\alpha_{0}$ -GaBO ( ), $\alpha_{\text{-}1}$ -GaBO ( ), $\mathbb{S}^{d}$ -Eucl. BO ( ), and BORIS ( ) for robotic multi-task control. Right: Snapshots of the optimal trajectory with target left ( ) and right ( ) hand positions.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Manifold	Kernel	Acq. Opt.	Median Regret	IQR
Sphere	Riemannian	Riemannian	$-3.728$	$0.670$
Sphere	Euclidean	Riemannian	$-1.473$	$2.072$
Sphere	Riemannian	Euclidean	$-2.880$	$0.959$
Sphere	Euclidean	Euclidean	$-1.480$	$1.987$
Simplex	Riemannian	Riemannian	$-3.634$	$0.496$
Simplex	Euclidean	Riemannian	$-1.500$	$1.718$

Appendix

Table 3: Ablation study on the 5-dimensional simplex with the Griewank objective, comparing Riemannian and Euclidean kernels and optimization methods.

Method	$d=2$	$d=5$	$d=10$
$\mathbb{S}^{d}$ -Eucl. BO	$0.046$ ( $0.028$ )	$0.068$ ( $0.025$ )	$0.112$ ( $0.008$ )
BORIS	$0.046$ ( $0.024$ )	$0.068$ ( $0.025$ )	$0.112$ ( $0.010$ )
$\alpha_{0}$ -GaBO	$0.235$ ( $0.049$ )	$0.241$ ( $0.042$ )	$0.270$ ( $0.033$ )
$\alpha_{-1}$ -GaBO	$0.302$ ( $0.099$ )	$0.231$ ( $0.079$ )	$0.245$ ( $0.071$ )

Appendix

Table 4: Median (interquartile range) iteration wall-clock times in seconds for the Ackley function. All experiments are run inside a virtual machine with 16 vCPUs AMD EPYC 7V12 with an NVIDIA Tesla T4.

Bayesian Optimization

Bayesian optimization uses probabilistic models and sequential experiments to find strong solutions when evaluating a black-box objective is costly.

See all 243 papers

Bayesian Optimization with Fisher Information Geometry: Gradient Bounds and Trust-Region Methods
Sep 25, 2026Saksham Kiroriwal, Julius Pfrommer, Jürgen BeyererBayesian Optimization Trust-Region Optimization
Cognitive Industrial Systems, Fraunhofer IOSB, Karlsruhe, Germany · Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany
Bayesian Optimization for Mixed-Variable Problems in the Natural Sciences
Apr 8, 2026Yuhao Zhang, Ti John, Matthias Stosiek +1Bayesian Optimization
Department of Applied Physics, Aalto University, Puumiehenkuja 2, Espoo, 02320, Finland · School of Natural Sciences Physics Department, Technical University of Munich, James-Franck-Str. 1, Garching bei München, 85748, Germany · Atomistic Modeling Center, Munich Data Science Institute, Technical University of Munich, Walther-von-Dyck Str. 10, Garching bei München, 85748, Germany +1
Amortized Acquisition Optimization for Bayesian Optimization with Variational Mutual Information
Jan 13, 2026Farhad MirkarimiBayesian Optimization Black-Box Optimization

Sep 25, 2026J/K move · Enter open · S save

Bayesian Optimization with Fisher Information Geometry: Gradient Bounds and Trust-Region Methods

Saksham Kiroriwal, Julius Pfrommer, Jürgen Beyerer

Cognitive Industrial Systems, Fraunhofer IOSB, Karlsruhe, Germany · Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany

Bayesian Optimization Trust-Region Optimization Information Geometry

Open paper AskPDF arXiv

We study Bayesian optimization (BO) through the lens of information geometry. Pulling back the Fisher information metric through the surrogate posterior map yields a local sensitivity tensor on the input space, which leads to an upper bound on the gradient of reparameterizable acquisition functions. This view explains vanishing-gradient behavior in high-dimensional BO and provides a common interpretation of heuristics such as RAASP and dimension-scaled lengthscales. Building on this analysis, we propose FITR, a trust-region-based BO method that replaces lengthscale-based scaling by local pullback-Fisher weights. FITR is not restricted to GP kernels with explicit lengthscales. On GP benchmarks with an SE kernel, experiments show competitive performance using FITR. The proposed method also easily generalizes to non-isotropic surrogates, although the gains are more task-dependent in that setting.

Abstract

Bayesian Optimization

Explore similar work

Bayesian Optimization with Fisher Information Geometry: Gradient Bounds and Trust-Region Methods

Bayesian Optimization for Mixed-Variable Problems in the Natural Sciences

Amortized Acquisition Optimization for Bayesian Optimization with Variational Mutual Information