Simplex-to-Euclidean Bijection for Conjugate and Calibrated Multiclass Gaussian Process Classification

Bernardo Williams; Harsha Vardhan Tetali; Arto Klami; Marcelo Hartmann

cs.LGMar 17, 2026

Simplex-to-Euclidean Bijection for Conjugate and Calibrated Multiclass Gaussian Process Classification

Authors: Bernardo Williams, Harsha Vardhan Tetali, Arto Klami, Marcelo Hartmann

Organizations: Department of Computer Science, University of Helsinki, Finland

Abstract

We propose a conjugate and calibrated Gaussian process (GP) model for multi-class classification by exploiting the geometry of the probability simplex. Our approach uses Aitchison geometry to map simplex-valued class probabilities to an unconstrained Euclidean representation, turning classification into a GP regression problem with fewer latent dimensions than standard multi-class GP classifiers. This yields conjugate inference and reliable predictive probabilities without relying on distributional approximations in the model construction. The method is compatible with standard sparse GP regression techniques, enabling scalable inference on larger datasets. Empirical results show well-calibrated and competitive performance across synthetic and real-world datasets.

Figures & tables

Panel of Figure 1 — Figure 1: Illustration of Exact-ILR on a $K=3$ toy problem. Top (data): Ground-truth class probabilities (blue, green and red), $\bm{\pi}(x)$ over $x\in[-1,1]$ , and three test inputs $x_{*}^{(1:3)}$ . Middle (likelihood): Discrete labels are represented by Gaussian pseudo-observations centered at $\bm{m}^{(k)}=\varphi(\bm{\mu}^{(k)})$ in Euclidean space (Sec. 3 ), where $\bm{\mu}^{(k)}=\lambda\,\mathbf{e}_{k}+(1-\lambda)\tfrac{1}{K}\bm{1}$ . The three panels visualize this class-weighted latent-space likelihood at $x_{*}^{(1:3)}$ . Bottom (posterior): For each $x_{*}^{(i)}$ , we draw posterior samples of the latent GP, map them through $\varphi^{-1}$ , and plot the resulting samples of $\bm{\pi}_{*}$ . Each point is one sampled probability vector (color = $\arg\max$ class) and the average is $\bar{\bm{\pi}}_{*}^{(i)}$ . The two lowermost panels show the posterior over the two latent GP coordinates across $x\in[-1,1]$ , with training points overlaid.

Figure 2: Likelihood in simplex and Euclidean spaces. The likelihood of observing classes $1$ , $2$ , or $3$ at input $x$ is a Gaussian mixture with modes at $\bm{m}^{(k)}=\varphi(\bm{\mu}^{(k)})$ in Euclidean space (right), which induces a pushforward likelihood on the simplex (left). The figure shows the likelihood for $\bm{\pi}(x)=(1/3,1/3,1/3)$ .

Panel of Figure 3 — Figure 3: Error, NLL and ECE for UCI datasets in the exact setting.

Panel of Figure 4 — Figure 4: Sparse models performance on datasets from the UCI repository.

Panel of Figure 5 — Figure 5: Error, NLL and ECE for increasing overlap of the input variables.

Panel of Figure 6 — Figure 6: Error, NLL and ECE for increasing number of categories $K$ .

Model	Error ( $y_{*}$ )	Error ( $f_{*}$ )
GPD	$0.076\pm 0.013$	$0.0\pm 0.0$
Exact-ILR	$0.0\pm 0.0$	$0.0\pm 0.0$

Table 1: Test misclassification rate when forming

\bm{\pi}_{*}

from either the noisy predictive distribution

p(\bm{z}_{*}\mid\mathcal{D},\bm{x}_{*})

or the latent predictive

p(\mathbf{f}_{*}\mid\mathcal{D},\bm{x}_{*})

. For GPD we compute

\bm{\pi}_{*}=\mathrm{softmax}(\cdot)

, applying softmax to samples of either

\bm{z}_{*}

\mathbf{f}_{*}

. For Exact-ILR we compute

\bm{\pi}_{*}=\varphi^{-1}(\cdot)

, applying

\varphi^{-1}

to samples of either

\bm{z}_{*}

\mathbf{f}_{*}

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	# Instances	Training Instances	Test Instances	# Attributes	# Classes	Inducing Points	Source
EEG	14980	10980	4000	14	2	200	UCI
HTRU2	17898	12898	5000	8	2	200	UCI
MAGIC	19020	14020	5000	10	2	200	UCI
MINIBOO	130064	120064	10000	50	2	400	UCI
LETTER	20000	15000	5000	16	26	200	UCI
DRIVE	58509	48509	10000	48	11	500	UCI

Appendix

Table 2: Details on all datasets from the UCI repository used in the experiments.

Panel of Figure 7 — Figure 7: Error, NLL and ECE for increasing overlap of the input variables for GPD and different values of $\alpha_{\varepsilon}$ .

Panel of Figure 8 — Figure 8: Error, NLL and ECE for increasing overlap of the input variables for different values of the error tolerance $\varepsilon$ and $\lambda=0.9$ .

Figure 9: The error rate of GDP for the estimate $\hat{\bm{\pi}}=\mathrm{softmax}(\bm{z})$ with $\bm{z}\sim\mathrm{Lognormal}(\tilde{y},\tilde{\sigma}^{2})$ . This error is due to the heavy tails of the Lognormal distribution that fall on the $\arg\max$ regions of the other categories.

Figure 10: Training points are shown with their true class labels. The background color indicates the predicted class, and its intensity of the color indicates the predicted probability. The method handles out of distribution data by reverting to uniform probabilities over classes. This can be seen by the probabilities smoothly transitioning to the uniform distribution (white) far from the data.

Exp	Model	Norm.	Target HP	LR	ACC $\uparrow$	NLL $\downarrow$	ECE $\downarrow$	Fit time (s)
Wine	LSM	$[-1,1]$	–	0.001	0.98 $\pm$ 0.017	0.52 $\pm$ 0.023	0.36 $\pm$ 0.010	–
	BSM	$[-1,1]$	–	0.001	0.97 $\pm$ 0.023	0.87 $\pm$ 0.011	0.55 $\pm$ 0.024	–
	GPD	$[-1,1]$ , Z	$\alpha_{\varepsilon}$ : 0.0001	0.01	0.98 $\pm$ 0.017	0.06 $\pm$ 0.021	0.07 $\pm$ 0.017	5.9
	E-ILR (ours)	$[-1,1]$ , Z	$\lambda$ : 0.999999	0.01	0.98 $\pm$ 0.017	0.06 $\pm$ 0.018	0.07 $\pm$ 0.018	6.2
Glass	LSM	$[-1,1]$	–	0.001	0.56 $\pm$ 0.041	1.24 $\pm$ 0.028	0.26 $\pm$ 0.049	–
	BSM	$[-1,1]$	–	0.001	0.66 $\pm$ 0.088	1.53 $\pm$ 0.010	0.42 $\pm$ 0.087	–

Appendix

Table 3: Full details on the hyperparameters selected for each model and experiment. Accuracy, NLL and ECE are computed on the test dataset, and the mean best value is boldfaced. Norm. refers to the input normalization method, HP to hyperparameter, and LR to the learning rate.

Gaussian Processes

Gaussian Processes are probabilistic models of functions that use kernels to make predictions with explicit uncertainty.

See all 112 papers

GGMPs: Generalized Gaussian Mixture Processes
Mar 11, 2026Vardaan Tekriwal, Mark D. Risser, Hengrui Luo +1Conditional Distribution Estimation Gaussian Processes
Applied Math & Computational Research Division, Lawrence Berkeley National Laboratory, Berkeley, CA · Department of Electrical Engineering & Computer Sciences, University of California, Berkeley, CA · Engineering Mathematics & Statistics Division, University of California, Berkeley, CA +3
Scalable Gaussian process inference via neural feature maps
May 11, 2026Anthony StephensonGaussian Processes Kernel Methods
School of Mathematics University of Bristol
Multiclass Classification without Labels via Posterior Simplex Geometry
Jul 27, 2026Raphaël Bonnet-Guerrini, Johann Ioannou-Nikolaides, Troels Petersen +1Multiclass Classification Weakly Supervised Learning
Niels Bohr Institute University of Copenhagen Copenhagen, 2100 · Computer Science Department Università degli Studi di Milano Milan, MI, 20133

Mar 11, 2026J/K move · Enter open · S save

GGMPs: Generalized Gaussian Mixture Processes

Vardaan Tekriwal, Mark D. Risser, Hengrui Luo, Marcus M. Noack

Applied Math & Computational Research Division, Lawrence Berkeley National Laboratory, Berkeley, CA · Department of Electrical Engineering & Computer Sciences, University of California, Berkeley, CA · Engineering Mathematics & Statistics Division, University of California, Berkeley, CA+3

Conditional Distribution Estimation Gaussian Processes Gaussian Mixture Models

Open paper AskPDF arXiv

Conditional density estimation is complicated by multimodality, heteroscedasticity, and strong non-Gaussianity. Gaussian processes (GPs) provide a principled nonparametric framework with calibrated uncertainty, but standard GP regression is limited by its unimodal Gaussian predictive form. We introduce the Generalized Gaussian Mixture Process (GGMP), a GP-based method for multimodal conditional density estimation in settings where each input may be associated with a complex output distribution rather than a single scalar response. GGMP combines local Gaussian mixture fitting, cross-input component alignment and per-component heteroscedastic GP training to produce a closed-form Gaussian mixture predictive density. The method is tractable, compatible with standard GP solvers and scalable methods, and avoids the exponentially large latent-assignment structure of naive multimodal GP formulations. Empirically, GGMPs improve distributional approximation on synthetic and real-world datasets with pronounced non-Gaussianity and multimodality.

Abstract

Gaussian Processes

Explore similar work

GGMPs: Generalized Gaussian Mixture Processes

Scalable Gaussian process inference via neural feature maps

Multiclass Classification without Labels via Posterior Simplex Geometry