GGMPs: Generalized Gaussian Mixture Processes

Vardaan Tekriwal; Mark D. Risser; Hengrui Luo; Marcus M. Noack

cs.LGMar 11, 2026

GGMPs: Generalized Gaussian Mixture Processes

Authors: Vardaan Tekriwal, Mark D. Risser, Hengrui Luo, Marcus M. Noack

Organizations: Applied Math & Computational Research Division, Lawrence Berkeley National Laboratory, Berkeley, CA · Department of Electrical Engineering & Computer Sciences, University of California, Berkeley, CA · Engineering Mathematics & Statistics Division, University of California, Berkeley, CA · Department of Industrial Engineering & Operations Research, University of California, Berkeley, CA · Climate & Ecosystem Sciences Division, Lawrence Berkeley National Laboratory, Berkeley, CA · Department of Statistics, Rice University, Houston, TX

Abstract

Conditional density estimation is complicated by multimodality, heteroscedasticity, and strong non-Gaussianity. Gaussian processes (GPs) provide a principled nonparametric framework with calibrated uncertainty, but standard GP regression is limited by its unimodal Gaussian predictive form. We introduce the Generalized Gaussian Mixture Process (GGMP), a GP-based method for multimodal conditional density estimation in settings where each input may be associated with a complex output distribution rather than a single scalar response. GGMP combines local Gaussian mixture fitting, cross-input component alignment and per-component heteroscedastic GP training to produce a closed-form Gaussian mixture predictive density. The method is tractable, compatible with standard GP solvers and scalable methods, and avoids the exponentially large latent-assignment structure of naive multimodal GP formulations. Empirically, GGMPs improve distributional approximation on synthetic and real-world datasets with pronounced non-Gaussianity and multimodality.

Figures & tables

Figure 1: Generalized Gaussian Mixture Processes (GGMPs) capture multimodal conditional distributions while preserving closed-form inference. Many real-world processes exhibit conditioning-dependent multimodality, asymmetry, and heteroscedasticity that standard GPs cannot represent. GGMPs address this by modeling $p(y\mid x)$ as a weighted mixture of GPs. Here, predictive densities at four held-out inputs are shown against empirical conditional distributions (blue histograms). With the number of components in the mixture $K{=}1$ (orange), GGMPs reduce to a standard heteroscedastic GP. Increasing $K$ progressively resolves multimodal structure, with the model remaining robust to overspecification as surplus components overlap with existing ones.

Figure 2: Synthetic distribution field with fixed-point slices. The background shows $\log_{10}p(y\mid x)$ , generated from a single latent mean function $f(x)$ (black curve), while red vertical slices depict sampled local conditional densities at selected $x$ -locations. Despite sharing the same underlying $f(x)$ , the local distributions vary strongly in modality, spread, and asymmetry.

Model	Bhattacharyya	Symmetric KL	Wasserstein-1	$L^{1}$
$\mathrm{GP}_{1}$	0.4869 $\pm$ 0.1513	4.2005 $\pm$ 1.7752	0.7589 $\pm$ 0.2126	1.3115 $\pm$ 0.1699
$\mathrm{MDN}_{1}$	0.2278 $\pm$ 0.2124	2.9952 $\pm$ 3.0088	0.3223 $\pm$ 0.3369	0.7633 $\pm$ 0.4602
$\mathrm{MDN}_{3}$	0.0169 $\pm$ 0.0049	0.1561 $\pm$ 0.0486	0.0429 $\pm$ 0.0357	0.1989 $\pm$ 0.0586
$\mathrm{MDN}_{5}$	0.0163 $\pm$ 0.0048	0.1606 $\pm$ 0.0533	0.0366 $\pm$ 0.0309	0.1936 $\pm$ 0.0522
$\mathrm{MDN}_{10}$	0.0156 $\pm$ 0.0038	0.1316 $\pm$ 0.0382	0.0470 $\pm$ 0.0415	0.1925 $\pm$ 0.0462
$\mathrm{MDN}_{25}$	0.0179 $\pm$ 0.0028	0.1357 $\pm$ 0.0446	0.0296 $\pm$ 0.0244	0.1839 $\pm$ 0.0282

Table 1: Divergence metrics for the synthetic dataset.

Model	Log Score	CRPS	PIT Mean	PIT Std	Cov 50%	Cov 90%	Cov 95%
$\mathrm{GP}_{1}$	-1.6003	1.4150	0.4774	0.2681	0.5648	0.8854	0.9330
$\mathrm{MDN}_{1}$	-0.9420	0.5585	0.4822	0.2779	0.5115	0.9143	0.9570
$\mathrm{MDN}_{3}$	-0.3572	0.4935	0.5004	0.2290	0.6710	0.9762	0.9890
$\mathrm{MDN}_{5}$	-0.3505	0.4945	0.4914	0.2286	0.6700	0.9767	0.9884
$\mathrm{MDN}_{10}$	-0.3458	0.4943	0.4988	0.2300	0.6679	0.9755	0.9886
$\mathrm{MDN}_{25}$	-0.3415	0.4934	0.5029	0.2289	0.6713	0.9766	0.9892

Table 2: Predictive scoring and calibration diagnostics on the synthetic dataset.

Model	Bhattacharyya	Symmetric KL	Wasserstein-1	$L^{1}$
$\mathrm{GP}_{1}$	0.1580 $\pm$ 0.1050	2.6399 $\pm$ 1.6705	2.2896 $\pm$ 0.9970	0.6090 $\pm$ 0.1987
$\mathrm{MDN}_{1}$	0.1480 $\pm$ 0.1033	2.6022 $\pm$ 1.0942	1.7232 $\pm$ 1.0110	0.6011 $\pm$ 0.1801
$\mathrm{MDN}_{3}$	0.1428 $\pm$ 0.1011	2.5923 $\pm$ 1.2212	1.2998 $\pm$ 1.0232	0.5801 $\pm$ 0.2336
$\mathrm{MDN}_{5}$	0.1414 $\pm$ 0.1006	2.5834 $\pm$ 1.6252	1.2121 $\pm$ 0.9547	0.5244 $\pm$ 0.2482
$\mathrm{MDN}_{10}$	0.1425 $\pm$ 0.0989	2.5901 $\pm$ 0.9998	1.0829 $\pm$ 1.1029	0.5328 $\pm$ 0.2434
$\mathrm{MDN}_{25}$	0.1382 $\pm$ 0.1001	2.5602 $\pm$ 1.4353	1.0021 $\pm$ 0.9887	0.4932 $\pm$ 0.2223

Table 3: Divergence metrics for the temperature dataset.

Model	Log Score	CRPS	PIT Mean	PIT Std	Cov 50%	Cov 90%	Cov 95%
$\mathrm{GP}_{1}$	-3.7951	13.7364	0.5134	0.3050	0.4194	0.9065	0.9624
$\mathrm{MDN}_{1}$	-3.7571	7.3935	0.5201	0.4387	0.4223	0.5997	0.6600
$\mathrm{MDN}_{3}$	-3.6779	6.1454	0.5030	0.3138	0.4868	0.7547	0.8015
$\mathrm{MDN}_{5}$	-3.6770	6.0630	0.4993	0.3055	0.4889	0.8013	0.8553
$\mathrm{MDN}_{10}$	-3.6699	5.7824	0.5001	0.3043	0.4989	0.8892	0.9342
$\mathrm{MDN}_{25}$	-3.6822	5.2364	0.4817	0.3570	0.4518	0.8438	0.8911

Table 4: Predictive scoring and calibration diagnostics on the temperature extremes dataset.

Figure 3: Joint density reconstruction on held-out conditions (multivariate, $K=5$ ). Blue contours show the empirical test distribution in the 2D output space (Axis 1–Axis 2), and red contours show the GGMP 5 predictive distribution at the same inputs. Each panel corresponds to a different held-out condition. Overall alignment of contour shape and multimodal structure indicates that the GGMP captures the major geometry of the joint distribution.

Model	Energy	Sliced Wasserstein-1
$\mathrm{GP}_{1}$	0.1764 $\pm$ 0.1182	0.2302 $\pm$ 0.0983
$\mathrm{MDN}_{1}$	0.0871 $\pm$ 0.0535	0.1980 $\pm$ 0.0555
$\mathrm{MDN}_{3}$	0.0820 $\pm$ 0.0739	0.1695 $\pm$ 0.0626
$\mathrm{MDN}_{5}$	0.0712 $\pm$ 0.0442	0.1565 $\pm$ 0.0468
$\mathrm{MDN}_{10}$	0.0652 $\pm$ 0.0539	0.1414 $\pm$ 0.0579
$\mathrm{MDN}_{25}$	0.0652 $\pm$ 0.0586	0.1423 $\pm$ 0.0551

Table 5: Joint metric summary across held-out distributions (mean

\pm

std).

Model	Bhattacharyya	Symmetric KL	Wasserstein-1	$L^{1}$
$\mathrm{GP}_{1}$	0.1526 $\pm$ 0.0662	1.9729 $\pm$ 0.9653	0.3436 $\pm$ 0.1769	0.6563 $\pm$ 0.1645
$\mathrm{MDN}_{1}$	0.1078 $\pm$ 0.0331	2.0924 $\pm$ 0.6608	0.1399 $\pm$ 0.1464	0.4713 $\pm$ 0.1012
$\mathrm{MDN}_{3}$	0.0884 $\pm$ 0.0361	1.4679 $\pm$ 0.5218	0.1674 $\pm$ 0.1453	0.4608 $\pm$ 0.1318
$\mathrm{MDN}_{5}$	0.0772 $\pm$ 0.0167	1.1918 $\pm$ 0.2770	0.1536 $\pm$ 0.1401	0.4595 $\pm$ 0.0683
$\mathrm{MDN}_{10}$	0.0734 $\pm$ 0.0330	1.2620 $\pm$ 0.5892	0.1154 $\pm$ 0.1391	0.4205 $\pm$ 0.1010
$\mathrm{MDN}_{25}$	0.0734 $\pm$ 0.0285	1.2057 $\pm$ 0.3635	0.1514 $\pm$ 0.1300	0.4152 $\pm$ 0.1306

Table 6: Marginal metric summary for Axis 1 (mean

\pm

std).

Model	Bhattacharyya	Symmetric KL	Wasserstein-1	$L^{1}$
$\mathrm{GP}_{1}$	0.1751 $\pm$ 0.1213	2.0973 $\pm$ 1.5235	0.0954 $\pm$ 0.0527	0.7393 $\pm$ 0.3263
$\mathrm{MDN}_{1}$	0.3139 $\pm$ 0.1338	5.2772 $\pm$ 1.7122	0.0579 $\pm$ 0.0537	0.9554 $\pm$ 0.2201
$\mathrm{MDN}_{3}$	0.2382 $\pm$ 0.1520	3.9513 $\pm$ 2.2437	0.0682 $\pm$ 0.0623	0.8096 $\pm$ 0.2654
$\mathrm{MDN}_{5}$	0.2192 $\pm$ 0.1645	3.5751 $\pm$ 2.4068	0.0708 $\pm$ 0.0437	0.7614 $\pm$ 0.2973
$\mathrm{MDN}_{10}$	0.2217 $\pm$ 0.1461	3.5517 $\pm$ 2.2105	0.0881 $\pm$ 0.0366	0.7930 $\pm$ 0.2525
$\mathrm{MDN}_{25}$	0.2063 $\pm$ 0.1501	3.4022 $\pm$ 2.3282	0.0695 $\pm$ 0.0488	0.7459 $\pm$ 0.2736

Table 7: Marginal metric summary for Axis 2 (mean

\pm

std).

Model	Log Score	CRPS	PIT Mean	PIT Std	Cov 50%	Cov 90%	Cov 95%
$\mathrm{GP}_{1}$	-6.6629	0.8140	0.6810	0.2852	0.4568	0.6856	0.7286
$\mathrm{MDN}_{1}$	-6.4721	0.2088	0.5088	0.2169	0.7114	0.9823	0.9936
$\mathrm{MDN}_{3}$	-6.1633	0.2071	0.5038	0.2434	0.6349	0.9720	0.9898
$\mathrm{MDN}_{5}$	-6.2886	0.2053	0.5102	0.2567	0.5860	0.9699	0.9885
$\mathrm{MDN}_{10}$	-5.5040	0.2024	0.4983	0.2566	0.5857	0.9681	0.9886
$\mathrm{MDN}_{25}$	-6.4337	0.2013	0.5055	0.2509	0.6040	0.9739	0.9884

Table 8: Predictive scoring and calibration diagnostics on the additive manufacturing dataset.

Dataset	GGMP K	$\mathcal{L}_{\mathrm{dens}}(w_{1/K})$	$\mathcal{L}_{\mathrm{dens}}(w^{*})$	$\Delta_{w_{1/K}\to w^{*}}$
Synthetic	3	$-$ 381.08	$-$ 380.64	0.12%
	5	$-$ 345.32	$-$ 345.18	0.04%
	10	$-$ 307.32	$-$ 306.12	0.39%
	25	$-$ 301.29	$-$ 299.62	0.55%
Temperature	3	$-$ 22,379.2	$-$ 22,378.1	$<$ 0.01%
	5	$-$ 22,306.1	$-$ 22,304.9	$<$ 0.01%

Table 9: Effect of weight optimization on the distributional log-likelihood.

\Delta_{w_{1/K}\to w^{*}}

is the relative improvement

(\mathcal{L}_{\mathrm{opt}}-\mathcal{L}_{\mathrm{equal}})/|\mathcal{L}_{\mathrm{equal}}|\times 100

$K$	$\mathcal{L}(w{=}1/K)$	$\mathcal{L}(w^{*})$	$\mathcal{L}(w^{*}(x))$	$\Delta_{w^{}\to w^{}(x)}$
3	$-$ 1,937,212	$-$ 1,881,962	$-$ 1,877,461	0.24%
5	$-$ 1,966,832	$-$ 1,917,670	$-$ 1,902,757	0.78%
10	$-$ 1,917,485	$-$ 1,878,141	$-$ 1,868,909	0.49%
25	$-$ 2,143,543	$-$ 2,101,179	$-$ 2,088,154	0.62%

Table 10: Distributional log-likelihood on the additive-manufacturing dataset under equal weights, optimized shared weights, and optimized input-dependent weights ( 24 ).

\Delta_{w^{*}\to w^{*}(x)}

is the relative improvement from shared to input-dependent weights,

(\mathcal{L}(w^{*}(x))-\mathcal{L}(w^{*}))/|\mathcal{L}(w^{*})|\times 100

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

$K$	GMM + Align	GP Training	Weight Opt	Prediction	GGMP Total	MDN Total
1	$<\!1$	39	0	1	$<\!41$	1,318
3	9	64	3	1	77	1,241
5	21	79	9	1	110	1,212
10	50	132	9	1	192	1,178
25	144	242	12	1	399	1,198

Appendix

Table 11: Wall-clock runtimes (seconds) for the synthetic function experiment (

N=300

N*T=600{,}000

p=1

$K$	GMM + Align	GP Training	Weight Opt	Prediction	GGMP Total	MDN Total
1	$<\!1$	593	0	57	$<\!651$	26,613
3	177	4,842	28	58	5,105	26,452
5	329	7,150	108	63	7,650	27,130
10	717	13,317	206	70	14,310	26,828
25	1,482	20,935	1,059	82	23,558	27,164

Appendix

Table 12: Wall-clock runtimes (seconds) for the U.S. temperature extremes experiment (

N\approx 7{,}000

N*T_{\text{GGMP}}\approx 50,000,000

N*T_{\text{MDN}}=10,000,000

p=1

$K$	GMM + Align	GP Training	Weight Opt		Prediction	GGMP Total	MDN Total
			$w^{*}$	$w^{*}(x)$		( $w^{*}$ )
1	$<\!1$	3	0	0	$<\!1$	$<\!5$	1,009
3	15	8	2	19	3	28	1,018
5	24	11	7	204	5	47	997
10	42	18	41	528	6	107	1,035
25	256	26	123	929	7	412	991

Appendix

Table 13: Wall-clock runtimes (seconds) for the additive manufacturing experiment (

N=24

N*T\approx 600{,}000

p=2

). Weight optimization is reported for shared weights (

w^{*}

) and input-dependent weights (

w^{*}(x)

); the GGMP total uses

w^{*}

Figure 4: Held-out distribution reconstruction across mixture complexity for the synthetic function. Each row shows a different number of mixture components; each column is one of four approximately equally spaced held-out inputs. In every panel, the ground-truth is shown in gray, and the model prediction is red. Increasing $K$ improves local shape fidelity—especially multimodal structure—while low $K$ underfits fine-scale features. When $K<K_{\text{true}}$ , we witness an averaging phenomenon; and when $K>K_{\text{true}}$ , we observe a distribution that looks like it could have come from a mixture with fewer components, indicating that components are likely stacked.

Explore similar work

Predictively Oriented Gaussian Process Posteriors
Oct 2, 2026Callum Lau, Jeremias Knoblauch, Louis SharrockGaussian Process Regression Uncertainty Quantification
University College London
Conditioning Gaussian Processes on Almost Anything
May 20, 2026Henry Moss, Lachlan Astfalck, Thomas Cowperthwaite +5Gaussian Processes Conditional Distribution Estimation
Lancaster University · University of New South Wales · University of Cambridge +1
Scalable Gaussian process inference via neural feature maps
May 11, 2026Anthony StephensonGaussian Processes Kernel Methods
School of Mathematics University of Bristol

Oct 2, 2026J/K move · Enter open · S save

Predictively Oriented Gaussian Process Posteriors

Callum Lau, Jeremias Knoblauch, Louis Sharrock

University College London

Gaussian Process Regression Uncertainty Quantification Uncertainty Calibration Bayesian Inference Model Misspecification

Open paper AskPDF arXiv

Gaussian Processes (GPs) are a powerful tool for modelling and quantifying uncertainty in functional relationships. However, they require practitioners to make a number of design decisions, such as the choice of the kernel and the observation model. Suboptimal choices can produce misspecified models that do not capture the underlying data generating process. We introduce Predictively Oriented Gaussian Processes (PrO-GPs), which treat predictive uncertainty as the primary inferential target and provide a robust alternative to standard GPs. Although direct computation of a PrO posterior for nonparametric models is intractable, we derive a reduced formulation and practical sampling scheme for efficient computation. Through synthetic and real data experiments, we show that PrO-GPs produce better calibrated predictive distributions under model misspecification compared to standard GP approaches.

Abstract

Explore similar work

Predictively Oriented Gaussian Process Posteriors

Conditioning Gaussian Processes on Almost Anything

Scalable Gaussian process inference via neural feature maps