Gaussian process (GP) regression is widely used for uncertainty quantification, yet the standard formulation assumes noise-free covariates. When inputs are measured with error, this errors-in-variables (EIV) setting can lead to optimistically narrow posterior intervals and biased decisions. We study GP regression under input measurement uncertainty by representing each noisy input as a probability measure and defining covariance through Wasserstein distances between these measures. Building on this perspective, we instantiate a deterministic projected Wasserstein ARD (PWA) kernel whose one-dimensional components admit closed-form expressions and whose product structure yields a scalable, positive-definite kernel on distributions. Unlike latent-input GP models, PWA-based GPs (\PWAGPs) handle input noise without introducing unobserved covariates or Monte Carlo projections, making uncertainty quantification more transparent and robust.
Figures & tables
Method (paper name)
Kernel input object
Dist.
Latent X
Det.
EIV mechanism
Citations
Regular GP (RBF / Matérn / Exp)
Point estimate (empirical mean mi )
N
N
Y
Ignores input uncertainty (variance / shape); standard baseline in Sec. 3 and Table 1.
Williams and Rasmussen, (2006)
Aggregated Regular GPs
Noisy samples {Uij} as point inputs (fit per- j GP; average)
N
N
Y
Ensemble over replicate-specific GPs; can amplify variance in sparse regions (as observed).
This paper.
IV-function / cov-only encoding
Gaussian summary (μx,Σx) but only Σx used in kernel
OT-based kernel on distributions; PD only in special cases (e.g., Gaussian p=2 , or 1D).
Candelieri et al., (2022) ; Peyré et al., (2019)
sliced WGP (Eq. (6), SWGP)
Measure μ ; average Wp over random 1D projections u
Y
N
MC
PD preserved via expectation over projections; stochastic projections add Monte Carlo variability and cost.
Meunier et al., (2022) ; Bonet et al., (2023)
PWA-GP (Eq. (7))
Coordinate-wise 1D marginals {μi}i=1d
Y
N
Y
Product of 1D Wasserstein kernels with dimension-specific hyperparameters (ARD-like); PD via product closure.
This paper.
Table 1: Compact comparison of EIV / uncertain-input methods mentioned in the manuscript (including experimental baselines and Wasserstein variants) plus an added distribution-input competitor family (KME/MMD). Dist. indicates whether the model explicitly consumes a distribution/set input (Y* = Gaussian-summary distribution but only covariance used). Det.: deterministic vs Monte Carlo (MC) due to random projections. OT = optimal transport; PD = positive definite; KME = kernel mean embedding; MMD = maximum mean discrepancy; PCA = principal component analysis.
Figure 1: Illustration of error-in-variable regression problem. In both panels, the true function is y=f(X)=2Xsin(10π⋅X)+(X−1)4 , but in each case the “true” input locations X are contaminated with measurement errors (with standard deviation of 0.01 and 0.05 on the left and right, respectively). Using a GP to accurately infer the true function (the blue line) must account for the fact that the input locations U are uncertain.
Scenario
Metric
Reg
Agg
WGP
SWGP
PWA
PCPWA
UI
KME
MMD
2D-mean
RMSE
8.500e-03
1.061e-01
5.280e-02
1.015e+07
4.620e-02
1.615e-01
4.000e-04
2.100e-03
2.600e-03
Cov.
4.250e-01
1.000e-01
1.000e+00
0.000e+00
6.250e-01
9.750e-01
1.000e+00
1.000e+00
9.750e-01
CRPS
5.100e-03
5.940e-02
4.540e-02
7.752e+06
3.970e-02
1.108e-01
2.000e-03
8.900e-03
1.010e-02
1D-EIV
RMSE
3.090e-01
5.712e-01
3.014e-01
2.995e-01
3.014e-01
3.014e-01
3.462e-01
4.179e-01
3.029e-01
Cov.
6.500e-01
5.000e-02
9.670e-01
9.670e-01
9.670e-01
9.670e-01
5.670e-01
5.330e-01
6.670e-01
CRPS
1.848e-01
4.791e-01
1.716e-01
1.722e-01
1.716e-01
1.716e-01
2.179e-01
2.844e-01
1.775e-01
Table 2: Simulated distributional regression experiments. For each scenario we report test RMSE, empirical coverage of nominal 90% predictive intervals (Cov.), and CRPS. “Reg” = Regular GP on empirical means, “Agg” = Aggregated GP, “UI” = uncertain-input GP. Extremely large SWGP values indicate numerical divergence. Values of Cov. above 0.9 are shown in bold .
5% error
0.5% error
Method
RMSE
Cov.
CRPS
RMSE
Cov.
CRPS
RBF GP
0.2186
0.910
0.0883
0.2177
0.920
0.0806
Matérn 3/2 GP
0.2351
0.930
0.1010
0.2378
0.930
0.0997
Matérn 5/2 GP
0.2228
0.940
0.0912
0.2242
0.940
0.0892
Exp. GP
0.2432
0.980
0.1380
0.2847
0.980
0.1392
PWA-GP
0.2798
0.980
0.1690
0.3373
0.980
0.1681
Table 3: Accelerator dataset: test RMSE, empirical coverage of nominal 90% predictive intervals (Cov.), and CRPS for two relative error levels. Euclidean GPs use empirical means as inputs; distributional GPs act on full input distributions.
Figure 2: RMSE (left) and CRPS (right) for GP-RBF vs. WGP -RBF across training/testing years 1987-2022. GP-RBF degrades when trained on recent and tested on older years, indicating poor generalization under temporal shifts. WGP -RBF remains robust, consistently achieving lower errors and better probabilistic calibration.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Method
Effective # inputs
Kernel assembly cost (big-O)
Regular GP on means ( Williams and Rasmussen,, 2006 )
N
O(N2d)
Aggregated GP on samples ( Williams and Rasmussen,, 2006 )
NM
O(N2M2d) (matrix) +O(N3M3) (training)
Uncertain-input GP ( Girard et al.,, 2002 ; McHutchon and Rasmussen,, 2011 )
N
O(N2d3) (full cov.) or O(N2d) (diag)
Full WGP ( Mallasto and Feragen,, 2017 ; Bachoc et al.,, 2017 ; Candelieri et al.,, 2022 )
N
O(N2M2d) to O(N2M3)
PWA-GP (separable Wasserstein)
N
O(N2dMlogM)
PCPWA (PCA-based PWA)
N
O(NMd2) (PCA) +O(N2kMlogM)
Appendix
Table F.1: Leading-order cost of assembling the N×N kernel matrix for the different methods, assuming mi≍M samples per cloud. All methods then incur a shared O(N3) cost for exact GP training.
Gaussian process (GP) regression with a single global GP (GP-glo) incurs cubic computational cost, limiting scalability to large datasets. Product-of-experts GP models (GP-pro), which combine local GP models to capture global correlations, alleviate this computational burden. However, training local experts on disjoint data subsets can lead to overestimated posterior variances. We propose GP-pro-c, a product-of-experts GP model that calibrates these variances using an information-based method. The method exploits the monotonicity and submodularity of information gain in GPs to define a calibration ratio that reduces the posterior variance of individual local GP models. We evaluate GP-pro-c using negative log-likelihood (NLL), root mean squared error (RMSE), and expected normalised calibration error (ENCE). Experiments on four synthetic functions and six regression datasets show that GP-pro-c achieves average reductions of 2.3% in NLL and 12.0% in ENCE compared with the uncalibrated GP-pro model. The proposed method mitigates posterior variance overestimation while maintaining predictive accuracy and reducing computational complexity. GP-pro-c provides a promising approach for uncertainty estimation in scalable GP models and may serve as a useful surrogate model for Bayesian optimisation with high-dimensional and large-scale data.
Yean Hoon Ong, Paolo Barucca, Wei Pan +1
University College London, London, UK · The University of Manchester, Manchester, UK
Gaussian Processes (GPs) are a powerful tool for modelling and quantifying uncertainty in functional relationships. However, they require practitioners to make a number of design decisions, such as the choice of the kernel and the observation model. Suboptimal choices can produce misspecified models that do not capture the underlying data generating process. We introduce Predictively Oriented Gaussian Processes (PrO-GPs), which treat predictive uncertainty as the primary inferential target and provide a robust alternative to standard GPs. Although direct computation of a PrO posterior for nonparametric models is intractable, we derive a reduced formulation and practical sampling scheme for efficient computation. Through synthetic and real data experiments, we show that PrO-GPs produce better calibrated predictive distributions under model misspecification compared to standard GP approaches.
Multi-output Gaussian Processes provide principled uncertainty-aware learning of vector-valued fields but are difficult to deploy in large-scale, distributed, and streaming settings due to their computational and centralized nature. This paper proposes a Consensus-based Recursive Multi-Output Gaussian Process (CRMGP) framework that combines recursive inference on shared basis vectors with neighbour-to-neighbour information-consensus updates. The resulting method supports parallel, fully distributed learning with bounded per-step computation while preserving inter-output correlations and calibrated uncertainty. Experiments on synthetic wind fields and real LiDAR data demonstrate that CRMGP achieves competitive predictive performance and reliable uncertainty calibration, offering a scalable alternative to centralized Gaussian process models for multi-agent sensing applications.
Yogesh Prasanna Kumar Rao, Tamas Keviczky, Raj Thilak Rajan