Outlier-Robust Multi-Output Gaussian Processes

Joshua Rooijakkers; Leiv Rønneberg; François-Xavier Briol; Jeremias Knoblauch; Matias Altamirano

stat.MLOct 30, 2025

Outlier-Robust Multi-Output Gaussian Processes

Authors: Joshua Rooijakkers, Leiv Rønneberg, François-Xavier Briol, Jeremias Knoblauch, Matias Altamirano

Organizations: Econometric Institute, Erasmus University Rotterdam, The Netherlands · Department of Mathematics, University of Oslo, Norway · Department of Statistical Science, University College London, United Kingdom

Abstract

Multi-output Gaussian process (MOGP) regression allows modelling dependencies among multiple correlated response variables. Similarly to standard Gaussian processes (GPs), MOGPs are sensitive to model misspecification and outliers, which can distort predictions within individual outputs. In the multi-output case, this situation can be further exacerbated as anomalous observations in one response can adversely influence predictions for other correlated outputs. To handle this situation, we propose R-MOGP, an MOGP that is provably robust to outliers in any individual response. Our approach preserves conjugacy, achieving robustness without sacrificing computational efficiency, and integrates naturally with sparse variational inference for scalability. Empirically, we show that R-MOGP achieves performance comparable to existing state-of-the-art robust MOGPs at a fraction of their computational costs.

Figures & tables

Figure 1: Navitoclax dose-response with outlier. We show the viability of two cancer cell lines exposed to varying doses of the drug Navitoclax.

Figure 2: Synthetic imputation experiment with focused outliers. We consider a regression problem (

T=2

) with structured missingness in

y_{1}

and outliers in

y_{2}

. The MOGP predictives are sensitive to outliers, with errors propagating across outputs, whereas R-MOGP is robust in both outputs.

Outliers	Method	RMSE	NLPD
✓	MOGP	$0.20\pm 0.03$	$-0.37\pm 0.04$
	R-MOGP ( $w_{\textup{RCGP}}$ )	$0.18\pm 0.04$	$-0.44\pm 0.07$
	R-MOGP ( $w_{\textup{RMOGP}}$ )	$0.11\pm 0.01$	$-0.67\pm 0.08$
✗	MOGP	$0.093\pm 0.004$	$-0.94\pm 0.04$
	R-MOGP ( $w_{\textup{RCGP}}$ )	$0.093\pm 0.004$	$-0.94\pm 0.04$
	R-MOGP ( $w_{\textup{RMOGP}}$ )	$0.093\pm 0.004$	$-0.95\pm 0.04$

Table 1: Average RMSE and NLPD and standard deviation over 20 seeds under well-specified and outlier-contaminated settings.

Figure 3: Trade-off between average total runtime and NLPD across several outlier scenarios and methods.

Figure 5: Average runtime in seconds (left) and RMSE (right) across 10 runs on Electricity Load forecasting.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 6 — Figure 6: Synthetic imputation experiment using R-MOGPs based on the Linear Model of Coregionalisation (LMC) and Process Convolution (PC) frameworks.

Figure 7: Synthetic imputation experiment for MOGP, R-MOGP,

t

-MOGP, and L-MOGP.

Figure 8: Coverage plots for MOGP and R-MOGP in the synthetic imputation experiment with

N=1000

, with and without outliers. Coverage is the proportion of uncontaminated observations that fall within predictive intervals with nominal coverage

q

Figure 9: Posterior mean of R-MOGP (left), R-MOGP without shrinkage term (middle) and R-MOGP without noise term (right) for varying values of

\epsilon_{2}

on the synthetic imputation experiment.

Outliers	Method	Unweighted		Weighted		Squared
		RMSE	NLPD	RMSE	NLPD	RMSE	NLPD
	MOGP	$0.20\pm 0.03$	$-0.37\pm 0.04$	$0.18\pm 0.03$	$-0.48\pm 0.05$	$0.18\pm 0.03$	$-0.52\pm 0.07$
✓	R-MOGP ( $w_{\textup{RCGP}}$ )	$0.19\pm 0.03$	$-0.38\pm 0.04$	$0.19\pm 0.04$	$-0.40\pm 0.04$	$0.18\pm 0.04$	$-0.44\pm 0.07$
	R-MOGP ( $w_{\textup{RMOGP}}$ )	$0.18\pm 0.04$	$-0.38\pm 0.04$	$0.13\pm 0.03$	$-0.53\pm 0.06$	$0.11\pm 0.01$	$-0.67\pm 0.08$
	MOGP	$0.093\pm 0.003$	$-0.95\pm 0.04$	$0.093\pm 0.004$	$-0.95\pm 0.04$	$0.093\pm 0.004$	$-0.95\pm 0.04$
✗	R-MOGP ( $w_{\textup{RCGP}}$ )	$0.093\pm 0.003$	$-0.95\pm 0.03$	$0.093\pm 0.004$	$-0.94\pm 0.04$	$0.093\pm 0.004$	$-0.94\pm 0.04$

Appendix

Table 3: Comparison of MOGP and R-MOGP (

w_{\textup{RCGP}}

) and R-MOGP (

w_{\textup{RMOGP}}

) under different LOO-CV weighting schemes (unweighted, weighted, squared weights). The results show that R-MOGP (

w_{\textup{RMOGP}}

) combined with a squared weighted hyperparameter optimisation objective is most robust to outliers.

Figure 10: Effect of the weight function on the robustness of R-MOGP. R-MOGP (

w_{\scriptscriptstyle\textup{RMOGP}}

) uses the weight function described in this paper, while the R-MOGP (

w_{\scriptscriptstyle\textup{RCGP}}

) use the original RCGP weight with

\gamma_{t}(\mathbf{x},\mathbf{y})=m_{t}(\mathbf{x})=\frac{1}{N}\sum^{N}_{i=1}y_{i,t}

. Among all methods, only R-MOGP (

w_{\scriptscriptstyle\textup{RMOGP}}

) is robust to the contamination.

Panel of Figure 11 — Figure 11: Performance of MOGP and R-MOGP as the number of outliers in the observed vector increases. We generate $N=80$ data points from an ICM across $T=10$ outputs. We then contaminate $\epsilon_{s}=10\%$ of the observations with multivariate outliers for $s\in\{1,\dots,S\}$ with $S\leq T$ increasing incrementally. The prior mean is $m_{t}(\mathbf{x})=\frac{1}{N}\sum_{i=1}^{N}y_{i,t}$ . As the percentage of outliers grows, the performance of the MOGP drops substantially faster than that of the R-MOGP.

Outliers	Method	RMSE	NLPD
None	MOGP	0.12 (0.01)	$-0.92$ (0.08)
	R-MOGP	0.12 (0.01)	$-0.88$ (0.07)
	MLP	0.10 (0.004)	–
	BNN	0.21 (0.01)	0.18 (0.13)
	XGBoost	0.083 (0.01)	–
Uniform	MOGP	2.27 (0.29)	1.26 (0.06)

Appendix

Table 4: RMSE and NLPD for MOGP, R-MOGP, and baseline machine learning methods on the Energy Efficiency dataset under different outlier settings.

Figure 12: Navitoclax dose-response analysis with outlier in separate plots for MOGP, R-MOGP,

t

-MOGP and L-MOGP.

Method	RMSE	NLPD	Runtime (s)
MOGP (no outliers)	0.066 (0.012)	$-1.23$ (0.11)	0.048
MOGP	0.170 (0.020)	$-0.21$ (0.05)	0.049
R-MOGP	0.081 (0.011)	$-1.00$ (0.07)	0.040
$t$ -MOGP	0.067 (0.013)	$-1.06$ (0.06)	3.820
L-MOGP	0.068 (0.013)	$-0.65$ (0.73)	1.060

Appendix

Table 5: Predictive performance and runtime of MOGP variants on the Navitoclax experiment.

Figure 13: Robustness of MOGP and R-MOGP in MBS trades. The outlier at 4 pm in the 5.5% MBS distorts the MOGP predictions for that output and propagates errors to the 6% MBS. In contrast, R-MOGP is robust.

Gaussian Processes

Gaussian Processes are probabilistic models of functions that use kernels to make predictions with explicit uncertainty.

See all 121 papers

Variational Outlier-Robust Gaussian Process Regression with Generative Modeling
Aug 17, 2026Arslan Majal, Aamir Hussain ChughtaiGaussian Process Regression Robust Regression
Zunitech, LLC, Houston, USA · Institute of Data Science, University of Engineering and Technology, Lahore, Pakistan
Transformed Latent Variable Multi-Output Gaussian Processes
May 6, 2026Xiaoyu Jiang, Xinxing Shi, Sokratia Georgaka +2Deep Kernel Learning Latent Variable Models
Department of Computer Science, University of Manchester, Manchester, UK · Faculty of Biology, Medicine and Health, University of Manchester, Manchester, UK
Adapting Nonstationary Multi-output Gaussian Processes to Bayesian Optimization
Sep 26, 2026Zikai XieMulti-Output Gaussian Processes Bayesian Optimization
State Key Laboratory of Precision and Intelligent Chemistry University of Science and Technology of China Hefei, Anhui 230000, China

Aug 17, 2026J/K move · Enter open · S save

Variational Outlier-Robust Gaussian Process Regression with Generative Modeling

Arslan Majal, Aamir Hussain Chughtai

Zunitech, LLC, Houston, USA · Institute of Data Science, University of Engineering and Technology, Lahore, Pakistan

Gaussian Process Regression Robust Regression Generative Modeling

Open paper AskPDF arXiv

Outliers can substantially distort Gaussian process regression (GPR) due to its conventional Gaussian observation likelihood, leading to inaccurate model learning and prediction. To address this limitation, this article introduces a generative GPR model that captures observation-specific contamination and adaptively mitigates the influence of outliers. Subsequently, a variational generalized expectation-maximization procedure is used to learn the latent variables and GPR model parameters. Experiments on synthetic and real datasets under different contamination settings demonstrate that the proposed method remains competitive with-and in several cases outperforms-robust GPR baselines in prediction accuracy. Moreover, the proposed method shares the cubic computational scaling of the compared GPR methods.

Abstract

Gaussian Processes

Explore similar work

Variational Outlier-Robust Gaussian Process Regression with Generative Modeling

Transformed Latent Variable Multi-Output Gaussian Processes

Adapting Nonstationary Multi-output Gaussian Processes to Bayesian Optimization