Multi-Output Gaussian Processes

Latest papers

11 papers3 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Sep 26, 2026

Adapting Nonstationary Multi-output Gaussian Processes to Bayesian Optimization

Multi-objective Bayesian optimization (MOBO) commonly relies on independent Gaussian processes (GPs) with stationary kernels, limiting its ability to represent nonstationary structure and share information between objectives. However, expressive nonstationary GPs do not necessarily make reliable BO decisions. We study this mismatch for the multi-output low-rank nonstationary (MO-LRN) GP: strong training fit can coexist with large off-design errors and optimistic acquisition predictions. We introduce MOLRN-BO, which combines a regularized shared-spectral surrogate with objective-specific residuals, prequential mean correction and tempered covariance scaling, and Pareto-local qLogEHVI optimization with periodic global search. Experiments on 12 deterministic bi-objective benchmarks show that MOLRN-BO substantially improves upon the original MO-LRN and achieves the best average problem ranks for final normalized hypervolume and normalized inverted generational distance among nine evaluated algorithms. It also achieves the strongest adverse-tail performance while remaining competitive with the leading baselines in anytime optimization. Ablation studies further show that the shared spectral construction improves off-design prediction, the local--global decision policy improves optimization performance, and hierarchical calibration reduces systematic candidate bias. These results demonstrate that nonstationary multi-output surrogates can deliver strong and robust MOBO performance when their structure and use are explicitly adapted to the demands of sequential optimization.

Zikai Xie

State Key Laboratory of Precision and Intelligent Chemistry University of Science and Technology of China Hefei, Anhui 230000, China

Multi-Output Gaussian Processes Bayesian Optimization Multi-Objective Optimization Gaussian Processes Gaussian Process Regression

Ask PDF

Sep 24, 2026

Residual Correlation as a Diagnostic for Joint-Uncertainty Gains from GP Coregionalisation

In multi-target regression, correlated targets are often coupled through multi-output Gaussian processes with an intrinsic model of coregionalisation (GP-ICM), assuming that sharing statistical strength improves overall performance. In practice, the benefits are inconsistent. Across the settings studied, we find that the main benefit of coregionalisation is joint uncertainty quantification rather than point prediction. Raw target correlation does not predict when coupling helps; in the separable GP-ICM settings studied here, residual correlation, the cross-target dependence left unexplained by independent per-target predictors, is the strongest predictor of joint-uncertainty gains. We introduce a lightweight diagnostic,

D_{\rm logdet}=-\frac{1}{2}\log\det R_{\rm res}

, which represents the idealised joint negative log-likelihood (NLL) gain from modelling a full rather than diagonal residual covariance and is computable from independent GPs alone. Across a controlled synthetic study, 16 multi-target benchmarks, and frozen transformer and convolutional neural network representations for keypoint regression, point prediction remains largely unchanged (

ΔR^2\approx 0

). In contrast,

D_{\rm logdet}

strongly predicts observed ICM NLL improvements (

ρ_s=-0.83

p<0.001

), outperforming heuristics such as the feature-to-sample ratio. We also propose Residual-ICM, which preserves independent marginal variances while adding residual-correlation structure to the joint covariance. Residual-ICM achieves the best average joint NLL among the compared methods, while the diagnostic indicates when covariance coupling is likely to be useful. The diagnostic is specific to global Gaussian residual dependence, the structure captured by separable coregionalisation.

Fangqin Zhou, Joaquin Vanschoren

Eindhoven University of Technology

Uncertainty Quantification Multi-Task Learning Gaussian Processes Multi-Output Gaussian Processes Gaussian Process Regression

Ask PDF

Sep 22, 2026

Resource-Efficient Distributed Recursive Gaussian Processes

Gaussian processes (GPs) provide a flexible framework for learning unknown functions from noisy measurements while quantifying predictive uncertainty, making them well suited for estimation in multi-agent systems. However, when measurements are collected by multiple agents, maintaining a unified GP model without centralized processing requires efficient distributed algorithms that can operate using local measurements and communication with neighboring agents. In this work, we develop two distributed recursive GP (RGP) algorithms for multi-output GP regression: ADMM-RGP and PDMM-RGP. We analyze the stability and convergence of both algorithms and develop parameter selection strategies to accelerate convergence, thus reducing the communication burden. The proposed methods are validated on a real-world multi-output wind dataset, and their convergence behavior is examined across communication graphs with varying connectivity. Numerical experiments demonstrate that ADMM-RGP and PDMM-RGP can significantly reduce communication relative to the state of the art, while maintaining comparable estimation accuracy and network-wide consensus.

Josephine King, Ali Emre Balci, Raj Thilak Rajan

Signal Processing Systems, Faculty of EEMCS, Delft University of Technology, Mekelweg 4, 2628CD, Delft, The Netherlands

Distributed Optimization Gaussian Processes Multi-Output Gaussian Processes Gaussian Process Regression

Ask PDF

Aug 26, 2026

Multi-output Gaussian process prediction of physical fields under linear equality constraints

We address the simultaneous prediction of multiple high-dimensional physical fields governed by linear equality constraints, a setting that arises in many real-world applications in physics machine learning. Gaussian process (GP) regression is a widely used surrogate modeling approach due to its effectiveness in small-sample regimes and its ability to provide uncertainty quantification. However, applying GP models in this setting raises two major challenges: the high dimensionality of the discretized output fields and the enforcement of the physical constraint in predictions. For the latter, a common strategy consists in deducing one output from the others via the constraint relation. Through a benchmark, we show that this deductive approach is sensitive to the arbitrary choice of which output to deduce, affecting both predictive accuracy and uncertainty quantification. Consequently, there is a need for an approach that treats all fields symmetrically while strictly respecting the underlying physics. Motivated by these limitations, we propose a robust framework for jointly modeling constrained multi-field data. Our approach first leverages a specific PCA procedure for multi-field data, coined row-wise PCA, which has the interesting property of preserving the constraint in the latent space. Since standard PCA strategies for multi-field data do not preserve such constraints, we investigate theoretically the optimality of the row-wise choice. In a second step, we consider a linearly-constrained multi-output GP approach based on a specific kernel parametrization which is trained on the latent space of row-wise PCA. The proposed framework is validated on a population dynamics problem and on an industrial CFD application, which involves the prediction of Reynolds stress tensor components under the incompressibility constraint.

Mahamat Hamdan Nassouradine, Clément Gauchy, Pierre-Emmanuel Angeli +1

Université Paris-Saclay, CEA, Service de Génie Logiciel pour la Simulation, Gif-sur-Yvette, 91191, France · Univ Rennes, Ensai, CNRS, CREST - UMR 9194, Rennes, F-35000, France · Université Paris-Saclay, CEA, Service de Thermohydraulique et de Mécanique des Fluides, Gif-sur-Yvette, 91191, France

Multi-Output Gaussian Processes Surrogate Modeling Gaussian Process Regression Physics-Informed ML

Ask PDF

Aug 12, 2026

A Factor Graph Approach to Scalable Multi-Output Gaussian Process Regression

Multi-output Gaussian process regression scales cubically in the number of observations times outputs, and dense kernel-matrix methods need bespoke handling whenever different outputs are observed at different inputs. We express multi-output Gaussian process regression as a Forney-style factor graph in which a nearest-neighbor chain orders a fixed candidate set of

C

inputs into a one-dimensional sequence. Along this chain, latent Matérn processes evolve through linear-Gaussian transition factors, while the linear model of coregionalization mixes

L

latent processes into

D

outputs through a deterministic mixing factor and per-output scalar observation factors. Posterior computation reduces to exact Gaussian message passing on the chain at cost

\mathcal{O}(C(DL^2 + L^3))

after chain construction, and missing observations omit their local factor without any covariance-matrix restructuring. The formulation therefore scales in the number of data samples and in the rate of missing observations, while remaining best suited to candidate sets in low input dimension. We compare the factor-graph formulation against an exact kernel-matrix baseline, a sparse-variational inducing-point baseline, and a nearest-neighbor baseline on a synthetic input-dimension sweep and on electricity time series forecasting. At low input dimension the factor-graph posterior tracks the exact kernel-matrix posterior closely, and the gap grows gradually as input dimension increases while staying competitive with both approximate baselines. On the electricity time series our factor-graph formulation matches all three baselines in forecast accuracy while scaling linearly in the number of data points, where the exact kernel-matrix method becomes infeasible and the inducing-point baseline remains substantially slower.

Wouter W. L. Nuijten, Esther G. van Pelt, Albert Podusenko +2

Eindhoven University of Technology & Lazy Dynamics B.V., the Netherlands · Eindhoven University of Technology, Eindhoven, the Netherlands · Lazy Dynamics B.V., the Netherlands

Multi-Output Gaussian Processes Gaussian Process Regression

Ask PDF

Jun 4, 2026

Gaussian Process Latent Factor Regression for Low-Data, High-Dimensional Output Problems

In the sciences, regression tasks often require predicting high-dimensional outputs from few training examples. Multi-output Gaussian processes excel in low-data regimes but typically struggle with high-dimensional outputs. Compress-then-predict pipelines such as PCA-GP (principal component analysis plus Gaussian process regression) handle high dimensionality, but rely on bases optimized for reconstruction rather than prediction. To address this gap, we propose a model that represents each output as a linear-Gaussian decoding of a low-dimensional latent state drawn from a Gaussian process prior. By analytically marginalizing the decoder weights, we couple compression and prediction in a single objective that scales to high-dimensional outputs. We refer to this model as Gaussian process latent factor regression (GPLFR). We demonstrate GPLFR by building the first spatially resolved emulator of global climate models for rocky exoplanets.

Edward T. Stevenson, Eric T. Wolf, Mei Ting Mak +2

University of Cambridge · University of Colorado Boulder · University of Oxford +1

Latent Variable Models Climate Emulation Multi-Output Gaussian Processes Dimensionality Reduction Gaussian Process Regression

Ask PDF

May 13, 2026

Safe Bayesian Optimization for Uncertain Correlation Matrices in Linear Models of Co-Regionalization

This paper extends safety guarantees for multi-task Bayesian optimization with uncertain co-regionalization matrices from intrinsic co-regionalization models to linear models of co-regionalization. The latter allows for more flexible modeling of the inter-task correlations by composing multiple features. We derive uniform error bounds for vector-valued functions sampled from a Gaussian process with a linear model of co-regionalization kernel. Furthermore, we show the potential performance gains of linear models of co-regionalization in a numerical comparison on a safe multi-task Bayesian optimization benchmark.

Jannis Lübsen, Annika Eichler

Institute of Control Systems, Hamburg University of Technology, Hamburg, Germany · Institute of Control Systems, Hamburg University of Technology · Deutsches Elektronen-Synchrotron, Hamburg, Germany

Bayesian Optimization Gaussian Processes Safe Bayesian Optimization Multi-Output Gaussian Processes

Ask PDF

May 6, 2026

Transformed Latent Variable Multi-Output Gaussian Processes

Multi-Output Gaussian Processes (MOGPs) provide a principled probabilistic framework for modelling correlated outputs but face scalability bottlenecks when applied to datasets with high-dimensional output spaces. To maintain tractability, existing methods typically resort to restrictive assumptions, such as employing low-rank or sum-of-separable kernels, which can limit expressiveness. We propose the Transformed Latent Variable MOGP (T-LVMOGP), a novel framework that scales MOGPs to a massive number of outputs while preserving the capacity to capture meaningful inter-output dependencies. T-LVMOGP constructs a flexible multi-output deep kernel by mapping inputs and output-specific latent variables into an embedding space using a Lipschitz-regularised neural network. Combined with stochastic variational inference, our model effectively scales to high-dimensional output settings. Across diverse benchmarks, including climate modelling with over 10,000 outputs and zero-inflated spatial transcriptomics data, T-LVMOGP outperforms baselines in both predictive accuracy and computational efficiency.

Xiaoyu Jiang, Xinxing Shi, Sokratia Georgaka +2

Department of Computer Science, University of Manchester, Manchester, UK · Faculty of Biology, Medicine and Health, University of Manchester, Manchester, UK

Deep Kernel Learning Latent Variable Models Gaussian Processes Multi-Output Gaussian Processes Kernel Methods

Ask PDF

Apr 11, 2026

Consensus-based Recursive Multi-Output Gaussian Process

Multi-output Gaussian Processes provide principled uncertainty-aware learning of vector-valued fields but are difficult to deploy in large-scale, distributed, and streaming settings due to their computational and centralized nature. This paper proposes a Consensus-based Recursive Multi-Output Gaussian Process (CRMGP) framework that combines recursive inference on shared basis vectors with neighbour-to-neighbour information-consensus updates. The resulting method supports parallel, fully distributed learning with bounded per-step computation while preserving inter-output correlations and calibrated uncertainty. Experiments on synthetic wind fields and real LiDAR data demonstrate that CRMGP achieves competitive predictive performance and reliable uncertainty calibration, offering a scalable alternative to centralized Gaussian process models for multi-agent sensing applications.

Yogesh Prasanna Kumar Rao, Tamas Keviczky, Raj Thilak Rajan

Multi-Output Gaussian Processes Gaussian Processes Gaussian Process Regression

Ask PDF

Nov 22, 2025

Improving Forecasts of Suicide Attempts for Patients with Little Data

Ecological Momentary Assessment (EMA) studies provide real-time data on suicidal thoughts and behaviors, but forecasting suicide attempts remains challenging: attempts are rare, and the pathways patients take to them are heterogeneous. Here, we investigate a cohort of patients from an EMA study with recorded suicide-related events. We show that a single model fit to all patients forecasts poorly, while idiographic (per-patient) models show improvement but overfit for those with little data. Based on this result, one may hypothesize that patients should be partitioned into subgroups---this way, similar patients' data can be pooled together to improve forecasts. However, we show that grouping patients at random already improves forecasts, with performance increasing monotonically with the number of groups. Moreover, we show that grouping patients by demographics yields worse forecasts than random groupings. From these results, we hypothesize that patient similarity is continuous, rather than discrete, and must be inferred from the data. This motivated us to use Latent Variable Multiple Output Gaussian Processes (LVMOGPs), adapted to our data. Preliminary results show that, even without careful kernel design, LVMOGPs already match the strongest baseline models on most metrics, and their latent spaces yield a similarity between patients that we can inspect directly. Because the cohort is conditioned on the outcome and the splits are not temporal, we read these results as evidence that idiographic structure exists and can be recovered, not as deployable forecasting performance---an area for future work.

Genesis Hang, Annie Chen, Hope Neveux +2

Wellesley College · Harvard University

Latent Variable Models Gaussian Processes Multi-Output Gaussian Processes Clinical Prediction Suicide Risk Modeling Clinical Risk Prediction

Ask PDF

Feb 17, 2025

Spectral structure learning for clinical time series

We develop and evaluate a structure learning algorithm for clinical time series. Clinical time series are multivariate time series observed in multiple patients and irregularly sampled, challenging existing structure learning algorithms. We assume that our times series are realizations of StructGP, a k-dimensional multi-output or multi-task stationary Gaussian process (GP), with independent patients sharing the same covariance function. StructGP encodes ordered conditional relations between time series, represented in a directed acyclic graph. We implement an adapted NOTEARS algorithm, which based on a differentiable definition of acyclicity, recovers the graph by solving a series of continuous optimization problems. Simulation results show that up to mean degree 3 and 20 tasks, we reach a median recall of 0.93% [IQR, 0.86, 0.97] while keeping a median precision of 0.71% [0.57-0.84], for recovering directed edges. We further show that the regularization path is key to identifying the graph. With StructGP, we proposed a model of time series dependencies, that flexibly adapt to different time series regularity, while enabling us to learn these dependencies from observations.

Ivan Lerner, Anita Burgun, Francis Bach

Université Paris Cité, Inria, Inserm, HeKA, F-75015 Paris, France · Service d’informatique biomédicale, AP-HP, Hôpital Georges Pompidou, F-75015 Paris, France · SIERRA, Inria Paris, F-75015 Paris, France

Multi-Output Gaussian Processes Time-Series Causal Discovery Causal Structure Learning Irregular Time-Series Modeling Gaussian Processes

Ask PDF