Multi-output Gaussian process prediction of physical fields under linear equality constraints

Mahamat Hamdan Nassouradine; Clément Gauchy; Pierre-Emmanuel Angeli; Sébastien da Veiga

stat.MLAug 26, 2026

Multi-output Gaussian process prediction of physical fields under linear equality constraints

Authors: Mahamat Hamdan Nassouradine, Clément Gauchy, Pierre-Emmanuel Angeli, Sébastien da Veiga

Organizations: Université Paris-Saclay, CEA, Service de Génie Logiciel pour la Simulation, Gif-sur-Yvette, 91191, France · Univ Rennes, Ensai, CNRS, CREST - UMR 9194, Rennes, F-35000, France · Université Paris-Saclay, CEA, Service de Thermohydraulique et de Mécanique des Fluides, Gif-sur-Yvette, 91191, France

Abstract

We address the simultaneous prediction of multiple high-dimensional physical fields governed by linear equality constraints, a setting that arises in many real-world applications in physics machine learning. Gaussian process (GP) regression is a widely used surrogate modeling approach due to its effectiveness in small-sample regimes and its ability to provide uncertainty quantification. However, applying GP models in this setting raises two major challenges: the high dimensionality of the discretized output fields and the enforcement of the physical constraint in predictions. For the latter, a common strategy consists in deducing one output from the others via the constraint relation. Through a benchmark, we show that this deductive approach is sensitive to the arbitrary choice of which output to deduce, affecting both predictive accuracy and uncertainty quantification. Consequently, there is a need for an approach that treats all fields symmetrically while strictly respecting the underlying physics. Motivated by these limitations, we propose a robust framework for jointly modeling constrained multi-field data. Our approach first leverages a specific PCA procedure for multi-field data, coined row-wise PCA, which has the interesting property of preserving the constraint in the latent space. Since standard PCA strategies for multi-field data do not preserve such constraints, we investigate theoretically the optimality of the row-wise choice. In a second step, we consider a linearly-constrained multi-output GP approach based on a specific kernel parametrization which is trained on the latent space of row-wise PCA. The proposed framework is validated on a population dynamics problem and on an industrial CFD application, which involves the prediction of Reynolds stress tensor components under the incompressibility constraint.

Figures & tables

Figure 1: Distribution of the RMSE on

f_{2}

(Branin) across

200

replications, for training set sizes

N\in\{20,50,100\}

and the three deduction scenarios. Each panel corresponds to a different choice of deduced output

l

. The CMOGP (blue) is identical across panels. The deductive methods exhibit sensitivity to the choice of

l

, with the independent GP showing systematic degradation when

f_{2}

is deduced (middle panel).

Figure 2: Win rate (%) of each method at

N=20

on the RMSE of each output (

f_{1}

f_{2}

f_{3}

), across the three deduction scenarios (

l\in\{1,2,3\}

). The CMOGP (blue) is the same model in all scenarios. Aggregated over 200 replications.

Figure 3: Left: normalized dominance criterion of Theorem 3 as a function of

m

, for

N\in\{30,15,10\}

. Right: RRMSE on output

p

for all methods and the three training set sizes (

m=1,\ldots,10

). Error bars represent

\pm 1

standard deviation over

10

seeds. The dominance criterion converges to zero beyond

m\approx 4

, and Row-CMO surpasses all alternatives at higher latent dimensions.

Figure 4: Zoomed view of the RRMSE on output

p

for

m\in\{5,\ldots,10\}

and training set sizes

N\in\{30,15,10\}

. Row-CMO (red, solid) consistently achieves the lowest RRMSE. The spread of Col-Indep , Fw-Indep , and Fw-LCM curves across deduction scenarios (

l\in\{0,1,2,3\}

) illustrates the sensitivity of the deductive approaches to the arbitrary choice of deduced output.

Figure 5: Scheme of the experiment carried out in ( Buice and Eaton, 2000b ) . The wall of the channel are defined by straight lines and circular arcs of centers

O_{1}

and

O_{2}

Figure 6: Left: dominance criterion of Theorem 3 for the CFD test case, showing the normalized difference

\text{Tr}(\mathbf{D}_{\text{col},m}^{2})-\text{Tr}(\mathbf{D}_{\text{row},m}^{2})

as a function of

m

. The small and decreasing values confirm near-equivalence of the two PCA strategies. Right: RRMSE on each output field (

\tau_{11}

\tau_{22}

k

) for all methods (

m=1,\ldots,10

). Row-CMO achieves the lowest RRMSE at

m\geq 6

. Error bars represent

\pm 1

standard deviation over

10

seeds.

Figure 7: RRMSE at

m=8

for each output field (

\tau_{11}

\tau_{22}

k

) across all methods and deduction scenarios. Row-CMO achieves the lowest mean RRMSE with the smallest variability. The spread across deduction scenarios within each family ( Col-Indep , Fw-Indep , Fw-LCM ) and the larger error bars of the deductive methods illustrate the residual sensitivity to the output selection, even in a regime of strong inter-field correlation.

Figure 8: Row-CMO predictions on the CFD test case for a representative test input ( $M=8$ ). From top to bottom: $\tau_{11}$ , $\tau_{22}$ , $k$ . Left to right: simulated field, predicted field, predictive standard deviation, and SRSE. The model accurately captures the spatial structure. Errors and uncertainty are concentrated in the recirculation zone of the diffuser.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

$C_{\upmu}$	$\upsigma_{k}$	$\upsigma_{\upvarepsilon}$	$C_{\upvarepsilon_{1}}$	$C_{\upvarepsilon_{2}}$
0.09	1.00	1.30	1.44	1.92

Appendix

Table 1: Usual values of the parameters of the standard

k-\upvarepsilon

model.

Panel of Figure 9 — Figure 9: Experimental data of Buice and Eaton (2000b) and probability density estimate of $C_{\upmu}$

$\tau_{11}$	$\tau_{22}$	$\tau_{33}$	$\tau_{12}$
$4.020\text{e-}3$	$1.576\text{e-}3$	$2.798\text{e-}3$	$1.020\text{e-}3$

Appendix

Table 2: Measured values for the Reynolds stress tensor component for

y/H=4

Figure 10: Histogram and gaussian KDE of the distribution of

C_{\upvarepsilon_{2}}

using a 1,000-sized sample of

\upbeta

values.

Panel of Figure 11 — Figure 11: Win rate (%) of each method on the RRMSE of each output ( $f_{1}$ , $f_{2}$ , $f_{3}$ ) for $N\in\{20,50,100\}$ , across all deduction scenarios ( $k_{*}\in\{1,2,3\}$ ). The highlighted column indicates the scenario where the evaluated output is the deduced one. The CMOGP (blue) is the same model in all scenarios. Results are aggregated over $200$ independent replications.

Figure 12: Distribution of the average

90\%

predictive interval length for

f_{2}

across 200 replications, for training set sizes

N\in{20,50,100}

and the three deduction scenarios. All methods achieve comparable empirical coverage rates. Shorter intervals thus indicate more informative predictions. The CMOGP maintains stable and shorter intervals across all scenarios.

Figure 13: RRMSE as a function of

m

for all four Lotka–Volterra outputs (

p

q

r

s

, columns) and training set sizes (

N=30,15,10

, rows). Row-CMO (red) is compared against Col-Indep and Fw-Indep with all deduction scenarios

k_{*}\in\{0,1,2,3\}

. Error bars represent

\pm 1

standard deviation over

10

seeds.

Explore similar work

Gaussian Process Latent Factor Regression for Low-Data, High-Dimensional Output Problems
Jun 4, 2026Edward T. Stevenson, Eric T. Wolf, Mei Ting Mak +2Latent Variable Models Dimensionality Reduction
University of Cambridge · University of Colorado Boulder · University of Oxford +1
Constrained Probabilistic Numerics: Gaussian Process Solvers for Linear PDEs under Linear Inequality Constraints
Oct 9, 2026Boyuan Deng, Kshitiz Upadhyay, Michael ShieldsPDE Solving Gaussian Processes
Department of Civil and Systems Engineering, Johns Hopkins University · Department of Aerospace Engineering and Mechanics, University of Minnesota
Conditioning Gaussian Processes on Almost Anything
May 20, 2026Henry Moss, Lachlan Astfalck, Thomas Cowperthwaite +5Gaussian Processes Conditional Distribution Estimation
Lancaster University · University of New South Wales · University of Cambridge +1

Jun 4, 2026, cs.LGJ/K move · Enter open · S save

Gaussian Process Latent Factor Regression for Low-Data, High-Dimensional Output Problems

Edward T. Stevenson, Eric T. Wolf, Mei Ting Mak, N. J. Mayne+1

University of Cambridge · University of Colorado Boulder · University of Oxford+1

Latent Variable Models Dimensionality Reduction Gaussian Process Regression

Open paper AskPDF arXiv

In the sciences, regression tasks often require predicting high-dimensional outputs from few training examples. Multi-output Gaussian processes excel in low-data regimes but typically struggle with high-dimensional outputs. Compress-then-predict pipelines such as PCA-GP (principal component analysis plus Gaussian process regression) handle high dimensionality, but rely on bases optimized for reconstruction rather than prediction. To address this gap, we propose a model that represents each output as a linear-Gaussian decoding of a low-dimensional latent state drawn from a Gaussian process prior. By analytically marginalizing the decoder weights, we couple compression and prediction in a single objective that scales to high-dimensional outputs. We refer to this model as Gaussian process latent factor regression (GPLFR). We demonstrate GPLFR by building the first spatially resolved emulator of global climate models for rocky exoplanets.

Abstract

Explore similar work

Gaussian Process Latent Factor Regression for Low-Data, High-Dimensional Output Problems

Constrained Probabilistic Numerics: Gaussian Process Solvers for Linear PDEs under Linear Inequality Constraints

Conditioning Gaussian Processes on Almost Anything