Organizations: Université Paris-Saclay, CEA, Service de Génie Logiciel pour la Simulation, Gif-sur-Yvette, 91191, France · Univ Rennes, Ensai, CNRS, CREST - UMR 9194, Rennes, F-35000, France · Université Paris-Saclay, CEA, Service de Thermohydraulique et de Mécanique des Fluides, Gif-sur-Yvette, 91191, France
We address the simultaneous prediction of multiple high-dimensional physical fields governed by linear equality constraints, a setting that arises in many real-world applications in physics machine learning. Gaussian process (GP) regression is a widely used surrogate modeling approach due to its effectiveness in small-sample regimes and its ability to provide uncertainty quantification. However, applying GP models in this setting raises two major challenges: the high dimensionality of the discretized output fields and the enforcement of the physical constraint in predictions. For the latter, a common strategy consists in deducing one output from the others via the constraint relation. Through a benchmark, we show that this deductive approach is sensitive to the arbitrary choice of which output to deduce, affecting both predictive accuracy and uncertainty quantification. Consequently, there is a need for an approach that treats all fields symmetrically while strictly respecting the underlying physics. Motivated by these limitations, we propose a robust framework for jointly modeling constrained multi-field data. Our approach first leverages a specific PCA procedure for multi-field data, coined row-wise PCA, which has the interesting property of preserving the constraint in the latent space. Since standard PCA strategies for multi-field data do not preserve such constraints, we investigate theoretically the optimality of the row-wise choice. In a second step, we consider a linearly-constrained multi-output GP approach based on a specific kernel parametrization which is trained on the latent space of row-wise PCA. The proposed framework is validated on a population dynamics problem and on an industrial CFD application, which involves the prediction of Reynolds stress tensor components under the incompressibility constraint.
Figures & tables
Figure 1: Distribution of the RMSE on f2 (Branin) across 200 replications, for training set sizes N∈{20,50,100} and the three deduction scenarios. Each panel corresponds to a different choice of deduced output l . The CMOGP (blue) is identical across panels. The deductive methods exhibit sensitivity to the choice of l , with the independent GP showing systematic degradation when f2 is deduced (middle panel).
Figure 2: Win rate (%) of each method at N=20 on the RMSE of each output ( f1 , f2 , f3 ), across the three deduction scenarios ( l∈{1,2,3} ). The CMOGP (blue) is the same model in all scenarios. Aggregated over 200 replications.
Figure 3: Left: normalized dominance criterion of Theorem 3 as a function of m , for N∈{30,15,10} . Right: RRMSE on output p for all methods and the three training set sizes ( m=1,…,10 ). Error bars represent ±1 standard deviation over 10 seeds. The dominance criterion converges to zero beyond m≈4 , and Row-CMO surpasses all alternatives at higher latent dimensions.
Figure 4: Zoomed view of the RRMSE on output p for m∈{5,…,10} and training set sizes N∈{30,15,10} . Row-CMO (red, solid) consistently achieves the lowest RRMSE. The spread of Col-Indep , Fw-Indep , and Fw-LCM curves across deduction scenarios ( l∈{0,1,2,3} ) illustrates the sensitivity of the deductive approaches to the arbitrary choice of deduced output.
Figure 5: Scheme of the experiment carried out in ( Buice and Eaton, 2000b ) . The wall of the channel are defined by straight lines and circular arcs of centers O1 and O2 .
Figure 6: Left: dominance criterion of Theorem 3 for the CFD test case, showing the normalized difference Tr(Dcol,m2)−Tr(Drow,m2) as a function of m . The small and decreasing values confirm near-equivalence of the two PCA strategies. Right: RRMSE on each output field ( τ11 , τ22 , k ) for all methods ( m=1,…,10 ). Row-CMO achieves the lowest RRMSE at m≥6 . Error bars represent ±1 standard deviation over 10 seeds.
Figure 7: RRMSE at m=8 for each output field ( τ11 , τ22 , k ) across all methods and deduction scenarios. Row-CMO achieves the lowest mean RRMSE with the smallest variability. The spread across deduction scenarios within each family ( Col-Indep , Fw-Indep , Fw-LCM ) and the larger error bars of the deductive methods illustrate the residual sensitivity to the output selection, even in a regime of strong inter-field correlation.
Figure 8: Row-CMO predictions on the CFD test case for a representative test input ( M=8 ). From top to bottom: τ11 , τ22 , k . Left to right: simulated field, predicted field, predictive standard deviation, and SRSE. The model accurately captures the spatial structure. Errors and uncertainty are concentrated in the recirculation zone of the diffuser.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
C\upmu
\upsigmak
\upsigma\upvarepsilon
C\upvarepsilon1
C\upvarepsilon2
0.09
1.00
1.30
1.44
1.92
Appendix
Table 1: Usual values of the parameters of the standard k−\upvarepsilon model.
Figure 9: Experimental data of Buice and Eaton (2000b) and probability density estimate of C\upmu
τ11
τ22
τ33
τ12
4.020e-3
1.576e-3
2.798e-3
1.020e-3
Appendix
Table 2: Measured values for the Reynolds stress tensor component for y/H=4 .
Figure 10: Histogram and gaussian KDE of the distribution of C\upvarepsilon2 using a 1,000-sized sample of \upbeta values.
Figure 11: Win rate (%) of each method on the RRMSE of each output ( f1 , f2 , f3 ) for N∈{20,50,100} , across all deduction scenarios ( k∗∈{1,2,3} ). The highlighted column indicates the scenario where the evaluated output is the deduced one. The CMOGP (blue) is the same model in all scenarios. Results are aggregated over 200 independent replications.
Figure 12: Distribution of the average 90% predictive interval length for f2 across 200 replications, for training set sizes N∈20,50,100 and the three deduction scenarios. All methods achieve comparable empirical coverage rates. Shorter intervals thus indicate more informative predictions. The CMOGP maintains stable and shorter intervals across all scenarios.
Figure 13: RRMSE as a function of m for all four Lotka–Volterra outputs ( p , q , r , s , columns) and training set sizes ( N=30,15,10 , rows). Row-CMO (red) is compared against Col-Indep and Fw-Indep with all deduction scenarios k∗∈{0,1,2,3} . Error bars represent ±1 standard deviation over 10 seeds.
In the sciences, regression tasks often require predicting high-dimensional outputs from few training examples. Multi-output Gaussian processes excel in low-data regimes but typically struggle with high-dimensional outputs. Compress-then-predict pipelines such as PCA-GP (principal component analysis plus Gaussian process regression) handle high dimensionality, but rely on bases optimized for reconstruction rather than prediction. To address this gap, we propose a model that represents each output as a linear-Gaussian decoding of a low-dimensional latent state drawn from a Gaussian process prior. By analytically marginalizing the decoder weights, we couple compression and prediction in a single objective that scales to high-dimensional outputs. We refer to this model as Gaussian process latent factor regression (GPLFR). We demonstrate GPLFR by building the first spatially resolved emulator of global climate models for rocky exoplanets.
Edward T. Stevenson, Eric T. Wolf, Mei Ting Mak +2
University of Cambridge · University of Colorado Boulder · University of Oxford +1
Gaussian process solvers for linear partial differential equations (PDEs) return posteriors rather than point estimates, but they condition only on equality information: the PDE residual, boundary data, and measurements. Many solutions of interest also obey inequality constraints, including nonnegativity, monotonicity, and absence of arbitrage, which equality conditioning cannot enforce. We prove that conditioning a Gaussian process prior jointly on linear equality observations and a finite-dimensional inequality event is equivalent to conditioning sequentially in terms of Borel measures on the solution space. The physics-informed posterior is therefore a valid prior for the inequality update at virtual points. The combined law is non-Gaussian yet computable through truncated-Gaussian moments, and neither the PDE conditioning nor the inequality update requires modification. We evaluated the construction on a Poisson benchmark, option-price surfaces, and air-quality fields. Under incomplete boundary data, a monotonicity constraint lowered the error of a two-dimensional Poisson reconstruction by 63%. On a real S&P 500 option chain thinned to 20 retained quotes, no-arbitrage constraints reduced held-out extrapolation error by 22% and cut the calendar-arbitrage rate from 10% to 1%. On four real air-quality snapshots, nonnegativity reduced a Gaussian moment-based sub-zero-tail proxy by 74% at the sparsest monitor count. Changes in mean accuracy and constraint diagnostics differed across data regimes. The probability of the noisy virtual constraint event under the equality posterior measures agreement with the specified bounds before the update. The construction turns admissibility from a post-processing step into conditioning information for probabilistic PDE solvers.
Boyuan Deng, Kshitiz Upadhyay, Michael Shields
Department of Civil and Systems Engineering, Johns Hopkins University · Department of Aerospace Engineering and Mechanics, University of Minnesota
Gaussian processes (GPs) offer a principled probabilistic model over functions, but exact inference is restricted to the linear-Gaussian regime. We establish an explicit equivalence between GPs and a class of linear diffusion models, recasting predictive sampling as an ODE with closed-form Gaussian dynamics and a likelihood-dependent guidance term that admits a simple Monte Carlo approximation. In the linear-Gaussian setting, we recover standard GP conditioning exactly; beyond conjugacy, the same machinery handles any conditioning statement admitting point-wise likelihood evaluation -- including non-linear physics, and, for the first time, natural language via large language models. Whitening isolates the irreducible non-Gaussian dynamics, minimising Wasserstein-2 transport cost and eliminating numerical stiffness. The result is a general-purpose GP inference scheme requiring no bespoke derivations. Together, these results provide a general mechanism for incorporating the full richness of real-world knowledge as conditioning information, opening a new frontier for the probabilistic modelling of real-world problems.
Henry Moss, Lachlan Astfalck, Thomas Cowperthwaite +5
Lancaster University · University of New South Wales · University of Cambridge +1