Learn Only Where You Can't Compute: Structured Corrections to Gaussian Process Posteriors

Tobias Weber; Carla Sagebiel; Philipp Hennig

cs.LGOct 15, 2026

Learn Only Where You Can't Compute: Structured Corrections to Gaussian Process Posteriors

Authors: Tobias Weber, Carla Sagebiel, Philipp Hennig

Organizations: Tübingen AI Center, Tübingen, Germany

Abstract

Physics-informed Gaussian processes (GPs) provide a principled framework for scientific inference by expressing uncertainty over empirical measurements, boundary conditions, and governing equations. However, they struggle to scale to larger datasets and nonlinear information. Conversely, diffusion models excel at learning complex distributions over functions from massive datasets, but struggle in scientific contexts where training data are scarce and physical or observational information must be incorporated. We propose a structured posterior-correction framework that combines a tractable Gaussian reference with data-driven corrections via generative learning. After a mean correction and covariance calibration, a diffusion model learns a targeted correction in a low-dimensional subspace selected to capture discrepancies from the reference, while the complement retains its Gaussian law. Across three scientific inference tasks, these corrections recover hidden modes, improve sample structure, and reduce prediction error relative to the Gaussian reference. Comparisons with full-field generative models reveal task-dependent benefits and tradeoffs between predictive accuracy and computational cost.

Figures & tables

Figure 1: Illustrative correction pipeline for a CR3BP trajectory in the rotating Earth–Moon frame: target orbit ( ), Gaussian reference ( ), and learned corrections ( ); thin curves are samples. Left: reference mean and samples. Middle: mean correction shifts the reference toward the target. Right: covariance and diffusion correction refines the ensemble spread. Earth and Moon mark their normalized locations; marks the trajectory start.

Method	CRPS	$d_{\mathrm{C}}$	$C_{90}$	PCD	CRPS	$d_{\mathrm{C}}$	$C_{90}$	PCD	CRPS	ES	$C_{90}$	Jacobi
	Synthetic				Smoke				CR3BP
Scaffold	0.2859	0.1884	0.30	667	0.0714	0.0585	0.92	7072	0.1588	11.26	0.89	1.46
KO	0.2130	0.0427	0.90	1242	0.0714	0.0584	0.92	7079	0.2726	19.03	0.96	5.50
Diff	0.2208	0.0575	0.86	1256	0.0434	0.0014	0.89	37	0.1785	10.91	0.99	2.89
[0.4pt/2pt] Scaffold + M	0.2744	0.1659	0.33	512	0.0619	0.0385	0.94	6906	0.1178	7.83	0.93	2.01
Rescale	0.2014	0.0193	0.94	406	0.0595	0.0336	0.92	5190	0.0916	7.60	0.94	0.79

Table 1: Main results, mean over three seeds. Synthetic :

d{=}512

n{=}2000

; Smoke :

d{=}1024

n{=}8192

; CR3BP :

d{=}2004

n{=}7500

. Lower is better except

C_{90}

(target

0.90

). Baselines are above the dashed line, scaffold corrections below it, and exact references below the solid rule (not available for CR3BP ). Bold: mean

\pm

one s.d. overlaps the best non-reference method.

Figure 2: Synthetic . Top: Cramér distance

d_{\mathrm{C}}

to the exact posterior versus training-set size; mean and range over three seeds. Bottom: pooled bump amplitudes at

n=1000

and

2000

(dotted lines; single seed) against the exact density (black). Rescale cannot split the sign modes; Gain-Diff does so by

n=1000

, Full-Diff only by

n=2000

Figure 3: Smoke training and sampling costs at

n=8192

d=1024

. Methods: Gain-Mix ( ), Gain-Diff ( ), Full-Diff ( ), Diff ( ). The arrow indicates improvement; points show timing medians over three seeds. Training includes selection and excludes evaluation; sampling produces 100 draws for one context.

Figure 4: Smoke posterior draws for one set of sensor observations (first three draws per method; rows are not paired). Exact draws are narrow plumes in different directions; the scaffold smears smoke across directions, while Gain-Diff and Diff recover localized plumes. Stars: source; circles: sensors.

Figure 5: CR3BP (

n=7500

, single seed). Top: first five draws per method for one initial state, with the reference trajectory in black; Gain-Diff ( ) stays close to the reference orbit, while Full-Diff ( ) and Diff ( ) scatter. Bottom: CRPS over time, averaged over the four state coordinates and all test contexts.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Illustration of the correction principle in two dimensions. Dashed contours mark two target modes. Correcting the mean and covariance accounts for location and spread, but leaves the Gaussian unable to represent both modes. Replacing the law along the horizontal direction recovers the bimodality while retaining a Gaussian in the vertical direction.

Method	Anchor	Gaussian part	Learned law	Problems	Defined in
Scaffold	$m_{c}$	$C_{c}$	–	all	Section 2
Scaffold + M	$\hat{\mu}_{c}$	$C_{c}$	–	all	Section 2
Rescale	$\hat{\mu}_{c}$	$L_{c}L_{c}^{\top}$	–	all	Section A.2
Gain-Mix	$\hat{\mu}_{c}$	$L_{c}(I-UU^{\top})L_{c}^{\top}$	mixture on $k$ coordinates	all	Section A.4
Gain-Diff	$\hat{\mu}_{c}$	$L_{c}(I-UU^{\top})L_{c}^{\top}$	DDPM on $k$ coordinates	all	Section A.4
Full-Diff	$\hat{\mu}_{c}$	–	DDPM on all $d$ coordinates	all	Section A.4

Appendix

Table 2: Method roster.

L_{c}

: calibrated factor ( Section A.2 );

k

: block size; mixtures are products of one-dimensional Gaussian mixtures; KO is shown before conditioning on readings. Grey rows require the exact posterior.

	Synthetic	Smoke	CR3BP
$d$	$512$	$1024$ ( $32\times 32$ )	$2004$ ( $501\times 4$ )
$n_{\mathrm{train}}$ / $n_{\mathrm{val}}$	$2000$ / $500$	$8192$ / $500$	$7500$ / $500$
Mean	linear (ridge)	MLP $2\times 128$ , span-projected	MLP $2\times 256$
Floor $\sigma$	log-score grid	$\tau$	– (per context)
$c_{\tau}$ / contrast	$0$ / pooled	$0.2$ / conditional	$0.2$ / conditional
Search span	resolved	resolved	$64$

Appendix

Table 3: Per-problem settings. Learning rates are for the block DDPM and Diff ; budgets are in optimizer steps.

Method	CRPS	$d_{\mathrm{C}}$	$C_{90}$	PCD	ES	RMSE	$k$
Scaffold	0.2859 $\pm$ 0.0016	0.1884 $\pm$ 0.0006	0.30 $\pm$ 0.00	667 $\pm$ 5	7.86 $\pm$ 0.04	0.4036 $\pm$ 0.0016	–
KO	0.2130 $\pm$ 0.0011	0.0427 $\pm$ 0.0002	0.90 $\pm$ 0.00	1242 $\pm$ 5	6.46 $\pm$ 0.03	0.4055 $\pm$ 0.0017	–
Diff	0.2208 $\pm$ 0.0013	0.0575 $\pm$ 0.0012	0.86 $\pm$ 0.00	1256 $\pm$ 18	6.79 $\pm$ 0.04	0.4241 $\pm$ 0.0024	–
[0.4pt/2pt] Scaffold + M	0.2744 $\pm$ 0.0012	0.1659 $\pm$ 0.0002	0.33 $\pm$ 0.00	512 $\pm$ 2	7.69 $\pm$ 0.03	0.3955 $\pm$ 0.0012	0
Rescale	0.2014 $\pm$ 0.0007	0.0193 $\pm$ 0.0002	0.94 $\pm$ 0.00	406 $\pm$ 0	6.26 $\pm$ 0.02	0.3955 $\pm$ 0.0012	0
Gain-Mix	0.1936 $\pm$ 0.0007	0.0026 $\pm$ 0.0001	0.89 $\pm$ 0.00	23 $\pm$ 3	6.27 $\pm$ 0.02	0.3955 $\pm$ 0.0012	16

Appendix

Table 4: Synthetic (

d=512

n=2000

): mean

\pm

standard deviation over three seeds (500 contexts

\times

100 draws). Corrected methods share the RMSE of their mean stage;

k

is the block size.

Figure 7: Synthetic learning curves for six further metrics, using 500 test contexts and 100 draws per context. Lines show means and bands span three seeds. Dashed and dotted lines show Exact-GP and Exact , respectively; the horizontal rule in the coverage panel marks

0.90

. Distance panels use logarithmic scales with a linear region near zero.

Figure 8: Synthetic posterior draws at

n=2000

(seed 10): three draws per method for the first test context, over the true function (dashed) and the readings (dots). Hatching marks the exact pointwise 90% intervals and dotted lines the sensors. The readings do not determine the bump signs, so valid draws need not match the true function between sensors.

Figure 9: The draws of Figure 8 and three exact draws on the first four sensor gaps, centered on the exact posterior mean

m_{G}(y)

; dotted lines mark the sensors.

Method	Fit + evaluation (s)
Rescale	40
Gain-Mix	349
Gain-Diff	361
Full-Diff	276
Diff	208

Appendix

Table 5: Synthetic ,

n=2000

: median fit-plus-evaluation time (s) over three seeds on an RTX 2080 Ti. Includes selection; excludes data preparation.

Figure 10: Smoke . (a) Prior over the wind direction. (b) Smoke plume shapes $g_{\varphi}$ for six directions, with the source ( $\star$ ) and the sensors ( $\circ$ ). (c–e) Three draws of the field: a sharp ray whose direction changes from draw to draw, on a quiet background.

	field						source
Method	CRPS	$d_{\mathrm{C}}$	Cov90	PCD	ES	RMSE	CRPS	$d_{\mathrm{C}}$	Cov90	$k$
Scaffold	0.0714 $\pm$ 0.0002	0.0585 $\pm$ 0.0001	0.92	7072.3 $\pm$ 10.9	3.561 $\pm$ 0.012	0.1631 $\pm$ 0.0005	0.1342 $\pm$ 0.0020	0.1306 $\pm$ 0.0013	0.93	–
KO	0.0714 $\pm$ 0.0002	0.0584 $\pm$ 0.0001	0.92	7078.7 $\pm$ 18.5	3.560 $\pm$ 0.012	0.1631 $\pm$ 0.0006	0.1342 $\pm$ 0.0021	0.1307 $\pm$ 0.0014	0.93	–
Diff	0.0434 $\pm$ 0.0009	0.0014 $\pm$ 0.0002	0.89	36.5 $\pm$ 1.3	2.515 $\pm$ 0.029	0.1353 $\pm$ 0.0017	0.0714 $\pm$ 0.0015	0.0028 $\pm$ 0.0004	0.89	–
[0.4pt/2pt] Scaffold + M	0.0619 $\pm$ 0.0005	0.0385 $\pm$ 0.0001	0.94	6906.3 $\pm$ 38.3	2.966 $\pm$ 0.017	0.1372 $\pm$ 0.0012	0.1135 $\pm$ 0.0020	0.0882 $\pm$ 0.0006	0.94	0
Rescale	0.0595 $\pm$ 0.0006	0.0336 $\pm$ 0.0001	0.92	5190.5 $\pm$ 35.1	2.914 $\pm$ 0.018	0.1372 $\pm$ 0.0012	0.1082 $\pm$ 0.0020	0.0775 $\pm$ 0.0006	0.92	0

Appendix

Table 6: Smoke , reported methods: field and source-disk scores, 500 test contexts

\times

100 draws, mean

\pm

std over seeds 10, 11, 12 (

n_{\mathrm{train}}=8192

d_{\mathrm{C}}

: Cramér distance to the exact posterior (Exact: its split-half floor); PCD: the Cramér distance between posterior log-density score distributions for generated and exact draws; RMSE: root mean squared error of the predictive mean;

k

: the selected block’s size. Best non-reference value in bold, references in grey.

Figure 11: Smoke posterior draws for all retained methods and the two exact references. The readings, training size ( $n=8192$ ), training seed ( $10$ ) and first three draws match Figure 4 ; rows are not paired realizations. Exact-GP matches the exact posterior’s conditional mean and covariance. All panels share a concentration scale: brown is positive, blue is negative. Stars mark the source; circles mark sensors.

Figure 12: Smoke learning curve: share of the scaffold-to-exact CRPS gap closed. Lines show seed means and bands span seeds. Filled points have three seeds; open points have two (scratch diffusion at

n=1024

and

4096

). The curve starts at

n=1024

, where all methods have results. Scratch diffusion leads throughout the measured range.

Method	Fit + evaluation (s)
Rescale	80
Gain-Mix	527
Gain-Diff	687
Full-Diff	1270
Diff	2983

Appendix

Table 7: Smoke ,

n=8192

: median fit-plus-evaluation time (s) over three seeds on an L40S. Includes selection; excludes data preparation.

Method	CRPS	ES	Jacobi err.	C 90	CRPS pos	CRPS vel	CRPS final	$k$
Scaffold	0.1588 $\pm$ 0.0002	11.26 $\pm$ 0.05	1.46 $\pm$ 0.01	0.89	0.1713 $\pm$ 0.0020	0.1462 $\pm$ 0.0023	0.592 $\pm$ 0.003	–
KO	0.2726 $\pm$ 0.0108	19.03 $\pm$ 0.59	5.50 $\pm$ 0.26	0.96	0.2910 $\pm$ 0.0119	0.2541 $\pm$ 0.0097	1.091 $\pm$ 0.025	–
Diff	0.1785 $\pm$ 0.0036	10.91 $\pm$ 0.27	2.89 $\pm$ 0.01	0.99	0.2031 $\pm$ 0.0038	0.1538 $\pm$ 0.0034	0.384 $\pm$ 0.014	–
[0.4pt/2pt] Scaffold + M	0.1178 $\pm$ 0.0028	7.83 $\pm$ 0.26	2.01 $\pm$ 0.03	0.93	0.1217 $\pm$ 0.0023	0.1138 $\pm$ 0.0036	0.397 $\pm$ 0.015	0
Rescale	0.0916 $\pm$ 0.0024	7.60 $\pm$ 0.23	0.79 $\pm$ 0.02	0.94 $\pm$ 0.01	0.0911 $\pm$ 0.0015	0.0921 $\pm$ 0.0035	0.376 $\pm$ 0.011	0
Gain-Mix	0.0844 $\pm$ 0.0028	7.24 $\pm$ 0.26	0.68 $\pm$ 0.02	0.96	0.0821 $\pm$ 0.0025	0.0866 $\pm$ 0.0033	0.372 $\pm$ 0.011	64

Appendix

Table 8: CR3BP (

d=2004

n=7500

): mean

\pm

std over seeds 10–12 (500 contexts

\times

100 draws). Jacobi error is defined in Section E.3 ;

k

is the block size. Best mean in bold (coverage closest to

0.90

Method	Fit + evaluation (s)
Rescale	275
Gain-Mix	1081
Gain-Diff	1362
Full-Diff	875
Diff	932

Appendix

Table 9: CR3BP ,

n=7500

: median fit-plus-evaluation time (s) over three seeds on an RTX 2080 Ti. Includes selection; excludes data preparation.

Gaussian Processes

Gaussian Processes are probabilistic models of functions that use kernels to make predictions with explicit uncertainty.

See all 113 papers

Predictively Oriented Gaussian Process Posteriors
Oct 2, 2026Callum Lau, Jeremias Knoblauch, Louis SharrockGaussian Process Regression Uncertainty Quantification
University College London
Conditioning Gaussian Processes on Almost Anything
May 20, 2026Henry Moss, Lachlan Astfalck, Thomas Cowperthwaite +5Gaussian Processes Conditional Distribution Estimation
Lancaster University · University of New South Wales · University of Cambridge +1
Input Bias Correction in Gaussian Process Regression
Oct 22, 2026Muzaffar Qureshi, Tochukwu E. Ogri, Trivikram Satharasi +2Gaussian Process Regression Uncertainty Calibration
Department of Mechanical and Aerospace Engineering, University of Florida, Gainesville, FL, USA · Torch Technologies, Shalimar, FL, USA

Oct 2, 2026J/K move · Enter open · S save

Predictively Oriented Gaussian Process Posteriors

Callum Lau, Jeremias Knoblauch, Louis Sharrock

University College London

Gaussian Process Regression Uncertainty Quantification Uncertainty Calibration Bayesian Inference Model Misspecification

Open paper AskPDF arXiv

Gaussian Processes (GPs) are a powerful tool for modelling and quantifying uncertainty in functional relationships. However, they require practitioners to make a number of design decisions, such as the choice of the kernel and the observation model. Suboptimal choices can produce misspecified models that do not capture the underlying data generating process. We introduce Predictively Oriented Gaussian Processes (PrO-GPs), which treat predictive uncertainty as the primary inferential target and provide a robust alternative to standard GPs. Although direct computation of a PrO posterior for nonparametric models is intractable, we derive a reduced formulation and practical sampling scheme for efficient computation. Through synthetic and real data experiments, we show that PrO-GPs produce better calibrated predictive distributions under model misspecification compared to standard GP approaches.

Abstract

Gaussian Processes

Explore similar work

Predictively Oriented Gaussian Process Posteriors

Conditioning Gaussian Processes on Almost Anything

Input Bias Correction in Gaussian Process Regression