Generative Priors Conditioned on Natural Language for Bayesian Inversion in PDEs

Pengyu Zhang; Mark Girolami; Arnaud Vadeboncoeur

cs.LGSep 26, 2026

Generative Priors Conditioned on Natural Language for Bayesian Inversion in PDEs

Authors: Pengyu Zhang, Mark Girolami, Arnaud Vadeboncoeur

Organizations: Department of Engineering University of Cambridge Cambridge, CB3 0FA, UK

Abstract

Inferring quantities of interest (QoI) from data is a central task in Science and Engineering. In such contexts, we often have access to both quantitative data and qualitative data. Quantitative data may be represented by noisy sensor measurements, simulation data, re-analysis data; qualitative data may be in the form of text descriptions of experimental setups, expected experiment outcomes, and human-perceived system behaviours. The task we address in this paper is the following. Given a training set of paired qualitative text and quantitative QoI data, we learn to exploit the inherent correlation between the two modalities to learn a highly informative data-driven natural-language-conditional Bayesian prior, such that when presented with a new physical system, we can coherently combine (i) the training dataset, (ii) qualitative text describing the new system, and (iii) a small number of noisy sensor readings from that new system, to perform inference and uncertainty quantification (UQ) over the QoI. To achieve this task, we develop two parallel approaches, one uses conditional diffusion and the other conditional autoencoders, and compare both against classical Bayesian methodology, unconditional generative models and deterministic supervised methods. Each approach has specific strengths and tradeoffs; conditional autoencoder offers theoretical tractability, allows for fast posterior sampling, and provides better-calibrated UQ, whereas conditional diffusion is explored for greater expressiveness and capturing complex posteriors with irregular QoI fields. The approach is tested on the steady-state heat equation, damped Helmholtz equation, and UK weather reanalysis data.

Figures & tables

Figure 1

	$n_{\text{obs}}=4$						$n_{\text{obs}}=8$						Cost
Model	NMSE $(\times 10^{-3})$		$\%C_{68}$		$\%C_{95}$		NMSE $(\times 10^{-3})$		$\%C_{68}$		$\%C_{95}$		Param(M)	Train(hr)	Pred(s)
GP	$178.87$ $\pm$	$103.15$	$71.7$	(14.8)	$93.6$	( 5.5)	$97.03$ $\pm$	$59.79$	$64.6$	(13.8)	$89.7$	( 7.5)	–	–	0.0004
Uncon Supervised	$54.66$ $\pm$	$203.76$	–		–		$4.87$ $\pm$	$12.89$	–		–		5.13	1.91	0.0003
Cond Supervised	$9.52$ $\pm$	$17.02$	–		–		$3.54$ $\pm$	$6.18$	–		–		5.13	0.93	0.0003
Uncon AE	$46.45$ $\pm$	$132.21$	$61.5$	(22.4)	$88.7$	(12.0)	$4.85$ $\pm$	$15.95$	$67.6$	(17.8)	$92.6$	( 8.5)	9.48	3.63	28.51
Cond AE	$6.13$ $\pm$	$11.70$	$61.6$	(23.4)	$88.2$	( 8.2)	$1.40$ $\pm$	$3.01$	$62.5$	(20.8)	$88.1$	( 7.9)	9.87	2.40	10.25

Table 1: Inference results on the steady-state heat equation at

\mathbf{n_{\textbf{obs}}=\{4,8\}}

. We report NMSE of posterior mean (with quartiles in Appendix Table 6 ), and coverages of the ground truth by the centered

68\%

and

95\%

posterior intervals (target values:

68\%

and

95\%

). NMSE entries are mean

\pm

1 std across

200

test fields; coverages report the mean, values in parentheses are the mean absolute difference from the target coverage (smaller is better). Pred. is the mean per-field inference time.

Figure 2: Performance of posterior calibration. We plot calibration curves (observed coverage over target central coverage) for three experiments, where the diagonal represents perfect calibration, with more details about calibration curve calculation in Appendix F .

Figure 3: Inference results of one Helmholtz test system (Real part) at $n_{\mathrm{\text{obs}}}=6$ .

	$n_{\text{obs}}=6$						$n_{\text{obs}}=10$						Cost
Model	NMSE $(\times 10^{-2})$		$\%C_{68}$		$\%C_{95}$		NMSE $(\times 10^{-2})$		$\%C_{68}$		$\%C_{95}$		Param(M)	Train(hr)	Pred(s)
GP	$86.74$ $\pm$	$9.20$	$83.3$	(23.0)	$94.0$	( 7.4)	$77.12$ $\pm$	$11.96$	$82.3$	(23.1)	$93.6$	( 7.7)	–	–	0.005
Uncon Supervised	$25.93$ $\pm$	$23.18$	–		–		$10.17$ $\pm$	$7.68$	–		–		8.22	13.64	0.002
Cond Supervised	$19.77$ $\pm$	$25.27$	–		–		$10.84$ $\pm$	$16.48$	–		–		8.32	10.91	0.002
Uncon AE	$66.61$ $\pm$	$338.05$	$45.9$	(28.7)	$68.6$	(28.3)	$16.07$ $\pm$	$30.13$	$46.1$	(26.9)	$68.4$	(28.5)	19.71	19.37	145.25
Cond AE	$7.15$ $\pm$	$17.18$	$71.4$	(13.6)	$94.5$	( 5.5)	$3.16$ $\pm$	$7.25$	$69.7$	(11.8)	$93.8$	( 5.4)	20.66	15.91	43.90

Table 2: Inference results on 100 Helmholtz test fields, more details in Appendix Tables 9 and 10 .

Figure 4: Example of a 3-channel weather data slice with corresponding text description. Green boxes mark the regions referenced in the text, and grey dashed boxes mark available land regions.

Figure 5: Inference results of one UK weather test system ( $v$ ) with $n_{\mathrm{\text{obs}}}=8$ .

	$n_{\text{obs}}=8$						$n_{\text{obs}}=12$						Cost
Model	NMSE $(\times 10^{-2})$		$\%C_{68}$		$\%C_{95}$		NMSE $(\times 10^{-2})$		$\%C_{68}$		$\%C_{95}$		Params(M)	Train(hr)	Pred.(s)
GP	$12.12$ $\pm$	$16.06$	$68.3$	( 8.5)	$91.7$	( 5.0)	$10.16$ $\pm$	$13.80$	$69.7$	( 8.1)	$92.1$	( 4.6)	–	–	0.002
Uncon Supervised	$7.46$ $\pm$	$11.38$	–		–		$6.22$ $\pm$	$10.47$	–		–		8.23	22.34	0.002
Cond Supervised	$6.78$ $\pm$	$9.95$	–		–		$5.89$ $\pm$	$8.81$	–		–		8.32	14.89	0.002
Uncon AE	$8.18$ $\pm$	$11.55$	$67.5$	( 6.5)	$93.6$	( 3.3)	$6.90$ $\pm$	$10.50$	$65.8$	( 5.9)	$92.3$	( 3.6)	16.92	14.08	47.02
Cond AE	$6.93$ $\pm$	$9.81$	$65.9$	( 5.6)	$92.5$	( 3.5)	$6.12$ $\pm$	$9.58$	$63.8$	( 6.6)	$90.9$	( 4.5)	17.71	15.73	47.08

Table 3: Inference results on 100 UK weather test fields, more details in Appendix Tables 12 and 13 .

Appendix figures & tables34 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Description	Example
$u\in\mathcal{U}$	Quantity of Interest (QoI)	Heat distribution over the physical domain
$\theta\in\Theta$	Human-perceivable quantity related to $u$	The left boundary condition
$s\in\mathcal{S}$	Qualitative data: description of $\theta$	"the left boundary is hot"
$\tau\in\mathcal{T}$	Euclidean description of $s$	Text encoding of "the left boundary is hot"
$y\in\mathbb{R}^{d_{y}}$	Quantitative data: incomplete/indirect measurements of $u$	Sparse sensor measurements of heat on domain

Appendix

Table 4: Symbols and concepts.

Figure 6: Ground truth fields and Voronoi inputs to the supervised model at $n_{\mathrm{obs}}=\{4,8\}$ .

Figure 7: Latent histograms across all dimensions over the test set in three experiments, compared with the ideal $\mathbb{Q}_{z}=\mathcal{N}(0,\mathrm{I})$ .

Figure 8: Examples of heat fields with corresponding BC values and text descriptions.

Figure 9: Prior samples generated given text. The leftmost panel shows the ground-truth field together with the corresponding text. Each of the four rows shows three prior samples drawn using four models, the text is supplied to conditional models and not provided to unconditional ones.

Figure 10: Posterior reconstructions of one heat equation test system at $\mathbf{n_{\mathrm{\textbf{obs}}}=4}$ (same system as in Figure 1 ). Each row indicates posterior mean, posterior standard deviation, $|$ error $|$ of the mean respectively. NMSE of posterior mean of each model is reported as well.

n_{\text{obs}}=2

n_{\text{obs}}=4

n_{\text{obs}}=6

Model NMSE

\downarrow

(\times 10^{-3})

CRPS

\downarrow

(\times 10^{-2})

\%C_{68}

\%C_{95}

NMSE

\downarrow

(\times 10^{-3})

CRPS

\downarrow

(\times 10^{-2})

\%C_{68}

\%C_{95}

NMSE

\downarrow

(\times 10^{-3})

CRPS

\downarrow

(\times 10^{-2})

\%C_{68}

\%C_{95}

315.79

\pm

176.38

30.68

\pm

7.54

73.1

(13.0)

95.6

( 5.0)

178.87

\pm

103.15

21.68

\pm

6.17

71.7

(14.8)

93.6

( 5.5)

135.62

\pm

82.94

18.19

\pm

5.63

68.3

(14.8)

91.8

( 6.4) Uncon Supervised

208.16

\pm

292.41

– – –

54.66

\pm

203.76

– – –

12.99

\pm

45.38

– – – Cond Supervised

17.95

\pm

34.60

– – –

9.52

\pm

17.02

– – –

5.82

\pm

12.28

– – – Uncon AE

248.75

\pm

377.03

21.20

\pm

17.44

47.6

(26.5)

80.9

(17.8)

46.45

\pm

132.21

6.63

\pm

8.83

61.5

(22.4)

88.7

(12.0)

9.75

\pm

29.25

3.14

\pm

3.76

65.0

(19.3)

92.0

( 9.0) Cond AE

15.42

\pm

24.84

4.80

\pm

3.76

63.2

(24.5)

88.3

( 8.6)

6.13

\pm

11.70

2.89

\pm

2.52

61.6

(23.4)

88.2

( 8.2)

3.14

\pm

8.68

1.99

\pm

2.00

62.0

(22.5)

87.8

( 8.4) Uncon Diffusion

239.28

\pm

291.64

19.93

\pm

13.37

62.0

(22.5)

90.1

(10.7)

37.71

\pm

107.23

5.02

\pm

6.40

64.9

(25.2)

89.2

(12.2)

7.36

\pm

22.12

2.30

\pm

2.92

59.0

(27.0)

86.5

(14.8) Cond Diffusion

15.79

\pm

37.39

4.69

\pm

4.47

60.3

(26.4)

90.4

(11.7)

7.12

\pm

17.64

2.88

\pm

3.02

62.3

(25.2)

89.8

(12.0)

3.11

\pm

8.93

1.91

\pm

2.10

63.9

(24.3)

89.5

(12.2) Uncon Diffusion (DPS)

235.67

\pm

310.17

19.50

\pm

13.23

59.0

(21.4)

80.9

(16.5)

39.47

\pm

118.32

5.34

\pm

6.72

37.4

(35.8)

65.2

(31.3)

7.18

\pm

20.47

2.55

\pm

2.88

29.6

(41.1)

54.8

(41.1) Cond Diffusion (DPS)

16.46

\pm

38.82

5.07

\pm

4.91

43.0

(31.9)

71.4

(25.2)

7.05

\pm

18.35

3.13

\pm

3.33

37.3

(35.5)

67.8

(29.2)

3.23

\pm

9.87

2.14

\pm

2.34

39.2

(32.9)

67.8

(28.6)

n_{\text{obs}}=8

n_{\text{obs}}=10

Model NMSE

\downarrow

(\times 10^{-3})

CRPS

\downarrow

(\times 10^{-2})

\%C_{68}

\%C_{95}

NMSE

\downarrow

(\times 10^{-3})

CRPS

\downarrow

(\times 10^{-2})

\%C_{68}

\%C_{95}

97.03

\pm

59.79

14.75

\pm

4.77

64.6

(13.8)

89.7

( 7.5)

78.78

\pm

57.13

12.72

\pm

4.44

63.2

(13.5)

88.8

( 7.8) Uncon Supervised

4.87

\pm

12.89

– – –

2.87

\pm

5.10

– – – Cond Supervised

3.54

\pm

6.18

– – –

2.90

\pm

5.41

– – – Uncon AE

4.85

\pm

15.95

2.32

\pm

2.94

67.6

(17.8)

92.6

( 8.5)

4.25

\pm

14.05

2.10

\pm

2.83

67.7

(17.7)

92.8

( 8.3) Cond AE

1.40

\pm

3.01

1.46

\pm

1.16

62.5

(20.8)

88.1

( 7.9)

1.03

\pm

2.20

1.28

\pm

0.99

60.4

(21.3)

87.1

( 8.7) Uncon Diffusion

2.45

\pm

8.95

1.44

\pm

1.62

59.8

(26.0)

87.0

(14.3)

1.09

\pm

2.77

1.17

\pm

1.06

58.9

(26.7)

85.4

(15.4) Cond Diffusion

1.44

\pm

3.54

1.40

\pm

1.29

64.3

(22.7)

90.3

(11.2)

1.16

\pm

3.31

1.24

\pm

1.23

64.5

(23.0)

90.1

(11.1) Uncon Diffusion (DPS)

2.74

\pm

8.71

1.64

\pm

1.57

31.5

(39.3)

55.9

(40.1)

1.40

\pm

4.67

1.40

\pm

1.25

30.5

(39.0)

54.8

(41.1) Cond Diffusion (DPS)

1.43

\pm

3.44

1.59

\pm

1.41

40.2

(31.9)

69.8

(26.8)

1.14

\pm

3.05

1.41

\pm

1.29

40.5

(31.3)

70.2

(26.2)

Appendix

Table 5: Full inference results on the heat equation test set with

\mathbf{n_{\textbf{obs}}}

varying from 2 to 10. This table is the extended version of Table 1 , and also presents the diffusion inference results obtained via the standard DPS scheme (last two rows). Continuous ranked probability score (CRPS) is also reported, with details in Appendix F .

Figure 11: Per-system coverage over 200 steady-state heat equation test systems at $\mathbf{n_{\textbf{obs}}=4}$ . Each panel shows the distribution of coverages across all systems, i.e. the percentage of systems falling in each coverage bin.

Figure 12: NMSE performance and text gain at different $\mathbf{n_{obs}}$ for steady-state heat equation. (a) NMSE across 200 test systems: median (solid), mean (dashed), and 25th–75th percentile bands, see Table 6 for exact values. (b) Per-system relative gain from text conditioning, $\log_{2}(\mathrm{NMSE}_{\mathrm{uncon}}/\mathrm{NMSE}_{\mathrm{cond}})$ , with median and interquartile bands. Zero values indicate that the text adds nothing beyond the observations.

	$n_{\text{obs}}=2$			$n_{\text{obs}}=4$			$n_{\text{obs}}=6$			$n_{\text{obs}}=8$			$n_{\text{obs}}=10$
Model NMSE $(\times 10^{-3})$	$Q_{25}$	Median	$Q_{75}$	$Q_{25}$	Median	$Q_{75}$	$Q_{25}$	Median	$Q_{75}$	$Q_{25}$	Median	$Q_{75}$	$Q_{25}$	Median	$Q_{75}$
Uncon AE	50.64	158.02	287.87	1.31	5.20	23.29	0.58	1.60	4.65	0.41	0.83	2.19	0.29	0.69	1.46
Cond AE	2.02	5.96	17.10	0.52	1.59	5.45	0.30	0.72	2.12	0.18	0.50	1.12	0.16	0.38	0.91
Uncon Diffusion	60.89	159.12	312.58	0.73	2.99	26.18	0.27	0.72	2.46	0.15	0.40	1.06	0.14	0.27	0.75
Cond Diffusion	1.70	5.08	13.97	0.44	1.33	4.87	0.25	0.66	1.93	0.18	0.44	0.96	0.13	0.32	0.74

Appendix

Table 6: Quartiles of per-system NMSE across 200 heat-equation test systems.

Q_{25}

, median and

Q_{75}

over the same 200 test systems across different

n_{obs}

are shown.

	$n_{\text{obs}}=2$		$n_{\text{obs}}=4$		$n_{\text{obs}}=6$		$n_{\text{obs}}=8$		$n_{\text{obs}}=10$
Model	$\text{ESS}_{\min}$	$\text{ESS}_{\text{mean}}$	$\text{ESS}_{\min}$	$\text{ESS}_{\text{mean}}$	$\text{ESS}_{\min}$	$\text{ESS}_{\text{mean}}$	$\text{ESS}_{\min}$	$\text{ESS}_{\text{mean}}$	$\text{ESS}_{\min}$	$\text{ESS}_{\text{mean}}$
Uncon AE	$53.7$	$116.2$	$62.0$	$124.1$	$64.6$	$127.0$	$65.6$	$128.4$	$63.9$	$124.0$
Cond AE	$102.9$	$191.7$	$100.5$	$190.2$	$100.4$	$188.7$	$99.7$	$186.2$	$100.8$	$193.4$

Appendix

Table 7: Effective sample size (ESS) of NUTS on the heat equation, averaged over the 200 test systems. This table reports

\text{ESS}_{\min}

and

\text{ESS}_{\text{mean}}

, the minimum and the mean over the

16

-dimensional latent vector of

M=200

posterior samples.

Figure 13: Posterior inference of one steady-state heat equation test system as the number of observation points increases. The leftmost panel shows the ground truth field together with the text description. Columns give $n_{obs}=\{2,4,6,8,10\}$ with nested observation sets. NMSE is also provided at the bottom of each field.

Figure 14: Examples of Helmholtz fields with forcing fields, related parameters and descriptions.

Forcing magnitude $A$		Damping $\gamma$		Radial position $\\|c\\|_{2}$
Level	Range	Level	Range	Level	Range
faint	1.00–2.15	very light	0.020–0.036	centre	0.00–0.25
moderate	2.15–4.64	light	0.036–0.063	mid	0.25–0.55
strong	4.64–10.0	moderate	0.063–0.113	rim	0.55–0.80
		heavy	0.113–0.200

Appendix

Table 8: Binning of the Helmholtz parameters for text generation. Only the bin labels are passed to the language model and the numerical values are never shown.

Figure 15: Prior samples generated from four models (with text provided to conditional models).

Figure 16: Posterior marginals of one Helmholtz equation test system (Real part) (same system as in Figure 3 ) at four unobserved query locations when $n_{\mathrm{\text{obs}}}=6$ .

n_{\text{obs}}=4

n_{\text{obs}}=6

n_{\text{obs}}=8

Model NMSE

\downarrow

(\times 10^{-2})

CRPS

\downarrow

(\times 10^{-2})

\%C_{68}

\%C_{95}

NMSE

\downarrow

(\times 10^{-2})

CRPS

\downarrow

(\times 10^{-2})

\%C_{68}

\%C_{95}

NMSE

\downarrow

(\times 10^{-2})

CRPS

\downarrow

(\times 10^{-2})

\%C_{68}

\%C_{95}

92.45

\pm

8.42

74.49

\pm

31.10

81.4

(23.0)

93.1

( 8.1)

86.74

\pm

9.20

74.66

\pm

33.42

83.3

(23.0)

94.0

( 7.4)

81.62

\pm

10.47

69.13

\pm

28.71

81.4

(23.0)

93.0

( 8.2) Uncon Supervised

54.07

\pm

28.83

– – –

25.93

\pm

23.18

– – –

14.17

\pm

12.89

– – – Cond Supervised

28.39

\pm

34.74

– – –

19.77

\pm

25.27

– – –

13.13

\pm

17.64

– – – Uncon AE

94.21

\pm

186.28

45.20

\pm

36.74

37.7

(33.7)

60.4

(35.9)

66.61

\pm

338.05

28.47

\pm

42.73

45.9

(28.7)

68.6

(28.3)

29.20

\pm

64.45

22.53

\pm

26.28

46.5

(27.6)

68.4

(28.4) Cond AE

23.89

\pm

46.08

18.68

\pm

20.59

67.7

(18.6)

89.6

(10.7)

7.15

\pm

17.18

9.62

\pm

9.43

71.4

(13.6)

94.5

( 5.5)

4.07

\pm

8.99

7.79

\pm

6.78

70.3

(12.5)

94.1

( 5.6) Uncon Diffusion

48.08

\pm

30.47

32.30

\pm

14.43

57.5

(19.7)

88.8

(10.5)

22.57

\pm

24.20

18.26

\pm

13.38

64.4

(18.0)

93.4

( 6.7)

9.00

\pm

13.21

9.95

\pm

8.67

68.7

(18.0)

93.9

( 7.0) Cond Diffusion

20.14

\pm

30.17

18.96

\pm

16.23

50.5

(26.1)

76.6

(21.2)

7.15

\pm

12.63

10.51

\pm

9.22

54.5

(23.5)

83.0

(15.3)

4.30

\pm

7.95

7.90

\pm

7.09

54.8

(21.7)

82.2

(15.9) Uncon Diffusion (DPS)

47.79

\pm

27.75

32.16

\pm

13.37

56.6

(19.1)

88.8

( 9.7)

24.18

\pm

23.35

19.23

\pm

12.81

61.5

(17.3)

90.6

( 8.0)

10.48

\pm

13.98

10.91

\pm

8.83

62.7

(18.2)

90.2

( 8.7) Cond Diffusion (DPS)

19.12

\pm

28.91

19.21

\pm

15.71

45.2

(28.3)

72.6

(24.0)

7.19

\pm

13.22

11.40

\pm

9.50

47.4

(26.8)

76.9

(19.7)

4.42

\pm

8.75

7.91

\pm

7.30

46.9

(24.9)

75.2

(20.9)

n_{\text{obs}}=10

n_{\text{obs}}=12

Model NMSE

\downarrow

(\times 10^{-2})

CRPS

\downarrow

(\times 10^{-2})

\%C_{68}

\%C_{95}

NMSE

\downarrow

(\times 10^{-2})

CRPS

\downarrow

(\times 10^{-2})

\%C_{68}

\%C_{95}

77.12

\pm

11.96

67.76

\pm

29.42

82.3

(23.1)

93.6

( 7.7)

72.21

\pm

12.94

65.36

\pm

28.73

82.3

(23.0)

93.7

( 7.6) Uncon Supervised

10.17

\pm

7.68

– – –

8.45

\pm

8.23

– – – Cond Supervised

10.84

\pm

16.48

– – –

8.84

\pm

13.23

– – – Uncon AE

16.07

\pm

30.13

17.42

\pm

18.26

46.1

(26.9)

68.4

(28.5)

12.72

\pm

29.08

14.82

\pm

17.04

46.9

(26.8)

69.0

(27.7) Cond AE

3.16

\pm

7.25

6.77

\pm

5.80

69.7

(11.8)

93.8

( 5.4)

2.35

\pm

4.83

6.00

\pm

4.51

68.9

(12.0)

93.1

( 5.7) Uncon Diffusion

5.36

\pm

9.24

7.20

\pm

6.99

67.6

(19.1)

92.0

( 8.4)

3.09

\pm

6.62

5.38

\pm

5.46

66.8

(17.6)

92.7

( 7.6) Cond Diffusion

3.45

\pm

7.54

6.75

\pm

6.64

54.7

(20.7)

83.6

(14.1)

2.52

\pm

4.92

5.80

\pm

5.63

54.5

(20.3)

83.9

(13.8) Uncon Diffusion (DPS)

5.93

\pm

9.74

7.61

\pm

7.04

61.6

(18.8)

88.3

(10.1)

3.95

\pm

7.94

6.02

\pm

5.96

58.2

(18.3)

86.5

(10.5) Cond Diffusion (DPS)

3.63

\pm

8.95

6.79

\pm

7.13

47.2

(24.1)

75.8

(20.3)

2.71

\pm

5.94

5.85

\pm

5.97

47.0

(23.9)

76.4

(19.3)

Appendix

Table 9: Full inference table on the Helmholtz equation test set. This table extends Table 2 with

n_{\text{obs}}

between 4 and 12 over

100

test fields.

Figure 17: Per-system coverages over 100 Helmholtz test systems at $n_{\text{obs}}=6.$

Figure 18: NMSE performance and text gain at different $\mathbf{n_{\textbf{obs}}}$ for Helmholtz equation. (a) NMSE across 100 test systems: median (solid), mean (dashed), and 25th–75th percentile bands, see Table 10 for exact values. (b) Per-system relative gain from text conditioning, $\log_{2}(\mathrm{NMSE}_{\mathrm{uncon}}/\mathrm{NMSE}_{\mathrm{cond}})$ , with median and interquartile bands.

	$n_{\text{obs}}=4$			$n_{\text{obs}}=6$			$n_{\text{obs}}=8$			$n_{\text{obs}}=10$			$n_{\text{obs}}=12$
Model NMSE $(\times 10^{-2})$	$Q_{25}$	Median	$Q_{75}$	$Q_{25}$	Median	$Q_{75}$	$Q_{25}$	Median	$Q_{75}$	$Q_{25}$	Median	$Q_{75}$	$Q_{25}$	Median	$Q_{75}$
Uncon AE	6.98	52.42	118.06	1.36	6.62	44.52	0.91	4.04	22.46	0.78	2.65	17.39	0.55	2.14	9.19
Cond AE	0.99	4.55	18.83	0.46	1.50	5.11	0.35	1.14	4.41	0.31	0.80	2.99	0.25	0.60	2.26
Uncon Diffusion	21.18	48.36	66.97	4.16	13.37	36.19	0.66	3.42	11.01	0.18	1.37	6.66	0.09	0.54	2.40
Cond Diffusion	1.78	7.79	29.61	0.62	2.10	6.79	0.34	0.94	4.86	0.21	0.70	2.91	0.12	0.48	2.39

Appendix

Table 10: Quartiles of per-system NMSE across 100 Helmholtz test systems at different

n_{\text{obs}}

Figure 19: Posterior inference of one Helmholtz equation test system (Imaginary part) as the number of observation points increases.

Figure 20: Posterior inference of one Helmholtz equation test system (Real part) as the number of observation points increases.

Figure 21: Posterior marginals of one UK weather test system ( $v$ ) (same system as in Figure 5 ) at four unobserved query locations when $n_{\mathrm{\text{obs}}}=8$ .

Temperature (°C)		Wind speed (m s -1 )
Level	Range	Level	Range
freezing	$<3$	calm	$<2.2$
cold	$2$ – $6$	light	$1.8$ – $4.0$
chilly	$5$ – $9$	moderate	$3.5$ – $5.5$
cool	$8$ – $12$	breezy	$5.0$ – $7.5$
mild	$11$ – $15$	strong	$>7.0$

Appendix

Table 11: Binning of the regional averages for text generation in UK weather dataset. Bands overlap at their edges: a value falling in two bands is described by one of them, chosen at random.

	$n_{\text{obs}}=4$								$n_{\text{obs}}=6$								$n_{\text{obs}}=8$
Model	NMSE $\downarrow$ $(\times 10^{-2})$		CRPS $\downarrow$ $(\times 10^{-2})$		$\%C_{68}$		$\%C_{95}$		NMSE $\downarrow$ $(\times 10^{-2})$		CRPS $\downarrow$ $(\times 10^{-2})$		$\%C_{68}$		$\%C_{95}$		NMSE $\downarrow$ $(\times 10^{-2})$		CRPS $\downarrow$ $(\times 10^{-2})$		$\%C_{68}$		$\%C_{95}$
GP	$15.05$ $\pm$	$19.33$	$18.16$ $\pm$	$11.44$	$70.0$	( 9.5)	$92.6$	( 4.7)	$13.10$ $\pm$	$16.83$	$16.71$ $\pm$	$10.37$	$71.0$	( 9.4)	$93.0$	( 4.3)	$12.12$ $\pm$	$16.06$	$15.91$ $\pm$	$10.12$	$68.3$	( 8.5)	$91.7$	( 5.0)
Uncon Supervised	$10.81$ $\pm$	$14.02$	–		–		–		$8.76$ $\pm$	$11.89$	–		–		–		$7.46$ $\pm$	$11.38$	–		–		–
Cond Supervised	$8.96$ $\pm$	$12.98$	–		–		–		$7.51$ $\pm$	$10.54$	–		–		–		$6.78$ $\pm$	$9.95$	–		–		–
Uncon AE	$11.30$ $\pm$	$15.16$	$15.21$ $\pm$	$9.38$	$69.5$	( 8.0)	$94.6$	( 3.5)	$9.28$ $\pm$	$12.43$	$13.57$ $\pm$	$8.50$	$68.6$	( 7.3)	$94.3$	( 3.2)	$8.18$ $\pm$	$11.55$	$12.58$ $\pm$	$8.02$	$67.5$	( 6.5)	$93.6$	( 3.3)
Cond AE	$9.27$ $\pm$	$13.45$	$13.70$ $\pm$	$8.67$	$66.3$	( 6.7)	$92.6$	( 3.9)	$7.68$ $\pm$	$10.44$	$12.45$ $\pm$	$7.64$	$66.4$	( 6.0)	$92.8$	( 3.4)	$6.93$ $\pm$	$9.81$	$11.68$ $\pm$	$7.37$	$65.9$	( 5.6)	$92.5$	( 3.5)

Appendix

Table 12: Full inference results on 100 UK weather test systems, which is an extension of Table 3 .

Figure 22: Per-system coverage over 100 Weather test fields with $n_{\text{obs}}=8$ .

Figure 23: Posterior reconstructions of one UK Weather test system ( $T$ ) when $n_{\mathrm{\text{obs}}}=8$ .

Figure 24: Posterior marginals of one UK weather test system ( $T$ ) (same system as in Figure 23 ) at four unobserved query locations when $n_{\mathrm{\text{obs}}}=8$ .

Figure 25: Posterior reconstructions of one UK Weather test system ( $T$ ) when $n_{\mathrm{\text{obs}}}=8$ .

Figure 26: Posterior marginals of one UK weather system ( $T$ ) (same system as in Figure 25 ) at four unobserved query locations when $n_{\mathrm{\text{obs}}}=8$ .

Figure 27: NMSE performance at different $\mathbf{n_{obs}}$ for UK Weather data. (a) NMSE across 100 test systems: median (solid), mean (dashed), and 25th–75th percentile bands, see Table 13 for exact values. (b) Per-system relative gain from text conditioning, $\log_{2}(\mathrm{NMSE}_{\mathrm{uncon}}/\mathrm{NMSE}_{\mathrm{cond}})$ , with median and interquartile bands.

	$n_{\text{obs}}=4$			$n_{\text{obs}}=6$			$n_{\text{obs}}=8$			$n_{\text{obs}}=10$			$n_{\text{obs}}=12$			$n_{\text{obs}}=14$
Model NMSE $(\times 10^{-2})$	$Q_{25}$	Median	$Q_{75}$	$Q_{25}$	Median	$Q_{75}$	$Q_{25}$	Median	$Q_{75}$	$Q_{25}$	Median	$Q_{75}$	$Q_{25}$	Median	$Q_{75}$	$Q_{25}$	Median	$Q_{75}$
Uncon AE	2.52	4.74	13.58	1.79	3.27	11.32	1.71	2.75	8.53	1.54	2.49	7.45	1.43	2.36	7.25	1.41	2.43	6.23
Cond AE	2.04	3.56	9.45	1.66	3.21	8.57	1.46	2.63	7.49	1.40	2.32	7.40	1.34	2.36	7.29	1.32	2.31	6.11
Uncon Diffusion	2.49	4.39	11.91	1.74	3.29	9.42	1.43	2.50	7.09	1.31	2.32	6.20	1.18	2.19	5.88	1.13	2.06	5.77
Cond Diffusion	1.84	3.70	9.15	1.45	2.80	7.85	1.30	2.28	6.99	1.20	2.26	6.25	1.09	1.99	6.03	1.05	1.81	6.07

Appendix

Table 13: Quartiles of per-system NMSE across 100 UK weather slices.

Figure 28: Mean of per-system text gain from conditioning for UK weather test set . The figure plots the text gain $\log_{2}(\mathrm{NMSE}_{\text{uncon}}/\mathrm{NMSE}_{\text{cond}})$ , averaged over $100$ test systems, inside the text-described regions (solid) and outside them (dashed).

Model	NMSE $(\times 10^{-2})$	$\%C_{68}$	$\%C_{95}$
Uncon Supervised	5.95 $\pm$ 5.31	–	–
Cond Supervised	5.11 $\pm$ 4.36	–	–
Uncon AE	5.22 $\pm$ 4.86	68.9 (4.6)	94.0 (2.3)
Cond AE	4.41 $\pm$ 4.01	65.9 (5.6)	92.3 (3.6)
Uncon Diffusion	5.04 $\pm$ 4.72	56.7 (11.6)	86.3 (9.0)
Cond Diffusion	4.20 $\pm$ 3.70	57.2 (11.5)	86.7 (8.5)

Appendix

Table 14: Ablation on a different observation operator of the UK weather experiment. Temperature is measured at

8

locations and the two wind components at

8

different locations, which is a sensor layout the supervised models were not trained on. Results are over

20

test systems.

Bayesian Inverse Problems

Bayesian inverse problems infer hidden signals or physical parameters from indirect, noisy measurements while quantifying uncertainty.

See all 118 papers

May 20, 2026

Conditioning Gaussian Processes on Almost Anything

Gaussian processes (GPs) offer a principled probabilistic model over functions, but exact inference is restricted to the linear-Gaussian regime. We establish an explicit equivalence between GPs and a class of linear diffusion models, recasting predictive sampling as an ODE with closed-form Gaussian dynamics and a likelihood-dependent guidance term that admits a simple Monte Carlo approximation. In the linear-Gaussian setting, we recover standard GP conditioning exactly; beyond conjugacy, the same machinery handles any conditioning statement admitting point-wise likelihood evaluation -- including non-linear physics, and, for the first time, natural language via large language models. Whitening isolates the irreducible non-Gaussian dynamics, minimising Wasserstein-2 transport cost and eliminating numerical stiffness. The result is a general-purpose GP inference scheme requiring no bespoke derivations. Together, these results provide a general mechanism for incorporating the full richness of real-world knowledge as conditioning information, opening a new frontier for the probabilistic modelling of real-world problems.

Henry Moss, Lachlan Astfalck, Thomas Cowperthwaite +5

Lancaster University · University of New South Wales · University of Cambridge +1

Gaussian Processes Conditional Distribution Estimation

Ask PDF

Sep 26, 2026

Efficient Message Passing for Partial Differential Equation Priors

Prior information for real-world physical quantities is most elegantly expressed via partial differential equations (PDEs). In this paper, we propose a novel way to solve PDEs using probabilistic inference on a factor graph. In general, factor graphs provide a natural way to encode prior knowledge into a model as explicit factors; here, this knowledge is provided by a governing PDE, which narrows the solution space, while observed data further shape the posterior over the parameters. The approximate parameter posterior is inferred using message passing based on moment matching, without posterior sampling or global gradient-based optimization. We demonstrate our approach on the first-order advection and the second-order semi-linear Fisher-KPP equations, where it achieves predictive accuracy comparable to a standard baseline while providing structured predictive uncertainty. Moreover, the inferred posterior marginal means and uncertainty structure match more closely those obtained using Hamiltonian Monte Carlo than the evaluated mean-field variational inference baseline, while requiring up to 10x less training time in our experiments, with inference speed comparable to variational inference.

Anna Kazachkova, Leonhard Hennicke, Rainer Schlosser +1

Hasso Plattner Institute University of Potsdam Potsdam, Germany

PDE Solving Partial Differential Equations Physics-Informed ML Bayesian Inference

Ask PDF

Jun 3, 2026

The Right Measure for Physics-Constrained Generation: A Co-Area Correction for Posterior-Consistent PDE Inverse Problems

Generative models -- diffusion and flow matching -- are increasingly used to solve partial differential equation (PDE) inverse problems, enforcing the governing physics as a hard constraint (via projection or guidance) and reporting the resulting samples as a Bayesian posterior with calibrated uncertainty. We show that this widely adopted recipe samples the wrong distribution. Conditioning a generative prior on a hard PDE constraint is conditioning on a measure-zero manifold -- an operation that is intrinsically ambiguous (the Borel--Kolmogorov paradox) and whose physically correct resolution, the small-residual-noise limit, carries a co-area (Fixman) Jacobian factor

[det(JJ^{\top})]^{-1/2}

that projection- and guidance-based methods silently omit. We make the bias precise, show that it grows with the heterogeneity of the constraint sensitivity, and validate it on controlled problems against an i.i.d. ground-truth arbiter. The omitted factor is not a second-order detail: removing it inflates the posterior error to

20\times

the sampling-noise floor; minimal-displacement projection (as in PCFM) is biased at

9\times

the floor; and a naive scalar reweighting does not fix it. We introduce CoCoS, a measure-aware constrained sampler that targets the correct co-area posterior, and show that it matches the gold-standard posterior to within sampling noise. Our results imply that satisfying the physics'' is not the same as sampling the posterior,'' and give a principled correction for uncertainty-aware scientific inference.

Jian Xu, Yanning Wu, Delu Zeng +2

1RIKEN iTHEMS · 2RIKEN AIP · 3South China University of Technology +1

Constrained Generative Modeling Physics-Informed Generative Modeling Bayesian Inverse Problems Diffusion-Based Inverse Problems Diffusion Posterior Sampling PDE Inverse Problems+1

Ask PDF

Abstract

Bayesian Inverse Problems

Explore similar work

Conditioning Gaussian Processes on Almost Anything

Efficient Message Passing for Partial Differential Equation Priors

The Right Measure for Physics-Constrained Generation: A Co-Area Correction for Posterior-Consistent PDE Inverse Problems