Latent Annealed Posterior Sampling with a Variance-Preserving Bridge for Sparsely Observed Random Fields

Jiankun Lei; Tian Qi

stat.MLOct 14, 2026

Latent Annealed Posterior Sampling with a Variance-Preserving Bridge for Sparsely Observed Random Fields

Authors: Jiankun Lei, Tian Qi

Organizations: School of Statistics and Mathematics, Yunnan University of Finance and Economics, No. 237 Longquan Road, Wuhua District, Kunming, 650221, Yunnan, China

Abstract

Posterior inference for random fields from sparse observations presents two major challenges: accurately characterizing complex spatial uncertainty and preventing conditional sampling costs from growing substantially as the spatial grid is refined. To address these challenges, we propose latent annealed posterior sampling (LAPS). LAPS first maps the random field into a latent space with reduced dimensionality, while a coordinate decoder composed of Branch and Trunk networks enables field evaluation at arbitrary spatial locations. Flow matching is then used to learn the latent prior distribution. Based on this learned prior, posterior sampling is performed entirely in the latent space, with the observation likelihood incorporated progressively through an annealing procedure. To control local covariance inflation, we introduce a variance-preserving (VP) bridge that allocates a covariance budget between the transported latent state and a Gaussian perturbation. Numerical experiments demonstrate that LAPS accurately approximates posterior distributions, preserves distinct posterior modes, and reduces the dependence of conditional sampling costs on spatial resolution. Across two Gaussian benchmarks, LAPS attains small errors against exact posterior references, with sliced Wasserstein distances only 2 to 3 percent higher than the reference values obtained from exact posterior samples. In a bimodal posterior problem, LAPS reduces the sliced Wasserstein distance from 0.2693 to 0.1499 and consistently recovers both modes across all test scenarios. For a two dimensional Navier-Stokes random field, LAPS substantially improves marginal posterior calibration, while its evaluation time remains nearly constant as the output grid is refined from 64^2 to 512^2. These results demonstrate that LAPS achieves a favorable tradeoff among posterior accuracy, uncertainty calibration, and computational efficiency.

Figures & tables

Method	SWD $\downarrow$	MMD $\downarrow$	$\mathrm{RMSE}_{m}$ $\downarrow$	$\mathrm{MAE}_{s}$ $\downarrow$	s/case †
Matérn (stationary)
CNP	$0.2087_{\pm 0.0072}$	$0.2002_{\pm 0.0098}$	$0.1521_{\pm 0.0084}$	$0.0671_{\pm 0.0022}$	0.07
ANP	$0.1380_{\pm 0.0019}$	$0.1200_{\pm 0.0035}$	$0.0895_{\pm 0.0032}$	$0.0359_{\pm 0.0007}$	0.08
FlowNP	$0.2149_{\pm 0.0027}$	$0.1978_{\pm 0.0051}$	$0.1367_{\pm 0.0047}$	$0.1403_{\pm 0.0016}$	1.21
FAPS	$0.1397_{\pm 0.0009}$	$0.1321_{\pm 0.0010}$	$0.1206_{\pm 0.0008}$	$0.0269_{\pm 0.0006}$	12.52
LAPS (ours)	$\mathbf{0.0614}_{\pm 0.0007}$	$\mathbf{0.0147}_{\pm 0.0012}$	$\mathbf{0.0266}_{\pm 0.0011}$	$\mathbf{0.0133}_{\pm 0.0005}$	5.49

Table 1: Posterior accuracy on the stationary Matérn and non-stationary Gibbs benchmarks at

n=128

. The reference row uses an independent ensemble of exact posterior draws.

\dagger

Times include scoring and come from different devices; they are recorded evaluation times, not a controlled speed comparison ( Runtime measurements ).

Figure 1: Representative posterior estimates for the Matérn and Gibbs benchmarks at

n=128

. The gray band denotes the central

90\%

interval of the analytic posterior; FAPS and LAPS intervals are empirical posterior quantiles. The lower panels show the pointwise posterior standard deviation.

	distance to the exact posterior			template regions
Method	SWD $\downarrow$	MMD $\downarrow$	$\mathrm{RMSE}_{m}$ $\downarrow$	ModeW-MAE $\downarrow$	ModeCov $\uparrow$	Valley $\to 0.04$
CNP	$0.4666_{\pm 0.0292}$	$0.4124_{\pm 0.0223}$	$0.3137_{\pm 0.0306}$	$0.3636_{\pm 0.0666}$	$0.378_{\pm 0.199}$	$0.703_{\pm 0.172}$
ANP	$0.3579_{\pm 0.0044}$	$0.3255_{\pm 0.0072}$	$0.1721_{\pm 0.0069}$	$0.2860_{\pm 0.0093}$	$0.320_{\pm 0.029}$	$0.685_{\pm 0.038}$
FlowNP	$0.3267_{\pm 0.1306}$	$0.2476_{\pm 0.1124}$	$0.1934_{\pm 0.1117}$	$0.2103_{\pm 0.1348}$	$0.856_{\pm 0.130}$	$0.092_{\pm 0.034}$
FAPS	$0.2693_{\pm 0.0007}$	$0.1750_{\pm 0.0006}$	$0.1964_{\pm 0.0006}$	$0.1502_{\pm 0.0013}$	$0.730_{\pm 0.007}$	$0.099_{\pm 0.003}$
LAPS (ours)	$\mathbf{0.1499}_{\pm 0.0090}$	$\mathbf{0.0529}_{\pm 0.0079}$	$\mathbf{0.0678}_{\pm 0.0056}$	$\mathbf{0.0661}_{\pm 0.0071}$	$\mathbf{1.000}_{\pm 0.000}$	$\mathbf{0.045}_{\pm 0.001}$

Table 2: Posterior accuracy and mode structure on the bimodal benchmark at

n=128

: five training seeds, mean

\pm

standard deviation. Distributional errors use exact posterior references. ModeW-MAE and ModeCov use a fixed template assignment; the valley mass is the fraction of draws between the two configurations. The reference row scores five independent ensembles of exact posterior draws ( B.2 Evaluation Metrics ).

Figure 2: Posterior on the bimodal benchmark. Rows: two test scenarios, in which the exact weight of the configuration with the bump at $x=0.35$ is $0.50$ (top) and $0.80$ (bottom). (a–c, e–g) Pointwise posterior density of $u(x)$ for the exact posterior, FAPS and LAPS; darker is denser, on a common square-root scale within each row. Circles are the observations and $w$ is the mass assigned to the first configuration. (d, h) Density of the bump coordinate $\delta$ , the fitted amplitude of the bump at $0.35$ minus that of the bump at $0.65$ . Sampler densities pool the draws of the five training seeds, and all densities are smoothed with the same Gaussian kernel.

Method	CRPS $\downarrow$	SSR $\to 1$	CE $\downarrow$	rel. $L^{2}$ $\downarrow$	s/case †
FAPS	$\mathbf{0.0121}_{\pm 0.0001}$	$1.136_{\pm 0.014}$	$0.0352_{\pm 0.0021}$	$\mathbf{0.0315}_{\pm 0.0003}$	227.7
LAPS (ours)	$0.0158_{\pm 0.0003}$	$\mathbf{0.989}_{\pm 0.012}$	$\mathbf{0.0047}_{\pm 0.0011}$	$0.0369_{\pm 0.0007}$	22.3

Table 3: Posterior inference on the

64\times 64

Navier–Stokes benchmark from

64

noisy observations. CRPS and relative

L^{2}

error assess predictive accuracy, CE assesses marginal calibration, and SSR compares spread with reconstruction error.

\dagger

Recorded times include scoring; unmatched hardware prevents a controlled speed comparison ( Runtime measurements ).

Figure 3: Spatial posterior comparison for a representative Navier–Stokes test case. (a) Reference vorticity and the $64$ observation locations. (b,c) FAPS posterior mean and standard deviation. (d,e) LAPS posterior mean and standard deviation. All estimates use $128$ posterior draws; posterior-mean panels share one color scale and posterior-standard-deviation panels another.

Figure 4: Calibration diagnostics for the Navier–Stokes benchmark, pooled over the five training seeds. (a) Empirical versus nominal central-interval coverage; bands show one standard deviation over seeds. (b) Probability integral transform histogram, with uniformity indicating calibrated marginal uncertainty. (c) Spread–skill relationship; the diagonal denotes equality between posterior spread and reconstruction error.

Figure 5: Recorded evaluation-time ratios for

128\rightarrow 512

points in one dimension and

64^{2}\rightarrow 128^{2}

in two dimensions. Ratios are within each implementation. The two-dimensional FAPS bar uses a two-scenario refined-grid probe and an unmatched reference protocol; it is exploratory ( Runtime measurements ).

Figure 6: Likelihood-free evaluation of the VP bridge with

K=40

, averaged over five training seeds; shaded bands indicate the seed ranges. (a,b) Single local updates using centered learned-prior anchors, matched estimated covariance

\Sigma_{\theta}

and unfloored VP coefficients, with

5000

prior draws per benchmark and seed. (a) Covariance-trace ratio relative to the learned prior; the dashed line gives the original-coefficient factor

1+(1-t_{k})^{2}

. (b) Mean absolute excess kurtosis of

\Sigma_{\theta}

-whitened coordinates; the dashed line indicates the

\alpha_{k}^{4}

scaling for Navier–Stokes. (c) Complete annealing recursion using learned transport, the implemented

\Sigma_{z}

, residual noise and the width floor, with

2000

particles. All panels use

S=200

Euler steps. The matched conditions in (a,b) differ from those of the complete recursion in (c).

Benchmark	Metric	Original	VP	VP $-$ original [95% CI]	Seeds
Matérn	SWD $\downarrow$	$0.0809_{\pm 0.0011}$	$0.0614_{\pm 0.0007}$	$-0.0195$ [ $-0.0204$ , $-0.0186$ ]	5/5
	$\mathrm{MAE}_{s}$ $\downarrow$	$0.0356_{\pm 0.0009}$	$0.0133_{\pm 0.0005}$	$-0.0224$ [ $-0.0232$ , $-0.0216$ ]	5/5
	width $\to 1$	$1.108_{\pm 0.003}$	$0.972_{\pm 0.002}$	$-0.080$ [ $-0.086$ , $-0.074$ ] ∗	5/5
Gibbs	SWD $\downarrow$	$0.1008_{\pm 0.0009}$	$0.0762_{\pm 0.0006}$	$-0.0245$ [ $-0.0255$ , $-0.0236$ ]	5/5
	$\mathrm{MAE}_{s}$ $\downarrow$	$0.0395_{\pm 0.0005}$	$0.0147_{\pm 0.0004}$	$-0.0248$ [ $-0.0252$ , $-0.0245$ ]	5/5
	width $\to 1$	$1.090_{\pm 0.003}$	$0.972_{\pm 0.002}$	$-0.062$ [ $-0.067$ , $-0.056$ ] ∗	5/5

Table 4: Original versus VP bridge coefficients on the same trained models and reported annealing schedules. Results are means

\pm

standard deviations over five training seeds and

100

test scenarios, with

128

posterior draws per scenario. The VP configuration corresponds to the reported LAPS results. Differences are paired by training seed, with

95\%

t

-intervals; “Seeds” counts improvements under VP. For the rows marked

\ast

(the pointwise width ratio and SSR), differences and improvement counts refer to the distance from one. The width ratio, ModeCov and valley mass are defined in B.2 Evaluation Metrics .

	Bimodal			Navier–Stokes
$(K,L)$	SWD $\downarrow$	$\mathrm{RMSE}_{m}$ $\downarrow$	Valley $\to 0.04$	CRPS $\downarrow$	CE $\downarrow$	SSR $\to 1$
reported	$0.1499_{\pm 0.0090}$	$0.0678_{\pm 0.0056}$	$0.045_{\pm 0.001}$	$0.0158_{\pm 0.0003}$	$0.0047_{\pm 0.0011}$	$0.989_{\pm 0.012}$
$(10,\,N_{\rm MALA}/10)$	$0.1410_{\pm 0.0061}$	$0.0615_{\pm 0.0028}$	$0.066_{\pm 0.007}$	$0.0198_{\pm 0.0001}$	$0.0155_{\pm 0.0010}$	$1.004_{\pm 0.004}$
$(1,\,N_{\rm MALA})$	$0.2018_{\pm 0.0005}$	$0.0386_{\pm 0.0023}$	$0.307_{\pm 0.006}$	$0.0340_{\pm 0.0001}$	$0.0196_{\pm 0.0009}$	$0.988_{\pm 0.003}$

Table 5: Effect of the annealing schedule at fixed MALA-update budgets,

N_{\rm MALA}=KL=2000

for the bimodal benchmark and

8000

for Navier–Stokes. Results are means

\pm

standard deviations over five training seeds and

100

test scenarios. The reported configurations use

(K,L)=(40,50)

and

(80,100)

, respectively. Transport work and wall-clock costs are not matched. The

K=1

configuration replaces the learned latent prior with a Gaussian distribution. Valley mass is defined in B.2 Evaluation Metrics .

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Benchmark	Grid	Train / Val.	Observations	Noise $\sigma$	Exact posterior
Matérn	$128$	$20{,}000/1{,}000$	$7$ random	$0.1$	Gaussian
Gibbs	$128$	$20{,}000/1{,}000$	$7$ random	$0.1$	Gaussian
Bimodal	$128$	$20{,}000/1{,}000$	$5$ fixed	$0.05$	Gaussian mixture
Navier–Stokes	$64\times 64$	$30{,}000/2{,}000$	$64$ random	$0.0434$	None

Appendix

Table 1: Benchmark configurations. Each benchmark uses

100

test scenarios. Training and validation sample counts are reported together.

Benchmark	Metric	Published	This work	Difference
Matérn, $n=128$	SWD	$0.142$	$0.1405$	$-1.1\%$
	MMD	$0.128$	$0.1314$	$+2.7\%$
Matérn, $n=512$	SWD	$0.147$	$0.1467$	$-0.2\%$
	MMD	$0.136$	$0.1360$	$+0.0\%$
Gibbs, $n=128$	SWD	$0.194$	$0.1706$	$-12.1\%$
	MMD	$0.167$	$0.1392$	$-16.6\%$

Appendix

Table 2: Reproduction of the FAPS results reported by Shi et al. [32] . The re-run uses the released prior weights. Differences are relative to the published values; positive values indicate larger errors in the re-run.

Benchmark	Output grid	LAPS	FAPS
Matérn	$128$	$5.49$	$12.52$
Matérn	$512$	$5.75$	$49.87$
Navier–Stokes	$64^{2}$	$22.3$	$227.7$
Navier–Stokes	$128^{2}$	$21.8$	$9609.5$
Navier–Stokes	$256^{2}$	$21.8$	–
Navier–Stokes	$512^{2}$	$22.2$	–

Appendix

Table 3: Recorded evaluation time per scenario (s), including sampling, decoding and scoring. One-dimensional entries are those of the Matérn benchmark, averaged over five training seeds; the LAPS Navier–Stokes sweep uses training seed

0

with the sampler configuration fixed. The FAPS

64^{2}

entry averages five sampler seeds and its

128^{2}

entry is a two-scenario probe. Devices are not matched across methods. A dash denotes a setting that was not run.

Setting	Matérn / Gibbs	Bimodal	Navier–Stokes
Latent model
Latent dimension $d$	$32$	$32$	$128$
Decoder rank $r$	$64$	$64$	$256$
Spatial Fourier frequencies	$64$	$64$	$64$
Flow widths	$256\times 2$	$512\times 4$	$64,128,256,512,1024$
Training

Appendix

Table 1: Model, training and posterior sampling settings for the reported LAPS experiments. Flow widths are those of the hidden layers. The autoencoder validation error is the relative

L^{2}

reconstruction error on the validation fields, averaged over the five training seeds and given separately for Matérn and Gibbs.

	Matérn		Gibbs	Bimodal		Navier–Stokes
Sampler	SWD $\downarrow$	MAE s $\downarrow$	SWD $\downarrow$	SWD $\downarrow$	ModeW-MAE $\downarrow$	rel. $L^{2}$ $\downarrow$	CE $\downarrow$	SSR $\to 1$
LAPS	$0.0614_{\pm 0.0007}$	$0.0133_{\pm 0.0005}$	$0.0762_{\pm 0.0006}$	$0.1499_{\pm 0.0090}$	$0.0661_{\pm 0.0071}$	$0.0369_{\pm 0.0007}$	$0.0047_{\pm 0.0011}$	$0.989_{\pm 0.012}$
direct pCN	$0.0617_{\pm 0.0012}$	$0.0159_{\pm 0.0006}$	$0.0756_{\pm 0.0014}$	$0.1401_{\pm 0.0054}$	$0.0568_{\pm 0.0043}$	$0.0399_{\pm 0.0004}$	$0.1185_{\pm 0.0024}$	$1.689_{\pm 0.010}$
direct MALA	$0.1302_{\pm 0.0336}$	$0.0571_{\pm 0.0252}$	$0.1207_{\pm 0.0262}$	$0.2710_{\pm 0.0205}$	$0.1489_{\pm 0.0070}$	$0.2307_{\pm 0.0027}$	$0.1505_{\pm 0.0051}$	$1.571_{\pm 0.018}$

Appendix

Table 1: Comparison of LAPS and direct MCMC under finite computational budgets. Results are means

\pm

standard deviations over five training seeds and

100

test scenarios, with

128

posterior draws per scenario. Direct MCMC uses the same learned latent prior and approximately matched or more generous wall-clock budgets.

Figure 1: Sampled versus exact posterior mode weights on the bimodal benchmark across

100

test scenarios and five training seeds. (a) Linear scale: faint points represent individual training seeds and solid markers their means. The gray band shows

\pm 2

standard deviations for

128

independent exact-posterior draws, and dotted lines indicate the

5\%

and

95\%

thresholds. (b) Logit scale: sampled weights are pooled across five seeds (

640

draws per scenario), with solid lines showing least-squares fits. Open markers denote scenarios in which all pooled draws belong to one configuration and are excluded from the fits.

Benchmark	Metric	Original	VP	VP $-$ original [95% CI]	Seeds
Matérn	SWD $\downarrow$	$0.0809_{\pm 0.0011}$	$0.0614_{\pm 0.0007}$	$-0.0195$ [ $-0.0204$ , $-0.0186$ ]	5/5
	MMD $\downarrow$	$0.0268_{\pm 0.0017}$	$0.0147_{\pm 0.0012}$	$-0.0121$ [ $-0.0144$ , $-0.0098$ ]	5/5
	$\mathrm{RMSE}_{m}$ $\downarrow$	$0.0312_{\pm 0.0012}$	$0.0266_{\pm 0.0011}$	$-0.0046$ [ $-0.0052$ , $-0.0040$ ]	5/5
	$\mathrm{MAE}_{s}$ $\downarrow$	$0.0356_{\pm 0.0009}$	$0.0133_{\pm 0.0005}$	$-0.0224$ [ $-0.0232$ , $-0.0216$ ]	5/5
	width $\to 1$	$1.108_{\pm 0.003}$	$0.972_{\pm 0.002}$	$-0.080$ [ $-0.086$ , $-0.074$ ] ∗	5/5
Gibbs	SWD $\downarrow$	$0.1008_{\pm 0.0009}$	$0.0762_{\pm 0.0006}$	$-0.0245$ [ $-0.0255$ , $-0.0236$ ]	5/5

Appendix

Table 2: Original versus VP bridge coefficients, all scored metrics. Conventions are those of Table 4 .

Posterior Sampling

Posterior sampling generates draws from Bayesian posteriors to quantify uncertainty and support inference, prediction, and decision-making.

See all 69 papers

Jun 21, 2026

Flow Annealing Posterior Sampling for Function-Space Regression and Inverse Problems

Principled regression for stochastic processes is a long-standing challenge with deep connections to scientific inverse problems. We introduce Flow Annealing Posterior Sampling (FLAPS), to our knowledge the first function-space posterior sampling framework that unifies stochastic-process regression and PDE inverse problems. Built on pretrained function-space flow-matching priors, FLAPS enables likelihood-guided posterior inference from sparse and noisy observations, supports variable query discretizations, and avoids explicit prior-density evaluation. Its Langevin correction uses a low-rank covariance preconditioner to exploit dominant function-space correlations across discretizations. Across Gaussian and non-Gaussian stochastic-process regression benchmarks and diverse PDE inverse problems, FLAPS produces coherent posterior samples with well-calibrated uncertainty quantification, significantly outperforming existing functional regression baselines and achieving competitive or better noisy PDE inverse performance than diffusion-based posterior samplers while reducing test-time sampling cost.

Yaozhong Shi, Zachary E. Ross, Yisong Yue

California Institute of Technology

Bayesian Inverse Problems Posterior Sampling Flow-Based Generative Modeling PDE Inverse Problems Gaussian Process Regression

Ask PDF

Oct 9, 2026

Scalable Sparse Autoregressive Variational Inference for Non-Gaussian Spatial Fields

Latent field models are essential for spatial statistics, but the posterior distribution of the latent field, even with a Gaussian process (GP) prior, is analytically intractable whenever the likelihood is non-Gaussian. Existing approximations are either too slow or too restrictive for large non-Gaussian datasets. We present a scalable variational inference framework that mirrors the sparse nearest-neighbor autoregressive structure of the prior in the variational family, uses normalizing-flow conditionals for expressiveness, and amortizes all variational parameters via a single shared neural conditioner. Doubly stochastic optimization with reduced ancestor sets achieves polylogarithmic complexity per iteration.

Paul F. V. Wiemann, Matthias Katzfuss

Department of Statistics, The Ohio State University, Columbus OH 43210, USA · Department of Statistics, University of Wisconsin–Madison, Madison WI 53706, USA

Variational Inference

Ask PDF

Jun 25, 2026

Latent Diffusion Posterior Sampling with Surrogate Likelihood Guidance for PDE Inverse Problems

We propose latent-space diffusion posterior sampling (L-DPS), an approximate Bayesian framework for high-dimensional inverse problems governed by partial differential equations (PDEs). The method addresses three challenges in PDE-constrained inversion: implicit sample-based priors without tractable densities, high-dimensional spatially distributed parameters, and the high cost of repeated forward-model evaluations during posterior sampling. L-DPS combines a variational autoencoder, an unconditional latent diffusion model, diffusion posterior sampling, and a differentiable neural surrogate. The VAE maps the parameter field to a lower-dimensional latent space, the diffusion model learns an implicit prior score in this latent space, and DPS combines this learned prior with likelihood-based guidance. The likelihood gradient is evaluated through the decoder-surrogate composition, avoiding repeated calls to the full numerical PDE solver. We evaluate the method on an inverse Darcy flow problem with an unknown spatially distributed permeability field inferred from sparse and noisy pressure observations. L-DPS produces accurate and robust inverse solutions, reduces inference cost relative to full-space DPS, and outperforms amortized inverse baselines such as conditional latent diffusion and inverse FNO in sparse and noisy regimes. We further compare L-DPS with a KLE-MAP baseline and study mixed-prior generalization and the sensitivity of inversion accuracy to surrogate forward-model error.

Yuanzhe Wang, Alexandre M. Tartakovsky

Department of Civil and Environmental Engineering University of Illinois Urbana-Champaign Urbana, IL 61801

Latent Diffusion Models Bayesian Inverse Problems Diffusion-Based Inverse Problems Diffusion Posterior Sampling PDE Inverse Problems

Ask PDF

Abstract

Posterior Sampling

Explore similar work

Flow Annealing Posterior Sampling for Function-Space Regression and Inverse Problems

Scalable Sparse Autoregressive Variational Inference for Non-Gaussian Spatial Fields

Latent Diffusion Posterior Sampling with Surrogate Likelihood Guidance for PDE Inverse Problems