Dual-space posterior sampling for Bayesian inference in constrained inverse problems

Ali Siahkoohi; Kamal Aghazade; Ali Gholami

physics.geo-phFeb 28, 2026

Dual-space posterior sampling for Bayesian inference in constrained inverse problems

Authors: Ali Siahkoohi, Kamal Aghazade, Ali Gholami

Organizations: Department of Computer Science, University of Central Florida, Orlando, FL, USA · Institute of Geophysics, Polish Academy of Sciences, Warsaw, Poland

Abstract

Inverse problems constrained by partial differential equations are often ill-conditioned due to noisy, incomplete data or inherent non-uniqueness. A prominent example is full waveform inversion (FWI), which estimates Earth's subsurface properties by fitting seismic measurements subject to the wave equation, where ill-conditioning stems from noisy, band-limited, finite-aperture measurements and complex geological structures. A Bayesian framework describes the solution more comprehensively: instead of a single estimate, a posterior distribution of plausible solutions characterizes the non-uniqueness and can be sampled to quantify uncertainty. However, no clear procedure exists for translating hard physical constraints, such as the wave equation, into priors amenable to existing sampling techniques. We address this by sampling the posterior in the dual space via an augmented Lagrangian formulation, which converts hard constraints into penalties suited to sampling algorithms while enforcing them progressively through multiplier updates, so they are satisfied in the limit. We integrate the alternating direction method of multipliers (ADMM) with Stein variational gradient descent (SVGD), a particle-based sampler: the constraint is relaxed at each iteration and the multiplier updates progressively enforce its satisfaction. This enables posterior sampling under hard constraints while inheriting the favorable conditioning of dual-space solvers, where partial constraint relaxation permits productive updates even when the current model is far from the true solution. We validate the method on a stylized Rosenbrock conditional inference problem and on frequency-domain FWI for a Gaussian anomaly model and the Marmousi II benchmark, demonstrating physically consistent uncertainty estimates and posterior contraction with increasing data coverage.

Figures & tables

Panel of Figure 1 — Figure 1: Conditional sampling problem. (a) Rosenbrock prior $p(\bm{x})$ with four test instances (pink triangles). (b) Noisy observations, $\sigma=0.5$ .

Figure 2: ADMM-SVGD posteriors for the four test observations: prior samples (gray), posterior samples (pink), the noisy observation ( $\triangledown$ ), and the true value ( $\triangle$ ).

Figure 3: ADMM-SVGD convergence diagnostics, four test instances. (a) Constraint residual $|z-x_{1}^{2}|$ . (b) Ensemble-mean multiplier $\bar{\varepsilon}$ . (c) Ensemble-mean penalty term. (d) Average log-posterior. (e) Kernel bandwidth. (f)–(g) Posterior mean, $x_{1}$ and $x_{2}$ . (h) Posterior standard deviation.

Figure 4: Posteriors from both methods for the four test observations: ADMM-SVGD (pink), reduced-space SVGD (green), the noisy observation ( $\triangledown$ ), and the true value ( $\triangle$ ).

Figure 5: Quantile–quantile plots of the reduced-space SVGD against the ADMM-SVGD marginals. Each column is one test instance; the top and bottom rows are the $x_{1}$ and $x_{2}$ marginals. Diagonal points indicate agreement.

Figure 6: Samples overlaid on the analytic conditional posterior $p(\bm{x}|\bm{y})$ for the four test instances; brighter background is higher density. ADMM-SVGD (pink), reduced-space SVGD (green), the noisy observation ( $\triangledown$ ), and the true value ( $\triangle$ ).

Figure 7: Simulation-based calibration rank histograms; $L=256$ replications, $N=99$ posterior samples each. (a)–(b) Ranks of the true $x_{1}$ for ADMM-SVGD and reduced-space SVGD; (c)–(d) the same for $x_{2}$ . Dashed: expected count under uniformity. Shaded band with dotted bounds: $99\%$ binomial interval.

Parameter	Gaussian anomaly	Marmousi II
Model and domain
Domain size ( $x\times z$ )	2 km $\times$ 2 km	17.0 km $\times$ 3.5 km
Grid spacing	10 m (200 $\times$ 200)	25 m (140 $\times$ 680)
Acquisition
Sources	50, $\Delta s=40$ m	$N_{s}=17,34,68$
Receivers	200, $\Delta r=10$ m	114, $\Delta r=150$ m

Table 1: Settings for the Gaussian-anomaly and Marmousi II experiments.

Figure 8: Gaussian-anomaly setup. (a) True velocity; sources (colored dots) at the surface, receivers at the bottom. (b)–(c) Two prior particles. (d) Shot gather in the true model. (e)–(f) Its traces interleaved with those of (b) and (c).

Figure 9: ADMM-SVGD against reduced-space SVGD, Gaussian-anomaly model; 4 Hz data, 60 iterations, identical initial ensemble. (a)–(c) Reduced-space SVGD: conditional mean, pointwise standard deviation, interleaved shot gather (middle source). (d)–(f) ADMM-SVGD, same quantities. (g) Relative model error, (h) normalized data residual. (i) PDE residual and (j) multiplier norm per ADMM-SVGD particle; ensemble mean dashed black.

Figure 10: Normalized posterior density (color) along the

z=1.0

km profile through the Gaussian anomaly: reduced-space SVGD (left), ADMM-SVGD (right). Overlaid: conditional mean (solid red), prior mean (dotted blue), prior particles (gray), and true model (dashed orange).

Figure 11: Marmousi II setup. (a) True P-wave velocity; receivers (inverted triangles), sources (red stars). (b) Velocity profiles at $x=3$ and $8.75$ km (white dashed lines): initial particles (colored), true model (black dotted). (c) Shot gather in the true model, then its traces interleaved with those of particles 10, 20, and 40.

Figure 12: Conditional mean velocity at successive cycle–frequency stages for $N_{s}=17$ , $34$ , and $68$ (columns). Rows are cycle–frequency pairs; RME is reported per panel.

Figure 13: Pointwise posterior standard deviation for the stages and ensemble sizes of Figure 12 .

Figure 14: Velocity estimates for particles 10, 20, and 40 (columns) with $N_{s}=17$ sources. Rows are successive cycle–frequency stages.

Figure 15: Same as Figure 14 but for $N_{s}=34$ .

Figure 16: Same as Figure 14 but for $N_{s}=68$ .

Figure 17: Marmousi II results for $N_{s}=17$ , $34$ , and $68$ sources (left to right). Top: conditional mean velocity with RME. Middle: pointwise standard deviation. Bottom: absolute difference from the true model.

Figure 18: Marmousi II convergence. (a) Mean pointwise standard deviation and (b) relative model error versus iteration; the vertical dashed line separates cycles. (c) Normalized data residual for

N_{s}=17

34

, and

68

(top to bottom), cycle 1 solid and cycle 2 dotted, with frequency transitions dashed.

Figure 19: Interleaved shot gathers for (a) $N_{s}=17$ , (b) $34$ , and (c) $68$ sources, alternating trace segments from the true seismogram and from the conditional-mean model.

Figure 20: AL terms for Marmousi II at the lowest frequency: dual norm (a)–(c) and constraint-residual norm (d)–(f), for $N_{s}=17$ , $34$ , and $68$ (columns). Individual particles colored, ensemble mean black dotted. The vertical dashed line marks the cycle boundary, where the multipliers are reinitialized.

Figure 21: Vertical profiles of the normalized posterior density at three lateral positions for

N_{s}=17

34

, and

68

sources (left to right). True velocity profile dotted red.

Figure 22: Pointwise posterior density at four locations in the Marmousi II model for

N_{s}=17

34

, and

68

sources. Prior in red, true velocity dashed black.

Experiment	Method	$N_{s}$	Runtime (min)	Peak memory (GB)
Gaussian anomaly	Reduced-space SVGD	50	20.83	9.29
Gaussian anomaly	ADMM-SVGD	50	32.76	12.00
Marmousi II	ADMM-SVGD	17	329.87	10.031
		34	463.62	14.934
		68	829.09	25.968
	Reduced-space SVGD	34	363.67	10.098

Table 2: Total runtime and peak memory for reduced-space SVGD and ADMM-SVGD in the Gaussian-anomaly (Figure 9 ) and Marmousi II (Figure 23 ) experiments, with ADMM-SVGD resolved by source count

N_{s}

Figure 23: Reduced-space SVGD (a) and ADMM-SVGD (b), Marmousi II; $N_{s}=34$ , $N_{p}=50$ , identical initial ensemble. Top to bottom: conditional mean, pointwise standard deviation, absolute error against the true model. (c) Relative model error versus iteration.

Figure 24: Deterministic ADMM against ADMM-SVGD, Marmousi II (

N_{s}=34

). (a) Final per-particle data residual (2-norm): deterministic (blue circles), ADMM-SVGD (red squares); ensemble means black dashed and red dash-dot. (b) Per-particle relative model error, with means.

Figure 25: Magnitude of the 3 Hz data in the source–receiver domain. Top left: noise-free; the rest add 10%, 15%, and 20% complex Gaussian noise per frequency. White dashed line: source gather no. 9, inset showing its magnitude versus receiver number, noise-free (black) and noisy (red).

Figure 26: Marmousi II inversion under increasing noise (left to right). Top: conditional mean velocity with RME. Middle: pointwise posterior standard deviation. Bottom: absolute difference from the true model.

Figure 27: Vertical profiles of the normalized posterior density at three lateral positions for

N_{s}=34

, with 10%, 15%, and 20% noise (left to right). True velocity profile dotted red.

Figure 28: Marmousi II convergence at different noise levels. (a) Mean pointwise posterior standard deviation and (b) relative model error versus iteration.

Figure 29: Per-particle data residual (2-norm) versus iteration over two cycles, for (a) 10%, (b) 15%, and (c) 20% noise. Particles colored by index (colorbar), ensemble mean black. Vertical dashed lines: frequency transitions, labeled atop (a). Horizontal dotted lines: per-frequency noise level, colored to match.

Bayesian Inverse Problems

Bayesian inverse problems infer hidden signals or physical parameters from indirect, noisy measurements while quantifying uncertainty.

See all 118 papers

Jun 25, 2026

Latent Diffusion Posterior Sampling with Surrogate Likelihood Guidance for PDE Inverse Problems

We propose latent-space diffusion posterior sampling (L-DPS), an approximate Bayesian framework for high-dimensional inverse problems governed by partial differential equations (PDEs). The method addresses three challenges in PDE-constrained inversion: implicit sample-based priors without tractable densities, high-dimensional spatially distributed parameters, and the high cost of repeated forward-model evaluations during posterior sampling. L-DPS combines a variational autoencoder, an unconditional latent diffusion model, diffusion posterior sampling, and a differentiable neural surrogate. The VAE maps the parameter field to a lower-dimensional latent space, the diffusion model learns an implicit prior score in this latent space, and DPS combines this learned prior with likelihood-based guidance. The likelihood gradient is evaluated through the decoder-surrogate composition, avoiding repeated calls to the full numerical PDE solver. We evaluate the method on an inverse Darcy flow problem with an unknown spatially distributed permeability field inferred from sparse and noisy pressure observations. L-DPS produces accurate and robust inverse solutions, reduces inference cost relative to full-space DPS, and outperforms amortized inverse baselines such as conditional latent diffusion and inverse FNO in sparse and noisy regimes. We further compare L-DPS with a KLE-MAP baseline and study mixed-prior generalization and the sensitivity of inversion accuracy to surrogate forward-model error.

Yuanzhe Wang, Alexandre M. Tartakovsky

Department of Civil and Environmental Engineering University of Illinois Urbana-Champaign Urbana, IL 61801

Latent Diffusion Models Bayesian Inverse Problems Diffusion-Based Inverse Problems Diffusion Posterior Sampling PDE Inverse Problems

Ask PDF

Feb 11, 2026

A Gibbs posterior sampler for inverse problem based on prior diffusion model

This paper addresses the issue of inversion in cases where (1) the observation system is modeled by a linear transformation and additive error, (2) the problem is ill-posed and regularization relies on a Bayesian strategy, (3)~the prior is modeled by a diffusion process adjusted on an available large set of examples. In this context, it is known that the issue of posterior sampling is a thorny one and the paper introduces a Gibbs algorithm. It appears that this avenue has not been explored, and we show that it is particularly effective and remarkably simple. In addition, it provides clear elements regarding convergence guarantees in a specific case and arguments supporting such guarantees in practical cases. The results are clearly confirmed by numerical simulations based on a toy example.

Jean-François Giovannelli

Groupe Signal-Image, IMS (Univ. Bordeaux, CNRS, BINP), Talence, France

Markov Chain Monte Carlo Bayesian Inverse Problems Posterior Sampling Diffusion-Based Inverse Problems Diffusion Posterior Sampling

Ask PDF

Jun 3, 2026

The Right Measure for Physics-Constrained Generation: A Co-Area Correction for Posterior-Consistent PDE Inverse Problems

Generative models -- diffusion and flow matching -- are increasingly used to solve partial differential equation (PDE) inverse problems, enforcing the governing physics as a hard constraint (via projection or guidance) and reporting the resulting samples as a Bayesian posterior with calibrated uncertainty. We show that this widely adopted recipe samples the wrong distribution. Conditioning a generative prior on a hard PDE constraint is conditioning on a measure-zero manifold -- an operation that is intrinsically ambiguous (the Borel--Kolmogorov paradox) and whose physically correct resolution, the small-residual-noise limit, carries a co-area (Fixman) Jacobian factor

[det(JJ^{\top})]^{-1/2}

that projection- and guidance-based methods silently omit. We make the bias precise, show that it grows with the heterogeneity of the constraint sensitivity, and validate it on controlled problems against an i.i.d. ground-truth arbiter. The omitted factor is not a second-order detail: removing it inflates the posterior error to

20\times

the sampling-noise floor; minimal-displacement projection (as in PCFM) is biased at

9\times

the floor; and a naive scalar reweighting does not fix it. We introduce CoCoS, a measure-aware constrained sampler that targets the correct co-area posterior, and show that it matches the gold-standard posterior to within sampling noise. Our results imply that satisfying the physics'' is not the same as sampling the posterior,'' and give a principled correction for uncertainty-aware scientific inference.

Jian Xu, Yanning Wu, Delu Zeng +2

1RIKEN iTHEMS · 2RIKEN AIP · 3South China University of Technology +1

Constrained Generative Modeling Physics-Informed Generative Modeling Bayesian Inverse Problems Diffusion-Based Inverse Problems Diffusion Posterior Sampling PDE Inverse Problems+1

Ask PDF

Abstract

Bayesian Inverse Problems

Explore similar work

Latent Diffusion Posterior Sampling with Surrogate Likelihood Guidance for PDE Inverse Problems

A Gibbs posterior sampler for inverse problem based on prior diffusion model

The Right Measure for Physics-Constrained Generation: A Co-Area Correction for Posterior-Consistent PDE Inverse Problems