Proximal-IMH: Proximal Posterior Proposals for Independent Metropolis-Hastings with Approximate Operators

Youguang Chen; George Biros

cs.LGFeb 24, 2026

Proximal-IMH: Proximal Posterior Proposals for Independent Metropolis-Hastings with Approximate Operators

Authors: Youguang Chen, George Biros

Organizations: Oden Institute for Computational Engineering and Sciences, The University of Texas at Austin, Austin, Texas, USA

Abstract

We consider the problem of sampling from a posterior distribution arising in Bayesian inverse problems in science, engineering, and imaging. Our method belongs to the family of independence Metropolis-Hastings (IMH) sampling algorithms, which are common in Bayesian inference. Relying on the existence of an approximate posterior distribution that is cheaper to sample from but may have significant bias, we introduce Proximal-IMH, a scheme that removes this bias by correcting samples from the approximate posterior through an auxiliary optimization problem. This yields a local adjustment that trades off adherence to the exact model against stability around the approximate reference point. For idealized settings, we prove that the proximal correction tightens the match between approximate and exact posteriors, thereby improving acceptance rates and mixing. The method applies to both linear and nonlinear input-output operators and is particularly suitable for inverse problems where exact posterior sampling is too expensive. We present numerical experiments including multimodal and data-driven priors with nonlinear input-output operators. The results show that Proximal-IMH reliably outperforms existing IMH variants.

Figures & tables

Figure 1: Sensitivity of the expected KL divergence of the Approx , Latent , and Proximal proposals (relative to the Exact posterior ) for different noise levels, operator errors, observation ratios, and dimensions. Details of the experimental setup are given in Table 2 .

Figure 2: Comparison of sampling performance in the bimodal test. Rows correspond to Tests I–III. From left to right, panels show Metropolis–Hastings acceptance rates for three IMH methods, convergence of the relative mean error for the IMH methods together with NUTS and MALA, and histograms of the projected samples

w^{\top}x

for two levels of operator error. In the histogram panels, Approx-NUTS denotes samples drawn from the Approx posterior

\pi_{a}(x\mid y)

. For Proximal-IMH , we set

\beta=\sigma^{2}

. Results in the left panel are averaged over 5 independent trials for each method.

Figure 3: Operator discrepancy in the bimodal test. The quantities

\|\mathbf{I}-\widetilde{{\bf F}}^{-1}{\bf F}\|

and

\|\mathbf{I}-{\bf K}^{-1}\|

are reported for Latent-IMH and Proximal-IMH , respectively, where

{\bf K}

is defined in Equation 7 . For Proximal-IMH , the hyperparameter is set to

\beta=\sigma^{2}

Figure 4: Effect of

\beta

on Proximal-IMH in the bimodal test. Values in parentheses represent

\|{\bf A}-\widetilde{{\bf A}}\|/\|{\bf A}\|

. Blue curves show acceptance rates (left y-axis), and pink curves show relative mean errors after

10^{5}

MH steps (right y-axis).

Panel of Figure 5 — Figure 5: Ground truth, posterior mean and MAP estimates for the MNIST inverse problem in Tests IV–VII . The MAP estimates are obtained from different random initial guesses for the optimization problem. We observe that in several cases the reconstructed MAP point is wrong. This is due to the nonconvexity of solving for a MAP point.

Figure 6: Sampling performance on MNIST imaging inverse problems: convergence of the relative mean error and relative second-moment error for different sampling methods. Dashed lines indicate the bias induced by the Approx posterior

\pi_{a}(x\mid y)

. For Proximal-IMH , we set

\beta=\sigma^{2}

. Results in the convergence plots are reported over 5 independent trials.

Figure 7: Effect of the hyperparameter

\beta

on Proximal-IMH in the inverse problem in MNIST. Values in parentheses represent

\|{\bf A}-\widetilde{{\bf A}}\|/\|{\bf A}\|

. Blue curves show acceptance rates (left y-axis), and pink curves show relative mean errors after

10^{5}

MH steps (right y-axis). The results are not sensitive to

\beta

Panel of Figure 8 — Figure 8: Nonlinear Helmholtz inverse problem. Ground truth, MAP estimate, and posterior mean for the acoustic scattering test, together with convergence of the relative mean error.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Posterior	Distribution	Mean	Covariance	Pseudo-inverse
Exact	$\pi(x\mid y)=\mathcal{N}(\mu,\bm{\Sigma})$	$\mu={\bf A}^{\dagger}y$	$\bm{\Sigma}={\bf I}-{\bf A}^{\dagger}{\bf A}$	${\bf A}^{\dagger}={\bf A}^{\top}\left({\bf A}{\bf A}^{\top}+\sigma^{2}{\bf I}\right)^{-1}$
Approx	$\pi_{a}(x\mid y)=\mathcal{N}(\mu_{a},\bm{\Sigma}_{a})$	$\mu_{a}={\bf A}_{a}^{\dagger}y$	$\bm{\Sigma}_{a}={\bf I}-{\bf A}_{a}^{\dagger}\widetilde{{\bf A}}$	${\bf A}_{a}^{\dagger}=\widetilde{{\bf A}}^{\top}\left(\widetilde{{\bf A}}\widetilde{{\bf A}}^{\top}+\sigma^{2}{\bf I}\right)^{-1}$
Latent	$\pi_{l}(x\mid y)=\mathcal{N}(\mu_{l},\bm{\Sigma}_{l})$	$\mu_{l}={\bf A}_{l}^{\dagger}y$	$\bm{\Sigma}_{l}={\bf F}^{-1}\widetilde{{\bf F}}\bm{\Sigma}_{a}\widetilde{{\bf F}}^{\top}{\bf F}^{-\top}$	${\bf A}_{l}^{\dagger}={\bf F}^{-1}\widetilde{{\bf F}}{\bf A}_{a}^{\dagger}$
Proximal	$\pi_{p}(x\mid y)=\mathcal{N}(\mu_{p},\bm{\Sigma}_{p})$	$\mu_{p}={\bf A}_{p}^{\dagger}y$	$\bm{\Sigma}_{p}={\bf K}\bm{\Sigma}_{a}{\bf K}^{\top}$	${\bf A}_{p}^{\dagger}={\bf K}{\bf A}_{a}^{\dagger}$

Appendix

Table 1: Distributions of Exact , Approx , Latent and Proximal posteriors for Gaussian prior

p(x)=\mathcal{N}({\bf 0},{\bf I})

and noise

q(e)=\mathcal{N}({\bf 0},\sigma^{2}{\bf I})

. For Proximal posterior ,

{\bf K}=({\bf A}^{\top}{\bf A}+\beta{\bf I})^{-1}({\bf A}^{\top}\widetilde{{\bf A}}+\beta{\bf I})

Effects	$\log_{10}\mathrm{SNR}$	$\frac{\\|{\bf F}-\widetilde{{\bf F}}\\|}{\\|{\bf F}\\|}$	$d_{y}/d$	$d$
Noise level	0.5 —4.0	6%	0.2	500
Operator error	2.5	2% —21%	0.2	500
Observation ratio	2.5	6%	0.05 —0.5	500
Dimension	2.5	6%	0.2	100 —2000

Appendix

Table 2: Experimental setup for comparing

\mathbb{D}_{a}

\mathbb{D}_{l}

and

\mathbb{D}_{p}

Figure 9: Histogram of the projection of prior samples in the bimodal test (see Equation 21 ).

$d_{z}$	$d_{x}$	$d_{u}$	$\widetilde{d}_{u}$	$d_{y}$
128	784 ( $=28^{2}$ )	$10^{4}$ ( $=100^{2}$ )	$\{35^{2},\,20^{2}\}$	100

Appendix

Table 3: Dimensions of parameters in Tests VI and VII in Section 4.2 .

Panel of Figure 10 — Figure 10: Matrix constructions of ${\bf A}$ and $\widetilde{{\bf A}}$ in Tests VI and VII in Section 4.2 . Left: ${\bf A}={\bf O}\,{\bf F}\,{\bf P}$ . Right: $\widetilde{{\bf A}}={\bf O}\,{\bf P}_{u}\,\widetilde{{\bf F}}\,{\bf P}_{x}$ . In our setting $d_{y}<d_{x}<d_{u}$ , and $\widetilde{d}_{u}<d_{u}$ .

Figure 11: Jacobian log-determinant diagnostics for the Gauss–Newton correction. Each row corresponds to one test case. Left: Sorted values of the log determinant of the Jacobian of a single Gauss–Newton transformation, evaluated at 2000 randomly selected posterior samples from

\pi_{a}(x\mid y)

. Right: Order statistics of ratios of Jacobian determinants between 2000 randomly selected pairs of posterior samples from

\pi_{a}(x\mid y)

, i.e.,

\det J_{\mathrm{GN}}(x_{i})/\det J_{\mathrm{GN}}(x_{j})

. The numbers shown in the legends indicate the relative operator approximation error

\|{\bf A}-\widetilde{{\bf A}}\|/\|{\bf A}\|

. In the right panels, dashed lines denote the 5%–95% quantile range.

$d_{x}$	$d_{u}$	$\widetilde{d}_{u}$	$d_{y}$
1024 ( $=32^{2}$ )	16,384 ( $=128^{2}$ )	4,096 ( $=64^{2}$ )	512

Appendix

Table 4: Dimensions of parameters in the nonlinear Helmholtz test in Section 4.3 .

Figure 12: Effect of preconditioning on GMRES convergence for the nonlinear Helmholtz test.

Panel of Figure 13 — Figure 13: Wavefields $\{u_{i}=\mathcal{L}(x)^{-1}f_{i}\}_{i=1}^{4}$ for the nonlinear Helmholtz test. Each column corresponds to a distinct source $f_{i}$ . Top: Ground-truth wavefields. Middle: Wavefields obtained from the MAP estimate. Bottom: Posterior mean wavefields computed from posterior samples.

Explore similar work

Flow Matching for Fast Posterior Sampling in Bayesian Inverse Problems
Oct 1, 2026Jan Blechschmidt, Oliver G. Ernst, Moritz Poguntke +1Posterior Sampling Bayesian Inverse Problems
When, why, and how do diffusion posterior samplers fail? A finite-sample lens
May 28, 2026Benjamin A. Burns, Sara Fridovich-KeilBayesian Inverse Problems Posterior Sampling
Georgia Institute of Technology
Alternating Diffusion for Proximal Sampling with Zeroth Order Queries
Mar 20, 2026Hirohane Takagi, Atsushi NitandaDiffusion Sampling
Graduate School of Information Science and Technology, The University of Tokyo, Japan · Institute of High Performance Computing, Agency for Science, Technology and Research, Singapore · Centre for Frontier AI Research, Agency for Science, Technology and Research, Singapore +1

Oct 1, 2026J/K move · Enter open · S save

Flow Matching for Fast Posterior Sampling in Bayesian Inverse Problems

Jan Blechschmidt, Oliver G. Ernst, Moritz Poguntke, Björn Sprungk

Posterior Sampling Bayesian Inverse Problems Amortized Inference Conditional Flow Matching PDE Inverse Problems

Open paper AskPDF arXiv

Sampling from the posterior is the central task of computational Bayesian inverse problems. The standard workhorse in Bayesian inference - Markov chain Monte Carlo (MCMC) - is sequential, yields correlated samples, and must be rerun for each observation. Conditional flow matching offers an amortized alternative: a transport map, trained once on joint samples of parameter and data, that yields independent approximate posterior samples for any observation at negligible online cost, without new likelihood evaluations. We give a careful, MCMC-literate assessment of flow matching for PDE-based inverse problems with function-valued parameters. Exploiting the flow's tractable density, we derive computable accuracy estimates of the underlying approximate posterior in total-variation distance and Kullback-Leibler divergence and, moreover, propose a hybrid sampler that is asymptotically exact by Metropolization. We validate the accuracy estimates and demonstrate the amortization in several numerical examples, including electrical impedance tomography and a likelihood-free Lotka-Volterra model.

Abstract

Explore similar work

Flow Matching for Fast Posterior Sampling in Bayesian Inverse Problems

When, why, and how do diffusion posterior samplers fail? A finite-sample lens

Alternating Diffusion for Proximal Sampling with Zeroth Order Queries