Diffusion Posterior Sampling with Uncertain Forward Models
Organizations: Department of Electrical and Computer Engineering, Boston University, Boston, MA 02215, USA
Abstract
Diffusion posterior samplers reconstruct signals by combining a pretrained diffusion prior with a measurement likelihood specified by a forward model and a noise model. Calibration and reference data can characterize uncertainty in the forward model's parameters, such as blur kernels or detector responses. Fixing these parameters at estimated values leaves their uncertainty out of the likelihood, and parameter errors are thus absorbed into the reconstruction. We introduce an uncertainty-aware likelihood (UAL), a Gaussian approximation to the marginal likelihood obtained by integrating over the uncertain parameters. For each candidate signal, UAL propagates parameter uncertainty into a structured measurement covariance, reducing the quadratic penalty along residual directions that parameter variation can explain. Existing samplers can use UAL in their data updates while retaining their pretrained diffusion priors and sampling schedules. Under a fixed Gaussian nuisance model, we bound likelihood error from linearization and average divergence between the exact marginal and UAL posteriors. We compare samplers with and without UAL across six inverse problems spanning Gaussian and motion deblurring, CT, MRI, ultrasound, and fluorescence microscopy, and find improvements in reconstruction and ensemble prediction. UAL reduces geometric mean MSE by 9% to 20% across samplers in the main Gaussian-deblurring setting and by 51% to 92% in motion deblurring when non-blind samplers use the population-mean kernel. In motion deblurring, DAPS and PnP-DM with UAL also give 6.2 to 6.4 dB higher ensemble-mean PSNR than published BlindDPS and Fast Diffusion EM. On the ablation set, UAL outperforms trace-matched isotropic inflation for PnP-DM in both deblurring tasks; in motion deblurring, it also exceeds the best-tested fixed scalar data weighting by 3.3 dB for PnP-DM and 4.2 dB for DAPS in average PSNR.
Figures & tables
| Sampler / task | Version | PSNR T/M | SSIM T/M | CRPS | Energy | LVS |
|---|---|---|---|---|---|---|
| Gaussian blur | ||||||
| Published | 29.49 / 30.60 | .7416 / .8143 | .02871 | 20.128 | 11.240 | |
| DAPS | UAL | 29.88 / 30.94 | .7468 / .8171 | .02796 | 19.449 | 11.021 |
| Published | 26.97 / 29.53 | .5603 / .7616 | .03209 | 20.874 | 15.286 | |
| PnP-DM | UAL | 27.50 / 30.35 | .5721 / .7776 | .02956 | 18.803 | 14.252 |
| Published | 29.13 / 30.28 | .7687 / .8180 | .02948 | 21.263 | 10.788 | |
| Gaussian blur | Motion blur | |||||
|---|---|---|---|---|---|---|
| Metric | Published | Isotropic | UAL | Published | Isotropic | UAL |
| PSNR T (dB) | 26.944 | 27.020 | 27.475 | 17.872 | 24.084 | 26.641 |
| CRPS ( ) | 32.53 | 32.42 | 29.81 | 131.53 | 58.55 | 31.49 |
| Metric | Published | Fixed covariance | Derivatives stopped | UAL |
|---|---|---|---|---|
| PSNR T (dB) | 26.944 | 26.964 | 27.506 | 27.475 |
| CRPS ( ) | 32.53 | 34.84 | 29.76 | 29.81 |
Appendix figures & tables26 assets
Supplementary material from the paper’s appendix.
Appendix
| Quantity | Point Gaussian likelihood | UAL |
|---|---|---|
| Mean | ||
| Covariance | ||
| Negative log likelihood |
| Task | Nuisance coordinates | Nuisance statistics | UAL construction |
|---|---|---|---|
| Gaussian blur | Kernel widths | Width mean and covariance and | Width-sensitivity covariance (Sections C.1 – C.2 ). |
| Motion blur | Coefficients in the retained kernel basis | Mean and covariance from kernels | Gaussian moment matching in retained coordinates (Section C.1 ). |
| CT | Flat and dark reference fields | Mean and covariance across 20 flat and 20 dark frames | Linearized flat–dark calibration map (Section C.3 ). |
| MRI | Real coil-map coefficients | 32 calibration replicates; calibrated maps for the mean prediction | Exact Gaussian marginal for a forward model affine in Gaussian coil-map coefficients (Section C.4 ). |
| Ultrasound | Sound speed | Gaussian scale from CUBDL reference spread; truncated quadrature | Per-mode variances from nonlinear sound-speed perturbations (Section C.5 ). |
| Microscopy | Five pupil-phase coefficients (Noll modes 5, 6, 7, 8, 11) | Prescribed Gaussian phase law, also used to generate the synthetic observations | Derivatives of the normalized PSF; energy with candidate-dependent covariance (Section C.6 ). |
| Sampler / task | PSNR gain 95% CI | CRPS reduction 95% CI |
|---|---|---|
| Gaussian blur: mean width 3 pixels, standard deviation 0.5 pixels | ||
| DAPS | ||
| PnP-DM | ||
| DiffPIR | ||
| BlindDPS | ||
| Motion blur | ||
| Likelihood | iter. 0 | iter. 25 | iter. 50 | iter. 74 | iter. 99 | |
|---|---|---|---|---|---|---|
| 0.25 | Published | |||||
| Isotropic | ||||||
| UAL | ||||||
| 0.5 | Published | |||||
| Isotropic | ||||||
| UAL |
| Sampler | Treatment | PSNR T/M | SSIM T/M | CRPS | Energy | LVS |
|---|---|---|---|---|---|---|
| DAPS | Published | 17.68 / 18.10 | .3303 / .3779 | .15043 | 96.013 | 57.076 |
| UAL | 28.84 / 29.58 | .7103 / .7726 | .03502 | 23.388 | 12.774 | |
| Oracle | 30.65 / 32.50 | .7698 / .8537 | .02265 | 15.048 | 8.783 | |
| PnP-DM | Published | 17.04 / 17.90 | .2766 / .3897 | .14485 | 91.587 | 61.032 |
| UAL | 26.54 / 29.43 | .5552 / .7628 | .03236 | 20.943 | 14.959 | |
| Oracle | 27.65 / 30.68 | .5752 / .7816 | .02878 | 17.908 | 13.728 |
| Pair | PSNR M (dB) | PSNR T (dB) | SSIM M | SSIM T | CRPS | Energy | LVS |
|---|---|---|---|---|---|---|---|
| DAPS with UAL vs BlindDPS | +6.37 [5.81, 6.95] 50 | +6.72 [6.25, 7.19] 50 | +0.1463 [0.1330, 0.1606] 50 | +0.1544 [0.1385, 0.1713] 50 | +28.90 [25.12, 32.81] 50 | +23.24 [20.35, 26.31] 50 | +5.52 [4.51, 6.58] 48 |
| DAPS with UAL vs Fast Diffusion EM | +6.31 [5.69, 6.87] 48 | +6.08 [5.53, 6.60] 49 | +0.2201 [0.1960, 0.2435] 48 | +0.1868 [0.1621, 0.2114] 48 | +42.87 [37.43, 48.27] 48 | +28.48 [24.98, 31.91] 48 | +11.50 [10.15, 12.83] 49 |
| PnP-DM with UAL vs BlindDPS | +6.22 [5.69, 6.75] 50 | +4.42 [3.98, 4.85] 49 | +0.1364 [0.1237, 0.1500] 50 | 0.0007 [ 0.0199, 0.0190] 23 | +31.55 [28.30, 34.94] 50 | +25.69 [23.11, 28.33] 50 | +3.33 [2.17, 4.45] 42 |
| PnP-DM with UAL vs Fast Diffusion EM | +6.16 [5.58, 6.70] 50 | +3.78 [3.25, 4.29] 47 | +0.2103 [0.1859, 0.2338] 49 | +0.0316 [0.0052, 0.0582] 31 | +45.53 [40.55, 50.38] 50 | +30.92 [27.58, 34.11] 50 | +9.31 [7.73, 10.88] 47 |
| Fast Diffusion EM with UAL vs BlindDPS | +7.52 [6.97, 8.05] 50 | +6.98 [6.46, 7.51] 50 | +0.1845 [0.1717, 0.1981] 50 | +0.2076 [0.1941, 0.2215] 50 | +36.84 [33.50, 40.45] 50 | +28.30 [25.60, 31.16] 50 | +8.66 [7.68, 9.74] 50 |
| Fast Diffusion EM with UAL vs Fast Diffusion EM | +7.45 [6.83, 8.02] 50 | +6.34 [5.71, 6.93] 48 | +0.2583 [0.2329, 0.2832] 50 | +0.2399 [0.2150, 0.2640] 49 | +50.81 [45.32, 56.30] 50 | +33.53 [30.09, 36.86] 50 | +14.64 [13.21, 16.05] 50 |
| Metric | Mean gain | 95% interval |
|---|---|---|
| PSNR T (dB) | 0.0127 | |
| PSNR M (dB) | 0.0141 | |
| SSIM T | 0.000313 | |
| SSIM M | 0.000119 | |
| CRPS | 0.0000331 | |
| Energy | 0.0220 |
| Metric | Published | UAL |
|---|---|---|
| CNR (dB), higher | 9.9292 | 12.3625 |
| gCNR | 0.9141 | 0.9743 |
| Speckle SNR, target | 2.0616 | 2.0184 |
| Axial FWHM (mm) | 0.2506 | 0.2497 |
| Lateral FWHM (mm) | 0.5729 | 0.5722 |
| 1 | 3 | 6 | 12 | 24 | 48 | 96 | 192 | 384 | |
|---|---|---|---|---|---|---|---|---|---|
| PnP-DM | 17.872 | 20.429 | 21.652 | 22.569 | 23.243 | 23.369 | 23.210 | 22.503 | 21.419 |
| DAPS | 18.441 | 21.098 | 22.431 | 23.448 | 24.113 | 24.413 | 24.360 | 23.965 | 23.245 |
| Likelihood | PSNR T | PSNR M | CRPS | Energy | LVS |
|---|---|---|---|---|---|
| Published | 26.944 | 29.388 | 32.53 | 21.327 | 15.22 |
| Isotropic | 27.020 | 29.472 | 32.42 | 21.078 | 15.12 |
| UAL | 27.475 | 30.259 | 29.81 | 19.052 | 14.18 |
| Metric | Published | Quadratic only | UAL |
|---|---|---|---|
| PSNR T (dB) | |||
| PSNR M (dB) | |||
| SSIM T | |||
| SSIM M | |||
| CRPS ( ) | |||
| Energy |
| Metric | Published | Covariance weighting | Quadratic only | UAL |
|---|---|---|---|---|
| PSNR T (dB) | 22.76 | 23.80 | 28.49 | 29.10 |
| PSNR M (dB) | 23.27 | 25.30 | 30.64 | 30.72 |
| SSIM T | .5236 | .5663 | .7537 | .7635 |
| SSIM M | .5525 | .6243 | .8131 | .8108 |
| CRPS | .07789 | .05472 | .02731 | .02708 |
| Energy | 51.865 | 37.077 | 18.360 | 18.332 |
| Task | Candidate | |||
|---|---|---|---|---|
| Gaussian blur | Published | |||
| Isotropic | ||||
| UAL | ||||
| Motion blur | Published | |||
| Isotropic | ||||
| UAL |
| Task | Covariance error | PSNR T | PSNR M | CRPS (%) |
|---|---|---|---|---|
| Gaussian | ||||
| Gaussian | ||||
| Gaussian | ||||
| Gaussian | ||||
| Gaussian | ||||
| Motion |
| MSE | PSNR | CRPS | |||||||
|---|---|---|---|---|---|---|---|---|---|
| (pixels) | Published | Isotropic | UAL | Published | Isotropic | UAL | Published | Isotropic | UAL |
| .25 | .007301 | .007312 | .007202 | 27.508 | 27.502 | 27.551 | 29.93 | 29.97 | 29.23 |
| .50 | .008459 | .008263 | .007348 | 26.944 | 27.020 | 27.475 | 32.53 | 32.42 | 29.81 |
| 1.00 | .014066 | .010826 | .008800 | 25.368 | 26.018 | 26.818 | 40.95 | 38.24 | 33.26 |
| 1.50 | .020463 | .012810 | .012655 | 24.043 | 25.318 | 25.789 | 49.58 | 43.24 | 39.90 |
| DAPS | PnP-DM | DiffPIR | BlindDPS | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Version | PSNR | CRPS | PSNR | CRPS | PSNR | CRPS | PSNR | CRPS | |
| Published | 29.49 | 28.71 | 26.97 | 32.09 | 29.13 | 29.48 | 28.31 | 26.98 | |
| UAL | 29.88 | 27.96 | 27.50 | 29.56 | 30.12 | 25.83 | 29.21 | 24.76 | |
| Published | 28.16 | 32.89 | 26.35 | 35.58 | 27.52 | 32.67 | 26.96 | 32.38 | |
| UAL | 28.65 | 32.22 | 26.84 | 33.93 | 28.60 | 28.64 | 27.37 | 29.79 | |
| Published | 28.05 | 35.42 | 25.80 | 38.56 | 26.70 | 43.55 | 28.43 | 27.01 | |
| Sampler | Version | PSNR T/M | SSIM T/M | CRPS | Energy | LVS |
|---|---|---|---|---|---|---|
| pixels | ||||||
| DAPS | Published | 28.16 / 29.32 | .6888 / .7685 | .03289 | 22.999 | 12.581 |
| UAL | 28.65 / 29.67 | .6949 / .7702 | .03222 | 22.512 | 12.631 | |
| PnP-DM | Published | 26.35 / 28.55 | .5325 / .7290 | .03558 | 23.459 | 16.258 |
| UAL | 26.84 / 29.12 | .5366 / .7338 | .03393 | 22.008 | 15.759 | |
| DiffPIR | Published | 27.52 / 29.11 | .7028 / .7712 | .03267 | 22.846 | 11.400 |
| Sampler | Treatment | PSNR T/M | SSIM T/M | CRPS | Energy | LVS |
|---|---|---|---|---|---|---|
| DAPS | Published | 18.44 / 18.89 | .3677 / .4190 | 0.13913 | 88.578 | 52.203 |
| UAL, | 28.63 / 29.34 | .7030 / .7660 | 0.03572 | 24.046 | 13.079 | |
| UAL, | 28.56 / 29.29 | .7002 / .7659 | 0.03568 | 23.962 | 12.990 | |
| PnP-DM | Published | 17.87 / 18.79 | .3095 / .4334 | 0.13153 | 83.874 | 54.395 |
| UAL, | 26.64 / 29.53 | .5541 / .7619 | 0.03149 | 20.351 | 14.766 | |
| UAL, | 26.25 / 29.31 | .5420 / .7517 | 0.03209 | 20.788 | 14.997 |
| Sampler / task | Published (s) | UAL (s) |
|---|---|---|
| Gaussian deblurring | ||
| PnP-DM | 77.3 | 76.9 |
| DAPS | 39.4 | 116.6 |
| DiffPIR | 6.0 | 6.1 |
| BlindDPS | 129.6 | 371.8 |
| Motion deblurring | ||
Diffusion-Based Inverse Problems
Diffusion-based inverse problems use generative priors to recover images and physical fields from incomplete, noisy, or indirect measurements.
See all 145 papers