Beyond Best-of-$k$: Prior-Consistent DAPS for Ambiguous Nonlinear Inverse Problems

Merve Kocabas; Benjamin A. Burns; Andreas Geiger; Kent Gauen; Sara Fridovich-Keil

cs.CVOct 22, 2026

Beyond Best-of- $k$ : Prior-Consistent DAPS for Ambiguous Nonlinear Inverse Problems

Authors: Merve Kocabas, Benjamin A. Burns, Andreas Geiger, Kent Gauen, Sara Fridovich-Keil

Organizations: University of Tübingen · Zuse School ELIZA · Max Planck Institute for Intelligent Systems · Georgia Institute of Technology ·

Abstract

Diffusion models have become strong priors for solving inverse problems. However, nonlinear inverse problems such as phase retrieval remain challenging when multiple measurement-consistent solutions are distinguishable only by the prior; we call such measurements ambiguous. Existing diffusion-based samplers rely on local approximations during reconstruction. For an ambiguous measurement, these approximations can allow measurement guidance to favor a measurement-consistent mode that is close to a prior-supported mode but is itself not supported by the prior. Evaluation often masks this type of failure by reporting the best-of- $k$ runs, where the best is selected by comparison to the ground truth. We introduce Prior-Consistent DAPS (PC-DAPS), a variant of Decoupled Annealing Posterior Sampling (DAPS) designed to address this failure case by maintaining multiple candidate solutions within a single sampling run. As annealing noise decreases, PC-DAPS prunes and replaces weaker candidates according to their prior consistency, without any ground-truth access. PC-DAPS reduces the incidence of prior-inconsistent samples and improves sample quality on nonlinear inverse problems while preserving sample quality and diversity on linear inverse problems.

Figures & tables

Figure 1: Measurement consistency is not sufficient for reconstruction. A single Fourier magnitude measurement is shared by rotations, translations, and color-channel inversions of an image. Existing solvers may return these prior-inconsistent modes, while PC-DAPS selects reconstructions supported by the learned image prior. Each sample was selected randomly, either from four independent runs for other methods or from the four particles generated in a single PC-DAPS run.

Panel of Figure 2 — Figure 2: Premature mode selection. We consider a nonlinear inverse problem with quadratic $\mathcal{A}$ and a two-atom prior; details are provided in Section A.2 . We select $y$ such that one atom is likelihood-consistent while the other is likelihood-inconsistent but neighboring a likelihood-consistent prior-inconsistent mode. The exact posterior, DAPS, and PC-DAPS are shown as distributions across noise levels $\sigma$ . The exact posterior assigns positive weight to both atoms, with majority weight on the likelihood-consistent atom. However, DAPS assigns majority mass to the prior-inconsistent likelihood mode, as highlighted in the last panel, which is not present in the true posterior. PC-DAPS’s selection algorithm prunes this mode, sampling only from the posterior-consistent atom. While neither DAPS nor PC-DAPS samples from the likelihood-inconsistent atom with adequate probability, PC-DAPS eliminates the hallucinated prior-inconsistent mode produced by DAPS.

Task	Method	FFHQ			ImageNet
Task	Method	PSNR ( $\uparrow$ )	SSIM ( $\uparrow$ )	LPIPS ( $\downarrow$ )	PSNR ( $\uparrow$ )	SSIM ( $\uparrow$ )	LPIPS ( $\downarrow$ )
Super-resolution $4\times$	PC-DAPS	30.25	0.841	0.166	26.44	0.714	0.245
	DAPS	$\underline{29.36}$	$0.782$	$\underline{0.173}$	$\underline{25.51}$	$0.639$	$\underline{0.281}$
	DDNM	$28.01$	$0.797$	$0.196$	$23.92$	$0.606$	$0.472$
	DPS	$25.86$	$0.753$	$0.269$	$21.13$	$0.489$	$0.361$
	FPS-SMC	$28.45$	$\underline{0.812}$	$0.206$	$24.79$	$\underline{0.705}$	$0.311$

Table 1: Quantitative evaluation on FFHQ and ImageNet. Comparison of methods on FFHQ (left) and ImageNet (right) across seven linear and three nonlinear inverse problems (defined in Section A.6 ) using PSNR, SSIM, and LPIPS, evaluated over 100 test images. For the nonlinear problems, mean

\pm

standard deviation over 100 images is reported to highlight variation across samples. PC-DAPS produces 4 samples for each input, so the metrics reported for PC-DAPS are averaged over these 4 samples. For phase retrieval, for all methods other than PC-DAPS, we report oracle results from best-of-4 runs. We indicate the best and second-best results for each experiment with bold and underline annotations. We use a noisy measurement with noise level

\beta_{y}=0.05

in all tasks. Baseline implementation details are provided in Section A.6.1 .

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

ODE steps	Final Langevin	Mean reconstruction			Oracle selection
ODE steps	Final Langevin	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
5	Included	29.50	0.787	0.168	29.63	0.789	0.167
5	Dropped	30.01	0.831	0.144	30.14	0.833	0.143
1	Included	25.02	0.692	0.274	28.43	0.780	0.188
1	Dropped	25.41	0.739	0.254	28.93	0.831	0.164
$1{\to}5$	Included	28.11	0.774	0.181	28.34	0.778	0.179

Appendix

Table 3: Comparison of the ODE schedule and final Langevin refinement. Ablated over 10 randomly selected FFHQ images using the PC-DAPS-1K configuration in phase retrieval task.

Figure 5: Effect of the number of ODE steps on PC-DAPS. Increasing the number of ODE steps generally improves perceptual quality (LPIPS) but can reduce PSNR, with the effect depending on the inverse problem.

Figure 6: Effect of the number of annealing noise scheduling steps on PC-DAPS. Increasing the number of annealing steps has little effect on super-resolution and Gaussian deblurring, while phase retrieval shows substantial improvement with a finer annealing schedule.

Figure 7: Effect of the number of particles in a bag on phase retrieval. Increasing the bag size improves PC-DAPS reconstruction quality, with both PSNR and LPIPS improving rapidly up to

K=8

–

16

and then beginning to saturate. PC-DAPS is competitive with DAPS best-of-

k

at comparable values of

k

, while requiring less sampling time for larger

k

Task	Method	FFHQ			ImageNet
Task	Method	PSNR ( $\uparrow$ )	SSIM ( $\uparrow$ )	LPIPS ( $\downarrow$ )	PSNR ( $\uparrow$ )	SSIM ( $\uparrow$ )	LPIPS ( $\downarrow$ )
Phase retrieval	PC-DAPS	$\textbf{30.09}_{\pm 1.77}$	$\textbf{0.834}_{\pm 0.089}$	$\textbf{0.152}_{\pm 0.088}$	$\textbf{21.75}_{\pm 6.10}$	$\textbf{0.469}_{\pm 0.132}$	$\textbf{0.373}_{\pm 0.128}$
	DAPS	$25.11_{\pm 5.73}$	$0.666_{\pm 0.141}$	$0.283_{\pm 0.138}$	$15.70_{\pm 3.90}$	$0.317_{\pm 0.153}$	$0.516_{\pm 0.101}$
	DPS	$14.13_{\pm 2.72}$	$0.398_{\pm 0.106}$	$0.508_{\pm 0.085}$	$13.38_{\pm 4.03}$	$0.311_{\pm 0.208}$	$0.560_{\pm 0.123}$
	PnP-DM	$17.68_{\pm 5.50}$	$0.458_{\pm 0.191}$	$0.523_{\pm 0.149}$	$14.80_{\pm 5.60}$	$0.382_{\pm 0.170}$	$0.570_{\pm 0.110}$
	DPnP	$16.57_{\pm 6.76}$	$0.479_{\pm 0.213}$	$0.499_{\pm 0.193}$	$14.69_{\pm 3.61}$	$0.427_{\pm 0.180}$	$0.584_{\pm 0.101}$

Appendix

Table 4: Phase retrieval under a mean-of-

k

evaluation. All methods are evaluated using the mean performance across

k

reconstructions, rather than selecting the reconstruction closest to the ground truth. This exposes failures that best-of-

k

evaluation can mask, since individual reconstructions may be measurement-consistent but differ from the ground truth due to color inversion, rotation, or translation. PC-DAPS achieves higher reconstruction quality compared to other baselines.

Figure 8: Comparison of single input sampling time and reconstruction quality between DAPS and PC-DAPS. The

x

-axis shows the sampling time for a single input sample measured on an NVIDIA A100-SXM4-80GB GPU, while the

y

-axis reports LPIPS. Evaluation is performed on 100 FFHQ images. For phase retrieval, DAPS requires four independent runs for an input image; therefore, the reported DAPS sampling time corresponds to the total time of all four runs.

Figure 9: PC-DAPS samples for phase retrieval on FFHQ. Samples are shown without manual selection to better visualize success rate and sample quality.

Figure 10: PC-DAPS samples for phase retrieval on ImageNet. Samples are shown without manual selection to better visualize the success rate and sample quality. As discussed in Section 5 , ImageNet samples in a bag still contain failed (diverged/noisy) particles and underperform compared to FFHQ.

Figure 11: PC-DAPS samples for $16\times$ super-resolution. PC-DAPS is able to keep the diversity in the multimodal posterior distribution.

Figure 12: PC-DAPS samples for $190{\times}190$ box inpainting. PC-DAPS is able to keep the diversity in the multimodal posterior distribution.

Figure 13: PC-DAPS samples for various tasks on FFHQ. PC-DAPS obtains competitive reconstructions for linear and nonlinear tasks.

Figure 14: PC-DAPS samples for various tasks on ImageNet. PC-DAPS obtains competitive reconstructions for linear and nonlinear tasks.

Diffusion-Based Inverse Problems

Diffusion-based inverse problems use generative priors to recover images and physical fields from incomplete, noisy, or indirect measurements.

See all 145 papers

DAPS++: Rethinking Diffusion Inverse Problems with Decoupled Posterior Annealing
Nov 21, 2025Hao Chen, Renzheng Zhang, Scott S. HowardDiffusion Models for Image Restoration Bayesian Inverse Problems
University of Notre Dame, Notre Dame, IN 46556, USA
Provable diffusion-based posterior sampling for linear inverse problems via DDIM
Jul 21, 2026Yuchen Jiao, Na Li, Changxiao Cai +2Diffusion Models for Image Restoration Bayesian Inverse Problems
Department of Statistics and Data Science, Chinese University of Hong Kong, Hong Kong · Department of Industrial and Operations Engineering, University of Michigan, Ann Arbor, USA · Department of Statistics and Data Science, the Wharton School, University of Pennsylvania, PA, USA
Solving Inverse Problems via Diffusion-Based Priors: An Approximation-Free Ensemble Sampling Approach
Jun 4, 2025Haoxuan Chen, Yinuo Ren, Martin Renqiang Min +2Diffusion Posterior Sampling Bayesian Inverse Problems
ICME, Stanford University, Stanford, CA 94305 · Machine Learning Department, NEC Labs America, Princeton, NJ 08540 · Department of Mathematics and ICME, Stanford University, Stanford, CA 94305

Nov 21, 2025J/K move · Enter open · S save

DAPS++: Rethinking Diffusion Inverse Problems with Decoupled Posterior Annealing

Hao Chen, Renzheng Zhang, Scott S. Howard

University of Notre Dame, Notre Dame, IN 46556, USA

Diffusion Models for Image Restoration Bayesian Inverse Problems Diffusion-Based Inverse Problems Diffusion Posterior Sampling Diffusion Model Inference Acceleration Image Inverse Problems

Open paper AskPDF arXiv

From a Bayesian perspective, score-based diffusion solves inverse problems through joint inference, embedding the likelihood with the prior to guide the sampling process. However, this formulation fails to explain its practical behavior: the prior offers limited guidance, while reconstruction is largely driven by the measurement-consistency term, leading to an inference process that is effectively decoupled from the diffusion dynamics. We show that the diffusion prior in these solvers functions primarily as a warm initializer that places estimates near the data manifold, while reconstruction is driven almost entirely by measurement consistency. Based on this observation, we introduce DAPS++, which fully decouples diffusion-based initialization from likelihood-driven refinement, allowing the likelihood term to guide inference more directly while maintaining numerical stability and providing insight into why unified diffusion trajectories remain effective in practice. By requiring fewer function evaluations (NFEs) and measurement-optimization steps, DAPS++ achieves high computational efficiency and robust reconstruction performance across diverse image restoration tasks.

Abstract

Diffusion-Based Inverse Problems

Explore similar work

DAPS++: Rethinking Diffusion Inverse Problems with Decoupled Posterior Annealing

Provable diffusion-based posterior sampling for linear inverse problems via DDIM

Solving Inverse Problems via Diffusion-Based Priors: An Approximation-Free Ensemble Sampling Approach