AdaptivePnP: Fast Auto-Tuned Plug-and-Play for Deconvolution
Authors: Haley Duba-Sullivan, Emma J. Reid, Charles A. Bouman, Gregery T. Buzzard
Organizations: Department of Mathematics, Purdue University, West Lafayette, IN 47907, USA · National Security Sciences Directorate, Oak Ridge National Laboratory, Oak Ridge, TN 37830, USA · School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 47907, USA
Plug-and-play (PnP) methods reconstruct images by alternating a data-fitting step with an image denoiser. Their performance depends on two parameters: the step size γ and the measurement noise level σ. In practice, γ is often tuned by grid search over validation images and must be retuned whenever the problem setting changes, and σ is assumed known or estimated separately. We propose AdaptivePnP, a PnP framework that jointly estimates σ and γ at every iteration from the data-fitting residual. To do so, we apply the PnP virtual model to the data-fitting proximal map and show that the residual is zero-mean Gaussian with covariance parameterized by γ and σ. However, when one parameter is much larger than the other, the residual carries little information about the smaller one and using the maximum likelihood (ML) estimator can degrade the PnP iterations. We therefore introduce the identifiable maximum likelihood (IML) estimator, which restricts the ML estimator to an identifiable window in which the normalized Cramér--Rao bound of every parameter is bounded by a user-specified tolerance. For the virtual model, this window is an interval of ratios ρ=γ/σ, so computing the IML estimate reduces to a one-dimensional search over ρ. We apply AdaptivePnP to two standard PnP methods, yielding algorithms that require neither a tuned γ nor a known σ. On image deblurring and super-resolution, AdaptivePnP matches or outperforms PnP baselines tuned with the true σ. Code is available at https://github.com/hdubasullivan/AdaptivePnP.
Figures & tables
Fig. 1: Overview of the proposed AdaptivePnP framework. Standard PnP methods alternate a data-fitting proximal map with an image denoiser (gray boxes); AdaptivePnP adds a parameter estimation step in each iteration (blue boxes). Namely, AdaptivePnP applies the identifiable maximum likelihood (IML) estimator to the data-fitting residual, resulting in a joint estimate of the noise level σ and the step size γ . The estimates are fed back to both the denoiser and the data-fitting proximal map, so neither a tuned step size nor a known noise level is required.
Figure 2
Fig. 2: The twelve 25×25 blur kernels used in experiments.
Isotropic Gaussian
Anisotropic Gaussian
Motion
Tuned PnP
AdaptivePnP
Tuned PnP
AdaptivePnP
Tuned PnP
AdaptivePnP
Deblurring
PGM
27.83 / 0.768
28.39 / 0.807
25.49 / 0.701
25.83 / 0.719
26.64 / 0.719
28.01 / 0.777
ADMM
28.27 / 0.793
28.32 / 0.808
25.77 / 0.722
25.79 / 0.719
28.02 / 0.800
27.96 / 0.782
2 × Super-Resolution
PGM
24.07 / 0.603
26.69 / 0.737
24.83 / 0.634
25.23 / 0.675
22.24 / 0.470
25.47 / 0.650
ADMM
23.88 / 0.623
26.55 / 0.737
25.13 / 0.671
25.28 / 0.685
23.03 / 0.578
25.79 / 0.701
TABLE I: Mean PSNR / SSIM of AdaptivePnP and tuned PnP reconstructions, averaged over the CBSD68 dataset with four kernels in each family (G1–G4, A1–A4, M1–M4) and three noise levels (1%, 3%, and 5%). Tuned PnP uses the true σ and a step size selected by grid search, whereas AdaptivePnP uses neither.The best result within each algorithm and kernel family is bolded.
Fig. 3: Mean reconstruction PSNR and ratio ρk=γk/σk of AdaptivePnP and tuned PnP over 10 iterations for (a) deblurring and (b) 2× SR. Results are averaged over the CBSD68 dataset with all 12 kernels and three noise levels. AdaptivePnP rapidly adapts ρk and reaches its final reconstruction quality within the first few iterations without requiring a validation-tuned parameter choice.
Fig. 4: Deblurring reconstructions using AdaptivePnP and tuned PnP; rows show an isotropic Gaussian kernel (G4) with 3% measurement noise, an anisotropic Gaussian kernel (A1) with 1% noise, and a motion blur kernel (M1) with 5% noise. The ground truth image and blurred noisy measurement are shown for reference, and the highlighted regions show magnified image details. AdaptivePnP produces visual quality comparable to or better than the tuned PnP baselines despite estimating both σ and γ directly from the reconstruction.
Fig. 5: 2× SR reconstructions using AdaptivePnP and tuned PnP; rows show an isotropic Gaussian kernel (G3) with 1% measurement noise, an anisotropic Gaussian kernel (A3) with 3% noise, and a motion blur kernel (M3) with 5% noise. The ground truth image and low-resolution noisy measurement are shown for reference, and the highlighted regions show magnified image details. AdaptivePnP recovers visibly sharper image detail than the tuned PnP baselines, consistent with the PSNR and SSIM improvements in Table I .
Fig. 6: Mean PSNR of AdaptivePnP after 10 iterations using varying tolerances τ for (a) deblurring and (b) 2× SR. Results are averaged over the first 20 images from CBSD68 with all 12 kernels and three noise levels. Reconstruction quality is insensitive to the precise choice of the tolerance once it is sufficiently large; we use τ=0.05 (dashed line) for all experiments.
Fig. 7: Identifiable windows Wτ at τ=0.05 for all kernels under (a) deblurring and (b) 2× SR. Horizontal intervals show [ρinf,ρsup] , with arrows indicating ρsup>106 . All twelve windows are nonempty in both settings.
Fig. 8: Mean PSNR of AdaptivePnP over 10 iterations with the proposed IML estimator and the unconstrained ML estimator for (a) deblurring and (b) 2× SR. Results are averaged over the first 20 images from CBSD68 with all 12 kernels and three noise levels. At each iteration, the mean PSNR excludes any runs that have diverged to a non-finite PSNR, which occurs only for the ML estimator. With IML, both algorithms converge within a few iterations, while using ML degrades quality over iterations.
Fig. 9: Mean PSNR of AdaptivePnP over 10 iterations with estimated σk and known σ for (a) deblurring and (b) 2× SR. Results are averaged over the first 20 images of CBSD68 with all 12 kernels and three noise levels; γ is estimated by the IML estimator in both settings. AdaptivePnP removes the need for a known σ at no cost in reconstruction quality.
Plug-and-Play (PnP) methods for imaging inverse problems replace the proximal operator of the prior, a maximum a posteriori (MAP) denoiser, with a pretrained minimum mean squared error (MMSE) denoiser. MMSE estimates are oversmoothed, motivating recent work to target the MAP with diffusion-model scores. We show that MAP-targeting iterations converge to cartoon-like images with learned scores, but to realistic ones with the exact scores of a Gaussian mixture. The failure thus seems to come from score errors, and it only appears near convergence. We therefore propose ProxiMAP, an iterative MAP approximation that queries the denoiser at the residual noise level of the current iterate. This keeps the denoiser in-distribution and slows the iterations down before they reach the cartoon regime. On deblurring, super-resolution, inpainting, HDR and phase retrieval, replacing the MMSE denoiser with ProxiMAP in DPIR, DiffPIR and DAPS improves perceptual quality in most experiments, sometimes at the expense of PSNR. To make ProxiMAP cheaper, we also propose a one-step distillation and a hybrid schedule that applies it only at the last PnP iteration. In most cases, one or the other retains much of the improvement at a fraction of the cost.
Kenta Vert, Giacomo Meanti, Scott Pesme +2
Univ. Grenoble Alpes, Inria, CNRS, Grenoble INP, LJK · MaLGa Centre, DIBRIS, Università di Genova, MMS, Italian Institute of Technology, Genova, Italy
Estimating a camera pose from 3D-2D correspondences - Perspective-n-Point - is one of the most fundamental problems in computer vision. Existing methods either solve a minimal problem or replace the objective with an easier one. Surprisingly, no scalable certifiable solver for the actual Maximum Likelihood Estimation (MLE) exists. We propose PnP*, the first practical solver that guarantees efficient computation and returns a certifiable globally optimal solution. We prove that the underlying convex relaxation is exact under moderate noise, which to our knowledge is the first stability guarantee for any relaxation for the MLE of PnP. Furthermore, we reduce the relaxation by exploiting chordal sparsity pattern and quotient ring reduction such that the relaxation is tractable and scalable. The stability and scalability results are verified numerically, as PnP* returns a provably correct camera pose at over 10 Hz.
Bang-Shien Chen, Caitrin E. Eaton, David M. Rosen
Northeastern University, Boston, MA 02115, USA. · Draper, Cambridge, MA 02139, USA.
Plug-and-Play (PnP) methods solve imaging inverse problems by incorporating deep denoisers into iterative optimization algorithms. Although practical implementations often decrease the denoiser noise level σ along iterations, most existing convergence analyses assume a fixed denoiser. In this work, we establish convergence guarantees for a broad family of Plug-and-Play algorithms with annealed noise level, spanning deterministic methods (RED--GD and PnP--PGD) and stochastic methods (SNORE, equivariant RED, and a variant of PnP--Flow). For each method, we identify an explicit, nonconvex objective associated with the terminal denoising level and prove asymptotic stationarity of the iterates with respect to this objective. Our analysis does not prescribe any decay rate for the noise schedule, and our assumptions cover both learned gradient-step denoisers and exact MMSE denoisers. Overall, our theoretical results bridge the gap between existing PnP convergence theory and the decreasing-denoising practices used by state-of-the-art image restoration methods. We empirically demonstrate the benefits of such schedules and illustrate the predicted convergence behavior on several imaging inverse problems, including inpainting, super-resolution, demosaicing and tomography.