Decoupled Data Consistency with Diffusion Purification for Image Restoration

Xiang Li; Soo Min Kwon; Shijun Liang; Ismail R. Alkhouri; Saiprasad Ravishankar; Qing Qu

eess.IVMar 10, 2024

Decoupled Data Consistency with Diffusion Purification for Image Restoration

Authors: Xiang Li, Soo Min Kwon, Shijun Liang, Ismail R. Alkhouri, Saiprasad Ravishankar, Qing Qu

Organizations: Department of Electrical Engineering and Computer Science, University of Michigan · Departments of Computational Mathematics, Science and Engineering, and Biomedical Engineering, Michigan State University

Abstract

Diffusion models have recently gained traction as a powerful class of deep generative priors, excelling in a wide range of image restoration tasks due to their exceptional ability to model data distributions. To solve image restoration problems, many existing techniques achieve data consistency by incorporating additional likelihood gradient steps into the reverse sampling process of diffusion models. However, the additional gradient steps pose a challenge for real-world practical applications as they incur a large computational overhead, thereby increasing inference time. They also present additional difficulties when using accelerated diffusion model samplers, as the number of data consistency steps is limited by the number of reverse sampling steps. In this work, we propose a novel diffusion-based image restoration solver that addresses these issues by decoupling the reverse process from the data consistency steps. Our method involves alternating between a reconstruction phase to maintain data consistency and a refinement phase that enforces the prior via diffusion purification. Our approach demonstrates versatility, making it highly adaptable for efficient problem-solving in latent space. Additionally, it reduces the necessity for numerous sampling steps through the integration of consistency models. The efficacy of our approach is validated through comprehensive experiments across various image restoration tasks, including image denoising, deblurring, inpainting, and super-resolution.

Figures & tables

Figure 1: Illustration of the difference between our proposed method and existing posterior sampling techniques for solving inverse problems. Our method decouples data consistency from the diffusion process, unlike existing approaches that enforce it via gradient steps during reverse sampling.

Figure 2: Demonstrating the regularization effects of diffusion purification. The estimates from the data fidelity step contain artifacts, making the overall reconstruction inaccurate, whereas regularization via diffusion purification can remove such artifacts.

Figure 3: Pixel space diffusion models for solving various linear inverse problems. These results demonstrate the effectiveness of our algorithm. We can consistently recover images with the highest PSNR.

Method	Super Resolution $4\times$			Inpainting (Box)
Method	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$
DPS [ 10 ]	$0.225\pm 0.05$	$25.37\pm 3.17$	$0.717\pm 0.09$	$0.155\pm 0.04$	$\underline{23.58}\pm 3.49$	$0.795\pm 0.09$
MCG [ 13 ]	$0.282\pm 0.11$	$18.94\pm 4.34$	$0.561\pm 0.15$	$0.159\pm 0.04$	$20.32\pm 3.54$	$0.789\pm 0.06$
ADMM-PnP [ 2 ]	$0.349\pm 0.04$	$21.56\pm 2.16$	$0.676\pm 0.07$	$0.232\pm 0.02$	$18.41\pm 2.09$	$0.728\pm 0.03$
DDNM [ 47 ]	$0.233\pm 0.06$	$26.00\pm 2.56$	$0.756\pm 0.08$	$0.105\pm 0.06$	$23.42\pm 3.45$	$0.882\pm 0.08$
RED-Diff [ 29 ]	$0.267\pm 0.05$	$25.78\pm 1.59$	$0.746\pm 0.07$	$0.115\pm 0.08$	$23.22\pm 3.16$	$0.878\pm 0.05$

Table 1: Quantitative results of super resolution and inpainting on the LSUN-Bedroom dataset for both pixel space diffusion models and LDMs. The results colored in blue and in orange are for LDMs and CMs respectively, while the others are for pixel space diffusion models. The best results are in bold and second best results are underlined for pixel space models and LDMs, respectively.

Method	Gaussian Deblurring			Motion Deblurring
Method	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$
DPS [ 10 ]	$0.213\pm 0.08$	$24.45\pm 3.72$	$0.691\pm 0.132$	$0.182\pm 0.03$	$24.45\pm 2.93$	$0.736\pm 0.06$
MCG [ 13 ]	$0.311\pm 0.13$	$17.54\pm 5.06$	$0.551\pm 0.191$	$0.365\pm 0.11$	$20.17\pm 3.73$	$0.515\pm 0.36$
ADMM-PnP [ 2 ]	$0.437\pm 0.04$	$20.76\pm 1.94$	$0.595\pm 0.09$	$0.524\pm 0.04$	$18.05\pm 2.05$	$0.493\pm 0.10$
DDNM [ 47 ]	$0.217\pm 0.07$	$\underline{27.30}\pm 1.67$	$\underline{0.815}\pm 0.08$	–	–	–
RED-Diff [ 29 ]	$0.228\pm 0.12$	$23.98\pm 1.23$	$0.687\pm 0.067$	$0.178\pm 0.06$	$24.15\pm 2.34$	$0.704\pm 0.12$

Table 2: Quantitative results of Gaussian and motion deblurring on the LSUN-Bedroom dataset for both pixel space diffusion models and LDMs. The results colored in blue and orange are for LDMs and CMs respectively, while the others are for pixel space diffusion models. The best results are in bold and second best results are underlined for pixel space models and LDMs respectively.

Figure 4: Demonstrating the differences between the DCDP methods on SR ( $4\times$ ). The two columns show the reconstructions with PSNRs and their corresponding Canny edges.

Figure 5: Qualitative results for various image restoration problems by using CMs. Our proposed framework is capable of utilizing CMs for image restoration.

Figure 6: Latent space diffusion models for solving various image restoration problems. These results demonstrate the effectiveness of our algorithm. We can consistently recover images with the highest PSNR.

Method	Nonlinear Deblurring
Method	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$
DPS [ 10 ]	$0.355\pm 0.09$	$22.60\pm 3.36$	$0.588\pm 0.16$
RED-Diff [ 29 ]	$\textbf{0.211}\pm 0.05$	$\textbf{29.51}\pm 0.76$	$\textbf{0.828}\pm 0.08$
DCDP-DDIM (Ours)	$0.323\pm 0.06$	$24.08\pm 2.42$	$0.634\pm 0.112$
DCDP-Tweedie (Ours)	$\underline{0.320}\pm 0.08$	$\underline{25.36}\pm 2.77$	$\underline{0.712}\pm 0.10$
Latent-DPS	$0.483\pm 0.08$	$21.77\pm 3.02$	$0.513\pm 0.16$

Table 3: Quantitative results of nonlinear deblurring on the ImageNet dataset for both pixel space diffusion models and LDMs. The results colored in blue are for LDMs, while the others are for pixel space diffusion models. The best results are in bold and second best results are underlined for pixel space models and LDMs respectively.

Figure 7: Comparison of inference times on Motion Deblurring. The experiment setting is the same as that introduced in Section 4.1 and all methods are run using the exact same compute. Here V1 and V2 stand for Approach I and Approach II introduced in Section 3.4 , respectively. The inference time for our algorithm is the fastest, while enjoying similar (or better) performance. The runtime is normalized by dividing by the largest value, which corresponds to DPS with a total runtime of 387 seconds.

Figure 8: Convergence of our methods in terms of LPIPS. The orange, green, red and blue curves correspond to the reconstruction process with data fidelity optimization only, DCDP with a constant (large) purification strength, constant (small) purification strength and a linearly decaying purification strength, respectively. The x-axis denotes the total number of data fidelity gradient steps. See Figure 21 in Appendix D for convergence behavior measured using PSNR and SSIM.

Figure 9: Reconstruction process for super-resolution (4x) in the pixel space. To compensate the artifacts introduced by the measurement noise, the reconstruction process takes $K=10$ iterations in total with a time schedule starting at $t_{1}=400$ and linearly decaying to $t_{10}=200$ . The learning rate for data fidelity optimization is $lr=100$ .

		Super Resolution ( $4\times$ )			Motion Deblurring
Metric		DPS	DCDP-DDIM	DCDP-Tweedie	DPS	DCDP-DDIM	DCDP-Tweedie
( $\sigma_{\bm{y}}=0.050$ )	LPIPS $\downarrow$	$\mathbf{0.287}\pm 0.03$	$\underline{0.321}\pm 0.05$	$0.363\pm 0.05$	$\mathbf{0.328}\pm 0.07$	$\underline{0.332}\pm 0.05$	$0.361\pm 0.05$
	PSNR $\uparrow$	$\underline{24.50}\pm 2.88$	$24.23\pm 2.45$	$\mathbf{24.66}\pm 2.55$	$22.43\pm 2.83$	$\underline{23.47}\pm 2.23$	$\mathbf{24.18}\pm 2.31$
	SSIM $\uparrow$	$\underline{0.697}\pm 0.08$	$0.687\pm 0.09$	$\mathbf{0.709}\pm 0.09$	$0.642\pm 0.09$	$\underline{0.669}\pm 0.09$	$\mathbf{0.703}\pm 0.09$
( $\sigma_{\bm{y}}=0.075$ )	LPIPS $\downarrow$	$\underline{0.342}\pm 0.04$	$\mathbf{0.325}\pm 0.05$	$0.363\pm 0.05$	$\underline{0.343}\pm 0.07$	$\mathbf{0.340}\pm 0.05$	$0.357\pm 0.05$
	PSNR $\uparrow$	$23.72\pm 2.59$	$\underline{24.12}\pm 2.41$	$\mathbf{24.62}\pm 2.54$	$22.09\pm 2.77$	$\underline{23.17}\pm 2.20$	$\mathbf{24.13}\pm 2.29$

Table 4: Case study of noise robustness of our methods compared to DPS on two image restoration problems in the pixel space. We compare our algorithm to DPS on super resolution (

4\times

) and motion deblurring to demonstrate that our method is effective even in the presence of measurement noise. Best results are bolded and second best results are underlined.

Figure 10: Impact of number of DDIM steps and total number of iterations $K$ . Figure (a) illustrates the performance of DCDP-DDIM with different numbers of DDIM steps for diffusion purification. Figure (b) illustrates the impact of the total number of iterations $K$ on the performance of DCDP-DDIM and DCDP-Tweedie respectively.

Figure 11: Impact of decaying schedules. (a) Different decaying schedules considered in our experiments. (b) Super-resolution performance of DCDP under different schedules, averaged over 10 randomly selected images.

Notation	Definition
$K\in\mathbb{N}$	Number of iterations for overall algorithm
$\tau\in\mathbb{N}$	Total number of iterations for data consistency optimization
$lr\in\mathbb{R}$	Learning rate for data consistency optimization
$t_{k}\in\mathbb{R}$	Diffusion purification strength at the $k^{\text{th}}$ iteration
$\bm{v}_{k}\in\mathbb{R}^{n}$	Prior consistent sample at $k^{\text{th}}$ iteration
$\bm{x}_{k}\in\mathbb{R}^{n}$	Data consistent sample at $k^{\text{th}}$ iteration

Table 5: Summary of the notation used throughout this work

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 12: Reconstruction process for super-resolution with pixel space diffusion models. The reconstruction process takes $K=10$ iterations in total with a time schedule starting at $t_{1}=400$ and linearly decaying to $t_{10}=0$ . We demonstrate reconstructions at timesteps 1, 5, and 10.

Figure 13: Reconstruction Process for Gaussian Deblurring with Pixel Space Diffusion Models. The reconstruction process takes $K=10$ iterations in total with a time schedule starting at $t_{1}=400$ and linearly decaying to $t_{10}=0$ . Due to the space limit, only reconstructions at timesteps 1, 5 and 10 are shown.

Figure 14: Reconstruction Process for Inpainting with Pixel Space Diffusion Models. The reconstruction process takes $K=20$ iterations in total with a time schedule starting at $t_{1}=700$ and linearly decaying to $t_{20}=0$ . Due to the space limit, only reconstructions at timesteps 1, 10 and 20 are shown.

Figure 15: Reconstruction Process for Motion Deblurring with Pixel Space Diffusion Models. The reconstruction process takes $K=20$ iterations in total with a time schedule starting at $t_{1}=400$ and linearly decaying to $t_{20}=0$ . Due to the space limit, only reconstructions at timesteps 1, 10 and 20 are shown.

Figure 16: Comparison between Pixel Space Optimization and Latent Space Optimization for Motion Deblurring. Initial reconstructions obtained via Pixel Space Optimization have much better quality compared to the ones obtained from Latent Space Optimization which in turn result in final reconstructions with higher quality.

Figure 17: Comparison between Pixel Space Optimization and Latent Space Optimization for Gaussian Deblurring. Initial reconstructions obtained via Pixel Space Optimization have much better quality compared to the ones obtained from Latent Space Optimization which in turn result in final reconstructions with higher quality.

Dataset	Super Resolution ( $4\times$ )					Gaussian Deblurring					Motion Deblurring					Inpainting (Box)
Dataset	$lr$	$K$	$\tau$	$t_{1}$	$t_{K}$	$lr$	$K$	$\tau$	$t_{1}$	$t_{K}$	$lr$	$K$	$\tau$	$t_{1}$	$t_{K}$	$lr$	$K$	$\tau$	$t_{1}$	$t_{K}$
LSUN-Bedroom	$10^{3}$	$10$	$100$	$400$	$0$	$10^{5}$	$10$	$100$	$400$	$0$	$10^{5}$	$20$	$50$	$400$	$0$	$10^{3}$	$20$	$50$	$700$	$0$
FFHQ	$10^{3}$	$10$	$100$	$400$	$0$	$10^{5}$	$10$	$100$	$400$	$0$	$10^{5}$	$20$	$50$	$400$	$0$	$10^{3}$	$20$	$50$	$700$	$0$

Appendix

Table 6: Hyperparameters of DCDP with pixel space diffusion models for solving linear inverse problems. For linear inverse problems, DCDP-Tweedie and DCDP-DDIM share the same set of hyperparameters.

Dataset	Super Resolution ( $4\times$ )					Gaussian Deblurring					Motion Deblurring					Inpainting (Box)
Dataset	$lr$	$K$	$\tau$	$t_{1}$	$t_{K}$	$lr$	$K$	$\tau$	$t_{1}$	$t_{K}$	$lr$	$K$	$\tau$	$t_{1}$	$t_{K}$	$lr$	$K$	$\tau$	$t_{1}$	$t_{K}$
LSUN-Bedroom	$10^{3}$	$10$	$100$	$400$	$0$	$10^{5}$	$10$	$100$	$400$	$0$	$10^{5}$	$10$	$100$	$400$	$0$	$10^{3}$	$20$	$50$	$500$	$0$
FFHQ	$10^{3}$	$10$	$100$	$400$	$0$	$10^{5}$	$4$	$250$	$400$	$0$	$10^{5}$	$10$	$100$	$400$	$0$	$10^{3}$	$20$	$50$	$500$	$0$

Appendix

Table 7: Hyperparameters of DCDP with latent space diffusion models for solving linear inverse problems. For linear inverse problems, DCDP-Tweedie and DCDP-DDIM share the same set of hyperparameters.

Dataset	DCDP-DDIM					DCDP-Tweedie					DCDP-LDM-DDIM					DCDP-LDM-Tweedie
Dataset	$lr$	$K$	$\tau$	$t_{1}$	$t_{K}$	$lr$	$K$	$\tau$	$t_{1}$	$t_{K}$	$lr$	$K$	$\tau$	$t_{1}$	$t_{K}$	$lr$	$K$	$\tau$	$t_{1}$	$t_{K}$
ImageNet	$10^{3}$	$20$	$100$	$400$	$100$	$10^{3}$	$20$	$100$	$400$	$50$	$5\times 10^{3}$	$20$	$100$	$300$	$50$	$5\times 10^{3}$	$20$	$100$	$300$	$50$

Appendix

Table 8: Hyperparameters of DCDP for solving nonlinear deblurring.

Dataset	Super Resolution ( $4\times$ )					Gaussian Deblurring					Motion Deblurring					Inpainting (Box)
Dataset	$lr$	$K$	$\tau$	$t_{1}$	$t_{K}$	$lr$	$K$	$\tau$	$t_{1}$	$t_{K}$	$lr$	$K$	$\tau$	$t_{1}$	$t_{K}$	$lr$	$K$	$\tau$	$t_{1}$	$t_{K}$
LSUN-Bedroom	$10^{3}$	$20$	$50$	$1$	$0$	$10^{5}$	$20$	$50$	$1$	$0$	$10^{5}$	$20$	$50$	$1$	$0$	$10^{3}$	$20$	$50$	$5$	$0$

Appendix

Table 9: Hyperparameters of DCDP with CMs. Different from the diffusion models in previous experiments which specify the noise level via discrete forward steps (ranging from 0 to 1,000), CMs are trained on a continuous range of Gaussian noise with standard deviation spanning [0.002,80]. Therefore,

t_{1}

and

t_{K}

in this table stand for the standard deviation of the noise rather than the timestep.

Method	Super Resolution $4\times$			Inpainting (Box)
Method	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$
DPS [ 10 ]	$\underline{0.176}$ $\pm 0.04$	$28.45\pm 1.99$	$0.792\pm 0.05$	$0.115\pm 0.02$	$23.14\pm 3.14$	$0.829\pm 0.03$
MCG [ 13 ]	$0.182\pm 0.03$	$24.45\pm 2.93$	$0.736\pm 0.06$	$0.119\pm 0.02$	$22.77\pm 3.06$	$0.827\pm 0.03$
ADMM-PnP [ 2 ]	$0.307\pm 0.03$	$21.54\pm 3.12$	$0.766\pm 0.03$	$0.251\pm 0.02$	$18.26\pm 1.86$	$0.724\pm 0.02$
DDNM [ 47 ]	$0.197\pm 0.04$	$28.03\pm 1.46$	$0.795\pm 0.08$	$0.235\pm 0.06$	$24.47\pm 2.38$	$0.837\pm 0.08$
RED-Diff [ 29 ]	$0.212\pm 0.06$	$27.18\pm 1.23$	$0.781\pm 0.07$	$0.165\pm 0.08$	$24.22\pm 3.16$	$0.824\pm 0.05$

Appendix

Table 10: Quantitative results of super resolution and inpainting on the FFHQ dataset for both pixel-based diffusion models and LDMs. The results colored in blue are for LDMs, while the others are for pixel space diffusion models. The best results are in bold and second best results are underlined for pixel space models and LDMs respectively.

Method	Gaussian Deblurring			Motion Deblurring
Method	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$
DPS [ 10 ]	$\underline{0.162}\pm 0.03$	$27.97\pm 2.01$	$0.782\pm 0.05$	$0.219\pm 0.04$	$25.34\pm 2.21$	$0.690\pm 0.07$
MCG [ 13 ]	$0.164\pm 0.03$	$25.03\pm 3.62$	$0.790\pm 0.06$	$0.343\pm 0.04$	$24.88\pm 2.01$	$0.708\pm 0.05$
DDNM [ 47 ]	$0.217\pm 0.04$	$28.23\pm 1.69$	$0.805\pm 0.08$	–	–	–
RED-Diff [ 29 ]	$0.242\pm 0.06$	$25.25\pm 2.17$	$0.791\pm 0.05$	$0.268\pm 0.08$	$24.32\pm 2.67$	$0.754\pm 0.05$
DiffPIR [ 57 ]	$0.236\pm 0.06$	$27.34\pm 2.15$	$0.778\pm 0.04$	$0.255\pm 0.07$	$26.69\pm 1.49$	$0.856\pm 0.06$

Appendix

Table 11: Quantitative results of Gaussian and motion deblurring on the FFHQ dataset for both pixel-based diffusion models and LDMs. The results colored in blue are for LDMs, while the others are for pixel space diffusion models. The best results are in bold and second best results are underlined for pixel space models and LDMs respectively.

Figure 21: Convergence of our methods in terms of PSNR, SSIM and LPIPS for solving nonlinear deblurring. DCDP greatly improves upon the pure data fidelity optimization by leveraging the image priors embedded in the diffusion models.

Diffusion Models for Image Restoration

Diffusion models for image restoration use generative priors to denoise, deblur, upscale, and reconstruct images from incomplete or corrupted measurements.

See all 189 papers

Residual Diffusion Implicit Models
Sep 26, 2026João Guerreiro, Pedro Tomás, Helena Aidos +1Diffusion Models for Image Restoration Diffusion-Based Inverse Problems
INESC-ID, Instituto Superior Técnico, Universidade de Lisboa, Lisboa, Portugal · ISR-Lisboa, Instituto Superior Técnico, Universidade de Lisboa, Lisboa, Portugal · LASIGE, Faculdade de Ciências, Universidade de Lisboa, Lisboa, Portugal
Local Epistemic Uncertainty Guided Active Sampling for Plug-and-play Diffusive Image Restoration
Aug 7, 2026Jiaqi Zhang, Zheng Pang, Rongrong Gao +2Diffusion Models for Image Restoration Diffusion Model Guidance
Jiangsu University, Zhenjiang, China · Xi’an University of Technology, Xi’an, China
Image Restoration via Diffusion Models with Dynamic Resolution
May 14, 2026Yang Zheng, Wen Li, Zhaoqiang LiuDiffusion Models for Image Restoration Diffusion-Based Inverse Problems
School of Computer Science and Engineering, University of Electronic Science and Technology of China.

Sep 26, 2026J/K move · Enter open · S save

Residual Diffusion Implicit Models

João Guerreiro, Pedro Tomás, Helena Aidos, Jacinto C. Nascimento

INESC-ID, Instituto Superior Técnico, Universidade de Lisboa, Lisboa, Portugal · ISR-Lisboa, Instituto Superior Técnico, Universidade de Lisboa, Lisboa, Portugal · LASIGE, Faculdade de Ciências, Universidade de Lisboa, Lisboa, Portugal

Diffusion Models for Image Restoration Diffusion-Based Inverse Problems Diffusion Models Image Denoising Diffusion Model Inference Acceleration Image Super-Resolution

Open paper AskPDF arXiv

Diffusion models achieve state-of-the-art results across multiple tasks. However, in inverse problems, standard initialization from pure Gaussian noise misaligns the generative process with real-world degradations. More recent methods such as diffusion bridges impose strict endpoint constraints and often require long reverse processes that are prone to hallucinations. Alternative consistency models provide noise-invariant, one-step mappings but lack inherent variance modeling and can degrade under severe corruption. Hence, residual diffusion implicit models (RDIMs) are proposed, constituting a generalized framework that explicitly models the residuals between high-quality (HQ) and low-quality (LQ) images, aligning the forward process with the actual degradation. A non-Markovian implicit reverse sampler is derived, which can skip intermediate timesteps, enabling accurate few-step or even single-step reconstruction, while mitigating the hallucinations inherent to long diffusion chains. RDIM also introduces a controllable variance mechanism that interpolates between deterministic and stochastic sampling, balancing fidelity and diversity. Furthermore, it enables the straightforward use of perceptual losses, when needed. Experiments on denoising and super-resolution benchmarks demonstrate that RDIMs consistently outperforms the state of the art, including bridge and consistency models, in terms of PSNR, SSIM, and LPIPS, reducing hallucinations while requiring only a few sampling steps (often just one). The results position RDIMs as an efficient solution for a broad range of image restoration tasks.

Abstract

Diffusion Models for Image Restoration

Explore similar work

Residual Diffusion Implicit Models

Local Epistemic Uncertainty Guided Active Sampling for Plug-and-play Diffusive Image Restoration

Image Restoration via Diffusion Models with Dynamic Resolution