Decoupled Data Consistency with Diffusion Purification for Image Restoration

Xiang Li; Soo Min Kwon; Shijun Liang; Ismail R. Alkhouri; Saiprasad Ravishankar; Qing Qu

eess.IVMar 10, 2024

Decoupled Data Consistency with Diffusion Purification for Image Restoration

Authors: Xiang Li, Soo Min Kwon, Shijun Liang, Ismail R. Alkhouri, Saiprasad Ravishankar, Qing Qu

Organizations: Department of Electrical Engineering and Computer Science, University of Michigan · Departments of Computational Mathematics, Science and Engineering, and Biomedical Engineering, Michigan State University

Abstract

Diffusion models have recently gained traction as a powerful class of deep generative priors, excelling in a wide range of image restoration tasks due to their exceptional ability to model data distributions. To solve image restoration problems, many existing techniques achieve data consistency by incorporating additional likelihood gradient steps into the reverse sampling process of diffusion models. However, the additional gradient steps pose a challenge for real-world practical applications as they incur a large computational overhead, thereby increasing inference time. They also present additional difficulties when using accelerated diffusion model samplers, as the number of data consistency steps is limited by the number of reverse sampling steps. In this work, we propose a novel diffusion-based image restoration solver that addresses these issues by decoupling the reverse process from the data consistency steps. Our method involves alternating between a reconstruction phase to maintain data consistency and a refinement phase that enforces the prior via diffusion purification. Our approach demonstrates versatility, making it highly adaptable for efficient problem-solving in latent space. Additionally, it reduces the necessity for numerous sampling steps through the integration of consistency models. The efficacy of our approach is validated through comprehensive experiments across various image restoration tasks, including image denoising, deblurring, inpainting, and super-resolution.

Figures & tables

Figure 1: Illustration of the difference between our proposed method and existing posterior sampling techniques for solving inverse problems. Our method decouples data consistency from the diffusion process, unlike existing approaches that enforce it via gradient steps during reverse sampling.

Figure 2: Demonstrating the regularization effects of diffusion purification. The estimates from the data fidelity step contain artifacts, making the overall reconstruction inaccurate, whereas regularization via diffusion purification can remove such artifacts.

Figure 3: Pixel space diffusion models for solving various linear inverse problems. These results demonstrate the effectiveness of our algorithm. We can consistently recover images with the highest PSNR.

Method	Super Resolution $4\times$			Inpainting (Box)
Method	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$
DPS [ 10 ]	$0.225\pm 0.05$	$25.37\pm 3.17$	$0.717\pm 0.09$	$0.155\pm 0.04$	$\underline{23.58}\pm 3.49$	$0.795\pm 0.09$
MCG [ 13 ]	$0.282\pm 0.11$	$18.94\pm 4.34$	$0.561\pm 0.15$	$0.159\pm 0.04$	$20.32\pm 3.54$	$0.789\pm 0.06$
ADMM-PnP [ 2 ]	$0.349\pm 0.04$	$21.56\pm 2.16$	$0.676\pm 0.07$	$0.232\pm 0.02$	$18.41\pm 2.09$	$0.728\pm 0.03$
DDNM [ 47 ]	$0.233\pm 0.06$	$26.00\pm 2.56$	$0.756\pm 0.08$	$0.105\pm 0.06$	$23.42\pm 3.45$	$0.882\pm 0.08$
RED-Diff [ 29 ]	$0.267\pm 0.05$	$25.78\pm 1.59$	$0.746\pm 0.07$	$0.115\pm 0.08$	$23.22\pm 3.16$	$0.878\pm 0.05$

Table 1: Quantitative results of super resolution and inpainting on the LSUN-Bedroom dataset for both pixel space diffusion models and LDMs. The results colored in blue and in orange are for LDMs and CMs respectively, while the others are for pixel space diffusion models. The best results are in bold and second best results are underlined for pixel space models and LDMs, respectively.

Method	Gaussian Deblurring			Motion Deblurring
Method	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$
DPS [ 10 ]	$0.213\pm 0.08$	$24.45\pm 3.72$	$0.691\pm 0.132$	$0.182\pm 0.03$	$24.45\pm 2.93$	$0.736\pm 0.06$
MCG [ 13 ]	$0.311\pm 0.13$	$17.54\pm 5.06$	$0.551\pm 0.191$	$0.365\pm 0.11$	$20.17\pm 3.73$	$0.515\pm 0.36$
ADMM-PnP [ 2 ]	$0.437\pm 0.04$	$20.76\pm 1.94$	$0.595\pm 0.09$	$0.524\pm 0.04$	$18.05\pm 2.05$	$0.493\pm 0.10$
DDNM [ 47 ]	$0.217\pm 0.07$	$\underline{27.30}\pm 1.67$	$\underline{0.815}\pm 0.08$	–	–	–
RED-Diff [ 29 ]	$0.228\pm 0.12$	$23.98\pm 1.23$	$0.687\pm 0.067$	$0.178\pm 0.06$	$24.15\pm 2.34$	$0.704\pm 0.12$

Table 2: Quantitative results of Gaussian and motion deblurring on the LSUN-Bedroom dataset for both pixel space diffusion models and LDMs. The results colored in blue and orange are for LDMs and CMs respectively, while the others are for pixel space diffusion models. The best results are in bold and second best results are underlined for pixel space models and LDMs respectively.

Figure 4: Demonstrating the differences between the DCDP methods on SR ( $4\times$ ). The two columns show the reconstructions with PSNRs and their corresponding Canny edges.

Figure 5: Qualitative results for various image restoration problems by using CMs. Our proposed framework is capable of utilizing CMs for image restoration.

Figure 6: Latent space diffusion models for solving various image restoration problems. These results demonstrate the effectiveness of our algorithm. We can consistently recover images with the highest PSNR.

Method	Nonlinear Deblurring
Method	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$
DPS [ 10 ]	$0.355\pm 0.09$	$22.60\pm 3.36$	$0.588\pm 0.16$
RED-Diff [ 29 ]	$\textbf{0.211}\pm 0.05$	$\textbf{29.51}\pm 0.76$	$\textbf{0.828}\pm 0.08$
DCDP-DDIM (Ours)	$0.323\pm 0.06$	$24.08\pm 2.42$	$0.634\pm 0.112$
DCDP-Tweedie (Ours)	$\underline{0.320}\pm 0.08$	$\underline{25.36}\pm 2.77$	$\underline{0.712}\pm 0.10$
Latent-DPS	$0.483\pm 0.08$	$21.77\pm 3.02$	$0.513\pm 0.16$

Table 3: Quantitative results of nonlinear deblurring on the ImageNet dataset for both pixel space diffusion models and LDMs. The results colored in blue are for LDMs, while the others are for pixel space diffusion models. The best results are in bold and second best results are underlined for pixel space models and LDMs respectively.

Figure 7: Comparison of inference times on Motion Deblurring. The experiment setting is the same as that introduced in Section 4.1 and all methods are run using the exact same compute. Here V1 and V2 stand for Approach I and Approach II introduced in Section 3.4 , respectively. The inference time for our algorithm is the fastest, while enjoying similar (or better) performance. The runtime is normalized by dividing by the largest value, which corresponds to DPS with a total runtime of 387 seconds.

Figure 8: Convergence of our methods in terms of LPIPS. The orange, green, red and blue curves correspond to the reconstruction process with data fidelity optimization only, DCDP with a constant (large) purification strength, constant (small) purification strength and a linearly decaying purification strength, respectively. The x-axis denotes the total number of data fidelity gradient steps. See Figure 21 in Appendix D for convergence behavior measured using PSNR and SSIM.

Figure 9: Reconstruction process for super-resolution (4x) in the pixel space. To compensate the artifacts introduced by the measurement noise, the reconstruction process takes $K=10$ iterations in total with a time schedule starting at $t_{1}=400$ and linearly decaying to $t_{10}=200$ . The learning rate for data fidelity optimization is $lr=100$ .

		Super Resolution ( $4\times$ )			Motion Deblurring
Metric		DPS	DCDP-DDIM	DCDP-Tweedie	DPS	DCDP-DDIM	DCDP-Tweedie
( $\sigma_{\bm{y}}=0.050$ )	LPIPS $\downarrow$	$\mathbf{0.287}\pm 0.03$	$\underline{0.321}\pm 0.05$	$0.363\pm 0.05$	$\mathbf{0.328}\pm 0.07$	$\underline{0.332}\pm 0.05$	$0.361\pm 0.05$
	PSNR $\uparrow$	$\underline{24.50}\pm 2.88$	$24.23\pm 2.45$	$\mathbf{24.66}\pm 2.55$	$22.43\pm 2.83$	$\underline{23.47}\pm 2.23$	$\mathbf{24.18}\pm 2.31$
	SSIM $\uparrow$	$\underline{0.697}\pm 0.08$	$0.687\pm 0.09$	$\mathbf{0.709}\pm 0.09$	$0.642\pm 0.09$	$\underline{0.669}\pm 0.09$	$\mathbf{0.703}\pm 0.09$
( $\sigma_{\bm{y}}=0.075$ )	LPIPS $\downarrow$	$\underline{0.342}\pm 0.04$	$\mathbf{0.325}\pm 0.05$	$0.363\pm 0.05$	$\underline{0.343}\pm 0.07$	$\mathbf{0.340}\pm 0.05$	$0.357\pm 0.05$
	PSNR $\uparrow$	$23.72\pm 2.59$	$\underline{24.12}\pm 2.41$	$\mathbf{24.62}\pm 2.54$	$22.09\pm 2.77$	$\underline{23.17}\pm 2.20$	$\mathbf{24.13}\pm 2.29$

Table 4: Case study of noise robustness of our methods compared to DPS on two image restoration problems in the pixel space. We compare our algorithm to DPS on super resolution (

4\times

) and motion deblurring to demonstrate that our method is effective even in the presence of measurement noise. Best results are bolded and second best results are underlined.

Figure 10: Impact of number of DDIM steps and total number of iterations $K$ . Figure (a) illustrates the performance of DCDP-DDIM with different numbers of DDIM steps for diffusion purification. Figure (b) illustrates the impact of the total number of iterations $K$ on the performance of DCDP-DDIM and DCDP-Tweedie respectively.

Figure 11: Impact of decaying schedules. (a) Different decaying schedules considered in our experiments. (b) Super-resolution performance of DCDP under different schedules, averaged over 10 randomly selected images.

Notation	Definition
$K\in\mathbb{N}$	Number of iterations for overall algorithm
$\tau\in\mathbb{N}$	Total number of iterations for data consistency optimization
$lr\in\mathbb{R}$	Learning rate for data consistency optimization
$t_{k}\in\mathbb{R}$	Diffusion purification strength at the $k^{\text{th}}$ iteration
$\bm{v}_{k}\in\mathbb{R}^{n}$	Prior consistent sample at $k^{\text{th}}$ iteration
$\bm{x}_{k}\in\mathbb{R}^{n}$	Data consistent sample at $k^{\text{th}}$ iteration

Table 5: Summary of the notation used throughout this work

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 12: Reconstruction process for super-resolution with pixel space diffusion models. The reconstruction process takes $K=10$ iterations in total with a time schedule starting at $t_{1}=400$ and linearly decaying to $t_{10}=0$ . We demonstrate reconstructions at timesteps 1, 5, and 10.

Figure 13: Reconstruction Process for Gaussian Deblurring with Pixel Space Diffusion Models. The reconstruction process takes $K=10$ iterations in total with a time schedule starting at $t_{1}=400$ and linearly decaying to $t_{10}=0$ . Due to the space limit, only reconstructions at timesteps 1, 5 and 10 are shown.

Figure 14: Reconstruction Process for Inpainting with Pixel Space Diffusion Models. The reconstruction process takes $K=20$ iterations in total with a time schedule starting at $t_{1}=700$ and linearly decaying to $t_{20}=0$ . Due to the space limit, only reconstructions at timesteps 1, 10 and 20 are shown.

Figure 15: Reconstruction Process for Motion Deblurring with Pixel Space Diffusion Models. The reconstruction process takes $K=20$ iterations in total with a time schedule starting at $t_{1}=400$ and linearly decaying to $t_{20}=0$ . Due to the space limit, only reconstructions at timesteps 1, 10 and 20 are shown.

Figure 16: Comparison between Pixel Space Optimization and Latent Space Optimization for Motion Deblurring. Initial reconstructions obtained via Pixel Space Optimization have much better quality compared to the ones obtained from Latent Space Optimization which in turn result in final reconstructions with higher quality.

Figure 17: Comparison between Pixel Space Optimization and Latent Space Optimization for Gaussian Deblurring. Initial reconstructions obtained via Pixel Space Optimization have much better quality compared to the ones obtained from Latent Space Optimization which in turn result in final reconstructions with higher quality.

Dataset	Super Resolution ( $4\times$ )					Gaussian Deblurring					Motion Deblurring					Inpainting (Box)
Dataset	$lr$	$K$	$\tau$	$t_{1}$	$t_{K}$	$lr$	$K$	$\tau$	$t_{1}$	$t_{K}$	$lr$	$K$	$\tau$	$t_{1}$	$t_{K}$	$lr$	$K$	$\tau$	$t_{1}$	$t_{K}$
LSUN-Bedroom	$10^{3}$	$10$	$100$	$400$	$0$	$10^{5}$	$10$	$100$	$400$	$0$	$10^{5}$	$20$	$50$	$400$	$0$	$10^{3}$	$20$	$50$	$700$	$0$
FFHQ	$10^{3}$	$10$	$100$	$400$	$0$	$10^{5}$	$10$	$100$	$400$	$0$	$10^{5}$	$20$	$50$	$400$	$0$	$10^{3}$	$20$	$50$	$700$	$0$

Appendix

Table 6: Hyperparameters of DCDP with pixel space diffusion models for solving linear inverse problems. For linear inverse problems, DCDP-Tweedie and DCDP-DDIM share the same set of hyperparameters.

Dataset	Super Resolution ( $4\times$ )					Gaussian Deblurring					Motion Deblurring					Inpainting (Box)
Dataset	$lr$	$K$	$\tau$	$t_{1}$	$t_{K}$	$lr$	$K$	$\tau$	$t_{1}$	$t_{K}$	$lr$	$K$	$\tau$	$t_{1}$	$t_{K}$	$lr$	$K$	$\tau$	$t_{1}$	$t_{K}$
LSUN-Bedroom	$10^{3}$	$10$	$100$	$400$	$0$	$10^{5}$	$10$	$100$	$400$	$0$	$10^{5}$	$10$	$100$	$400$	$0$	$10^{3}$	$20$	$50$	$500$	$0$
FFHQ	$10^{3}$	$10$	$100$	$400$	$0$	$10^{5}$	$4$	$250$	$400$	$0$	$10^{5}$	$10$	$100$	$400$	$0$	$10^{3}$	$20$	$50$	$500$	$0$

Appendix

Table 7: Hyperparameters of DCDP with latent space diffusion models for solving linear inverse problems. For linear inverse problems, DCDP-Tweedie and DCDP-DDIM share the same set of hyperparameters.

Dataset	DCDP-DDIM					DCDP-Tweedie					DCDP-LDM-DDIM					DCDP-LDM-Tweedie
Dataset	$lr$	$K$	$\tau$	$t_{1}$	$t_{K}$	$lr$	$K$	$\tau$	$t_{1}$	$t_{K}$	$lr$	$K$	$\tau$	$t_{1}$	$t_{K}$	$lr$	$K$	$\tau$	$t_{1}$	$t_{K}$
ImageNet	$10^{3}$	$20$	$100$	$400$	$100$	$10^{3}$	$20$	$100$	$400$	$50$	$5\times 10^{3}$	$20$	$100$	$300$	$50$	$5\times 10^{3}$	$20$	$100$	$300$	$50$

Appendix

Table 8: Hyperparameters of DCDP for solving nonlinear deblurring.

Dataset	Super Resolution ( $4\times$ )					Gaussian Deblurring					Motion Deblurring					Inpainting (Box)
Dataset	$lr$	$K$	$\tau$	$t_{1}$	$t_{K}$	$lr$	$K$	$\tau$	$t_{1}$	$t_{K}$	$lr$	$K$	$\tau$	$t_{1}$	$t_{K}$	$lr$	$K$	$\tau$	$t_{1}$	$t_{K}$
LSUN-Bedroom	$10^{3}$	$20$	$50$	$1$	$0$	$10^{5}$	$20$	$50$	$1$	$0$	$10^{5}$	$20$	$50$	$1$	$0$	$10^{3}$	$20$	$50$	$5$	$0$

Appendix

Table 9: Hyperparameters of DCDP with CMs. Different from the diffusion models in previous experiments which specify the noise level via discrete forward steps (ranging from 0 to 1,000), CMs are trained on a continuous range of Gaussian noise with standard deviation spanning [0.002,80]. Therefore,

t_{1}

and

t_{K}

in this table stand for the standard deviation of the noise rather than the timestep.

Method	Super Resolution $4\times$			Inpainting (Box)
Method	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$
DPS [ 10 ]	$\underline{0.176}$ $\pm 0.04$	$28.45\pm 1.99$	$0.792\pm 0.05$	$0.115\pm 0.02$	$23.14\pm 3.14$	$0.829\pm 0.03$
MCG [ 13 ]	$0.182\pm 0.03$	$24.45\pm 2.93$	$0.736\pm 0.06$	$0.119\pm 0.02$	$22.77\pm 3.06$	$0.827\pm 0.03$
ADMM-PnP [ 2 ]	$0.307\pm 0.03$	$21.54\pm 3.12$	$0.766\pm 0.03$	$0.251\pm 0.02$	$18.26\pm 1.86$	$0.724\pm 0.02$
DDNM [ 47 ]	$0.197\pm 0.04$	$28.03\pm 1.46$	$0.795\pm 0.08$	$0.235\pm 0.06$	$24.47\pm 2.38$	$0.837\pm 0.08$
RED-Diff [ 29 ]	$0.212\pm 0.06$	$27.18\pm 1.23$	$0.781\pm 0.07$	$0.165\pm 0.08$	$24.22\pm 3.16$	$0.824\pm 0.05$

Appendix

Table 10: Quantitative results of super resolution and inpainting on the FFHQ dataset for both pixel-based diffusion models and LDMs. The results colored in blue are for LDMs, while the others are for pixel space diffusion models. The best results are in bold and second best results are underlined for pixel space models and LDMs respectively.

Method	Gaussian Deblurring			Motion Deblurring
Method	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$
DPS [ 10 ]	$\underline{0.162}\pm 0.03$	$27.97\pm 2.01$	$0.782\pm 0.05$	$0.219\pm 0.04$	$25.34\pm 2.21$	$0.690\pm 0.07$
MCG [ 13 ]	$0.164\pm 0.03$	$25.03\pm 3.62$	$0.790\pm 0.06$	$0.343\pm 0.04$	$24.88\pm 2.01$	$0.708\pm 0.05$
DDNM [ 47 ]	$0.217\pm 0.04$	$28.23\pm 1.69$	$0.805\pm 0.08$	–	–	–
RED-Diff [ 29 ]	$0.242\pm 0.06$	$25.25\pm 2.17$	$0.791\pm 0.05$	$0.268\pm 0.08$	$24.32\pm 2.67$	$0.754\pm 0.05$
DiffPIR [ 57 ]	$0.236\pm 0.06$	$27.34\pm 2.15$	$0.778\pm 0.04$	$0.255\pm 0.07$	$26.69\pm 1.49$	$0.856\pm 0.06$

Appendix

Table 11: Quantitative results of Gaussian and motion deblurring on the FFHQ dataset for both pixel-based diffusion models and LDMs. The results colored in blue are for LDMs, while the others are for pixel space diffusion models. The best results are in bold and second best results are underlined for pixel space models and LDMs respectively.

Figure 21: Convergence of our methods in terms of PSNR, SSIM and LPIPS for solving nonlinear deblurring. DCDP greatly improves upon the pure data fidelity optimization by leveraging the image priors embedded in the diffusion models.

Diffusion Models for Image Restoration

Diffusion models for image restoration use generative priors to denoise, deblur, upscale, and reconstruct images from incomplete or corrupted measurements.

See all 189 papers

Sep 26, 2026

Residual Diffusion Implicit Models

Diffusion models achieve state-of-the-art results across multiple tasks. However, in inverse problems, standard initialization from pure Gaussian noise misaligns the generative process with real-world degradations. More recent methods such as diffusion bridges impose strict endpoint constraints and often require long reverse processes that are prone to hallucinations. Alternative consistency models provide noise-invariant, one-step mappings but lack inherent variance modeling and can degrade under severe corruption. Hence, residual diffusion implicit models (RDIMs) are proposed, constituting a generalized framework that explicitly models the residuals between high-quality (HQ) and low-quality (LQ) images, aligning the forward process with the actual degradation. A non-Markovian implicit reverse sampler is derived, which can skip intermediate timesteps, enabling accurate few-step or even single-step reconstruction, while mitigating the hallucinations inherent to long diffusion chains. RDIM also introduces a controllable variance mechanism that interpolates between deterministic and stochastic sampling, balancing fidelity and diversity. Furthermore, it enables the straightforward use of perceptual losses, when needed. Experiments on denoising and super-resolution benchmarks demonstrate that RDIMs consistently outperforms the state of the art, including bridge and consistency models, in terms of PSNR, SSIM, and LPIPS, reducing hallucinations while requiring only a few sampling steps (often just one). The results position RDIMs as an efficient solution for a broad range of image restoration tasks.

João Guerreiro, Pedro Tomás, Helena Aidos +1

INESC-ID, Instituto Superior Técnico, Universidade de Lisboa, Lisboa, Portugal · ISR-Lisboa, Instituto Superior Técnico, Universidade de Lisboa, Lisboa, Portugal · LASIGE, Faculdade de Ciências, Universidade de Lisboa, Lisboa, Portugal

Diffusion Models for Image Restoration Diffusion-Based Inverse Problems Diffusion Models Image Denoising Diffusion Model Inference Acceleration Image Super-Resolution

Ask PDF

Aug 7, 2026

Local Epistemic Uncertainty Guided Active Sampling for Plug-and-play Diffusive Image Restoration

Diffusion models have demonstrated remarkable effectiveness in image restoration tasks. However, when guiding image reconstruction, existing Diffusion Model-based Image Restoration (DMIR) methods typically rely on fixed data constraints and uniform step sizes, thereby overlooking the dynamic nature of the generative process. Such rigid designs render the models vulnerable to spatially non-uniform degradations, thus resulting in structural distortions and loss of fine details. Meanwhile, uniform step sizes introduce computational redundancy, whereas naïve step reduction strategies tend to accumulate approximation errors. To address these limitations, we propose a Local Epistemic Uncertainty Guided Active Sampling framework (LEADer). In the spatial domain, LEADer leverages pixel-wise uncertainty to dynamically modulate the prior strength within the null space, which effectively balances detail preservation and artifact suppression. In the temporal domain, it quantifies sampling stability via the uncertainty trace to enable adaptive trajectory pruning, thereby accelerating convergence. Theoretical proofs demonstrate that our framework achieves strict data consistency, while the trajectory pruning strategy admits a deterministic error bound, thereby guaranteeing stable convergence under skip sampling. Notably, our plug-and-play method can be seamlessly integrated into various DMIR baselines. Extensive experiments show that LEADer improves the performance of multiple state-of-the-art DMIR methods, while significantly reducing sampling time with negligible memory overhead. Code is available at https://www.jiaqizhang-sengoku.com/Code/LEADer.

Jiaqi Zhang, Zheng Pang, Rongrong Gao +2

Jiangsu University, Zhenjiang, China · Xi’an University of Technology, Xi’an, China

Diffusion Models for Image Restoration Diffusion Model Guidance Diffusion Sampling Diffusion-Based Inverse Problems Diffusion Model Inference Acceleration

Ask PDF

May 14, 2026

Image Restoration via Diffusion Models with Dynamic Resolution

Diffusion models (DMs) have exhibited remarkable efficacy in various image restoration tasks. However, existing approaches typically operate within the high-dimensional pixel space, resulting in high computational overhead. While methods based on latent DMs seek to alleviate this issue by utilizing the compressed latent space of a variational autoencoder, they require repeated encoder-decoder inference. This introduces significant additional computational burdens, often resulting in runtime performance that is even inferior to that of their pixel-space counterparts. To mitigate the computational inefficiency, this work proposes projecting data into lower-dimensional subspaces using dynamic resolution DMs to accelerate the inference process. We first fine-tune pre-trained DMs for dynamic resolution priors and adapt DPS and DAPS, which are two widely used pixel-space methods for general image restoration tasks, into the proposed framework, yielding methods we refer to as SubDPS and SubDAPS, respectively. Given the favorable inference speed and reconstruction fidelity of SubDAPS, we introduce an enhanced variant termed SubDAPS++ to further boost both reconstruction efficiency and quality. Empirical evaluations across diverse image datasets and various restoration tasks demonstrate that the proposed methods outperform recent DM-based approaches in the majority of experimental scenarios. The code is available at https://github.com/StarNextDay/SubDAPS.git.

Yang Zheng, Wen Li, Zhaoqiang Liu

School of Computer Science and Engineering, University of Electronic Science and Technology of China.

Diffusion Models for Image Restoration Diffusion-Based Inverse Problems Diffusion Posterior Sampling Diffusion Model Inference Acceleration Diffusion Models

Ask PDF

Abstract

Diffusion Models for Image Restoration

Explore similar work

Residual Diffusion Implicit Models

Local Epistemic Uncertainty Guided Active Sampling for Plug-and-play Diffusive Image Restoration

Image Restoration via Diffusion Models with Dynamic Resolution