Diffusion-Based Inverse Problems

Overview

These methods reconstruct hidden images or physical fields from measurements such as sparse MRI data, blurred images, or limited-angle CT projections, where many solutions may fit the observations. A pretrained diffusion model supplies a prior over plausible solutions, combined with measurement guidance through posterior sampling, plug-and-play optimization, or physics-informed conditioning. Current work also explores latent-space and distilled models to reduce inference costs, while using ensembles or sampling to estimate uncertainty. A central challenge is balancing measurement fidelity with realistic detail without hallucinating unsupported content, especially under model mismatch or distribution shift, while making posterior estimates reliable and computation practical.

Latest papers

156 papers12 papers in the last four weeks, down 56% on the four weeks before. 0.1% of all new papers.

Jun 4, 2025

Solving Inverse Problems via Diffusion-Based Priors: An Approximation-Free Ensemble Sampling Approach

Diffusion models (DMs) have proven to be effective in modeling high-dimensional distributions, leading to their widespread adoption for representing complex priors in Bayesian inverse problems (BIPs). However, current DM-based posterior sampling methods proposed for solving common BIPs rely on heuristic approximations to the generative process. To exploit the generative capability of DMs and avoid the usage of such approximations, we propose an ensemble-based algorithm that performs posterior sampling without the use of heuristic approximations. Our algorithm is motivated by existing work that combines DM-based methods with the sequential Monte Carlo (SMC) method. By examining how the prior evolves through the diffusion process encoded by the pre-trained score function, we derive a modified partial differential equation (PDE) governing the evolution of the corresponding posterior distribution. This PDE includes a modified diffusion term and a reweighting term, which can be simulated via stochastic weighted particle methods. Theoretically, we prove that the error between the true posterior and the empirical distribution of the generated samples can be bounded in terms of the training error of the pre-trained score function and the number of particles in the ensemble. Empirically, we validate our algorithm on several inverse problems in imaging to show that our method gives more accurate reconstructions compared to existing DM-based methods. Our code is available at the following Github repository~https://github.com/HaoxuanSteveC00/AFDPS-TMLR.
May 30, 2025

Diff-SPORT: Diffusion-based Sensor Placement Optimization and Reconstruction of Turbulent flows in urban environments

Rapid urbanization demands efficient monitoring of turbulent wind and pollutant dispersion, yet existing reconstruction and sensor placement strategies fail under realistic sparsity constraints. Here, we introduce Diff--SPORT, a diffusion-based framework that combines a generative diffusion prior with maximum a posteriori inference and Shapley-value attribution for high-fidelity flow reconstruction and optimal sensor placement. By training a diffusion prior model once over a domain, Diff--SPORT enables non-linear optimal sensor placement and near-real-time flow reconstruction from sparse measurements orders of magnitude faster than RANS or LES simulations, consistently outperforming state-of-the-art methods. The framework also extends, without algorithmic modification, to experimental passive scalar concentration dataset, a direct proxy for pollutant dispersion, measured in a 1:2400 scale water-flume model of the Beijing Haidian neighbourhood under realistic urban flow conditions. Shapley-guided sensor placement achieves up to 57% lower reconstruction error than randomly placed sensors at extreme sparsity, identifying compact and physically interpretable configurations. These results establish Diff--SPORT as a modular foundation offering a zero-shot alternative to retraining-intensive downstream strategies, supporting scalable urban flow monitoring for air quality management and resilient city design.
May 29, 2025

EquiReg: Equivariance Regularized Diffusion for Inverse Problems

Diffusion models represent the state-of-the-art for solving inverse problems such as image restoration tasks. Diffusion-based inverse solvers incorporate a likelihood term to guide prior sampling, generating data consistent with the posterior distribution. However, due to the intractability of the likelihood, most methods rely on isotropic Gaussian approximations, which can push estimates off the data manifold and produce inconsistent, poor reconstructions. We propose Equivariance Regularized (EquiReg) diffusion, a general plug-in framework that improves posterior sampling by penalizing trajectories that deviate from the data manifold. EquiReg formalizes manifold-preferential equivariant functions that exhibit low equivariance error for on-manifold samples and high error for off-manifold ones, thereby guiding sampling toward symmetry-preserving regions of the solution space. We highlight that such functions naturally emerge when training non-equivariant models with augmentation or on data with symmetries. EquiReg's largest gains are under reduced sampling and measurement consistency steps, where many methods suffer severe quality degradation. By regularizing trajectories toward the manifold, EquiReg implicitly accelerates convergence and enables high-quality reconstructions. EquiReg consistently improves performance in linear and nonlinear image restoration tasks and solving partial differential equations. Our code is available at https://github.com/Anima-Lab/EquiReg
Dec 3, 2024

Filterless Snapshot Hyperspectral Imaging using Guided Patch Diffusion

We consider the problem of reconstructing a HxWx31 hyperspectral image from a H×WH\times W grayscale snapshot measurement that is captured using only a single diffractive lens and a filterless panchromatic photosensor. This problem is severely ill-posed, but we present a model that produces high-quality results in simulation and experiment. We make efficient use of limited training data by creating a conditional denoising diffusion model that operates on small patches in a shift-invariant manner. During inference, we synchronize per-patch hyperspectral predictions using guidance by physical consistency with the system's optical point spread function. Our experiments reveal that the patch size can be as small as the point spread function, with local optical cues being the main source of information about complete spectra. Also, by drawing multiple samples, our model provides per-pixel uncertainty estimates that strongly correlate with reconstruction error.
Jul 1, 2024

An Expectation-Maximization Algorithm for Training Clean Diffusion Models from Corrupted Observations

Diffusion models excel in solving imaging inverse problems due to their ability to model complex image priors. However, their reliance on large, clean datasets for training limits their practical use where clean data is scarce. In this paper, we propose EMDiffusion, an expectation-maximization (EM) approach to train diffusion models from corrupted observations. Our method alternates between reconstructing clean images from corrupted data using a known diffusion model (E-step) and refining diffusion model weights based on these reconstructions (M-step). This iterative process leads the learned diffusion model to gradually converge to the true clean data distribution. We validate our method through extensive experiments on diverse computational imaging tasks, including random inpainting, denoising, and deblurring, achieving new state-of-the-art performance.
Mar 10, 2024

Decoupled Data Consistency with Diffusion Purification for Image Restoration

Diffusion models have recently gained traction as a powerful class of deep generative priors, excelling in a wide range of image restoration tasks due to their exceptional ability to model data distributions. To solve image restoration problems, many existing techniques achieve data consistency by incorporating additional likelihood gradient steps into the reverse sampling process of diffusion models. However, the additional gradient steps pose a challenge for real-world practical applications as they incur a large computational overhead, thereby increasing inference time. They also present additional difficulties when using accelerated diffusion model samplers, as the number of data consistency steps is limited by the number of reverse sampling steps. In this work, we propose a novel diffusion-based image restoration solver that addresses these issues by decoupling the reverse process from the data consistency steps. Our method involves alternating between a reconstruction phase to maintain data consistency and a refinement phase that enforces the prior via diffusion purification. Our approach demonstrates versatility, making it highly adaptable for efficient problem-solving in latent space. Additionally, it reduces the necessity for numerous sampling steps through the integration of consistency models. The efficacy of our approach is validated through comprehensive experiments across various image restoration tasks, including image denoising, deblurring, inpainting, and super-resolution.