Overview

Image inverse problems arise when measurements are blurred, undersampled, noisy, or transformed by an imperfectly known imaging system, as in MRI, CT, microscopy, and remote sensing. Methods combine physical forward models with iterative optimization and regularization, learned or unrolled networks, and plug-and-play denoisers; newer approaches also use diffusion and flow-based generative priors for restoration and posterior sampling. Work spans tasks such as deblurring, inpainting, super-resolution, sparse-view tomography, and phase retrieval, with training-free and self-supervised methods addressing settings where paired data are scarce. A central tension is how to use strong learned priors to fill in missing information without hallucinating details, while maintaining measurement fidelity, uncertainty estimates, robustness to model mismatch, and reliable convergence at practical computational cost.

Latest papers

127 papers7 papers in the last four weeks, down 53% on the four weeks before. 0.1% of all new papers.

Aug 4, 2025

A Morse-Bott Framework for Blind Inverse Problems: Local Recovery Guarantees and the Failure of the MAP

Maximum A Posteriori (MAP) estimation is a cornerstone framework for blind inverse problems, where an image and a forward operator are jointly estimated as the maximizers of a posterior distribution. In applications such as blind deblurring, this principle is used to recover sharp images from degraded observations. In this paper, we analyze the recovery guarantees of MAP-based methods by adopting a Morse--Bott framework. We model the image potential as a Morse--Bott function, where natural images are modeled as residing locally on a critical submanifold. This means that while the potential is locally flat along the natural'' directions of the image manifold, it is strictly convex in the directions normal to it. We demonstrate that this Morse--Bott hypothesis aligns with the structural properties of state-of-the-art learned priors, a finding we validate through an experimental analysis of the potential landscape and its Hessian spectrum. Our theoretical results show that, in a neighborhood of the ground-truth image and operator, the posterior admits local minimizers that are stable both with respect to initialization (gradient descents converge to the same minimizer) and to small perturbations of the data (solutions vary smoothly with the observations). This local stability potentially provides a theoretical justification for the empirical success of well designed gradient-based optimization in these settings. However, we also demonstrate that this local stability is a **local** property: the blurry trap'', well-known for sparse priors in blind deconvolution, persists even with state-of-the-art learned priors. Our findings demonstrate that the failure of MAP in blind deconvolution is not a limitation of prior quality, but an intrinsic characteristic of the landscape. We conclude that successful recovery depends on strategic initialization around favorable local minima.
Jul 19, 2025

QUTCC: Quantile Uncertainty Training and Conformal Calibration for Imaging Inverse Problems

While deep learning offers tremendous promise for scientific and medical imaging, any failures and hallucinations (predictions that do not coincide with reality) are hard to pinpoint and can have serious downstream consequences. Uncertainty estimation techniques, such as conformal prediction, can help by predicting statistically valid error bars for a model's prediction. However, popular conformal prediction methods were not designed for high-dimensional image-valued problems and do not take into account spatial correlations within an image during conformal calibration, resulting in larger-than-necessary uncertainty intervals. We propose a practical simultaneous quantile regression method that enables non-linear, spatially-adaptive scaling during conformal calibration. Our method, QUTCC uses a U-Net architecture with a quantile embedding to learn a full conditional quantile distribution during training, and then leverages this non-linear, learned function for spatially-adaptive conformal calibration. At test time, our method can efficiently estimate uncertainty intervals with pixel-marginal coverage guarantees. In addition, QUTCC can also predict pixel-wise conditional probability density estimates without any built-in distributional assumptions. We evaluate our method on several denoising problems, accelerated magnetic resonance imaging, and quantitative phase microscopy. Our method consistently produces tighter uncertainty intervals than prior conformal methods at the same coverage level, can predict plausible conditional distributions for different tasks, and in some cases, high-uncertainty regions can help us locate hallucinations in a model's prediction.
Jul 13, 2025

prNet: Data-Driven Phase Retrieval via Stochastic Refinement

Phase retrieval is an ill-posed inverse problem in which existing methods often struggle to balance reconstruction fidelity and perceptual quality. We propose a novel Langevin-based framework for phase retrieval that bridges model-driven and data-driven approaches by combining stochastic posterior sampling, learned denoising, and model-based updates to better navigate the perception-distortion tradeoff. The framework consists of three progressively more advanced variants. The first, prNet-Small, provides a theoretically grounded and lightweight pipeline that integrates Langevin dynamics with learned noise and denoising processes and alternating projection-based updates. Building upon this foundation, prNet-Large exploits parallel sampling of diverse reconstructions and aggregates them to approximate the MMSE solution, significantly improving distortion metrics. Finally, prNet-Large-Adversarial enhances perceptual quality by replacing simple averaging with an adversarially trained aggregation network while preserving reconstruction fidelity. In addition, we incorporate test time augmentation to further improve the performance. Extensive experiments demonstrate that the proposed framework consistently outperforms classical, deep learning-based, and diffusion-based baselines across multiple benchmarks in both reconstruction fidelity and perceptual quality. The source code and trained models are available at https://github.com/METU-SPACE-Lab/prNet-for-Phase-Retrieval
Jun 4, 2025

Solving Inverse Problems via Diffusion-Based Priors: An Approximation-Free Ensemble Sampling Approach

Diffusion models (DMs) have proven to be effective in modeling high-dimensional distributions, leading to their widespread adoption for representing complex priors in Bayesian inverse problems (BIPs). However, current DM-based posterior sampling methods proposed for solving common BIPs rely on heuristic approximations to the generative process. To exploit the generative capability of DMs and avoid the usage of such approximations, we propose an ensemble-based algorithm that performs posterior sampling without the use of heuristic approximations. Our algorithm is motivated by existing work that combines DM-based methods with the sequential Monte Carlo (SMC) method. By examining how the prior evolves through the diffusion process encoded by the pre-trained score function, we derive a modified partial differential equation (PDE) governing the evolution of the corresponding posterior distribution. This PDE includes a modified diffusion term and a reweighting term, which can be simulated via stochastic weighted particle methods. Theoretically, we prove that the error between the true posterior and the empirical distribution of the generated samples can be bounded in terms of the training error of the pre-trained score function and the number of particles in the ensemble. Empirically, we validate our algorithm on several inverse problems in imaging to show that our method gives more accurate reconstructions compared to existing DM-based methods. Our code is available at the following Github repository~https://github.com/HaoxuanSteveC00/AFDPS-TMLR.
May 29, 2025

EquiReg: Equivariance Regularized Diffusion for Inverse Problems

Diffusion models represent the state-of-the-art for solving inverse problems such as image restoration tasks. Diffusion-based inverse solvers incorporate a likelihood term to guide prior sampling, generating data consistent with the posterior distribution. However, due to the intractability of the likelihood, most methods rely on isotropic Gaussian approximations, which can push estimates off the data manifold and produce inconsistent, poor reconstructions. We propose Equivariance Regularized (EquiReg) diffusion, a general plug-in framework that improves posterior sampling by penalizing trajectories that deviate from the data manifold. EquiReg formalizes manifold-preferential equivariant functions that exhibit low equivariance error for on-manifold samples and high error for off-manifold ones, thereby guiding sampling toward symmetry-preserving regions of the solution space. We highlight that such functions naturally emerge when training non-equivariant models with augmentation or on data with symmetries. EquiReg's largest gains are under reduced sampling and measurement consistency steps, where many methods suffer severe quality degradation. By regularizing trajectories toward the manifold, EquiReg implicitly accelerates convergence and enables high-quality reconstructions. EquiReg consistently improves performance in linear and nonlinear image restoration tasks and solving partial differential equations. Our code is available at https://github.com/Anima-Lab/EquiReg
Dec 3, 2024

Filterless Snapshot Hyperspectral Imaging using Guided Patch Diffusion

We consider the problem of reconstructing a HxWx31 hyperspectral image from a H×WH\times W grayscale snapshot measurement that is captured using only a single diffractive lens and a filterless panchromatic photosensor. This problem is severely ill-posed, but we present a model that produces high-quality results in simulation and experiment. We make efficient use of limited training data by creating a conditional denoising diffusion model that operates on small patches in a shift-invariant manner. During inference, we synchronize per-patch hyperspectral predictions using guidance by physical consistency with the system's optical point spread function. Our experiments reveal that the patch size can be as small as the point spread function, with local optical cues being the main source of information about complete spectra. Also, by drawing multiple samples, our model provides per-pixel uncertainty estimates that strongly correlate with reconstruction error.
Aug 24, 2022

Maximum Likelihood on the Joint (Data, Condition) Distribution for Solving Ill-Posed Problems with Conditional Flow Models

In the absence of conditioning information, normalizing flow models are trained using maximum likelihood. Previous extensions to conditional settings have abandoned the simple maximum-likelihood approach, as obtaining the full joint distribution of data and conditions for training purposes (or equivalently, the true conditional distribution for each point in the training dataset) is impossible. However, under special conditions that nevertheless apply to a wide class of useful problems, it is possible to obtain a simple, computationally cheap proxy with the same global minimum as the conditional likelihood. Conditional flow models trained using this proxy do not require explicit knowledge of the conditional distribution, auxiliary networks or other specific architecture, or additional loss terms beyond maximum likelihood. They provide a natural and powerful way to solve a class of ill-posed inverse problems. I demonstrate these properties on easily visualized toy problems, then use the method to successfully generate class-conditional images and to reconstruct semantically degraded images via super-resolution.