Posterior Information Dynamics of Diffusion Models for Linear Inverse Problems

Xiangming Meng

cs.LGAug 22, 2026

Posterior Information Dynamics of Diffusion Models for Linear Inverse Problems

Authors: Xiangming Meng

Organizations: ZJU-UIUC Institute, Zhejiang University Haining, Zhejiang 314400, China

Abstract

Diffusion models are widely used as priors for linear inverse problems, yet endpoint quality does not reveal when measurement information enters reverse denoising or how it is allocated across signal directions. We study this process through the smoothed likelihood force, the difference between exact posterior and prior scores at each noise level. For a fixed measurement, its expected squared norm gives both posterior--prior relative-entropy dissipation and reverse-path relative-entropy growth. Averaging over measurements yields an information--minimum mean-square error (I-MMSE) identity linking information gain to denoising-error reduction. Under finite second moments, the force energy and its ratio to prior-score energy decay quadratically in the noising kernel's signal coefficient at high noise. Solvable models show that conditioning removes class separation already explained by the measurement, reduces a uniform index entropy over $n$ empirical samples from $\log n$ to $H(I\mid r)$ , and makes assimilation depend on operator--prior alignment even for identical singular values. Experiments in models with tractable posteriors evaluate these predictions. In a separate illustration with a frozen FFHQ model, masks sharing the same spectrum yield different prior-normalized null-space trajectory statistics.

Figures & tables

Figure 1: Matched-spectrum orientation dependence in an anisotropic Gaussian prior. (a) Two rank-one, row-orthonormal measurements have the same singular values but select different prior eigendirections. (b) Their exact measurement-assimilation curves

\mathcal{M}_{\gamma}(A)=\operatorname{I}(R_{A};Y_{\gamma})

differ throughout the Gaussian channel and equal the corresponding mean posterior–prior reverse-path KL at

\gamma=\gamma_{t}

; the dashed asymptotes are the clean measurement information, equivalently half the log posterior-covariance volume contraction. Parameters are

\Sigma=\operatorname{diag}(4,0.25)

and

\sigma_{y}=0.5

; the curves evaluate Theorem 5.9 in closed form.

Aspect	Exact coordinates	Role
Measurement assimilation	$\mathcal{K}_{t}(r)$ , $\mathcal{M}_{\gamma}$ , $\mathbb{E}\\|g_{t}(X_{t},R)\\|^{2}$	fixed-measurement information, its measurement average, and the corresponding force-energy rate
Class resolution	$\eta(r)$ , $\delta=(\mathrm{I}_{d}-KA)m$ , $\widetilde{\phi}_{r}(t)$	measurement evidence, residual separation, and baseline-corrected commitment
Sample-index resolution	$H(I\mid r)$ , $\mathcal{J}_{t}(r)$ , $\overline{\mathcal{J}}_{\gamma}$	remaining index budget and its fixed-measurement and averaged resolution
Directional allocation	$\rho_{i}(t)$ , $\mathcal{M}_{\gamma}(A)$	direction-wise force strength and operator-dependent information assimilation

Table 1: Summary of the exact coordinates used in Sections 4–5. The rows describe complementary aspects of conditioning and need not change at a common noise level.

Figure 2: Residual class-information diagnostics in an exact conditional Gaussian mixture. (a) At zero field, the residual coordinate

\kappa_{t}

crosses the local-instability reference

\kappa_{t}=1

. (b) With a measurement-induced field, subtracting the high-noise cloning baseline isolates evidence added by the noisy state. (c) Residual separation

d^{-1}\|(\mathrm{I}_{d}-KA)m\|^{2}

decreases only in observed directions.

Figure 3: Exact empirical-posterior diagnostics on a 1500-sample MNIST subset (100 references; bands and bars are standard errors). (a)

H(I\mid X_{t},r)

starts at the measurement-dependent budget

H(I\mid r)

and decays as components separate; the colored dotted horizontals are the corresponding mean budgets. (b) Posterior budgets; the gray dashed line is the unconditional reference

H(I)=\log n

. (c) Measurement-induced high-noise class-cloning baselines.

Figure 4: Information diagnostics on common exact-score conditional reverse trajectories (MNIST empirical prior; twenty references, 32 chains). (a) Fixed-reference row-space force ratio; dotted colored curves are the corresponding global ratios, and the gray dashed line is a unit-slope reference for the

a_{t}^{2}

high-noise scaling. (b) Class entropy, with static measurement baselines dashed. (c) Denoised residual approaching the compatible-pool sampling floor. (d) Index information, decomposed into the measurement head start (colored dashes) and subsequent trajectory resolution; hollow circles show independent forward conditional samples at every fourth saved checkpoint.

Figure 5: Total-covariance calibration for the exact finite empirical posterior. Solid curves are posterior-mean dispersion as a fraction of clean posterior variance; dashed curves are remaining conditional covariance. (a)–(b) Super-resolution row and null coordinates. (c)–(d) Inpainting row and null coordinates. Shaded regions are 95% reference-bootstrap intervals; at most checkpoints they are narrower than the plotted curves.

Figure 6: Method-specific contrasts between five contiguous and five random masks from the fixed library. Points are means over held-out references; bars are 95% reference-bootstrap intervals conditional on the masks. Positive values in (a) mean that the guided-to-prior null-space AUC ratio is larger for contiguous masks. Panel (b) shows the corresponding row-space contrasts.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Task (steps 200/400/800)	index $t_{1/2}$	class $t_{1/2}$	$R_{\rm end}/F_{r}$	entropy gap
Denoising	0.848/0.906/0.871	0.887/0.998/0.992	0.997/1.062/0.975	0.094/0.074/0.074
Super-resolution	0.915/0.921/0.922	0.868/0.861/0.870	0.992/1.001/1.024	0.136/0.091/0.065
Inpainting	0.891/0.893/0.906	0.868/0.868/0.890	1.012/1.029/1.016	0.206/0.115/0.069

Appendix

Table 2: Euler–Maruyama stability for the exact-score reverse SDE. Half-times are interpolated within each reference before averaging. The last column is the maximum difference between the reference-averaged reverse and forward conditional index-entropy curves.

Figure 7: Joint-law high-noise calculation for the finite empirical MNIST prior (1,024 samples; bands are standard errors). (a) Measurement-force energy per dimension. (b) Prior-score energy. (c) Their ratio. Dotted guides in (a) and (c) have unit slope against

a_{t}^{2}

Figure 8: Sensitivity of the finite-empirical-posterior diagnostics to the assumed likelihood width. (a) Posterior index budget

H(I\mid r)

. (b) High-noise class baseline

\phi_{r}(\infty)

. Curves average 100 in-support references; shaded bands are standard errors. Dashed lines mark the corresponding unconditional limits.

Task	DPS scale	DDNM+ $\eta$	PiGDM $\lambda$ ( $\eta$ )	DMPS scale
SR $4\times$	0.40	0.85	0.05 (1.0)	1.50
Box inpainting	0.65	0.85	0.10 (1.0)	1.75
Random inpainting	0.65	0.85	0.10 (1.0)	1.75

Appendix

Table 3: Configurations selected on the ten validation references. Every run uses 1000 reverse steps and

\sigma_{y}=0.05

Figure 9: Fixed-spectrum mask library. Black pixels are missing. All masks remove the same number of spatial positions and therefore have identical singular values.

Diffusion Models

Diffusion models learn to reverse gradual data corruption, enabling flexible generation, editing, forecasting, and inference.

See all 1,176 papers

Jun 15, 2026

Exact Posterior Score Estimation for Solving Linear Inverse Problems

Diffusion and flow-based models learn powerful data priors by training a denoiser to reverse Gaussian corruption. To use this prior to solve a linear inverse problem, one needs to sample from the posterior, but the score that the prior provides is the unconditional score, not the posterior score. Existing methods either steer a fixed pretrained denoiser with approximate measurement-matching corrections, or train a conditional restoration model that abandons the denoising structure of the prior. We derive the exact posterior score in closed form for linear Gaussian inverse problems under general Gaussian interpolants, and show that posterior sampling reduces to a denoising problem at an operator-dependent shifted pivot under an anisotropic noise covariance. We turn this identity into Exact Posterior Score (EPS), a denoising training objective that preserves the input/output structure of standard pretraining and can therefore be trained from scratch or fine-tuned from a pretrained denoiser. At inference, EPS uses the same sampler as the underlying backbone, with no likelihood gradients or projections. We evaluate EPS on five linear inverse problems across FFHQ and ImageNet, where it outperforms training-free and training-based baselines on fidelity, perceptual, and distributional metrics, while using roughly an order of magnitude fewer denoiser evaluations than gradient-based posterior samplers.

Abbas Mammadov, Ozgur Kara, Kaan Oktay +5

University of Oxford · UIUC · fal +1

Diffusion Models for Image Restoration Diffusion-Based Inverse Problems Score-Based Generative Modeling Diffusion Posterior Sampling Inverse Problems

Ask PDF

Jul 21, 2026

Provable diffusion-based posterior sampling for linear inverse problems via DDIM

Diffusion-based methods have achieved remarkable empirical success in solving inverse problems. However, many existing posterior samplers either lack rigorous theoretical guarantees or incur substantial computational overhead. We propose a simple and efficient algorithm, called \pddim, for solving linear inverse problems with diffusion priors via a DDIM-type sampler. Our method requires only lightweight, coordinate-wise modifications to the standard DDIM update, while explicitly incorporating the measurement model. The key idea is to perform posterior sampling separately along each singular direction of the measurement operator: for each direction, the sampler follows the learned diffusion prior when the observation signal-to-noise ratio (SNR) is below the corresponding diffusion SNR, and switches to a calibrated measurement-based predictor otherwise. We prove that the proposed sampler converges to the Bayesian posterior conditioned on the measurements. Empirical results show that the proposed sampler performs favorably against existing diffusion-based posterior samplers across a range of image restoration tasks, achieving the best performance on the majority of evaluation metrics considered. Overall, our results convert posterior sampling for noisy linear inverse problems to simple coordinate-wise DDIM updates, yielding an efficient, easy-to-implement algorithm with provable posterior consistency.

Yuchen Jiao, Na Li, Changxiao Cai +2

Department of Statistics and Data Science, Chinese University of Hong Kong, Hong Kong · Department of Industrial and Operations Engineering, University of Michigan, Ann Arbor, USA · Department of Statistics and Data Science, the Wharton School, University of Pennsylvania, PA, USA

Diffusion Models for Image Restoration Bayesian Inverse Problems Diffusion Sampling Diffusion-Based Inverse Problems Diffusion Posterior Sampling Inverse Problems

Ask PDF

Aug 15, 2026

A Posterior-Dynamics Framework for Imaging Inverse Problems with Pretrained Diffusion Priors

Pretrained diffusion models represent image distributions through a continuum of progressively smoothed distributions. This multiscale structure organizes generation from global structure to fine detail and supports high-quality, diverse samples. We exploit the same multiscale diffusion prior for linear imaging inverse problems. Rather than using the pretrained model only as a denoiser in an outer iteration, we define a surrogate likelihood whose center is aligned with the clean-image coordinate and whose covariance accounts for residual diffusion uncertainty. This construction defines an explicit surrogate posterior path, from which we derive continuous posterior dynamics. A tunable Langevin component supports target tracking and allows the amount of posterior exploration to be adapted to the application. We prove endpoint consistency and a finite-horizon tracking bound and, in the exact-score setting, first-order weak accuracy. For computation, we derive the Posterior-Dynamics Implicit--Explicit sampler (PD-IMEX), a stable method using one score evaluation per diffusion scale and an implicit data-consistency update. Experiments on deblurring, super-resolution, and inpainting show strong reconstruction quality at 100 score evaluations, coarse-grid stability, and controllable fidelity--diversity behavior.

Zhaoqiang Liu, Tongyao Pang, Ruibing Wang +1

School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China · Yau Mathematical Sciences Center, Shuangqing Complex, Tsinghua University, Beijing 100084, China

Diffusion Models for Image Restoration Bayesian Inverse Problems Diffusion-Based Inverse Problems Diffusion Posterior Sampling Image Inverse Problems

Ask PDF

Abstract

Diffusion Models

Explore similar work

Exact Posterior Score Estimation for Solving Linear Inverse Problems

Provable diffusion-based posterior sampling for linear inverse problems via DDIM

A Posterior-Dynamics Framework for Imaging Inverse Problems with Pretrained Diffusion Priors