ANaLOG: Anisotropic Native-Latent Operator Guidance for Solving Inverse Problems

Darshan Thaker; Lachlan Ewen MacDonald; René Vidal

eess.IVSep 25, 2026

ANaLOG: Anisotropic Native-Latent Operator Guidance for Solving Inverse Problems

Authors: Darshan Thaker, Lachlan Ewen MacDonald, René Vidal

Organizations: University of Pennsylvania

Abstract

Native-latent guidance is a recent paradigm for solving inverse problems with latent diffusion models. It replaces repeated evaluations of the image-space forward model, each requiring a decoder pass, with efficient guidance computed using a learned latent-space surrogate. However, existing methods apply guidance uniformly across latent dimensions, ignoring that measurements are informative only along certain directions and that the reliability of model predictions varies across inputs and timesteps. We propose ANaLOG, a framework for efficient uncertainty-aware guidance with pretrained latent diffusion models. ANaLOG models uncertainty by learning an anisotropic, input- and time-dependent covariance that is integrated into the guidance mechanism to emphasize reliable directions and downweight uncertain ones. We theoretically analyze this framework in a linear model setting and prove that anisotropic, uncertainty-aware weighting is necessary for correct sampling, whereas isotropic guidance induces sampling errors. Experiments across five challenging inverse problems show that ANaLOG improves perceptual reconstruction quality over existing methods while preserving efficiency.

Figures & tables

Figure 1: Left: Representative results of our method on the Gaussian deblurring, 8x super-resolution, and nonlinear JPEG decompression tasks. Right: ANaLOG models anisotropic covariance during guidance without expensive decoding steps. On a Gaussian deblurring task, ANaLOG preserves identity-relevant detail that isotropic approximations wash out.

	Efficient Diffusion	Efficient Guidance	Uncertainty- Aware
Pixel-space	✗	✓	✓
LDPS, PSLD	✓	✗	✗
STSL	✓	✗	✓
SILO	✓	✓	✗
ANaLOG (Ours)	✓	✓	✓

Table 1: Comparison of posterior sampling methods across three desiderata.

Panel of Figure 2 — Figure 2: Guidance pathways at each sampling step.

Panel of Figure 3 — Figure 3: Visualizing our theorem bounds.

Figure 4: Qualitative comparison on FFHQ deconvolution tasks. Best viewed zoomed in.

Panel of Figure 5 — Figure 5: Learned variance maps and ablation of covariance structure. (a) Task-dependent variance maps show the model learns distinct spatial patterns for each inverse problem. (b) Progressively richer covariance models improve reconstruction quality.

				Gaussian Blur			4x Super-Resolution			8x Super-Resolution
		Method	Time (s)	FID $\downarrow$	LPIPS $\downarrow$	PSNR $\uparrow$	FID $\downarrow$	LPIPS $\downarrow$	PSNR $\uparrow$	FID $\downarrow$	LPIPS $\downarrow$	PSNR $\uparrow$
FFHQ	Decode	LDPS	52.2	84.8	0.564 (.004)	21.48 (.16)	68.6	0.554 (.003)	20.70 (.11)	55.5	0.536 (.003)	21.24 (.08)
		PSLD	62.2	70.8	0.535 (.003)	22.65 (.14)	53.5	0.505 (.003)	22.52 (.10)	54.0	0.513 (.003)	21.68 (.08)
		STSL	433	41.7	0.419 (.002)	27.36 (.07)	39.7	0.415 (.002)	26.88 (.10)	38.6	0.439 (.002)	25.58 (.08)
	Native	SILO	24.0	47.3	0.367 (.002)	25.03 (.06)	38.8	0.331 (.002)	26.26 (.06)	35.0	0.356 (.002)	25.81 (.06)
		ANaLOG	23.7	29.8	0.326 (.002)	27.34 (.09)	27.0	0.303 (.002)	27.56 (.08)	38.8	0.357 (.002)	25.39 (.05)

Table 2: Quantitative results on deconvolution tasks on the FFHQ and ImageNet datasets. Bolded entries indicate best performance, underlined entries indicate second best performance, and subscripts denote standard error of mean. We bold multiple rows in the event of approximately tied results. Gaussian deblurring is the representative task for timings since timings do not vary significantly across the different forward operators.

				Box Inpainting			JPEG
		Method	Time (s)	FID $\downarrow$	LPIPS $\downarrow$	PSNR $\uparrow$	FID $\downarrow$	LPIPS $\downarrow$	PSNR $\uparrow$
FFHQ	Decode	LDPS	52.2	68.2	0.205 (.001)	16.61 (.12)	101.0	0.612 (.003)	17.83 (.14)
		PSLD	62.2	62.7	0.196 (.001)	17.09 (.12)	N/A	N/A	N/A
		STSL	433	63.1	0.191 (.001)	18.19 (.13)	63.5	0.502 (.003)	22.09 (.17)
	Native	SILO	24.0	32.8	0.142 (.001)	22.59 (.08)	68.3	0.471 (.002)	20.87 (.06)
		ANaLOG	23.7	22.8	0.135 (.001)	21.31 (.11)	46.3	0.393 (.002)	23.91 (.06)

Table 3: Quantitative results on center box inpainting and JPEG decompression on the FFHQ dataset. Note PSLD does not apply to nonlinear inverse problems.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 6 — Figure 6: Commuting diagram for the latent forward operator. The Noble identity guarantees that encoding after degradation ( $\mathcal{E}\circ\mathbf{A}$ , right then down) equals latent degradation after encoding ( $\mathbf{H}\circ\mathcal{E}$ , down then right), so the diagram commutes: $\mathcal{E}(\mathbf{A}\mathbf{x}_{0})=\mathbf{H}\,\mathcal{E}(\mathbf{x}_{0})=\mathbf{H}\mathbf{z}_{0}$ .

Figure 7: KL lower bound ( 18 ) vs. measurement noise

\sigma_{y}

t=0.05

for different spectral decay rates

\alpha

. The isotropic penalty depends non-trivially on

\sigma_{y}

: at low noise the bound is set by the operator’s frequency response, while at high noise the encoder’s whitening (

\sigma_{y}^{2}/\lambda_{\omega}

) dominates.

Figure 8: Covariance eigenvalues

c_{\omega}(t)

t=0.05

for different forward operators (

\alpha=2.5

\sigma_{y}=0.001

). Eigenvalues are plotted against nonnegative frequency magnitude

|\omega|

. Each operator produces a distinct spectral profile. The identity is nearly flat; blurs suppress high frequencies progressively;

4\times

SR introduces a sharp cutoff. An isotropic covariance cannot adapt to these task-dependent structures.

Layer	Operation	Output shape	# Params
Input	$\mathbf{f}=[\mathbf{f}_{\mathrm{agg}}\,\\|\,\mathbf{e}_{\sigma}]$	$385\times 64\times 64$	—
Conv-1	$\mathrm{Conv2d}(385\!\to\!128,\,3\!\times\!3,\,\text{pad}=1)$	$128\times 64\times 64$	$443{,}648$
	$\mathrm{SiLU}$	$128\times 64\times 64$	—
Conv-2	$\mathrm{Conv2d}(128\!\to\!32,\,3\!\times\!3,\,\text{pad}=1)$	$32\times 64\times 64$	$36{,}896$
	$\mathrm{SiLU}$	$32\times 64\times 64$	—
Conv-3	$\mathrm{Conv2d}(32\!\to\!4,\,1\!\times\!1)$	$4\times 64\times 64$	$132$

Appendix

Table 4: Architecture of the log-variance head

h_{\phi}

. Input is the pre-head feature

\mathbf{f}\in\mathbb{R}^{B\times 385\times 64\times 64}

(aggregated UNet features

\mathbf{f}_{\mathrm{agg}}\in\mathbb{R}^{384\times 64\times 64}

concatenated with the sigma embedding map

\mathbf{e}_{\sigma}\in\mathbb{R}^{1\times 64\times 64}

). Output is per-pixel

\log\sigma^{2}\in\mathbb{R}^{B\times 4\times 64\times 64}

Figure 9: Gaussian Blur task on FFHQ dataset.

Figure 10: 4x Super-Resolution task on FFHQ dataset.

Figure 11: 8x Super-Resolution task on FFHQ dataset.

Figure 12: JPEG Decompression task on FFHQ dataset.

Figure 13: Box Inpainting task on FFHQ dataset.

Task	Method	FID $\downarrow$	LPIPS $\downarrow$	PSNR $\uparrow$
Gaussian deblurring	SILO	47.3	0.367	25.03
	ANaLOG	29.8	0.326	27.34
	ANaLOG -DCT	30.9	0.338	26.36
	ANaLOG -DCT-block	22.0	0.292	26.48
JPEG decompression	SILO	68.3	0.471	20.87
	ANaLOG	46.3	0.393	23.91

Appendix

Table 5: Richer covariance parameterizations improve restoration quality.

Task	$\mathbb{E}[z^{2}]$	$\overline{z}$	$\mathrm{std}(z)$	$\mathrm{Var}_{b}[z_{i}]$ across coords			$\Pr[\,\mathrm{Var}_{b}[z_{i}]\!\in\![0.8,1.25]\,]$	$\|\overline{z_{i}}\|$
				mean	median	std		avg over coords
gb	1.033	$+0.024$	1.016	1.030	1.010	0.199	93.7%	0.068
sr4	1.069	$-0.139$	1.024	0.943	0.947	0.125	87.4%	0.326
sr8	1.015	$-0.074$	1.005	0.958	0.964	0.109	93.1%	0.213
ip	0.993	$+0.020$	0.996	0.992	0.981	0.137	93.6%	0.058
jpeg	0.999	$-0.030$	0.999	0.997	0.992	0.102	97.0%	0.059

Appendix

Table 6: Per-coordinate calibration of the learned diagonal covariance

\Sigma_{\theta}

on FFHQ (

N\!=\!256

held-out images per task).

\mathbb{E}[z^{2}]

\overline{z}

and

\mathrm{std}(z)

are pooled over all latent coordinates. The remaining columns characterise the distribution of

\mathrm{Var}_{b}[z_{i}]

over the

4\!\times\!64\!\times\!64=16{,}384

latent coordinates, and the per-coordinate residual bias

|\overline{z_{i}}|

averaged over coordinates. The sampling-noise floor on

\mathrm{Var}_{b}[z_{i}]

N\!=\!256

\approx 0.089

Figure 14: Calibration of the learned diagonal covariance

\Sigma_{\theta}

on FFHQ, all five tasks overlaid ( gb , sr4 , sr8 , ip , jpeg ;

N\!=\!256

held-out images each). (a) Distribution of the per-coordinate empirical variance

\mathrm{Var}_{b}[z_{i}]

over the

16{,}384

latent coordinates; the dashed line marks the calibrated value

1

. All five tasks are sharply peaked at

1

. (b) Reliability diagram on log-log axes: coordinates binned into 12 quantiles by mean predicted variance; the dashed line is

y\!=\!x

. All five tasks track

y\!=\!x

across roughly two orders of magnitude of predicted variance, indicating that the variance head is calibrated not only on average but conditional on its own predictions.

Figure 15: Sensitivity to the guidance scale hyperparameter on FFHQ dataset.

Guidance	FID $\downarrow$	LPIPS $\downarrow$	PSNR $\uparrow$
Stop-gradient (ours)	61.7	0.317	27.47
Full covariance gradient	60.9	0.325	27.44

Appendix

Table 7: Effect of stopping gradients through the learned covariance.

Diffusion Model Guidance

Diffusion model guidance steers denoising toward desired conditions, rewards, constraints, or away from unwanted concepts.

See all 364 papers

Sparse Scheduled Diffusion Guidance for Inverse Problems
Mar 9, 2026Abduragim Shtanchaev, Albina Ilina, Yazid Janati +3Diffusion-Based Inverse Problems Diffusion Posterior Sampling
MBZUAI, Abu Dhabi, United Arab Emirates · Institute of Foundation Models · EPITA, Paris, France
Consistency Regularised Gradient Flows for Inverse Problems
May 8, 2026Alessio Spagnoletti, Tim Y. J. Wang, Marcelo Pereyra +1Latent Diffusion Models Wasserstein Gradient Flows
Laboratoire MAP5, UMR 8145, Université Paris Cité, CNRS · Department of Mathematics, Imperial College London · Heriot-Watt University, MACS & Maxwell Institute
Learning Normalized Energy Models for Linear Inverse Problems
May 15, 2026Nicolas Zilberstein, Santiago Segarra, Eero Simoncelli +1Bayesian Inverse Problems Energy-Based Models
Rice University, Houston, TX, USA · Flatiron Institute, New York, NY, USA · New York University, New York, NY, USA

Mar 9, 2026J/K move · Enter open · S save

Sparse Scheduled Diffusion Guidance for Inverse Problems

Abduragim Shtanchaev, Albina Ilina, Yazid Janati, Arip Asadulaev+2

MBZUAI, Abu Dhabi, United Arab Emirates · Institute of Foundation Models · EPITA, Paris, France

Diffusion-Based Inverse Problems Diffusion Posterior Sampling Diffusion Guidance Diffusion Timestep Scheduling

Open paper AskPDF arXiv

Pretrained diffusion models are effective priors for Bayesian inverse problems, but posterior sampling with these priors is often costly because data-consistency guidance is applied throughout the full reverse trajectory. Existing methods have shown that vector-Jacobian products through the denoiser can sometimes be avoided, yet they typically still rely on dense guidance through the full trajectory or expensive inner solves. We introduce Sparse Scheduled Diffusion Guidance for Inverse Problems (Spin), a solver that avoids starting posterior sampling from pure noise. Spin first samples from a posterior time-marginal at an intermediate timestep

t_*

, and then uses that state as a warm start for a guided reverse diffusion process. At guidance time, instead of enforcing the measurement constraint at every denoising step, Spin applies lightweight corrections only at scheduled timesteps where the denoiser can still clean up artifacts. The resulting procedure decouples prior refinement from data consistency: the prior supplies denoising, while lightweight pixel-space optimization enforces the measurement constraint without backpropagation through the denoiser or decoder. Across linear and nonlinear inverse problems on FFHQ and ImageNet, Spin achieves competitive reconstruction quality with a substantially better runtime--memory profile, running 2x faster on pixel-space models and up to 50x faster on latent diffusion models, with lower memory costs.

Abstract

Diffusion Model Guidance

Explore similar work

Sparse Scheduled Diffusion Guidance for Inverse Problems

Consistency Regularised Gradient Flows for Inverse Problems

Learning Normalized Energy Models for Linear Inverse Problems