Perturb-and-Solve: Efficient Learned-Operator Conditioning for Latent Diffusion Inverse Problems

Abduragim Shtanchaev; Arip Asadulaev; Luiza Labazanova; Aidar Alimbayev; Karim Salta; Eric Moulines

Figure 4: Anchor choices in coefficient space, corresponding to Table 11 . The horizontal axis weights recycled bridge noise; the vertical axis weights fresh noise. The blue arc keeps

c_{1}^{2}+c_{2}^{2}=1

, preserving the variance of the lifted prior-only proposal. The orange line joins the same endpoints but reduces its standard deviation to

0.71

of the bridge value at

\beta=\tfrac{1}{2}

. At a fixed

\beta

, both choices use the same fresh-noise coefficient and differ in recycled noise. The point

(1,1)

is the added-noise (“variance only”) variant, with standard deviation

\sqrt{2}

times the bridge value. These are statements about the prior-only proposal, not posterior exactness.

method	anchor	trust $\tau$	how the system is solved	idealization
DDS	denoised estimate	finite	$M$ -step CG in $\mathbb{R}^{d}$	converged solve
DiffPIR	denoised estimate	finite	closed form (FFT)	linear ${H}$
DDNM	denoised estimate	$\tau\to\infty$	projection by ${H}^{+}$	Moore–Penrose inverse
DING	perturbed prior mean	$\eta_{z}^{2}/\sigma_{{\mathbf{y}}}^{2}$	closed form, trivial system	in distribution
PASEO	either	tuned $r$	$M$ -step CG in $\mathbb{R}^{d}$

Appendix

Table 3: The four measurement steps as instances of equation D.1 . Trust is

\tau=\eta_{z}^{2}/r^{2}

; each row is recovered by one substitution under the stated idealization.

method	trust $\tau$	where the correction is applied	inner loop
FlowChef	$2s^{\prime}/\alpha_{t}$	the state	$N$ steps, velocity frozen
PnP-Flow	$\gamma_{n}/\sigma^{2}$	the clean estimate	none

Appendix

Table 4: The two first-order cases. Both anchor at the denoised estimate, keep only the gradient term of equation D.29 , and discard the preconditioner.

A. Agreement with the reference operator
Prior	Task	$r$	Value RMS error $\downarrow$		Direction cosine $\uparrow$
			Clean	Re-encoded	Clean	Re-encoded
RV-v5.1	SR $\times 4$	0.05	0.19	0.18	0.37	0.44
	SR $\times 8$	0.02	0.16	0.14	0.59	0.64
	Blur	0.005	0.31	0.31	0.63	0.62
	Inpaint	0.05	0.22	0.20	0.37	0.55

Appendix

Table 5: Does the learned operator predict both measurements and their local changes? Eight test images and four random directions per image and configuration. Panel A: value error is RMS distance from

{\mathcal{E}}\circ{\mathcal{A}}\circ{\mathcal{D}}

; compare it with

r

, the solver’s assumed operator-noise scale. Direction agreement is the cosine between learned and reference Jacobian–vector products (

1

is best). “Clean” means

{\mathcal{E}}({\mathbf{x}})

; “re-encoded” means the VAE mean encoding of a saved arc reconstruction, not an internal sampler state. Panel B: magnitude error compares directional-change lengths (

0.1

means

10\%

relative error). Adjoint error compares

\langle{\mathbf{u}},{\mathbf{J}}{\mathbf{v}}\rangle

with

\langle{\mathbf{J}}^{T}{\mathbf{u}},{\mathbf{v}}\rangle

. Both errors should approach

0

. The norm is estimated by power iteration; the condition bound estimate is

1+250^{2}\|{\mathbf{J}}\|_{2}^{2}/(r^{2}+\lambda)

(Appendix A.3 ), not a measured condition number. Reference derivatives use the fp32 chain with finite-difference checks; the JPEG check is unreliable across step sizes.

Task	$P$	$C$	$\rho$	$\lambda$	$r$	CFG
RV-v5.1
SR $\times 4$	3	3	0.15	0.005	0.05	1
SR $\times 8$	4	2	0.15	0.005	0.02	1
Gaussian blur	5	3	0.15	0.005	0.005	1
Inpainting	3	12	0.6	0.005	0.05	1
JPEG	4	1	0.3	0.05	0.08	1

Appendix

Table 6: PASEO hyperparameters on FFHQ. Both priors use

28

reverse steps and

53

NFEs for every task.

Task	Steps	NFEs	$P$	$C$	$\rho$	$\lambda$	$r$	CFG
SR $\times 4$	50	97	3	3	0.3	0	0.01	1
SR $\times 8$	100	197	3	1	0.45	0.005	0.025	1
Gaussian blur	50	97	4	1	0.4	0.005	0.02	4
Inpainting	100	197	2	2	1	0	0.005	1
JPEG	50	97	5	1	0.3	0.02	0.08	4

Appendix

Table 7: PASEO hyperparameters on COCO with SD3.5-medium and LSDIR-trained operators. Steps denotes the number of reverse steps, and NFEs the number of denoiser evaluations.

Algorithm 1 PASEO : Matrix-free local learned-operator conditioning

1. Ideal update: reference
Update rule	mean err.	cov. err.	$90\%$ coverage	spread
exact solve	0.008	0.045	0.900	1.000
2. Approximate update: fewer CG steps or partial correction
CG $C{=}$ 1	0.438	0.272	0.842	1.093
CG $C{=}$ 2	0.197	0.107	0.885	1.027
CG $C{=}$ 4	0.051	0.047	0.899	1.003

Appendix

Table 8: Checking samples against a known Gaussian posterior. Reading guide: mean and covariance errors should approach

0

, coverage

0.90

, and spread

1

; spread below

1

means too little variation. The problem has state dimension

d=64

, measurement dimension

q=32

, dense random

{H}

,

\eta=1

, and

r=0.3

. Each rule uses

20{,}000

draws, except the fixed-noise rows: their statistics are computed from

2{,}000

draws per fixed

{\mathbf{w}}^{\prime}

and averaged over

20

choices of

{\mathbf{w}}^{\prime}

. Mean error is

\|\bar{{\mathbf{z}}}-{\mathbf{m}}_{p}\|/\|{\mathbf{m}}_{p}-{\bm{\mu}}_{z}\|

; covariance error is the relative Frobenius error to

\mathbf{\Sigma}_{p}

. Coverage is the fraction of sample coordinates inside the true

90\%

marginal intervals; spread is

\sqrt{\operatorname{tr}\operatorname{Cov}/\operatorname{tr}\mathbf{\Sigma}_{p}}

.

1. Exact update: random-hole mask
	Truth drawn from the model’s prior				Real FFHQ face
Sampler	spread	TV	finest energy	LPIPS	spread	LPIPS
DING	1.029	0.019	—	0.038	1.085	0.045
PASEO , $\beta{=}1$	1.029	0.020	—	0.038	1.085	0.045
PASEO , arc	1.024	0.016	—	0.037	1.077	0.044
PASEO , denoiser anchor	1.023	0.015	—	0.037	1.077	0.044

Appendix

Table 9: Checking the full sampler against a known posterior, with exact linear latent operators. Reading guide: spread and finest energy should be

1

, and TV

0

; spread below

1

means the draws vary too little or are off-center, and finest energy above

1

means detail that the truth does not have. Spread is the RMS distance between draws divided by the RMS distance from the draws to the truth (in the holes, for the mask). TV is the total-variation distance of the rank histogram of the truth from uniform. Finest energy is the energy of the draws in the finest Fourier band divided by that of the truth (Table 10 gives all bands). Blocks 1 and 2 use

100

images and block 3 uses

64

, each with

20

draws. Bold (block 3): lowest LPIPS for each anchor; —: not measured.

Figure 5: Random-hole inpainting with the exact mask operator on three FFHQ faces. PASEO uses the arc anchor. The last column is the per-pixel standard deviation over

20

draws: the variation sits in the holes.

		$\beta=1$		arc
Band (cycles per latent cell)	Blur keeps	exact	$C=5$	exact	$C=5$
coarse ( $0$ – $0.1$ )	91%	1.00 [1.00, 1.00]	1.00 [1.00, 1.00]	1.00 [1.00, 1.00]	1.00 [1.00, 1.00]
medium ( $0.1$ – $0.3$ )	41%	1.15 [1.14, 1.16]	1.09 [1.09, 1.10]	1.16 [1.15, 1.17]	1.14 [1.13, 1.15]
fine ( $0.3$ – $0.4$ )	9%	2.56 [2.49, 2.64]	0.79 [0.79, 0.80]	2.84 [2.74, 2.93]	1.19 [1.18, 1.21]
finest ( $0.4$ – $0.75$ )	1%	14.54 [13.79, 15.30]	0.92 [0.91, 0.93]	18.73 [17.76, 19.69]	1.19 [1.17, 1.20]

Appendix

Table 10: Energy of the draws per frequency band, divided by that of the truth, on the latent blur with truths from the model’s prior (

100

images). Blur keeps is the mean gain of the operator in the band. The exact solve adds content where the blur keeps almost nothing.

Figure 6: Circular latent blur on one FFHQ face, one draw per CG budget. The first column shows the ground truth and the blurred measurement (top two rows) with the same zoom below. Few iterations give a smooth face; a few more give a sharp one; many iterations and the exact solve add a mesh-like texture that the truth does not have.

Figure 7: Rank of the truth among

20

draws, pooled over latent coordinates and

100

images, with truths from the model’s prior. A flat histogram at the dotted line is calibrated. (a) Inpainting: all samplers overlap. (b) Blur: the exact solve looks as calibrated as

C=5

here; Table 10 shows what these histograms miss.

Figure 8: CG budget on the exact latent blur (a–c, truths from the model’s prior,

64

images,

95\%

intervals) and on the learned-operator blur of the main experiments (d, Tables 13 and 12 ). The budget with the lowest LPIPS (circles) is where the fine-detail energy crosses

1

, with the spread closest to

1

one budget earlier; the arc is best near the

C=2

–

3

used with the learned operator.

Prior	Task	Arc interpolation	Denoiser estimate	Linear interpolation	Denoiser + noise	Perturbed prior mean
RV-v5.1	SR $\times 4$	0.135	0.134	0.145	0.161	0.214
	SR $\times 8$	0.205	0.198	0.281	0.667	0.374
	Blur	0.157	0.151	0.214	0.534	0.292
	Inpaint	0.140	0.132	0.147	0.430	0.139
	JPEG	0.255	0.223	0.362	0.689	0.679
SD3.5	SR $\times 4$	0.180	0.178	0.178	0.534	0.659

Appendix

Table 11: How does the anchor affect reconstruction quality? The table compares LPIPS (

\downarrow

) on the first

200

FFHQ images at the reference solver settings for each task and model, shared by all five anchors within each row. Inpainting uses the hard mask. Bold marks the lowest LPIPS and underline the second-lowest distinct value within each row; ties at the displayed precision share the same marking.

Relinearization count $P$
Setting	Level	SR $\times 4$	SR $\times 8$	Blur	Inpaint	JPEG
Reference LPIPS		0.131	0.199	0.150	0.138	0.248
	1	+0.127	+0.182	+0.176	+0.002	+0.244
	2	+0.021	+0.042	+0.050	-0.007	+0.066
	3	0	+0.012	+0.021	0	+0.015
	5	+0.007	-0.006	0	+0.096	+0.011

Appendix

Table 12: Solver sensitivity on RV-v5.1: one setting changes at a time on the first

100

FFHQ images, using the arc anchor and a hard mask for inpainting. The first row gives reference LPIPS; subsequent rows give

\Delta\mathrm{LPIPS}

(negative: better; positive: worse). Bold zero marks the reference setting, not the best result. These are individual changes, not joint retuning.

Relinearization count $P$
Setting	Level	SR $\times 4$	SR $\times 8$	Blur	Inpaint	JPEG
Reference LPIPS		0.173	0.314	0.189	0.117	0.335
	1	+0.039	0	0	0	+0.141
	2	0	+0.128	+0.013	+0.001	+0.040
	3	+0.025	+0.135	+0.038	+0.001	0
	5	+0.050	+0.140	+0.057	+0.001	-0.011

Appendix

Table 13: Solver sensitivity on SD3.5: one setting changes at a time on the first

100

FFHQ images, using the arc anchor and a hard mask for inpainting. The first row gives reference LPIPS; subsequent rows give

\Delta\mathrm{LPIPS}

(negative: better; positive: worse). Bold zero marks the reference setting, not the best result. These are individual changes, not joint retuning. For SR

\times 8

, the effective

r

is half the listed value.

Task	Native $\rho$	Ref. $C$	Ref.	Full deriv. ( $C=1$ )	$\Delta$ [95% CI]	Ref. (s)	Full deriv. ( $C=1$ ) (s)	Reduction	Speedup	Ref.	Full deriv. ( $C=1$ )
			Alex-LPIPS ( $n=100$ )			Sampler cost ( $20$ images, two-call sampler)				Decoded peak allocated (GiB)
SR $\times 4$	0.6	2	0.1727	0.1723	$-0.0004$ [ $-0.0030$ , $+0.0024$ ]	11.461	9.428	17.7%	$1.216\times$	16.764	16.764
SR $\times 8$	1.0	2	0.3136	0.2606	$-0.0530$ [ $-0.0591$ , $-0.0470$ ]	8.997	7.827	13.0%	$1.149\times$	16.751	16.751
Blur	0.6	2	0.1889	0.2151	$+0.0262$ [ $+0.0196$ , $+0.0330$ ]	8.467	8.222	2.9%	$1.030\times$	16.751	16.751
Inpaint	1.0	5	0.1172	0.1212	$+0.0040$ [ $+0.0028$ , $+0.0054$ ]	10.716	8.029	25.1%	$1.335\times$	16.794	16.794
JPEG	0.3	3	0.3350	0.4212	$+0.0861$ [ $+0.0760$ , $+0.0963$ ]	14.577	10.980	24.7%	$1.328\times$	16.784	16.784

Appendix

Table 14: Full-derivative (

C=1

) quality–cost ablation for SD3.5 at seed

0

, using the arc anchor and the supplied hard mask for inpainting. With zero-initialized CG,

C=1

is the full-derivative case: its single iterate gives a correction in the complete Jacobian-transpose measurement derivative direction

{\mathbf{J}}^{T}{\mathbf{b}}

, scaled by the CG line-search coefficient. It is not a converged linear-system solve. Alex-LPIPS (

\downarrow

) uses the first

100

FFHQ images; the interval is a paired bootstrap over images. Sampler time is the two-call sampler’s mean over the first

20

FFHQ images (one run per arm, both arms in the same job); peak memory comes unchanged from the earlier synchronized benchmark. The reference uses the task-specific CG budget shown; the full-derivative case keeps every other setting fixed, including the native correction fraction

\rho

. The LPIPS difference is full derivative minus reference.

		14 steps 25 denoiser evaluations			28 steps 53 denoiser evaluations			56 steps 109 denoiser evaluations			112 steps 221 denoiser evaluations
Prior	Task	LPIPS $\downarrow$	PSNR $\uparrow$	Time $\downarrow$	LPIPS $\downarrow$	PSNR $\uparrow$	Time $\downarrow$	LPIPS $\downarrow$	PSNR $\uparrow$	Time $\downarrow$	LPIPS $\downarrow$	PSNR $\uparrow$	Time $\downarrow$
RV-v5.1	SR $\times 4$	0.213	26.6	6	0.135	28.1	11	0.163	28.4	23	0.238	28.1	46
	SR $\times 8$	0.272	25.5	5	0.205	26.8	12	0.224	27.1	24	0.303	26.9	47
	Blur	0.216	26.6	8	0.157	27.8	17	0.165	28.4	34	0.205	28.3	70
	Inpaint	0.146	21.5	15	0.140	21.5	30	0.152	21.8	65	0.158	22.0	127
	JPEG	0.438	20.5	4	0.255	24.6	8	0.295	25.9	17	0.410	25.5	34

Appendix

Table 15: Reconstruction quality and runtime as the reverse-step count changes, using the same first

200

FFHQ images per configuration. Other solver settings are fixed at those selected for

K=28

. All runs use the arc anchor; inpainting uses the hard mask. Timings are separate from Table 1 . PSNR is in dB and time in seconds per image. For each task and each quality metric, bold marks the best and underline the second-best distinct value across step counts; ties at the reported precision share the same marking. Denoiser-evaluation counts and the Time column are those of the two-call sampler (

2(K-2)+1

evaluations for

K

steps); time is the mean over the first

20

images, one job per row.

Figure 9: Mean relative CG residual by step index, pooled by prior and CG budget. The plot shows only the range up to

1.05

; larger values are clipped, and Table 16 reports the pooled summaries. Because task and parameter mixtures differ between curves, this is not a controlled comparison of CG budgets on one fixed problem.

Prior	CG budget $C$	Logged solves	Mean relative residual $\downarrow$	Median relative residual $\downarrow$	Fraction with residual $>0.5$ $\downarrow$
RV-v5.1	1	202k	2.44	2.65	0.97
	2	202k	1.75	1.32	0.95
	3	306k	1.42	1.27	0.98
	4	49k	1.41	1.31	0.84
	8	49k	1.26	1.14	0.74
	12	122k	0.07	0.03	0.00

Appendix

Table 16: Accuracy of the truncated CG solves, grouped by prior and CG iteration budget

C

. Smaller relative residuals indicate more accurate solutions;

k

denotes thousands of logged solves. Rows pool different tasks and parameter settings, so their differences do not isolate the effect of changing

C

alone.

A. Variation across draws
Prior	Task	RMS distance ratio	Pixel std. whole (hole / visible)	LPIPS mean $\pm$ std.
RV-v5.1	SR $\times 4$	0.89	0.039	0.137 $\pm$ 0.008
	SR $\times 8$	0.79	0.039	0.210 $\pm$ 0.012
	Blur	0.83	0.037	0.159 $\pm$ 0.009
	Inpaint	0.87	0.070 (0.210 / 0.024)	0.158 $\pm$ 0.014
	JPEG	0.90	0.065	0.262 $\pm$ 0.015

Appendix

Table 17: Variation across eight seeded reconstructions (

10

–

17

) on the first

50

images per configuration, using the arc anchor. The inpainting rows use the corrected fixed-measurement protocol (measurement seed

0

), together with the mask-free soft likelihood weights from the main results. The retained SR, blur, and JPEG rows use the legacy joint-seed protocol, in which the run seed affects measurement construction and sampling. Panel A describes variation across runs; panel B measures RMS error against the noiseless degraded reference

{\mathcal{A}}({\mathbf{x}})

. These are descriptive statistics, not a posterior-calibration test.

Figure 10: Repeated PASEO reconstructions. In the inpainting rows, the ground-truth image and measurement are held fixed while the sampling seed varies; the four runs are seeds

12

,

13

,

14

and

16

. The inpainting image and these four seeds were selected by visual screening of all

50

measurements under both priors and all eight seeds, prioritizing clean boundaries and coherent facial structure rather than LPIPS. These rows use the current mask-free soft likelihood weights. The retained SR

\times 8

rows show legacy end-to-end repetitions with seeds

10

–

13

: the displayed measurement is from run seed

10

, and measurement noise is regenerated with the run seed for the other reconstructions. Table 17 summarizes all

50

images.

Anchor	Weighting	PSNR $\uparrow$ (dB)	LPIPS $\downarrow$ Whole image	LPIPS $\downarrow$ Hole	KID $\downarrow$ ( $\times 10^{3}$ )	Time $\downarrow$ (s/image)
Denoiser	Hard mask	21.97	0.133	0.330	3.50	30.4
Arc	Hard mask	21.64	0.141	0.354	16.26	30.9
Arc	Learned weight †	21.76	0.144	0.351	14.00	38.4
Arc	Fixed weight	21.13	0.150	0.362	14.88	33.0
Perturbed prior	Hard mask	21.77	0.139	0.342	8.89	31.8
Perturbed prior	Learned weight	19.85	0.247	0.676	60.78	36.0

Appendix

Table 18: Time re-measured with the two-call sampler on the first

20

images; the rows printed in Tables 1 and 4.2 carry those tables’ values. Anchor and weighting choices for centre inpainting on

1000

FFHQ images (first two groups) and

1000

COCO images (last group). Within each group and metric, bold marks the best value and underline the second-best distinct value. Shaded rows marked

\dagger

are the configurations reported in Tables 1 and 4.2 .

Image scored	Denoiser anchor		Arc anchor
	KID ( $\times 10^{3}$ ) $\downarrow$	FID $\downarrow$	KID ( $\times 10^{3}$ ) $\downarrow$	FID $\downarrow$
Whole reconstruction	3.50	21.2	16.26	34.4
Reconstructed hole crop	12.06	35.3	27.30	50.8
Reconstructed background + reference hole	-0.23	6.1	-0.05	6.6
Reconstructed hole + reference background	2.89	17.9	17.31	32.6

Appendix

Table 19: Locating the inpainting quality gap on the same

1000

test images with RV-v5.1 and the hard mask. For each metric within a row, bold marks the lower value and underline the higher value. Compare anchors within rows; scores from hole crops and complete images are not directly comparable.

Figure 11: The

256\times 256

hole, RV-v5.1 center inpainting, six test images. Top: the true image. Middle: the

{\mathbf{z}}_{0}^{(0)}

anchor. Bottom: the arc. The arc examples show recurring small dark blemishes on skin. Across the full dataset, the whole-image KID is about

13

points higher than with the denoiser anchor (Table 19 ). The regional comparison locates the gap in the filled hole, without establishing what causes the artifacts.

Task	Wins	$79$	$999$	$\Delta$	$79$	$999$	$\Delta$	$79$	$999$
		PSNR $\uparrow$			LPIPS $\downarrow$			Time (s)
SR $\times 4$	$21/30$	30.34	31.08	$+0.74$	0.1779	0.1730	$-0.0049$	130	1565
SR $\times 8$	$12/30$	27.53	27.90	$+0.37$	0.3652	0.3682	$+0.0030$	130	1565
Gaussian blur	$19/30$	30.30	31.13	$+0.83$	0.2028	0.1931	$-0.0097$	138	1657
Center inpainting	$12/30$	21.19	20.47	$-0.71$	0.2207	0.2226	$+0.0020$	267	3625
JPEG $q{=}10$	$\phantom{0}4/30$	29.59	31.17	$+1.58$	0.1940	0.2229	$+0.0289$	141	1668

Appendix

Table 20: ReSample under SD3.5 at

999

NFEs, compared with the

79

-NFE budget of Table 1 , paired on the same

30

images per task. “Wins” counts images on which the 999-NFE run attains the lower LPIPS, and a negative

\Delta

LPIPS favors

999

. The two time columns are measured under different conventions; see the text.

Task	GML-DPS	LDPS	PSLD
SR $\times 4$	28.65 / 0.253 / 20.51 / 606	26.42 / 0.343 / 29.71 / 497	22.44 / 0.490 / 40.36 / 606
SR $\times 8$	25.52 / 0.471 / 31.08 / 606	20.61 / 0.519 / 90.35 / 497	22.56 / 0.553 / 40.06 / 607
Gaussian blur	22.15 / 0.458 / 44.58 / 615	22.14 / 0.514 / 68.82 / 506	21.28 / 0.529 / 64.56 / 615
Center inpainting	16.90 / 0.313 / 80.73 / 606	18.07 / 0.363 / 64.87 / 497	17.84 / 0.339 / 63.28 / 605
JPEG $q{=}10$	29.47 / 0.295 / 13.69 / 617	29.50 / 0.296 / 22.63 / 512	n/a

Appendix

Table 21: GML-DPS , LDPS , and PSLD on FFHQ with SD3.5-medium at

1{,}000

steps, the authors’ default, with

1000

samples per cell. Each entry is PSNR / LPIPS / KID

\times 10^{3}

/ seconds per image; peak memory is

23.4

,

22.5

, and

23.4

GiB, respectively. For PSLD , JPEG is unsupported (n/a) because its update requires a linear operator adjoint (Appendix G.3 ).

	PSNR $\uparrow$		LPIPS $\downarrow$		Time (s/image) $\downarrow$		PASEO
Task	FlowDPS	PASEO	FlowDPS	PASEO	FlowDPS	PASEO	Wins	Speedup
SR $\times 4$	$13.44\pm 2.20$	$\mathbf{25.43\pm 3.08}$	$.866\pm.083$	$\mathbf{.241\pm.102}$	$20.88\pm.01$	$\mathbf{7.55\pm.32}$	$200/200$	$2.77\times$
SR $\times 8$	$10.01\pm 1.73$	$\mathbf{22.27\pm 3.00}$	$.808\pm.081$	$\mathbf{.537\pm.126}$	$20.87\pm.02$	$\mathbf{5.59\pm.17}$	$197/200$	$3.73\times$
Gaussian blur	$23.44\pm 3.02$	$\mathbf{23.81\pm 2.80}$	$.510\pm.146$	$\mathbf{.397\pm.088}$	$43.35\pm.05$	$\mathbf{6.24\pm.21}$	$168/200$	$6.95\times$
Center inpainting	$17.04\pm 2.34$	$\mathbf{18.65\pm 2.42}$	$.396\pm.090$	$\mathbf{.193\pm.030}$	$20.80\pm.01$	$\mathbf{7.75\pm.32}$	$199/200$	$2.68\times$

Appendix

Table 22: Exact-budget comparison on the same 200 COCO-val2017 images at 27 denoiser evaluations. PSNR, LPIPS and sampler time are mean

\pm

sample standard deviation. “Wins” is the number of paired images on which PASEO has lower LPIPS; PASEO is faster on all 800 pairs.

		RV-v5.1					SD3.5-medium
Task	Method	PSNR $\uparrow$	LPIPS $\downarrow$	KID $\downarrow$	Time $\downarrow$	Mem $\downarrow$	PSNR $\uparrow$	LPIPS $\downarrow$	KID $\downarrow$	Time $\downarrow$	Mem $\downarrow$
SR $\times 4$	PASEO	27.98	0.139	1.81	11.7	3.6	29.92	0.182	10.27	10.6	16.8
	PASEO †	28.15	0.130	2.96	14.4	3.6	28.76	0.221	9.49	25.9	16.8
	SILO	27.03	0.182	5.10	149	7.4	29.28	0.303	29.36	20	21.2
	ReSample	24.62	0.433	25.50	1418	7.0	30.35	0.182	7.70	130.4	22.5
	PSLD	28.23	0.249	10.11	390	7.9	22.88	0.494	47.23	47	23.4

Appendix

Table 23: FFHQ restoration. Left: PASEO with RV-v5.1 and published baselines; right: SD3.5-medium, baselines at

79

NFEs and PASEO at

53

NFEs. Bold/underline mark best/second-best within each block. KID is scaled by

10^{3}

. PASEO † uses the original anchor of Algorithm 3 .

Prior / data	Task	Changes from PASEO
RV-v5.1 / FFHQ	SR $\times 4$	$P$ : $3\to 4$ , $\rho$ : $0.15\to 0.3$
	SR $\times 8$	$C$ : $2\to 3$ , $\rho$ : $0.15\to 0.6$
	Gaussian blur	$C$ : $3\to 32$
	Inpainting	fixed weight
	JPEG	$P$ : $4\to 1$ , $C$ : $1\to 16$ , $\rho$ : $0.3\to 0.6$
SD3.5 / FFHQ	SR $\times 4$	$P$ : $2\to 3$ , $C$ : $2\to 8$ , $r$ : $0.01\to 0.005$

Appendix

Table 24: Settings of PASEO † that differ from PASEO (Tables 6 and 7 ); all other settings, including the reverse steps, are unchanged. Fixed weight is the inpainting rule of Appendix F.6.1 .

Task	Method	NFE $\downarrow$	PSNR $\uparrow$	LPIPS $\downarrow$	KID $\downarrow$	Time $\downarrow$	Mem $\downarrow$
SR $\times 4$	PASEO (arc)	97	26.69	0.208	1.14	26.4	16.8
	PASEO †	97	26.76	0.196	0.73	27.8	16.8
	ReSample	145	27.23	0.219	1.77	208.9	22.5
	DAPS	145	23.01	0.681	29.65	233.4	18.7
	RedDiff	145	23.91	0.387	9.23	51.2	18.7
	FlowDPS	145	26.31	0.380	12.38	115.7	18.7

Appendix

Table 25: Full COCO-val2017 comparison supporting Table 4.2 ,

n{=}1000

(canonical

0

–

999

), SD3.5-medium with LSDIR-trained latent operators, reported from the arc anchor ( interp_arc_std ) run on our shipped configurations. Every method runs at the NFE listed in the table. KID is

\times 10^{3}

, Time is s/image, Mem is GiB. LPIPS is computed over the whole image on every task, inpainting included, matching Table 1 ; as there, our inpainting row is never given the mask and restricts the likelihood with the learned operator’s sensitivity weight (Appendix F.6.1 ). Baselines are given the true forward operator while ours has only a learned latent surrogate. PSLD is omitted for JPEG because its update requires a linear operator adjoint (Appendix G.3 ). PASEO † uses the original anchor of Algorithm 3 , at the same NFEs as PASEO .

Figure 12: SR

\times 4

with

\sigma_{{\mathbf{y}}}=0.01

, FFHQ, RV-v5.1 prior. Below each row, the region boxed on

{\mathbf{x}}

in every column. PASEO is from the runs of Table 23 . The other baselines’ rows there quote SILO’s published numbers, so their panels come from our own runs (Appendix I ).

Figure 13: SR

\times 4

with

\sigma_{{\mathbf{y}}}=0.01

, FFHQ, SD3.5-medium prior. Below each row, the region boxed on

{\mathbf{x}}

in every column. Panels are from the runs of Table 23 .

Figure 14: SR

\times 8

with

\sigma_{{\mathbf{y}}}=0.01

, FFHQ, RV-v5.1 prior. Below each row, the region boxed on

{\mathbf{x}}

in every column. PASEO is from the runs of Table 23 . The other baselines’ rows there quote SILO’s published numbers, so their panels come from our own runs (Appendix I ).

Figure 15: SR

\times 8

with

\sigma_{{\mathbf{y}}}=0.01

, FFHQ, SD3.5-medium prior. Below each row, the region boxed on

{\mathbf{x}}

in every column. Panels are from the runs of Table 23 .

Figure 16: Gaussian blur with

\sigma_{{\mathbf{y}}}=0.01

, FFHQ, RV-v5.1 prior. Below each row, the region boxed on

{\mathbf{x}}

in every column. PASEO is from the runs of Table 23 . The other baselines’ rows there quote SILO’s published numbers, so their panels come from our own runs (Appendix I ).

Figure 17: Gaussian blur with

\sigma_{{\mathbf{y}}}=0.01

, FFHQ, SD3.5-medium prior. Below each row, the region boxed on

{\mathbf{x}}

in every column. Panels are from the runs of Table 23 .

Figure 18: Box inpainting with

\sigma_{{\mathbf{y}}}=0.01

, FFHQ, RV-v5.1 prior. Below each row, the region boxed on

{\mathbf{x}}

in every column. PASEO and DING are from the runs of Table 23 . The other baselines’ rows there quote SILO’s published numbers, so their panels come from our own runs (Appendix I ). In those runs ReSample does not fill the box, so its column is worse than its published row.

Figure 19: Box inpainting with

\sigma_{{\mathbf{y}}}=0.01

, FFHQ, SD3.5-medium prior. Below each row, the region boxed on

{\mathbf{x}}

in every column. Panels are from the runs of Table 23 .

Figure 20: JPEG (

q{=}10

) with

\sigma_{{\mathbf{y}}}=0.01

, FFHQ, RV-v5.1 prior. Below each row, the region boxed on

{\mathbf{x}}

in every column. PASEO is from the runs of Table 23 . The other baselines’ rows there quote SILO’s published numbers, so their panels come from our own runs (Appendix I ).

Figure 21: JPEG (

q{=}10

) with

\sigma_{{\mathbf{y}}}=0.01

, FFHQ, SD3.5-medium prior. Below each row, the region boxed on

{\mathbf{x}}

in every column. Panels are from the runs of Table 23 .

Figure 22: SR

\times 4

with

\sigma_{{\mathbf{y}}}=0.01

, COCO, SD3.5-medium prior. Below each row, the region boxed on

{\mathbf{x}}

in every column. Panels are from the runs of Table 25 .

Figure 23: SR

\times 8

with

\sigma_{{\mathbf{y}}}=0.01

, COCO, SD3.5-medium prior. Below each row, the region boxed on

{\mathbf{x}}

in every column. Panels are from the runs of Table 25 .

Figure 24: Gaussian blur with

\sigma_{{\mathbf{y}}}=0.01

, COCO, SD3.5-medium prior. Below each row, the region boxed on

{\mathbf{x}}

in every column. Panels are from the runs of Table 25 .

Figure 25: Box inpainting with

\sigma_{{\mathbf{y}}}=0.01

, COCO, SD3.5-medium prior. Below each row, the region boxed on

{\mathbf{x}}

in every column. Panels are from the runs of Table 25 .

Figure 26: JPEG (

q{=}10

) with

\sigma_{{\mathbf{y}}}=0.01

, COCO, SD3.5-medium prior. Below each row, the region boxed on

{\mathbf{x}}

in every column. Panels are from the runs of Table 25 .

Perturb-and-Solve: Efficient Learned-Operator Conditioning for Latent Diffusion Inverse Problems

Abstract

Figures & tables

Appendix

Diffusion-Based Inverse Problems

A Posterior-Dynamics Framework for Imaging Inverse Problems with Pretrained Diffusion Priors

Dynamic Time Step Prediction in Inverse Heat Dissipation for Blur-Like Image Restoration Tasks

Learning Normalized Energy Models for Linear Inverse Problems

Abstract

Diffusion-Based Inverse Problems

Explore similar work

A Posterior-Dynamics Framework for Imaging Inverse Problems with Pretrained Diffusion Priors

Dynamic Time Step Prediction in Inverse Heat Dissipation for Blur-Like Image Restoration Tasks

Learning Normalized Energy Models for Linear Inverse Problems