Pre-trained diffusion models have become a widely adopted zero-shot prior for image inverse problems, among which annealed decoupling methods achieve leading reconstruction performance on nonlinear tasks. We find, however, that annealed methods that directly optimize the denoiser input xt exhibit radial drift in later layers: within-layer optimization drives the noise-component norm of xt away from its typical value, pushing the denoiser out of its high-performance regime and degrading reconstruction quality. To address this, we propose FANO (Anchored Forward-Noise Optimization). At each annealing layer, FANO fixes the previous denoiser estimate as an anchor and optimizes only the forward noise e, whose distribution is known, instead of xt, thereby preserving the forward-diffusion structure of the denoiser input. The standard Gaussian prior on e concentrates its norm near the typical value, and deviations degrade denoiser performance. We accordingly decompose e in polar coordinates into a direction and a radial magnitude, and apply a soft potential to regulate the radius, keeping xt in the high-density region of the forward distribution. On multiple linear and nonlinear inverse problems, FANO matches the strongest current baselines on linear tasks and achieves consistent improvements on nonlinear tasks.
Figures & tables
Figure 1: Visual comparison on FFHQ and ImageNet across four inverse problems ( σy=0.05 ): (a) nonlinear deblurring, (b) phase retrieval, (c) random inpainting, (d) high dynamic range (HDR). Each panel shows the measurement, DAPS, SITCOM, FANO, and the reference. Metrics below each panel are averaged over the two examples, with the best in bold ; FANO recovers sharper structures and finer textures across all tasks, with the advantage most pronounced on nonlinear problems.
Figure 2: Radial drift on FFHQ ( σy=0.05 , N=20 ). (a) Mean PSNR and normalized radius r on FFHQ100 for SITCOM ( Alkhouri et al., 2025 ) and FANO; insets magnify late-layer PSNR, and shading marks layers after SITCOM’s peak. (b) Denoiser PSNR drop relative to r=1 when varying the radius along fixed random noise directions. (c) SITCOM reconstructions at layers 2, 18, and 20, illustrating late-layer degradation as r increases.
Figure 3: Overview of FANO. (a) Blue: direct optimization of xt ; red: FANO fixes the anchor xˉ=x^k−1 and optimizes only the forward noise e . Circular diagrams contrast radial drift (blue) with controlled search (red). (b) The radial potential Φd(r) is minimized at r=1 , softly penalizing deviations from the typical noise shell. FANO decomposes e=ru and jointly optimizes direction u and magnitude r .
FFHQ
ImageNet
Task
Method
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
SR ×4
DAPS
29.35 ± 1.47
0.782 ± 0.028
0.268 ± 0.035
25.47 ± 2.86
0.638 ± 0.087
0.376 ± 0.061
SITCOM
30.55 ± 1.87
0.868 ± 0.033
0.223 ± 0.034
27.09 ± 3.84
0.757 ± 0.116
0.307 ± 0.077
FAST-DIPS
29.59 ± 1.75
0.843 ± 0.041
0.241 ± 0.042
26.55 ± 3.65
0.725 ± 0.125
0.325 ± 0.072
DiffRGD
29.24 ± 1.93
0.838 ± 0.042
0.214 ± 0.037
25.96 ± 3.70
0.712 ± 0.136
0.322 ± 0.084
INO
29.24 ± 1.80
0.809 ± 0.037
0.260 ± 0.038
25.65 ± 3.50
0.694 ± 0.131
0.310 ± 0.090
Table 1: Quantitative results at σy=0.05 , reported as the mean ± standard deviation over 100 test images. The best results are in bold , and the second-best results are underlined .
Component
HDR
NL deblur
Variant
Polar
Radial freedom
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
FANO
✓
✓
29.88
0.878
0.157
30.83
0.873
0.183
FANO-Sphere
✓
×
29.06
0.858
0.167
30.72
0.870
0.183
FANO-Euc
×
✓
29.18
0.859
0.171
30.80
0.871
0.182
Table 2: Ablation of the polar parameterization and radial freedom (FFHQ100, σy=0.05 ). The three variants share the e -search framework ( N , K , and residual-based early stopping) and differ in two design choices: Polar (the polar decomposition e=ru ) and Radial freedom (allowing r to vary freely; otherwise, r≡1 is fixed). The best results are in bold .
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Task
Type
Parameters
SR ×4
Linear
Bicubic, factor 4
Gauss. deblur
Linear
61×61 Gaussian, σblur=3.0
Motion deblur
Linear
61×61 motion, intensity 0.5
Inpaint (box)
Linear
128×128 random mask
Inpaint (rand.)
Linear
Pixel mask prob. 0.7
Phase retrieval
Nonlinear
Oversampling ratio 1.5
Appendix
Table 3: Degradation-operator parameters for each task.
Task
Annealing layers N
Inner steps K
λrad
SR ×4
50
30
0
Gauss. deblur
50
30
0.0015
Motion deblur
50
30
0.01
Inpaint (box)
20
20
0
Inpaint (rand.)
20
30
1
Phase retrieval
50
30
0.01
Appendix
Table 4: Annealing configurations and radial regularization coefficients used in the main FANO experiments on FFHQ and ImageNet ( σy=0.05 ).
FFHQ
ImageNet
Task
SITCOM
DAPS
Sphere
SITCOM
DAPS
Sphere
SR ×4
30.55
29.35
30.57
27.09
25.47
27.14
Gauss. deblur
29.89
29.61
30.66
26.38
25.88
27.01
Motion deblur
32.37
31.65
33.03
29.38
28.65
29.89
Inpaint (box)
24.47
24.32
25.14
19.62
21.05
20.17
Inpaint (rand.)
32.40
30.72
34.06
28.90
27.35
29.90
Appendix
Table 5: PSNR comparison of FANO-Sphere ( r≡1 , without tuning λrad ) against annealed baselines ( σy=0.05 ). The best result for each combination is in bold .
Figure C.1: Sensitivity to λrad on three representative tasks (the first 30 FFHQ images, σy=0.05 ). The top row shows PSNR and the bottom row shows LPIPS (VGG16), both averaged over images. Dashed lines and open circles mark the default value used for each task in the main text.
FFHQ
ImageNet
Task
Method
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
SR ×4
SITCOM-TB
30.61
0.869
0.226
27.11
0.759
0.311
FANO
30.68
0.872
0.232
27.03
0.755
0.321
Gauss. deblur
SITCOM-TB
30.09
0.854
0.244
26.56
0.732
0.337
FANO
30.65
0.868
0.225
27.01
0.746
0.317
Motion deblur
SITCOM-TB
32.62
0.902
0.187
29.68
0.838
0.231
Appendix
Table 6: Matched-budget comparison of SITCOM-TB and FANO at σy=0.05 . For each task, both methods use the same number of annealing layers N , inner steps K , residual-based early-stopping criterion, and number of candidate runs. The best results are in bold .
Task
DAPS
FANO
SR ×4
83
74
Gauss. deblur
68
88
Motion deblur
68
88
Inpaint (box)
46
86
Inpaint (rand.)
46
34
Phase retrieval
107
87
Appendix
Table 7: Average core reconstruction time (seconds) of FANO and DAPS on FFHQ, executed sequentially on a single A100.
FFHQ
ImageNet
Task
Config
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
SR ×4
N=50,K=30
30.68
0.872
0.232
27.03
0.755
0.321
N=20,K=75
30.64
0.869
0.226
27.10
0.756
0.315
Gauss. deblur
N=50,K=30
30.65
0.868
0.225
27.01
0.746
0.317
N=20,K=75
30.55
0.866
0.217
27.08
0.750
0.306
Motion deblur
N=50,K=30
33.04
0.909
0.167
29.88
0.840
0.225
Appendix
Table 8: Results for two FANO configurations on FFHQ100 and ImageNet100 ( σy=0.05 ): N=50,K=30 and N=20,K=75 , both with a maximum NFE of 1500 . The best metric for each task and dataset is in bold .
Figure D.1: FANO reconstruction performance under different NFE budgets. The left column shows motion deblurring and the right column nonlinear deblurring; the top row shows PSNR ( ↑ ) and the bottom row LPIPS ( ↓ ). Each curve corresponds to a number of annealing layers N , with total NFE ( =N×K ) on the horizontal axis.
Figure E.1: Reconstruction performance of FANO and DAPS at different measurement noise levels σy . The left column shows motion deblurring and the right column nonlinear deblurring; the top row shows PSNR ( ↑ ) and the bottom row LPIPS ( ↓ ). FANO uses N=50,K=30 and DAPS uses its recommended 1k configuration. All results are averaged over 20 FFHQ images.
Figure E.2: Phase retrieval without oversampling ( OS=1 , noiseless) on 100 FFHQ images. (a) LPIPS as a function of per-image wall-clock time on a single A100: solid curves show FANO and DAPS under varying compute budgets; dashed lines mark HIO at three oversampling rates. (b) Three representative examples comparing HIO ( OS=1 ), HIO ( OS=2 ), DAPS, FANO, and the reference. At roughly 300 s per image, FANO reaches perceptual quality comparable to HIO at OS=2 , recovering clean structures without any oversampling.
Figure F.1: Super resolution ( left ) and Gaussian deblurring ( right ) results. First (resp. last) three rows are for the FFHQ (resp. ImageNet) dataset.
Figure F.3: Box inpainting ( left ) and random inpainting ( right ) results. First (resp. last) three rows are for the FFHQ (resp. ImageNet) dataset.