The perception-distortion trade-off poses a fundamental challenge in single-image super-resolution (SR). Although diffusion-based SR methods excel at generating perceptually realistic images, achieving high fidelity remains a key limitation. Recent advances in diffusion-based SR have shown promise in improving fidelity, but these methods often compromise perceptual quality due to their high reliance on a high-fidelity image. To address this, we introduce UGDiff, a novel diffusion guidance paradigm designed to further improve the perception-distortion balance. In particular, we first estimate the reconstruction uncertainty of the latent features corresponding to a high-fidelity image. This uncertainty is then used to guide the diffusion process to selectively restore high-frequency details in high-uncertainty regions, while preserving fidelity elsewhere. Furthermore, our guidance method adaptively identifies the high-uncertainty regions by considering not only the estimated uncertainty but also the posterior variance of the diffusion sampler at each timestep. This relaxes the reliance on the high-fidelity image in the later stages of sampling, thereby achieving a better perception-distortion balance. Extensive experimental results demonstrate that our method performs favorably against state-of-the-art diffusion-based SR methods.
Figures & tables
Figure 1: Perception-Distortion trade-off of diffusion-based SR methods on RealSR. The DiffBIR curve corresponds to various values of s , while ours varies γ at a fixed s=100 .
Figure 2: Overview of our method. Given a high-fidelity image Irm restored from the low-quality image Ilq by a regression-based restoration model R , our uncertainty estimation model E′ produces a variance map Σu specifically for μ=E(Irm) . During the reverse process of a pretrained latent diffusion model, we rectify μθ(zt,z^0∣t) , i.e . , the mean of the posterior distribution p(zt−1∣zt,z^0∣t) , at each timestep t . Our guidance aims to push μθ(zt,z^0∣t) to the direction of μ according to the uncertainty at each pixel. Repeating this process, we get the final restored image Ihq at timestep 0 , which is expected to have plentiful details in textured regions while remaining robust against hallucinations and artifacts elsewhere.
Figure 3: Comparison of 4× upsampling on DIV2K-Val (upper) and RealSR (bottom). We set s=100 for all of our methods.
Dataset
Method
PSNR ↑
SSIM ↑
LPIPS ↓
NIQE ↓
DIV2K-Val
StableSR [ 5 ]
21.6100
0.5267
0.3113
4.7581
ResShift [ 4 ]
22.6640
0.5884
0.3077
6.9142
SeeSR [ 17 ]
21.9703
0.5592
0.3194
4.8097
FaithDiff [ 12 ]
21.7965
0.5348
0.3118
4.9149
SUPIR [ 9 ]
21.4792
0.5005
0.3625
6.3373
PiSA-SR [ 11 ]
22.2189
0.5624
0.2823
4.5577
Table 1: Quantitative comparisons of 4× upsampling. Underline indicates superior performance between DiffBIR and ours at comparable PSNR levels. Red and blue respectively indicate the best and the second best performance among all methods. We set s=100 for all of our methods.
Figure 4: Relative ratio of weighting over time under different uncertainties σu2 . The ratio is defined by ρ=σt2/(αˉt−1σu2) . Note that the curves are depicted using the scaled linear noise scheduler with β1=8.5×10−4 and βT=1.2×10−2 .
Irm
PSNR ↑
SSIM ↑
LPIPS ↓
NIQE ↓
BSRNet(Ilq)
20.2307
0.4222
0.4388
4.6974
D(Enll(Ilq))
19.5111
0.3844
0.4854
5.6106
Table 2: Ablation study on mean estimation. Bold indicates superior performance. We conduct the experiments using DiffBIR [ 8 ] without guidance ( s=0 ).
s
γ
PSNR ↑
SSIM ↑
LPIPS ↓
NIQE ↓
1
1.0
20.2278
0.4222
0.4387
4.6801
10
1.0
20.2040
0.4219
0.4371
4.5573
100
1.0
19.8139
0.4113
0.4271
3.7287
200
1.0
18.9496
0.3779
0.4628
4.0655
100
1.0
19.8139
0.4113
0.4271
3.7287
100
0.5
20.2100
0.4340
0.4185
3.8541
Table 3: Ablation study on guidance parameters. Bold indicates the best performance.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Param.
PSNR ↑
SSIM ↑
LPIPS ↓
NIQE ↓
σu2
20.7987
0.4676
0.4118
4.3496
σu
20.9637
0.4780
0.4095
4.5425
Appendix
Table 4: Ablation study on parameterization. Bold indicates the better performance. We set s=100 and γ=0.1 for the experiments.
Figure 5: Visual comparison under different uncertainty scales on DIV2K-Val (upper) and RealSR (bottom). We set s=100 for all of our methods.
Figure 6: Visualization and analysis of the sampling process with and w/o our guidance. We set s=100 and γ=0.1 for the guidance.
Figure 7: Qualitative comparison of 4× upsampling on DRealSR. We set s=100 for all of our methods.
Figure 8: Qualitative comparison of 4× upsampling on DIV2K-Val. We set s=100 for all of our methods.