LPNSR: Learnable Noise Prediction for Diffusion-Based Image Super-Resolution

Shuwei Huang; Shizhuo Liu; Zijun Wei

cs.CVMar 22, 2026

LPNSR: Learnable Noise Prediction for Diffusion-Based Image Super-Resolution

Authors: Shuwei Huang, Shizhuo Liu, Zijun Wei

Organizations: Huazhong University of Science and Technology

Abstract

Diffusion-based image super-resolution (SR) aims to reconstruct high-resolution (HR) images from low-resolution (LR) observations. A key property of diffusion models is that, once the starting point of the reverse chain and the denoising network are fixed, the quality of the generated image is dictated by the noise maps sampled at the intermediate steps, which are drawn from an unconstrained standard Gaussian distribution in conventional pipelines. This property raises a natural question: does there exist a noise sampling distribution better than the standard Gaussian that improves the quality of the generated images? To this end, we use a parameterized deep neural network to predict the mean and the variance of the Gaussian noise sampling distribution at each intermediate step, and design two training schemes, one supervised by the quality of the final generated image and the other by aligning each reverse step with the forward posterior. Experiments show that, compared with the standard Gaussian distribution, the learnable noise sampling distribution improves the quality of the generated images. The source code of our method can be found at https://github.com/Faze-Hsw/LPNSR.

Figures & tables

Figure 1: Qualitative comparison of SR results generated with different random seeds. (Zoom in for best view)

Figure 2: Overview of the LPNSR inference framework. The noise predictor

\boldsymbol{g}_{\omega}

outputs a diagonal Gaussian

(\boldsymbol{\mu}_{\omega},\boldsymbol{\sigma}_{\omega})

whose sample

\boldsymbol{z}_{t-1}

replaces the unconstrained Gaussian noise in the reverse iteration of Eq. ( 7 ). The reverse process runs from

t=T

t=1

, and the final estimate is decoded by the VQGAN decoder into the SR image.

Panel of Figure 3 — Figure 3: Qualitative comparison of the SR results generated under different noise sampling schemes. (Zoom in for best view)

Dataset	Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	DISTS $\downarrow$	MUSIQ $\uparrow$	MANIQA $\uparrow$	CLIPIQA $\uparrow$	FID $\downarrow$
LSDIR	VOSR	20.04	0.4917	0.2448	0.1524	72.1925	0.5025	0.6303	47.79
	InvSR	19.19	0.4845	0.2657	0.1636	72.7310	0.5113	0.7156	69.30
	OSEDiff	20.03	0.5094	0.2675	0.1627	71.4740	0.4573	0.6946	58.98
	SinSR	20.39	0.5136	0.2281	0.1447	69.1867	0.4648	0.6687	40.82
	ResShift (Origin)	20.70	0.5281	0.2249	0.1420	67.4741	0.4363	0.6445	36.76
	LPNSR-A (Ours)	19.77	0.5140	0.2099	0.1336	71.7904	0.5618	0.7389	42.39

Table 1: Quantitative comparison on the LSDIR and ImageNet datasets. The last three rows of each group report our noise sampling schemes (Origin, LPNSR-A, and LPNSR-B), and the remaining rows list several representative SR methods for reference. The best and second-best results of each metric are highlighted in red and blue , respectively.

Method	PSNR $\uparrow$	LPIPS $\downarrow$	DISTS $\downarrow$	MUSIQ $\uparrow$	CLIPIQA $\uparrow$
Origin	$20.6951\pm 0.0066$	$0.2239\pm 0.0009$	$0.1419\pm 0.0003$	$67.5117\pm 0.0617$	$0.6436\pm 0.0024$
PreSet-A	$19.7679\pm 0.0017$	$0.2103\pm 0.0002$	$0.1339\pm 0.0002$	$71.8248\pm 0.0269$	$0.7393\pm 0.0008$
PreSet-B	$20.7975\pm 0.0072$	$0.2195\pm 0.0007$	$0.1394\pm 0.0003$	$68.5293\pm 0.0640$	$0.6726\pm 0.0019$

Table 2: Statistics of the three noise sampling schemes on the LSDIR dataset. We randomly sample 100 random seeds; for each seed, every metric is averaged over the test images to obtain a per-seed mean, and the mean and the standard deviation of these per-seed means across the 100 seeds are reported as mean

\pm

std.

Method	$t=4$			$t=3$			$t=2$
Method	$D_{KL}^{\text{mean}}$	$D_{KL}^{\text{var}}$	$\bar{\sigma}_{\omega}^{2}$	$D_{KL}^{\text{mean}}$	$D_{KL}^{\text{var}}$	$\bar{\sigma}_{\omega}^{2}$	$D_{KL}^{\text{mean}}$	$D_{KL}^{\text{var}}$	$\bar{\sigma}_{\omega}^{2}$
Origin	0.0611	0.0000	1.0000	0.1782	0.0000	1.0000	9.2725	0.0000	1.0000
PreSet-A	0.8988	5.8690	$1.47\times 10^{-4}$	0.4707	6.1599	$6.97\times 10^{-5}$	15.1685	5.4199	$2.49\times 10^{-4}$
PreSet-B	0.0577	0.0000	1.0000	0.1710	0.0000	1.0000	9.2190	0.0000	1.0000

Table 3: Decomposition of the KL divergence between the forward posterior and the model reverse distribution at each intermediate step (

t=4,3,2

); all values are computed per image and then averaged over the test images of the LSDIR dataset. Per Eq. ( 17 ),

D_{KL}

is split into a mean term

D_{KL}^{\text{mean}}

and a variance term

D_{KL}^{\text{var}}

, where a smaller value indicates a better alignment, and

\bar{\sigma}_{\omega}^{2}

denotes the mean injected noise variance, i.e., the mean of the diagonal entries of

\boldsymbol{\Sigma}_{\omega}

Method	Hyperparameters		Metrics
Method	$\lambda_{l}$ (LPIPS loss)	$\lambda_{g}$ (GAN loss)	PSNR $\uparrow$	LPIPS $\downarrow$	DISTS $\downarrow$	MUSIQ $\uparrow$	CLIPIQA $\uparrow$	$D_{KL}^{\text{mean}}$ $\downarrow$	$\bar{\sigma}_{\omega}^{2}$
A-Baseline1	0.0	0.0	22.05	0.2683	0.1592	64.2017	0.6184	8.8653	$1.93\times 10^{-4}$
A-Baseline2	1.0	0.0	20.35	0.2155	0.1372	70.4506	0.7180	12.3537	$2.56\times 10^{-4}$
A-Baseline3	0.0	0.1	19.28	0.2230	0.1428	73.0554	0.7495	17.0581	$2.91\times 10^{-4}$
B-Baseline1	0.0	\	21.55	0.2602	0.1571	64.3558	0.6202	8.9522	1.0000
PreSet-A	1.0	0.1	19.77	0.2099	0.1336	71.7904	0.7389	15.1685	$2.49\times 10^{-4}$

Table 4: Quantitative ablation studies on the loss functions used for training PreSet-A and PreSet-B, wherein the hyperparameters

\lambda_{l}

and

\lambda_{g}

control the weight importance of the LPIPS loss and the GAN loss, respectively. The results are evaluated on the LSDIR dataset under the 4-step sampling setting, with the last two columns measured at the intermediate step

t=2

Panel of Figure 4 — Figure 4: Qualitative comparison with other methods. (Zoom in for best view)

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Hyperparameter	Value
Base channels	160
Channel multipliers	$(1,2,2,4)$
Residual blocks per resolution	2
Attention resolutions	$64,32,16,8$
Channels per attention head	32
Scale-shift normalization	Enabled

Appendix

Table 5: Architecture hyperparameters of the noise predictor

\boldsymbol{g}_{\omega}

Hyperparameter	Value
Training iterations	200k
Batch size	16
Optimizer	AdamW
Learning rate	$5\times 10^{-5}$
Weight decay	0
LR schedule	None

Appendix

Table 6: Training hyperparameters of the noise predictor.

Method	#Params (M) $\downarrow$	FLOPs (G) $\downarrow$	Latency (ms) $\downarrow$	GPU Memory Usage (MB) $\downarrow$
Origin	173.92	5410.3	321.8	1968
PreSet-A	292.52	7032.3	442.1	2432
PreSet-B	292.51	7031.8	438.5	2432

Appendix

Table 7: Inference efficiency of the three noise sampling schemes on the LSDIR dataset (

128\times 128

\rightarrow

512\times 512

SR), measured on a single NVIDIA RTX 5090 GPU with a batch size of 1 and mixed precision enabled. The number of parameters and the FLOPs are reported per image, and the latency and the GPU memory usage are averaged over all test images.

Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	DISTS $\downarrow$	MUSIQ $\uparrow$	MANIQA $\uparrow$	CLIPIQA $\uparrow$
Deterministic noise map	19.73	0.5174	0.2315	0.1456	70.0219	0.4491	0.6452
Noise sampling distribution	19.77	0.5140	0.2099	0.1336	71.7904	0.5618	0.7389

Appendix

Table 8: Ablation on the form of the noise predictor under the end-to-end supervised training scheme, i.e., predicting the mean and the variance of the noise sampling distribution versus directly predicting a deterministic noise map. The results are evaluated on the LSDIR dataset under the 4-step sampling setting.

Panel of Figure 5 — Figure 5: Additional qualitative comparison with other methods. (Zoom in for best view)

Panel of Figure 6 — Figure 6: Additional qualitative comparison with other methods. (Zoom in for best view)

Image Super-Resolution

Image super-resolution reconstructs high-resolution images from lower-resolution observations, balancing detail, fidelity, and computational cost.

See all 248 papers

DNF-SR: Dual-Input and Negative-Aware Feature Fine-Tuning for Real-World Image Super-Resolution
Sep 14, 2026Shuhao Han, Wenjie Liao, Hayden Vance +4Diffusion Models for Image Restoration Diffusion Model Fine-Tuning
VCIP, CS, Nankai University · Independent Researcher
QUSR: Quality-Aware and Uncertainty-Guided Image Super-Resolution Diffusion Model
Mar 10, 2026Junjie Yin, Jiaju Li, Hanfa XingImage Super-Resolution Diffusion Models for Image Restoration
School of BeiDou Research Institute, South China Normal University, China
Noise-Started One-Step Real-World Super-Resolution via LR-Conditioned SplitMeanFlow and GAN Refinement
May 10, 2026Wei Zhu, Kai Zhang, Yu Zheng +3Flow Matching Diffusion Model Distillation
Nanjing University of Science and Technology · Nanjing University · Huawei

Sep 14, 2026J/K move · Enter open · S save

DNF-SR: Dual-Input and Negative-Aware Feature Fine-Tuning for Real-World Image Super-Resolution

Shuhao Han, Wenjie Liao, Hayden Vance, Hang Dong+3

VCIP, CS, Nankai University · Independent Researcher

Diffusion Models for Image Restoration Diffusion Model Fine-Tuning Image Super-Resolution

Open paper AskPDF arXiv

Benefiting from the powerful generative priors of diffusion models, diffusion-based real-world image super-resolution (Real-ISR) methods have demonstrated impressive performance.To achieve efficient Real-ISR, several recent works have designed one-step diffusion-based models.Howerver, unmediatedly feeding LR into a diffusion model creates a distributional gap with the model's original input.A straightforward approach to reduce the distribution gap is to introduce noise to the LR latents. However, directly adding noise inevitably corrupts the content of the LR images.In this study, we propose DNF-SR, a Dual-input and Negative-aware Feature fine-tuning method for Real-ISR.Specifically, we use a dual-input strategy that concatenates the original LR image with the noisy LR input and feeds them into a diffusion-based image editing model, ensuring both high-fidelity one-step super-resolution and improved perceptual and content consistency.Additionally, the noise present in the noisy LR input introduces randomness and diversity into the outputs. We exploit this property and propose a post-training optimization method, Negative-aware Feature Fine-Tuning (NF2T), which guides the model toward producing higher-quality results.NF^2T classifies multiple outputs into positive and negative subsets and then defines implicit policy improvement directions in both the image and feature spaces, thereby further enhancing the stability of the optimization.Extensive experiments show that DNF-SR outperforms other methods.Code will be released.

Abstract

Image Super-Resolution

Explore similar work

DNF-SR: Dual-Input and Negative-Aware Feature Fine-Tuning for Real-World Image Super-Resolution

QUSR: Quality-Aware and Uncertainty-Guided Image Super-Resolution Diffusion Model

Noise-Started One-Step Real-World Super-Resolution via LR-Conditioned SplitMeanFlow and GAN Refinement