GDPO-SR: Group Direct Preference Optimization for One-Step Generative Image Super-Resolution

Qiaosi Yi; Shuai Li; Rongyuan Wu; Lingchen Sun; Zhengqiang Zhang; Lei Zhang

cs.CVMar 16, 2026

GDPO-SR: Group Direct Preference Optimization for One-Step Generative Image Super-Resolution

Authors: Qiaosi Yi, Shuai Li, Rongyuan Wu, Lingchen Sun, Zhengqiang Zhang, Lei Zhang

Organizations: The Hong Kong Polytechnic University · OPPO Research Institute

Abstract

Recently, reinforcement learning (RL) has been employed for improving generative image super-resolution (ISR) performance. However, the current efforts are focused on multi-step generative ISR, while one-step generative ISR remains underexplored due to its limited stochasticity. In addition, RL methods such as Direct Preference Optimization (DPO) require the generation of positive and negative sample pairs offline, leading to a limited number of samples, while Group Relative Policy Optimization (GRPO) only calculates the likelihood of the entire image, ignoring local details that are crucial for ISR. In this paper, we propose Group Direct Preference Optimization (GDPO), a novel approach to integrate RL into one-step generative ISR model training. First, we introduce a noise-aware one-step diffusion model that can generate diverse ISR outputs. To prevent performance degradation caused by noise injection, we introduce an unequal-timestep strategy to decouple the timestep of noise addition from that of diffusion. We then present the GDPO strategy, which integrates the principle of GRPO into DPO, to calculate the group-relative advantage of each online generated sample for model optimization. Meanwhile, an attribute-aware reward function is designed to dynamically evaluate the score of each sample based on its statistics of smooth and texture areas. Experiments demonstrate the effectiveness of GDPO in enhancing the performance of one-step generative ISR models. Code: https://github.com/Joyies/GDPO.

Figures & tables

Figure 1: Noise regulates the diversity of generated samples. Different noise inputs yield both high-quality samples ( e.g . , noise1) and low-quality ones ( e.g . , noise2 and noise3). After preference learning, the model produces more visually pleasing results.

Figure 2: The framework of GDPO, which consists of two core stages: (a) advantage calculation and (b) policy optimization. Firstly, we employ a pre-trained one-step Real-ISR model as the reference model to generate a group of diverse outputs by injecting different random noises. Subsequently, we compute the advantage $\mathcal{A}$ for each sample by evaluating its reward with our designed attribute-aware reward functions and converting these rewards into group-relative advantages. In the policy optimization stage, we feed these samples along with noises into both the policy model and the reference ISR model, and update the parameters of the policy ISR model by minimizing the proposed GDPO loss, steering it to favor generating high-reward samples.

Figure 3: The structure of NAOSD, which uses the $t_{add}$ to control the intensity of injected noise.

Figure 4: The pipeline of calculating smooth and detailed regions.

Dataset	Method	PSNR↑	SSIM↑	LPIPS↓	FID↓	DISTS↓	MANIQA↑	MUSIQ↑	CLIPIQA↑	AFINE↓
	NAOSD	27.97	0.7831	0.2859	140.37	0.2113	0.6041	65.17	0.6886	20.52
DrealSR	GDPO-SR	28.18	0.7839	0.2851	138.87	0.2112	0.6180	65.63	0.7020	18.72
	NAOSD	25.25	0.7346	0.2689	114.91	0.2001	0.6459	69.06	0.6617	18.90
RealSR	GDPO-SR	25.48	0.7328	0.2675	112.13	0.1980	0.6615	69.42	0.6760	17.73
	NAOSD	23.83	0.6131	0.2848	25.16	0.1922	0.6297	68.20	0.6758	41.76
DIV2K-val	GDPO-SR	23.92	0.6117	0.2897	26.44	0.1965	0.6423	68.80	0.6929	41.56

Table 1: Performance comparison with the base model NAOSD on real-world and synthetic datasets. Metrics with a blue background denote those utilized in the reward function, while yellow-shaded ones correspond to metrics that are excluded from it. Arrows denote if higher (↑) or lower (↓) values represent better performance. The best results are highlighted in red .

Method	PSNR↑	SSIM↑	LPIPS↓	FID↓	DISTS↓	MANIQA↑	MUSIQ↑	CLIPIQA↑	AFINE↓
StableSR	28.03	0.7536	0.3284	148.98	0.2269	0.5592	58.51	0.6356	35.77
DiffBIR	26.71	0.6571	0.4557	166.79	0.2748	0.5927	61.07	0.6395	39.42
SeeSR	28.07	0.7684	0.3174	147.39	0.2315	0.6054	65.08	0.6905	22.79
PASD	27.36	0.7073	0.3760	156.13	0.2531	0.6160	64.87	0.6808	33.89
OSEDiff	27.92	0.7835	0.2968	135.30	0.2165	0.5899	64.65	0.6963	21.39
InvSR	25.79	0.7176	0.3471	166.42	0.2381	0.6212	64.92	0.6918	21.69

Table 2: Quantitative comparison with different methods on real-world and synthetic datasets. The best and second best results are highlighted in red and blue , respectively. Arrows denote if higher (↑) or lower (↓) values represent better performance.

Dataset	Method	PSNR↑	SSIM↑	LPIPS↓	FID↓	DISTS↓	MANIQA↑	MUSIQ↑	CLIPIQA↑	AFINE↓
	DP 2 O-SR	23.95	0.6294	0.4412	157.76	0.2763	0.6674	71.09	0.7842	11.39
DrealSR	GDPO-SR	28.18	0.7839	0.2851	138.87	0.2112	0.6180	65.63	0.7020	18.72
	DP 2 O-SR	22.39	0.6467	0.3513	131.79	0.2442	0.7036	73.24	0.7484	8.654
RealSR	GDPO-SR	25.48	0.7328	0.2675	112.13	0.1980	0.6615	69.42	0.6760	17.73

Table 3: Quantitative comparison with DP 2 O-SR on real-world datasets. The best results are highlighted in red .

Figure 5: Visual comparison with SD-based Real-ISR methods. Please zoom in for a better view.

Methods	StableSR	DiffBIR	SeeSR	PASD	OSEDiff	InvSR	GDPO-SR
Para.(B)	1.56	1.68	2.51	2.31	1.77	1.33	1.77
Time(s)	10.03	2.72	4.30	2.80	0.11	0.12	0.11
FLOPs(T)	79.94	24.31	65.86	29.13	2.27	2.40	2.27

Table 4: Comparison of mode size, running-time and FLOPs.

Figure 6: Visual comparison with DP 2 O-SR.

Method	PSNR↑	FID↓	DISTS↓	MUSIQ↑	AFINE↓
NAOSD	25.25	114.91	0.2001	69.06	20.52
Diffusion-DPO	25.41	112.87	0.2010	69.16	20.22
DanceGRPO	25.10	113.74	0.2049	69.95	16.52
GDPO (ours)	25.48	112.13	0.1980	69.42	18.72

Table 5: Ablation studies on GDPO on the RealSR dataset.

Method	LPIPS↓	DISTS↓	MUSIQ↑	CLIPIQA↑
NAOSD	0.2689	0.2001	69.06	0.6617
ARF w/ FR	0.2642	0.1978	67.68	0.6359
ARF w/ NR	0.2866	0.2102	69.80	0.6914
ARF w/o AW	0.2660	0.1979	68.42	0.6331
ARF (ours)	0.2675	0.1980	69.42	0.6760

Table 6: Ablation studies on ARF on the RealSR dataset.

G	PSNR $\uparrow$	DISTS $\downarrow$	MANIQA $\uparrow$	CLIPIQA $\uparrow$
NAOSD	25.25	0.2001	0.6459	0.6617
GDPO-SR (G=4)	25.45	0.1996	0.6491	0.6610
GDPO-SR (G=6)	25.48	0.1980	0.6615	0.6760
GDPO-SR (G=8)	25.34	0.1955	0.6619	0.6788

Table 7: Ablation studies on Group Size on the RealSR dataset.

Figure 7: Ablation studies on the timestep setting in NAOSD.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Method	PSNR↑	SSIM↑	LPIPS↓	FID↓	DISTS↓	MANIQA↑	MUSIQ↑	CLIPIQA↑	AFINE↓
NAOSD	25.27	0.7354	0.2685	116.67	0.1997	0.6458	68.85	0.6620	19.28
GDPO-SR	25.49	0.7336	0.2673	116.03	0.1978	0.6607	69.21	0.6766	18.19

Appendix

Table 8: Performance comparison (averaged over 50 stochastic runs) on the RealSR dataset. Arrows denote if higher (↑) or lower (↓) values represent better performance. The best results are highlighted in red .

FR	PSNR $\uparrow$	DISTS $\downarrow$	MANIQA $\uparrow$	CLIPIQA $\uparrow$
NAOSD	25.25	0.2001	0.6459	0.6617
PSNR	25.48	0.1980	0.6615	0.6760
LPIPS	25.28	0.1997	0.6605	0.6835
PSNR+LPIPS	25.33	0.1970	0.6548	0.6701

Appendix

Table 9: Ablation studies on FR metrics on the RealSR dataset.

Method	PSNR↑	SSIM↑	LPIPS↓	DISTS↓	MANIQA↑	MUSIQ↑	CLIPIQA↑	AFINE↓
GDPO-SR ( $t_{add}$ =150)	25.85	0.7499	0.2541	0.1938	0.6521	68.09	0.6331	19.89
GDPO-SR ( $t_{add}$ =250)	25.48	0.7328	0.2675	0.1980	0.6615	69.42	0.6760	17.73
GDPO-SR ( $t_{add}$ =350)	24.84	0.7074	0.2900	0.2088	0.6646	70.14	0.6994	17.41

Appendix

Table 10: The impact of

t_{add}

on the generation capability of GDPO-SR. Arrows denote if higher (↑) or lower (↓) values represent better performance. The best and second best results are highlighted in red and blue , respectively.

Method	PSNR↑	SSIM↑	LPIPS↓	DISTS↓	MANIQA↑	MUSIQ↑	CLIPIQA↑	AFINE↓
NAOSD	25.25	0.7346	0.2689	0.2001	0.6459	69.06	0.6617	18.90
GDPO-SR (CFG)	25.06	0.7174	0.2782	0.2049	0.6543	69.47	0.6815	19.61
GDPO-SR ( $t_{add}$ )	25.29	0.7294	0.2871	0.2133	0.6477	69.46	0.6766	17.12
GDPO-SR	25.48	0.7328	0.2675	0.1980	0.6615	69.42	0.6760	17.73

Appendix

Table 11: Ablation study on sample generation methods on the Real-ISR dataset. Arrows denote if higher (↑) or lower (↓) values represent better performance.

Datasets	Methods	PSNR↑	SSIM↑	LPIPS↓	FID↓	DISTS↓	MANIQA↑	MUSIQ↑	CLIPIQA↑	AFINE ↓
	RealESRGAN	24.29	0.6371	0.3112	37.64	0.2141	0.5501	61.06	0.5277	56.20
	BSRGAN	24.58	0.6269	0.3351	44.23	0.2275	0.5247	61.20	0.5071	62.21
	LDL	23.83	0.6344	0.3256	42.29	0.2227	0.5350	60.04	0.5180	53.19
DIV2K-val	GDPO-SR	23.92	0.6117	0.2897	26.44	0.1965	0.6423	68.80	0.6929	41.56
	RealESRGAN	25.69	0.7616	0.2727	135.18	0.2063	0.5487	60.18	0.4449	39.53
	BSRGAN	26.39	0.7654	0.2670	141.28	0.2121	0.5399	63.21	0.5001	45.69

Appendix

Table 12: Quantitative comparison between GDPO-SR and the state-of-the-art GAN-based Real-ISR methods on synthetic and real-world datasets. The best and second best results are highlighted in red and blue , respectively. Arrows denote if higher (↑) or lower (↓) values represent better performance.

Figure 8: Visual comparison with SD-based Real-ISR methods. Please zoom in for a better view.

Figure 9: Visual comparison with GAN-based Real-ISR methods. Please zoom in for a better view.

Image Super-Resolution

Image super-resolution reconstructs high-resolution images from lower-resolution observations, balancing detail, fidelity, and computational cost.

See all 247 papers

Noise-Started One-Step Real-World Super-Resolution via LR-Conditioned SplitMeanFlow and GAN Refinement
May 10, 2026Wei Zhu, Kai Zhang, Yu Zheng +3Flow Matching Diffusion Model Distillation
Nanjing University of Science and Technology · Nanjing University · Huawei
Bridging Restoration and Generation Manifolds in One-Step Diffusion for Real-World Super-Resolution
Apr 27, 2026Shyang-En Weng, Yi-Cheng Liao, Yu-Syuan Xu +2Diffusion Model Inversion Diffusion Models for Image Restoration
National Yang Ming Chiao Tung University, Hsinchu, Taiwan · MediaTek Inc., Taiwan
Diff-Instruct with Diffused Reward: Towards Principled One-step Generator RL
May 18, 2026Junyi Wu, Weijian Luo, Haoyang Zheng +2Diffusion Model Alignment One-Step Generative Modeling
Purdue University · hi-lab, Xiaohongshu Inc.

May 10, 2026J/K move · Enter open · S save

Noise-Started One-Step Real-World Super-Resolution via LR-Conditioned SplitMeanFlow and GAN Refinement

Wei Zhu, Kai Zhang, Yu Zheng, Lei Luo+2

Nanjing University of Science and Technology · Nanjing University · Huawei

Flow Matching Diffusion Model Distillation One-Step Generative Modeling Mean Flow Image Super-Resolution

Open paper AskPDF arXiv

Pre-trained text-to-image (T2I) diffusion models have shown strong potential for real-world image super-resolution (Real-ISR), owing to their noise-started generation process that enables realistic texture synthesis and captures the one-to-many nature of super-resolution. However, diffusion-based Real-ISR methods still face a fundamental efficiency-quality trade-off. Multi-step methods generate high-quality results by iteratively denoising random Gaussian noise under LR conditioning, but suffer from slow sampling. Recent one-step methods greatly improve efficiency, yet they typically replace noise-started generation with direct LR-to-HR restoration, which weakens stochasticity and limits realistic detail synthesis. To address this issue, we propose SMFSR, a noise-started one-step Real-ISR framework via LR-conditioned SplitMeanFlow and GAN refinement. SMFSR preserves the random-noise starting point of diffusion models and learns a direct noise-to-HR mapping conditioned on the LR image. To this end, Interval Splitting Consistency distills the multi-step generative trajectory into a single average-velocity prediction, enabling efficient one-step generation. To compensate for the reduced opportunity for progressive refinement, we further introduce a GAN refinement stage, where a DINOv3-based discriminator enhances realistic texture synthesis and variational score distillation aligns the generated outputs with the natural image distribution under a frozen diffusion teacher. Extensive experiments demonstrate that SMFSR achieves state-of-the-art perceptual quality among one-step diffusion-based Real-ISR methods while retaining fast single-step inference.

Abstract

Image Super-Resolution

Explore similar work

Noise-Started One-Step Real-World Super-Resolution via LR-Conditioned SplitMeanFlow and GAN Refinement

Bridging Restoration and Generation Manifolds in One-Step Diffusion for Real-World Super-Resolution

Diff-Instruct with Diffused Reward: Towards Principled One-step Generator RL