Recently, reinforcement learning (RL) has been employed for improving generative image super-resolution (ISR) performance. However, the current efforts are focused on multi-step generative ISR, while one-step generative ISR remains underexplored due to its limited stochasticity. In addition, RL methods such as Direct Preference Optimization (DPO) require the generation of positive and negative sample pairs offline, leading to a limited number of samples, while Group Relative Policy Optimization (GRPO) only calculates the likelihood of the entire image, ignoring local details that are crucial for ISR. In this paper, we propose Group Direct Preference Optimization (GDPO), a novel approach to integrate RL into one-step generative ISR model training. First, we introduce a noise-aware one-step diffusion model that can generate diverse ISR outputs. To prevent performance degradation caused by noise injection, we introduce an unequal-timestep strategy to decouple the timestep of noise addition from that of diffusion. We then present the GDPO strategy, which integrates the principle of GRPO into DPO, to calculate the group-relative advantage of each online generated sample for model optimization. Meanwhile, an attribute-aware reward function is designed to dynamically evaluate the score of each sample based on its statistics of smooth and texture areas. Experiments demonstrate the effectiveness of GDPO in enhancing the performance of one-step generative ISR models. Code: https://github.com/Joyies/GDPO.
Figures & tables
Figure 1: Noise regulates the diversity of generated samples. Different noise inputs yield both high-quality samples ( e.g . , noise1) and low-quality ones ( e.g . , noise2 and noise3). After preference learning, the model produces more visually pleasing results.
Figure 2: The framework of GDPO, which consists of two core stages: (a) advantage calculation and (b) policy optimization. Firstly, we employ a pre-trained one-step Real-ISR model as the reference model to generate a group of diverse outputs by injecting different random noises. Subsequently, we compute the advantage A for each sample by evaluating its reward with our designed attribute-aware reward functions and converting these rewards into group-relative advantages. In the policy optimization stage, we feed these samples along with noises into both the policy model and the reference ISR model, and update the parameters of the policy ISR model by minimizing the proposed GDPO loss, steering it to favor generating high-reward samples.
Figure 3: The structure of NAOSD, which uses the tadd to control the intensity of injected noise.
Figure 4: The pipeline of calculating smooth and detailed regions.
Dataset
Method
PSNR↑
SSIM↑
LPIPS↓
FID↓
DISTS↓
MANIQA↑
MUSIQ↑
CLIPIQA↑
AFINE↓
NAOSD
27.97
0.7831
0.2859
140.37
0.2113
0.6041
65.17
0.6886
20.52
DrealSR
GDPO-SR
28.18
0.7839
0.2851
138.87
0.2112
0.6180
65.63
0.7020
18.72
NAOSD
25.25
0.7346
0.2689
114.91
0.2001
0.6459
69.06
0.6617
18.90
RealSR
GDPO-SR
25.48
0.7328
0.2675
112.13
0.1980
0.6615
69.42
0.6760
17.73
NAOSD
23.83
0.6131
0.2848
25.16
0.1922
0.6297
68.20
0.6758
41.76
DIV2K-val
GDPO-SR
23.92
0.6117
0.2897
26.44
0.1965
0.6423
68.80
0.6929
41.56
Table 1: Performance comparison with the base model NAOSD on real-world and synthetic datasets. Metrics with a blue background denote those utilized in the reward function, while yellow-shaded ones correspond to metrics that are excluded from it. Arrows denote if higher (↑) or lower (↓) values represent better performance. The best results are highlighted in red .
Dataset
Method
PSNR↑
SSIM↑
LPIPS↓
FID↓
DISTS↓
MANIQA↑
MUSIQ↑
CLIPIQA↑
AFINE↓
StableSR
28.03
0.7536
0.3284
148.98
0.2269
0.5592
58.51
0.6356
35.77
DiffBIR
26.71
0.6571
0.4557
166.79
0.2748
0.5927
61.07
0.6395
39.42
SeeSR
28.07
0.7684
0.3174
147.39
0.2315
0.6054
65.08
0.6905
22.79
PASD
27.36
0.7073
0.3760
156.13
0.2531
0.6160
64.87
0.6808
33.89
OSEDiff
27.92
0.7835
0.2968
135.30
0.2165
0.5899
64.65
0.6963
21.39
InvSR
25.79
0.7176
0.3471
166.42
0.2381
0.6212
64.92
0.6918
21.69
Table 2: Quantitative comparison with different methods on real-world and synthetic datasets. The best and second best results are highlighted in red and blue , respectively. Arrows denote if higher (↑) or lower (↓) values represent better performance.
Dataset
Method
PSNR↑
SSIM↑
LPIPS↓
FID↓
DISTS↓
MANIQA↑
MUSIQ↑
CLIPIQA↑
AFINE↓
DP 2 O-SR
23.95
0.6294
0.4412
157.76
0.2763
0.6674
71.09
0.7842
11.39
DrealSR
GDPO-SR
28.18
0.7839
0.2851
138.87
0.2112
0.6180
65.63
0.7020
18.72
DP 2 O-SR
22.39
0.6467
0.3513
131.79
0.2442
0.7036
73.24
0.7484
8.654
RealSR
GDPO-SR
25.48
0.7328
0.2675
112.13
0.1980
0.6615
69.42
0.6760
17.73
Table 3: Quantitative comparison with DP 2 O-SR on real-world datasets. The best results are highlighted in red .
Figure 5: Visual comparison with SD-based Real-ISR methods. Please zoom in for a better view.
Methods
StableSR
DiffBIR
SeeSR
PASD
OSEDiff
InvSR
GDPO-SR
Para.(B)
1.56
1.68
2.51
2.31
1.77
1.33
1.77
Time(s)
10.03
2.72
4.30
2.80
0.11
0.12
0.11
FLOPs(T)
79.94
24.31
65.86
29.13
2.27
2.40
2.27
Table 4: Comparison of mode size, running-time and FLOPs.
Figure 6: Visual comparison with DP 2 O-SR.
Method
PSNR↑
FID↓
DISTS↓
MUSIQ↑
AFINE↓
NAOSD
25.25
114.91
0.2001
69.06
20.52
Diffusion-DPO
25.41
112.87
0.2010
69.16
20.22
DanceGRPO
25.10
113.74
0.2049
69.95
16.52
GDPO (ours)
25.48
112.13
0.1980
69.42
18.72
Table 5: Ablation studies on GDPO on the RealSR dataset.
Method
LPIPS↓
DISTS↓
MUSIQ↑
CLIPIQA↑
NAOSD
0.2689
0.2001
69.06
0.6617
ARF w/ FR
0.2642
0.1978
67.68
0.6359
ARF w/ NR
0.2866
0.2102
69.80
0.6914
ARF w/o AW
0.2660
0.1979
68.42
0.6331
ARF (ours)
0.2675
0.1980
69.42
0.6760
Table 6: Ablation studies on ARF on the RealSR dataset.
G
PSNR ↑
DISTS ↓
MANIQA ↑
CLIPIQA ↑
NAOSD
25.25
0.2001
0.6459
0.6617
GDPO-SR (G=4)
25.45
0.1996
0.6491
0.6610
GDPO-SR (G=6)
25.48
0.1980
0.6615
0.6760
GDPO-SR (G=8)
25.34
0.1955
0.6619
0.6788
Table 7: Ablation studies on Group Size on the RealSR dataset.
Figure 7: Ablation studies on the timestep setting in NAOSD.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Method
PSNR↑
SSIM↑
LPIPS↓
FID↓
DISTS↓
MANIQA↑
MUSIQ↑
CLIPIQA↑
AFINE↓
NAOSD
25.27
0.7354
0.2685
116.67
0.1997
0.6458
68.85
0.6620
19.28
GDPO-SR
25.49
0.7336
0.2673
116.03
0.1978
0.6607
69.21
0.6766
18.19
Appendix
Table 8: Performance comparison (averaged over 50 stochastic runs) on the RealSR dataset. Arrows denote if higher (↑) or lower (↓) values represent better performance. The best results are highlighted in red .
FR
PSNR ↑
DISTS ↓
MANIQA ↑
CLIPIQA ↑
NAOSD
25.25
0.2001
0.6459
0.6617
PSNR
25.48
0.1980
0.6615
0.6760
LPIPS
25.28
0.1997
0.6605
0.6835
PSNR+LPIPS
25.33
0.1970
0.6548
0.6701
Appendix
Table 9: Ablation studies on FR metrics on the RealSR dataset.
Method
PSNR↑
SSIM↑
LPIPS↓
DISTS↓
MANIQA↑
MUSIQ↑
CLIPIQA↑
AFINE↓
GDPO-SR ( tadd =150)
25.85
0.7499
0.2541
0.1938
0.6521
68.09
0.6331
19.89
GDPO-SR ( tadd =250)
25.48
0.7328
0.2675
0.1980
0.6615
69.42
0.6760
17.73
GDPO-SR ( tadd =350)
24.84
0.7074
0.2900
0.2088
0.6646
70.14
0.6994
17.41
Appendix
Table 10: The impact of tadd on the generation capability of GDPO-SR. Arrows denote if higher (↑) or lower (↓) values represent better performance. The best and second best results are highlighted in red and blue , respectively.
Method
PSNR↑
SSIM↑
LPIPS↓
DISTS↓
MANIQA↑
MUSIQ↑
CLIPIQA↑
AFINE↓
NAOSD
25.25
0.7346
0.2689
0.2001
0.6459
69.06
0.6617
18.90
GDPO-SR (CFG)
25.06
0.7174
0.2782
0.2049
0.6543
69.47
0.6815
19.61
GDPO-SR ( tadd )
25.29
0.7294
0.2871
0.2133
0.6477
69.46
0.6766
17.12
GDPO-SR
25.48
0.7328
0.2675
0.1980
0.6615
69.42
0.6760
17.73
Appendix
Table 11: Ablation study on sample generation methods on the Real-ISR dataset. Arrows denote if higher (↑) or lower (↓) values represent better performance.
Datasets
Methods
PSNR↑
SSIM↑
LPIPS↓
FID↓
DISTS↓
MANIQA↑
MUSIQ↑
CLIPIQA↑
AFINE ↓
RealESRGAN
24.29
0.6371
0.3112
37.64
0.2141
0.5501
61.06
0.5277
56.20
BSRGAN
24.58
0.6269
0.3351
44.23
0.2275
0.5247
61.20
0.5071
62.21
LDL
23.83
0.6344
0.3256
42.29
0.2227
0.5350
60.04
0.5180
53.19
DIV2K-val
GDPO-SR
23.92
0.6117
0.2897
26.44
0.1965
0.6423
68.80
0.6929
41.56
RealESRGAN
25.69
0.7616
0.2727
135.18
0.2063
0.5487
60.18
0.4449
39.53
BSRGAN
26.39
0.7654
0.2670
141.28
0.2121
0.5399
63.21
0.5001
45.69
Appendix
Table 12: Quantitative comparison between GDPO-SR and the state-of-the-art GAN-based Real-ISR methods on synthetic and real-world datasets. The best and second best results are highlighted in red and blue , respectively. Arrows denote if higher (↑) or lower (↓) values represent better performance.
Figure 8: Visual comparison with SD-based Real-ISR methods. Please zoom in for a better view.
Figure 9: Visual comparison with GAN-based Real-ISR methods. Please zoom in for a better view.
Pre-trained text-to-image (T2I) diffusion models have shown strong potential for real-world image super-resolution (Real-ISR), owing to their noise-started generation process that enables realistic texture synthesis and captures the one-to-many nature of super-resolution. However, diffusion-based Real-ISR methods still face a fundamental efficiency-quality trade-off. Multi-step methods generate high-quality results by iteratively denoising random Gaussian noise under LR conditioning, but suffer from slow sampling. Recent one-step methods greatly improve efficiency, yet they typically replace noise-started generation with direct LR-to-HR restoration, which weakens stochasticity and limits realistic detail synthesis. To address this issue, we propose SMFSR, a noise-started one-step Real-ISR framework via LR-conditioned SplitMeanFlow and GAN refinement. SMFSR preserves the random-noise starting point of diffusion models and learns a direct noise-to-HR mapping conditioned on the LR image. To this end, Interval Splitting Consistency distills the multi-step generative trajectory into a single average-velocity prediction, enabling efficient one-step generation. To compensate for the reduced opportunity for progressive refinement, we further introduce a GAN refinement stage, where a DINOv3-based discriminator enhances realistic texture synthesis and variational score distillation aligns the generated outputs with the natural image distribution under a frozen diffusion teacher. Extensive experiments demonstrate that SMFSR achieves state-of-the-art perceptual quality among one-step diffusion-based Real-ISR methods while retaining fast single-step inference.
Wei Zhu, Kai Zhang, Yu Zheng +3
Nanjing University of Science and Technology · Nanjing University · Huawei
Pretrained diffusion models have revolutionized real-world image super-resolution (Real-ISR) but suffer from computational bottlenecks due to iterative sampling. Recent single-step distillation accelerates inference but faces a stark perception-distortion trade-off due to rigid timestep initialization, distributional trajectory mismatches, and fragile stochastic modulation. To address this, we present Adaptive Inversion and Degradation-aware Sampling for Real-ISR (IDaS-SR), a one-step framework bridging the deterministic restoration and stochastic generation manifolds. At its core, the Manifold Inversion Noise Estimator (MINE) resolves these initialization and trajectory mismatches by predicting a severity-aware timestep and inversion noise, precisely anchoring low-quality latents onto the diffusion trajectory. Furthermore, to mitigate fragile stochastic modulation, we propose CHARIOT, a continuous generative steering mechanism. By rescheduling trajectories and interpolating noise, it enables explicit navigation of the perception-distortion boundary without compromising structural priors. Extensive experiments demonstrate that IDaS-SR outperforms state-of-the-art methods, seamlessly transitioning from a rigorous structural restorer to a sophisticated texture hallucinator in a single inference step.
Shyang-En Weng, Yi-Cheng Liao, Yu-Syuan Xu +2
National Yang Ming Chiao Tung University, Hsinchu, Taiwan · MediaTek Inc., Taiwan
Recent advances in one-step text-to-image generation have enabled real-time synthesis with remarkable efficiency and quality. Previous reinforcement learning methods for one-step generators combine image-space reward optimization with diffusion noisy-space distribution matching. This paradigm brings challenges due to a mismatch between terminal reward optimization and the underlying generative dynamics. As a result, optimization tends to exploit stochastic degrees of freedom, often improving reward at the expense of image fidelity. To address this issue, we propose Diff-Instruct with Diffused Reward (DIDR), a data-free trajectory-level alignment framework derived from Integral KL minimization. DIDR propagates the RLHF-optimal reward-tilted clean-image distribution across all noise levels along the diffusion trajectory. We show that this objective admits the same minimizer as clean-image RLHF, while naturally inducing the Diffused Reward Score (DRS), which acts as a reward-driven correction to the reference score function. To make this practical, we further introduce the Diffused Reward Proxy (DRP), an efficient estimator of DRS based on differentiable short-step denoising. Extensive experiments demonstrate that DIDR consistently Pareto-dominates existing one-step SDXL baselines. Moreover, when transferred to a 6B DiT backbone (Z-Image), DIDR surpasses its 50-step teacher in preference alignment while requiring only a single generation step.