OARS: Process-Aware Online Alignment for Generative Real-World Image Super-Resolution

Shijie Zhao; Xuanyu Zhang; Bin Chen; Weiqi Li; Qunliang Xing; Kexin Zhang; Yan Wang; Junlin Li

cs.CVMar 13, 2026

OARS: Process-Aware Online Alignment for Generative Real-World Image Super-Resolution

Authors: Shijie Zhao, Xuanyu Zhang, Bin Chen, Weiqi Li, Qunliang Xing, Kexin Zhang, Yan Wang, Junlin Li,

Organizations: ByteDance Inc. · Peking University · The Chinese University of Hong Kong

Abstract

Aligning generative real-world image super-resolution models with human visual preference is challenging due to the perception--fidelity trade-off and diverse, unknown degradations. Prior approaches rely on offline preference optimization and static metric aggregation, which are often non-interpretable and prone to pseudo-diversity under strong conditioning. We propose OARS, a process-aware online alignment framework built on COMPASS, a MLLM-based reward that evaluates the LR to SR transition by jointly modeling fidelity preservation and perceptual gain with an input-quality-adaptive trade-off. To train COMPASS, we curate COMPASS-20K spanning synthetic and real degradations, and introduce a three-stage perceptual annotation pipeline that yields calibrated, fine-grained training labels. Guided by COMPASS, OARS performs progressive online alignment from cold-start flow matching to full-reference and finally reference-free RL via shallow LoRA optimization for on-policy exploration. Extensive experiments and user studies demonstrate consistent perceptual improvements while maintaining fidelity, achieving state-of-the-art performance on Real-ISR benchmarks.

Figures & tables

Figure 1: Overview of the COMPASS-20K dataset construction and the COMPASS reward framework. (a) Data generation and score label annotation pipeline. (b) COMPASS uses an input-quality-adaptive mechanism to balance fidelity preservation and perceptual gain conditioned on input quality.

Figure 2: Overview of the proposed OARS framework. (a) Policy optimization process of our OARS framework; (b) Cold-start stage with paired LR–HR supervision; (c) Full-reference RL stage; (d) Non-reference RL stage on unpaired data guided by our reward.

Type	Method	Reg-Acc	Gen-Acc	All-Acc
GT-Ref	PSNR	80.7	41.7	34.7
	SSIM [ 51 ]	83.0	45.3	37.4
	LPIPS [ 73 ]	84.7	63.8	72.2
	DISTS [ 16 ]	86.0	63.9	71.4
	AHIQ [ 23 ]	71.0	71.5	69.6
	TOPIQ [ 9 ]	78.3	73.0	77.7

Table 1: Comparison of different IQA methods on SRIQA-Bench [ 11 ] . Throughout this paper, the best, second-best, and third-best results are highlighted in bold red , underlined blue , and italic green .

Dataset	Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	DISTS $\downarrow$	LIQE $\uparrow$	MUSIQ $\uparrow$	MANIQA $\uparrow$	Q-Insight $\uparrow$	TOPIQ $\uparrow$
RealSR	DiffBIR [ 33 ]	23.20	0.6346	0.3350	0.2162	3.5529	65.25	0.4620	3.530	0.6033
	OSEDiff [ 56 ]	23.07	0.6850	0.2941	0.2109	4.0681	68.95	0.4876	3.712	0.6441
	SeeSR [ 59 ]	24.34	0.7187	0.2754	0.2134	3.3938	65.53	0.4856	3.285	0.6246
	SinSR [ 50 ]	23.68	0.6649	0.3490	0.2445	3.2255	61.03	0.4230	3.264	0.5383
	StableSR [ 47 ]	23.73	0.6979	0.2792	0.2023	3.0532	61.65	0.3826	3.480	0.5201
	PURE [ 52 ]	21.31	0.5738	0.3859	0.2468	3.7881	66.57	0.4829	3.569	0.6301

Table 2: Quantitative comparison on three classical SR datasets, including RealSR, DIV2K, and RealSet80.

\uparrow/\downarrow

indicates higher/lower is better.

Figure 3: Subjective user study results. OARS achieves the highest preference rate, receiving 47.62% of the total votes.

Figure 4: Qualitative comparison of OARS against state-of-the-art Real-ISR methods under complex degradations.

Case	Score Calibration	Explicit Fidelity	Quality-Adaptive $\gamma$	Accuracy
1	✗	✗	✗	78.8
2	✓	✗	✗	81.5
3	✓	✓	✗	82.3
4	✓	✓	5	82.7
5	✓	✓	7	83.1
6	✓	✓	9	82.8

Table 3: Ablation study of dataset calibration, fidelity modeling, and adaptive

\gamma

Method	LIQE $\uparrow$	MUSIQ $\uparrow$	MANIQA $\uparrow$	Q-Insight $\uparrow$	TOPIQ $\uparrow$
Qwen-SFT	4.1602	69.7866	0.5171	3.701	0.6539
+COMPASS	4.5465 (+0.39)	72.9598 (+3.17)	0.5364 (+0.02)	3.823 (+0.12)	0.6967 (+0.04)
+COMPASS+TTT	4.5792 (+0.03)	73.2551 (+0.30)	0.5428 (+0.01)	3.841 (+0.02)	0.7025 (+0.01)
DP 2 O-SR [ 58 ]	4.2159	67.9658	0.5703	3.865	0.6867
+COMPASS	4.5796 (+0.36)	72.8912 (+4.93)	0.6123 (+0.04)	3.949 (+0.08)	0.6982 (+0.01)

Table 4: Quantitative comparison between our OARS and DPO-based SR methods on RealSet80. Performance gains are reported in parentheses.

Method	RL Stage	SFT / Base	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	DISTS $\downarrow$	LIQE $\uparrow$	MUSIQ $\uparrow$	MANIQA $\uparrow$	Q-Insight $\uparrow$	TOPIQ $\uparrow$
Qwen-SFT	-	-	22.71	0.6462	0.3100	0.2203	3.8146	68.5687	0.4897	3.545	0.6397
case (1)	stage1	base	22.52	0.6494	0.3115	0.2224	4.2345	71.0159	0.5244	3.685	0.6741
case (2)	stage1+stage2	base	22.36	0.6481	0.3095	0.2244	4.3045	71.4136	0.5277	3.701	0.6803
case (3)	stage1	sft	22.15	0.6266	0.3249	0.2272	4.0784	70.6030	0.5233	3.617	0.6756
case (4)	stage1+stage2	sft	21.31	0.5956	0.3326	0.2334	4.0937	70.5618	0.5293	3.667	0.6768

Table 5: Ablation study on different RL stages and initialization settings .

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Examples from the real-world low-quality subset of COMPASS-20K.

Figure 6: Effect of perceptual-gain-only optimization. Optimizing the policy using only $\Delta Q=Q_{\mathrm{SR}}-Q_{\mathrm{LR}}$ as the RL reward leads to severe artifacts and poor consistency with the input image, revealing clear reward hacking behavior. In contrast, COMPASS preserves fidelity while achieving clear perceptual quality improvement.

Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	DISTS $\downarrow$	LIQE $\uparrow$	MUSIQ $\uparrow$	MANIQA $\uparrow$	TOPIQ $\uparrow$
Qwen-SFT	22.71	0.6462	0.3100	0.2203	3.8146	68.5687	0.4897	0.6397
Qwen-SFT+OARS	22.36	0.6481	0.3095	0.2244	4.3045	71.4136	0.5277	0.6803
Flux-SFT	23.62	0.6617	0.3224	0.2168	3.8442	67.1802	0.4686	0.6031
Flux+OARS	23.10	0.6603	0.3335	0.2174	4.1277	68.3608	0.5035	0.6427

Appendix

Table 6: Ablation Study on different generative backbones.

\uparrow

\downarrow

indicate higher / lower is better.

Method	RL Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	DISTS $\downarrow$	LIQE $\uparrow$	MUSIQ $\uparrow$	MANIQA $\uparrow$	TOPIQ $\uparrow$
Qwen-SFT	–	22.71	0.6462	0.3100	0.2203	3.8146	68.5687	0.4897	0.6397
+HPSv2	OARS	22.91	0.6547	0.3154	0.2238	3.8294	68.6830	0.4966	0.6383
+Qwen25	OARS	23.08	0.6618	0.3042	0.2197	3.7682	68.2220	0.4835	0.6273
+RALI	OARS	20.83	0.5283	0.3777	0.2472	3.9657	71.0329	0.5399	0.6732
+COMPASS (Ours)	OARS	22.36	0.6481	0.3095	0.2244	4.3045	71.4136	0.5277	0.6803
+RALI	Flow-GRPO	21.08	0.5781	0.3548	0.2379	3.8757	70.3903	0.5127	0.6639

Appendix

Table 7: Quantitative comparison of different methods under full-reference and no-reference image quality assessment metrics.

\uparrow

\downarrow

indicates higher / lower is better.

Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	DISTS $\downarrow$	LIQE $\uparrow$	MUSIQ $\uparrow$	MANIQA $\uparrow$	TOPIQ $\uparrow$
Qwen-SFT	22.71	0.6462	0.3100	0.2203	3.8146	68.5687	0.4897	0.6397
+Formulation 1	21.38	0.6039	0.3436	0.2409	4.3654	71.8348	0.5596	0.7058
+Formulation 2	21.89	0.6203	0.3321	0.2320	4.2445	70.9428	0.5426	0.6904
+COMPASS (Ours)	22.36	0.6481	0.3095	0.2244	4.3045	71.4136	0.5277	0.6803

Appendix

Table 8: Ablation on reward formulation. Formulation 1 uses only perceptual gain,

\Delta Q=Q_{\mathrm{SR}}-Q_{\mathrm{LR}}

, while Formulation 2 adds a gating term,

F^{Q_{\mathrm{LR}}/\gamma}\!\cdot\!\Delta Q

. COMPASS further introduces an input-quality-aware fidelity term, enabling perceptual improvement with better fidelity preservation.

Dataset	Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	DISTS $\downarrow$	LIQE $\uparrow$	MUSIQ $\uparrow$	MANIQA $\uparrow$	Q-Insight $\uparrow$	TOPIQ $\uparrow$
RealSR	DiffBIR [ 33 ]	23.20	0.6346	0.3350	0.2162	3.5529	65.25	0.4620	3.530	0.6033
	OSEDiff [ 56 ]	23.07	0.6850	0.2941	0.2109	4.0681	68.95	0.4876	3.712	0.6441
	PASD [ 64 ]	24.50	0.7115	0.2716	0.1954	2.8541	58.52	0.3831	3.173	0.4969
	ResShift [ 70 ]	24.17	0.6528	0.4336	0.2812	2.6610	53.38	0.3412	2.781	0.4210
	S3Diff [ 71 ]	23.16	0.6810	0.2748	0.1986	4.0080	67.57	0.4677	3.674	0.6301
	SeeSR [ 59 ]	24.34	0.7187	0.2754	0.2134	3.3938	65.53	0.4856	3.285	0.6246

Appendix

Table 9: Quantitative comparison on four SR datasets , including RealSR, DRealSR, DIV2K, and RealSet80.

\uparrow/\downarrow

indicates higher/lower is better.

Figure 7: More qualitative comparisons of OARS against state-of-the-art Real-ISR methods under complex degradations.

Figure 8: More qualitative comparisons of OARS against state-of-the-art Real-ISR methods under complex degradations.

Image Super-Resolution

Image super-resolution reconstructs high-resolution images from lower-resolution observations, balancing detail, fidelity, and computational cost.

See all 248 papers

RefReward-SR: LR-Conditioned Reward Modeling for Preference-Aligned Super-Resolution
Mar 25, 2026Yushuai Song, Weize Quan, Weining Wang +8Image Super-Resolution Human Preference Modeling
Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · OPPO AI Center, OPPO Inc.
FreqOrtho-SR: Frequency-Guided Orthogonal Expert Learning for Real-World Image Super-Resolution
Jun 27, 2026Minh Son Hoang, Dinh Phu Tran, Quyen Nguyen Duc +2Diffusion Models for Image Restoration Mixture of Experts
School of Computing, KAIST, Republic of Korea
PixelIR: Fidelity-Perception Decoupling via Pixel-Space Image-Residual Flow Matching for Efficient One-Step Real-World Super-Resolution
Aug 31, 2026Bingtian Qiao, Yue Shi, Yong Guo +2Image Super-Resolution
Shanghai Jiao Tong University · Fuzhou University · Shanghai AI Laboratory

Mar 25, 2026J/K move · Enter open · S save

RefReward-SR: LR-Conditioned Reward Modeling for Preference-Aligned Super-Resolution

Yushuai Song, Weize Quan, Weining Wang, Jiahui Sun+7

Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · OPPO AI Center, OPPO Inc.

Image Super-Resolution Human Preference Modeling RL for Generative Models

Open paper AskPDF arXiv

Recent advances in generative super-resolution (SR) have greatly improved visual realism, yet existing evaluation and optimization frameworks remain misaligned with human perception. Full-Reference and No-Reference metrics often fail to reflect perceptual preference, either penalizing semantically plausible details due to pixel misalignment or favoring visually sharp but inconsistent artifacts. Moreover, most SR methods rely on ground-truth (GT)-dependent distribution matching, which does not necessarily correspond to human judgments. In this work, we propose RefReward-SR, a low-resolution (LR) reference-aware reward model for preference-aligned SR. Instead of relying on GT supervision or NR evaluation, RefReward-SR assesses high-resolution (HR) reconstructions conditioned on their LR inputs, treating the LR image as a semantic anchor. Leveraging the visual-linguistic priors of a Multimodal Large Language Models (MLLM), it evaluates semantic consistency and plausibility in a reasoning-aware manner. To support this paradigm, we construct RefSR-18K, the first large-scale LR-conditioned preference dataset for SR, providing pairwise rankings based on LR-HR consistency and HR naturalness. We fine-tune the MLLM with Group Relative Policy Optimization (GRPO) using LR-conditioned ranking rewards, and further integrate GRPO into SR model training with RefReward-SR as the core reward signal for preference-aligned generation. Extensive experiments show that our framework achieves substantially better alignment with human judgments, producing reconstructions that preserve semantic consistency while enhancing perceptual plausibility and visual naturalness. Code, models, and datasets will be released upon paper acceptance.

Abstract

Image Super-Resolution

Explore similar work

RefReward-SR: LR-Conditioned Reward Modeling for Preference-Aligned Super-Resolution

FreqOrtho-SR: Frequency-Guided Orthogonal Expert Learning for Real-World Image Super-Resolution

PixelIR: Fidelity-Perception Decoupling via Pixel-Space Image-Residual Flow Matching for Efficient One-Step Real-World Super-Resolution