BeautyGRPO: Aesthetic Alignment for Face Retouching via Dynamic Path Guidance and Fine-Grained Preference Modeling
Authors: Jiachen Yang, Xianhui Lin, Yi Dong, Zebiao Zheng, Xing Liu, Hong Gu, Yanmei Fang
Organizations: School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University, China · vivo BlueImage Lab, vivo Mobile Communication Co., Ltd., China · Guangdong Provincial Key Laboratory of Information Security Technology, China
Face retouching requires removing subtle imperfections while preserving unique facial identity features, in order to enhance overall aesthetic appeal. However, existing methods suffer from a fundamental trade-off. Supervised learning on labeled data is constrained to pixel-level label mimicry, failing to capture complex subjective human aesthetic preferences. Conversely, while online reinforcement learning (RL) excels at preference alignment, its stochastic exploration paradigm conflicts with the high-fidelity demands of face retouching and often introduces noticeable noise artifacts due to accumulated stochastic drift. To address these limitations, we propose BeautyGRPO, a reinforcement learning framework that aligns face retouching with human aesthetic preferences. We construct FRPref-10K, a fine-grained preference dataset covering five key retouching dimensions, and train a specialized reward model capable of evaluating subtle perceptual differences. To reconcile exploration and fidelity, we introduce Dynamic Path Guidance (DPG). DPG stabilizes the stochastic sampling trajectory by dynamically computing an anchor-based ODE path and replanning a guided trajectory at each sampling timestep, effectively correcting stochastic drift while maintaining controlled exploration. Extensive experiments show that BeautyGRPO outperforms both specialized face retouching methods and general image editing models, achieving superior texture quality, more accurate blemish removal, and overall results that better align with human aesthetic preferences.
Figures & tables
Figure 1: Comparison of face retouching methods and training paradigms. (a) Our approach outperforms specialized (RetouchFormer) and general (NanoBanana) models in blemish removal, identity preservation, and human preference. (b) By stabilizing stochastic exploration, BeautyGRPO overcomes the limitations of Supervised Fine-Tuning (SFT) and standard online RL (FlowGRPO), achieving natural and highly realistic results.
Figure 2: Overview of our FRPref-10K construction, reward model training, and BeautyGRPO with Dynamic Path Guidance (DPG). Top: FRPref-10K dataset curation pipeline. Multiple retouched candidates are generated with diverse editing models, preference pairs are formed via output-vs-output/label comparisons, and are annotated by VLMs across five quality dimensions before human verification. Bottom left: Three-stage reward model training, including SFT, self-training with consistency filtering, and GRPO. Bottom right: BeautyGRPO training with DPG on a FluxKontext-LoRA backbone.
Figure 3: Overview of different sampling trajectories. (a) Standard flow-matching ODE trajectory for SFT. (b) Uncontrolled SDE trajectory in FlowGRPO, which gradually drifts from the high-fidelity anchor point and introduces noise artifacts. (c) Proposed BeautyGRPO with Dynamic Path Guidance (DPG). At each timestep, DPG dynamically computes an anchor-based ODE path and replans a guided trajectory by linearly blending a correction vector with standard Gaussian noise, correcting stochastic drift while maintaining controlled exploration.
FFHQR
In-the-wild
NIQE ↓
NIMA ↑
MUSIQ ↑
MANIQA ↑
NRQM ↑
TOPIQ ↑
FID ↓
ArcFace ↑
NIQE ↓
NIMA ↑
MUSIQ ↑
MANIQA ↑
NRQM ↑
TOPIQ ↑
ArcFace ↑
Retouch Model
ABPN [ 20 ]
11.152
4.861
4.599
1.027
8.219
0.611
4.713
0.971
11.979
5.005
4.651
0.997
7.019
0.603
0.968
RestoreFormer [ 44 ]
12.281
4.712
4.361
0.958
7.411
0.587
15.898
0.799
12.801
4.819
4.534
0.978
7.372
0.601
0.761
RestoreFormer++ [ 45 ]
12.144
4.618
4.273
0.935
7.415
0.572
15.311
0.907
12.624
4.713
4.386
0.959
7.307
0.594
0.890
VRetouchEr [ 55 ]
12.948
4.586
4.169
0.962
6.167
0.551
2.229
0.978
13.605
4.658
4.308
0.957
5.635
0.565
0.974
Table 1: Quantitative evaluation of state-of-the-art methods on the FFHQR and In-the-wild datasets, utilizing diverse no-reference metrics to assess perceptual quality and aesthetic appeal. Our BeautyGRPO significantly outperforms specialized retouching models, general editing baselines, and alternative RL strategies across nearly all benchmarks. These results validate the effectiveness of our preference alignment framework in achieving high-quality, natural retouching outcomes that generalize well across diverse scenarios. Best results are in bold .
Figure 4: Visual comparison of face retouching results across different methods on FFHQR and in-the-wild datasets, where Flux.K denotes FluxKontext. Our BeautyGRPO removes blemishes cleanly while retaining natural texture, skin gloss, and moles, unlike existing models that exhibit incomplete blemish removal, over-smoothing, or artificial appearance. Please refer to the supplementary materials for more results.
VretouchEr
RetouchFormer
NanoBanana
Kontext Lora
Ours
Win rate
6.50
8.50
9.75
12.00
63.25
Table 2: Comparison of user preference win rates for various face retouching methods.
Figure 5: Radar chart comparing human judgment accuracy for our reward model against various VLMs.
Figure 6: Face retouching results of Qwen-Image-Edit with various training paradigms.
Figure 7: Ablation of DPG steps K on FFHQR.
NIMA ↑
MUSIQ ↑
MANIQA ↑
TOPIQ ↑
EditReward [ 48 ]
4.754
4.462
1.028
0.613
EditScore [ 26 ]
4.737
4.576
1.045
0.621
UnifiedReward-Edit [ 43 ]
4.756
4.571
1.037
0.625
Our Reward Model
5.123
4.906
1.079
0.676
Table 3: Quantitative evaluation of BeautyGRPO across various reward models on FFHQR.
NIMA ↑
MUSIQ ↑
MANIQA ↑
TOPIQ ↑
Qwen-Image-Edit
4.571
4.535
0.991
0.563
Qwen-Image-Edit+LoRA
4.824
4.745
1.028
0.595
Qwen-Image-Edit+LoRA w/Ours
5.351
5.133
1.075
0.664
Table 4: Quantitative comparison of Qwen-Image-Edit variants, demonstrating the superior performance of integrating BeautyGRPO.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Method
PSNR ↑
SSIM ↑
LPIPS ↓
MUSIQ ↑
MANIQA ↑
NRQM ↑
Supervised
RetouchFormer
42.088
0.997
0.024
4.465
1.036
8.175
Xu (2025) [ 53 ]
45.480
0.993
0.008
–
–
–
RetouchGPT
46.210
–
0.002
–
–
–
Generative
ICEdit
32.463
0.882
0.152
4.192
0.978
7.424
SeedDream
25.598
0.739
0.170
4.503
0.993
7.538
NanoBanana
28.841
0.824
0.106
4.681
1.009
8.065
Appendix
Table 5: Distortion vs. perceptual quality metrics on FFHQR.
Figure 8: Visual examples of imperfect ground-truth references in the FFHQR dataset, which are actively corrected by our method.
Figure 9: Age and racial distribution of the FRPref-10K dataset.
Figure 10: User interface of Questionnaire I.
Figure 11: User interface of Questionnaire II.
Figure 12: Comparison of user preference win rates for various face retouching methods across different ages and gender.
Figure 13: Qualitative examples of face retouching CoT reward reasoning. Given an input image and two retouched candidates, our reward model performs chain-of-thought quality assessment across five dimensions: skin smoothing, blemish removal, skin texture quality, clarity, and identity preservation.
Figure 14: Visual comparison of face retouching results across different methods on the FFHQR dataset, where Flux.K denotes FluxKontext. Compared with existing methods that suffer from incomplete blemish removal, over-smoothing, or unnatural visual appearance, our BeautyGRPO cleanly removes blemishes while preserving identity-consistent natural skin texture, gloss, and fine details such as wrinkles.
Figure 15: Visual comparison of face retouching results across different methods on the FFHQR dataset, where Flux.K denotes FluxKontext. Compared with existing methods that suffer from incomplete blemish removal, over-smoothing, or unnatural visual appearance, our BeautyGRPO cleanly removes blemishes while preserving identity-consistent natural skin texture, gloss, and fine details such as wrinkles.
Figure 16: Visual comparison of face retouching results across different methods on in-the-wild datasets, where Flux.K denotes FluxKontext. Compared with existing methods that suffer from incomplete blemish removal, over-smoothing, or unnatural visual appearance, our BeautyGRPO cleanly removes blemishes while preserving identity-consistent natural skin texture, gloss, and fine details such as moles.