BeautyGRPO: Aesthetic Alignment for Face Retouching via Dynamic Path Guidance and Fine-Grained Preference Modeling
Authors: Jiachen Yang, Xianhui Lin, Yi Dong, Zebiao Zheng, Xing Liu, Hong Gu, Yanmei Fang
Organizations: School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University, China · vivo BlueImage Lab, vivo Mobile Communication Co., Ltd., China · Guangdong Provincial Key Laboratory of Information Security Technology, China
Face retouching requires removing subtle imperfections while preserving unique facial identity features, in order to enhance overall aesthetic appeal. However, existing methods suffer from a fundamental trade-off. Supervised learning on labeled data is constrained to pixel-level label mimicry, failing to capture complex subjective human aesthetic preferences. Conversely, while online reinforcement learning (RL) excels at preference alignment, its stochastic exploration paradigm conflicts with the high-fidelity demands of face retouching and often introduces noticeable noise artifacts due to accumulated stochastic drift. To address these limitations, we propose BeautyGRPO, a reinforcement learning framework that aligns face retouching with human aesthetic preferences. We construct FRPref-10K, a fine-grained preference dataset covering five key retouching dimensions, and train a specialized reward model capable of evaluating subtle perceptual differences. To reconcile exploration and fidelity, we introduce Dynamic Path Guidance (DPG). DPG stabilizes the stochastic sampling trajectory by dynamically computing an anchor-based ODE path and replanning a guided trajectory at each sampling timestep, effectively correcting stochastic drift while maintaining controlled exploration. Extensive experiments show that BeautyGRPO outperforms both specialized face retouching methods and general image editing models, achieving superior texture quality, more accurate blemish removal, and overall results that better align with human aesthetic preferences.
Figures & tables
Figure 1: Comparison of face retouching methods and training paradigms. (a) Our approach outperforms specialized (RetouchFormer) and general (NanoBanana) models in blemish removal, identity preservation, and human preference. (b) By stabilizing stochastic exploration, BeautyGRPO overcomes the limitations of Supervised Fine-Tuning (SFT) and standard online RL (FlowGRPO), achieving natural and highly realistic results.
Figure 2: Overview of our FRPref-10K construction, reward model training, and BeautyGRPO with Dynamic Path Guidance (DPG). Top: FRPref-10K dataset curation pipeline. Multiple retouched candidates are generated with diverse editing models, preference pairs are formed via output-vs-output/label comparisons, and are annotated by VLMs across five quality dimensions before human verification. Bottom left: Three-stage reward model training, including SFT, self-training with consistency filtering, and GRPO. Bottom right: BeautyGRPO training with DPG on a FluxKontext-LoRA backbone.
Figure 3: Overview of different sampling trajectories. (a) Standard flow-matching ODE trajectory for SFT. (b) Uncontrolled SDE trajectory in FlowGRPO, which gradually drifts from the high-fidelity anchor point and introduces noise artifacts. (c) Proposed BeautyGRPO with Dynamic Path Guidance (DPG). At each timestep, DPG dynamically computes an anchor-based ODE path and replans a guided trajectory by linearly blending a correction vector with standard Gaussian noise, correcting stochastic drift while maintaining controlled exploration.
FFHQR
In-the-wild
NIQE ↓
NIMA ↑
MUSIQ ↑
MANIQA ↑
NRQM ↑
TOPIQ ↑
FID ↓
ArcFace ↑
NIQE ↓
NIMA ↑
MUSIQ ↑
MANIQA ↑
NRQM ↑
TOPIQ ↑
ArcFace ↑
Retouch Model
ABPN [ 20 ]
11.152
4.861
4.599
1.027
8.219
0.611
4.713
0.971
11.979
5.005
4.651
0.997
7.019
0.603
0.968
RestoreFormer [ 44 ]
12.281
4.712
4.361
0.958
7.411
0.587
15.898
0.799
12.801
4.819
4.534
0.978
7.372
0.601
0.761
RestoreFormer++ [ 45 ]
12.144
4.618
4.273
0.935
7.415
0.572
15.311
0.907
12.624
4.713
4.386
0.959
7.307
0.594
0.890
VRetouchEr [ 55 ]
12.948
4.586
4.169
0.962
6.167
0.551
2.229
0.978
13.605
4.658
4.308
0.957
5.635
0.565
0.974
Table 1: Quantitative evaluation of state-of-the-art methods on the FFHQR and In-the-wild datasets, utilizing diverse no-reference metrics to assess perceptual quality and aesthetic appeal. Our BeautyGRPO significantly outperforms specialized retouching models, general editing baselines, and alternative RL strategies across nearly all benchmarks. These results validate the effectiveness of our preference alignment framework in achieving high-quality, natural retouching outcomes that generalize well across diverse scenarios. Best results are in bold .
Figure 4: Visual comparison of face retouching results across different methods on FFHQR and in-the-wild datasets, where Flux.K denotes FluxKontext. Our BeautyGRPO removes blemishes cleanly while retaining natural texture, skin gloss, and moles, unlike existing models that exhibit incomplete blemish removal, over-smoothing, or artificial appearance. Please refer to the supplementary materials for more results.
VretouchEr
RetouchFormer
NanoBanana
Kontext Lora
Ours
Win rate
6.50
8.50
9.75
12.00
63.25
Table 2: Comparison of user preference win rates for various face retouching methods.
Figure 5: Radar chart comparing human judgment accuracy for our reward model against various VLMs.
Figure 6: Face retouching results of Qwen-Image-Edit with various training paradigms.
Figure 7: Ablation of DPG steps K on FFHQR.
NIMA ↑
MUSIQ ↑
MANIQA ↑
TOPIQ ↑
EditReward [ 48 ]
4.754
4.462
1.028
0.613
EditScore [ 26 ]
4.737
4.576
1.045
0.621
UnifiedReward-Edit [ 43 ]
4.756
4.571
1.037
0.625
Our Reward Model
5.123
4.906
1.079
0.676
Table 3: Quantitative evaluation of BeautyGRPO across various reward models on FFHQR.
NIMA ↑
MUSIQ ↑
MANIQA ↑
TOPIQ ↑
Qwen-Image-Edit
4.571
4.535
0.991
0.563
Qwen-Image-Edit+LoRA
4.824
4.745
1.028
0.595
Qwen-Image-Edit+LoRA w/Ours
5.351
5.133
1.075
0.664
Table 4: Quantitative comparison of Qwen-Image-Edit variants, demonstrating the superior performance of integrating BeautyGRPO.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Method
PSNR ↑
SSIM ↑
LPIPS ↓
MUSIQ ↑
MANIQA ↑
NRQM ↑
Supervised
RetouchFormer
42.088
0.997
0.024
4.465
1.036
8.175
Xu (2025) [ 53 ]
45.480
0.993
0.008
–
–
–
RetouchGPT
46.210
–
0.002
–
–
–
Generative
ICEdit
32.463
0.882
0.152
4.192
0.978
7.424
SeedDream
25.598
0.739
0.170
4.503
0.993
7.538
NanoBanana
28.841
0.824
0.106
4.681
1.009
8.065
Appendix
Table 5: Distortion vs. perceptual quality metrics on FFHQR.
Figure 8: Visual examples of imperfect ground-truth references in the FFHQR dataset, which are actively corrected by our method.
Figure 9: Age and racial distribution of the FRPref-10K dataset.
Figure 10: User interface of Questionnaire I.
Figure 11: User interface of Questionnaire II.
Figure 12: Comparison of user preference win rates for various face retouching methods across different ages and gender.
Figure 13: Qualitative examples of face retouching CoT reward reasoning. Given an input image and two retouched candidates, our reward model performs chain-of-thought quality assessment across five dimensions: skin smoothing, blemish removal, skin texture quality, clarity, and identity preservation.
Figure 14: Visual comparison of face retouching results across different methods on the FFHQR dataset, where Flux.K denotes FluxKontext. Compared with existing methods that suffer from incomplete blemish removal, over-smoothing, or unnatural visual appearance, our BeautyGRPO cleanly removes blemishes while preserving identity-consistent natural skin texture, gloss, and fine details such as wrinkles.
Figure 15: Visual comparison of face retouching results across different methods on the FFHQR dataset, where Flux.K denotes FluxKontext. Compared with existing methods that suffer from incomplete blemish removal, over-smoothing, or unnatural visual appearance, our BeautyGRPO cleanly removes blemishes while preserving identity-consistent natural skin texture, gloss, and fine details such as wrinkles.
Figure 16: Visual comparison of face retouching results across different methods on in-the-wild datasets, where Flux.K denotes FluxKontext. Compared with existing methods that suffer from incomplete blemish removal, over-smoothing, or unnatural visual appearance, our BeautyGRPO cleanly removes blemishes while preserving identity-consistent natural skin texture, gloss, and fine details such as moles.
While text-guided image editing has made remarkable progress, it remains limited in structural portrait retouching. Textual descriptions struggle to convey fine-grained changes to facial features and body proportions. To address this gap, we introduce Exemplar-Based Portrait Photo Retouching, where the model is given an exemplar pair and tasked with inferring and applying the same retouching operations to a new query image. Existing exemplar-based editing methods primarily focus on tasks with pronounced visual transformations. In contrast, structural portrait retouching involves extremely delicate and localized modifications, making accurate extraction and transfer of these edits challenging. To tackle this, we propose MirrorPPR, a novel framework designed to capture and transfer subtle structural retouching operations. Our method uses a Retouching Operation Extractor to capture the subtle differences from the exemplar pair. The extracted representations are then injected into a pre-trained Diffusion Transformer (DiT) through a connector and Low-Rank Adaptation (LoRA) modules. Furthermore, constructing perfectly aligned cross-identity training pairs is severely hindered by operation misalignment. To overcome this, we propose an advanced data self-augmentation paradigm that ensures strictly aligned retouching operations. To alleviate data scarcity and support this novel task, we introduce MirrorPPR47M, a large-scale dataset with over 47 million retouched pairs. By structuring the dataset into simulated and professional subsets, we enable progressive curriculum learning to smoothly optimize the network. Extensive experiments demonstrate that MirrorPPR significantly outperforms existing baselines in both retouching quality and identity preservation. The project page is available at https://sjtu-deng-lab.github.io/MirrorPPR.
Language-guided photo retouching aims to adjust color and tone while preserving geometry and texture. Recently, diffusion-based retouching shows a superior visual quality, but often struggles with both fidelity issues due to its generative nature and efficiency because of its iterative sampling process. In this work, we propose an efficient and fidelity-preserving retouching method using bilateral space manipulation, which is both compact and content-decoupled. Specifically, instead of directly editing pixels or image latents, our model predicts a low-resolution bilateral grid of affine transforms, which are sliced using a learned guidance map and then applied to the full-resolution image. This approach yields both high fidelity and improved efficiency. To retain strong priors of a pretrained generative model, we distill a multi-step diffusion model into our bilateral grid framework using Variational Score Distillation, complemented by a prompt alignment loss to guide instruction-following behavior. Additionally, we introduce a new benchmark and evaluate our method across multiple dimensions: fidelity, instruction following, and efficiency. Compared to the latest retouch methods, like Gemini-2.5-Flash (Nano-Banana), our method can avoid content drift, significantly improve latency, and generate visually pleasing edits, while maintaining a high level of fidelity. Project page: https://openimaginglab.github.io/InstantRetouch/.
Jiarui Wu, Yujin Wang, Ruikang Li +3
Shanghai AI Laboratory · CUHK MMLab · CPII under InnoHK
Automatic portrait retouching has advanced rapidly, yet its objective is inherently subjective: the same portrait admits multiple professionally valid results, and users disagree about which one is best. Most existing methods optimize a population-level aesthetic standard and therefore cannot capture individual taste, while fine-tuning a separate editing model for every user incurs prohibitive training, storage, and data costs. We propose PALATE, a shared reward-evolution framework that keeps the image editor fixed and instead personalizes the selection among retouched candidates of the same source portrait. PALATE decomposes the reward for each user into a global backbone shared by all users, category-level residuals shared by aesthetically similar users, and a lightweight user adapter, with anti-collapse regularizers keeping the three levels complementary.A cyclic dual-level distillation scheme first distills user-specific preferences into category rewards and then consolidates the resulting category-level knowledge into the global backbone, which is redistributed to initialize the next evolution round. In this way, the shared initialization improves progressively across rounds, enabling unseen users to be calibrated from only a few rankings. On expert-retouched candidates from PPR10K with held-out users and held-out images, PALATE attains 72.83% pairwise preference-prediction accuracy, surpassing all reward, aesthetic, and image-quality baselines, of which the strongest, PickScore, reaches 58.06%. Each new user costs only 512 bytes of user-specific parameters and millisecond-level scoring.
Jingxuan Wang, Yifan Mei, Yuxia Niu +2
Harbin Institute of Technology · MT Lab, Meitu Inc. · Xiamen University +1