PALATE: Personalized Aesthetic Learning through Adaptive Taste Evolution for Multi-User Portrait Retouching

Jingxuan Wang; Yifan Mei; Yuxia Niu; Chaowan Jiao; Qijin Shen

cs.CVAug 19, 2026

PALATE: Personalized Aesthetic Learning through Adaptive Taste Evolution for Multi-User Portrait Retouching

Authors: Jingxuan Wang, Yifan Mei, Yuxia Niu, Chaowan Jiao, Qijin Shen

Organizations: Harbin Institute of Technology · MT Lab, Meitu Inc. · Xiamen University · Central South University

Abstract

Automatic portrait retouching has advanced rapidly, yet its objective is inherently subjective: the same portrait admits multiple professionally valid results, and users disagree about which one is best. Most existing methods optimize a population-level aesthetic standard and therefore cannot capture individual taste, while fine-tuning a separate editing model for every user incurs prohibitive training, storage, and data costs. We propose PALATE, a shared reward-evolution framework that keeps the image editor fixed and instead personalizes the selection among retouched candidates of the same source portrait. PALATE decomposes the reward for each user into a global backbone shared by all users, category-level residuals shared by aesthetically similar users, and a lightweight user adapter, with anti-collapse regularizers keeping the three levels complementary.A cyclic dual-level distillation scheme first distills user-specific preferences into category rewards and then consolidates the resulting category-level knowledge into the global backbone, which is redistributed to initialize the next evolution round. In this way, the shared initialization improves progressively across rounds, enabling unseen users to be calibrated from only a few rankings. On expert-retouched candidates from PPR10K with held-out users and held-out images, PALATE attains 72.83% pairwise preference-prediction accuracy, surpassing all reward, aesthetic, and image-quality baselines, of which the strongest, PickScore, reaches 58.06%. Each new user costs only 512 bytes of user-specific parameters and millisecond-level scoring.

Figures & tables

Figure 1: Overview of PALATE. (a) During training, a source portrait and its retouched candidates are encoded into retouching features. A shared reward backbone and user residual adapters jointly form personalized teacher rewards. User-level preference knowledge is first distilled within each aesthetic category to update the corresponding category reward, after which category rewards are consolidated into an evolved global reward. The evolved rewards are redistributed to initialize the next round of category modeling and user adaptation. (b) During inference, a frozen image editor generates multiple retouched candidates for a source portrait, and a few-shot calibrated personalized reward ranks the candidates and selects the result that best matches the target user’s preference.

Figure 2: Global–category–user cyclic reward evolution in PALATE. Shared knowledge is distributed from the global reward to category rewards and user adapters, while personalized knowledge is distilled in the reverse direction, from users to categories and then to the global reward. The evolved global reward is redistributed to initialize the next evolution round.

Figure 3: Qualitative comparison of candidate-selection results. For each source portrait, three expert-retouched candidates are evaluated by PALATE, PickScore, and CLIPScore. The rightmost column shows the candidate selected by the user.

Method	Pairwise Acc.	Top-1	MRR	NDCG@3	Exact Rank	Hit@2	Kendall $\tau$	Spearman $\rho$	Macro Pairwise	Worst-prof. Pairwise	Macro Top-1	Worst-prof. Top-1
Aesthetic and quality assessment
FGAesQ	38.31	21.17	52.24	73.01	9.17	49.92	$-0.234$	$-0.262$	39.60	35.72	23.08	17.33
CLIP-IQA	43.72	25.92	56.92	75.92	13.42	59.42	$-0.097$	$-0.112$	43.94	42.33	26.21	24.00
Vision–language similarity
CLIP image similarity	41.17	22.25	54.53	74.39	12.67	58.21	$-0.148$	$-0.165$	41.28	40.00	22.96	20.83
SigLIP similarity	48.67	35.83	62.69	78.75	18.38	65.88	0.002	0.000	48.78	47.67	36.42	34.67

Table 1: Comparison under the same held-out users and source images. All metrics except Kendall’s

\tau

and Spearman’s

\rho

are percentages. Macro scores average user profiles equally; worst-profile scores report the lowest-performing profile. Best results are bold.

Figure 4: Deployment modes under the user–image double-held-out protocol. Correct routing improves global-only scoring; wrong routing degrades it.

Figure 5: Few-shot user calibration under different support budgets. Safe calibration is stable; the vanilla adapter degrades.

Persistent storage
Component	Parameters	Storage
Global reward	309,511	1.23 MiB
Five category rewards	$5\times 309{,}511$	6.08 MiB
One user adapter	128	512 B
Computation time
Operation	Time	Setting

Table 2: Storage and deployment cost measured on one NVIDIA A800-SXM4-80GB GPU with cached 195-dimensional input features. The shared PALATE state, consisting of one global and five category rewards, occupies 7.31 MiB. The adaptation timing is measured for the vanilla user adapter at

K=10

with 50 optimization steps and is reported as its mean and observed range over six test users.

Cyclic knowledge flow
Variant	Pairwise acc. (%)
Full PALATE	72.83
w/o user-to-category distillation	71.50
w/o category-to-global distillation	72.72
w/o global broadcast	72.03
w/o double teacher	72.33

Table 3: Ablation results under the same user–image double-held-out protocol. All variants are selected using the validation set. The three blocks evaluate cyclic knowledge flow, input representations, and anti-collapse regularization, respectively. Pairwise accuracy is reported in percent.

Personalized Reward Modeling

See all 10 papers

RewardHarness: Self-Evolving Agentic Post-Training
May 9, 2026Yuxuan Zhang, Cong Wei, Penghui Du +11Reward Modeling Image Editing Evaluation
University of British Columbia · Vector Institute · Kolors Team, Kuaishou Technology +5
PrefLUT: Reusable and Refinable Personalized Color Editing from Pairwise Preferences
Sep 28, 2026Chuanzhi Xu, Langyi Chen, Chengkun Yue +5Pairwise Preference Learning User Preference Modeling
The University of Sydney · Indiana University · The University of Hong Kong
BeautyGRPO: Aesthetic Alignment for Face Retouching via Dynamic Path Guidance and Fine-Grained Preference Modeling
Mar 1, 2026Jiachen Yang, Xianhui Lin, Yi Dong +4Image Editing RL for Image Editing
School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University, China · vivo BlueImage Lab, vivo Mobile Communication Co., Ltd., China · Guangdong Provincial Key Laboratory of Information Security Technology, China

May 9, 2026J/K move · Enter open · S save

RewardHarness: Self-Evolving Agentic Post-Training

Yuxuan Zhang, Cong Wei, Penghui Du, Bo Li+10

University of British Columbia · Vector Institute · Kolors Team, Kuaishou Technology+5

Reward Modeling Image Editing Evaluation Multimodal Reward Modeling Image Generation Evaluation

Open paper AskPDF arXiv

Aligning generative models with human preferences remains a central challenge. While RLHF is widely adopted, it relies on large-scale preference data. This limitation is particularly evident in evaluating instruction-guided image edits, even though humans can often learn such preferences from only a few examples. We present RewardHarness, a training-free, self-evolving agentic reward framework that reframes reward modeling as context evolution rather than weight optimization. Instead of learning from large-scale annotations, RewardHarness aligns with human preferences by iteratively evolving a library of tools and skills from as few as 100 preference demonstrations. Given a source image, an edited image, and an editing instruction, an orchestrator agent selects the most relevant subset of tools and skills from the maintained library, and sub-agents invoke them to construct a reasoning chain that produces a reward score. By comparing predicted scores with ground-truth preferences and analyzing successes and failures in the reasoning process, the orchestrator automatically refines its library of tools and skills without additional human annotation. Using only 0.05% of the EditReward preference data, RewardHarness achieves 47.4% average accuracy on EditReward-Bench K=2/3/4 and GenAI-Bench, surpassing GPT-5 by 5.3 points. When used as a reward signal in FlowGRPO, RL-tuned models achieve 3.52 on ImgEdit-Bench. Code and model are available at the project page: https://rewardharness.com/

Abstract

Personalized Reward Modeling

Explore similar work

RewardHarness: Self-Evolving Agentic Post-Training

PrefLUT: Reusable and Refinable Personalized Color Editing from Pairwise Preferences

BeautyGRPO: Aesthetic Alignment for Face Retouching via Dynamic Path Guidance and Fine-Grained Preference Modeling