Attractive Faces Skew Text-to-Image Reward Models and Leaderboards

Harvey Mannering; Zhiwu Huang; Adam Prügel-Bennett

cs.CVOct 20, 2026

Attractive Faces Skew Text-to-Image Reward Models and Leaderboards

Authors: Harvey Mannering, Zhiwu Huang, Adam Prügel-Bennett

Organizations: University of Southampton

Abstract

Human preference is now central to the evaluation, guidance, finetuning, and ranking of text-to-image models. However, the susceptibility of human preferences to bias in image generation remains underexplored. We identify a key observation: for human faces, perceived attractiveness overrides image quality in preference judgments, even when comparing real photographs of comparable fidelity. In a controlled user study, annotators rank more attractive faces as higher quality, even when unattractive real faces are compared with attractive AI-generated ones. We examine the downstream consequences of this bias. Firstly, simulating Elo rankings from user study data shows attractiveness can distort a leaderboard's ranking, elevating synthetic attractive faces over real, unattractive ones. Secondly, the same bias reappears in the human preference models. Editing 40 annotated facial attributes in isolation, we find that attractiveness is the strongest correlate of score for four of five human preference models (PickScore, MPS, HPSv2.1, HPSv3), and the strongest predictor of score in regression analysis. We further show that preference aligned generation shifts faces toward younger, more feminine faces, and can override explicit prompts for unattractive faces, thereby limiting user control. Finally, we explore mitigation strategies, both for data collection and post training on human preference models. These findings highlight the need for preference models that disentangle image quality from human cognitive bias, enabling more reliable evaluation.

Figures & tables

Panel of Figure 1 — Figure 1: Two AI-generated faces and the scores assigned by four widely used preference models. Stable Diffusion v1.5 (2022) was prompted for an attractive face; GPT-Image-2 (2026) was given a blurred version of the attractive face and prompted for an unattractive face, preserving its structure. Preference models mark the attractive face as better despite the lower quality.

Panel of Figure 2 — Figure 2: User study overview. Participants rank four faces by preference (Question 1) and rate three subjective attributes per face on a 0–100 scale (Question 2).

	AI More Attractive		Real More Attractive
	( $\Delta=-33.8$ )		( $\Delta=+9.8$ )
Instruction	Real Images	AI Images	Real Images	AI Images
Fidelity	1441.3 $\pm 0.6$	1558.7 $\pm 0.6$	1498.6 $\pm 0.6$	1501.4 $\pm 0.6$
Aesthetic	1434.4 $\pm 0.6$	1565.6 $\pm 0.6$	1512.1 $\pm 0.6$	1487.9 $\pm 0.6$
All	1438.1 $\pm 0.6$	1561.9 $\pm 0.6$	1505.8 $\pm 0.6$	1494.2 $\pm 0.6$

Table 1: Mean final Elo ratings (± standard error) of real and AI-generated CelebA-HQ images over 10000 simulated matchmaking runs using user study preferences. Left: less attractive real images against more attractive AI images. Right: the reverse selection.

\Delta

is the difference in mean attractiveness rating (Real

-

AI).

		Absolute Correlation Coeff. Ranking
		1st		2nd		3rd		4th		5th
Reward Model	Human Pref. Model	Attribute	$r$	Attribute	$r$	Attribute	$r$	Attribute	$r$	Attribute	$r$
Aesthetic [ 64 ]	✗	Blond Hair	$0.37$	Pointy Nose	$0.35$	Wearing Earrings	$0.34$	Arched Eyebrows	$0.32$	Mustache	$-0.31$
CLIP [ 58 ]	✗	Heavy Makeup	$-0.33$	Bushy Eyebrows	$0.33$	Wearing Lipstick	$-0.31$	Male	$0.28$	Wearing Necklace	$-0.27$
ImageReward [ 78 ]	✓	Narrow Eyes	$-0.39$	Wearing Necklace	$-0.34$	Bushy Eyebrows	$0.32$	Attractive	$0.30$	Mouth Slightly Open	$-0.27$
PickScore [ 38 ]	✓	Attractive	$0.64$	Young	$0.56$	Oval Face	$0.55$	Arched Eyebrows	$0.52$	Wearing Lipstick	$0.51$

Table 2: Attributes that correlate most strongly with reward model scores. We edit individual attributes in CelebA-HQ images using DiffAE. For each image the reward model scores are calculated. Correlation coefficients between reward model scores and attribute edit strengths are shown here, with the five strongest correlations being shown for each reward model.

Panel of Figure 3 — Figure 3: Example Diffusion Autoencoder editing of a CelebA-HQ image. Diffusion Autoencoder used to edit the “Pale Skin” attribute, with the original unedited image showing in the middle.

Figure 5: Left: the SCUT-FBP5500 dataset’s facial beauty labels against HPSv3 scores. Right: distribution of HPSv3 scores on real CelebA-HQ images, separated by the binary attractive label.

Figure 7: Human preference finetuning of Stable Diffusion v1.5 and a DDPM model using AlignProp and DRaFT, respectively. Each series shows the mean predicted facial attractiveness at each epoch of training (± standard deviation). Epoch 0 is the base mode (i.e. no finetuning).

Figure 8: Left: HPSv3 scores vs. facial attractiveness across 9,000 Stable Diffusion v3.5 outputs. Right: Scores after applying our debiasing approach.

	PickScore		HPSv2.1		HPSv3		MPS
T2I Model	Orig.	Adj.	Orig.	Adj.	Orig.	Adj.	Orig.	Adj.
FLUX.2 [ 42 ]	0.19	0.03	0.10	-0.01	0.09	-0.01	0.07	0.00
LongCat [ 70 ]	0.24	0.03	0.26	0.06	0.25	0.11	0.21	0.07
SD1.5 [ 61 ]	0.13	0.04	0.24	0.08	0.35	0.09	0.17	0.07
SD3.5 [ 18 ]	0.23	0.08	0.29	0.07	0.33	0.09	0.17	0.02
Z-Image [ 71 ]	0.25	0.11	0.26	0.10	0.17	0.04	0.31	0.00

Table 5: Correlation between predicted attractiveness and reward model scores before and after applying our debiasing approach. Orig. is the raw correlation; Adj. is the correlation after debiasing.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 9: Loss plots and accuracy of our facial attractiveness predictor over 20 epochs. From left to right the subplots show: total training loss, total validation loss, MSE loss on the validation set of SCUT-FBP5500, BCE loss on the validation set of CelebA, and accuracy of binary attractiveness prediction on the validation set of CelebA.

Figure 10: Human preference guided generation using different face-based prompts. Human preference alignment is done using ReNO with HPSv2.1 as the reward model and PixArt- $\alpha$ as the text-to-image model. The plot shows the average predicted facial attractiveness over 50 optimization steps.

Figure 11: Confusion matrices that show the age and gender of faces before and after ReNO optimization. PixArt- $\alpha$ is the base text-to-image model used and HPSv2.1 is the reward model used.

Figure 12: Confusion matrices that show the age, race, gender of faces before and after DRaFT finetuning. A DDPM model trained on CelebA faces is used as the base diffusion model and PickScore is used for reward model finetuning.

Panel of Figure 13 — Figure 13: Qualitative AlignProp finetuning results across epochs for two samples. The prompts used for generation are “She has a extremely mild smile. This female is a young adult and has no eyeglasses. This female shows the whole forehead without any bangs.” and “There is not any glasses on his face. This person looks extremely young and has no mustache, and no bangs. This guy looks serious with no smile in his face.” for the top and bottom rows, respectively.

Panel of Figure 14 — Figure 14: Qualitative DRaFT finetuning results across epochs for two samples.

Figure 15: Screenshot from the first question of our Qualtrics user study.

Figure 16: Screenshot from the second question of our Qualtrics user study.

Figure 17: Average final Elo ratings against differential attractiveness (Real $-$ AI). Shaded regions indicate standard deviation within 10000 simulations. As real images become more attractive relative to AI-generated images, their average final Elo ratings increase, suggesting that attractiveness can systematically bias leaderboard rankings.

T2I Generation Evaluation

Text-to-image generation evaluation measures how accurately, safely, and consistently image models turn prompts into images.

See all 162 papers

Personalizing Text-to-Image Generation to Individual Taste
Apr 8, 2026Anne-Sofie Maerten, Juliane Verwiebe, Shyamgopal Karthik +3Personalized Reward Modeling Personalized Image Aesthetics Assessment
Tübingen AI Center, University of Tübingen · Department of Brain and Cognition, KU Leuven
Fidelity Preference, Not Demographic Preference: A Pixel-Level Attribute-Sensitivity Audit of Image Aesthetic/Preference Scorers
Aug 17, 2026Mingyang XuAlgorithmic Bias Algorithmic Fairness
Peking University Beijing, China
Post-Training Frontier Text-to-Image Models by Composing Preference and Rubric Rewards
Oct 2, 2026Yuanhao Ban, I-Hung Hsu, Anastasios Angelopoulos +3T2I Generation Reward Design for RL
Arena Intelligence Inc · UCLA

Apr 8, 2026J/K move · Enter open · S save

Personalizing Text-to-Image Generation to Individual Taste

Anne-Sofie Maerten, Juliane Verwiebe, Shyamgopal Karthik, Ameya Prabhu+2

Tübingen AI Center, University of Tübingen · Department of Brain and Cognition, KU Leuven

Personalized Reward Modeling Personalized Image Aesthetics Assessment Reward Modeling T2I Generation

Open paper AskPDF arXiv

Modern text-to-image (T2I) models generate high-fidelity visuals but remain indifferent to individual user preferences. While existing reward models optimize for "average" human appeal, they fail to capture the inherent subjectivity of aesthetic judgment. In this work, we introduce a novel dataset and predictive framework, called PAMELA, designed to model personalized image evaluations. Our dataset comprises 70,000 ratings across 5,000 diverse images generated by state-of-the-art models (Flux 2 and Nano Banana). Each image is evaluated by 15 unique users, providing a rich distribution of subjective preferences across domains such as art, design, fashion, and cinematic photography. Leveraging this data, we propose a personalized reward model trained jointly on our high-quality annotations and existing aesthetic assessment subsets. We demonstrate that our model predicts individual liking with higher accuracy than the majority of current state-of-the-art methods predict population-level preferences. Using our personalized predictor, we demonstrate how simple prompt optimization methods can be used to steer generations towards individual user preferences. Our results highlight the importance of data quality and personalization to handle the subjectivity of user preferences. We release our dataset and model to facilitate standardized research in personalized T2I alignment and subjective visual quality assessment.

Abstract

T2I Generation Evaluation

Explore similar work

Personalizing Text-to-Image Generation to Individual Taste

Fidelity Preference, Not Demographic Preference: A Pixel-Level Attribute-Sensitivity Audit of Image Aesthetic/Preference Scorers

Post-Training Frontier Text-to-Image Models by Composing Preference and Rubric Rewards