Measuring Preferred Representation in AI-generated Images with Community Scorers
Authors: Melanie F. Pradier, Camilla Longden, Anja Thieme, Rita F. Marques, Nari Johnson, Sidhika Balachandar, Xinnuo Xu, Linda Y. Wen, Martin Grayson, Katja Hofmann, Cecily Morrison, Daniela Massiceti
Organizations: Microsoft Research, Cambridge, United Kingdom · Carnegie Mellon University; Pittsburgh, PA USA · University of California; Berkeley, CA USA · Canva Research, Sydney, Australia
AI-generated images often misrepresent historically marginalised communities, with real-world consequences beyond the images themselves. However, conventional metrics fail to measure how communities want to be represented in AI-generated images. To address this, we introduce the concept of preferred representation. Building on previous work, we propose a measurement instrument that operationalises this concept through a Community Scorer - a model that evaluates a specific community's preferred representation in AI-generated images. We interrogate the validity of Community Scorers on three diverse disability communities that span the Global North and South. The results show that Community Scorers align significantly better with community ratings compared to other widely used text-to-image metrics. We then demonstrate the usefulness of these scorers in a downstream content filtering task. We release the Community Preference Dataset to enable others to optimise how Community Scorers can be scaled for culturally specific evaluation of AI-generated visual media.
Figures & tables
Figure 1: Our contributions. We define, operationalise, and validate an automated metric of preferred representation. The Community Library corresponds to DCL and the Community Preference Dataset to DCP . The Preferred Representation box (bottom left) represents the target function we seek to approximate via a scalable proxy (bottom middle), the Community Scorer ϕ .
Method
BLV
DWF-S
DWF-N
CLIPScore
0.10±0.06
0.02±0.08
0.13±0.08
HPSv2
0.14±0.07
0.02±0.09
0.17±0.08
LLaVAScore
0.15±0.07
0.20±0.09
0.34±0.07
VQ2
0.00±0.06
−0.09±0.08
0.09±0.07
CAIRE
0.17±0.07
0.08±0.11
0.09±0.02
Zero-Shot Scorer
−0.05±0.02
−0.06±0.02
0.00±0.14
Table 1: Metric–human agreement across communities. Kendall’s τ correlation between automated metrics and community ratings on the held-out set, reported per community. Values are mean ± standard deviation estimated via 1000 bootstrap resamples with replacement.
Figure 2: Alignment improves with community supervision. We report QWK between Community Scorers and community ratings on the held-out set as a function of training data size. Community Scorers improve consistently with more data, while zero-shot Generic Scorers remain poorly aligned. Experiments are run with 5 random seeds controlling variability due to model parameter initialisation. The dashed BLV reference is within-lead repeat agreement (QWK 0.54).
Figure 3: Community Scorers improve retained-pool composition. Proportion of good and bad images retained by an automated evaluator after manual screening. Good images correspond to designated community lead ratings of 4 or 5, and bad images to ratings of 1 or 2, computed over all images that pass the automated filtering stage. GS and CS stand for Generic and Community Scorer.
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: An example from the BLV Community Library (left) and its synthetic counterpart from the Community Preference Dataset (right). Generating prompt: “ A photo of a VI (with total sight loss) shop owner arranging products on shelves with the help of a sighted assistant, showcasing her entrepreneurship and effective organizational skills for managing inventory. Make sure to include: A middle aged woman shop owner who is blind holding two beauty products; Sighted assistant who is helping in arranging products in the shop systematically; Tagged and systematically arranged products on the shop shelves. The scene is in Kenya. ”
Figure 5: Web-based rating interface for data collection. This was used to collect community ratings for the Community Preference Dataset. Community annotators are shown a text prompt, a corresponding AI-generated image, and a discrete rating scale with optional free-text comments.
Figure 6: Number of samples generated from different prompt sources.
Figure 7: Intra-rater agreement across two annotation rounds conducted six months apart.
Figure 8: Representative off-diagonal examples where ratings differed between annotation rounds.
Table 3: Vision LoRA adapter configuration. Target modules are all self-attention ( qkv_proj , o_proj ) and MLP ( gate_up_proj , down_proj ) projections across all transformer layers. The image encoder parameters are additionally unfrozen.
Hyperparameter
Value
Optimisation
Optimiser
AdamW ( adamw_torch )
Learning rate
4×10−5
LR scheduler
Linear decay
Warmup steps
50
Weight decay
0.01
Appendix
Table 4: Training hyperparameters for Phi-4 multimodal fine-tuning.
Experiment
Training Time
BLV
3 h 20 min
DWF-S
2 h 15 min
DWF-N
1 h 50 min
Appendix
Table 5: Approximate training times per experiment on a single A100 80 GB GPU.
Figure 9: Alignment between QWK and downstream quality metrics. Higher Quadratic Weighted Kappa (QWK) corresponds to a higher fraction of images rated positively and a lower fraction rated negatively by the community.
Method
BLV
DWF-N
DWF-S
CAIRE
-0.00
0.07
-0.04
Generic Scorer
-0.02
0.05
-0.00
Community Scorer
0.24
0.44
0.46
Appendix
Table 6: Quadratic Weighted Kappa (QWK) on a held-out set by community and method. QWK is only reported for methods producing ordinal scores. Community Scorers achieve the highest agreement across all communities.
Figure 10: Confusion matrices for the best Community Scorers. Predictions concentrate along the diagonal, with most disagreements limited to neighbouring rating levels.
Figure 11: Marginal score distributions. Predicted and human rating distributions are closely aligned, indicating good coverage of the community rating scale.
Figure 12: Calibration curves for Community Scorers. Predicted scores closely track empirical human ratings, indicating good calibration across most of the scale.
Figure 13: Qualitative impact of Community Scorers in content filtering. Example outputs from the filtering pipeline. Images in the first row are identified as poor representations (low alignment with community ratings) and filtered out, while images in the second row are retained as better aligned with the community’s preferred representation.
Figure 14: Threshold ablation for content filtering. Intermediate thresholds balance removal of low-rated content with retention of high-quality images.
Figure 15: Qualitative results for community-aligned content filtering across communities. For each community, the Community Scorer filters out images with subtle representational issues or misalignment, yielding final outputs that better reflect community preferences and norms.