Organizations: Department of Information Technology, Rajkiya Engineering College Banda, 210201, Uttar Pradesh, India. · School of AI and Data Science, Indian Institute of Technology Jodhpur, 342030, Rajasthan, India.
Text-to-image (T2I) generative models achieve impressive visual fidelity but inherit and amplify demographic imbalances and cultural biases embedded in training data. We introduce T2I-BiasBench, a unified evaluation framework of thirteen complementary metrics that jointly captures demographic bias, element omission, and cultural collapse in diffusion models - the first framework to address all three dimensions simultaneously. We evaluate three open-source models - Stable Diffusion v1.5, BK-SDM Base, and Koala Lightning - against Gemini 2.5 Flash (RLHF-aligned) as a reference baseline. The benchmark comprises 1,574 generated images across five structured prompt categories. T2I-BiasBench integrates six established metrics with seven additional measures: four newly proposed (Composite Bias Score, Grounded Missing Rate, Implicit Element Missing Rate, Cultural Accuracy Ratio) and three adapted (Hallucination Score, Vendi Score, CLIP Proxy Score). Three key findings emerge: (1) Stable Diffusion v1.5 and BK-SDM exhibit bias amplification (>1.0) in beauty-related prompts; (2) contextual constraints such as surgical PPE substantially attenuate professional-role gender bias (Doctor CBS = 0.06 for SD v1.5); and (3) all models, including RLHF-aligned Gemini, collapse to a narrow set of cultural representations (CAS: 0.54-1.00), confirming that alignment techniques do not resolve cultural coverage gaps. T2I-BiasBench is publicly released to support standardized, fine-grained bias evaluation of generative models. The project page is available at: https://gyanendrachaubey.github.io/T2I-BiasBench/
Figures & tables
Figure 1: Overview of the proposed Bias and Fairness Evaluation Framework for text-to-image models. The pipeline consists of prompt generation (demographic and contextual), controlled image synthesis across multiple models, attribute extraction via vision-language methods, and a multi-metric evaluation module comprising 13 fairness, diversity, and alignment metrics. The framework produces composite bias and diversity scores, enabling comparative analysis of model behavior across demographic attributes and contextual scenarios.
Model
Type
Scale
#Images
Stable Diffusion v1.5 Rombach et al. (2022)
Open-source
∼860 M params
496
BK-SDM Base Kim et al. (2024)
Open-source (distilled)
Compact
499
Koala Lightning Lee et al. (2024)
Open-source (distilled)
Compact
500
Gemini 2.5 Flash Imran and Almusharraf (2024)
Proprietary (RLHF)
Large-scale
75 (baseline)
Table 1: Models selected for evaluation.
Prompt ID
Prompt Text
Bias Dimension
Beauty
“A fashion magazine cover featuring the most beautiful person in the world.”
“An insect resting on a flower in soft morning sunlight.”
Non-human baseline — lighting and species diversity
Table 2: Prompt design. Rows marked (*) are non-human contextual baselines used to isolate capability gaps from demographic bias.
Metric
Formula / Range
Interpretation
Reference
Representation Parity
pg=Ng/N
Raw group proportion; foundation metric
Friedrich et al. (2024)
Parity Difference
∣pa−pb∣∈[0,1]
0 = equal groups; 1 = only one group
Friedrich et al. (2024)
Bias Amplification
∑∣pi−1/k∣
>1.0 amplifies beyond training data
Zhao et al. (2017)
Shannon Entropy
H=−∑plog2p
Higher = more diverse output distribution
Info. theory
KL Divergence
KL(P∥U)
0 = perfectly fair distribution
Li et al. (2025)
CAS Score
S/(S+D+ε)
0 = diverse; 1 = fully stereotyped
Vice et al. (2023)
Table 3: Thirteen-metric evaluation framework. Rows marked † (blue-shaded in colour) are newly proposed metrics; rows marked ‡ (grey-shaded in colour) are adapted from existing work. PD = Parity Difference; H = normalised entropy; k = group count; S , D = stereotype / diverse term counts; ε = smoothing constant. 0 0 footnotetext: †Newly proposed metrics (this work): CBS = Composite Bias Score; GMR = Grounded Missing Rate; IEMR = Implicit Element Missing Rate; CAR = Cultural Accuracy Ratio. ‡Adapted from existing work and applied to T2I bias evaluation.
Table 5: 5×4 Generated Image Gallery (continued, rows 4–5). Colour-coding as per Table 4 : Low ( ≤0.30 , green-shaded), Moderate ( 0.31 – 0.55 , amber-shaded), High ( >0.55 , red-shaded). * * footnotetext: Gemini Doctor CBS =1.00 reflects counter-stereotyping (100% female); BA GEM,Doctor=0.00 confirms no active stereotype reinforcement (see Section 4.4 ). VAOP = Visual Attribute Occlusion Prompting (Section 4.4 ).
Model
Beauty
Doctor
Animal
Nature
Culture
μˉ
SD v1.5
0.50
0.06
0.30
0.32
0.66
0.37
Koala Lightning
0.51
0.76
0.43
0.23
0.48
0.48
BK-SDM Base
0.59
0.20
0.28
0.30
0.79
0.43
Gemini 2.5 Flash
0.33
1.00∗
0.20
0.20
0.60
0.47
Best model
Gemini
SD v1.5
Gemini
Gemini/Koala
Koala
SD v1.5
Table 6: Composite scores by model and prompt. Cells are colour-coded: low bias ( ≤0.30 , green-shaded), moderate ( 0.31 – 0.55 , amber-shaded), high ( >0.55 , red-shaded). ∗ Gemini Doctor CBS reflects female over-representation (counter-stereotyping); BA GEM,Doctor=0.00 . μˉ denotes the per-model mean across all prompts. Boldface marks the per-column optimum.
Figure 2: Composite bias and diversity scores ( 0 = fair, 1 = biased) across all five prompt categories for all four models. Horizontal threshold bands demarcate low ( ≤0.30 ), moderate ( 0.31 – 0.55 ), and high ( >0.55 ) severity regions. The Culture axis is consistently elevated across all models, providing quantitative evidence of systemic cultural representation collapse.
Figure 3: Ethnic composition P^(e∣Beauty) for e∈E across all four models. White-dominant distributions in SD v1.5 and BK-SDM contrast sharply with Gemini’s near-uniform ethnic coverage.
Figure 4: Beauty prompt — KL Divergence from the uniform reference KL(P^∥U) (Eq. 8 , left) and normalised Shannon Entropy H (Eq. 7 , right).
Figure 5: Bias Amplification (BA, Eq. 9 ) for the Beauty and Doctor prompts. Bars exceeding the dashed threshold BA =1.0 (red) indicate active stereotype reinforcement beyond the training distribution.
Figure 6: Gender distribution P^(g∣Doctor) for g∈{M,F,U} across all four models. Koala exhibits severe male dominance ( P^(M)=0.91 ); Gemini over-corrects to full female dominance ( P^(F)=1.00 ).
Figure 7: Animal baseline — Grounded Missing Rate (GMR, Eq. 17 ) for puzzle fidelity and Implicit Element Missing Rate (IEMR, Eq. 19 ) for laboratory context across all four models. Lower bars indicate better prompt fidelity.
Figure 8: Culture prompt — Contextual Association Score CAS (Eq. 12 , stereotype breadth, red bars) versus Cultural Accuracy Ratio CAR (Eq. 22 , green bars). The accuracy–breadth paradox is most visible in SD v1.5 (high CAR, high CAS) and Gemini (maximal CAS, maximal CAR).
Figure 9: Vendi Score (VS, Eq. 14 , caption lexical diversity) and CLIP Proxy Score (CPS, Eq. 15 , prompt–caption semantic alignment) across all models and prompts. High VS with variable CPS reveals a diversity–alignment decoupling in T2I models.
Figure 10: Bias profile radar chart across all five prompts for all four models. Smaller enclosed area Am indicates an overall fairer model. The universally extended Culture axis confirms systemic cultural representation collapse across all architectures, including the RLHF-aligned Gemini baseline.
Finding
Metric
Value
Model
Prompt
Largest ethnic skew
KL ↓
0.765
SD v1.5
Beauty
Smallest ethnic skew
KL ↓
0.063
Gemini
Beauty
Max bias amplification
BA ↓
1.15
Koala
Doctor
Min bias amplification
BA ↓
0.00
Gemini
Doctor
Worst gender parity
PD ↓
0.85
Koala
Doctor
Best composite score
CBS ↓
0.06
SD v1.5
Doctor
Table 7: Summary of key quantitative findings. ↓ = lower is better; ↑ = higher is better. All metrics defined in Section 3.5 . 0 0 footnotetext: ∗ Gemini Doctor CBS =1.00 reflects the metric polarity artefact (Section 4.4 ): female dominance, not male-dominant bias reinforcement.
Text-to-image (T2I) models often produce stereotypical images when stereotype-relevant attributes are left unspecified. We ask whether explicit requests can overcome these defaults, especially when the requested attribute contradicts the stereotype. We introduce \mbox{IMPLICIT-Bench}, a challenge benchmark of 1,831 prompt triplets built on a bias knowledge graph (KG) of stereotypes recorded in StereoSet and CrowS-Pairs. Each triplet describes one scene with a neutral prompt, a stereotype prompt, and an anti-stereotype prompt. The neutral prompt measures the model's default, and the two explicit prompts measure whether the model can follow a request on either side of the stereotype. On this benchmark, Qwen-Image depicts the stereotype in 52% of neutral images and the anti-stereotype in 5%. On both Qwen-Image and Stable Diffusion~3, explicit requests that contradict this default succeed far less often than requests that agree with it. Where Qwen-Image's default is strongest, explicitly requesting the stereotype succeeds in 97% of cases, whereas requesting the anti-stereotype succeeds in only 54%. In these scenes, 31% of the images for anti-stereotype requests show the stereotypical attribute instead. We further test whether prompt rewriting and embedding steering can reduce the default on neutral prompts. With the prompt's own KG unit, the best configurations improve a combined bias and alignment score by up to 9.5 points, and with a unit retrieved from the bias KG, by at most 2.9. Debiasing thus depends on knowing which stereotype a prompt evokes.
Reward models (RMs) are inherently non-neutral value functions designed and trained to encode specific objectives, such as human preferences or text-image alignment. RMs have become crucial components of text-to-image (T2I) generation systems where they are used at various stages for dataset filtering, as evaluation metrics, as a supervisory signal during optimization of parameters, and for post-generation safety and quality filtering of T2I outputs. While specific problems with the integration of RMs into the T2I pipeline have been studied (e.g. reward hacking or mode collapse), their robustness and fairness as scoring functions remains largely unknown. We conduct a large scale audit of RM robustness with respect to demographic biases during T2I model training and generation. We provide quantitative and qualitative evidence that while originally developed as quality measures, RMs encode demographic biases, which cause reward-guided optimization to disproportionately sexualize female image subjects reinforce gender/racial stereotypes, and collapse demographic diversity. These findings highlight shortcomings in current reward models, challenge their reliability as quality metrics, and underscore the need for improved data collection and training procedures to enable more robust scoring.
As Text-to-Image (T2I) systems rapidly advance, evaluating the cultural authenticity of synthesized content has become increasingly important for fair and trustworthy generative AI. Existing T2I evaluation metrics and multimodal judges often rely on visual-semantic representations that underrepresent implicit cultural norms, leading to biased preference judgments and the omission of fine-grained cultural cues. In addition, visual question answering (VQA)-based evaluators typically depend on autoregressive text generation, which limits their scalability for real-time reward modeling. To address these limitations, we introduce an Implicit Cultural Alignment Reward Model built upon a lightweight 4.2-billion-parameter Multimodal Large Language Model (MLLM). Our framework integrates an Implicit Cultural Probe with a Skip-connection Cross-Attention (SkipCA) mechanism, enabling late-stage semantic features to directly attend to early-stage visual representations and better preserve culturally salient details. Evaluations on 3,323 challenging and carefully curated image pairs from the CulturalFrames benchmark show that our approach achieves 83.49% pairwise accuracy, with Pearson and Kendall correlation coefficients of 0.5268 and 0.3749, respectively, outperforming representative vision-language metrics and MLLM-based evaluators. Moreover, by bypassing autoregressive text generation, our model processes each evaluation in 0.21 seconds under our local inference setup, achieving a 10× speedup over standard VQA-based evaluators. These results suggest that the proposed reward model can provide an efficient and culturally aware scalar signal for preference optimization pipelines such as Reinforcement Learning from Human Feedback and Direct Preference Optimization. Additional resources are available on our project page at https://bensonch1214.github.io/Implicit_Cultural_Alignment/.
Bo-An Chang, Yu-Chih Chen
National Tsing Hua University, Hsinchu 300044, Taiwan · National Yang Ming Chiao Tung University, Hsinchu 300093, Taiwan