Whose Values Are Generated? Measuring Visual Value Leakage in Multilingual Text-to-Image Generation
Organizations: University of Wisconsin–Madison · Microsoft Research Asia
Abstract
Text-to-image (T2I) systems serve users across languages and cultural contexts. However, evaluations of cultural representation focus mainly on recognizable elements such as landmarks, clothing, and activities, overlooking the cultural values that generated images convey. To bridge this gap, we introduce VisionValueBench, a multilingual benchmark that operationalizes six of Hofstede's cultural value dimensions (e.g., individualism versus collectivism) to investigate how T2I systems frame neutral social situations through a particular cultural orientation. We find that value-neutral prompts, which admit diverse cultural interpretations, nonetheless yield images that favor particular value orientations, a phenomenon we call visual value leakage. We evaluate prompts varying in language and geographic cues across 960 social scenarios and six T2I configurations. Generated images consistently favor the same value orientations across all six dimensions, for example, equality over hierarchy (low power distance), individual autonomy over group obligation (individualism), and enjoyment over restraint (indulgence). Prompt language has limited influence on these orientations, whereas geographic cues shift some dimensions toward the associated cultural values while largely preserving the prevailing orientations. These findings reveal substantial homogeneity in the cultural values conveyed by T2I systems worldwide.
Figures & tables
Appendix figures & tables29 assets
Supplementary material from the paper’s appendix.
Appendix
| Code | Value dimension | Orientation A | Orientation B |
|---|---|---|---|
| PDI | Power distance | Acceptance of unequal authority | Equal standing and influence |
| IDV | Individualism–collectivism | Personal autonomy | In-group obligation |
| MAS | Motivation towards achievement and success | Achievement and competition | Care and quality of life |
| UAI | Uncertainty avoidance | Need for certainty | Tolerance of ambiguity |
| LTO | Long-term orientation | Perseverance and pragmatic adaptation | Quicker returns and normative continuity |
| IVR | Indulgence–restraint | Relatively free enjoyment | Restraint by norms and duties |
| Country | PDI | IDV | MAS | UAI | LTO | IVR |
|---|---|---|---|---|---|---|
| United States | 40 | 60 | 62 | 46 | 50 | 68 |
| China | 80 | 43 | 66 | 30 | 77 | 24 |
| India | 77 | 24 | 56 | 40 | 51 | 26 |
| Thailand | 64 | 19 | 34 | 64 | 67 | 45 |
| Benchmark | Cultural focus | Prompt design | Multilingual | Geographic cues | Evaluation method |
|---|---|---|---|---|---|
| CUBE (2024) | Cultural awareness and diversity | Artifacts / broad concepts | ✗ | Country / no cue | Human ratings; diversity metrics |
| ViSAGe (2024) | Visual stereotypes | National identities and attributes | ✗ | Nationality | Human labels; embedding / caption metrics |
| CuRe (2025) | Long-tail cultural representativeness | Artifacts with varying detail | ✗ | Region / no cue | Embedding / perceptual metrics; human ratings |
| CultDiff (2025) | Cultural fidelity | Architecture, clothing, food | ✗ | Country | Human ratings; learned similarity |
| Nationalities audit (2025 / rev. 2026) | Attire stereotypes in activities | Everyday activities | ✗ | Nationality | Human labels; embedding / VLM alignment |
| CulturalFrames (2025) | Explicit / implicit expectations | Social practices and scenes | ✗ | Country context | Human ratings |
| PDI | IDV | MAS | UAI | LTO | IVR | |
|---|---|---|---|---|---|---|
| Initial | 512 | 472 | 452 | 530 | 485 | 502 |
| Removed | 25 | 28 | 44 | 27 | 25 | 31 |
| Retained | 487 | 444 | 408 | 503 | 460 | 471 |
| Model | Prompt processing | Conditions | Output resolution |
|---|---|---|---|
| Commercial API systems | |||
| Nano Banana 2 | Built-in thinking | 14 | |
| GPT-Image-2 | Internal processing | 14 | Auto |
| MAI-Image-2.6-Flash | Internal processing | 14 | |
| Open-weight image models | |||
| HiDream-O1-Image | Without Prompt Refiner | 6 | |
| Prompt language | No cue | US cue | China cue | Thailand cue | India cue |
|---|---|---|---|---|---|
| English | ✓ ∗ | ✓ ∗ | ✓ ∗ | ✓ | ✓ |
| Chinese | ✓ ∗ | ✓ ∗ | ✓ ∗ | – | – |
| Thai | ✓ | ✓ | – | ✓ | – |
| Hindi | ✓ | ✓ | – | – | ✓ |
| Component | Model identifier | Settings |
|---|---|---|
| Nano Banana 2 | gemini-3.1-flash-image | Minimal thinking; seed 20260728 |
| GPT-Image-2 | gpt-image-2 | Auto size |
| MAI | MAI-Image-2.6-Flash | ; seed unspecified |
| HiDream | HiDream-O1-Image-Dev-2604 | With Refiner: flash scheduler, 28 image steps |
| HiDream Refiner | HiDream-ai/Prompt-Refine | , top- , top- ; 2,048 output tokens |
| FLUX | black-forest-labs/FLUX.2-dev | 50 steps; guidance 4; bfloat16 |
| Configuration | Shared conditions | All configured conditions |
|---|---|---|
| MAI | 5,701 / 5,760 | 13,302 / 13,440 |
| GPT-Image-2 | 5,650 / 5,760 | 13,243 / 13,440 |
| Nano Banana 2 | 5,753 / 5,760 | 13,426 / 13,440 |
| FLUX + Upsampler | 5,760 / 5,760 | 5,760 / 5,760 |
| HiDream + Refiner | 5,760 / 5,760 | 5,760 / 5,760 |
| ideogram-4 Magic | 5,753 / 5,760 | 5,753 / 5,760 |
| Configuration | PDI | IDV | MAS | UAI | LTO | IVR | |
|---|---|---|---|---|---|---|---|
| SAE | |||||||
| MAI | 5701 | 927 | 3165 | 2302 | 674 | 1650 | 923 |
| GPT-Image-2 | 5650 | 897 | 2443 | 1911 | 631 | 1242 | 800 |
| Nano Banana 2 | 5753 | 952 | 2245 | 1605 | 514 | 963 | 1017 |
| FLUX + Upsampler | 5760 | 678 | 1708 | 1005 | 350 | 561 | 765 |
| HiDream + Refiner | 5760 | 741 | 1812 | 1257 | 364 | 639 | 768 |
| SAE versus SAE-H | IAE versus IAE-H | |||||
|---|---|---|---|---|---|---|
| Dim. | Agree. | 95% CI | Agree. | 95% CI | ||
| All | 77.5 | [0.717, 0.829] | 0.329 | 80.4 | [0.738, 0.871] | 0.562 |
| PDI | 82.5 | [0.725, 0.925] | 0.307 | 75.0 | [0.575, 0.900] | 0.322 |
| IDV | 55.0 | [0.425, 0.675] | 0.137 | 67.5 | [0.525, 0.825] | 0.387 |
| MAS | 75.0 | [0.625, 0.875] | 0.342 | 80.0 | [0.675, 0.925] | 0.614 |
| UAI | 75.0 | [0.625, 0.875] | 0.000 | 87.5 | [0.775, 0.950] | 0.600 |
| Metric | Automatic method | SAE-H | IAE-H |
|---|---|---|---|
| Overall agreement (%) | SAE | 77.50 | 67.50 |
| IAE | 82.92 | 80.42 | |
| UAI agreement (%) | SAE | 75.0 | 75.0 |
| IAE | 87.5 | 87.5 | |
| Human-supported UAI pairs detected | SAE | 0/10 | 0/10 |
| IAE | 5/10 | 5/10 |
| SAE | IAE | |||||||
|---|---|---|---|---|---|---|---|---|
| Verifier | Support | Accept | Agree. | Support | Accept | Agree. | ||
| Luna | 13.0 | 13.7 | 78.8 | 0.375 | 29.4 | 69.3 | 80.8 | 0.575 |
| Gemini | 6.7 | 9.0 | 74.6 | 0.223 | 13.2 | 36.7 | 71.2 | 0.328 |
| Muse | 12.4 | 14.5 | 80.0 | 0.420 | 30.7 | 72.1 | 85.0 | 0.677 |
| # | Prompt | Key inputs | Main outputs | Calls |
|---|---|---|---|---|
| Scenario construction (Section 3 ) | ||||
| – | Concept extraction | CDEval item + metadata | Concept + rationale | 1/item |
| – | Scenario generation | Same as concept extraction + concept | Scenario + rationale | 1/concept |
| – | Translation | English scenario | Chinese, Thai, Hindi scenarios | 1/scenario |
| Annotation (Section 4 ) | ||||
| F.4 | SAE proposal | Scenario | Potential orientation evidence | 3/scenario |
| Dim. | Defining issue | Orientation criteria | Insufficient alone |
|---|---|---|---|
| PDI | Authority or control over decisions, resources or participation, and how unequal power is treated | High: higher standing is treated as entitling control, deference, or limits on others’ voice. Low: their standing to contribute, question or seek accountability is acknowledged or asserted, even if a leader keeps final authority. | Following instructions (for High); disagreement, informality or friendliness (for Low) |
| IDV | A personal choice, goal, responsibility or belonging relative to a lasting in-group | Individualism: the matter is the person’s own to direct, including privacy and self-reliance. Collectivism: membership, harmony, loyalty or mutual commitments shape the choice, responsibility or identity. | Separate or shared activity, which may be task allocation or coordination; help and group ties |
| MAS | What the depiction treats as worth pursuing or maintaining | Achievement: accomplishment, advancement, competitive standing or material success is valued. Care: needs, well-being, relationships or quality of life matter. | Working, helping or holding a care role; effort, wealth, mood or gender composition |
| UAI | What is uncertain, ambiguous or unfamiliar for the participants | High: unease with uncertainty, or definite answers and fixed procedures treated as needed assurance. Low: ambiguity, provisional answers or flexibility accepted while something remains unsettled. | Error prevention, information gathering, precautions against known hazards |
| LTO | Present investment and expected returns, or established practices and expectations | Long-term: commitments serve returns worth waiting for, or a practice is pragmatically adapted. Short-term: quicker returns guide effort, or traditions and established expectations guide conduct. | Future benefits, scheduling or fast completion; old and new elements shown together |
| IVR | The place given to enjoying life, leisure or fun | Indulgence: enjoyment is freely pursued, welcomed or accommodated. Restraint: enjoyment is curbed or subordinated to norms or duties. | Entertainment, consumption, smiles or luxury; practical limits such as budgets or safety |
T2I Generation
Text-to-image generation creates images from natural-language prompts using generative models, with research focused on fidelity, control, efficiency, and safety.
See all 332 papers