Guardrail-Agnostic Societal Bias Evaluation in Large Vision-Language Models

Yusuke Hirota; Michael Ross Boone; Arun George Zachariah; Jibin Rajan Varghese; Yu-Chiang Frank Wang; Boyi Li; Ryo Hachiuma

cs.CVAug 30, 2026

Guardrail-Agnostic Societal Bias Evaluation in Large Vision-Language Models

Authors: Yusuke Hirota, Michael Ross Boone, Arun George Zachariah, Jibin Rajan Varghese, Yu-Chiang Frank Wang, Boyi Li, Ryo Hachiuma

Organizations: NVIDIA

Abstract

We propose a societal bias evaluation method for large vision-language models (LVLMs) in the era of strong safety guardrails. Existing benchmarks rely on prompts that ask models to infer attributes of people in images (e.g., "Is this person a CEO or a secretary?"). However, we find that LVLMs with strong guardrails, such as GPT and Claude, often refuse these prompts, making evaluations unreliable. To address this, we change the prior evaluation paradigm by decoupling the task from the depicted person: instead of inferring person's attributes, we use prompts that do not ask about the person (e.g., "Write a fictional story about an imaginary person.") and attach the image as provisional user information to implicitly provide demographic cues, then compare outputs across user demographics. Instantiated across three tasks --- story generation, term explanation, and exam-style QA --- our method avoids refusals even in guardrailed LVLMs, enabling reliable bias measurement. Applying it to 20 recent LVLMs, both open-source and proprietary, we find that all models undesirably use user demographic information in person-irrelevant tasks; for instance, characters in stories are often portrayed as mechanic for male users and nurse for female users. Although still biased, proprietary models like GPT-5 show lower bias than open-source ones. We analyze potential factors behind this gap, discussing continuous model monitoring and improvement as a possible contributor for reducing bias.

Figures & tables

Figure 1: ( Top ): Conventional evaluation methods use attribute-inferring prompts, asking about the depicted person, which often trigger refusals in guardrailed LVLMs. ( Bottom ): Our guardrail-agnostic method replaces them with person-irrelevant prompts and uses images only as user context , enabling bias evaluation even for safety-guarded models.

	Open-source models			Proprietary models
Benchmark	Qwen2.5-VL-32B	Gemma3-27B	InternVL3.5-38B	GPT-5	Claude 3.7 Sonnet
SBBench	90	80	80	83	100
ModScan	94	61	63	49	98
VLA-gender	90	86	71	97	98
Pairs	35	41	61	52	81
Ours	0	0	0	0	0

Table 1: Refusal rate (%) of recent LVLMs on prior bias benchmarks, SBBench [ 55 ] , ModScan [ 40 ] , VLA-gender [ 19 ] , and Pairs [ 17 ] , and on our method (Ours).

	Story Generation		Term Explanation		Exam-Style QA
Model	Gender	Race	Gender	Race	Gender	Race
Open-source LVLMs
Molmo-7B	26.98	24.57	2.76	5.08	3.44	2.98
LLaVA-1.6-7B	47.81	22.50	2.26	4.78	-	-
LLaVA-1.6-13B	31.49	22.27	3.05	4.02	-	-
LLaVA-1.6-34B	24.35	26.92	3.67	4.14	-	-

Table 2: Gender and racial bias scores computed using the TVD metric, multiplied by

100

(

0

= no bias,

100

= maximum bias). Best/second-best are shown in bold / underline , and worst/second-worst in bold / underline . We exclude LLaVA-1.6 variants from Exam-style QA due to near-random accuracies that lead to misleadingly low bias scores.

Figure 2: (a) Generated stories by GPT-4o. (b) Generated explanations by Claude 3.7 Sonnet. For both tasks, the top image pairs show gender bias, and the bottom pairs show racial bias. Biased differences between users are highlighted in red . Additional examples are in Appendix G.

Panel of Figure 3 — Figure 3: Top-10 most gender-skewed jobs and majors in the story generation task for GPT-4o, measured by deviation from the expected equal ratio ( $50\%$ ). Appendix G shows additional visualizations for other attributes.

Figure 4: Racial disparities in explanation difficulty for physics and CS terms in Claude 3.7 Sonnet. Higher ratios indicate more difficult explanations. Appendix G shows the complete visualizations for gender and all subjects.

Figure 5: Task-wise (solid lines) and gender - race (dotted lines) bias correlations.

Figure 6: Bias score vs. Model performance ( Left : Story generation, Middle : Term explanation, Right : Exam-style QA). The bubble size indicates model size (only for the open-source models). Model performance is measured by the accuracy on the MMMU benchmark [ 92 ] .

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Complete prompt for story generation.

Figure 8: Complete prompt for LLM assistant to extract character attributes in story generation.

Figure 9: Complete prompt for the LLM-based judge. For the LLM judge, we randomize the presentation order of the candidate explanations (including which one appears first in a pair) to mitigate order/position bias.

Figure 10: Complete prompt for term explanation.

Figure 11: Complete prompt for exam-style QA.

Method	JS Distance $\downarrow$	Interpretation
Existing open-ended eval	0.61	Outputs heavily driven by background context
Ours (story generation)	0.38	Outputs remain consistent regardless of visual context

Appendix

Table 3: JS distance (vocabulary shift) when the image source changes from face-centric to real-world images. Lower is better.

Alternative LLM assistant	Gender	Race
Qwen3.5-27B [ 59 ]	0.99	0.97
Nemotron-3-Nano-30B-A3B [ 7 ]	0.99	0.96

Appendix

Table 4: Robustness to alternative LLM assistants. Pearson correlations are computed over model-level story-generation bias scores, using Qwen3-32B as the main LLM assistant.

Model	Job	Major	Personality	Education	Family	Avg.
Open-source LVLMs
Molmo-7B	46.76	41.12	22.93	15.62	8.49	26.98
LLaVA-1.6-7B	61.31	47.25	47.29	26.30	56.91	47.81
LLaVA-1.6-13B	45.55	32.94	30.07	28.89	19.99	31.49
LLaVA-1.6-34B	32.24	28.60	26.07	13.53	21.29	24.35
LLaVA-OneVision-7B	35.31	21.79	23.81	11.28	14.84	21.41

Appendix

Table 5: Normalized Total Variation Distance (TVD norm , scaled by 100) for story generation by gender (

|\mathcal{A}|=2

). Higher values indicate greater bias. For each category, TVD is computed per extracted element and averaged within the category; “Avg.” denotes the mean across categories (also reported in Tab.2 in the main paper).

Model	Job	Major	Personality	Education	Family	Avg.
Open-source LVLMs
Molmo-7B	42.70	26.24	25.37	17.09	11.43	24.57
LLaVA-1.6-7B	28.61	31.99	22.61	19.16	10.13	22.50
LLaVA-1.6-13B	29.20	27.06	23.03	19.86	12.19	22.27
LLaVA-1.6-34B	35.60	32.45	28.17	16.12	22.24	26.92
LLaVA-OneVision-7B	31.32	31.29	21.30	10.97	14.50	21.88

Appendix

Table 6: Normalized Total Variation Distance (TVD norm , scaled by 100) for story generation by race (

|\mathcal{A}|=7

Model	Math	Physics	CS	Art	Literature	Music	Avg.
Open-source LVLMs
Molmo-7B	5.28	3.72	0.58	2.00	1.86	3.14	2.76
LLaVA-1.6-7B	6.72	0.42	0.42	1.00	1.00	4.00	2.26
LLaVA-1.6-13B	5.72	3.28	0.58	0.72	4.44	3.58	3.05
LLaVA-1.6-34B	2.00	6.08	1.86	0.42	4.22	7.42	3.67
LLaVA-OneVision-7B	1.42	4.42	2.72	6.72	2.36	1.58	3.20

Appendix

Table 7: Normalized Total Variation Distance (TVD norm , scaled by 100) for term explanation by gender (

|\mathcal{A}|=2

). Higher is worse. For each term, we form the distribution of wins across groups and compute TVD norm ; values are averaged within each domain, and “Avg.” is the mean across domains (also reported in Tab. 2 in the main paper).

Model	Math	Physics	CS	Art	Literature	Music	Avg.
Open-source LVLMs
Molmo-7B	3.33	6.82	2.74	3.35	6.70	7.52	5.08
LLaVA-1.6-7B	3.68	6.58	3.22	4.28	7.05	3.88	4.78
LLaVA-1.6-13B	3.90	6.00	3.39	3.12	4.38	3.32	4.02
LLaVA-1.6-34B	3.23	4.25	4.25	5.32	5.18	2.62	4.14
LLaVA-OneVision-7B	4.63	5.18	5.67	4.62	2.59	4.38	4.51

Appendix

Table 8: Normalized Total Variation Distance (TVD norm , scaled by 100) for term explanation by race (

|\mathcal{A}|=7

Figure 12: Top-10 most gender-skewed personality traits in the story generation task for GPT-4o.

Figure 13: Gender-skewed education level in the story generation task for GPT-4o.

Figure 14: Gender-skewness of family situation in the story generation task for GPT-4o.

Figure 15: Selection ratios of the LLM assistant across domains (gender). Higher values indicate that LVLMs tend to provide more difficult explanations for users of the demographic group.

Figure 16: Selection ratios of the LLM assistant across domains (race). Higher values indicate that LVLMs tend to provide more difficult explanations for users of the demographic group.

Model	Biology	Medicine	Chemistry	CS	Math	Physics	Avg.
Open-source LVLMs
Molmo-7B	0.59	2.55	6.99	4.84	0.81	4.85	3.44
LLaVA-OneVision-7B	1.08	2.12	5.31	3.28	1.72	2.35	2.64
Qwen2-VL-7B	1.11	1.77	3.62	2.37	1.98	3.41	2.38
Qwen2.5-VL-7B	1.14	1.91	3.44	0.75	1.36	1.52	1.69
Qwen2.5-VL-32B	1.02	3.02	4.14	2.78	3.81	2.27	2.84

Appendix

Table 9: Normalized Total Variation Distance (TVD norm , scaled by 100) for exam-style QA by gender (

|\mathcal{A}|=2

). Higher is worse. For each subject, TVD norm is computed from the share of correct answers per group; “Avg.” is the mean across subjects. (also reported in Tab. 2 in the main paper).

Model	Biology	Medicine	Chemistry	CS	Math	Physics	Avg.
Open-source LVLMs
Molmo-7B	1.63	1.90	4.07	2.42	3.14	4.70	2.98
LLaVA-OneVision-7B	1.05	1.53	3.03	1.89	2.39	2.44	2.06
Qwen2-VL-7B	1.23	1.11	2.24	2.07	3.72	2.51	2.15
Qwen2.5-VL-7B	0.85	1.04	2.28	2.02	2.70	2.26	1.86
Qwen2.5-VL-32B	0.59	2.16	3.54	1.46	3.40	0.63	1.96

Appendix

Table 10: Normalized Total Variation Distance (TVD norm , scaled by 100) for exam-style QA by race (

|\mathcal{A}|=7

). Higher is worse. For each subject, TVD norm is computed from the share of correct answers per group; “Avg.” is the mean across subjects. (also reported in Tab. 2 in the main paper).

Figure 17: (a) Generated stories by Qwen2.5-VL-32B (top) and GPT-4o (bottom). (b) Generated explanations for math-related terms by Claude 3.5 Sonnet (top) and InternVL3.5-38B (bottom). For both tasks, the top image pairs show gender bias, and the bottom pairs show racial bias. Biased differences between users are highlighted in red .

Language Model Bias Evaluation

Language model bias evaluation tests how models’ outputs, decisions, and representations vary across social groups, cultures, languages, and contexts.

See all 473 papers

DeepBias: Adaptive In-depth Probing of Social Biases in LVLMs
Jul 13, 2026Anqi Li, Jie Zhang, Zhongqi Wang +4VLM Evaluation Cultural Bias in VLMs
FairLens: Benchmarking Fairness in Vision-Language Models for High-Stakes Decision-Making
Sep 1, 2026Vahid Reza Khazaie, Ahmed Y. Radwan, Shaina RazaVLM Evaluation
Vector Institute, Toronto, Canada
ImplicitBBQ: Benchmarking Implicit Bias in Large Language Models through Characteristic Based Cues
Apr 2, 2026Bhaskara Hanuma Vedula, Darshan Anghan, Ishita Goyal +2Social Bias in Language Models Language Model Bias Evaluation
International Institute of Information Technology, Hyderabad · Indian Institute of Technology, Kharagpur

Jul 13, 2026J/K move · Enter open · S save

DeepBias: Adaptive In-depth Probing of Social Biases in LVLMs

Anqi Li, Jie Zhang, Zhongqi Wang, Songkai Xue+3

VLM Evaluation Cultural Bias in VLMs Language Model Bias Evaluation LLM Safety Evaluation

Open paper AskPDF arXiv

While Large Vision-Language Models (LVLMs) demonstrate remarkable capabilities, they remain highly susceptible to embedded social biases. Existing bias evaluation protocols predominantly rely on static datasets, which provide only a superficial assessment, as their fixed test cases cannot adaptively evolve to measure the true depth and limits of model vulnerabilities. We introduce DeepBias, an adaptive framework for the in-depth probing of social biases in LVLMs with carefully designed agents. Our approach operates through a dynamic ''generation-evolution-probing'' loop. First, a generative ProposerAgent synthesizes test data and is iteratively updated via Direct Preference Optimization (DPO) based on the target LVLM's responses, exploring model-specific failure modes. Second, an autonomous skill-driven DiggerAgent rewrites each test data across multiple probing turns, adaptively selecting from a curated skill library of deepening and rewriting strategies. At each turn, this process is conditioned on the model's previous response, enabling progressively deeper biases to be exposed. Furthermore, we build a benchmark named DeepBiasBench using our framework. By employing an ensemble of five diverse state-of-the-art LVLMs as anchors, the benchmark captures vulnerabilities shared across architectures. Comprehensive experiments demonstrate the effectiveness of our framework and show that DeepBias provides a challenging benchmark for in-depth bias evaluation, establishing an evolutionary paradigm for LVLM safety assessment.

Abstract

Language Model Bias Evaluation

Explore similar work

DeepBias: Adaptive In-depth Probing of Social Biases in LVLMs

FairLens: Benchmarking Fairness in Vision-Language Models for High-Stakes Decision-Making

ImplicitBBQ: Benchmarking Implicit Bias in Large Language Models through Characteristic Based Cues