Unequal Voices: How LLMs Construct Constrained Queer Narratives

Atreya Ghosal; Ashim Gupta; Vivek Srikumar

cs.CYJul 21, 2025

Unequal Voices: How LLMs Construct Constrained Queer Narratives

Authors: Atreya Ghosal, Ashim Gupta, Vivek Srikumar

Organizations: University of Utah

Abstract

One way social groups are marginalized in discourse is that the narratives told about them often default to a narrow, stereotyped range of topics. In contrast, default groups are allowed the full complexity of human existence. We describe the constrained representations of queer people in LLM generations in terms of harmful representations, narrow representations, and discursive othering and formulate hypotheses to test for these phenomena. Our results show that LLMs are significantly limited in their portrayals of queer personas.

Figures & tables

Figure 1: gemma-3-27b-it when asked to assume the persona of a cis man vs. when asked to assume the persona of a gay man. The response by the gay man persona brings up identity-related conflict, while the response by the cis man persona talks about holiday planning with family. Neither response is problematic by itself, but a repeated focus on identity-related themes for queer personas indicates a systematic association bias at play.

Figure 2: Topic divergence between (a) Unmarked and Queer responses (upper line) and (b) Marked Non-Queer and Queer responses (lower line), for models gemma-3-27b-it and Qwen3.5-35B-A3B . There is a significant difference between pairwise topic divergence scores for all models and contexts presented ( $p<0.0001$ ). The confidence interval bars denote $95\%$ CI. Note that topic divergence scores cannot be compared across different contexts or models, as the topic distribution is calculated individually for each context-model pair.

Model	Group	$p$	$\mathrm{std}(p)$	$\theta$	$\mathrm{std}(\theta)$	Likelihood Ratio
gemma-3-27b	Q	$0.62^{***}$	$2.2\times 10^{-8}$	$0.99^{***}$	$1.6\times 10^{-8}$	69.38
	NQ	$0.70$	0.08	$0.37$	0.18	1.77
Llama-3.3-70B	Q	$0.26^{***}$	$1.6\times 10^{-8}$	$1.00^{***}$	$6.1\times 10^{-9}$	225.81
	NQ	${2.2\times 10^{-7}}$	$1.2\times 10^{-8}$	$0.29^{***}$	$6.0\times 10^{-10}$	53.46

Table 1: Maximum likelihood estimated parameters from the human labeled data, for groups Q and NQ i.e. Queer and Non-Queer .

\mathrm{std}(param)

is the standard deviation of the parameter over multiple runs of EM with random initializations. We look at the order of magnitude of this value as an indicator of the stability of the associated parameter. Significance stars are for the corresponding null hypotheses:

1-p_{q}>0

1-p_{nq}>0

\theta_{q}>0.5

1-\theta_{nq}>0.5

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 1: Topic divergence between (a) Unmarked and Queer responses (upper line) and (b) Marked Non-Queer and Queer responses (lower line), for all four models tested. The omnibus test shows that there is a significant difference between pairwise topic divergence scores for all models and contexts presented ( $p<0.0001$ ). All pairwise topic divergence scores are also statistically significant at $p<0.0001$ . The confidence interval bars denote $95\%$ CI calculated using the bootstrap test with $10^{4}$ resamplings. Note that the horizontal axes of the two graphs are not the same.

Midpoint	Group	$p$	$\mathrm{std}(p)$	$\theta$	$\mathrm{std}(\theta)$	Likelihood Ratio
Queer	Q	$0.62^{***}$	$2.2\times 10^{-8}$	$0.99^{***}$	$1.6\times 10^{-8}$	69.38
	NQ	$0.70$	0.08	$0.37$	0.18	1.77
Nonqueer	Q	$0.63^{***}$	0.08	$0.99^{***}$	0.39	65.84
	NQ	$0.49^{***}$	0.15	$0.09^{***}$	0.27	66.29

Appendix

Table 1: Results for generations from model gemma-3-27b-it . Significance stars are for the corresponding null hypotheses:

1-p_{q}>0

1-p_{nq}>0

\theta_{q}>0.5

1-\theta_{nq}>0.5

Midpoint	Group	$p$	$\mathrm{std}(p)$	$\theta$	$\mathrm{std}(\theta)$	Likelihood Ratio
Queer	Q	$0.26^{***}$	$1.62\times 10^{-8}$	$1.00^{***}$	$6.15\times 10^{-9}$	225.81
	NQ	${2.23\times 10^{-7}}$	$1.23\times 10^{-8}$	$0.29^{***}$	$6.04\times 10^{-10}$	53.46
Nonqueer	Q	$0.30^{***}$	$3.28\times 10^{-9}$	$1.00^{***}$	$1.08\times 10^{-9}$	201.23
	NQ	$0.04$	$3.64\times 10^{-3}$	${5.22\times 10^{-3}}^{***}$	$1.11\times 10^{-3}$	365.44

Appendix

Table 2: Results for generations from model Llama-3.3-70B-Instruct . Significance stars are for the corresponding null hypotheses:

1-p_{q}>0

1-p_{nq}>0

\theta_{q}>0.5

1-\theta_{nq}>0.5

File name	Queer $>$ non-queer	Queer $>$ unmarked
Housing	$(0.87>0.79)$ ***	$(0.87>0.73)$ ***
Medical	$(0.84>0.82)$ ***	$(0.84>0.75)$ ***
Persona	$(0.94>0.83)$ ***	$(0.94>0.76)$ ***
Recomm.	$(0.92>0.67)$ ***	$(0.92>0.62)$ ***
Work	$(0.88>0.77)$ ***	$(0.88>0.73)$ ***

Appendix

Table 3: Pairwise tests of per-label classification accuracy across contexts for the model gemma-3-27b-it . Pairwise entries report the classification accuracy for Queer responses followed by the comparison group.

File name	Queer $>$ non-queer	Queer $>$ unmarked
Housing	$(0.85>0.83)$ ***	$(0.85>0.67)$ ***
Medical	$(0.91>0.92)$ -	$(0.91>0.80)$ ***
Persona	$(0.94>0.91)$ ***	$(0.94>0.83)$ ***
Recomm.	$(0.89>0.79)$ ***	$(0.89>0.79)$ ***
Work	$(0.92>0.82)$ ***	$(0.92>0.75)$ ***

Appendix

Table 4: Pairwise tests of per-label classification accuracy across contexts for the model Llama-3.3-70B-Instruct . Pairwise entries report the classification accuracy for Queer responses followed by the comparison group.

File name	Queer $>$ non-queer	Queer $>$ unmarked
Housing	$(0.85>0.38)$ ***	$(0.85>0.33)$ ***
Medical	$(0.82>0.57)$ ***	$(0.82>0.30)$ ***
Persona	$(0.79>0.48)$ ***	$(0.79>0.34)$ ***
Recomm.	$(0.87>0.38)$ ***	$(0.87>0.40)$ ***
Work	$(0.86>0.46)$ ***	$(0.86>0.42)$ ***

Appendix

Table 5: Pairwise tests of per-label classification accuracy across contexts for the model gpt-oss-120b . Pairwise entries report the classification accuracy for Queer responses followed by the comparison group.

File name	Queer $>$ non-queer	Queer $>$ unmarked
Housing	$(0.82>0.64)$ ***	$(0.82>0.18)$ ***
Medical	$(0.94>0.82)$ ***	$(0.94>0.24)$ ***
Persona	$(0.81>0.66)$ ***	$(0.81>0.33)$ ***
Recomm.	$(0.73>0.41)$ ***	$(0.73>0.84)$ -
Work	$(0.75>0.56)$ ***	$(0.75>0.49)$ ***

Appendix

Table 6: Pairwise tests of per-label classification accuracy across contexts for the model Qwen3.5-35B-A3B . Pairwise entries report the classification accuracy for Queer responses followed by the comparison group.

Context	Queer	Marked Non-Queer	Unmarked
Housing	binary, problem, regardless, pen, nerve, fashioned, appropriate, eyes, appreciated, termination	professionally, https, road, equipped, intended, poems, blood, org, scripts	ceiling, leak, inquired, traveling, establish, strengths, rats, rectify, priced, mold
Medical	transition, binary, exploring, research, practice, wellbeing, growing, respecting, includes, pronouns	problems, engineer, mistakes, chatbot, thyroid, prints, manager, organized, screen, nature	spinning, cramps, dog, unimportant, meditation, scan, optimistic, malaise, tremendous, unpleasant
Persona	slip, journey, misgendered, navigating, altogether, correcting, god, hormones, outside, chosen	subtly, video, involve, major, smaller, property, forgot, seven, reeling, reliable	oven, drip, volume, legendary, piles, villain, taxidermist, alarm, review, meals
Recomm.	binary, identifies, resilience, transition, navigating, strength, rights, elements, historical, receive	attended, witnessed, ancient, julia, housing, stopping, rise, robot, tree, deception	gaps, strengthen, facts, ambassador, ambiguity, impressively, sticking, honors
Work	neutral, partner, orientation, sensitivity, shared, probably, experiences, inclusivity, identifies, upset	delay, cleaning, initiate, text, weekly, sick, perform, pm	sounding, promotion, reconciliation, attendance, advances, troubleshoot, unforeseen, effect, possibilities

Appendix

Table 7: Most strongly label-associated words returned by the BOW classifier for model gemma-3-27b-it

Context	Queer	Marked Non-Queer	Unmarked
Housing	partner, inclusive, welcoming, identity, assure, regardless, furnish, backgrounds	guy, breaks, steps, control, terminated, sold, id, multiple, delighted, unfortunate	familiar, clauses, windows, stopped, machine, washing, special, residential, reality
Medical	hormone, replacement, transition, pronouns, identity, partner, use, healthcare, providing, acknowledge	yesterday, skeptical, wellbeing, killing, pinpoint, notch, screenings, clarification, quality, horrible	exert, directly, tissue, main, tightness, wasted, toll, seat, end, stairs
Persona	pronouns, journey, accept, books, bookstore, transition, called, experiences	thanks, counting, gain, loud, stand, cookie, sandwich, engineering, sports, brands	rivalries, account, paying, stable, solve, shaped, wise, following, clashes, footsteps
Recomm.	perspective, lifestyle, choices, resilience, encouragement, open, face, transition, life, space	comments, marginalized, behavior, dismissive, hostile, groups, short, jokes, fallen, foster	remind, assisted, moments, drives, hoped, mediocre, evidence, booster, passing, pool
Work	mention, inclusive, personal, unique, note, identity, diversity, comfortable, inclusion	specifications, oversights, headway, fits, deliverable, accuracy, software, documents, specs, miscommunications	uncertain, weren, resulting, business, stuck, slowed, term, differently, walk, sustainable

Appendix

Table 8: Most strongly label-associated words returned by the BOW classifier for model Llama-3.3-70B-Instruct

Context	Queer	Marked Non-Queer	Unmarked
Housing	pronouns, supportive, discriminatory, accommodations, diversity, regardless, respects, inclusivity, orientation, discrimination	police, officer, comedic, fitness, revealing, comedy, begins, noisy, brand, properly	construct, noinput, faced, acceptable, chatcompletion, score, safely, davinci, roleplayer
Medical	pronouns, affirming, transition, hormone, resources, therapy, space, identity, aspects, orientation	cardiovascular, month, insurance, developer, thro, fulfill, const, cough, evaluation	ellipsis, adds, rabbit, sir, triggered, picture, kwargs, dynamic, modern
Persona	pronouns, workplaces, acceptance, advocacy, inclusivity, diversity, affirming, rights, queer, binary	arguments, established, introduction, regex, strategies, successes, construction, traveler	suffix, brothers, capricorn, taurus, gemini, batteries, jupiter, cycle
Recomm.	social, discriminate, diversity, identifies, navigating, discrimination, identity, inclusion, discriminatory, faced	assert, msg, dark, souls, status, rephrasing, failed, remains	nite, excelling, advisory, div, pdf, cuts, adept, spanish, lakeshore, unparalleled
Work	pronouns, accommodations, diversity, inclusivity, inclusion, prefer, perspectives, identifies, reflect, comfortable	approximately, expert, cursor, korean, vote, source, verification, analyst, strain	outliers, revamped, coworkers, js, expectation, considerate, researched, assistantfinalyes, endpoint, crafting

Appendix

Table 9: Most strongly label-associated words returned by the BOW classifier for model gpt-oss-120b

Context	Queer	Marked Non-Queer	Unmarked
Housing	inclusivity, pronouns, identity, judgment, discrimination, diversity, worrying, subtly, dignity, diverse	accountant, marketing, residential, sector, sustain, quietness, coin, doorman	simplify, prices, allotted, entails, urgency, insight, appealing, hovering, appliance
Medical	pronouns, hormone, affirming, identity, inclusive, correct, dysphoria, misgendering, preferred, sexual	irregular, menstrual, balances, grief, thyroid, hypochondriac, energetic, brushed, hydration, town	meticulously, lowest, garden, useless, dog, controlling, throb, grandchildren, breathless, callback
Persona	pronouns, chosen, inclusive, diversity, acceptance, community, diverse, advocacy, center, companies	electrician, sports, stuffing, generational, buddies, demand, pin, hvac, ride, plumbing	disagreements, travelling, crafting, scenarios, harmony, rude, histories, hypothetically, flowery, owns
Recomm.	marginalized, lived, alliance, welcoming, ally, communities, intersectionality, justice, refuses, identities	weird, audiences, experimental, irrelevant, protocols, reliability, communicate, odd, setbacks, biological	consumer, dropped, emphasis, marketing, shortcomings, tutelage, municipal, oversaw, hinges
Work	pronouns, inclusivity, inclusive, diversity, inclusion, respected, partner, ally, diverse, subtly	reallocating, bugs, intend, risks, contractor, reliable, considerably, reached, control, corresponding	upload, tenure, resignation, pixel, recommendation

Appendix

Table 10: Most strongly label-associated words returned by the BOW classifier for model Qwen3.5-35B-A3B

Context	queer vs neutral	non-queer vs neutral	queer vs non-queer
Housing	(smile, hand, come, small), (noise, complaints, night, received), (scenario, roleplaying, request, ai), (wanted, just, 15th, signed), (rent, month, lease, renewal), (discrimination, housing, gender, orientation), (im, lease, landlord, rent), (im, designer, graphic, really), (home, im, quiet, place), (income, credit, check, history)	(notice, vacate, understand, apologize), (smile, hand, come, small), (noise, complaints, night, received), (application, approved, lease, glad), (scenario, roleplaying, request, ai), (wanted, just, 15th, signed), (ai, assistant, helpful, sexually), (im, designer, graphic, really), (home, im, quiet, place), (discrimination, housing, gender, orientation)	(natural, light, quiet, looking), (application, approved, lease, glad), (im, place, looking, safe), (rent, month, lease, renewal), (lease, wanted, just, desk), (income, credit, check, history), (chair, desk, gesture, come), (landlord, youre, im, little)
Medical	(enjoy, used, things, just), (desk, chair, dr, smile), (doctor, tired, sleep, ive), (stress, im, complaining, told), (hrt, changes, im, therapy), (therapy, hormone, testosterone, im), (headaches, headache, tired, worried), (pronouns, use, want, correct), (tired, sleep, ive, work), (ive, feeling, just, really)	(doctor, tired, sleep, ive), (stress, im, complaining, told), (headaches, headache, tired, worried), (pronouns, use, want, correct), (tired, sleep, ive, work), (ive, feeling, just, really), (desk, opposite, warm, gesture), (gender, ive, really, nonbinary), (like, really, feeling, just), (hormone, levels, results, good)	(results, looks, normal, good), (enjoy, used, things, just), (just, tired, im, frustrated), (nights, ive, sleep, tired), (hrt, changes, im, therapy), (therapy, hormone, testosterone, im), (pronouns, use, want, correct), (gender, ive, really, nonbinary), (just, im, really, like), (desk, opposite, warm, gesture)
Persona	(vent, just, having, like), (boss, work, shortstaffed, im), (promotion, did, spill, leave), (okay, sorry, dont, week), (work, design, creative, youre), (design, youre, good, creative), (holidays, year, family, questions), (year, holidays, gathering, im), (year, gingerbread, im, cookies), (shes, pronouns, trying, complicated)	(youre, creative, design, jobs), (year, gingerbread, im, cookies), (year, holidays, gathering, im), (holidays, christmas, year, im), (plans, holidays, year, trying)	(vent, just, having, like), (youre, creative, design, jobs), (promotion, did, spill, leave), (boss, work, shortstaffed, im), (holidays, year, family, questions), (work, design, creative, youre), (okay, sorry, dont, week), (holidays, christmas, year, im), (shes, pronouns, trying, complicated), (friend, supportive, playing, saying)
Recomm.	(really, said, internship, voice), (internship, really, im, class), (finds, professors, hope, request), (upset, professor, youre, response), (really, im, internship, good), (gender, want, letter, youre), (consistently, students, positive, candidate), (manager, committeehiring, title, admissions)	(letter, recommendation, personal, knowledge), (really, said, internship, voice), (really, environmental, internship, im), (upset, professor, youre, response), (manager, committeehiring, title, admissions), (internship, really, im, class), (ai, letter, ethical, recommendation), (gender, want, letter, youre)	(letter, recommendation, personal, knowledge), (really, said, internship, voice), (student, draft, applying, letter), (ai, letter, ethical, recommendation), (really, environmental, internship, im), (consistently, students, positive, candidate), (committeehiring, manager, admissions, title), (desk, come, smile, chair), (gender, want, letter, youre), (really, im, internship, good)
Work	(im, overwhelmed, feeling, just), (ive, noticed, project, reports), (gender, supervisor, harmful, identity), (understand, reports, deadlines, missed), (im, little, wanted, really), (henderson, report, specifically, pattern), (marketing, data, campaign, media), (really, things, taking, check), (im, ive, wanted, want), (deadlines, missed, employee, noticed)	(im, overwhelmed, feeling, just), (ive, noticed, project, reports), (im, overwhelmed, worried, tasks), (really, things, taking, check), (henderson, account, really, good), (doing, im, really, check), (marketing, data, campaign, media), (employee, noticed, wanted, value), (sharing, supervisor, aiming, courage)	(pronouns, nonbinary, gender, theythem), (gender, supervisor, harmful, identity), (understand, reports, deadlines, missed), (im, overwhelmed, worried, tasks), (im, little, wanted, really), (noticed, dip, want, ive), (deadlines, missed, employee, noticed), (henderson, report, specifically, pattern), (doing, im, really, check), (eager, im, contribute, responsibility)

Appendix

Table 11: Over-represented topics for label A vs. label B for the model gemma-3-27b-it . Blank spaces denote that no topics were significantly over-represented for label A of that label pair.

Context	queer vs neutral	non-queer vs neutral	queer vs non-queer
Housing	(making, weve, really, partner), (youre, property, rental, whats), (youve, tenants, recently, regarding), (unfortunately, payments, youve, afraid), (thats, place, im, quiet), (ive, yeah, base, touch), (im, tenants, causing, landlord), (rental, property, available, units), (neighborhood, quiet, looking, hi), (ive, look, got, paperwork)	(im, weve, 1200, lease), (unfortunately, payments, youve, afraid), (notice, youve, thank, afraid), (rental, property, available, units), (assure, application, providing, weve)	(im, weve, 1200, lease), (youre, property, rental, whats), (making, weve, really, partner), (unfortunately, payments, youve, afraid), (thats, place, im, quiet), (youve, tenants, recently, regarding), (ive, yeah, base, touch), (rental, property, available, units), (ive, signed, just, lease), (notice, youve, thank, afraid)
Medical	(ive, having, lately, fatigue), (pronouns, want, welcome, glad), (taking, medical, little, brings), (tell, youre, listen, way), (tell, whats, mind, thats), (plan, gratitude, really, manage), (ive, im, assistant, today), (struggling, life, able, gratitude), (options, ive, research, transition), (recovery, weve, treatment, challenging)	(ive, having, lately, fatigue), (tell, youre, listen, way), (tell, whats, mind, thats), (ive, im, assistant, today), (plan, gratitude, really, manage), (primary, physician, care, medical), (oh, finally, waiting, 20), (youre, clarify, upset, whats), (weve, results, looks, news), (recovery, weve, treatment, challenging)	(ive, having, lately, fatigue), (pronouns, want, welcome, glad), (taking, medical, little, brings), (time, 20, minutes, oh), (weve, treatment, news, recovery), (primary, physician, care, medical), (plan, gratitude, really, manage), (struggling, life, able, gratitude), (options, ive, research, transition), (great, hormone, therapy, news)
Persona	(im, assistant, really, know), (youd, design, nonprofit, thought), (grind, shop, daily, called), (came, mom, journey, course), (dont, parents, frustrating, used), (havent, caught, ages, pretty), (youre, tough, sorry, bad), (eye, family, siblings, love), (books, bookstore, page, day), (work, lunch, firm, marketing)	(im, assistant, really, know), (youd, design, nonprofit, thought), (youre, tough, sorry, bad), (havent, caught, ages, pretty), (sales, great, youd, think), (eye, family, siblings, love), (youre, tough, im, want), (youre, tough, week, im), (field, design, thought, graphic), (celebration, smile, tell, face)	(im, assistant, really, know), (grind, shop, daily, called), (havent, caught, ages, pretty), (came, mom, journey, course), (dont, parents, frustrating, used), (youre, tough, sorry, bad), (work, lunch, firm, marketing), (eye, family, siblings, love), (books, bookstore, page, day), (excited, holidays, planning, christmas)
Recomm.	(dont, havent, feel, requires), (certain, say, afraid, values), (students, academic, recipient, exceptional), (wondering, enjoyed, think, really), (ive, understand, trying, thought), (world, means, thrilled, heartfelt), (think, im, ive, wondering), (students, assistant, academic, ive), (having, dont, letter, youve), (students, individual, future, recipient)	(just, endorsement, thrilled, express), (dont, havent, feel, requires), (certain, say, afraid, values), (students, academic, recipient, exceptional), (think, hoping, ive, letter), (wondering, enjoyed, think, really), (having, dont, letter, youve), (good, wont, ive, relationship), (students, assistant, academic, ive), (students, academic, attest, confidently)	(just, endorsement, thrilled, express), (dont, havent, feel, requires), (ive, understand, trying, thought), (wondering, enjoyed, think, really), (world, means, thrilled, heartfelt), (think, im, ive, wondering), (good, wont, ive, relationship), (students, assistant, academic, ive), (committee, students, admissions, academic), (students, individual, future, recipient)
Work	(grow, think, develop, areas), (marketing, campaign, new, current), (smith, taking, account, hard), (10, account, weeks, sales), (really, best, hours, extra), (reviewed, completing, noticed, whats), (draft, hours, extra, latest), (meet, im, good, ive), (team, youve, want, coming), (campaign, marketing, launch, product)	(seat, consistently, moment, say), (marketing, campaign, new, current), (10, account, weeks, sales), (seat, dedication, extremely, say), (team, youve, want, coming), (downs, ups, pattern, understand), (mistakes, minor, job, youve), (deadline, project, im, bit), (media, social, enjoying, marketing)	(grow, think, develop, areas), (seat, consistently, moment, say), (marketing, campaign, new, current), (smith, taking, account, hard), (10, account, weeks, sales), (reviewed, completing, noticed, whats), (really, best, hours, extra), (seat, dedication, extremely, say), (ive, im, wanted, assistant), (meet, im, good, ive)

Appendix

Table 12: Over-represented topics for label A vs. label B for the model Llama-3.3-70B-Instruct . Blank spaces denote that no topics were significantly over-represented for label A of that label pair.

Context	queer vs neutral	non-queer vs neutral	queer vs non-queer
Housing	(housing, looking, talking, chatgpt), (prospective, interviewing, requirements, talk), (approved, lease, enquire, notifying)	(housing, looking, talking, chatgpt), (approved, lease, enquire, notifying), (im, space, place, apartment), (eviction, notice, evicting, tenant)	(im, ive, know, let), (informing, signed, user, chatgpt), (im, space, place, apartment), (sorry, like, im, help), (user, landlord, chatgpt, roleplay)
Medical	(json, role, assistant, content), (pronouns, use, male, female), (good, news, giving, doctor), (doctor, patient, words, user), (chatgpt, model, message, trained), (im, today, like, hello)		(pronouns, use, male, female), (content, disallowed, assistant, roleplay), (patient, words, doctor, respond), (good, news, giving, doctor), (im, today, like, hello)
Persona	(community, center, workshops, events), (im, holidays, holiday, ive), (work, team, tech, hey), (plans, words, holiday, holidays)	(jobs, ask, suited, interests), (fit, roles, design, youre), (plans, words, holiday, holidays)	(reassurance, supportive, bad, friend), (community, center, workshops, events), (im, holidays, holiday, ive), (jobs, ask, suited, interests), (work, team, tech, hey), (fit, roles, design, youre), (plans, words, holiday, holidays), (openai, api, def, create), (yaml, role, content, assistant), (chatgpt, model, trained, large)
Recomm.
Work		(employee, user, supervisor, work), (assistant, user, supervisor, performance), (chatgpt, user, supervisor, employee), (chatgpt, model, large, trained)	(chatgpt, user, model, trained), (openai, user, policy, comply), (employee, user, supervisor, work), (chatgpt, model, large, trained)

Appendix

Table 13: Over-represented topics for label A vs. label B for model gpt-oss-120b . Blank spaces denote that no topics were significantly over-represented for label A of that label pair.

Context	queer vs neutral	non-queer vs neutral	queer vs non-queer
Housing	(notice, decision, vacate, tenancy), (reason, state, purpose, eviction), (ive, wanted, really, just), (mention, greeting, confirm, signed), (im, month, looking, id), (mention, budget, location, greeting)	(budget, 1500, need, looking), (leave, sit, need, dont), (mention, greeting, confirm, signed), (mention, budget, location, greeting)	(budget, 1500, need, looking), (reason, state, purpose, eviction), (notice, decision, vacate, tenancy), (leave, sit, need, dont), (ive, wanted, really, just), (mention, greeting, confirm, signed), (lease, inform, property, application), (mention, budget, location, greeting), (im, month, looking, id), (specific, requirements, want, needs)
Medical	(ive, im, sure, bit), (im, like, need, dont), (today, want, dr, feel), (im, know, just, like), (thank, youve, felt, feel), (results, plan, markers, news), (levels, results, work, hormone), (ive, lately, weeks, im), (im, just, like, ive), (thank, difference, really, feel)		(ive, im, sure, bit), (im, like, need, dont), (today, want, dr, feel), (im, know, just, like), (thank, youve, felt, feel), (results, plan, markers, news), (levels, results, work, hormone), (ive, lately, weeks, im), (im, just, like, ive), (thank, difference, really, feel)
Persona	(home, good, doing, maybe), (just, okay, dont, youre), (sorry, youre, strong, im), (year, im, thinking, just), (offer, validate, remind, acknowledge), (gender, tech, think, dont), (downtown, busy, work, marketing), (downtown, work, new, hear), (gender, think, tech, youre), (wow, tell, celebrate, im)	(youre, people, maybe, think), (just, okay, dont, youre), (year, im, thinking, just), (plans, greeting, ask, family), (maybe, question, thats, creative)	(home, good, doing, maybe), (youre, people, maybe, think), (just, okay, dont, youre), (sorry, youre, strong, im), (tech, think, youre, maybe), (year, im, thinking, just), (downtown, busy, work, marketing), (downtown, work, new, hear), (gender, think, tech, youre), (offer, validate, remind, acknowledge)
Recomm.	(know, thank, really, time), (letter, professional, wish, encourage), (program, recommend, community, possesses), (program, recommend, possesses, complex), (salutation, signoff, identity, student), (elena, program, community, recommend), (really, im, know, time), (thought, worked, good, need), (time, thank, know, really), (safety, discrimination, roleplay, based)	(biology, program, research, recommend), (program, recommend, community, possesses), (program, recommend, possesses, complex), (biology, program, research, exceptional), (salutation, signoff, identity, student), (time, thank, know, really), (elena, program, community, recommend), (recommend, sincerely, dear, committee), (thanks, lot, means, yes), (thanks, seeing, grad, school)	(know, thank, really, time), (thanks, seeing, grad, school), (letter, professional, wish, encourage), (program, recommend, possesses, complex), (thank, wanted, time, know), (biology, program, research, recommend), (program, recommend, community, possesses), (biology, program, research, exceptional), (elena, program, community, recommend), (really, im, know, time)
Work	(think, ive, wanted, q3), (youve, thank, difference, confidence), (employees, highlight, purpose, state), (mention, ask, purpose, greet), (supervisor, ask, mention, word), (im, like, need, just), (mention, greet, ask, sit), (mention, ask, count, word), (great, talk, lets, thanks), (express, mention, lack, acknowledge)	(im, like, need, just), (thanks, say, lot, seeing), (express, mention, lack, acknowledge), (great, talk, lets, thanks), (reiterate, gratitude, impact, request), (think, ive, wanted, q3), (employees, highlight, purpose, state)	(think, ive, wanted, q3), (youve, thank, difference, confidence), (employees, highlight, purpose, state), (mention, ask, purpose, greet), (purpose, offer, state, specific), (ensure, need, performance, mention), (reiterate, gratitude, impact, request), (q3, want, team, work), (mention, greet, ask, sit), (want, team, quarter, work)

Appendix

Table 14: Over-represented topics for label A vs. label B, for model Qwen3.5-35B-A3B . Blank spaces denote that no topics were significantly over-represented for label A of that label pair.

Figure 2: Instructions given to annotators for data collection for the human study

Panel of Figure 3 — Figure 3: Examples given to annotators for data collection for the human study

Panel of Figure 4 — Figure 4: Examples given to annotators for data collection for the human study

Social Bias in Language Models

Social bias in language models is the study and mitigation of unfair stereotypes and disparities in generated text, judgments, and decisions.

See all 199 papers

Apr 24, 2026

Representational Harms in LLM-Generated Narratives Against Global Majority Nationalities

Large language models (LLMs) are increasingly used for text generation tasks from everyday use to high-stakes enterprise and government applications, including simulated interviews with asylum seekers. While many works highlight the new potential applications of LLMs, there are risks of LLMs encoding and perpetuating harmful biases about non-dominant communities across the globe. To better evaluate and mitigate such harms, more research examining how LLMs portray diverse individuals is needed. In this work, we study how national origin identities are portrayed by widely-adopted LLMs in response to open-ended narrative generation prompts. Our findings demonstrate the presence of persistent representational harms by national origin, including harmful stereotypes, erasure, and one-dimensional portrayals of Global Majority identities. Minoritized national identities are simultaneously underrepresented in power-neutral stories and overrepresented in subordinated character portrayals, which are over fifty times more likely to appear than dominant portrayals. The degree of harm is amplified when US nationality cues (e.g., ``American'') are present in input prompts. Notably, we find that the harms we identify cannot be explained away via sycophancy, as US-centric biases persist even when replacing US nationality cues with non-US national identities in the prompts. Based on our findings, we call for further exploration of cultural harms in LLMs through methodologies that center Global Majority perspectives and challenge the uncritical adoption of US-based LLMs for the classification, surveillance, and misrepresentation of the majority of our planet.

Ilana Nguyen, Harini Suresh, Thema Monroe-White +1

Brown University, USA · Brown University, Providence, Rhode Island, USA · George Mason University, Fairfax, Virginia, USA +1

Cultural Bias in Language Models Language Model Bias Evaluation Demographic Bias in Language Models

Ask PDF

Apr 2, 2026

Shiny Stories, Hidden Struggles: Investigating the Representation of Disability Through the Lens of LLMs

Modern Large Language Models (LLMs) have recently attracted much attention for their ability to simulate human behavior and generate text that reflects personas and demographic groups. While these capabilities can open up a multitude of diverse applications across fields, it is crucial to examine how such models represent various target groups since LLMs can perpetuate and amplify biases or discrimination against historically marginalized communities or, alternatively, as a result of debiasing efforts, overcorrect by portraying overly positive stereotypes. This overcompensation can idealize these groups, erasing the complexities and challenges they face in favor of unrealistic depictions. In this paper, we investigate how LLMs represent disability by simulating the perspectives of individuals with disabilities in generating social media posts. These posts are then compared with those written by real people with disabilities, focusing on emotional tone, sentiment, and representative words and themes. Our analysis reveals two key findings: (1) LLMs often idealize the experiences of people with disabilities, producing overly positive stereotypes that, despite appearing uplifting, fail to authentically capture their lived realities; and (2) a comparative analysis of posts simulating individuals with and without disabilities highlights a negative bias, where certain topics, such as career and entertainment, are disproportionately associated with nondisabled individuals. This reinforces exclusionary narratives and over-idealized portrayals of disability, misrepresenting the actual challenges faced by this community. These findings align with broader concerns and ongoing research showing that LLMs struggle to reflect the diverse realities of society, particularly the nuanced experiences of marginalized groups, and underscore the need for critical scrutiny of their representations.

Marco Bombieri, Simone Paolo Ponzetto, Marco Rospocher

Department of Information Engineering and Computer Science, University of Trento, Italy · Data and Web Science Group, University of Mannheim, Germany

Social Bias in Language Models Language Model Bias Evaluation Large Language Model-Based Role-Play Simulation

Ask PDF

May 11, 2026

StereoTales: A Multilingual Framework for Open-Ended Stereotype Discovery in LLMs

Multilingual studies of social bias in open-ended LLM generation remain limited: most existing benchmarks are English-centric, template-based, or restricted to recognizing pre-specified stereotypes. We introduce StereoTales, a multilingual dataset and evaluation pipeline for systematically studying the emergence of social bias in open-ended LLM generation. The dataset covers 10 languages and 79 socio-demographic attributes, and comprises over 650k stories generated by 23 recent LLMs, each annotated with the socio-demographic profile of the protagonist across 19 dimensions. From these, we apply statistical tests to identify more than 1{,}500 over-represented associations, which we then rate for harmfulness through both a panel of humans (N = 247) and the same LLMs. We report three main findings. (i) Every model we evaluate emits consequential harmful stereotypes in open-ended generation, regardless of size or capabilities, and these associations are largely shared across providers rather than isolated misbehaviors. (ii) Prompt language strongly shapes which stereotypes appear: rather than transferring as a shared set of biases, harmful associations adapt culturally to the prompt language and amplify bias against locally salient protected groups. (iii) Human and LLM harmfulness judgments are broadly aligned (Spearman

ρ=0.62

), with disagreements concentrating on specific attribute classes rather than specific providers. To support further analyses, we release the evaluation code and the dataset, including model generations, attribute annotations, and harmfulness ratings.

Pierre Le Jeune, Étienne Duchesne, Weixuan Xiao +4

Giskard AI · École Normale Supérieure, INSERM, Paris

Open-Ended Generation Multilingual Language Model Evaluation Language Model Generation Evaluation Social Bias in Language Models Cultural Bias in Language Models Language Model Bias Evaluation

Ask PDF

Abstract

Social Bias in Language Models

Explore similar work

Representational Harms in LLM-Generated Narratives Against Global Majority Nationalities

Shiny Stories, Hidden Struggles: Investigating the Representation of Disability Through the Lens of LLMs

StereoTales: A Multilingual Framework for Open-Ended Stereotype Discovery in LLMs