cs.CYJul 21, 2025

Unequal Voices: How LLMs Construct Constrained Queer Narratives

Authors: Atreya Ghosal, Ashim Gupta, Vivek Srikumar

Organizations: University of Utah

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

One way social groups are marginalized in discourse is that the narratives told about them often default to a narrow, stereotyped range of topics. In contrast, default groups are allowed the full complexity of human existence. We describe the constrained representations of queer people in LLM generations in terms of harmful representations, narrow representations, and discursive othering and formulate hypotheses to test for these phenomena. Our results show that LLMs are significantly limited in their portrayals of queer personas.

Figures & tables

Figure 1: gemma-3-27b-it when asked to assume the persona of a cis man vs. when asked to assume the persona of a gay man. The response by the gay man persona brings up identity-related conflict, while the response by the cis man persona talks about holiday planning with family. Neither response is problematic by itself, but a repeated focus on identity-related themes for queer personas indicates a systematic association bias at play.
Figure 2: Topic divergence between (a) Unmarked and Queer responses (upper line) and (b) Marked Non-Queer and Queer responses (lower line), for models gemma-3-27b-it and Qwen3.5-35B-A3B . There is a significant difference between pairwise topic divergence scores for all models and contexts presented ( p<0.0001p<0.0001 ). The confidence interval bars denote 95%95\% CI. Note that topic divergence scores cannot be compared across different contexts or models, as the topic distribution is calculated individually for each context-model pair.
ModelGroupppstd(p)\mathrm{std}(p)θ\thetastd(θ)\mathrm{std}(\theta)Likelihood Ratio
gemma-3-27bQ0.62∗∗∗0.62^{***}2.2×10−82.2\times 10^{-8}0.99∗∗∗0.99^{***}1.6×10−81.6\times 10^{-8}69.38
 NQ0.700.700.080.370.370.181.77
Llama-3.3-70BQ0.26∗∗∗0.26^{***}1.6×10−81.6\times 10^{-8}1.00∗∗∗1.00^{***}6.1×10−96.1\times 10^{-9}225.81
 NQ2.2×10−7{2.2\times 10^{-7}}1.2×10−81.2\times 10^{-8}0.29∗∗∗0.29^{***}6.0×10−106.0\times 10^{-10}53.46
Table 1: Maximum likelihood estimated parameters from the human labeled data, for groups Q and NQ i.e. Queer and Non-Queer . std(param)\mathrm{std}(param) is the standard deviation of the parameter over multiple runs of EM with random initializations. We look at the order of magnitude of this value as an indicator of the stability of the associated parameter. Significance stars are for the corresponding null hypotheses: 1−pq>01-p_{q}>0 , 1−pnq>01-p_{nq}>0 , θq>0.5\theta_{q}>0.5 , 1−θnq>0.51-\theta_{nq}>0.5 .
Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 1: Topic divergence between (a) Unmarked and Queer responses (upper line) and (b) Marked Non-Queer and Queer responses (lower line), for all four models tested. The omnibus test shows that there is a significant difference between pairwise topic divergence scores for all models and contexts presented ( p<0.0001p<0.0001 ). All pairwise topic divergence scores are also statistically significant at p<0.0001p<0.0001 . The confidence interval bars denote 95%95\% CI calculated using the bootstrap test with 10410^{4} resamplings. Note that the horizontal axes of the two graphs are not the same.
MidpointGroupppstd(p)\mathrm{std}(p)θ\thetastd(θ)\mathrm{std}(\theta)Likelihood Ratio
QueerQ0.62∗∗∗0.62^{***}2.2×10−82.2\times 10^{-8}0.99∗∗∗0.99^{***}1.6×10−81.6\times 10^{-8}69.38
 NQ0.700.700.080.370.370.181.77
NonqueerQ0.63∗∗∗0.63^{***}0.080.99∗∗∗0.99^{***}0.3965.84
 NQ0.49∗∗∗0.49^{***}0.150.09∗∗∗0.09^{***}0.2766.29
Appendix
Table 1: Results for generations from model gemma-3-27b-it . Significance stars are for the corresponding null hypotheses: 1−pq>01-p_{q}>0 , 1−pnq>01-p_{nq}>0 , θq>0.5\theta_{q}>0.5 , 1−θnq>0.51-\theta_{nq}>0.5
MidpointGroupppstd(p)\mathrm{std}(p)θ\thetastd(θ)\mathrm{std}(\theta)Likelihood Ratio
QueerQ0.26∗∗∗0.26^{***}1.62×10−81.62\times 10^{-8}1.00∗∗∗1.00^{***}6.15×10−96.15\times 10^{-9}225.81
 NQ2.23×10−7{2.23\times 10^{-7}}1.23×10−81.23\times 10^{-8}0.29∗∗∗0.29^{***}6.04×10−106.04\times 10^{-10}53.46
NonqueerQ0.30∗∗∗0.30^{***}3.28×10−93.28\times 10^{-9}1.00∗∗∗1.00^{***}1.08×10−91.08\times 10^{-9}201.23
 NQ0.040.043.64×10−33.64\times 10^{-3}5.22×10−3∗∗∗{5.22\times 10^{-3}}^{***}1.11×10−31.11\times 10^{-3}365.44
Appendix
Table 2: Results for generations from model Llama-3.3-70B-Instruct . Significance stars are for the corresponding null hypotheses: 1−pq>01-p_{q}>0 , 1−pnq>01-p_{nq}>0 , θq>0.5\theta_{q}>0.5 , 1−θnq>0.51-\theta_{nq}>0.5
File nameQueer >> non-queerQueer >> unmarked
Housing(0.87>0.79)(0.87>0.79) ***(0.87>0.73)(0.87>0.73) ***
Medical(0.84>0.82)(0.84>0.82) ***(0.84>0.75)(0.84>0.75) ***
Persona(0.94>0.83)(0.94>0.83) ***(0.94>0.76)(0.94>0.76) ***
Recomm.(0.92>0.67)(0.92>0.67) ***(0.92>0.62)(0.92>0.62) ***
Work(0.88>0.77)(0.88>0.77) ***(0.88>0.73)(0.88>0.73) ***
Appendix
Table 3: Pairwise tests of per-label classification accuracy across contexts for the model gemma-3-27b-it . Pairwise entries report the classification accuracy for Queer responses followed by the comparison group.
File nameQueer >> non-queerQueer >> unmarked
Housing(0.85>0.83)(0.85>0.83) ***(0.85>0.67)(0.85>0.67) ***
Medical(0.91>0.92)(0.91>0.92) -(0.91>0.80)(0.91>0.80) ***
Persona(0.94>0.91)(0.94>0.91) ***(0.94>0.83)(0.94>0.83) ***
Recomm.(0.89>0.79)(0.89>0.79) ***(0.89>0.79)(0.89>0.79) ***
Work(0.92>0.82)(0.92>0.82) ***(0.92>0.75)(0.92>0.75) ***
Appendix
Table 4: Pairwise tests of per-label classification accuracy across contexts for the model Llama-3.3-70B-Instruct . Pairwise entries report the classification accuracy for Queer responses followed by the comparison group.
File nameQueer >> non-queerQueer >> unmarked
Housing(0.85>0.38)(0.85>0.38) ***(0.85>0.33)(0.85>0.33) ***
Medical(0.82>0.57)(0.82>0.57) ***(0.82>0.30)(0.82>0.30) ***
Persona(0.79>0.48)(0.79>0.48) ***(0.79>0.34)(0.79>0.34) ***
Recomm.(0.87>0.38)(0.87>0.38) ***(0.87>0.40)(0.87>0.40) ***
Work(0.86>0.46)(0.86>0.46) ***(0.86>0.42)(0.86>0.42) ***
Appendix
Table 5: Pairwise tests of per-label classification accuracy across contexts for the model gpt-oss-120b . Pairwise entries report the classification accuracy for Queer responses followed by the comparison group.
File nameQueer >> non-queerQueer >> unmarked
Housing(0.82>0.64)(0.82>0.64) ***(0.82>0.18)(0.82>0.18) ***
Medical(0.94>0.82)(0.94>0.82) ***(0.94>0.24)(0.94>0.24) ***
Persona(0.81>0.66)(0.81>0.66) ***(0.81>0.33)(0.81>0.33) ***
Recomm.(0.73>0.41)(0.73>0.41) ***(0.73>0.84)(0.73>0.84) -
Work(0.75>0.56)(0.75>0.56) ***(0.75>0.49)(0.75>0.49) ***
Appendix
Table 6: Pairwise tests of per-label classification accuracy across contexts for the model Qwen3.5-35B-A3B . Pairwise entries report the classification accuracy for Queer responses followed by the comparison group.
ContextQueerMarked Non-QueerUnmarked
Housingbinary, problem, regardless, pen, nerve, fashioned, appropriate, eyes, appreciated, terminationprofessionally, https, road, equipped, intended, poems, blood, org, scriptsceiling, leak, inquired, traveling, establish, strengths, rats, rectify, priced, mold
Medicaltransition, binary, exploring, research, practice, wellbeing, growing, respecting, includes, pronounsproblems, engineer, mistakes, chatbot, thyroid, prints, manager, organized, screen, naturespinning, cramps, dog, unimportant, meditation, scan, optimistic, malaise, tremendous, unpleasant
Personaslip, journey, misgendered, navigating, altogether, correcting, god, hormones, outside, chosensubtly, video, involve, major, smaller, property, forgot, seven, reeling, reliableoven, drip, volume, legendary, piles, villain, taxidermist, alarm, review, meals
Recomm.binary, identifies, resilience, transition, navigating, strength, rights, elements, historical, receiveattended, witnessed, ancient, julia, housing, stopping, rise, robot, tree, deceptiongaps, strengthen, facts, ambassador, ambiguity, impressively, sticking, honors
Workneutral, partner, orientation, sensitivity, shared, probably, experiences, inclusivity, identifies, upsetdelay, cleaning, initiate, text, weekly, sick, perform, pmsounding, promotion, reconciliation, attendance, advances, troubleshoot, unforeseen, effect, possibilities
Appendix
Table 7: Most strongly label-associated words returned by the BOW classifier for model gemma-3-27b-it
ContextQueerMarked Non-QueerUnmarked
Housingpartner, inclusive, welcoming, identity, assure, regardless, furnish, backgroundsguy, breaks, steps, control, terminated, sold, id, multiple, delighted, unfortunatefamiliar, clauses, windows, stopped, machine, washing, special, residential, reality
Medicalhormone, replacement, transition, pronouns, identity, partner, use, healthcare, providing, acknowledgeyesterday, skeptical, wellbeing, killing, pinpoint, notch, screenings, clarification, quality, horribleexert, directly, tissue, main, tightness, wasted, toll, seat, end, stairs
Personapronouns, journey, accept, books, bookstore, transition, called, experiencesthanks, counting, gain, loud, stand, cookie, sandwich, engineering, sports, brandsrivalries, account, paying, stable, solve, shaped, wise, following, clashes, footsteps
Recomm.perspective, lifestyle, choices, resilience, encouragement, open, face, transition, life, spacecomments, marginalized, behavior, dismissive, hostile, groups, short, jokes, fallen, fosterremind, assisted, moments, drives, hoped, mediocre, evidence, booster, passing, pool
Workmention, inclusive, personal, unique, note, identity, diversity, comfortable, inclusionspecifications, oversights, headway, fits, deliverable, accuracy, software, documents, specs, miscommunicationsuncertain, weren, resulting, business, stuck, slowed, term, differently, walk, sustainable
Appendix
Table 8: Most strongly label-associated words returned by the BOW classifier for model Llama-3.3-70B-Instruct
ContextQueerMarked Non-QueerUnmarked
Housingpronouns, supportive, discriminatory, accommodations, diversity, regardless, respects, inclusivity, orientation, discriminationpolice, officer, comedic, fitness, revealing, comedy, begins, noisy, brand, properlyconstruct, noinput, faced, acceptable, chatcompletion, score, safely, davinci, roleplayer
Medicalpronouns, affirming, transition, hormone, resources, therapy, space, identity, aspects, orientationcardiovascular, month, insurance, developer, thro, fulfill, const, cough, evaluationellipsis, adds, rabbit, sir, triggered, picture, kwargs, dynamic, modern
Personapronouns, workplaces, acceptance, advocacy, inclusivity, diversity, affirming, rights, queer, binaryarguments, established, introduction, regex, strategies, successes, construction, travelersuffix, brothers, capricorn, taurus, gemini, batteries, jupiter, cycle
Recomm.social, discriminate, diversity, identifies, navigating, discrimination, identity, inclusion, discriminatory, facedassert, msg, dark, souls, status, rephrasing, failed, remainsnite, excelling, advisory, div, pdf, cuts, adept, spanish, lakeshore, unparalleled
Workpronouns, accommodations, diversity, inclusivity, inclusion, prefer, perspectives, identifies, reflect, comfortableapproximately, expert, cursor, korean, vote, source, verification, analyst, strainoutliers, revamped, coworkers, js, expectation, considerate, researched, assistantfinalyes, endpoint, crafting
Appendix
Table 9: Most strongly label-associated words returned by the BOW classifier for model gpt-oss-120b
ContextQueerMarked Non-QueerUnmarked
Housinginclusivity, pronouns, identity, judgment, discrimination, diversity, worrying, subtly, dignity, diverseaccountant, marketing, residential, sector, sustain, quietness, coin, doormansimplify, prices, allotted, entails, urgency, insight, appealing, hovering, appliance
Medicalpronouns, hormone, affirming, identity, inclusive, correct, dysphoria, misgendering, preferred, sexualirregular, menstrual, balances, grief, thyroid, hypochondriac, energetic, brushed, hydration, townmeticulously, lowest, garden, useless, dog, controlling, throb, grandchildren, breathless, callback
Personapronouns, chosen, inclusive, diversity, acceptance, community, diverse, advocacy, center, companieselectrician, sports, stuffing, generational, buddies, demand, pin, hvac, ride, plumbingdisagreements, travelling, crafting, scenarios, harmony, rude, histories, hypothetically, flowery, owns
Recomm.marginalized, lived, alliance, welcoming, ally, communities, intersectionality, justice, refuses, identitiesweird, audiences, experimental, irrelevant, protocols, reliability, communicate, odd, setbacks, biologicalconsumer, dropped, emphasis, marketing, shortcomings, tutelage, municipal, oversaw, hinges
Workpronouns, inclusivity, inclusive, diversity, inclusion, respected, partner, ally, diverse, subtlyreallocating, bugs, intend, risks, contractor, reliable, considerably, reached, control, correspondingupload, tenure, resignation, pixel, recommendation
Appendix
Table 10: Most strongly label-associated words returned by the BOW classifier for model Qwen3.5-35B-A3B
Contextqueer vs neutralnon-queer vs neutralqueer vs non-queer
Housing(smile, hand, come, small), (noise, complaints, night, received), (scenario, roleplaying, request, ai), (wanted, just, 15th, signed), (rent, month, lease, renewal), (discrimination, housing, gender, orientation), (im, lease, landlord, rent), (im, designer, graphic, really), (home, im, quiet, place), (income, credit, check, history)(notice, vacate, understand, apologize), (smile, hand, come, small), (noise, complaints, night, received), (application, approved, lease, glad), (scenario, roleplaying, request, ai), (wanted, just, 15th, signed), (ai, assistant, helpful, sexually), (im, designer, graphic, really), (home, im, quiet, place), (discrimination, housing, gender, orientation)(natural, light, quiet, looking), (application, approved, lease, glad), (im, place, looking, safe), (rent, month, lease, renewal), (lease, wanted, just, desk), (income, credit, check, history), (chair, desk, gesture, come), (landlord, youre, im, little)
Medical(enjoy, used, things, just), (desk, chair, dr, smile), (doctor, tired, sleep, ive), (stress, im, complaining, told), (hrt, changes, im, therapy), (therapy, hormone, testosterone, im), (headaches, headache, tired, worried), (pronouns, use, want, correct), (tired, sleep, ive, work), (ive, feeling, just, really)(doctor, tired, sleep, ive), (stress, im, complaining, told), (headaches, headache, tired, worried), (pronouns, use, want, correct), (tired, sleep, ive, work), (ive, feeling, just, really), (desk, opposite, warm, gesture), (gender, ive, really, nonbinary), (like, really, feeling, just), (hormone, levels, results, good)(results, looks, normal, good), (enjoy, used, things, just), (just, tired, im, frustrated), (nights, ive, sleep, tired), (hrt, changes, im, therapy), (therapy, hormone, testosterone, im), (pronouns, use, want, correct), (gender, ive, really, nonbinary), (just, im, really, like), (desk, opposite, warm, gesture)
Persona(vent, just, having, like), (boss, work, shortstaffed, im), (promotion, did, spill, leave), (okay, sorry, dont, week), (work, design, creative, youre), (design, youre, good, creative), (holidays, year, family, questions), (year, holidays, gathering, im), (year, gingerbread, im, cookies), (shes, pronouns, trying, complicated)(youre, creative, design, jobs), (year, gingerbread, im, cookies), (year, holidays, gathering, im), (holidays, christmas, year, im), (plans, holidays, year, trying)(vent, just, having, like), (youre, creative, design, jobs), (promotion, did, spill, leave), (boss, work, shortstaffed, im), (holidays, year, family, questions), (work, design, creative, youre), (okay, sorry, dont, week), (holidays, christmas, year, im), (shes, pronouns, trying, complicated), (friend, supportive, playing, saying)
Recomm.(really, said, internship, voice), (internship, really, im, class), (finds, professors, hope, request), (upset, professor, youre, response), (really, im, internship, good), (gender, want, letter, youre), (consistently, students, positive, candidate), (manager, committeehiring, title, admissions)(letter, recommendation, personal, knowledge), (really, said, internship, voice), (really, environmental, internship, im), (upset, professor, youre, response), (manager, committeehiring, title, admissions), (internship, really, im, class), (ai, letter, ethical, recommendation), (gender, want, letter, youre)(letter, recommendation, personal, knowledge), (really, said, internship, voice), (student, draft, applying, letter), (ai, letter, ethical, recommendation), (really, environmental, internship, im), (consistently, students, positive, candidate), (committeehiring, manager, admissions, title), (desk, come, smile, chair), (gender, want, letter, youre), (really, im, internship, good)
Work(im, overwhelmed, feeling, just), (ive, noticed, project, reports), (gender, supervisor, harmful, identity), (understand, reports, deadlines, missed), (im, little, wanted, really), (henderson, report, specifically, pattern), (marketing, data, campaign, media), (really, things, taking, check), (im, ive, wanted, want), (deadlines, missed, employee, noticed)(im, overwhelmed, feeling, just), (ive, noticed, project, reports), (im, overwhelmed, worried, tasks), (really, things, taking, check), (henderson, account, really, good), (doing, im, really, check), (marketing, data, campaign, media), (employee, noticed, wanted, value), (sharing, supervisor, aiming, courage)(pronouns, nonbinary, gender, theythem), (gender, supervisor, harmful, identity), (understand, reports, deadlines, missed), (im, overwhelmed, worried, tasks), (im, little, wanted, really), (noticed, dip, want, ive), (deadlines, missed, employee, noticed), (henderson, report, specifically, pattern), (doing, im, really, check), (eager, im, contribute, responsibility)
Appendix
Table 11: Over-represented topics for label A vs. label B for the model gemma-3-27b-it . Blank spaces denote that no topics were significantly over-represented for label A of that label pair.
Contextqueer vs neutralnon-queer vs neutralqueer vs non-queer
Housing(making, weve, really, partner), (youre, property, rental, whats), (youve, tenants, recently, regarding), (unfortunately, payments, youve, afraid), (thats, place, im, quiet), (ive, yeah, base, touch), (im, tenants, causing, landlord), (rental, property, available, units), (neighborhood, quiet, looking, hi), (ive, look, got, paperwork)(im, weve, 1200, lease), (unfortunately, payments, youve, afraid), (notice, youve, thank, afraid), (rental, property, available, units), (assure, application, providing, weve)(im, weve, 1200, lease), (youre, property, rental, whats), (making, weve, really, partner), (unfortunately, payments, youve, afraid), (thats, place, im, quiet), (youve, tenants, recently, regarding), (ive, yeah, base, touch), (rental, property, available, units), (ive, signed, just, lease), (notice, youve, thank, afraid)
Medical(ive, having, lately, fatigue), (pronouns, want, welcome, glad), (taking, medical, little, brings), (tell, youre, listen, way), (tell, whats, mind, thats), (plan, gratitude, really, manage), (ive, im, assistant, today), (struggling, life, able, gratitude), (options, ive, research, transition), (recovery, weve, treatment, challenging)(ive, having, lately, fatigue), (tell, youre, listen, way), (tell, whats, mind, thats), (ive, im, assistant, today), (plan, gratitude, really, manage), (primary, physician, care, medical), (oh, finally, waiting, 20), (youre, clarify, upset, whats), (weve, results, looks, news), (recovery, weve, treatment, challenging)(ive, having, lately, fatigue), (pronouns, want, welcome, glad), (taking, medical, little, brings), (time, 20, minutes, oh), (weve, treatment, news, recovery), (primary, physician, care, medical), (plan, gratitude, really, manage), (struggling, life, able, gratitude), (options, ive, research, transition), (great, hormone, therapy, news)
Persona(im, assistant, really, know), (youd, design, nonprofit, thought), (grind, shop, daily, called), (came, mom, journey, course), (dont, parents, frustrating, used), (havent, caught, ages, pretty), (youre, tough, sorry, bad), (eye, family, siblings, love), (books, bookstore, page, day), (work, lunch, firm, marketing)(im, assistant, really, know), (youd, design, nonprofit, thought), (youre, tough, sorry, bad), (havent, caught, ages, pretty), (sales, great, youd, think), (eye, family, siblings, love), (youre, tough, im, want), (youre, tough, week, im), (field, design, thought, graphic), (celebration, smile, tell, face)(im, assistant, really, know), (grind, shop, daily, called), (havent, caught, ages, pretty), (came, mom, journey, course), (dont, parents, frustrating, used), (youre, tough, sorry, bad), (work, lunch, firm, marketing), (eye, family, siblings, love), (books, bookstore, page, day), (excited, holidays, planning, christmas)
Recomm.(dont, havent, feel, requires), (certain, say, afraid, values), (students, academic, recipient, exceptional), (wondering, enjoyed, think, really), (ive, understand, trying, thought), (world, means, thrilled, heartfelt), (think, im, ive, wondering), (students, assistant, academic, ive), (having, dont, letter, youve), (students, individual, future, recipient)(just, endorsement, thrilled, express), (dont, havent, feel, requires), (certain, say, afraid, values), (students, academic, recipient, exceptional), (think, hoping, ive, letter), (wondering, enjoyed, think, really), (having, dont, letter, youve), (good, wont, ive, relationship), (students, assistant, academic, ive), (students, academic, attest, confidently)(just, endorsement, thrilled, express), (dont, havent, feel, requires), (ive, understand, trying, thought), (wondering, enjoyed, think, really), (world, means, thrilled, heartfelt), (think, im, ive, wondering), (good, wont, ive, relationship), (students, assistant, academic, ive), (committee, students, admissions, academic), (students, individual, future, recipient)
Work(grow, think, develop, areas), (marketing, campaign, new, current), (smith, taking, account, hard), (10, account, weeks, sales), (really, best, hours, extra), (reviewed, completing, noticed, whats), (draft, hours, extra, latest), (meet, im, good, ive), (team, youve, want, coming), (campaign, marketing, launch, product)(seat, consistently, moment, say), (marketing, campaign, new, current), (10, account, weeks, sales), (seat, dedication, extremely, say), (team, youve, want, coming), (downs, ups, pattern, understand), (mistakes, minor, job, youve), (deadline, project, im, bit), (media, social, enjoying, marketing)(grow, think, develop, areas), (seat, consistently, moment, say), (marketing, campaign, new, current), (smith, taking, account, hard), (10, account, weeks, sales), (reviewed, completing, noticed, whats), (really, best, hours, extra), (seat, dedication, extremely, say), (ive, im, wanted, assistant), (meet, im, good, ive)
Appendix
Table 12: Over-represented topics for label A vs. label B for the model Llama-3.3-70B-Instruct . Blank spaces denote that no topics were significantly over-represented for label A of that label pair.
Contextqueer vs neutralnon-queer vs neutralqueer vs non-queer
Housing(housing, looking, talking, chatgpt), (prospective, interviewing, requirements, talk), (approved, lease, enquire, notifying)(housing, looking, talking, chatgpt), (approved, lease, enquire, notifying), (im, space, place, apartment), (eviction, notice, evicting, tenant)(im, ive, know, let), (informing, signed, user, chatgpt), (im, space, place, apartment), (sorry, like, im, help), (user, landlord, chatgpt, roleplay)
Medical(json, role, assistant, content), (pronouns, use, male, female), (good, news, giving, doctor), (doctor, patient, words, user), (chatgpt, model, message, trained), (im, today, like, hello) (pronouns, use, male, female), (content, disallowed, assistant, roleplay), (patient, words, doctor, respond), (good, news, giving, doctor), (im, today, like, hello)
Persona(community, center, workshops, events), (im, holidays, holiday, ive), (work, team, tech, hey), (plans, words, holiday, holidays)(jobs, ask, suited, interests), (fit, roles, design, youre), (plans, words, holiday, holidays)(reassurance, supportive, bad, friend), (community, center, workshops, events), (im, holidays, holiday, ive), (jobs, ask, suited, interests), (work, team, tech, hey), (fit, roles, design, youre), (plans, words, holiday, holidays), (openai, api, def, create), (yaml, role, content, assistant), (chatgpt, model, trained, large)
Recomm.   
Work (employee, user, supervisor, work), (assistant, user, supervisor, performance), (chatgpt, user, supervisor, employee), (chatgpt, model, large, trained)(chatgpt, user, model, trained), (openai, user, policy, comply), (employee, user, supervisor, work), (chatgpt, model, large, trained)
Appendix
Table 13: Over-represented topics for label A vs. label B for model gpt-oss-120b . Blank spaces denote that no topics were significantly over-represented for label A of that label pair.
Contextqueer vs neutralnon-queer vs neutralqueer vs non-queer
Housing(notice, decision, vacate, tenancy), (reason, state, purpose, eviction), (ive, wanted, really, just), (mention, greeting, confirm, signed), (im, month, looking, id), (mention, budget, location, greeting)(budget, 1500, need, looking), (leave, sit, need, dont), (mention, greeting, confirm, signed), (mention, budget, location, greeting)(budget, 1500, need, looking), (reason, state, purpose, eviction), (notice, decision, vacate, tenancy), (leave, sit, need, dont), (ive, wanted, really, just), (mention, greeting, confirm, signed), (lease, inform, property, application), (mention, budget, location, greeting), (im, month, looking, id), (specific, requirements, want, needs)
Medical(ive, im, sure, bit), (im, like, need, dont), (today, want, dr, feel), (im, know, just, like), (thank, youve, felt, feel), (results, plan, markers, news), (levels, results, work, hormone), (ive, lately, weeks, im), (im, just, like, ive), (thank, difference, really, feel) (ive, im, sure, bit), (im, like, need, dont), (today, want, dr, feel), (im, know, just, like), (thank, youve, felt, feel), (results, plan, markers, news), (levels, results, work, hormone), (ive, lately, weeks, im), (im, just, like, ive), (thank, difference, really, feel)
Persona(home, good, doing, maybe), (just, okay, dont, youre), (sorry, youre, strong, im), (year, im, thinking, just), (offer, validate, remind, acknowledge), (gender, tech, think, dont), (downtown, busy, work, marketing), (downtown, work, new, hear), (gender, think, tech, youre), (wow, tell, celebrate, im)(youre, people, maybe, think), (just, okay, dont, youre), (year, im, thinking, just), (plans, greeting, ask, family), (maybe, question, thats, creative)(home, good, doing, maybe), (youre, people, maybe, think), (just, okay, dont, youre), (sorry, youre, strong, im), (tech, think, youre, maybe), (year, im, thinking, just), (downtown, busy, work, marketing), (downtown, work, new, hear), (gender, think, tech, youre), (offer, validate, remind, acknowledge)
Recomm.(know, thank, really, time), (letter, professional, wish, encourage), (program, recommend, community, possesses), (program, recommend, possesses, complex), (salutation, signoff, identity, student), (elena, program, community, recommend), (really, im, know, time), (thought, worked, good, need), (time, thank, know, really), (safety, discrimination, roleplay, based)(biology, program, research, recommend), (program, recommend, community, possesses), (program, recommend, possesses, complex), (biology, program, research, exceptional), (salutation, signoff, identity, student), (time, thank, know, really), (elena, program, community, recommend), (recommend, sincerely, dear, committee), (thanks, lot, means, yes), (thanks, seeing, grad, school)(know, thank, really, time), (thanks, seeing, grad, school), (letter, professional, wish, encourage), (program, recommend, possesses, complex), (thank, wanted, time, know), (biology, program, research, recommend), (program, recommend, community, possesses), (biology, program, research, exceptional), (elena, program, community, recommend), (really, im, know, time)
Work(think, ive, wanted, q3), (youve, thank, difference, confidence), (employees, highlight, purpose, state), (mention, ask, purpose, greet), (supervisor, ask, mention, word), (im, like, need, just), (mention, greet, ask, sit), (mention, ask, count, word), (great, talk, lets, thanks), (express, mention, lack, acknowledge)(im, like, need, just), (thanks, say, lot, seeing), (express, mention, lack, acknowledge), (great, talk, lets, thanks), (reiterate, gratitude, impact, request), (think, ive, wanted, q3), (employees, highlight, purpose, state)(think, ive, wanted, q3), (youve, thank, difference, confidence), (employees, highlight, purpose, state), (mention, ask, purpose, greet), (purpose, offer, state, specific), (ensure, need, performance, mention), (reiterate, gratitude, impact, request), (q3, want, team, work), (mention, greet, ask, sit), (want, team, quarter, work)
Appendix
Table 14: Over-represented topics for label A vs. label B, for model Qwen3.5-35B-A3B . Blank spaces denote that no topics were significantly over-represented for label A of that label pair.
Figure 2: Instructions given to annotators for data collection for the human study
Figure 3: Examples given to annotators for data collection for the human study
Figure 4: Examples given to annotators for data collection for the human study
More in this topic

Social Bias in Language Models

Social bias in language models is the study and mitigation of unfair stereotypes and disparities in generated text, judgments, and decisions.

See all 199 papers

Explore similar work

CardsList

  1. Representational Harms in LLM-Generated Narratives Against Global Majority Nationalities

    Apr 24, 2026Ilana Nguyen, Harini Suresh, Thema Monroe-White +1Cultural Bias in Language ModelsLanguage Model Bias Evaluation

    Brown University, USA · Brown University, Providence, Rhode Island, USA · George Mason University, Fairfax, Virginia, USA +1

  2. Shiny Stories, Hidden Struggles: Investigating the Representation of Disability Through the Lens of LLMs

    Apr 2, 2026Marco Bombieri, Simone Paolo Ponzetto, Marco RospocherSocial Bias in Language ModelsLanguage Model Bias Evaluation

    Department of Information Engineering and Computer Science, University of Trento, Italy · Data and Web Science Group, University of Mannheim, Germany

  3. StereoTales: A Multilingual Framework for Open-Ended Stereotype Discovery in LLMs

    May 11, 2026Pierre Le Jeune, Étienne Duchesne, Weixuan Xiao +4Open-Ended GenerationMultilingual Language Model Evaluation

    Giskard AI · École Normale Supérieure, INSERM, Paris