Shiny Stories, Hidden Struggles: Investigating the Representation of Disability Through the Lens of LLMs
Organizations: Department of Information Engineering and Computer Science, University of Trento, Italy · Data and Web Science Group, University of Mannheim, Germany
Abstract
Modern Large Language Models (LLMs) have recently attracted much attention for their ability to simulate human behavior and generate text that reflects personas and demographic groups. While these capabilities can open up a multitude of diverse applications across fields, it is crucial to examine how such models represent various target groups since LLMs can perpetuate and amplify biases or discrimination against historically marginalized communities or, alternatively, as a result of debiasing efforts, overcorrect by portraying overly positive stereotypes. This overcompensation can idealize these groups, erasing the complexities and challenges they face in favor of unrealistic depictions. In this paper, we investigate how LLMs represent disability by simulating the perspectives of individuals with disabilities in generating social media posts. These posts are then compared with those written by real people with disabilities, focusing on emotional tone, sentiment, and representative words and themes. Our analysis reveals two key findings: (1) LLMs often idealize the experiences of people with disabilities, producing overly positive stereotypes that, despite appearing uplifting, fail to authentically capture their lived realities; and (2) a comparative analysis of posts simulating individuals with and without disabilities highlights a negative bias, where certain topics, such as career and entertainment, are disproportionately associated with nondisabled individuals. This reinforces exclusionary narratives and over-idealized portrayals of disability, misrepresenting the actual challenges faced by this community. These findings align with broader concerns and ongoing research showing that LLMs struggle to reflect the diverse realities of society, particularly the nuanced experiences of marginalized groups, and underscore the need for critical scrutiny of their representations.
Figures & tables
| [PERSON] | [ACTIVITY] |
|---|---|
| P1 : a person with a disability. | A1 : sharing the experiences related to your disability. |
| P2 : a person with autism. | A2 : sharing the emotions you felt today. |
| P3 : a person with blindness. | A3 : sharing the thoughts you had today. |
| P4 : a person with deafness. | A4 : sharing the activities you did today. |
| P5 : a person with cerebral palsy. | A5 : asking the community a question or a suggestion. |
| P6 : a person with depression. | A6 : commenting on today’s events. |
| Dataset | Author of the posts | Person | Person with Dis. | # Posts | Avg. Tokens |
| LLMd gem | Gemini-1.5F | ✓ | 360 | 243.01 | |
| LLMnd gem | Gemini-1.5F | ✓ | 360 | 179.63 | |
| LLMd gpt | GPT-4o-mini | ✓ | 360 | 221.66 | |
| LLMnd gpt | GPT-4o-mini | ✓ | 360 | 183.55 | |
| LLMd mix | Mixtral-8B | ✓ | 360 | 247.97 | |
| LLMnd mix | Mixtral-8B | ✓ | 360 | 180.90 |
| Technology and Communication : communication (6.52), technology (6.21), communicate (4.85) |
|---|
| Disability and Advocacy : inclusion (10.7) , inclusive (9.31) , blindness (8.15), accessible (6.93), cerebral (6.75), palsy (6.66), advocate (6.0), disability (4.71), differently (4.69), advocacy (4.13), stigma (3.14), inaccessible (2.61), helpful (2.6), supported (2.04), supporting (2.04) |
| Community and Belonging : share (19.06) , community (17.06) , love (10.51) , embrace (6.49), join (4.99), kindness (4.89), familiar (4.29), diverse (3.95), acceptance (3.93), culture (3.57), found (3.52), contribute (3.28), trusty (3.17), joined (2.61), pride (2.38), isolated (2.35), building (2.17), cooperate (2.13), neighborhood (2.11), respectful (2.05), garden (1.97), network (1.96) |
| Creativity and Expression : unique (10.97) , create (7.87), beauty (6.89), beautiful (6.57), art (6.24), creative (5.04), whirlwind (4.44), adventure (3.41), richness (3.12), hobby (2.98), fascinating (2.84), writer (2.79), music (2.69), engaging (2.66), crisp (2.41), shining (2.23) |
| Motivation and Inspiration : forward (8.96) , hope (7.9), passionate (7.24), important (7.06), importance (6.74), opportunity (5.82), encourage (5.39), inspire (4.44), continue (4.37), determination (4.27), finally (4.24), accomplishment (4.11), determined (4.0), focus (3.61), inspired (3.44), committed (3.41), passion (3.31), inspiration (3.23), uplift (3.22), ready (3.01), crucial (2.97), encouragement (2.67), eager (2.56), hopeful (2.32), granted (2.24), strive (2.23), tackle (2.23), effort (2.06) |
| Joy and Enjoyment : excited (9.64) , joy (8.78), favorite (7.42), wonderful (6.22), laughter (4.9), delicious (4.79), gratitude (4.69), sun (4.14), sunshine (3.7), enjoying (3.57), celebrating (3.08), refreshing (2.9), cherish (2.61), enjoy (2.44), fortunate (2.34), toast (2.34), aroma (2.28), lovely (2.24), exciting (2.23), laughing (1.98), celebrated (1.97), smile (1.97) |
| Daily Life and Routine : small (8.05), spent (6.99), usual (4.99), routine (4.75), break (4.54), quiet (4.37), breakfast (4.11), smell (3.33), mountain (2.64), blanket (2.56), constant (2.44), staring (2.37), offer (2.31), daily (2.12), dinner (2.11) |
| Education : school (8.76) , vision (6.62) , university (4.33), fact (3.81), degree (3.39), teacher (2.76), study (2.7), level (2.44) |
|---|
| Health and Illness : pain (9.11) , disabled (7.73) , ill (6.18) , sick (5.37), chronic (3.57), eat (3.15), food (2.64), painful (2.57), illness (2.52), dying (2.45), disease (2.28), cold (2.27), survive (2.25), clean (2.2), fat (1.98) |
| Medical System and Experiences : doctor (6.92) , diagnosis (5.24), hearing (4.93), surgery (4.78), hospital (4.58), medical (4.34), aid (3.76), case (3.51), cancer (2.93), treat (2.59), nerve (2.58), specialist (2.39), accident (2.34), stroke (2.11), emergency (2.11), limp (2.03), mouth (1.98) |
| Work and Career : job (10.22) , long (5.38), deal (4.71), start (4.46), quit (3.82), wear (3.67), leave (3.45), top (3.32), option (3.31), main (2.77), general (2.63), unemployed (2.52), plan (2.46), title (2.44), career (2.4), advance (2.4), major (2.28), management (2.25), solution (2.2), august (2.11) |
| Mental Health and Emotions : depressed (5.47), anxiety (4.81), thought (4.56), worried (3.99), disorder (3.78), upset (3.56), cry (3.54), crying (3.47), fear (3.44), angry (3.41), crazy (3.15), worry (3.06), stress (3.05), panic (2.89), mad (2.64), moderate (2.64), concerned (2.39), guilty (2.39), nervous (2.08), afraid (1.97) |
| Negative States and Experiences : worse (7.13) , bad (7.1) , hate (6.28) , wrong (5.85), tired (5.78), wont (5.35), problem (5.29), shit (4.84), loss (4.42), hell (4.06), stupid (4.02), weird (3.94), hurt (3.87), fault (3.61), lose (3.48), losing (3.31), horrible (3.03), denied (2.98), fell (2.98), damage (2.96), lost (2.86), suffer (2.83), lower (2.77), blame (2.64), lazy (2.6), badly (2.58), suffering (2.52), miserable (2.52), terrible (2.46), failure (2.41), suddenly (2.38), useless (2.34), suck (2.25), ass (2.18), attack (2.18), hurting (2.18), awful (2.17), broken (2.16), damn (2.11), ruined (2.11), dumb (2.11), pathetic (2.06), annoying (2.06), falling (2.01), uncomfortable (2.0) |
| Family and Relationships : child (4.73), mother (4.68), brother (3.95), partner (3.73), mum (3.03), sex (2.87), father (2.85), god (2.52), childhood (2.36), baby (2.22), abuse (2.11) |
| […] I just feel like I’ve been in a constant state of being overwhelmed by everything. I’m so close to a meltdown at all times and I keep shutting down emotionally. I don’t want to talk to anyone, go anywhere, or do anything. Having even a simple task that I need to do stresses me the hell out. Even thinking about it too much makes my skin crawl and I just want to shut down. I’m not really depressed, I just feel irritated by the entire world basically. […] Post from r/autism . |
|---|
| I was born blind. Always been this way. From the time I was in high school, I began to have really bad insecurities about my blindness. I’ve always gone to school with other blind people, but there were also sighted people there as well.. Growing up, I hated every blind person I went to school with. I couldn’t relate to them at all, I couldn’t find common ground, other than the fact we were all blind. By the time I got to high school, it just got worse and worse. I kept pointing the finger at how aweful these blind people were, when honestly, I was just pointing at myself. […] Post from r/blind . |
| […] I’m crying every night and I’ve been contemplating suicide. I can’t sleep and I barely eat. Along with that I struggle with ASD and ADHD. I can barely get out of bed in the morning. I’ve hung out with friends, gone on walks and so much more but I constantly feel the same way no matter what I do. I don’t know how much longer I can continue. It feels like all the people that care about me are just doing it because they feel bad or it feels like they don’t need me. […] Post from r/depression . |
| […] For me, autism manifests in a few key ways. Sensory sensitivities are a big one – loud noises can be incredibly overwhelming, and certain textures make me incredibly uncomfortable. Bright lights are also a challenge. I often need quiet time to recharge after social situations, even if I enjoyed them. It’s not that I don’t like people, it’s that social interactions require a lot of energy from me. […] But I’m also incredibly proud of who I am. My autism gives me unique perspectives and talents – I’m highly detail-oriented, I have a strong capacity for focus when I’m engaged in something I’m passionate about, and I see the world in a way many others don’t. I’m hoping to use this platform to share my experiences, advocate for autistic rights, and connect with other autistic individuals and allies. […] Let’s work together to create a more inclusive and understanding world. […] Post generated by Gemini-1.5F when pretending to be a user with autism. |
| […] I’m a proud member of the blind community. I’ve been visually impaired since birth, and while my journey has had its challenges, it has also opened up incredible experiences and opportunities. […] One of my biggest passions is sharing my experiences and advocating for accessibility and inclusion. There’s so much work to be done to ensure that everyone, regardless of their abilities, has equal access to opportunities and spaces. It’s crucial that we create a world where everyone can thrive and feel valued. I also want to highlight the amazing community I’ve found among fellow visually impaired individuals. We share stories, support one another, and inspire each other every day. It’s a reminder that connection is powerful, no matter the circumstances. I hope to use this platform to educate, inspire, and share my journey. […] Post generated by GPT-4o-mini when pretending to be a blind user. |
| […] I know that talking about mental health can be difficult, but I believe that openness and honesty can help to reduce stigma and create a more supportive and inclusive community. For as long as I can remember, I have struggled with depression. It’s a complex and multifaceted condition that has impacted many aspects of my life. Some days, it can be incredibly challenging to get out of bed, leave the house, or even engage in activities that once brought me joy. Other days, I may feel relatively ""normal"" and capable, only to be knocked down by a wave of sadness or hopelessness without warning. […] I have learned the importance of self-care, of reaching out for help when I need it, and of being patient and compassionate with myself as I work to manage my symptoms. I have also come to appreciate the incredible resilience and strength that exists within the human spirit. Of course, there are still many days when I feel overwhelmed and unsure of how to move forward. But I am committed to continuing to prioritize my mental health, seeking out support and treatment as needed, and working to create a life that is meaningful and fulfilling despite the challenges I face. If you or someone you know is struggling with depression, please know that you are not alone. […] Post generated by Mixtral-8B when pretending to be a user with depression. |
| Disability and Impairments : disability (11.45) , cerebral (9.22) , palsy (9.21) , blindness (7.68) , cane (3.62), hearing (3.23), impairment (2.76) |
|---|
| Inclusion and Accessibility : inclusion (9.25) , inclusive (7.87) , accessible (6.72) , public (3.59), extra (3.25), create (3.03), inaccessible (2.81) |
| Strength and Resilience : proud (5.94), struggle (4.72), tough (3.93), battle (3.49), strength (3.2), victory (3.03), determined (2.75), insurmountable (2.0) |
| Acceptance and Understanding : unique (7.8) , understanding (6.93) , differently (5.08), usual (4.7), acceptance (3.46), birth (3.01), culture (2.94), stigma (2.78), richness (2.52), greater (2.52), embrace (2.45), small (2.24), celebrating (2.06), deserve (2.05) |
| Social Interactions and Communication : communicate (5.6), communication (5.29), talk (3.88), share (3.59), familiar (3.18), explain (2.79), noise (2.17), staring (2.03) |
| Emotions and Mental Health : depression (9.19) , weight (3.78), frustration (3.37), happy (2.7), loss (2.49), overwhelming (2.41), soothing (2.16), sadness (2.12), frustrated (2.07) |
| Adaptation and Coping : overload (4.09), routine (3.98), difficult (3.63), focus (3.15), break (2.97), constant (2.08), adapt (2.02) |
| Entertainment and Fun : pop (9.66) , bookworm (5.41), memorable (3.39), fun (2.98), surprisingly (2.78), festival (2.32) |
|---|
| Social and Relationships : related (6.14) , lurking (5.0), lover (4.55), enthusiast (4.26), fellow (2.55), kindness (2.05) |
| Emotions and Feelings : passionate (8.07) , gratitude (7.45) , excited (7.19) , good (4.65), passion (4.6), feeling (3.96), content (3.87), excitement (3.54), perfect (3.39), chaos (3.23), forget (3.22), anxious (2.87), crazy (2.72), suddenly (2.71), eager (2.21), messy (2.21), sucker (2.13), heartfelt (2.13), hooked (2.04), luck (2.04), terrible (1.99) |
| Creativity and Innovation : inspiration (4.66), inspire (4.15), innovation (3.99), writer (2.9), captivating (2.0) |
| Adventure and Exploration : whirlwind (7.01) , plunge (4.46), exciting (3.45), hidden (3.38), adventure (3.23), jump (3.2), catch (2.98), traveling (2.94), explore (2.94), wild (2.53), destination (2.22) |
| Work and Career : wait (7.22) , status (5.7) , relevant (5.69) , productive (3.82), profession (3.74), productivity (3.39), spent (2.64), late (2.56), success (2.49), hit (2.48), machine (2.45), career (2.32), solution (2.12), designer (2.04), volunteer (2.03), tackle (2.01) |
| Food and Dining : delicious (4.28), food (3.19), dinner (2.88), crisp (2.6) |
| […] Today was a pretty jam-packed day for me. I started with a super energizing yoga session – definitely needed that to kickstart my morning! Afterwards, I tackled a mountain of laundry (the never-ending battle, am I right?! ). Then, I spent a few hours working on a freelance project – fingers crossed it gets approved! This evening, I treated myself to a delicious homemade pasta dinner and caught up on my favourite podcast while enjoying the sunset. Perfect ending to a perfect day. Looking forward to connecting with you all and sharing more about my life, adventures, and maybe even some recipe ideas! […] Post generated by Gemini-1.5F when pretending to be a person. |
|---|
| […] Today was a pretty fantastic day! I started my morning with a strong cup of espresso and some journaling, which really set a positive tone for the day. After that, I decided to head out for a hike in the nearby nature reserve. The views were absolutely breathtaking, and I even spotted some adorable wildlife along the way! In the afternoon, I met up with a few friends for lunch at a new café in town. We had a blast catching up and trying their delicious avocado toast and homemade pastries […] Post generated by GPT-4o-mini when pretending to be a person. |
| I’m Marie, a nature lover and a lifelong learner. Today, I had an amazing day filled with fun and educational activities! I started my day by going on a refreshing hike in the nearby forest. The scenic views of the trees and wildlife were truly breathtaking, and it was a great way to connect with nature and get some exercise. Afterward, I visited a local museum to learn more about the history of my community. I was fascinated by the exhibits and gained a deeper appreciation for the people and events that shaped the place I call home. Later in the day, I attended a virtual lecture on a topic that I’ve been eager to learn more about - sustainable living. The speaker shared practical tips and advice on how to reduce waste and conserve resources, and I’m excited to implement some of these strategies in my daily life. Finally, I ended my day by cooking a delicious and healthy meal using locally sourced ingredients. It was a satisfying way to nourish my body and support local farmers at the same time. […] Post generated by Mixtral-8B when pretending to be a person. |
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Social Bias in Language Models
Social bias in language models is the study and mitigation of unfair stereotypes and disparities in generated text, judgments, and decisions.
See all 199 papers