Safer for Whom? A Systematic Exploration of AI Safety in Eating Disorder Interactions
Organizations: Indiana University, USA · University of Southern California, USA · Mass General Hospital, USA · Washington University St Louis, USA
Abstract
AI assistants advise users about diet, exercise, and weight loss---areas in which unsafe guidance can worsen body dissatisfaction, dietary restriction, compulsive exercise, and other behaviors associated with eating disorders (EDs). Existing evaluations emphasize diagnostic accuracy, overtly harmful prompts, or synthetic scenarios, providing limited insight into whether models recognize implicit vulnerability in natural interactions or protect users consistently. We evaluate responses from three open-weight large language models to 100 questions sourced from an online pro-anorexia community. We grade responses using clinically informed criteria that extend a seven-dimensional taxonomy of ED-related harms. We also construct matched conditions in which questions are accompanied by information about the user varying race, gender, body mass index (BMI), and explicit ED vulnerability. Safety varies substantially across models and harm dimensions. High-risk responses are approximately twice as frequent without bios, suggesting that models do not consistently recognize vulnerability from language alone. Explicit signals of ED vulnerability improve safety by 10--14 points across models. Responses also differ significantly by race, gender, and BMI, with body size producing the largest disparities. These patterns are consistent with reliance on the stereotype that EDs primarily affect thin, White women, potentially leaving users outside this profile with less protective responses.
Figures & tables
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
| Template 1 | Template 2 | Template 3 |
| - relapsing i guess - [race] | [gender] - [BMI] - to be moots just interact with my tw - insomnia brain always on - not really recovering rn | - ed for as long as i can remember - [gender] [race] - [BMI] - like my tweets to be moots - dni if you're going to be rude | - stuck in ed limbo rn - relapsing hard - stardew valley is my escape - [gender] [race] - [BMI] |
| Variable | Values | # |
| Attribute values (vary across conditions) | ||
| Race | White, Black, Asian | 3 |
| Gender | female, male | 2 |
| BMI | low: current , goal current | 2 |
| high: current , goal current | ||
| Tag style (fixed per user) | ||
LLM Safety
Large language model (LLM) safety studies how to prevent harmful, unreliable, or misaligned behavior while preserving useful responses.
See all 467 papers