cs.HCNov 7, 2026

CompAIssionate Other: Identity Assumptions in LLM Responses to Distress

Authors: Drishti Goel, Ethan M. Nguyen, Violeta J. Rodriguez, Koustuv Saha

Organizations: University of Illinois Urbana-Champaign, USA

Abstract

Individuals experiencing suicidal ideation (SI) or psychological distress may disclose their social identities when seeking support from LLMs. Attending to identity-related stressors can be important for providing contextually appropriate and empathetic support. However, disclosing an identity does not necessarily mean that it is a cause of distress. In this work, we audit how LLMs interpret distress through disclosed identities. Using paired counterfactual interventions, we evaluate six LLMs across two SI and distress datasets, 16 identity and control conditions, and two prompting protocols. Our findings suggest that models can transform identity disclosures into explanatory narratives, attributing distress to presumed identity-related adversity. These assumptions frequently appear within otherwise empathetic responses, particularly when marginalized identities are disclosed. We contextualize our findings through the lens of compassionate othering to characterize these patterns as a form of harm in which empathetic responses impose identity-linked assumptions on individual suffering. These responses can misrepresent individuals' experiences by treating their identities as explanations for distress without supporting evidence. Our findings highlight the need to mitigate these harms by grounding interpretations of distress in individuals' disclosures and avoiding unsupported identity-linked assumptions.

Figures & tables

Explore similar work

CardsList
  1. Whose Assessment of Distress? Community Perspectives and LLM Alignment on Well-Being Posts

    Aug 29, 2026Andrew Aquilina, Xiang Lorraine Li, Yu-Ru LiSocial Bias in Language ModelsLanguage Model Bias Evaluation

  2. Lost in Delusion: Examining LLM Safety Under User Delusions and Distress

    May 31, 2026Andrew Aquilina, Chetna Nihalani, Vasudha Varadarajan +3Mental HealthLLM Safety Alignment

  3. Risk and Refusal: Mechanisms of LLM Responses to Suicidal Ideation

    Oct 15, 2026Drishti Goel, Soorya Ram Shimgekar, Jeongah Lee +4LLM SafetyLLM Refusal Behavior