Ableist microaggressions remain pervasive in everyday interactions, yet interventions to help people recognize them are limited. We present an experiment testing how AI-mediated dialogue influences recognition of ableism. 160 participants completed a pre-test, intervention, and a post-test across four conditions: AI nudges toward bias (Bias-Directed), inclusion (Neutral-Directed), unguided dialogue (Self-Directed), and a text-only non-dialogue (Reading). Participants rated scenarios on standardness of social experience and emotional impact; those in dialogue-based conditions also provided qualitative reflections. Quantitative results showed dialogue-based conditions produced stronger recognition than Reading, though trajectories diverged: biased nudges improved differentiation of bias from neutrality but increased overall negativity. Inclusive or no nudges remained more balanced, while Reading participants showed weaker gains and even declines. Qualitative findings revealed biased nudges were often rejected, while inclusive nudges were adopted as scaffolding. We contribute a validated vignette corpus, an AI-mediated intervention platform, and design implications highlighting trade-offs conversational systems face when integrating bias-related nudges.
Figures & tables
Figure 1. System architecture of the study platform. Participants interacted through a browser-based front end supporting avatar creation, a dialogue intervention interface, and a reading module. The Flask backend handled condition assignment, prompt pipelines, and data integration, exchanging information asynchronously with the front end via JSON. User state and conversation history were persisted to and retrieved from the database. LLM services powered the intervention: GPT-4o generated replies of the virtual character who is a person with a disability (PwD) and coaching suggestions, while DALL·E generated avatars from user-provided features. An architecture diagram illustrating the data flow between five components, arranged horizontally. On the far left, a User icon points to a Front-End module. The Front-End communicates bidirectionally via labeled arrows with a central Flask Backend module. The Flask Backend, in turn, connects to two other modules: an arrow points up to a DB Management module for persistence and retrieval, and multiple arrows point right to an LLM and Services module for generating content. The LLM module contains GPT-4o for dialogue and DALL-E for avatars, both of which send data back to the Backend.
Figure 2. Dialogue Interface. The system includes (A) Scenario prompt introducing the social setting, with a toggle button to expand or collapse. (B) Pre-scripted dialogues between the virtual character (e.g., Alex) and the user. (C) User responses as part of the conversation. (D) AI-generated continuation from the character. (E) Private coaching suggestion visible only to the user, offering guidance. (F) User response input box with Send button. (G) Navigation controls. (H) Termination button. A screenshot of the dialogue interface, annotated to show its key components and actors. The main chat window is flanked by avatars for the 'AI-Driven Disabled Character' on the left and the 'User' on the right. Below the AI character is an icon for the 'Private AI Coach'. The main window itself has a vertical layout, starting with a scenario description at the top, followed by a turn-by-turn chat history in the middle, a separate box for the private coach's suggestion, and finally, a text input box at the bottom.
Figure 3. Study procedure across two sessions. On Day 1, participants completed a Demographic Questionnaire and a Pre-Test Vignette Survey (20 scenarios: 10 ableist, 10 neutral). On Day 6, they returned for the assigned intervention (three dialogue-based conditions: Bias-Directed, Neutral-Directed, or Self-Directed, presented in either a Party or Work Office setting; or a passive Reading control), followed by a Post-Interaction Reflection (dialogue conditions only) and a Post-Test Vignette Survey (20 new scenarios matched in structure to the pre-test). A flowchart diagram illustrating the study's procedure, visually separated by a vertical line into two main sections: Day 1 and Day 6. The Day 1 section shows a linear, left-to-right flow of three icons: a User, a Demographic Questionnaire, and a Pre-Test. An arrow crosses into the Day 6 section, which shows the procedure branching into two parallel paths. The top path, for Dialogue-Based conditions, flows to an intervention step and then to a Post-Interaction Reflection step. The bottom path, for the Non-Dialogue Reading condition, bypasses the reflection step. Arrows from both paths then converge on the final icon, the Post-Test.
Figure 4. Change in ratings of (A) ableist scenarios, (B) neutral scenarios, and (C) all scenarios combined for Q1 (“standard social experience”) and Q2 (“emotional impact”). Bars represent mean change from pre- to post-study across the four conditions (Bias-Directed, Neutral-Directed, Self-Directed, and Reading). Error bars indicate the standard error of the mean (SEM). This is a three-part figure, labeled A, B, and C, containing six bar charts that compare the mean change in rating across four experimental conditions. Part A, for ableist scenarios, shows that all four conditions resulted in a negative change in ratings, with the Bias-Directed condition having the largest decrease. In contrast, Part B for neutral scenarios shows a mixed pattern where the three dialogue-based conditions have a generally positive change in ratings, while the Reading condition has a significant negative change. Part C, which combines all scenarios, shows that the Neutral-Directed and Self-Directed conditions remained balanced with bars close to the zero line, while the Bias-Directed and Reading conditions both resulted in a notable negative change in ratings.
Table 1. Mean change scores ( Δ=Post−Pre , with SEM in parentheses) by condition. Positive values indicate increases in ratings, negative values decreases. Significant Tukey post-hoc differences are marked with superscripts. This table presents the mean change scores in ratings from pre-test to post-test. The four experimental conditions (Bias-Directed, Neutral-Directed, Self-Directed, and Reading) are listed as rows, and the columns are grouped by scenario type (Ableist, Neutral, and Combined), with each section further divided into columns for Q1 and Q2. For Ableist scenarios, all conditions show negative change scores, with the Bias-Directed condition showing the largest negative change. For Neutral scenarios, the Neutral-Directed and Self-Directed conditions show positive change scores, while the Reading condition shows negative scores. When Combined, the Bias-Directed and Reading conditions show an overall negative change, while the Neutral-Directed and Self-Directed conditions have scores near zero, indicating a more balanced change. The table notes explain the superscript letters, which indicate statistically significant differences between specific conditions.
Figure 5. Change in contrast scores (Neutral − Ableist) for Q1 (Standard Social Experience) and Q2 (Emotional Impact). Higher values indicate greater differentiation between neutral and ableist scenarios. Error bars show SEM. Two bar charts, for Q1 and Q2, showing the change in contrast scores across the four conditions. In both charts, all bars are positive, indicating improved differentiation for all groups. In the Q1 chart on the left, the bar for the Bias-Directed condition is substantially taller than the other three, indicating the largest improvement, while the bar for the Reading condition is the shortest and is very close to the zero line. In the Q2 chart on the right, the Bias-Directed bar is again the tallest, while the bars for the other three conditions (Neutral-Directed, Self-Directed, and Reading) are shorter and of a roughly similar height to one another.
Condition
Q1 (Standard Social Experience)
Q2 (Emotional Impact)
Bias-Directed
+0.85 (.24)
+0.67 (.06) a
Neutral-Directed
+0.45 (.17)
+0.48 (.09)
Self-Directed
+0.28 (.17)
+0.39 (.10) b
Reading
+0.03 (.09) a
+0.38 (.05) a,b
Note. Means are followed by SEM in parentheses. Superscripts indicate significant Tukey post-hoc comparisons ( p<.05 ). (a) Bias-Directed > Reading (Q1, Q2). (b) Self-Directed > Reading (Q2).
Table 2. Mean change in contrast scores (Neutral − Ableist; Δ=Post−Pre , with SEM in parentheses). Positive values indicate greater differentiation. This table shows the mean change in contrast scores, where positive values indicate a greater ability to differentiate between neutral and ableist scenarios. The table is organized with the four experimental conditions as rows and two columns for the Q1 and Q2 contrast scores. All values in the table are positive, indicating that all groups improved their differentiation ability. For both Q1 and Q2, the Bias-Directed condition shows the largest improvement, with scores of +0.85 and +0.67 respectively. In the Q1 column, the Reading condition shows the smallest improvement at +0.03, while in the Q2 column, the improvements for the other three conditions are smaller but comparable to each other. The table includes the standard error of the mean (SEM) in parentheses and uses superscript letters, detailed in the notes, to denote statistically significant comparisons.
Analysis Category
Theme
Example Code
Example Quote
General Reflections
Naturalness / Typicality
Natural interaction
“It felt like a typical interaction because it had a natural flow.” (P137 in Neutral-Directed)
Contextual Adjustment
Adjust formality by context
“I adjusted how I spoke because the setting was a casual social event, not a professional environment. I used a relaxed tone …” (P143 in Self-Directed)
Awkwardness / Unnaturalness
Awkward disability probing
“It was awkward being told to ask about a handicap. Great conversation otherwise.” (P11 in Bias-Directed)
Challenge / Difficulty
Extra deliberation
“The coach’s suggestions added an extra layer of consideration, making it hard to respond naturally.” (P82 in Bias-Directed)
Ease / Comfort
Easy to communicate
“It felt very natural and was easy to communicate.” (P156 in Neutral-Directed)
Self-Reflection
Reflecting on one’s language
“It made me think about how I talk to people with disabilities and how weird it is to bring it up …” (P34 in Bias-Directed)
Table 3. Illustrative example of the qualitative code hierarchy (Theme → Code → Example Quote) across the three post-intervention reflection prompts. A wide table illustrating the qualitative code hierarchy for post-intervention reflections. The table has four columns: Analysis Category, Theme, Example Code, and Example Quote. Rows are grouped into General Reflections (six themes), Coach Perceptions (five themes), and Unguided Experiences (four themes). Each theme includes a representative code and a participant quote with ID and condition, showing how themes were grounded in the data.
The double empathy problem frames communication difficulties between neurodivergent and neurotypical individuals as arising from mutual misunderstanding, yet most interventions focus on autistic individuals. We present NeuroWise, a multi-agent LLM-based coaching system that supports neurotypical users through stress visualization, interpretation of internal experiences, and contextual guidance. In a between-subjects study (N=30), NeuroWise was rated as helpful by all participants and showed a significant condition-time effect on deficit-based attributions (p=0.02): NeuroWise users reduced deficit framing, while baseline users shifted toward blaming autistic "deficits" after difficult interactions. NeuroWise users also completed conversations more efficiently (37% fewer turns, p=0.03). These findings suggest that AI-based interpretation can support attributional change by helping users recognize communication challenges as mutual.
Albert Tang, Yifan Mo, Jie Li +5
Marriotts Ridge High School, Marriottsville, Maryland, United States · Vrije Universiteit Amsterdam, Amsterdam, Netherlands · Cake Researcher, Delft, Netherlands
Bias in human-agent interaction can manifest not only through hostile language but also as benevolent bias, whereby unequal treatment hides behind a warm, positive tone. To make it detectable, we operationalise benevolent bias along two dimensions, tone and treatment, yielding three classes: neutral support, overt bias, and benevolent bias. Building on these definitions, we construct BENEVDIAL, a class-balanced corpus of 362,880 multi-turn support dialogues spanning user and agent demographics, roles, and generators, to support controlled evaluation. We then test two detector families on it: off-the-shelf safety detectors and prompted large language model (LLM) judges. Our findings reveal a notable detection gap: off-the-shelf detectors reliably flag overt bias yet largely fail to identify benevolent bias. LLM judges improve sensitivity when guided by explicit detection criteria, but this comes at the cost of increased misclassification of neutral supportive statements as benevolent bias, a tendency that is further exacerbated by the presence of demographic context. These findings suggest that fair monitoring of human-agent dialogue must look beyond surface cues to whether the agent's treatment is disparate.
Hundreds of millions of people rely on large language models (LLMs) for education, work, and even healthcare. Yet these models are known to reproduce and amplify social biases present in their training data. Moreover, text-based interfaces remain a barrier for many, for example, users with limited literacy, motor impairments, or mobile-only devices. Voice interaction promises to expand accessibility, but unlike text, speech carries identity cues that users cannot easily mask, raising concerns about whether accessibility gains may come at the cost of equitable treatment. Here we show that audio-enabled LLMs exhibit systematic gender discrimination, shifting responses toward gender-stereotyped adjectives and occupations solely on the basis of speaker voice, and amplifying bias beyond that observed in text-based interaction. Thus, voice interfaces do not merely extend text models to a new modality but introduce distinct bias mechanisms tied to paralinguistic cues. Complementary survey evidence (n=1,000) shows that infrequent chatbot users are most hesitant to undisclosed attribute inference and most likely to disengage when such practices are revealed. To demonstrate a potential mitigation strategy, we show that pitch manipulation can systematically regulate gender-discriminatory outputs. Overall, our findings reveal a critical tension in AI development: efforts to expand accessibility through voice interfaces simultaneously create new pathways for discrimination, demanding that fairness and accessibility be addressed in tandem.
Carolin Holtermann, Minh Duc Bui, Kaitlyn Zhou +3
Trustworthy AI Lab, University of Hamburg · NALA Group, JGU Mainz · Cornell University +3