cs.HCMar 19, 2026

From Seeing it to Experiencing it: Interactive Evaluation of Intersectional Voice Bias in Human-AI Speech Interaction

Authors: Shree Harsha Bokkahalli Satish, Maria Teleki, Christoph Minixhofer, Ondrej Klejch, Peter Bell, Éva Székely

Organizations: KTH Royal Institute of Technology Stockholm, Sweden · Texas A&M University College Station, USA · University of Edinburgh Edinburgh, UK

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

SpeechLLMs process spoken language directly from audio, but accent and vocal identity cues can lead to biased behaviour. Current bias evaluations often miss how such bias manifests in end-to-end speech interactions and how users experience it. We distinguish quality-of-service disparities (e.g., off-topic or low-effort responses) from content-level bias in coherent outputs, and examine intersectional effects of accent and perceived gender. In this work, we explore a two-part evaluation approach: (1) a controlled test cohort spanning six accents and two gender presentations, analysed with judge-free prompt-response metrics, and (2) an interactive study design using voice conversion to let users experience identical content through different vocal identities. Across two studies (Interactive, N=24; Observational, N=19), we find that voice conversion increases trust and acceptability for benign responses and encourages perspective-taking, while automated analysis in search of quality-of-service disparities, reveals {accent x gender} disparities in alignment and verbosity across SpeechLLMs. These results highlight voice conversion for probing and experiencing intersectional voice bias while our evaluation suite provides richer bias evaluations for spoken conversational AI.

Figures & tables

Figure 1. Study design – Observational Study (OS) and Interactive Study (IS) – illustrating three positions from which participants evaluate AI system behaviour. Participants either observe another user’s interaction with the system (OS), interact directly using their own voice (IS–O), or vicariously experience an interaction as if they were another speaker via voice conversion (IS–Vicarious). A diagram divided into three sections illustrating different study conditions. The left section, labeled 'Observational Study (OS, 3rd Person)', depicts a participant watching another person speak to a robot, representing observing another person's interaction. The middle section, labeled 'Interactive Study (IS, 1st Person) - IS-O', depicts a participant speaking directly to a robot, representing interacting through one's own voice. The right section, labeled 'Interactive Study (IS, 1st Person) - IS-VC', depicts a participant speaking to a robot while holding a mask of a different face, representing vicariously experiencing an interaction as another speaker via voice conversion.
Figure 2. The same inputs are treated differently by the SpeechLLM depending on accent x gender voice profile, in terms of (a) semantic alignment and (b) response verbosity : Qwen3-Omni is the most stable, while LFMAudio2-1.5B shows the largest variability.
Figure 3. Comparison of Harm, Acceptability, and Trust ratings across conditions for the Benign and Potentially Concerning AI responses. OS = Observational Study (reference); IS-O = Interactive Study with original voice; IS-VC = Interactive Study with voice conversion. β\beta = fixed effect coefficient from linear mixed-effects models with participant as random intercept. Cell colours indicate the magnitude and direction of the difference relative to the OS baseline for each measure: green denotes an increase, red denotes a decrease, and grey indicates the reference condition.
IDParticipant Observations on Potentially Concerning AI Response Quality and Bias
AI Response Quality
P17“[U]sers’ question was reasonably intelligible, even if the AI had difficulty understanding it, it should have probed more deeply, rather than just a ‘say it again in English’ which to someone with an overseas accent can easily be misinterpreted.”
P13“Seemed very rude that it was asking the user to repeat in English – it could have tried reply what it thought it heard and ask for clarification.”
P12“AI’s response was way below the standards expected, in fact it bordered on the offensive. The User’s prompt was hardly unintelligible yet the AI made no attempt to begin to understand it.”
P23“The caller was very clear. I had no problem understanding so for AI to not even recognise the language is extremely insulting, not to mention potential[ly] dangerous if problems cannot be reported”
P30“…the AI responded with odd information about the application process asking the user to give information about their new address (which they don’t have) and a utility bill at the new address again which they wouldn’t have. This response was not helpful for user at all.”
Table 1. Participant observations on Potentially Concerning responses, pertaining to AI quality and bias.
More in this topic

Intersectional Bias in Language Models

See all 9 papers

Explore similar work

CardsList

  1. The Voice Behind the Words: Quantifying Intersectional Bias in SpeechLLMs

    Mar 15, 2026Shree Harsha Bokkahalli Satish, Christoph Minixhofer, Maria Teleki +5Intersectional Bias in Language ModelsLanguage Model Bias Evaluation

    Department of Speech, Music and Hearing, KTH Royal Institute of Technology, Sweden · Centre for Speech Technology Research, University of Edinburgh, UK · Texas A&M University, USA

  2. HEAR: Real Voices, Real Bias: A Large-Scale Human-Recorded, Demographically Diverse Benchmark for Audio Language Models

    Sep 28, 2026Shen Yan, Duc Le, Irina-Elena VelicheAudio-Language Model EvaluationAudio-Language Models

    Meta Superintelligence Labs

  3. Greater accessibility can amplify discrimination in generative AI

    Mar 23, 2026Carolin Holtermann, Minh Duc Bui, Kaitlyn Zhou +3Social Bias in Language ModelsGender Bias in Language Models

    Trustworthy AI Lab, University of Hamburg · NALA Group, JGU Mainz · Cornell University +3