Dialect and Gender Bias in YouTube's Spanish Captioning System

Iris Dania Jimenez; Christoph Kern

cs.CLFeb 27, 2026

Dialect and Gender Bias in YouTube's Spanish Captioning System

Authors: Iris Dania Jimenez, Christoph Kern

Organizations: Department of Statistics, Ludwig-Maximilians-Universität München, Munich, Germany. · Munich Center for Machine Learning (MCML), Munich, Germany.

Abstract

Spanish is the official language of twenty-one countries and is spoken by over 441 million people. Naturally, there are many variations in how Spanish is spoken across these countries. Media platforms such as YouTube rely on automatic speech recognition systems to make their content accessible to different groups of users. However, YouTube offers only one option for automatically generating captions in Spanish. This raises the question: could this captioning system be biased against certain Spanish dialects? This study examines the potential biases in YouTube's automatic captioning system by analyzing its performance across various Spanish dialects. By comparing the quality of captions for female and male speakers from different regions, we identify systematic disparities which can be attributed to specific dialects. Our study provides further evidence that algorithmic technologies deployed on digital platforms need to be calibrated to the diverse needs and experiences of their user populations.

Figures & tables

Table 1: Summary of ASR’s (by YouTube, Google, Amazon, and custom-build) performance on different dialects

Country	Gender	Duration (min)	Speakers	Sentences
Argentina	F	28:40	30	177
	M	28:42	13	183
Chile	F	28:48	13	164
	M	28:50	18	167
Colombia	F	28:48	16	167
	M	28:45	17	171

Table 2: Summary of processed data with YouTube captions.

Table 3: Case sensitive WER, CER, and Entity Recall by country

Model	Predictors	AIC	Significant Variables
1	Country	-1537.32	Puerto Rico, Peru
2	Gender	-1509.96	None
3	Country + Gender	-1535.51	Puerto Rico
4	Country + Pitch	-1515.24	Pitch, Puerto Rico
5	Country + Intensity	-1780.80	Intensity
6	Gender + Pitch + Pitch:Gender	-1520.25	Pitch

Table 4: Summary of models predicting Word Error Rate.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Table 5: Case insensitive WER, CER, and Entity Recall by country

Country	Pitch	Intensity	Pitch (F)	Pitch (M)	Intensity (F)	Intensity (M)
Argentina	144	0.022	171	117	0.021	0.022
Chile	158	0.021	197	118	0.021	0.020
Colombia	146	0.024	178	114	0.021	0.026
Mexico	203	0.042	248	158	0.043	0.042
Peru	147	0.024	171	123	0.023	0.024
Puerto Rico	-	-	164	-	0.028	-

Appendix

Table 6: Average pitch and intensity by country and gender.

Table 7: Mixed-effects regression models predicting WER (ref.: Mexico (country) and Female (gender)).

Table 8

Table 9

Table 10

Demographic Bias in ASR

Demographic bias in automatic speech recognition (ASR) concerns unequal transcription accuracy and user burdens across speaker groups and language varieties.

See all 30 papers

A Sociolinguistic Analysis of Automatic Speech Recognition Bias in Newcastle English
Mar 25, 2026Dana Serditova, Kevin TangDemographic Bias in ASR Automatic Speech Recognition
University of Regensburg, Faculty of Languages, Literature, and Cultures, Department of English and American Studies, Regensburg, 93040, Bavaria, Germany · Heinrich Heine University Düsseldorf, Faculty of Arts and Humanities, Institute of English and American Studies, Department English Language and Linguistics, Düsseldorf, 40225, North Rhine-Westphalia, Germany · University of Florida, Department of Linguistics, Gainesville, 32611-5454, Florida, U.S.A.
Side-by-side Comparison Amplifies Dialect Bias in Language Models
May 23, 2026Kritee Kondapally, Claire J. Smerdon, Pooja C. Patel +5Social Bias in Language Models Algorithmic Fairness
University of Southern California, USA
Benchmarking Automatic Speech Recognition Tools for Iberian Languages
Sep 29, 2026Fernando López, Pablo Gómez, David Solans +2ASR Evaluation Automatic Speech Recognition
Telefónica Innovación Digital, Spain · Universidad Autónoma de Madrid, Spain

Mar 25, 2026J/K move · Enter open · S save

A Sociolinguistic Analysis of Automatic Speech Recognition Bias in Newcastle English

Dana Serditova, Kevin Tang

University of Regensburg, Faculty of Languages, Literature, and Cultures, Department of English and American Studies, Regensburg, 93040, Bavaria, Germany · Heinrich Heine University Düsseldorf, Faculty of Arts and Humanities, Institute of English and American Studies, Department English Language and Linguistics, Düsseldorf, 40225, North Rhine-Westphalia, Germany · University of Florida, Department of Linguistics, Gainesville, 32611-5454, Florida, U.S.A.

Demographic Bias in ASR Automatic Speech Recognition ASR Evaluation

Open paper AskPDF arXiv

Automatic Speech Recognition (ASR) systems are widely used in everyday communication, education, healthcare, and industry, yet their performance remains uneven across speakers, particularly when dialectal variation diverges from the mainstream accents represented in training data. This study investigates ASR bias through a sociolinguistic analysis of Newcastle English, a regional variety of North-East England that has been shown to challenge current speech recognition technologies. Using spontaneous speech from the Diachronic Electronic Corpus of Tyneside English (DECTE), we evaluate the output of a state-of-the-art commercial ASR system and conduct a fine-grained analysis of more than 3,000 transcription errors. Errors are classified by linguistic domain and examined in relation to social variables including gender, age, and socioeconomic status. In addition, an acoustic case study of selected vowel features demonstrates how gradient phonetic variation contributes directly to misrecognition. The results show that phonological variation accounts for the majority of errors, with recurrent failures linked to dialect-specific features like vowel quality and glottalisation, as well as local vocabulary and non-standard grammatical forms. Error rates also vary across social groups, with higher error frequencies observed for men and for speakers at the extremes of the age spectrum. These findings indicate that ASR errors are not random but socially patterned and can be explained from a sociolinguistic perspective. Thus, the study demonstrates the importance of incorporating sociolinguistic expertise into the evaluation and development of speech technologies and argues that more equitable ASR systems require explicit attention to dialectal variation and community-based speech data.

Abstract

Demographic Bias in ASR

Explore similar work

A Sociolinguistic Analysis of Automatic Speech Recognition Bias in Newcastle English

Side-by-side Comparison Amplifies Dialect Bias in Language Models

Benchmarking Automatic Speech Recognition Tools for Iberian Languages