cs.CLAug 24, 2026

The Dialect Tax: Dialectal Biases Persist throughout the Language Modeling Pipeline

Authors: Elle

Organizations: University of Oxford, Department of Computer Science

Abstract

Systematic dialectal performance gaps in language models (LMs) are well documented, but the source of these disparities within the modern language modeling pipeline remains unclear. Our study traces this "dialect tax" across the natural language processing pipeline. Using parallel English dialect corpora that hold meaning fixed while varying surface form, we first confirm that LMs recognize matched Standard American English (SAE) and dialectal texts as semantically equivalent. However, we discover further representational gaps corresponding to downstream performance gaps. Across model families and generations, modern LMs still encode dialectal texts unequally during tokenization, pre-training, post-training, and inference. Strikingly, bypassing traditional subword segmentation via a character-level counterfactual tokenizer removes neither input and output asymmetries nor dialectal accuracy gaps. During pre-training, dialect pairs induce more divergent gradient updates than pairs of entirely unrelated SAE documents, indicating that models find semantically equivalent dialectal content harder to learn from than unrelated SAE documents. During post-training, reward models show contextual, unstable dialect preferences, assigning higher values to isolated AAVE-exclusive tokens than to SAE-exclusive tokens, while full reasoning contexts receive task- and model-dependent dialect penalties. Overall, our findings suggest that the dialect tax is encoded and accumulated not by any one step in isolation, but at every step of the language modeling process.

Figures & tables

Appendix figures & tables22 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Side-by-side Comparison Amplifies Dialect Bias in Language Models

    May 23, 2026Kritee Kondapally, Claire J. Smerdon, Pooja C. Patel +5Social Bias in Language ModelsAlgorithmic Fairness

  2. Some Dialects Are More Equal Than Others: Non-Prestigious Arabic Dialectal Bias in LLMs

    Sep 21, 2026Mai Mohamed Eida, Ryan Dolan, Paul de Nijs +1Arabic NLPSocial Bias in Language Models

  3. DiaLLM: An Investigation into the Robustness-Generation Gap in English Dialect Adaptation

    Jul 8, 2026Jordan Painter, Dipankar Srirag, Adarsh Kappiyath +3Language ModelingLLM Alignment