Variation is the Norm: Embracing Sociolinguistics in NLP

Anne-Marie Lutgen; Alistair Plum; Verena Blaschke; Barbara Plank; Christoph Purschke

cs.CLMar 25, 2026

Variation is the Norm: Embracing Sociolinguistics in NLP

Authors: Anne-Marie Lutgen, Alistair Plum, Verena Blaschke, Barbara Plank, Christoph Purschke

Organizations: University of Luxembourg, Esch-sur-Alzette, Luxembourg · MaiNLP, LMU Munich, Germany · Munich Center for Machine Learning (MCML), Munich, Germany

Abstract

In Natural Language Processing (NLP), variation is typically seen as noise and "normalised away" before processing, even though it is an integral part of language. Conversely, studying language variation in social contexts is central to sociolinguistics. We present a framework to combine the sociolinguistic dimension of language with the technical dimension of NLP. We argue that by embracing sociolinguistics, variation can actively be included in a research setup, in turn informing the NLP side. To illustrate this, we provide a case study on Luxembourgish, an evolving language featuring a large amount of orthographic variation, demonstrating how NLP performance is impacted. The results show large discrepancies in the performance of models tested and fine-tuned on data with a large amount of orthographic variation in comparison to data closer to the (orthographic) standard. Furthermore, we provide a possible solution to improve the performance by including variation in the fine-tuning process. This case study highlights the importance of including variation in the research setup, as models are currently not robust to occurring variation. Our framework facilitates the inclusion of variation in the thought-process while also being grounded in the theoretical framework of sociolinguistics.

Figures & tables

Figure 1: Illustration of the container metaphor for language and variety.

Variant	Task	WER	CER
Destandardised	IC	78.09	14.12
	NER	63.78	12.94
	POS	65.61	13.00
	WNLI	83.23	16.02
	TC	64.57	9.89
Normalised	CM	28.81	4.97

Table 1: WER (in %) and CER (in %) between pairs of standard and non-standard variety sentences.

Table 3

Task	Train	Dev	Test
IC	698	149	159
NER	4298	459	770
POS	4278	460	388
WNLI	568	63	136
CM	6000	1000	2000
SC	1299	185	364

Table 5: Number of sentences in the training, development, and test sets for each task.

Task	batch size	LR	epochs
IC, WNLI, CM, SC	16	5 -5	5
NER, POS	16	5 -5	3
TC	16	2 -5	5

Table 6: Hyperparameter settings for each task, inspired by Lothritz et al. (2022)

Table 6

Language Model Robustness

Language model robustness studies how reliably language models preserve correct, safe behavior under input changes, distribution shifts, and adversarial pressure.

See all 493 papers

Measuring the Cost of Variety Conflation in Multilingual MT Evaluation: Adding Mozambican Xichangana, Nyanja and Sena to FLORES+
Sep 12, 2026Felermino D. M. A. Ali, Delfina Lázaro Mateus, Manuel Valente MangueAfrican Language Processing MT Evaluation
Faculdade de Engenharia da Universidade Lúrio, Bairro Eduardo Mondlane, Pemba, Mozambique · Universidade Eduardo Mondlane - Escola de Comunicação e Artes, Av. Samora Machel, Maputo, Mozambique
A Systematic Review of NLP for Ghanaian Languages: Datasets, Models, and a Research Roadmap
May 10, 2024Sheriff Issaka, Erick Rosas Gonzalez, Colene Agbo +18Low-Resource Language Processing African Language Processing
African Languages Lab
Versteasch du mi? Computational and Socio-Linguistic Perspectives on GenAI, LLMs, and Non-Standard Language
Mar 30, 2026Verena Platzgummer, John McCrae, Sina AhmadiSocial Bias in Language Models Low-Resource Language Processing
University of Galway · University of Zurich

Sep 12, 2026J/K move · Enter open · S save

Measuring the Cost of Variety Conflation in Multilingual MT Evaluation: Adding Mozambican Xichangana, Nyanja and Sena to FLORES+

Felermino D. M. A. Ali, Delfina Lázaro Mateus, Manuel Valente Mangue

Faculdade de Engenharia da Universidade Lúrio, Bairro Eduardo Mondlane, Pemba, Mozambique · Universidade Eduardo Mondlane - Escola de Comunicação e Artes, Av. Samora Machel, Maputo, Mozambique

African Language Processing MT Evaluation Low-Resource MT

Open paper AskPDF arXiv

In this paper, we extend FLORES+ with Portuguese-source evaluation sets for three Mozambican Bantu varieties: Xichangana, Mozambican Nyanja, and Sena. We compare Xichangana with the existing Tsonga reference and Mozambican Nyanja with Chichewa, and evaluate NLLB-200, Google Translate, GPT, and a variant-aware NLLB model. Holding system output fixed reveals substantial reference sensitivity. On devtest, changing only the reference from Tsonga to Xichangana reduces spBLEU by 13.10 points for NLLB-200 and 15.30 for Google. On matched Nyanja subsets, replacing Chichewa with Mozambican Nyanja produces smaller but consistent reductions of 3.03 and 6.10 spBLEU, respectively. Variant-aware fine-tuning reverses this pattern on the intended targets: relative to NLLB-200, it improves Xichangana by 7.04 spBLEU and Mozambican Nyanja by 5.33 on devtest, while losing performance on the sibling references. GPT is competitive on Tsonga and Chichewa but substantially weaker on the Mozambican varieties. For Sena, the finetuned model reaches 12.64 spBLEU and 36.21 chrF++ on devtest. These findings motivate variety-aware language identifiers, references, and reporting for cross-border languages or language dialects/variants. The data is publicly available on Hugging Face at https://huggingface.co/datasets/MOZNLP/FLORES_MOZ

Abstract

Language Model Robustness

Explore similar work

Measuring the Cost of Variety Conflation in Multilingual MT Evaluation: Adding Mozambican Xichangana, Nyanja and Sena to FLORES+

A Systematic Review of NLP for Ghanaian Languages: Datasets, Models, and a Research Roadmap

Versteasch du mi? Computational and Socio-Linguistic Perspectives on GenAI, LLMs, and Non-Standard Language