A Catalog of Basque Dialectal Resources: Online Collections and Standard-to-Dialectal Adaptations

Jaione Bengoetxea; Itziar Gonzalez-Dios; Rodrigo Agerri

cs.CLMar 26, 2026

A Catalog of Basque Dialectal Resources: Online Collections and Standard-to-Dialectal Adaptations

Authors: Jaione Bengoetxea, Itziar Gonzalez-Dios, Rodrigo Agerri

Organizations: HiTZ Center - Ixa, University of the Basque Country EHU

Abstract

Recent research on dialectal NLP has identified data scarcity as a primary limitation. To address this limitation, this paper presents a catalog of contemporary Basque dialectal data and resources, offering a systematic and comprehensive compilation of the dialectal data currently available in Basque. Two types of data sources have been distinguished: online data originally written in some dialect, and standard-to-dialect adapted data. The former includes all dialectal data that can be found online, such as news and radio sites, informal tweets, as well as online resources such as dictionaries, atlases, grammar rules, or videos. The latter consists of data that has been adapted from the standard variety to dialectal varieties, either manually or automatically. Regarding the manual adaptation, the test split of the XNLI Natural Language Inference dataset was manually adapted into three Basque dialects: Western, Central, and Navarrese-Lapurdian, yielding a high-quality parallel gold standard evaluation dataset. With respect to the automatic dialectal adaptation, the automatically adapted physical commonsense dataset (BasPhyCowest) underwent additional manual evaluation by native speakers to assess its quality and determine whether it could serve as a viable substitute for full manual adaptation (i.e., silver data creation).

Figures & tables

Figure 1: The Basque dialectal catalog consists of two different sources: online dialectal data and standard-to-dialect adapted data, either manually or automatically.

Source	Type	Dialect	Register	Modality	License
Bizkaia Irratia	Text & Audio	Western	Formal	News & Radio	cc-by-sa
Bizkaie!	Text & Audio	Western	Formal	News & Radio	CC-BY-SA
Xiberoko Botza	Text & Audio	Zuberoan	Formal	News & Radio	CC-BY-SA
Irulegiko Irratia	Text & Audio	Nav-Lap	Formal	News & Radio	CC-BY-SA
Gure Irratia	Text & Audio	Nav-Lap	Formal	News & Radio	CC-BY-SA
General Assemblies	Text	Western	Specialized	Minutes	N/A

Table 1: Summary of online Basque dialectal resources. N/A = Not Available.

Figure 2: Example of a linguistic atlas.

Figure 3: Illustration of XNLIeu dialectal adaptation. XNLIvar was a compilation of different instances in three different dialects. Parallel XNLIvar provides the same instances in three Basque dialects, offering completely parallel data.

Dialect	Birthplace	Age	Gender	Studies
Western	Elorrio	58	Male	Translation
Central	Arroa/Zumaia	34	Female	Translation
Nav-Lap	Donibane-Lohizune	33	Female	Translation

Table 2: Metadata from native Basque speakers who adapted the test partition of XNLIeu into dialects.

Figure 4: Levenshtein distance distribution for the three Parallel XNLIvar datasets.

Dialect	Distance
Western	15.56
Central	24.73
Nav-Lap	29.36

Table 3: The average Levenshtein Distance of the Parallel XNLIvar dialectal datasets to the Standard.

Figure 5: Manual evaluation results. From left to right, results for all sentence pairs ( all-pairs ), as well as results for different sentence pair combinations ( standard-west , standard-orthographic , and west-orthographic ).

Dialect	Birthplace	Age	Gender	Studies
Western	Oñati	33	Female	Translation
Western	Elorrio	59	Male	Translation

Table 4: Metadata from the native Western Basque speakers for the manual evaluation of the automatic adaptation.

Figure 6: The map of the classification of Basque dialects according to Zuazu (2008) .

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 7 — Figure 7: Dialectal adaptation prompt for the non-orthographic dataset version (BasPhyCo west ).

Panel of Figure 8 — Figure 8: Dialectal adaptation prompt for the orthographic dataset version (BasPhyCo west-new ).

Low-Resource Language Processing

Low-resource language processing builds and evaluates language technologies for languages with limited data, tools, or digital representation.

See all 565 papers

5-Dialects-BN: Unmasking the Impact of Transliteration on Bangla Dialectal LLMs
Sep 9, 2026Md Mahir Jawad, Galib Mahmud Jim, Rafid Ahmed +3Multilingual Language Model Evaluation Low-Resource Language Processing
BRAC University · Penta Global Limited · University of Central Florida +1
DiaLLM: An Investigation into the Robustness-Generation Gap in English Dialect Adaptation
Jul 8, 2026Jordan Painter, Dipankar Srirag, Adarsh Kappiyath +3Language Modeling LLM Alignment
Institute for People-Centred AI, University of Surrey, Surrey, United Kingdom · University of New South Wales, Sydney, Australia
Unified Multi-Dialectal Neural Machine Translation for Bangla Using the Dwadash Benchmark Corpus
Aug 12, 2026Rakib Ullah, Md. Ruhul Islam, Tanbir Ahmed +1Low-Resource MT Low-Resource Language Processing
Department of Computer Science and Engineering SYLHET ENGINEERING COLLEGE, SYLHET Sylhet, Bangladesh

Sep 9, 2026J/K move · Enter open · S save

5-Dialects-BN: Unmasking the Impact of Transliteration on Bangla Dialectal LLMs

Md Mahir Jawad, Galib Mahmud Jim, Rafid Ahmed, Mir Sazzat Hossain+2

BRAC University · Penta Global Limited · University of Central Florida+1

Multilingual Language Model Evaluation Low-Resource Language Processing

Open paper AskPDF arXiv

Large Language Models (LLMs) have achieved remarkable progress across natural language processing (NLP) tasks, yet their capabilities degrade sharply for low-resource languages and dialectally diverse settings. Bangla, the world's sixth most spoken language, exemplifies this gap: existing resources overwhelmingly target Standard Bangla, leaving its regional dialects without the benchmarks needed to develop or evaluate dialect-aware systems. We address this gap with 5-Dialects-BN, the first multi-annotation Bangla dialect benchmark to align Romanized transliteration with dialectal text, Standard Bangla, English, and subjectivity labels across five regional varieties. The dataset comprises 6,000 manually annotated entries spanning five major dialects: Chittagong, Barisal, Noakhali, Sylhet, and Rangpur (Chittagong 1,900; Noakhali 1,500; Sylhet 1,200; Barisal 700; Rangpur 700), reflecting natural online availability. Each entry is enriched with five aligned annotations: the original dialectal text, a Romanized transliteration, an English translation, a Standard Bangla translation, and a subjectivity label (subjective vs. objective). Annotations were produced and cross-validated by native speakers and undergraduate linguistics students to ensure dialectal authenticity and semantic fidelity. The resulting resource supports a diverse suite of tasks, including dialect identification, dialect-to-standard normalization, machine translation, subjectivity classification, and parameter-efficient fine-tuning (e.g., LoRA) of multilingual LLMs. By providing a standardized, multi-annotation benchmark, 5-Dialects-BN enables principled evaluation of LLMs on dialectally diverse Bangla and lays a foundation for further research in low-resource, dialect-aware NLP.

Abstract

Low-Resource Language Processing

Explore similar work

5-Dialects-BN: Unmasking the Impact of Transliteration on Bangla Dialectal LLMs

DiaLLM: An Investigation into the Robustness-Generation Gap in English Dialect Adaptation

Unified Multi-Dialectal Neural Machine Translation for Bangla Using the Dwadash Benchmark Corpus