cs.CLMar 5, 2026

Oral to Web: Digitizing 'Zero Resource'Languages of Bangladesh

Authors: Mohammad Mamun Or Rashid

Organizations: Consultant, EBLICT Project, Bangladesh Computer Council · Associate Professor, Jahangirnagar University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

We present the Multilingual Cloud Corpus, the first national-scale, parallel, multimodal linguistic dataset of Bangladesh's ethnic and indigenous languages. Despite being home to approximately 40 minority languages spanning four language families, Bangladesh has lacked a systematic, cross-family digital corpus for these predominantly oral, computationally "zero resource" varieties, 14 of which are classified as endangered. Our corpus comprises 85792 structured textual entries, each containing a Bengali stimulus text, an English translation, and an IPA transcription, together with approximately 107 hours of transcribed audio recordings, covering 42 language varieties from the Tibeto-Burman, Indo-European, Austro-Asiatic, and Dravidian families, plus two genetically unclassified languages. The data were collected through systematic fieldwork over 90 days across nine districts of Bangladesh, involving 16 data collectors, 77 speakers, and 43 validators, following a predefined elicitation template of 2224 unique items organized at three levels of linguistic granularity: isolated lexical items (475 words across 22 semantic domains), grammatical constructions (887 sentences across 21 categories including verbal conjugation paradigms), and directed speech (862 prompts across 46 conversational scenarios). Post-field processing included IPA transcription by 10 linguists with independent adjudication by 6 reviewers. The complete dataset is publicly accessible through the Multilingual Cloud platform (multiling.cloud), providing searchable access to annotated audio and textual data for all documented varieties. We describe the corpus design, fieldwork methodology, dataset structure, and per-language coverage, and discuss implications for endangered language documentation, low-resource NLP, and digital preservation in linguistically diverse developing countries.

Figures & tables

Figure 1: Taxonomic Classification of Language Varieties
Figure 2: Methodology for Corpus Construction
DomainItemsRepresentative Examples
Verbal Words112to sit, to come, to go, to eat, to sleep, to run, to give, to take
Animals and Birds37cow, duck, tiger, bird, fish, snake, elephant, crow, goat
Words about Time34yesterday, today, tomorrow, now, then, later, morning, night
Kinship Terms32mother, father, grandfather, grandmother, uncle, aunt, son, daughter
Body Parts29hand, finger, mouth, eye, head, leg, ear, nose, teeth
Fruits27mango, jackfruit, banana, guava, coconut, papaya, litchi
Table 1: The domains and examples of words
Sentence TypeItemsDesign Purpose
Travel Manual47Formulaic questions and requests; documents interrogative and polite request strategies
Interrogative49Yes/no and wh-questions; documents question formation and word order
Assertive-Negative46Negative declaratives; documents negation placement and morphology
Sensory48Perception and sensation descriptions; documents sensory verb constructions
Assertive-Affirmative33Positive declaratives; baseline clause structure documentation
Case Markers38Noun phrases with explicit case relations; documents case marking systems
Table 2: Pre-defined sentence types of the corpus
CategoryTopicsUniqueTotal
  ItemsEntries
Isolated Lexical2247518344
Items (Words)   
Grammatical Constr.2188734204
(Sentences)   
Directed Speech46 (+8)86233244
Table 3: Summary of the Predefined Template
FieldDescription
Language NameTarget minority language in English
Serial NoSequential identifier within each language subset
Data TypeCollection category: Words, Sentences, or Directed/Elicited
DistrictGeographic location where the data was recorded
Data CollectorName of the trained field researcher
SpeakerName of the native language consultant
Table 4: Data Structure
Figure 3: Per-Language Data Coverage
Figure 4: Audio Corpus Duration by Language
NameDesignationRole
Michael Mridul Kanti SangmaEthnic Language ExpertLead contributor for gathering linguistic diversity information from the field; data collection; language profiling; IPA standard definition and transcription
Charu HaqEthnic Language ExpertLead contributor for compiling linguistic diversity information from the field; compiling language profiles; data collection
Sharmin Akter ShovaLead LinguistIPA standard definition and transcription supervision; data collection
Samar Michael SorenEthnic Language SpecialistField data collection; language profiling; IPA standard definition and transcription
Ribeng DewanData Collection SpecialistLead contributor for gathering linguistic diversity information from the field; data collection; language profiling
Eduard SorenAssociate LinguistLead contributor for gathering linguistic diversity information from the field; data collection; language profiling; IPA standard definition and transcription
Table 5: Project Contributors
More in this topic

Low-Resource Language Processing

Low-resource language processing builds and evaluates language technologies for languages with limited data, tools, or digital representation.

See all 548 papers

Explore similar work

CardsList
Oct 1, 2026

BanglaDial-Abuse: A Corpus-Grounded Dataset for Regional Dialect Identification in Abusive Bangla Text

Regional linguistic variation remains an important challenge for Bangla natural language processing, particularly in informal and non-standard text. This paper introduces BanglaDial-Abuse, a balanced Bengali-script dataset developed for regional dialect identification in abusive and hostile Bangla text. The dataset contains 1,000 sentences distributed equally across four linguistic varieties: Standard Bangla, Chattagram, Sylhet, and Barishal, with 250 samples per class. The resource was constructed using a corpus-grounded synthetic procedure incorporating regional variation in pronouns, possessive forms, verb morphology, negation, interrogative structures, postpositions, vocabulary, and Bengali-script spelling conventions while preserving the underlying hostile or abusive meaning. Descriptive analysis shows broadly comparable sentence-length distributions but partially distinct lexical spaces across the four classes. Pairwise Jaccard vocabulary similarity ranges from 0.37 to 0.56. The primary task is four-class regional dialect identification rather than binary abusive-text detection. The dataset is publicly available through Zenodo under a Creative Commons Attribution 4.0 license. The current version is intended as a research and prototyping corpus rather than a native-speaker-validated gold-standard linguistic resource. Keywords: Bangla, Bengali, dialect identification, regional dialect, abusive language, low-resource NLP, Chattagram, Sylhet, Barishal, dataset

Hasin Almas Sifat

Department of Computer Science American International University-Bangladesh Dhaka, Bangladesh

Bengali NLPLow-Resource Language Processing
AskPDF
Sep 9, 2026

5-Dialects-BN: Unmasking the Impact of Transliteration on Bangla Dialectal LLMs

Large Language Models (LLMs) have achieved remarkable progress across natural language processing (NLP) tasks, yet their capabilities degrade sharply for low-resource languages and dialectally diverse settings. Bangla, the world's sixth most spoken language, exemplifies this gap: existing resources overwhelmingly target Standard Bangla, leaving its regional dialects without the benchmarks needed to develop or evaluate dialect-aware systems. We address this gap with 5-Dialects-BN, the first multi-annotation Bangla dialect benchmark to align Romanized transliteration with dialectal text, Standard Bangla, English, and subjectivity labels across five regional varieties. The dataset comprises 6,000 manually annotated entries spanning five major dialects: Chittagong, Barisal, Noakhali, Sylhet, and Rangpur (Chittagong 1,900; Noakhali 1,500; Sylhet 1,200; Barisal 700; Rangpur 700), reflecting natural online availability. Each entry is enriched with five aligned annotations: the original dialectal text, a Romanized transliteration, an English translation, a Standard Bangla translation, and a subjectivity label (subjective vs. objective). Annotations were produced and cross-validated by native speakers and undergraduate linguistics students to ensure dialectal authenticity and semantic fidelity. The resulting resource supports a diverse suite of tasks, including dialect identification, dialect-to-standard normalization, machine translation, subjectivity classification, and parameter-efficient fine-tuning (e.g., LoRA) of multilingual LLMs. By providing a standardized, multi-annotation benchmark, 5-Dialects-BN enables principled evaluation of LLMs on dialectally diverse Bangla and lays a foundation for further research in low-resource, dialect-aware NLP.

Md Mahir Jawad, Galib Mahmud Jim, Rafid Ahmed +3

BRAC University · Penta Global Limited · University of Central Florida +1

Multilingual Language Model EvaluationLow-Resource Language Processing
AskPDF
Sep 24, 2026

BanglaKontho: Closing the Long-Form Gap in Bangla Text-to-Speech

Bangla, the seventh most spoken language in the world, remains under-resourced for neural text-to-speech. Public Bangla speech corpora are dominated by short read-prompt utterances collected for speech recognition, leaving long-form prosody and consistent single-speaker narration uncovered. We present BanglaKontho, a single-speaker Bangla TTS corpus of 20 hours derived from professional audiobook recordings: 7,050 segmented utterances with verified transcripts at 24 kHz. We also release a reusable Bangla text normalizer covering Bangladeshi-style digit grouping, currency and date expressions, Danda punctuation and Unicode normalization, together with the full preprocessing pipeline. An MB-iSTFT-VITS baseline trained from scratch reaches 9.5% WER and 4.46 naturalness MOS, against 16.0% and 3.16 for the same architecture retrained on the 12-hour IndicTTS-Bn corpus. The corpus is released openly under CC BY-NC 4.0.

Mizbaul Haque Maruf

Vivasoft Limited, Dhaka, Bangladesh

TTS SynthesisLow-Resource TTS SynthesisLong-Form TTS SynthesisLow-Resource Language ProcessingSpeech Corpus Construction
AskPDF