cs.CLAug 24, 2026

Statistical Machine Translation Systems of English-Pnar Language Pair : Some Insights of the Emperical Study

Authors: Edawanbiang Dhar Surmila Thokchom, Thoudam Doren Singh

Organizations: Human Language Technology Lab, Department of Computer Science and Engineering, National Institute of Technology Meghalaya, Sohra, Meghalaya-793108, India · Department of Computer Science and Engineering, National Institute of Technology Meghalaya, Sohra, Meghalaya-793108, India

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Pnar, an Austroasiatic language spoken by approximately 0.4 million people in the Jaintia Hills of Meghalaya, lacks the digital corpora and natural language processing (NLP) resources. This paper presents the first machine translation study for the English and Pnar language pair. Using articles collected from the Wyrta newspaper, we built a parallel corpus comprising of 10,234 sentences and trained phrase-based statistical machine translation (SMT) systems the models using 9,563 parallel corpora under three configurations for each direction using Moses, GIZA++ , KenLM, varying lexicalized reordering and minimum error rate training (MERT) tuning. The models are evaluated on a held out test set of 371 sentences, the best performing system achieves a BLEU score of 14.97 (chrF2: 33.42, TER: 77.60) for Pnar to English and 11.16 (chrF2: 31.38, TER: 93.51) for English to Pnar, establishing the first quantitative benchmark for this language pair. Lexicalized reordering improves translation quality by 3.73 BLEU points for Pnar to English, reflecting the structural shift from the source language's SOV word order to the target language's SVO order, whereas MERT tuning degrades BLEU performance under low resource conditions. Finally, we analyze the remaining translation errors, including morphological out of vocabulary (OOV) words, long-distance reordering and Khasi code mixing and discuss future directions toward neural and multilingual machine translation for Pnar.

Figures & tables

MetricTrainDevTest
Sentences9,563300371
Pnar tokens∼\sim 234,800∼\sim 7,350∼\sim 5,777
English tokens∼\sim 229,400∼\sim 7,200∼\sim 5,865
Average Pnar sentence length∼\sim 24.6∼\sim 24.5∼\sim 15.6
Average English sentence length∼\sim 24.0∼\sim 24.0∼\sim 15.9
Table 1: Sentence counts, token counts and average sentence length for the train, development and test splits of the English-Pnar parallel corpus statistics
Figure 1: Phrase-based SMT model architecture : the decoder combines translation-model and language-model scores over the bilingual and monolingual training corpora to produce the target sentence.
SystemReorderingTuningLM
Config-1✓\checkmark msd-bidirectionalNone (untuned)5-gram KenLM
Config-2✓\checkmark msd-bidirectionalMERT (25 iter.)5-gram KenLM
Config-3×\times NoneNone (untuned)5-gram KenLM
Table 2: The three SMT configurations in each translation direction varying lexicalized reordering and MERT tuning
ConfigurationBLEU ↑\uparrowchrF2 ↑\uparrowTER ↓\downarrow
Config-311.2427.4491.70
Config-114.9733.4277.60
Config-212.4336.8695.04
Table 3: Translation performance on the 371-sentence test set.
DirectionConfigBLEU ↑\uparrowchrF2 ↑\uparrowTER ↓\downarrow
Pn →\to EnNo reorder11.2427.4491.70
Reorder14.9733.4277.60
Δ\Delta+3.73+5.98-14.10
En →\to PnNo reorder11.1631.3893.51
Reorder10.1430.9886.53
Δ\Delta-1.02-0.40-6.98
Table 4: BLEU, chrF, TER scores with vs. without lexicalized reordering, isolating the reordering effect from MERT tuning (both rows untuned).
DirectionComparisonΔ\Delta BLEU95% CIpp -valueSig.?
Pn →\rightarrow EnConfig-1 vs. Config-3 (reordering)+3.72+3.72[3.16, 4.30][3.16,\ 4.30]<0.001<0.001Yes †
Pn →\rightarrow EnConfig-1 vs. Config-2 (MERT)−2.50-2.50[−3.24, −1.75][-3.24,\ -1.75]<0.001<0.001Yes †
En →\rightarrow PnConfig-1 vs. Config-3 (reordering)−1.02-1.02[−2.19, 0.11][-2.19,\ 0.11]0.0840.084No
En →\rightarrow PnConfig-1 vs. Config-2 (MERT)−3.22-3.22[−3.96, −2.43][-3.96,\ -2.43]<0.001<0.001Yes †
Table 5: Paired bootstrap significance test ( B=1,000B{=}1{,}000 ) on the 371-sentence test set. †\dagger = significant at p<0.05p<0.05 .
TypeSentence
Source 1Sdang ka JAC yei yakhih wow khang yei lai i wan ki Truck ka Star Cement wa Dalmia.
ReferenceThe JAC began a protest to stop the movement of trucks belonging to Star Cement and Dalmia Cement.
SystemThe JAC yakhih to the khang the on the wan Truck Star Cement along with the Dalmia.
Source 2Mynsoo joor ki ar ngut hadien wa noh ka Truck chapoh riat ha Ri-Bhoi.
ReferenceTwo people were seriously injured after a truck fell into a gorge in Ri-Bhoi.
SystemTwo the ar ngut after the noh Truck chapoh riat in Ri-Bhoi.
Table 6: Sample translations from best Pnar →\to English system (Config-1, BLEU 14.97). System output is from the untuned, reordering-enabled configuration (Config-1); word-for-word artefacts are discussed in 5.5
TypeSentence
Source 1The JAC began a protest to stop the movement of trucks belonging to Star Cement and Dalmia Cement.
ReferenceSdang ka JAC yei yakhih wow khang yei lai i wan ki Truck ka Star Cement wa Dalmia.
SystemYa ka JAC began protest toh ka movement trucks belonging Star Cement wa u Dalmia Cement.
Source 2Two people were seriously injured after a truck fell into a gorge in Ri-Bhoi.
ReferenceMynsoo joor ki ar ngut hadien wa noh ka Truck chapoh riat ha Ri-Bhoi.
SystemYap seriously injured hadien ka truck fell into gorge wa Ri-Bhoi.
Table 7: Sample translations from best English →\to Pnar system (Config-1, BLEU 10.14).System output is from the untuned, reordering-enabled configuration (Config-1); word-for-word artefacts are discussed in 5.5
More in this topic

Machine Translation

Machine translation develops computational methods for converting text, speech, and visual content between languages while preserving meaning and purpose.

See all 267 papers

Explore similar work

CardsList
Mar 28, 2026

Towards High-Quality Machine Translation for Kokborok: A Low-Resource Tibeto-Burman Language of Northeast India

We present KokborokMT, a high-quality neural machine translation (NMT) system for Kokborok (ISO 639-3), a Tibeto-Burman language spoken primarily in Tripura, India with approximately 1.5 million speakers. Despite its status as an official language of Tripura, Kokborok has remained severely under-resourced in the NLP community, with prior machine translation attempts limited to systems trained on small Bible-derived corpora achieving BLEU scores below 7. We fine-tune the NLLB-200-distilled-600M model on a multi-source parallel corpus comprising 36,052 sentence pairs: 9,284 professionally translated sentences from the SMOL dataset, 1,769 Bible-domain sentences from WMT shared task data, and 24,999 synthetic back-translated pairs generated via Gemini Flash from Tatoeba English source sentences. We introduce as a new language token for Kokborok in the NLLB framework. Our best system achieves BLEU scores of 17.30 and 38.56 on held-out test sets, representing substantial improvements over prior published results. Human evaluation by three annotators yields mean adequacy of 3.74/5 and fluency of 3.70/5, with substantial agreement between trained evaluators.

Badal Nyalang, Biman Debbarma

MWire Labs Shillong, Meghalaya, India · Department of Kokborok, Tripura University Agartala, Tripura, India

Machine TranslationLow-Resource MTLow-Resource Language Processing
AskPDF
Mar 14, 2026

NepTam: A Nepali-Tamang Parallel Corpus and Baseline Machine Translation Experiments

Modern Translation Systems heavily rely on high-quality, large parallel datasets for state-of-the-art performance. However, such resources are largely unavailable for most of the South Asian languages. Among them, Nepali and Tamang fall into such category, with Tamang being among the least digitally resourced languages in the region. This work addresses the gap by developing NepTam20K, a 20K gold standard parallel corpus, and NepTam80K, an 80K synthetic Nepali-Tamang parallel corpus, both sentence-aligned and designed to support machine translation. The datasets were created through a pipeline involving data scraping from Nepali news and online sources, pre-processing, semantic filtering, balancing for tense and polarity (in NepTam20K dataset), expert translation into Tamang by native speakers of the language, and verification by an expert Tamang linguist. The dataset covers five domains: Agriculture, Health, Education and Technology, Culture, and General Communication. To evaluate the dataset, baseline machine translation experiments were carried out using various multilingual pre-trained models: mBART, M2M-100, NLLB-200, and a vanilla Transformer model. The fine-tuning on the NLLB-200 achieved the highest sacreBLEU scores of 40.92 (Nepali-Tamang) and 45.26 (Tamang-Nepali).

Rupak Raj Ghimire, Bipesh Subedi, Balaram Prasain +7

ILPRL, Kathmandu University, Nepal · Tribhuvan University, Nepal · Dublin City University, Ireland

Low-Resource MTMachine Translation
AskPDF
Jun 24, 2026

Neural Machine Translation for Low-Resource Tangkhul--English

We present a study on low-resource machine translation for the Tangkhul-English (nmf-en) language pair. Tangkhul is a severely under-resourced Tibeto-Burman language spoken primarily in Manipur, India, with virtually no prior natural language processing infrastructure. We describe two systems: (1) a primary system based on ByT5-large fine-tuned on 38,336 Tangkhul-English parallel sentence pairs, and (2) a contrastive system based on mT5-small fine-tuned on the same corpus. Our primary ByT5-large system achieves a corpus BLEU score of 39.97, chrF++ of 58.07, BERTScore F1 of 0.8104, and COMET (wmt22-comet-da) of 0.7302 on a held-out test set of 3,856 sentences. We further discuss the orthographic challenges specific to Tangkhul's Latin-script diacritics, the domain bias of our training corpus (which comprises biblical text, stories, and conversational data), and avenues for future improvement through data diversification and domain adaptation.

Chormi Zimik Vashai, Agniva Maiti

Independent Researcher · KIIT University

Low-Resource MTLow-Resource Language ProcessingMachine Translation
AskPDF