Organizations: Human Language Technology Lab, Department of Computer Science and Engineering, National Institute of Technology Meghalaya, Sohra, Meghalaya-793108, India · Department of Computer Science and Engineering, National Institute of Technology Meghalaya, Sohra, Meghalaya-793108, India
Pnar, an Austroasiatic language spoken by approximately 0.4 million people in the Jaintia Hills of Meghalaya, lacks the digital corpora and natural language processing (NLP) resources. This paper presents the first machine translation study for the English and Pnar language pair. Using articles collected from the Wyrta newspaper, we built a parallel corpus comprising of 10,234 sentences and trained phrase-based statistical machine translation (SMT) systems the models using 9,563 parallel corpora under three configurations for each direction using Moses, GIZA++ , KenLM, varying lexicalized reordering and minimum error rate training (MERT) tuning. The models are evaluated on a held out test set of 371 sentences, the best performing system achieves a BLEU score of 14.97 (chrF2: 33.42, TER: 77.60) for Pnar to English and 11.16 (chrF2: 31.38, TER: 93.51) for English to Pnar, establishing the first quantitative benchmark for this language pair. Lexicalized reordering improves translation quality by 3.73 BLEU points for Pnar to English, reflecting the structural shift from the source language's SOV word order to the target language's SVO order, whereas MERT tuning degrades BLEU performance under low resource conditions. Finally, we analyze the remaining translation errors, including morphological out of vocabulary (OOV) words, long-distance reordering and Khasi code mixing and discuss future directions toward neural and multilingual machine translation for Pnar.
Figures & tables
Metric
Train
Dev
Test
Sentences
9,563
300
371
Pnar tokens
∼ 234,800
∼ 7,350
∼ 5,777
English tokens
∼ 229,400
∼ 7,200
∼ 5,865
Average Pnar sentence length
∼ 24.6
∼ 24.5
∼ 15.6
Average English sentence length
∼ 24.0
∼ 24.0
∼ 15.9
Table 1: Sentence counts, token counts and average sentence length for the train, development and test splits of the English-Pnar parallel corpus statistics
Figure 1: Phrase-based SMT model architecture : the decoder combines translation-model and language-model scores over the bilingual and monolingual training corpora to produce the target sentence.
System
Reordering
Tuning
LM
Config-1
✓ msd-bidirectional
None (untuned)
5-gram KenLM
Config-2
✓ msd-bidirectional
MERT (25 iter.)
5-gram KenLM
Config-3
× None
None (untuned)
5-gram KenLM
Table 2: The three SMT configurations in each translation direction varying lexicalized reordering and MERT tuning
Configuration
BLEU ↑
chrF2 ↑
TER ↓
Config-3
11.24
27.44
91.70
Config-1
14.97
33.42
77.60
Config-2
12.43
36.86
95.04
Table 3: Translation performance on the 371-sentence test set.
Direction
Config
BLEU ↑
chrF2 ↑
TER ↓
Pn → En
No reorder
11.24
27.44
91.70
Reorder
14.97
33.42
77.60
Δ
+3.73
+5.98
-14.10
En → Pn
No reorder
11.16
31.38
93.51
Reorder
10.14
30.98
86.53
Δ
-1.02
-0.40
-6.98
Table 4: BLEU, chrF, TER scores with vs. without lexicalized reordering, isolating the reordering effect from MERT tuning (both rows untuned).
Direction
Comparison
Δ BLEU
95% CI
p -value
Sig.?
Pn → En
Config-1 vs. Config-3 (reordering)
+3.72
[3.16,4.30]
<0.001
Yes †
Pn → En
Config-1 vs. Config-2 (MERT)
−2.50
[−3.24,−1.75]
<0.001
Yes †
En → Pn
Config-1 vs. Config-3 (reordering)
−1.02
[−2.19,0.11]
0.084
No
En → Pn
Config-1 vs. Config-2 (MERT)
−3.22
[−3.96,−2.43]
<0.001
Yes †
Table 5: Paired bootstrap significance test ( B=1,000 ) on the 371-sentence test set. † = significant at p<0.05 .
Type
Sentence
Source 1
Sdang ka JAC yei yakhih wow khang yei lai i wan ki Truck ka Star Cement wa Dalmia.
Reference
The JAC began a protest to stop the movement of trucks belonging to Star Cement and Dalmia Cement.
System
The JAC yakhih to the khang the on the wan Truck Star Cement along with the Dalmia.
Source 2
Mynsoo joor ki ar ngut hadien wa noh ka Truck chapoh riat ha Ri-Bhoi.
Reference
Two people were seriously injured after a truck fell into a gorge in Ri-Bhoi.
System
Two the ar ngut after the noh Truck chapoh riat in Ri-Bhoi.
Table 6: Sample translations from best Pnar → English system (Config-1, BLEU 14.97). System output is from the untuned, reordering-enabled configuration (Config-1); word-for-word artefacts are discussed in 5.5
Type
Sentence
Source 1
The JAC began a protest to stop the movement of trucks belonging to Star Cement and Dalmia Cement.
Reference
Sdang ka JAC yei yakhih wow khang yei lai i wan ki Truck ka Star Cement wa Dalmia.
System
Ya ka JAC began protest toh ka movement trucks belonging Star Cement wa u Dalmia Cement.
Source 2
Two people were seriously injured after a truck fell into a gorge in Ri-Bhoi.
Reference
Mynsoo joor ki ar ngut hadien wa noh ka Truck chapoh riat ha Ri-Bhoi.
System
Yap seriously injured hadien ka truck fell into gorge wa Ri-Bhoi.
Table 7: Sample translations from best English → Pnar system (Config-1, BLEU 10.14).System output is from the untuned, reordering-enabled configuration (Config-1); word-for-word artefacts are discussed in 5.5
Machine translation develops computational methods for converting text, speech, and visual content between languages while preserving meaning and purpose.