Towards High-Quality Machine Translation for Kokborok: A Low-Resource Tibeto-Burman Language of Northeast India
Organizations: MWire Labs Shillong, Meghalaya, India · Department of Kokborok, Tripura University Agartala, Tripura, India
Abstract
We present KokborokMT, a high-quality neural machine translation (NMT) system for Kokborok (ISO 639-3), a Tibeto-Burman language spoken primarily in Tripura, India with approximately 1.5 million speakers. Despite its status as an official language of Tripura, Kokborok has remained severely under-resourced in the NLP community, with prior machine translation attempts limited to systems trained on small Bible-derived corpora achieving BLEU scores below 7. We fine-tune the NLLB-200-distilled-600M model on a multi-source parallel corpus comprising 36,052 sentence pairs: 9,284 professionally translated sentences from the SMOL dataset, 1,769 Bible-domain sentences from WMT shared task data, and 24,999 synthetic back-translated pairs generated via Gemini Flash from Tatoeba English source sentences. We introduce as a new language token for Kokborok in the NLLB framework. Our best system achieves BLEU scores of 17.30 and 38.56 on held-out test sets, representing substantial improvements over prior published results. Human evaluation by three annotators yields mean adequacy of 3.74/5 and fluency of 3.70/5, with substantial agreement between trained evaluators.
Figures & tables
| Source | Sentences | Type |
|---|---|---|
| SMOL (train) | 9,284 | Professional |
| WMT Bible (train) | 1,769 | Professional |
| Gemini BT (Tatoeba) | 24,999 | Synthetic |
| Total Train | 36,052 | |
| Dev (SMOL) | 500 | Professional |
| Test (SMOL) | 500 | Professional |
| Annotator | Adequacy | Fluency |
|---|---|---|
| Linguistic expert | 3.76 | 3.76 |
| Native speaker | 3.64 | 3.54 |
| Native researcher (Kokborok) | 3.84 | 3.80 |
| Mean | 3.74 | 3.70 |
| (expert vs researcher) | 0.672 | 0.677 |
| (expert vs native) | 0.134 | 0.109 |
| System | Direction | BLEU | chrF | ROUGE-L | METEOR | TER | Cos Sim | COMET |
|---|---|---|---|---|---|---|---|---|
| SMOL Test Set (general domain) | ||||||||
| Zero-Shot NLLB | en trp | 0.50 | 11.89 | 0.0261 | 0.0132 | 139.51 | 0.1939 | 0.2697 |
| Zero-Shot NLLB | trp en | 0.63 | 17.07 | 0.0675 | 0.0526 | 130.30 | 0.1872 | 0.2880 |
| System 1 (No BT) | en trp | 13.35 | 46.22 | 0.3873 | 0.3112 | 75.95 | 0.7707 | 0.6938 |
| System 1 (No BT) | trp en | 32.91 | 49.41 | 0.5498 | 0.5091 | 55.07 | 0.7466 | 0.6604 |
| System 2 (Full) | en trp | 15.25 | 47.67 | 0.3896 | 0.3138 | 74.26 | 0.7596 | 0.6958 |
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
| Epoch | Train Loss | Val Loss |
|---|---|---|
| System 2 (Full, With BT) | ||
| 1 | 0.3573 | 0.3816 |
| 2 | 0.2889 | 0.3187 |
| 3 | 0.2626 | 0.2896 |
| 4 | 0.2439 | 0.2723 |
| 5 | 0.2335 | 0.2607 |
Machine Translation
Machine translation develops computational methods for converting text, speech, and visual content between languages while preserving meaning and purpose.
See all 267 papers