Neural machine translation (NMT) from Chinese to low-resource Southeast Asian languages remains severely constrained by the extreme scarcity of clean parallel corpora and the pervasive noise in existing mined data. This chronic shortage not only impedes effective model training but also sustains a large performance gap with high-resource directions, leaving millions of speakers of languages such as Lao, Burmese, and Tagalog with persistently low-quality translation systems despite recent advances in large multilingual models. We introduce Multilingual Expert-Reward Informed Tuning (MERIT), a unified translation framework that transforms the traditional English-centric ALT benchmark into a Chinese-centric evaluation suite for five Southeast Asian low-resource languages (LRLs). Our framework combines language-specific token prefixing (LTP) with supervised fine-tuning (SFT) and a novel group relative policy optimization (GRPO) guided by the semantic alignment reward (SAR). These results confirm that, in LRL{\textrightarrow}Chinese translation, targeted data curation and reward-guided optimization dramatically outperform mere model scaling.
Figures & tables
Figure 1: Overview of the MERIT framework. The pipeline consists of (a) heuristic data selection utilizing the Elite Parallel Data Sampler (EPDS, algorithm 1 ) and Data Integrity Validation (DIV, algorithm 2 ), (b) translation scoring via a QE agent trained on expert cross-reviewed quality data using GRPO-SAR, (c) reward-based data distillation, which utilizes the QE agent to rescore the entire training set and filter for high-quality translation data, and (d) final model optimization using SFT-LTP based on this distilled dataset.
Languages
Speaker Population 1 1 1 Speaker numbers derive from the most recent national censuses or Ethnologue reports (2023–2025) and are expressed in millions (M). ( Eberhard et al., 2023 )
Filtered Subset 2 2 2 Each ALT language contains approximately 20k aligned sentence pairs from a shared English source. See https://www2.nict.go.jp/astrec-att/member/mutiyama/ALT/ .
LRL
Chinese ( zh )
1180M
✗
✗
Hindi ( hi )
345M
✗
✗
Bengali ( bn )
234M
✗
✗
Japanese ( ja )
121M
✗
✗
Vietnamese ( vi )
86M
10K
✓
Indonesian ( id )
43M
10K
✓
Table 1: The ALT corpus statistics sorted by L1 speaker population. All counts refer to L1 speakers and are rounded to the nearest million (M).
Figure 2: Performance–Scale Trade-offs of MERIT-3B and Baseline Models on Chinese-Centric Multilingual Translation. Comparison of BLEU-chrF scores against model size (log-scale) across MERIT-3B, open-source, and estimated closed-source models.
Strict-Overlap
BLEU-4
sacreBLEU
ROUGE-L
Model
fil
id
lo
my
vi
fil
id
lo
my
vi
fil
id
lo
my
vi
FT
OS
GPT-4o
45.26
47.77
28.10
30.48
45.03
43.60
45.81
27.38
29.53
44.50
33.62
31.40
12.33
23.57
28.25
✗
✗
Claude-3.5 Sonnet
42.97
47.35
32.20
30.92
45.14
43.38
46.54
32.65
30.14
44.55
35.03
31.17
14.00
24.44
28.45
✗
✗
Gemini-2.5 Flash
49.26
48.73
35.79
36.91
39.24
47.48
47.20
35.03
36.07
38.63
35.89
30.83
13.53
23.92
24.62
✗
✗
DeepSeek-V3
46.19
41.83
25.57
26.68
41.29
44.12
41.38
25.25
26.29
40.90
35.15
30.06
12.75
22.95
28.27
✗
✓
Qwen-2.5 32B
43.56
47.87
23.27
20.43
46.23
42.01
46.61
22.48
19.50
44.63
34.97
31.85
11.84
20.61
29.08
✗
✓
Table 2: Evaluation on five Southeast Asian languages. Strict-Overlap metrics include BLEU-4, sacreBLEU, and ROUGE-L. Semantic-Friendly metrics include chrF, METEOR, and BERTScore. For each metric column: Bold values indicate the highest score, and Underlined values indicate the second highest score across all models. FT: Fine-tuned; OS: Open Source.
Model
BLEU-4
chrF
Overall (BLEU-chrF)
fil
id
lo
my
vi
fil
id
lo
my
vi
Qwen2.5-0.5b
0.03
0.03
0.02
0.01
0.01
0.16
0.12
0.40
0.25
0.06
0.11
+ SFT-LTP
1.86 ↑1.83
4.02 ↑4.00
0.25 ↑0.22
0.15 ↑0.14
3.12 ↑3.11
4.85 ↑4.69
10.38 ↑10.26
1.41 ↑1.00
1.07 ↑0.82
8.55 ↑8.50
3.57 ↑3.46
+ GRPO-SAR
2.31 ↑2.28
4.32 ↑4.29
0.26 ↑0.24
0.16 ↑0.16
6.29 ↑6.27
5.00 ↑4.84
10.64 ↑10.52
1.38 ↑0.98
1.22 ↑0.96
12.36 ↑12.30
4.39 ↑4.28
+ LLME
0.32 ↑0.29
0.45 ↑0.42
0.08 ↑0.06
0.07 ↑0.06
0.79 ↑0.78
1.20 ↑1.04
1.66 ↑1.54
0.45 ↑0.05
1.25 ↑1.00
2.82 ↑2.76
0.91 ↑0.80
Avg.
1.13
2.21
0.15
0.10
2.55
2.80
5.70
0.91
0.95
5.95
2.25
Table 3: Evaluation results of Qwen-2.5-0.5B and 3B on five Southeast Asian languages. All values are rounded to two decimal places. Improvements over the baseline ( underlined rows ) are shown with arrows.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Method
# Training Size
Overall
fil
id
lo
my
vi
(Size / BLEU-chrF)
MERIT-3B
+ SFT-LTP
8,000
8,000
8,000
8,000
8,000
40,000 / 15.53
+ GRPO-SAR
1,851 ↓76.9%
1,779 ↓77.7%
2,058 ↓74.2%
2,462 ↓69.2%
976 ↓87.8%
9,126 ↓77.2% / 18.23 ↑17.4%
+ LLME
2,891 ↓63.9%
3,104 ↓61.2%
3,300 ↓58.8%
3,764 ↓53.0%
2,193 ↓72.6%
15,252 ↓61.9% / 19.94 ↑28.4%
Appendix
Table 4: Training size comparison across five low-resource languages for MERIT-3B. Overall column shows total training data (with percentage reduction relative to initial 40,000) and BLEU-chrF score (with percentage improvement relative to the SFT-LTP stage).
Figure 3: Training loss and reward evolution across SFT and GRPO strategies.
Combination
Spearman ρ↑
σ2↓
( BLEU + chrF )/2 (equal)
0.986
1.02
BLEU×chrF (geometric)
0.985
1.14
0.4⋅BLEU+0.6⋅chrF
0.985
1.19
0.6⋅BLEU+0.4⋅chrF
0.984
1.23
Appendix
Table 5: Empirical comparison of different combination methods for BLEU and chrF, evaluated on WMT22 test sets (5 languages × 16 systems). The results show that, while all variants exhibit very similar rank correlations with human judgments, the simple arithmetic mean yields the lowest system-level variance.
Metric
Mean Δ (v.s NLLB-200)
p-value
BLEU-4
+9.46
0.003
chrF2
+8.36
0.006
ROUGE-L
+1.18
0.041
METEOR
+2.65
0.038
COMET-22
+2.80
< 0.001
Appendix
Table 6: Statistical Significance of Improvements over Baseline. We conduct pairwise bootstrap resampling (1,000 iterations) to compare our final model (MERIT-3B) against the NLLB-200 baseline. All improvements are statistically significant.
Machine translation develops computational methods for converting text, speech, and visual content between languages while preserving meaning and purpose.
Multilingual neural machine translation models such as NLLB-200 cover 200 languages but leave thousands unsupported, including most Grassfields Bantu languages of Cameroon. When fine-tuning these models for an unseen language, practitioners must choose a proxy language token, yet no principled method exists for this selection. We implemented an embedding initialization strategy where a language token is the average of embeddings from multiple typologically related languages already in the mod el. We evaluate this approach on Limbum-to-English translation using a parallel corpus of 8,837 sentence pairs from New Testament text and a bilingual dictionary. We compare models: NLLB-200 zero-shot (chrF2++ = 12.5), a Transformer trained from scratch (chrF2++ = 14.5), NLLB-200 fine-tuned with a Swahili proxy token (chrF2++ = 47.3), and NLLB-200 with our averaged embedding initialization (chrF2++ = 46.7). We find that the multi-language initialization achieves performance comparable to the best single-language proxy. Both NLLB-200 variants improve over the from-scratch baseline by over 32 chrF2++ points. These results show that multilingual transfer is the dominant factor in extremely low-resource Bantu translation while eliminating the need for heuristic proxy selection. However, all systems fail to preserve tonal diacritics, highlighting an open challenge. We make our dataset and code available to support further research.
Samiratu Ntohsi, Neza David Tuyishimire, Anesu Kafesu +3
Ruzivo Research Lab, African Leadership University · African Leadership University
The landscape of extremely low-resource machine translation (MT) is characterized by perplexing variability in reported performance, often making results across different language pairs difficult to contextualize. For researchers focused on specific language groups -- such as ancient languages -- it is nearly impossible to determine if breakthroughs reported in other contexts (e.g., native African or American languages) result from superior methodologies or are merely artifacts of benchmark collection. To address this problem, we introduce the FRED Difficulty Metrics, which include the Fertility Ratio (F), Retrieval Proxy (R), Pre-training Exposure (E), and Corpus Diversity (D) and serve as dataset-intrinsic metrics to contextualize reported scores. These metrics reveal that a significant portion of result variability is explained by train-test overlap and pre-training exposure rather than model capability. Additionally, we identify that some languages -- particularly extinct and non-Latin indigenous languages -- suffer from poor tokenization coverage (high token fertility), highlighting a fundamental limitation of transferring models from high-resource languages that lack a shared vocabulary. By providing these indices alongside performance scores, we enable more transparent evaluation of cross-lingual transfer and provide a more reliable foundation for the XLR MT community.
Danlu Chen, Ka Sing He, Jiahe Tian +4
UC San Diego · New York University · Durham University +3
We present a study on low-resource machine translation for the Tangkhul-English (nmf-en) language pair. Tangkhul is a severely under-resourced Tibeto-Burman language spoken primarily in Manipur, India, with virtually no prior natural language processing infrastructure. We describe two systems: (1) a primary system based on ByT5-large fine-tuned on 38,336 Tangkhul-English parallel sentence pairs, and (2) a contrastive system based on mT5-small fine-tuned on the same corpus. Our primary ByT5-large system achieves a corpus BLEU score of 39.97, chrF++ of 58.07, BERTScore F1 of 0.8104, and COMET (wmt22-comet-da) of 0.7302 on a held-out test set of 3,856 sentences. We further discuss the orthographic challenges specific to Tangkhul's Latin-script diacritics, the domain bias of our training corpus (which comprises biblical text, stories, and conversational data), and avenues for future improvement through data diversification and domain adaptation.