Neural machine translation (NMT) from Chinese to low-resource Southeast Asian languages remains severely constrained by the extreme scarcity of clean parallel corpora and the pervasive noise in existing mined data. This chronic shortage not only impedes effective model training but also sustains a large performance gap with high-resource directions, leaving millions of speakers of languages such as Lao, Burmese, and Tagalog with persistently low-quality translation systems despite recent advances in large multilingual models. We introduce Multilingual Expert-Reward Informed Tuning (MERIT), a unified translation framework that transforms the traditional English-centric ALT benchmark into a Chinese-centric evaluation suite for five Southeast Asian low-resource languages (LRLs). Our framework combines language-specific token prefixing (LTP) with supervised fine-tuning (SFT) and a novel group relative policy optimization (GRPO) guided by the semantic alignment reward (SAR). These results confirm that, in LRL{\textrightarrow}Chinese translation, targeted data curation and reward-guided optimization dramatically outperform mere model scaling.
Figures & tables
Figure 1: Overview of the MERIT framework. The pipeline consists of (a) heuristic data selection utilizing the Elite Parallel Data Sampler (EPDS, algorithm 1 ) and Data Integrity Validation (DIV, algorithm 2 ), (b) translation scoring via a QE agent trained on expert cross-reviewed quality data using GRPO-SAR, (c) reward-based data distillation, which utilizes the QE agent to rescore the entire training set and filter for high-quality translation data, and (d) final model optimization using SFT-LTP based on this distilled dataset.
Languages
Speaker Population 1 1 1 Speaker numbers derive from the most recent national censuses or Ethnologue reports (2023–2025) and are expressed in millions (M). ( Eberhard et al., 2023 )
Filtered Subset 2 2 2 Each ALT language contains approximately 20k aligned sentence pairs from a shared English source. See https://www2.nict.go.jp/astrec-att/member/mutiyama/ALT/ .
LRL
Chinese ( zh )
1180M
✗
✗
Hindi ( hi )
345M
✗
✗
Bengali ( bn )
234M
✗
✗
Japanese ( ja )
121M
✗
✗
Vietnamese ( vi )
86M
10K
✓
Indonesian ( id )
43M
10K
✓
Table 1: The ALT corpus statistics sorted by L1 speaker population. All counts refer to L1 speakers and are rounded to the nearest million (M).
Figure 2: Performance–Scale Trade-offs of MERIT-3B and Baseline Models on Chinese-Centric Multilingual Translation. Comparison of BLEU-chrF scores against model size (log-scale) across MERIT-3B, open-source, and estimated closed-source models.
Strict-Overlap
BLEU-4
sacreBLEU
ROUGE-L
Model
fil
id
lo
my
vi
fil
id
lo
my
vi
fil
id
lo
my
vi
FT
OS
GPT-4o
45.26
47.77
28.10
30.48
45.03
43.60
45.81
27.38
29.53
44.50
33.62
31.40
12.33
23.57
28.25
✗
✗
Claude-3.5 Sonnet
42.97
47.35
32.20
30.92
45.14
43.38
46.54
32.65
30.14
44.55
35.03
31.17
14.00
24.44
28.45
✗
✗
Gemini-2.5 Flash
49.26
48.73
35.79
36.91
39.24
47.48
47.20
35.03
36.07
38.63
35.89
30.83
13.53
23.92
24.62
✗
✗
DeepSeek-V3
46.19
41.83
25.57
26.68
41.29
44.12
41.38
25.25
26.29
40.90
35.15
30.06
12.75
22.95
28.27
✗
✓
Qwen-2.5 32B
43.56
47.87
23.27
20.43
46.23
42.01
46.61
22.48
19.50
44.63
34.97
31.85
11.84
20.61
29.08
✗
✓
Table 2: Evaluation on five Southeast Asian languages. Strict-Overlap metrics include BLEU-4, sacreBLEU, and ROUGE-L. Semantic-Friendly metrics include chrF, METEOR, and BERTScore. For each metric column: Bold values indicate the highest score, and Underlined values indicate the second highest score across all models. FT: Fine-tuned; OS: Open Source.
Model
BLEU-4
chrF
Overall (BLEU-chrF)
fil
id
lo
my
vi
fil
id
lo
my
vi
Qwen2.5-0.5b
0.03
0.03
0.02
0.01
0.01
0.16
0.12
0.40
0.25
0.06
0.11
+ SFT-LTP
1.86 ↑1.83
4.02 ↑4.00
0.25 ↑0.22
0.15 ↑0.14
3.12 ↑3.11
4.85 ↑4.69
10.38 ↑10.26
1.41 ↑1.00
1.07 ↑0.82
8.55 ↑8.50
3.57 ↑3.46
+ GRPO-SAR
2.31 ↑2.28
4.32 ↑4.29
0.26 ↑0.24
0.16 ↑0.16
6.29 ↑6.27
5.00 ↑4.84
10.64 ↑10.52
1.38 ↑0.98
1.22 ↑0.96
12.36 ↑12.30
4.39 ↑4.28
+ LLME
0.32 ↑0.29
0.45 ↑0.42
0.08 ↑0.06
0.07 ↑0.06
0.79 ↑0.78
1.20 ↑1.04
1.66 ↑1.54
0.45 ↑0.05
1.25 ↑1.00
2.82 ↑2.76
0.91 ↑0.80
Avg.
1.13
2.21
0.15
0.10
2.55
2.80
5.70
0.91
0.95
5.95
2.25
Table 3: Evaluation results of Qwen-2.5-0.5B and 3B on five Southeast Asian languages. All values are rounded to two decimal places. Improvements over the baseline ( underlined rows ) are shown with arrows.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Method
# Training Size
Overall
fil
id
lo
my
vi
(Size / BLEU-chrF)
MERIT-3B
+ SFT-LTP
8,000
8,000
8,000
8,000
8,000
40,000 / 15.53
+ GRPO-SAR
1,851 ↓76.9%
1,779 ↓77.7%
2,058 ↓74.2%
2,462 ↓69.2%
976 ↓87.8%
9,126 ↓77.2% / 18.23 ↑17.4%
+ LLME
2,891 ↓63.9%
3,104 ↓61.2%
3,300 ↓58.8%
3,764 ↓53.0%
2,193 ↓72.6%
15,252 ↓61.9% / 19.94 ↑28.4%
Appendix
Table 4: Training size comparison across five low-resource languages for MERIT-3B. Overall column shows total training data (with percentage reduction relative to initial 40,000) and BLEU-chrF score (with percentage improvement relative to the SFT-LTP stage).
Figure 3: Training loss and reward evolution across SFT and GRPO strategies.
Combination
Spearman ρ↑
σ2↓
( BLEU + chrF )/2 (equal)
0.986
1.02
BLEU×chrF (geometric)
0.985
1.14
0.4⋅BLEU+0.6⋅chrF
0.985
1.19
0.6⋅BLEU+0.4⋅chrF
0.984
1.23
Appendix
Table 5: Empirical comparison of different combination methods for BLEU and chrF, evaluated on WMT22 test sets (5 languages × 16 systems). The results show that, while all variants exhibit very similar rank correlations with human judgments, the simple arithmetic mean yields the lowest system-level variance.
Metric
Mean Δ (v.s NLLB-200)
p-value
BLEU-4
+9.46
0.003
chrF2
+8.36
0.006
ROUGE-L
+1.18
0.041
METEOR
+2.65
0.038
COMET-22
+2.80
< 0.001
Appendix
Table 6: Statistical Significance of Improvements over Baseline. We conduct pairwise bootstrap resampling (1,000 iterations) to compare our final model (MERIT-3B) against the NLLB-200 baseline. All improvements are statistically significant.
Machine translation develops computational methods for converting text, speech, and visual content between languages while preserving meaning and purpose.