Ensemble Self-Training for Unsupervised Machine Translation

Ido Aharon; Jonathan Shaki; Sarit Kraus

cs.CLMar 17, 2026

Ensemble Self-Training for Unsupervised Machine Translation

Authors: Ido Aharon, Jonathan Shaki, Sarit Kraus

Organizations: Bar-Ilan University

Abstract

We present an ensemble-driven self-training framework for unsupervised neural machine translation (UNMT). Starting from a primary language pair, we train multiple UNMT models that share the same translation task but differ in an auxiliary language, inducing structured diversity across models. We then generate pseudo-translations for the primary pair using token-level ensemble decoding, averaging model predictions in both directions. These ensemble outputs are used as synthetic parallel data to further train each model, allowing the models to improve via shared supervision. At deployment time, we select a single model by validation performance, preserving single-model inference cost. Experiments show statistically significant improvements over single-model UNMT baselines, with mean gains of 1.7 chrF when translating from English and 0.67 chrF when translating into English.

Figures & tables

Model	Direction	Average Score
Llama-3.2-1B (Low-resource)	EN $\to$ PL	16.60
Llama-3.2-1B (Low-resource)	PL $\to$ EN	20.03
Qwen3-0.6B-Base (High-resource)	EN $\to$ FR	51.29
Qwen3-0.6B-Base (High-resource)	FR $\to$ EN	53.20
Qwen3-1.7B-Base (High-resource)	EN $\to$ FR	55.79
Qwen3-1.7B-Base (High-resource)	FR $\to$ EN	56.95

Table 1: Average chrF score of individual models for the main language pair, on 3000 sentences.

Figure 1: 100 sentences BLEU for Llama-3.2-1B : mean performance across all single models.

Model	Ensemble Size	Direction	Best Model Before +Single Training	Best Model After Ensemble Training
Llama-1B (low- resource)	3	EN $\to$ PL	18.93	21.44
	3	PL $\to$ EN	22.80	23.47
	6	EN $\to$ PL	18.93	21.01
	6	PL $\to$ EN	22.80	23.79
	10	EN $\to$ PL	18.93	21.99
	10	PL $\to$ EN	22.80	24.25

Table 2: Best individual model chrF score.

Model Name	Ensemble Size	Direction	Best Model Before	Best Model After	Avg. Models Before	Avg. Models After	Ensemble Before	Ensemble After
Llama-3.2-1B	3	EN $\to$ PL	17.57 (cs)	21.44 (cs)	17.30	21.27	18.51	21.09
	3	PL $\to$ EN	22.29 (bg)	23.47 (el)	20.62	23.36	20.41	23.54
	6	EN $\to$ PL	17.56 (cs)	21.01 (cs)	16.97	20.86	16.77	20.45
	6	PL $\to$ EN	22.17 (bg)	23.79 (cs)	19.79	23.56	20.35	23.55
	10	EN $\to$ PL	17.70 (cs)	21.99 (bg)	16.60	21.45	17.18	20.97
	10	PL $\to$ EN	22.35 (bg)	24.25 (sk)	20.03	23.85	20.53	23.91

Table 3: chrF score for the ensemble setting per model and ensemble size.

Figure 2: Llama-3.2-1B performance during ensemble training (10 models).

Figure 3: Qwen3-0.6B-Base performance during ensemble training (10 models).

Figure 4: Qwen3-1.7B-Base performance during ensemble training (10 models).

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Version (3rd lang)	PL->EN	EN->PL	3rd->EN	EN->3rd
BG	5.55	0.98	3.39	2.26
CS	4.76	1.73	2.62	1.82
EL	4.03	1.11	1.11	0.80
ET	4.17	1.30	2.57	1.52
FI	4.00	1.26	1.07	0.47
HU	4.53	1.17	3.43	2.50

Appendix

Table 4: Final single-model Llama-3.2-1B 3000 sentence BLEU by direction (low-resource).

Version (3rd lang)	FR->EN	EN->FR	3rd->EN	EN->3rd
BG	26.17	22.13	29.99	20.90
CS	25.78	21.58	25.12	14.47
DA	25.67	21.39	21.88	15.61
LV	25.42	21.16	7.72	5.15
NL	25.45	21.62	20.13	14.76
PL	25.92	22.09	23.91	13.51

Appendix

Table 5: Final single-model Qwen3-0.6B 3000 sentence BLEU by direction (high-resource).

Version (3rd lang)	FR->EN	EN->FR	3rd->EN	EN->3rd
BG	30.29	26.62	41.31	31.47
CS	30.26	26.33	36.53	24.72
DA	29.74	26.06	29.45	21.95
DE	30.13	25.95	29.59	18.70
ES	30.08	26.27	32.32	30.57
LV	30.44	26.12	29.44	16.90

Appendix

Table 6: Final single-model Qwen3-1.7B 3000 sentence BLEU by direction (high-resource).

Figure 5: 100 sentences BLEU for Qwen3-0.6B-Base : mean performance across all single models.

Figure 6: 100 sentences BLEU for Qwen3-1.7B-Base : mean performance across all single models.

Table 13

Model Name	Ensemble Size	Direction	Best Model Before	Best Model After	Avg. Models Before	Avg. Models After	Ensemble Before	Ensemble After
Llama-3.2-1B	3	EN $\to$ PL	1.70 (cs)	4.38 (cs)	1.24	4.15	1.87	4.03
	3	PL $\to$ EN	5.57 (bg)	6.24 (cs)	4.79	5.85	5.00	5.61
	6	EN $\to$ PL	1.75 (cs)	4.23 (et)	1.24	4.19	1.55	4.17
	6	PL $\to$ EN	5.51 (bg)	6.33 (hu)	4.50	6.23	5.07	6.26
	10	EN $\to$ PL	1.73 (cs)	4.54 (et)	1.22	4.36	1.69	4.36
	10	PL $\to$ EN	5.61 (bg)	6.70 (hu)	4.69	6.46	4.87	6.53

Appendix

Table 9: BLEU score for the ensemble setting per model and ensemble size.

Model Name	Ensemble Size	Direction	Best Model Before	Best Model After	Avg. Models Before	Avg. Models After	Ensemble Before	Ensemble After
Llama-3.2-1B	3	EN $\to$ PL	0.1806 (cs)	0.2206 (cs)	0.1707	0.2196	0.1831	0.2218
	3	PL $\to$ EN	0.1937 (bg)	0.1982 (cs)	0.1779	0.1939	0.1788	0.1943
	6	EN $\to$ PL	0.1812 (cs)	0.2151 (el)	0.1647	0.2097	0.1747	0.2098
	6	PL $\to$ EN	0.1924 (bg)	0.2078 (cs)	0.1697	0.2034	0.1819	0.2021
	10	EN $\to$ PL	0.1815 (cs)	0.2376 (hu)	0.1617	0.2280	0.1759	0.2257
	10	PL $\to$ EN	0.1926 (bg)	0.2092 (et)	0.1723	0.2005	0.1801	0.1988

Appendix

Table 10: COMET score for the ensemble setting per model and ensemble size.

Machine Translation

Machine translation develops computational methods for converting text, speech, and visual content between languages while preserving meaning and purpose.

See all 267 papers

Backtranslation Augmented Direct Preference Optimization for Neural Machine Translation
Apr 28, 2026Mehrdad Ghassabi, Spehr Rajabi, Hamidreza Baradaran Kashani +3Back-Translation Language Model Post-Training
Faculty of Computer Engineering University of Isfahan Isfahan, Iran · School of Computer Science University of Windsor Windsor, Canada
Omnilingual MT: Machine Translation for 1,600 Languages
Mar 17, 2026Omnilingual MT Team, Belen Alastruey, Niyati Bafna +29Machine Translation Low-Resource MT
FAIR at Meta
LatentMT: Machine Translation with Latent Reasoning
Jul 21, 2026Wei-Rui Chen, Samar M. Magdy, Chiyu Zhang +3Low-Resource MT Efficient Language Model Inference
The University of British Columbia · Arizona State University · Rice University +1

Apr 28, 2026J/K move · Enter open · S save

Backtranslation Augmented Direct Preference Optimization for Neural Machine Translation

Mehrdad Ghassabi, Spehr Rajabi, Hamidreza Baradaran Kashani, Sadra Hakim+2

Faculty of Computer Engineering University of Isfahan Isfahan, Iran · School of Computer Science University of Windsor Windsor, Canada

Back-Translation Language Model Post-Training Machine Translation

Open paper AskPDF arXiv

Contemporary neural machine translation (NMT) systems are almost exclusively built by training on supervised parallel data. Despite the tremendous progress achieved, these systems still exhibit persistent translation errors. This paper proposes that a post-training paradigm based on reinforcement learning (RL) can effectively rectify such mistakes. We introduce a novel framework that requires only a general text corpus and an expert translator which can be either human or an AI system to provide iterative feedback. In our experiments, we focus specifically on English-to-German translation as a representative high-resource language pair. Crucially, we implement this RL-based post-training using Direct Preference Optimization (DPO). Applying our DPO-driven framework to the gemma3-1b model yields a significant improvement in translation quality, elevating it's COMET score from 0.703 to 0.747 on the English to German task. The results demonstrate that DPO offers an efficient and stable pathway for enhancing pre-trained NMT models through preference-based post-training.

Abstract

Machine Translation

Explore similar work

Backtranslation Augmented Direct Preference Optimization for Neural Machine Translation

Omnilingual MT: Machine Translation for 1,600 Languages

LatentMT: Machine Translation with Latent Reasoning