Open Machine Translation for Esperanto

Ona de Gibert; Lluís de Gibert

cs.CLMar 31, 2026

Open Machine Translation for Esperanto

Authors: Ona de Gibert, Lluís de Gibert

Organizations: University of Helsinki, Department of Digital Humanities · Kataluna Esperanto-Asocio (KEA) · Sennacieca Asocio Tutmonda (SAT)

Abstract

Esperanto is a widespread constructed language, known for its regular grammar and productive word formation. Besides having substantial resources available thanks to its online community, it remains relatively underexplored in the context of modern machine translation (MT) approaches. In this work, we present the first comprehensive evaluation of open-source MT systems for Esperanto, comparing rule-based systems, encoder-decoder models, and LLMs across model sizes. We evaluate translation quality across six language directions involving English, Spanish, Catalan, and Esperanto using multiple automatic metrics as well as human evaluation. Our results show that the NLLB family achieves the best performance in all language pairs, followed closely by our trained compact models and a fine-tuned general-purpose LLM. Human evaluation confirms this trend, with NLLB translations preferred in approximately half of the comparisons, although noticeable errors remain. In line with Esperanto's tradition of openness and international collaboration, we release our code and best-performing models publicly.

Figures & tables

Figure 1: Cumulative number of ACL Anthology papers mentioning ’Esperanto’ and ’Esperanto and Machine Translation’. Early work involving Esperanto focused primarily on MT, while more recent work covers a broader range of topics.

Language Pair	Raw	Filtered	% Removed	Marian	LLM FT	Dev	Test
	Training Data			Used for Training		FLORES+
en–eo	70.34M	45.39M	35.5%	5.0M	100k	997	1012
es–eo	6.21M	4.68M	24.7%	4.7M	100k	997	1012
ca–eo	1.17M	673.93k	42.7%	673k	100k	997	1012
Total	77.73M	50.74M		10.35M	300k	2991

Table 1: Parallel data statistics before and after filtering, and final training sizes per model family.

Panel of Figure 2 — Figure 2: Prompt example for fine-tuning Llama-3.1-8B-Instruct

Panel of Figure 3 — Figure 3: Human evaluation win rates for both translation directions.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Hyperparameter	Value
Optimizer	Adam
Adam $\beta$	(0.9, 0.98)
Adam $\epsilon$	1e-9
Learning rate	3e-4
LR warmup	16000 steps
LR decay strategy	epoch+stalled

Appendix

Table 5: Hyperparameters for Transformer-base

Hyperparameter	Value
Optimizer	Adam
Adam $\beta$	(0.9, 0.98)
Adam $\epsilon$	1e-9
Learning rate	3e-4
LR warmup	16000 steps
LR decay strategy	epoch+stalled

Appendix

Table 6: Hyperparameters for Transfomer-tiny

Hyperparameter	Value
Learning Rate	5e-05
LR Scheduler Type	Linear
Warmup Ratio	0.3
Weight Decay	0.0
Per Device Train Batch Size	4
Gradient Accumulation Steps	4

Appendix

Table 7: Hyperparameters for Llama fine-tuning

Hyperparameter	Value
Optimizer	Adafactor
Learning Rate	1e-4
LR Scheduler Type	Constant
LR Warmup	1000 steps
Weight Decay	1e-3
Per Device Train Batch Size	32

Appendix

Table 8: Hyperparameters for NLLB fine-tuning

Transformer-base
Error Type	Source	Translation
Lexical	La epidemio igis la baratan registaron entrepreni tiajn rimedojn, kiel instalado de porkokaptiloj en la serioze damaĝitaj areoj, disdonado de miloj da kontraŭmoskitaj retoj kaj ŝprucado de pesticidoj.	La epidemia llevó al gobierno indio a emprender tales medios, como la instalación de trampas de cerdo en las áreas seriamente dañadas, la distribución de miles de redes anti-moscoquios y el brote de pesticidas.
Grammatical	Malgraŭ foresto de cunama minaco, loĝantoj ekpanikis kaj komencis forlasadi siajn komercojn kaj hejmojn.	A pesar de la ausencia de la amenaza tsunami, los residentes comenzaron a pánico y comenzaron a dejar sus negocios y casas.
Untranslated	Gregon konsistigas de unu ĝis tri parencaj plenkreskaj maskloj kaj multaj ĝis tridek inoj kaj idoj.	Gregon consiste de uno a tres varones adultos relacionados y muchos a treinta hembras y crías.
NLLB-200-3.3B
Mistranslation	Nekredantoj vivantaj en kristanaj landoj aŭ aliloke en la tuta mondo adoptis multajn el la tradicioj pri ĉi tiu festo.	Los incrédulos que viven en países cristianos o en otras partes del mundo han adoptado muchas de las tradiciones de esta fiesta.

Appendix

Table 12: Illustrative examples of error categories identified in the qualitative analysis for Esperanto into Spanish. Highlighted spans mark erroneous content.

Transformer-base
Error Type	Source	Translation

Grammatical	En verdad, el formato de 35 mm es algo confuso , ya que sus medidas son 36 mm de anchura por 24 mm de alto.	Fakte, la 35mm formato estas iom konfuzita , ĉar ĝiaj mezuroj estas 36 mm larĝa de 24 mm alta.

Mistranslation	Conforme surge de las imágenes infrarrojas, las variaciones de temperatura durante la noche y el día indican que, probablemente, se trate de cuevas.	Ĉar ĝi ekestiĝas de infraruĝaj bildoj, temperaturkatastrofoj dum la nokto kaj tago indikas ke ili estas verŝajne kavernoj.
Named Entities	El día de hoy se originó en el Océano Atlántico la tormenta número diez de su temporada de huracanes en ser nombrada, a la que se denominó Tormenta subtropical Jerry.	La atlantika ŝtormo numero dek el sia uragansezono estis nomita Jerry Storm.

Appendix

Table 13: Illustrative examples of error categories identified in the qualitative analysis for Spanish into Esperanto. Highlighted spans mark erroneous content.

Machine Translation

Machine translation develops computational methods for converting text, speech, and visual content between languages while preserving meaning and purpose.

See all 267 papers

Omnilingual MT: Machine Translation for 1,600 Languages
Mar 17, 2026Omnilingual MT Team, Belen Alastruey, Niyati Bafna +29Machine Translation Low-Resource MT
FAIR at Meta
Precision over Scale: A Polish-Silesian Benchmark and a Translation System Outperforming Open-Source and Commercial Models
Oct 1, 2026Grzegorz Kulik, Mikołaj Pokrywka, Adam Jatowt +1Low-Resource MT Machine Translation
NASK National Research Institute, Poland · University of Innsbruck, Austria
Beyond Prompts: A Systematic Study of LLM-Based Machine Translation Evaluators
Oct 24, 2026Dmitry Popov, Roman Bokhyan, Ekaterina Enikeeva +3MT Evaluation LLM-as-a-Judge
Yandex · HSE University

Mar 17, 2026J/K move · Enter open · S save

Omnilingual MT: Machine Translation for 1,600 Languages

Omnilingual MT Team, Belen Alastruey, Niyati Bafna, Andrea Caciolai+28

FAIR at Meta

Machine Translation Low-Resource MT Cross-Lingual Transfer MT Evaluation

Open paper AskPDF arXiv

High-quality machine translation (MT) can scale to hundreds of languages, setting a high bar for multilingual systems. However, compared to the world's 7,000 languages, current systems still offer only limited coverage: about 200 languages on the target side, and maybe a few hundreds more on the source side, supported due to cross-lingual transfer. And even these numbers have been hard to evaluate due to the lack of reliable benchmarks and metrics. We present Omnilingual Machine Translation (OMT), the first MT system supporting more than 1,600 languages. This scale is enabled by a comprehensive data strategy that integrates large public multilingual corpora with newly created datasets, including manually curated MeDLEY bitext. We explore two ways of specializing a Large Language model (LLM) for machine translation: as a decoder-only model (OMT-LLaMA) or as a module in an encoder-decoder architecture (OMT-NLLB). Notably, all our 1B to 8B parameter models match or exceed the MT performance of a 70B LLM baseline, revealing a clear specialization advantage and enabling strong translation quality in low-compute settings. Moreover, our evaluation of English-to-1,600 translations further shows that while baseline models can interpret undersupported languages, they frequently fail to generate them with meaningful fidelity; OMT-LLaMA models substantially expand the set of languages for which coherent generation is feasible. Additionally, OMT models improve in cross-lingual transfer, being close to solving the "understanding" part of the puzzle in MT for the 1,600 evaluated. Our leaderboard and main human-created evaluation datasets (BOUQuET and Met-BOUQuET) are dynamically evolving towards Omnilinguality and freely available.

Abstract

Machine Translation

Explore similar work

Omnilingual MT: Machine Translation for 1,600 Languages

Precision over Scale: A Polish-Silesian Benchmark and a Translation System Outperforming Open-Source and Commercial Models

Beyond Prompts: A Systematic Study of LLM-Based Machine Translation Evaluators