Mobilizing the Archive: Bitext Extraction at Scale from Archival Documents for Extremely Low-resource Languages

Chutong Meng; Alec J. Bateman; Antonios Anastasopoulos

cs.CLOct 29, 2026

Mobilizing the Archive: Bitext Extraction at Scale from Archival Documents for Extremely Low-resource Languages

Authors: Chutong Meng, Alec J. Bateman, Antonios Anastasopoulos

Organizations: George Mason University · Barron Associates

Abstract

While mining parallel sentences from web pages has proven effective for low-resource machine translation, the majority of the world's languages have little to no web presence. What they do have, though, are resources that are the result of centuries of efforts by linguists, such as grammars, dictionaries, and other documentation materials. We propose extracting translation examples from such linguistic resources, which offer parallel data for languages under-represented online and could potentially scale to thousands more languages. We develop a pipeline to achieve this, leveraging an Optical Character Recognition (OCR) model as well as a vision language model (VLM), to ensure identified content to be aligned with visual layout information. We demonstrate the utility of the extracted bitexts for bootstrapping translation systems across 20 languages, both by training machine translation models and by applying in-context learning for large language models. For in-context learning, we observe improvements of +15.78 and +6.94 chrF2++ over zero-shot translation for into low- and high-resource languages, respectively.

Figures & tables

Figure 1: We illustrate the rough relationship between mined bitext size and MT performance, using nllb-200-3.3B as an example. Archival linguistic resources could extend bitext coverage from around 200 languages to thousands more, while substantially improving few-shot translation quality toward NLLB-level performance.

Figure 2: Examples of incidental bitext in a Kalamang book Visser (2016) . Blue boxes indicate Kalamang sentences, and green boxes indicate English sentences.

LRL name	LRL code	HRL	NLLB support	Existing data
LRL name	LRL code	HRL	NLLB support	Data source	Script
Arapaho	arp	eng	No	SIGMORPHON 2023	Latin
Chuvash	chv	eng	No	FLORES, BOUQuET	Cyrillic
Tsez	ddo	eng	No	SIGMORPHON 2023	Latin
Dogri	dgo	eng	No	FLORES, BOUQuET	Devanagari
Gitxsan	git	eng	No	SIGMORPHON 2023	Latin

Table 1: Summary of languages and their resource status, separated by NLLB support. Kimbundu is paired with Portuguese, for which our collection contains substantially more books, while Uspanteco is paired with Spanish to match its available test set. BOUQuET is only used as a secondary evaluation set where available.

Model	CER (%) $\downarrow$
PaddleOCR-VL	34.13
HunyuanOCR	11.37
dots.ocr	7.88
dots.ocr with retry	5.66

Table 2: CER for three OCR models on the synthesized FLORES PDFs.

Lang	BLASER 2.0-QE $\uparrow$
Lang	Extracted	Test / FLORES
dik	2.935	3.281
gug	3.375	3.732
ilo	3.356	4.274
kac	3.129	3.570
kea	3.293	3.965

Table 3: QE scores for supported languages on extracted and FLORES bitexts. Bold indicates better performance. FLORES bitexts are of higher quality.

Without NLLB support
Lang	#Extracted	Existing Datasets
Lang	#Extracted	#Train	#Dev	#Test
arp	1,796	36,123	4,892	4,892
chv	454	–	997 / 504	1,012 / 854
ddo	1,403	3,547	445	445
dgo	137	–	498 / 504	499 / 854

Table 4: Number of sentences in the extracted and existing datasets. For FLORES, the devtest set is used as the test set. In cells with two numbers, the left one corresponds to FLORES and the right one to BOUQuET.

Without NLLB support
Lang	Qwen		nllb
Lang	0 shot	512 shot	OOB	fine-tuned
arp	3.86	* 32.90 \upbox $\uparrow$ 29.04	–	9.96
chv	20.99	* 30.49 \upbox $\uparrow$ 9.50	–	5.14
ddo	0.17	* 24.24 \upbox $\uparrow$ 24.07	–	1.93
dgo	21.18	* 20.68 \downbox $\downarrow$ 0.50	–	4.12

Table 5: Extracted data improves few-shot translation performance into LRLs . Results are reported in chrF2++(

\uparrow

). OOB denotes out-of-the-box performance. *

\text{p-value}<0.05

Without NLLB support †
Lang	Qwen		nllb
Lang	0 shot	512 shot	OOB	fine-tuned
arp	11.67	* 22.09 \upbox $\uparrow$ 10.42	8.62	* 12.59 \upbox $\uparrow$ 3.97
chv	47.46	* 48.66 \upbox $\uparrow$ 1.20	19.72	* 24.47 \upbox $\uparrow$ 4.75
ddo	14.76	* 27.64 \upbox $\uparrow$ 12.88	10.27	* 16.55 \upbox $\uparrow$ 6.28
dgo	47.52	* 50.50 \upbox $\uparrow$ 2.98	42.86	* 39.53 \downbox $\downarrow$ 3.33

Table 6: Extracted data improves few-shot translation performance into HRLs . Results are reported in chrF2++(

\uparrow

). † Proxy source language codes are used. *

\text{p-value}<0.05

Lang	LASER2	LABSE	SONAR
Lang	xsim $\downarrow$	xsim $\downarrow$	lang code	xsim $\downarrow$
Without NLLB support †
arp	96.89	94.60	eng_Latn	95.34
chv	76.19	33.70	tat_Cyrl	33.40
ddo	97.08	90.34	eng_Latn	90.79
dgo	40.48	0.80	hin_Deva	0.20

Table 7: xSIM error rate (%) on test sets. SONAR achieves the lowest error rate. † Proxy SONAR language codes are used.

Methods without prior knowledge
Alignment Method	Precision	Recall	F1
Global Mining	0.0610	0.0549	0.0578
Local Mining	0.0851	0.2473	0.1266
Vecalign	0.2776	0.4560	0.3451
Hunalign w/o dict.	0.2435	0.2582	0.2507
VLM (our method)	1.0000	0.7657	0.8673

Table 8: Alignment performance of different methods. VLM achieves the best performance. Its imperfect recall is due to pages dominated by word-level alignments, where sentence-level alignments are overlooked.

Without NLLB support
Lang	Into LRL		Into HRL
Lang	Gold	Book	Gold	Book
arp	48.32	* 32.90 \downbox $\downarrow$ 15.42	39.30	* 22.09 \downbox $\downarrow$ 17.21
chv	33.77	* 30.49 \downbox $\downarrow$ 3.28	49.96	* 48.66 \downbox $\downarrow$ 1.30
ddo	43.59	* 24.24 \downbox $\downarrow$ 19.35	42.38	* 27.64 \downbox $\downarrow$ 14.74
dgo	34.52	* 20.68 \downbox $\downarrow$ 13.84	54.62	* 50.50 \downbox $\downarrow$ 4.12

Table 9: chrF2++ scores for 512-shot translation with Qwen . Extracted bitexts are narrowing the gap to gold in-domain bitexts. † Excluded from the average as it has only 3 gold examples. *

\text{p-value}<0.05

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: Script-level CER for dots.ocr , HunyuanOCR , and PaddleOCR-VL . CER values exceeding 100% indicate that the corresponding script is not supported by the model.

Lang	Books	Existing Data	Normalization
arp	use diacritics (e.g., é, í, ó)	no diacritics	remove diacritics
chv	Latin script	Cyrillic script	script conversion
ddo	noisy OCR (e.g., ā, ç, ġ)	diacritics (e.g., ä, č, š)	correct OCR errors
dgo	IPA	Devanagari script	Codex-proposed conversion rules
dik	diacritics and noisy OCR (e.g., à, ç, ē)	diacritics (e.g., í, ó, ú)	correct OCR errors
git	use diacritics	no diacritics; use U+0332	remove diacritics

Appendix

Table 10: Orthographic differences between the archival books and the existing dataset, along with the normalization applied to reconcile them. Differences caused by OCR errors are also included.

Into-LRL translation
Lang	Standard Norm.	Further Norm.
ddo	24.24	25.86 \upbox $\uparrow$ 1.62
dik	16.89	19.92 \upbox $\uparrow$ 3.03
git	23.16	25.14 \upbox $\uparrow$ 1.98
gug	26.05	24.63 \downbox $\downarrow$ 1.42
kmb	16.19	16.52 \upbox $\uparrow$ 0.33

Appendix

Table 11: 512 shot chrF2++ scores with Qwen . Standard Norm. applies only the normalization steps in Section 4.3 to the extracted bitext, while Further Norm. additionally removes diacritics from both the extracted and test sentences.

Lang	BLASER 2.0-QE $\uparrow$
Lang	Extracted	Test
arp	2.798	2.755
chv	2.804	2.792
ddo	2.721	2.466
dgo	2.851	3.683
git	2.867	2.820

Appendix

Table 12: BLASER 2.0-QE scores for unsupported languages.

Without NLLB support
Lang	Into LRL		Into HRL
Lang	0 shot	8 shot	0 shot	8 shot
arp	5.02	* 23.39 \upbox $\uparrow$ 18.37	11.88	* 16.72 \upbox $\uparrow$ 4.84
chv	15.56	* 18.29 \upbox $\uparrow$ 2.73	33.42	* 33.94 \upbox $\uparrow$ 0.52
ddo	0.60	* 13.46 \upbox $\uparrow$ 12.86	16.19	* 19.56 \upbox $\uparrow$ 3.37
dgo	13.08	* 16.59 \upbox $\uparrow$ 3.51	36.84	36.60 \downbox $\downarrow$ 0.24

Appendix

Table 13: chrF2++ (

\uparrow

) scores for few-shot MT using Llama-3.1-8B . *

\text{p-value}<0.05

Figure 7: Few-shot MT experiments on arp dev set with varying numbers of parallel examples. Performance of Qwen3.6-27B continues to improve as more examples are provided, whereas Llama-3.1-8B-instruct plateaus quickly.

Without NLLB support
Lang	Qwen		nllb
Lang	0 shot	512 shot	OOB	fine-tuned
arp	0.14	* 16.73 \upbox $\uparrow$ 16.59	–	1.00
chv	6.69	* 20.85 \upbox $\uparrow$ 14.16	–	0.42
ddo	0.00	* 11.83 \upbox $\uparrow$ 11.83	–	0.08
dgo	4.21	4.10 \downbox $\downarrow$ 0.11	–	0.24

Appendix

Table 14: spBLEU scores for translation into LRL . *

\text{p-value}<0.05

Without NLLB support †
Lang	Qwen		nllb
Lang	0 shot	512 shot	OOB	fine-tuned
arp	2.37	* 7.98 \upbox $\uparrow$ 5.61	0.67	* 1.57 \upbox $\uparrow$ 0.90
chv	25.00	* 26.23 \upbox $\uparrow$ 1.23	2.76	* 5.10 \upbox $\uparrow$ 2.34
ddo	0.31	* 8.35 \upbox $\uparrow$ 8.04	0.20	* 1.47 \upbox $\uparrow$ 1.27
dgo	24.53	* 27.76 \upbox $\uparrow$ 3.23	19.39	* 14.09 \downbox $\downarrow$ 5.30

Appendix

Table 15: spBLEU scores for translation into HRL . † Proxy source language codes are used. *

\text{p-value}<0.05

Without NLLB support
Lang	Into LRL			Into HRL
	0 shot	512 shot		0 shot	512 shot
	0 shot	Book	Gold	0 shot	Book	Gold
chv	18.43	* 29.03 \upbox $\uparrow$ 10.60	* 36.30 \upbox $\uparrow$ 7.27	49.41	* 50.76 \upbox $\uparrow$ 1.35	* 52.45 \upbox $\uparrow$ 1.69
dgo	21.79	* 21.44 \downbox $\downarrow$ 0.35	* 41.62 \upbox $\uparrow$ 20.18	49.34	* 52.11 \upbox $\uparrow$ 2.77	* 61.37 \upbox $\uparrow$ 9.26
With NLLB support

Appendix

Table 16: chrF2++ (

\uparrow

) results for Qwen3.6-27B on the BOUQuET test set. 0-shot performance serves as the baseline for comparison with 512-shot using book examples. The 512-shot setting with gold in-domain examples serves as the topline for comparison with 512-shot prompting using book examples. *

\text{p-value}<0.05

. On the top is a visualization of the results.

Lang	Into Creole		Into English
Lang	0 shot	512 shot	0 shot	512 shot
aoa	9.64	* 23.35 \upbox $\uparrow$ 13.71	10.90	* 23.62 \upbox $\uparrow$ 12.72
bah	11.77	* 7.61 \downbox $\downarrow$ 4.16	51.31	51.89 \upbox $\uparrow$ 0.58
brc	10.52	* 39.29 \upbox $\uparrow$ 28.77	15.23	* 44.71 \upbox $\uparrow$ 29.48
bzj	25.12	* 28.25 \upbox $\uparrow$ 3.13	38.73	* 48.14 \upbox $\uparrow$ 9.41
bzk	13.10	* 36.45 \upbox $\uparrow$ 23.35	57.61	* 62.41 \upbox $\uparrow$ 4.80

Appendix

Table 17: chrF2++ scores with Qwen on the Kreyòl-MT public test set. *

\text{p-value}<0.05

Low-Resource Language Processing

Low-resource language processing builds and evaluates language technologies for languages with limited data, tools, or digital representation.

See all 567 papers

Back-Translation Still Matters: Leveraging Monolingual Data for Low-Resource Machine Translation with LLMs
Oct 13, 2026Divyanshu Bathla, Parag Singla, Jitin SinglaBack-Translation Low-Resource MT
IIT Roorkee · IIT Delhi
Can Linguistically Related Languages Guide LLM Translation in Low-Resource Settings?
Mar 17, 2026Aishwarya Ramasethu, Niyathi Allu, Rohin Garg +2Machine Translation Low-Resource MT
Prediction Guard · Independent · Scale AI +1
An Empirical Study of Many-Shot In-Context Learning for Machine Translation of Low-Resource Languages
Apr 3, 2026Yinhan Lu, Gaganpreet Jhajj, Chen Zhang +2In-Context Learning Low-Resource MT
Mila – Quebec AI Institute · McGill University · Athabasca University +2

Oct 13, 2026J/K move · Enter open · S save

Back-Translation Still Matters: Leveraging Monolingual Data for Low-Resource Machine Translation with LLMs

Divyanshu Bathla, Parag Singla, Jitin Singla

IIT Roorkee · IIT Delhi

Back-Translation Low-Resource MT Machine Translation LLM Fine-Tuning Self-Training for Language Models

Open paper AskPDF arXiv

Pre-trained Large Language Model (LLMs) are effective on many tasks in zero-shot and few-shot settings. For machine translation of low-resource languages, the obvious options are few-shot prompting and fine-tuning on a small parallel set. We ask: what if moderate amounts of low-resource monolingual data are also available? We argue that, rather than continued pretraining, a more effective use of this data is the pre-LLM era technique of back-translation (BT). The current fine-tuned LLM translates monolingual sentences in each language, and the resulting pairs train it in the reverse direction. Updates on the parallel set are interleaved for stable learning. With 1K parallel pairs on Odia and Swahili, BT beats continued pretraining into the low-resource language by 7.7 to 16.4 XCOMET-XXL points in domain. With beam-decoded intermediate translations, it also beats a model with 10 time more parallel data. In-context examples retrieved from the parallel data further improve BT. We also analyze the source, quality and amount of monolingual text. Across five low-resource languages, with Hindi as a high-resource control, and two 3B backbones, BT improves translation into the low-resource language by 16 to 173% XCOMET-XXL over parallel-only fine-tuning. Blind human evaluation on Odia and Sanskrit agrees with the automatic metrics.

Abstract

Low-Resource Language Processing

Explore similar work

Back-Translation Still Matters: Leveraging Monolingual Data for Low-Resource Machine Translation with LLMs

Can Linguistically Related Languages Guide LLM Translation in Low-Resource Settings?

An Empirical Study of Many-Shot In-Context Learning for Machine Translation of Low-Resource Languages