Text Restoration of Ancient Documents with Language Models

Shibingfeng Zhang; Edoardo Caraffa; Annafelicia Zuffrano; Maddalena Modesti; Giovanni Colavizza

cs.CLAug 28, 2026

Text Restoration of Ancient Documents with Language Models

Authors: Shibingfeng Zhang, Edoardo Caraffa, Annafelicia Zuffrano, Maddalena Modesti, Giovanni Colavizza

Organizations: Department of Classical Philology and Italian Studies, University of Bologna · Centre for Digital and Computational Humanities, University of Copenhagen

Abstract

Purpose - This study investigates the feasibility of restoring missing text caused by physical lacunae in damaged ancient manuscripts using language models. Methodology - The study proposes different scenarios to replicate real-world conditions. Language models of different architectures are applied according to their suitability to each scenario. We also propose several decoding strategies that further enhance performance and address the discrepancy between lacuna boundaries and the models' tokenization schemes. Findings - The results reveal that text restoration of these documents cannot be fully automated, but it can serve as a useful tool to assist paleographers in their work. Model performance varies greatly depending on which structural part of the document needs to be restored and whether the character length of missing text is available. Originality - This is the first study and to analyze model performance on formulaic and non-formulaic content and the impact of lacuna length awareness in manuscript restoration. Both are recurring challenges in paleographers' manual restoration work. Through systematic comparison and both qualitative and quantitative analysis of different models' performance under varying settings, this study offers a guideline for developing assistive tools to support paleographers.

Figures & tables

Damage type	Count	Percentage	Mean length	Median	Standard Deviation
Known length	5807	93.2%	11.61	7	14.44
Inferred text	3133	50.28%	10.19	6	16.56
Estimated length	2694	43.23%	12.41	9	11.47
Unknown length	424	6.8%	/	/	/

Table I: Statistics of textual damages in the manuscripts. Known length aggregates the inferred-text and estimated-length cases. Mean length , Median and Standard Deviation refer to character length and it may contain punctuations and spaces. “/” marks values that are undefined.

Panel of Figure 2 — Figure 2: An illustration of the misalignment between lacuna boundary and token boundary. The tokenizer in question is LaBERTa’s tokenizer. In this case, the lacuna edges fall within tokens.

Panel of Figure 3 — Figure 3: Text restoration workflow of LaBERTa and PhilBERTa models

Lacuna Length	Model	CER	Overlap	HR@1	HR@3	HR@5	HR@10
Overall	Aeneas	$\mathbf{\underline{30.26_{\pm 0.20}}}$	$\mathbf{\underline{60.79_{\pm 0.42}}}$	$\mathbf{\underline{50.71_{\pm 0.54}}}$	$\mathbf{\underline{58.08_{\pm 0.42}}}$	$\mathbf{\underline{59.69_{\pm 0.38}}}$	$\mathbf{\underline{61.20_{\pm 0.29}}}$
	LaBERTa	$59.77_{\pm 1.54}$	$53.94_{\pm 0.96}$	$32.95_{\pm 0.64}$	$40.66_{\pm 0.49}$	$43.03_{\pm 0.24}$	$45.17_{\pm 0.54}$
	PhilBERTa	$61.59_{\pm 1.31}$	$52.70_{\pm 0.57}$	$31.05_{\pm 0.67}$	$38.97_{\pm 0.72}$	$41.97_{\pm 0.73}$	$44.86_{\pm 1.18}$
	DeepSeek	$47.55_{\pm 0.58}$	$53.00_{\pm 0.86}$	$26.38_{\pm 1.12}$	$31.98_{\pm 1.13}$	$33.24_{\pm 1.24}$	$34.28_{\pm 1.71}$
	Llama	$70.78_{\pm 0.76}$	$33.82_{\pm 0.20}$	$5.62_{\pm 0.49}$	$8.08_{\pm 0.28}$	$10.27_{\pm 0.36}$	$12.22_{\pm 0.42}$
Short	Aeneas	$\mathbf{\underline{18.16_{\pm 0.27}}}$	$\mathbf{\underline{78.43_{\pm 0.41}}}$	$\mathbf{\underline{72.34_{\pm 0.54}}}$	$\mathbf{\underline{82.54_{\pm 0.58}}}$	$\mathbf{\underline{84.95_{\pm 0.43}}}$	$\mathbf{\underline{87.39_{\pm 0.44}}}$

Table II: Known-length setting results (mean

\pm

95% confidence interval over 5 checkpoints).

Component	Model	CER	Overlap	HR@1	HR@3	HR@5	HR@10
Protocol	Aeneas	$\mathbf{\underline{13.40_{\pm 2.38}}}$	$\mathbf{\underline{77.42_{\pm 2.93}}}$	$\mathbf{\underline{68.36_{\pm 4.02}}}$	$\mathbf{\underline{73.73_{\pm 2.11}}}$	$\mathbf{\underline{74.93_{\pm 2.42}}}$	$\mathbf{\underline{75.22_{\pm 2.11}}}$
	LaBERTa	$36.82_{\pm 0.91}$	$67.51_{\pm 1.27}$	$46.57_{\pm 2.42}$	$56.42_{\pm 1.55}$	$58.21_{\pm 1.85}$	$60.60_{\pm 1.66}$
	PhilBERTa	$39.19_{\pm 5.69}$	$63.49_{\pm 2.03}$	$40.30_{\pm 2.62}$	$51.64_{\pm 2.49}$	$54.93_{\pm 4.02}$	$59.10_{\pm 5.50}$
	DeepSeek	$51.11_{\pm 11.08}$	$51.96_{\pm 5.39}$	$24.40_{\pm 0.90}$	$32.87_{\pm 2.19}$	$34.06_{\pm 2.97}$	$36.15_{\pm 4.54}$
	Llama	$60.38_{\pm 2.55}$	$44.88_{\pm 1.13}$	$8.96_{\pm 2.27}$	$13.73_{\pm 3.05}$	$17.91_{\pm 3.47}$	$18.81_{\pm 2.81}$
Text Body	Aeneas	$\mathbf{\underline{30.64_{\pm 0.39}}}$	$\mathbf{\underline{60.46_{\pm 0.57}}}$	$\mathbf{\underline{50.11_{\pm 0.66}}}$	$\mathbf{\underline{58.31_{\pm 0.48}}}$	$\mathbf{\underline{59.78_{\pm 0.44}}}$	$\mathbf{\underline{61.37_{\pm 0.30}}}$

Table III: Known-length setting results divided by the functional components of notarial manuscripts(mean

\pm

95% confidence interval over 5 checkpoints).

Lacuna Length	Model	CER	Overlap	HR@1	HR@3	HR@5	HR@10
Overall	Aeneas	$70.33_{\pm 1.39}$	$41.49_{\pm 0.51}$	$24.05_{\pm 0.65}$	$\mathbf{\underline{37.61_{\pm 0.67}}}$	$\mathbf{\underline{40.66_{\pm 0.25}}}$	$\mathbf{\underline{43.85_{\pm 0.49}}}$
	LaTa	$72.72_{\pm 2.27}$	$47.71_{\pm 0.60}$	$\underline{24.86_{\pm 0.96}}$	$30.86_{\pm 1.71}$	$32.07_{\pm 2.44}$	$32.66_{\pm 3.03}$
	PhilTa	$130.29_{\pm 27.24}$	$28.71_{\pm 2.49}$	$11.20_{\pm 1.48}$	$13.86_{\pm 2.05}$	$14.20_{\pm 2.31}$	$14.31_{\pm 2.41}$
	DeepSeek	$\mathbf{\underline{67.15_{\pm 1.59}}}$	$\mathbf{\underline{49.94_{\pm 0.09}}}$	$23.86_{\pm 0.24}$	$28.41_{\pm 0.22}$	$30.04_{\pm 0.13}$	$30.76_{\pm 0.20}$
	Llama	$81.18_{\pm 0.94}$	$30.72_{\pm 0.31}$	$3.48_{\pm 0.16}$	$6.08_{\pm 0.40}$	$7.54_{\pm 0.42}$	$8.90_{\pm 0.54}$
Short	Aeneas	$\underline{69.83_{\pm 2.32}}$	$56.87_{\pm 0.44}$	$\mathbf{\underline{41.97_{\pm 0.72}}}$	$\mathbf{\underline{59.19_{\pm 1.38}}}$	$\mathbf{\underline{63.83_{\pm 0.35}}}$	$\mathbf{\underline{69.15_{\pm 0.88}}}$

Table IV: Unknown-length setting: results divided by text length (mean

\pm

95% CI, n=5 seeds).

Component	Model	CER	Overlap	HR@1	HR@3	HR@5	HR@10
Protocol	Aeneas	$\mathbf{\underline{39.45_{\pm 1.98}}}$	$56.94_{\pm 0.73}$	$37.91_{\pm 2.11}$	$\underline{51.34_{\pm 2.11}}$	$\underline{51.94_{\pm 2.42}}$	$\underline{53.13_{\pm 2.11}}$
	LaTa	$48.84_{\pm 5.42}$	$\underline{58.77_{\pm 1.21}}$	$\underline{40.90_{\pm 3.84}}$	$47.16_{\pm 2.81}$	$48.96_{\pm 2.42}$	$50.15_{\pm 2.81}$
	PhilTa	$65.33_{\pm 16.68}$	$46.19_{\pm 3.27}$	$25.67_{\pm 6.06}$	$35.22_{\pm 3.84}$	$35.52_{\pm 4.23}$	$36.12_{\pm 4.80}$
	DeepSeek	$71.49_{\pm 6.37}$	$52.24_{\pm 0.89}$	$20.00_{\pm 1.66}$	$25.67_{\pm 1.55}$	$25.97_{\pm 1.66}$	$28.06_{\pm 2.42}$
	Llama	$64.11_{\pm 0.28}$	$40.57_{\pm 0.67}$	$3.58_{\pm 1.02}$	$10.75_{\pm 0.83}$	$12.24_{\pm 0.83}$	$14.03_{\pm 1.02}$
Text Body	Aeneas	$72.00_{\pm 1.40}$	$40.44_{\pm 0.57}$	$22.76_{\pm 0.78}$	$\underline{36.61_{\pm 0.70}}$	$\mathbf{\underline{40.15_{\pm 0.39}}}$	$\mathbf{\underline{43.46_{\pm 0.61}}}$

Table V: Unknown-length setting: results divided by tag (mean

\pm

95% CI, n=5 seeds).

Setting	Prompting	CER	Overlap	HR@1	HR@3	HR@5	HR@10
Known length	Zero-shot	$57.62_{\pm 4.86}$	$46.10_{\pm 0.06}$	$20.15_{\pm 0.20}$	$25.29_{\pm 0.27}$	$26.20_{\pm 0.24}$	$26.61_{\pm 0.20}$
	Few-shot	$49.87_{\pm 0.30}$	$47.59_{\pm 0.19}$	$21.75_{\pm 0.33}$	$25.86_{\pm 0.43}$	$26.63_{\pm 0.47}$	$27.24_{\pm 0.60}$
	Concordance	$56.81_{\pm 5.04}$	$49.57_{\pm 0.17}$	$22.92_{\pm 0.28}$	$28.14_{\pm 0.41}$	$30.19_{\pm 0.29}$	$31.34_{\pm 0.16}$
	Few-shot + concordance	$\mathbf{\underline{47.55_{\pm 0.58}}}$	$\mathbf{\underline{53.00_{\pm 0.86}}}$	$\mathbf{\underline{26.38_{\pm 1.12}}}$	$\mathbf{\underline{31.98_{\pm 1.13}}}$	$\mathbf{\underline{33.24_{\pm 1.24}}}$	$\mathbf{\underline{34.28_{\pm 1.71}}}$
Unknown length	Zero-shot	$100.92_{\pm 6.22}$	$42.51_{\pm 0.19}$	$16.75_{\pm 0.30}$	$20.75_{\pm 0.30}$	$21.51_{\pm 0.34}$	$21.92_{\pm 0.30}$
	Few-shot	$73.09_{\pm 2.63}$	$44.48_{\pm 0.16}$	$19.27_{\pm 0.38}$	$22.49_{\pm 0.38}$	$23.14_{\pm 0.20}$	$23.76_{\pm 0.14}$

Table VI: Prompting ablation for DeepSeek on the full test set (mean

\pm

95% CI across 5 rounds).

Figure 4: Relationship between Hit Rate and Lacuna Length of Aeneas in known-length setting

Tag	Type-Token Ratio	Hapax-Token Ratio
Protocol	0.137	6.8%
Eschatocol	0.223	14.8%
Text Body	0.129	6.7%
Rogationes	0.310	19.7%

Table VII: Lexical richness and Hapax Legomena-to-token ratio across notary document sections.

Historical Document Processing

Historical document processing uses AI to transcribe, structure, restore, and analyze manuscripts, books, newspapers, and archival records.

See all 76 papers

Apollo Restore: A Foundation LLM for Historical Greek Optimized for Fill-in-the-Middle Restoration of Ancient Greek Texts
Sep 18, 2026Hope McGovern, Anna Dolganov, Samuel Belkadi +3Language Modeling
Mistral AI · Austrian Academy of Sciences · Reply +1
Think Before You Restore: Risk-Aware Manchu Manuscript Restoration with Stroke-Guided Attention
Sep 28, 2026Mingqiu Liang, Dongdong Wang, Siyang Lu +2Image Restoration Historical Document Processing
School of Computer Science and Technology, Beijing Jiaotong University, Beijing, China · College of Design, Construction, and Planning, University of Florida, Gainesville, FL, USA · School of Japanese Studies, Dalian University of Foreign Languages, Dalian, China
MESA: A Training-Free Multi-Exemplar Deep Framework for Restoring Ancient Inscription Textures
Apr 19, 2026Vasileios Toulatzis, Sofia Theodoridou, Ioannis FudosImage Generation Image Restoration
University of Ioannina, Greece

Sep 18, 2026J/K move · Enter open · S save

Apollo Restore: A Foundation LLM for Historical Greek Optimized for Fill-in-the-Middle Restoration of Ancient Greek Texts

Hope McGovern, Anna Dolganov, Samuel Belkadi, Guillaume Kunsch+2

Mistral AI · Austrian Academy of Sciences · Reply+1

Language Modeling

Open paper AskPDF arXiv

We present Apollo Restore, a 24-billion-parameter large language model for restoring lacunae---physical gaps---in fragmentary Ancient Greek texts. Fine-tuned from Mistral Small with a fill-in-the-middle objective, Apollo Restore reconstructs missing spans without requiring oracle knowledge of their length. To our knowledge, it is the first large-scale decoder model for historical Greek, and the first for any ancient Mediterranean language. Evaluated as in prior work, on short gaps of up to ten characters, Apollo Restore places the correct restoration among its top twenty candidates for 80.6%/54.6%/61.0% of documentary-papyrus, literary-papyrus, and stone-inscription lacunae, exceeding the strongest published models by

1.6\times

2.6\times

1.4\times

. Prior evaluation protocols, however, inflate scores through a bias toward trivially short gaps; under a length-balanced metric Apollo Restore's advantage over the strongest published models grows to

2.3\times

3.5\times

1.6\times

and degrades gracefully, even given incorrect length hints. In a blind study, 20 expert papyrologists, epigraphists, and philologists strongly preferred Apollo Restore to the strongest baseline and judged its performance at least as good as human restorations in 77% of cases. Apollo Restore also improves the published reading of PHerc. 1667---a papyrus roll carbonised in the eruption of Vesuvius in 79 CE and digitally unrolled and edited after Apollo Restore's training data was compiled. Apollo Restore is an output of the Decoding Antiquity initiative to build specialized LLMs for historical languages and manuscripts, led by the Austrian Academy of Sciences.

Abstract

Historical Document Processing

Explore similar work

Apollo Restore: A Foundation LLM for Historical Greek Optimized for Fill-in-the-Middle Restoration of Ancient Greek Texts

Think Before You Restore: Risk-Aware Manchu Manuscript Restoration with Stroke-Guided Attention

MESA: A Training-Free Multi-Exemplar Deep Framework for Restoring Ancient Inscription Textures