Text Restoration of Ancient Documents with Language Models

Shibingfeng Zhang; Edoardo Caraffa; Annafelicia Zuffrano; Maddalena Modesti; Giovanni Colavizza

cs.CLAug 28, 2026

Text Restoration of Ancient Documents with Language Models

Authors: Shibingfeng Zhang, Edoardo Caraffa, Annafelicia Zuffrano, Maddalena Modesti, Giovanni Colavizza

Organizations: Department of Classical Philology and Italian Studies, University of Bologna · Centre for Digital and Computational Humanities, University of Copenhagen

Abstract

Purpose - This study investigates the feasibility of restoring missing text caused by physical lacunae in damaged ancient manuscripts using language models. Methodology - The study proposes different scenarios to replicate real-world conditions. Language models of different architectures are applied according to their suitability to each scenario. We also propose several decoding strategies that further enhance performance and address the discrepancy between lacuna boundaries and the models' tokenization schemes. Findings - The results reveal that text restoration of these documents cannot be fully automated, but it can serve as a useful tool to assist paleographers in their work. Model performance varies greatly depending on which structural part of the document needs to be restored and whether the character length of missing text is available. Originality - This is the first study and to analyze model performance on formulaic and non-formulaic content and the impact of lacuna length awareness in manuscript restoration. Both are recurring challenges in paleographers' manual restoration work. Through systematic comparison and both qualitative and quantitative analysis of different models' performance under varying settings, this study offers a guideline for developing assistive tools to support paleographers.

Figures & tables

Damage type	Count	Percentage	Mean length	Median	Standard Deviation
Known length	5807	93.2%	11.61	7	14.44
Inferred text	3133	50.28%	10.19	6	16.56
Estimated length	2694	43.23%	12.41	9	11.47
Unknown length	424	6.8%	/	/	/

Table I: Statistics of textual damages in the manuscripts. Known length aggregates the inferred-text and estimated-length cases. Mean length , Median and Standard Deviation refer to character length and it may contain punctuations and spaces. “/” marks values that are undefined.

Panel of Figure 2 — Figure 2: An illustration of the misalignment between lacuna boundary and token boundary. The tokenizer in question is LaBERTa’s tokenizer. In this case, the lacuna edges fall within tokens.

Panel of Figure 3 — Figure 3: Text restoration workflow of LaBERTa and PhilBERTa models

Lacuna Length	Model	CER	Overlap	HR@1	HR@3	HR@5	HR@10
Overall	Aeneas	$\mathbf{\underline{30.26_{\pm 0.20}}}$	$\mathbf{\underline{60.79_{\pm 0.42}}}$	$\mathbf{\underline{50.71_{\pm 0.54}}}$	$\mathbf{\underline{58.08_{\pm 0.42}}}$	$\mathbf{\underline{59.69_{\pm 0.38}}}$	$\mathbf{\underline{61.20_{\pm 0.29}}}$
	LaBERTa	$59.77_{\pm 1.54}$	$53.94_{\pm 0.96}$	$32.95_{\pm 0.64}$	$40.66_{\pm 0.49}$	$43.03_{\pm 0.24}$	$45.17_{\pm 0.54}$
	PhilBERTa	$61.59_{\pm 1.31}$	$52.70_{\pm 0.57}$	$31.05_{\pm 0.67}$	$38.97_{\pm 0.72}$	$41.97_{\pm 0.73}$	$44.86_{\pm 1.18}$
	DeepSeek	$47.55_{\pm 0.58}$	$53.00_{\pm 0.86}$	$26.38_{\pm 1.12}$	$31.98_{\pm 1.13}$	$33.24_{\pm 1.24}$	$34.28_{\pm 1.71}$
	Llama	$70.78_{\pm 0.76}$	$33.82_{\pm 0.20}$	$5.62_{\pm 0.49}$	$8.08_{\pm 0.28}$	$10.27_{\pm 0.36}$	$12.22_{\pm 0.42}$
Short	Aeneas	$\mathbf{\underline{18.16_{\pm 0.27}}}$	$\mathbf{\underline{78.43_{\pm 0.41}}}$	$\mathbf{\underline{72.34_{\pm 0.54}}}$	$\mathbf{\underline{82.54_{\pm 0.58}}}$	$\mathbf{\underline{84.95_{\pm 0.43}}}$	$\mathbf{\underline{87.39_{\pm 0.44}}}$

Table II: Known-length setting results (mean

\pm

95% confidence interval over 5 checkpoints).

Component	Model	CER	Overlap	HR@1	HR@3	HR@5	HR@10
Protocol	Aeneas	$\mathbf{\underline{13.40_{\pm 2.38}}}$	$\mathbf{\underline{77.42_{\pm 2.93}}}$	$\mathbf{\underline{68.36_{\pm 4.02}}}$	$\mathbf{\underline{73.73_{\pm 2.11}}}$	$\mathbf{\underline{74.93_{\pm 2.42}}}$	$\mathbf{\underline{75.22_{\pm 2.11}}}$
	LaBERTa	$36.82_{\pm 0.91}$	$67.51_{\pm 1.27}$	$46.57_{\pm 2.42}$	$56.42_{\pm 1.55}$	$58.21_{\pm 1.85}$	$60.60_{\pm 1.66}$
	PhilBERTa	$39.19_{\pm 5.69}$	$63.49_{\pm 2.03}$	$40.30_{\pm 2.62}$	$51.64_{\pm 2.49}$	$54.93_{\pm 4.02}$	$59.10_{\pm 5.50}$
	DeepSeek	$51.11_{\pm 11.08}$	$51.96_{\pm 5.39}$	$24.40_{\pm 0.90}$	$32.87_{\pm 2.19}$	$34.06_{\pm 2.97}$	$36.15_{\pm 4.54}$
	Llama	$60.38_{\pm 2.55}$	$44.88_{\pm 1.13}$	$8.96_{\pm 2.27}$	$13.73_{\pm 3.05}$	$17.91_{\pm 3.47}$	$18.81_{\pm 2.81}$
Text Body	Aeneas	$\mathbf{\underline{30.64_{\pm 0.39}}}$	$\mathbf{\underline{60.46_{\pm 0.57}}}$	$\mathbf{\underline{50.11_{\pm 0.66}}}$	$\mathbf{\underline{58.31_{\pm 0.48}}}$	$\mathbf{\underline{59.78_{\pm 0.44}}}$	$\mathbf{\underline{61.37_{\pm 0.30}}}$

Table III: Known-length setting results divided by the functional components of notarial manuscripts(mean

\pm

95% confidence interval over 5 checkpoints).

Lacuna Length	Model	CER	Overlap	HR@1	HR@3	HR@5	HR@10
Overall	Aeneas	$70.33_{\pm 1.39}$	$41.49_{\pm 0.51}$	$24.05_{\pm 0.65}$	$\mathbf{\underline{37.61_{\pm 0.67}}}$	$\mathbf{\underline{40.66_{\pm 0.25}}}$	$\mathbf{\underline{43.85_{\pm 0.49}}}$
	LaTa	$72.72_{\pm 2.27}$	$47.71_{\pm 0.60}$	$\underline{24.86_{\pm 0.96}}$	$30.86_{\pm 1.71}$	$32.07_{\pm 2.44}$	$32.66_{\pm 3.03}$
	PhilTa	$130.29_{\pm 27.24}$	$28.71_{\pm 2.49}$	$11.20_{\pm 1.48}$	$13.86_{\pm 2.05}$	$14.20_{\pm 2.31}$	$14.31_{\pm 2.41}$
	DeepSeek	$\mathbf{\underline{67.15_{\pm 1.59}}}$	$\mathbf{\underline{49.94_{\pm 0.09}}}$	$23.86_{\pm 0.24}$	$28.41_{\pm 0.22}$	$30.04_{\pm 0.13}$	$30.76_{\pm 0.20}$
	Llama	$81.18_{\pm 0.94}$	$30.72_{\pm 0.31}$	$3.48_{\pm 0.16}$	$6.08_{\pm 0.40}$	$7.54_{\pm 0.42}$	$8.90_{\pm 0.54}$
Short	Aeneas	$\underline{69.83_{\pm 2.32}}$	$56.87_{\pm 0.44}$	$\mathbf{\underline{41.97_{\pm 0.72}}}$	$\mathbf{\underline{59.19_{\pm 1.38}}}$	$\mathbf{\underline{63.83_{\pm 0.35}}}$	$\mathbf{\underline{69.15_{\pm 0.88}}}$

Table IV: Unknown-length setting: results divided by text length (mean

\pm

95% CI, n=5 seeds).

Component	Model	CER	Overlap	HR@1	HR@3	HR@5	HR@10
Protocol	Aeneas	$\mathbf{\underline{39.45_{\pm 1.98}}}$	$56.94_{\pm 0.73}$	$37.91_{\pm 2.11}$	$\underline{51.34_{\pm 2.11}}$	$\underline{51.94_{\pm 2.42}}$	$\underline{53.13_{\pm 2.11}}$
	LaTa	$48.84_{\pm 5.42}$	$\underline{58.77_{\pm 1.21}}$	$\underline{40.90_{\pm 3.84}}$	$47.16_{\pm 2.81}$	$48.96_{\pm 2.42}$	$50.15_{\pm 2.81}$
	PhilTa	$65.33_{\pm 16.68}$	$46.19_{\pm 3.27}$	$25.67_{\pm 6.06}$	$35.22_{\pm 3.84}$	$35.52_{\pm 4.23}$	$36.12_{\pm 4.80}$
	DeepSeek	$71.49_{\pm 6.37}$	$52.24_{\pm 0.89}$	$20.00_{\pm 1.66}$	$25.67_{\pm 1.55}$	$25.97_{\pm 1.66}$	$28.06_{\pm 2.42}$
	Llama	$64.11_{\pm 0.28}$	$40.57_{\pm 0.67}$	$3.58_{\pm 1.02}$	$10.75_{\pm 0.83}$	$12.24_{\pm 0.83}$	$14.03_{\pm 1.02}$
Text Body	Aeneas	$72.00_{\pm 1.40}$	$40.44_{\pm 0.57}$	$22.76_{\pm 0.78}$	$\underline{36.61_{\pm 0.70}}$	$\mathbf{\underline{40.15_{\pm 0.39}}}$	$\mathbf{\underline{43.46_{\pm 0.61}}}$

Table V: Unknown-length setting: results divided by tag (mean

\pm

95% CI, n=5 seeds).

Setting	Prompting	CER	Overlap	HR@1	HR@3	HR@5	HR@10
Known length	Zero-shot	$57.62_{\pm 4.86}$	$46.10_{\pm 0.06}$	$20.15_{\pm 0.20}$	$25.29_{\pm 0.27}$	$26.20_{\pm 0.24}$	$26.61_{\pm 0.20}$
	Few-shot	$49.87_{\pm 0.30}$	$47.59_{\pm 0.19}$	$21.75_{\pm 0.33}$	$25.86_{\pm 0.43}$	$26.63_{\pm 0.47}$	$27.24_{\pm 0.60}$
	Concordance	$56.81_{\pm 5.04}$	$49.57_{\pm 0.17}$	$22.92_{\pm 0.28}$	$28.14_{\pm 0.41}$	$30.19_{\pm 0.29}$	$31.34_{\pm 0.16}$
	Few-shot + concordance	$\mathbf{\underline{47.55_{\pm 0.58}}}$	$\mathbf{\underline{53.00_{\pm 0.86}}}$	$\mathbf{\underline{26.38_{\pm 1.12}}}$	$\mathbf{\underline{31.98_{\pm 1.13}}}$	$\mathbf{\underline{33.24_{\pm 1.24}}}$	$\mathbf{\underline{34.28_{\pm 1.71}}}$
Unknown length	Zero-shot	$100.92_{\pm 6.22}$	$42.51_{\pm 0.19}$	$16.75_{\pm 0.30}$	$20.75_{\pm 0.30}$	$21.51_{\pm 0.34}$	$21.92_{\pm 0.30}$
	Few-shot	$73.09_{\pm 2.63}$	$44.48_{\pm 0.16}$	$19.27_{\pm 0.38}$	$22.49_{\pm 0.38}$	$23.14_{\pm 0.20}$	$23.76_{\pm 0.14}$

Table VI: Prompting ablation for DeepSeek on the full test set (mean

\pm

95% CI across 5 rounds).

Figure 4: Relationship between Hit Rate and Lacuna Length of Aeneas in known-length setting

Tag	Type-Token Ratio	Hapax-Token Ratio
Protocol	0.137	6.8%
Eschatocol	0.223	14.8%
Text Body	0.129	6.7%
Rogationes	0.310	19.7%

Table VII: Lexical richness and Hapax Legomena-to-token ratio across notary document sections.

Historical Document Processing

Historical document processing uses AI to transcribe, structure, restore, and analyze manuscripts, books, newspapers, and archival records.

See all 76 papers

Sep 18, 2026

Apollo Restore: A Foundation LLM for Historical Greek Optimized for Fill-in-the-Middle Restoration of Ancient Greek Texts

We present Apollo Restore, a 24-billion-parameter large language model for restoring lacunae---physical gaps---in fragmentary Ancient Greek texts. Fine-tuned from Mistral Small with a fill-in-the-middle objective, Apollo Restore reconstructs missing spans without requiring oracle knowledge of their length. To our knowledge, it is the first large-scale decoder model for historical Greek, and the first for any ancient Mediterranean language. Evaluated as in prior work, on short gaps of up to ten characters, Apollo Restore places the correct restoration among its top twenty candidates for 80.6%/54.6%/61.0% of documentary-papyrus, literary-papyrus, and stone-inscription lacunae, exceeding the strongest published models by

1.6\times

2.6\times

1.4\times

. Prior evaluation protocols, however, inflate scores through a bias toward trivially short gaps; under a length-balanced metric Apollo Restore's advantage over the strongest published models grows to

2.3\times

3.5\times

1.6\times

and degrades gracefully, even given incorrect length hints. In a blind study, 20 expert papyrologists, epigraphists, and philologists strongly preferred Apollo Restore to the strongest baseline and judged its performance at least as good as human restorations in 77% of cases. Apollo Restore also improves the published reading of PHerc. 1667---a papyrus roll carbonised in the eruption of Vesuvius in 79 CE and digitally unrolled and edited after Apollo Restore's training data was compiled. Apollo Restore is an output of the Decoding Antiquity initiative to build specialized LLMs for historical languages and manuscripts, led by the Austrian Academy of Sciences.

Hope McGovern, Anna Dolganov, Samuel Belkadi +3

Mistral AI · Austrian Academy of Sciences · Reply +1

Language Modeling

Ask PDF

Sep 28, 2026

Think Before You Restore: Risk-Aware Manchu Manuscript Restoration with Stroke-Guided Attention

Full-page blind restoration of historical Manchu manuscripts is challenging due to scarce annotations, unknown degradation regions, and fragile connected strokes. Generic restoration models may improve visual quality but often modify intact content, leading to over-restoration. We propose SAGE-Restore (Stroke-Aware Gated rEstoration), a selective restoration framework that first assesses where restoration is needed and then uses this assessment to guide restoration candidate generation and pixel-level selection. Its encoder predicts patch-level repair probabilities from complementary appearance and stroke-structural cues to condition restoration candidate generation, while the corresponding repair logits are refined into a pixel-level soft gate that selectively controls where the restoration candidate is applied. We further introduce a fidelity-aware evaluation protocol that jointly measures degraded-region recovery, intact-content preservation, and their balance. SAGE-Restore achieves the highest R-Recovery (0.463) and RFS (0.626), while maintaining high U-Fidelity (0.968), demonstrating an effective balance between restoration and content preservation.

Mingqiu Liang, Dongdong Wang, Siyang Lu +2

School of Computer Science and Technology, Beijing Jiaotong University, Beijing, China · College of Design, Construction, and Planning, University of Florida, Gainesville, FL, USA · School of Japanese Studies, Dalian University of Foreign Languages, Dalian, China

Image Restoration Historical Document Processing

Ask PDF

Apr 19, 2026

MESA: A Training-Free Multi-Exemplar Deep Framework for Restoring Ancient Inscription Textures

Ancient inscriptions frequently suffer missing or corrupted regions from fragmentation, erosion, or other damage, hindering reading, and analysis. We review prior image restoration methods and their applicability to inscription image recovery, then introduce MESA (Multi-Exemplar, Style-Aware) -an image-level restoration method that uses well-preserved exemplar inscriptions (from the same epigraphic monument, material, or similar letterforms) to guide reconstruction of damaged text. MESA encodes VGG19 convolutional features as Gram matrices to capture exemplar texture, style, and stroke structure; for each neural network layer it selects the exemplar minimizing Mean-Squared Displacement (MSD) to the damaged input. Layer-wise contribution weights are derived from Optical Character Recognition-estimated character widths in the exemplar set to bias filters toward scales matching letter geometry, and a training mask preserves intact regions so synthesis is restricted to damaged areas. We also summarize prior network architectures and exemplar and single-image synthesis, inpainting, and Generative Adversarial Network (GAN) approaches, highlighting limitations that MESA addresses. Comparative experiments demonstrate the advantages of MESA. Finally, we provide a practical roadmap for choosing restoration strategies given available exemplars and metadata.

Vasileios Toulatzis, Sofia Theodoridou, Ioannis Fudos

University of Ioannina, Greece

Image Generation Image Restoration Image Inpainting Texture Synthesis

Ask PDF

Abstract

Historical Document Processing

Explore similar work

Apollo Restore: A Foundation LLM for Historical Greek Optimized for Fill-in-the-Middle Restoration of Ancient Greek Texts

Think Before You Restore: Risk-Aware Manchu Manuscript Restoration with Stroke-Guided Attention

MESA: A Training-Free Multi-Exemplar Deep Framework for Restoring Ancient Inscription Textures