Which papyrus HTR is good enough? Character-error-rate tolerance of four papyrological tasks on Greek texts

Anton Repushko; Elena Chepel

cs.CLSep 29, 2026

Which papyrus HTR is good enough? Character-error-rate tolerance of four papyrological tasks on Greek texts

Authors: Anton Repushko, Elena Chepel

Organizations: Independent Researcher, Berlin, Germany. · University of Vienna, Vienna, Austria.

Abstract

Purpose: Most Greek papyri remain unpublished and undigitised; a handwritten text recognition (HTR) pipeline that transcribes them automatically would let scholars discover documents and literary works that have so far gone unread. Recognition systems for Ancient Greek papyri are in statu nascendi, and how accurate they must be for a given papyrological task has not been examined. To answer this and set a benchmark for Greek papyrus HTR, we test a range of character error rates (CER) against four papyrological tasks, using published editions as ground truth. Methods: From 63,846 current editions of Greek texts in papyri.info, we imitate a letters-only "perfect HTR" output by removing the editorial layer, then degrade it with a seeded algorithm to exact CERs of 1 - 50%, with lost lines and four error-shape variants. On these data we train small models (TF-IDF, fastText, a character CNN, ByT5-small) for document type, dating and documentary-versus-literary classification, and apply eight keyword search methods. We compare models trained on clean text with models retrained at a specific CER level, and evaluate across CERs. Results: Tolerance differs by task. With clean-trained models, documentary-versus-literary classification retains 90% of its metric up to 20% CER; document type up to 7.5%; subtypes and search up to 5%; dating only up to 3%. Retraining on text containing character errors largely eliminates the sharp degradation that otherwise sets in above 15% CER. Models generally tolerate concentrated damage in a long document better than small errors spread across a short text. Conclusion: The study provides a CER target for each of the four tasks and shows that models trained on noisy text make current, imperfect text recognition useful for them.

Figures & tables

Task	Labels from	Train / validation / test	Target	Headline metric	Baseline
T1 Document form	Grammateus type	1,776 (5-fold CV)	4 types	macro-F1	0.121
T1 subtype	Grammateus subtype	1,665 (5-fold CV)	16 subtypes ( $\geq$ 30 papyri)	macro-F1	0.028
T2 Dating	HGV date interval	43,803 / 4,629 / 4,681	year (interval midpoint)	MAE in years	184
T3 Documentary/literary	DDbDP vs DCLP	47,328 / 5,906 / 5,950	2 classes	macro-F1	0.491
T4 Search	dictionary, names, formulae, substrings	2,534 queries; 5,961 docs, 96,410 lines	relevant units	recall@20, filter F1	–

Table 1: The four tasks, the data behind each and how they are scored. Counts are documents with at least one letter; validation and test are the evaluated subsets. T1 uses five-fold grouped cross-validation over all its papyri

Edition (L0) as printed: restorations, accents, case, word division	1	’Απίων ’Επιμάχῳ τω̃ι πατρὶ ϰαὶ
	2	ϰυρίῳ πλει̃ςτα χαίρειν. πρ`ο μὲν πάν-
	$\vdots$
	19	Καπίτων[α] π̣ο̣λλὰ ϰαὶ το`̣υς̣ ἀδελφο´υς
	20	[μ]ου ϰαὶ Σε[ρηνί]λλαν ϰαὶ το[`υς] φίλους μο[υ].
Ink only editorial layer removed; $\square$ marks lost text	1	’Απίων ’Επιμάχῳ τω̃ι πατρὶ ϰαὶ
	2	ϰυρίῳ πλει̃ςτα χαίρειν. πρ`ο μὲν πάν

Figure 1: A real papyrus from the edition to letters-only text and its degradation. Lines 1–2 and 19–20 of the letter of Apion to his father (BGU II 423 = Chrest.Wilck. 480) as encoded in the DDbDP edition (L0; square brackets enclose restored letters, dots mark uncertain ones), with the editorial layer removed (ink only;

\square

marks lost text), and reduced to letters (L3), the text a perfect HTR system would return. The last three rows are the same lines degraded by the study’s algorithm to exactly 5, 15 and 30% CER with the default 3:1:1 mix: a substituted letter is bold, an inserted letter bold and underlined, and a deleted letter is marked by a bold dot. Every edit costs one Levenshtein step and gap tokens are never touched

Model	Tasks	Description
TF-IDF + logistic / ridge	T1–T3	letter 1–5-gram TF-IDF (sublinear), multinomial logistic regression (classes) or ridge regression (dates); $\min\mathrm{df}$ , $C$ or $\alpha$ and class weights tuned Pedregosa et al (2011)
fastText	T1–T3	supervised fastText Joulin et al (2017) with one token per letter, so its word $n$ -grams are letter $n$ -grams (3–5); dating as 25-year classes, prediction = probability-weighted bin midpoint; T3 trained with the literary class oversampled to balance
Char-CNN	T1–T3	2.6M parameters: letter embeddings (64), a $k{=}7$ convolution, three stages of two residual blocks with dilated $k{=}5$ convolutions (256 channels), masked max + mean pooling, MLP head Zhang et al (2015) ; inputs up to 4,096 letters; early stopping on the validation split
ByT5-small	T1–T3	the 218M-parameter byte-level encoder of ByT5-small Xue et al (2022) with masked mean pooling and a linear head; inputs up to 2,048 bytes ( $\approx$ 1,000 Greek letters, above the 90th percentile of document length); AdamW Loshchilov and Hutter (2019) , bf16, 3 epochs (8 for T2)
Line-length shortcut	T3	gradient-boosted trees on line-length statistics only: does layout give the answer away?
Search methods	T4	exact match; Levenshtein filters with at most 1 or 2 edits; normalised distance $d/m\leq 0.25$ ; ranking by minimum edit distance $d$ (semi-global, Myers’ bit-parallel algorithm Myers (1999) ); trigram overlap; TF-IDF cosine over letter 2–4-grams; Okapi BM25 Robertson and Zaragoza (2009) over letter $n$ -grams ( $n{=}6$ , $k_{1}{=}0.3$ , $b{=}0$ , tuned on clean validation queries). Matches never span a gap or a line break

Table 2: Models. All hyperparameters are tuned once on the clean validation split and frozen.

Figure 2: T1 document type: macro-F1 of every model against CER. a Models trained on clean text and tested at every CER. b Models retrained at the test CER (the char-CNN and ByT5-small at 5, 10, 15, 20 and 30% only). Both panels share the legend and the axes

Figure 3: T2 dating: mean absolute error in years (lower is better) of every model against CER. a Models trained on clean text. b Models retrained at the test CER (the char-CNN and ByT5-small at 5, 10, 15, 20 and 30% only). Both panels share the legend and the axes

Figure 4: T3 documentary versus literary: macro-F1 of every model against CER, with the line-length-only shortcut. a Models trained on clean text. b Models retrained at the test CER (the char-CNN and ByT5-small at 5, 10, 15, 20 and 30% only). Both panels share the legend and the axes

Figure 5: T4 search: document-level recall@20 of the ranking methods over all 2,534 queries against CER. The search methods need no training, so there is a single panel

Task	As is	Retrained	Also depends on
Doc. vs literary	20%	20%	$\geq$ 50 letters
Document type	7.5–12.5%	15%	$\geq$ 100 letters
Search, ranking	5%	–	query length (20% at 10+ letters)
Search, exact filter	5%	–	recall falls with CER, precision holds
Dating	3%	5%	text length; 50-year bar unmet

Table 3: CER targets per task, read from the per-task results of Sect. 8 and the per-model thresholds of Appendix 13 . “As is” is the highest CER at which a model built on editions keeps 90% of its performance; “retrained” the same for a model trained on degraded editions

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

EpiDoc element	Count	In the HTR view
supplied @reason=lost	721,518	removed, gap token
supplied @reason=omitted	17,257	removed, no gap (never written)
gap	797,539	gap token
choice : orig / sic vs reg / corr	143,904	orig / sic kept (what the scribe wrote)
expan with ex	903,536	written letters kept, expansion dropped
abbr	37,113	kept

Appendix

Table 1: What counts as ink: how EpiDoc elements of the edition are rendered in the HTR view. Counts are element occurrences over the 72,623 Greek editions of DDbDP and DCLP

Task	Model	Clean score	$c_{95}$ clean	$c_{90}$ clean	$c_{95}$ / $c_{90}$ retrained
T1	TF-IDF + LR	0.909 (0.895–0.923)	10	12.5	7.5 / 15
T1	fastText	0.857 (0.839–0.873)	5	7.5	5 / 12.5
T1	char-CNN	0.811 (0.793–0.829)	5	10	5 / 10
T1	ByT5-small	0.825 (0.808–0.843)	3	5	0 / 5
T2	fastText	55 y (52–58)	1	3	1 / 3
T2	char-CNN	62 y (60–64)	2	3	0 / 0

Appendix

Table 1: Per-model thresholds (% CER) on the headline metric. Clean: point estimate and 95% bootstrap interval at 0% CER. Retrained char-CNN and ByT5 stop at 30% CER, so their retrained thresholds cannot exceed it

Figure 1: Lost lines crossed with CER: 0, 10, 20 and 30% of the text-bearing lines dropped, models trained on clean text. a T1 document type (TF-IDF + LR, macro-F1). b T2 dating (TF-IDF + ridge, MAE in years, lower is better). c T3 documentary versus literary (TF-IDF + LR, macro-F1). d T4 search (min-edit ranking, document-level recall@20). All four panels share the legend

Figure 1: Performance by fragment size at 0, 10, 20 and 30% CER, models trained on clean text. Bin sizes are in the released result tables; bins with fewer than 30 units are not plotted. a T1 document form (TF-IDF + LR) by letters per document. b T2 dating error (char-CNN) by letters per document. c T3 documentary versus literary (TF-IDF + LR) by letters per document. d T4 recall@20 (min-edit ranking) by letters per query

Explore similar work

Sep 18, 2026cs.CL

Apollo Restore: A Foundation LLM for Historical Greek Optimized for Fill-in-the-Middle Restoration of Ancient Greek Texts

We present Apollo Restore, a 24-billion-parameter large language model for restoring lacunae---physical gaps---in fragmentary Ancient Greek texts. Fine-tuned from Mistral Small with a fill-in-the-middle objective, Apollo Restore reconstructs missing spans without requiring oracle knowledge of their length. To our knowledge, it is the first large-scale decoder model for historical Greek, and the first for any ancient Mediterranean language. Evaluated as in prior work, on short gaps of up to ten characters, Apollo Restore places the correct restoration among its top twenty candidates for 80.6%/54.6%/61.0% of documentary-papyrus, literary-papyrus, and stone-inscription lacunae, exceeding the strongest published models by

1.6\times

2.6\times

1.4\times

. Prior evaluation protocols, however, inflate scores through a bias toward trivially short gaps; under a length-balanced metric Apollo Restore's advantage over the strongest published models grows to

2.3\times

3.5\times

1.6\times

and degrades gracefully, even given incorrect length hints. In a blind study, 20 expert papyrologists, epigraphists, and philologists strongly preferred Apollo Restore to the strongest baseline and judged its performance at least as good as human restorations in 77% of cases. Apollo Restore also improves the published reading of PHerc. 1667---a papyrus roll carbonised in the eruption of Vesuvius in 79 CE and digitally unrolled and edited after Apollo Restore's training data was compiled. Apollo Restore is an output of the Decoding Antiquity initiative to build specialized LLMs for historical languages and manuscripts, led by the Austrian Academy of Sciences.

Hope McGovern, Anna Dolganov, Samuel Belkadi +3

Mistral AI · Austrian Academy of Sciences · Reply +1

Greek Historical Manuscripts Restoration Decoding

Ask PDF

Jun 23, 2026cs.CV

TrOCR for Medieval HTR: A Systematic Ablation Study with Cross-Dataset Validation

Fine-tuning transformer-based handwritten text recognition (HTR) models on medieval manuscripts is challenging because these models are pre-trained on modern text and must adapt to a very different visual domain. This paper studies how three controllable fine-tuning choices (contrast normalization, data augmentation, and layer freezing) affect recognition accuracy when adapting TrOCR to small historical datasets. We run controlled experiments on a 13th-century Italian manuscript (I-CT 91 "Cortonese") and replicate the same experimental grid on the public READ-16 benchmark as robustness evidence. On Cortonese, our best configuration achieves 8.03% character error rate (CER). Statistical comparisons across 13 configurations show that freezing up to three encoder layers or six decoder layers does not significantly harm accuracy, while deeper freezing becomes progressively detrimental. Removing contrast normalization (CLAHE) yields 7.84% CER, comparable to a domain-specialized baseline, suggesting strong optimization can reduce reliance on image preprocessing. Cross-dataset validation on READ-16 shows that decoder freezing thresholds transfer more robustly than encoder thresholds, and combined freezing strategies require dataset-specific re-validation. Finally, we use Grad-CAM gradient attributions and decoder cross-attention maps to diagnose error patterns and failure modes revealed by the ablations. Source code is available at https://github.com/LaudareProject/TrOCR-analysis

Sachin Sharma, Michele Flammini, Federico Simonetta

GSSI – Gran Sasso Science Institute L’Aquila, Italy

Handwritten Text Recognition Historical Manuscripts Maestro Dataset Frozen Backbone Character Error Rate Ablation+2

Ask PDF

Aug 7, 2026cs.CL

Stoicheia: Character-Level Masked Diffusion for Ancient Greek Textual Restoration, Parsing, and Metrical Scansion

We introduce Stoicheia, a 405M-parameter character-level masked-diffusion encoder for Ancient Greek whose input factors into five aligned, independently maskable planes: letters, word and sentence boundaries, diacritics, capitalization, and punctuation. A single backbone can therefore restore lacunae, re-segment, accentuate, and punctuate unspaced text without task-specific retokenization. We pretrain it on an open, revision-pinned corpus of 380M words and release eleven checkpoints: ten rotated, decontaminated folds, guaranteeing that for any given literary passage at least one released model has never seen its text, and one with no exposure to documentary texts. Three experiments - reconstruction of damaged inscriptions and papyri, morphosyntactic tagging and dependency parsing, and macronization with metrical scansion - each carry a matched random-initialization control, isolating what character-level diffusion pretraining contributes: 5.6 CER points on inscription reconstruction, 12.9 LAS on parsing, and 6.0 points of balanced accuracy on macronization. On Ithaca's own test split, with identical frozen samples and strict scoring, Stoicheia reduces character error relative to both prior state-of-the-art systems, from 24.6 (Ithaca) and 23.5 (its 2025 Aeneas-framework successor) to 15.5, and raises top-1 accuracy from 63.0 and 64.0 to 74.5.

Eric Cullhed, Albin Thörn Cleland

Department of Linguistics and Philology · Uppsala University · Sweden +2

Greek Character Error Rate Diffusion Language Models Document Parsing

Ask PDF