Recovering obscured text in palimpsests, i.e. the manuscripts where the original writing (under-text) was erased and overwritten for reuse, is vital for preserving lost information. Due to the nature of the material, such a task can be highly challenging and time-consuming. This paper presents an approach towards automated palimpsest handwritten text modeling for under-text recovery. We investigate generative models, including Generative Adversarial Networks (GANs) and diffusion models, and design a custom denoising diffusion model that learns structured noise patterns to reconstruct the under-text. The recovered text is subsequently processed by a fine-tuned, attention-based handwritten text recognition network. Furthermore, we investigate a language-guided recognition framework that leverages textual class descriptions as informative priors, enabling the text recognition model to distinguish meaningful features from noise. This is achieved by grounding task-specific descriptions, phrased in words or sentences, through visual attention maps generated by pretrained vision-language models like CLIP (Contrastive Language-Image Pre-training). To facilitate future research and benchmarking, we release new datasets for synthetic palimpsests (currently scarce), CLIP-compatible text descriptions, and pre-trained models with code.
Figures & tables
Figure 1: Architecture of our proposed framework, consisting of two main stages: (i) Under-text separation , where a trained generative model is used to recover the target under-text from the palimpsest. (ii) Language-guided HTR , where an HTR model is trained using a standard CE loss between the predicted text and the ground truth. To enhance recognition, we introduce a language-guided supervision strategy, which aligns the HTR model’s Grad-CAM visual feature maps with text-image alignments derived from CLIP via an activation loss. The example palimpsest image originates from the Synthetic Saint Gall dataset, and the example under-text image is a reconstructed image using cDDM2.
Figure 2: Combined and under-text patches from the processed synthetic Georgian dataset.
Dataset
MNIST
Saint Gall
Method
WAcc (%) ↑
WAcc (%) ↑
CER (%) ↓
NormED ↓
AHTR
99.34 ( ± 0.05)
94.64 ( ± 0.51)
1.26 ( ± 0.12)
0.0127 ( ± 0.0013)
G-AHTR
99.37 ( ± 0.06)
94.40 ( ± 0.37)
1.31 ( ± 0.10)
0.0131 ( ± 0.0009)
Table 1: Baseline recognition performance on the original MNIST and Saint Gall datasets, reported in WAcc(%) for MNIST and WAcc(%), CER(%), NormED for Saint Gall. Results are averaged over 10 runs with standard deviation ( ± ).
Dataset
Synthetic MNIST
Synthetic Saint Gall
Synthetic Georgian
Methods
SSIM ↑
PSNR ↑
FID ↓
SSIM ↑
PSNR ↑
FID ↓
SSIM ↑
PSNR ↑
FID ↓
Starynska et al. (2021)
-
17.781
-
-
-
-
-
-
-
Jampour (2025)
-
-
-
-
-
-
-
-
144.68*
Pix2pix
0.9383
23.242
3.849
0.9855
27.771
11.066
0.9913
37.619
20.367
cDDM1
0.8879
21.634
20.054
0.9804
26.238
21.285
0.9509
27.574
49.942
cDDM2
0.9419
23.238
3.837
0.9939
32.753
19.678
0.9750
31.384
31.055
Table 2: Text separation results on synthetic palimpsest datasets using Pix2pix and cDDMs. Results from related works are included for comparison. *Note that Jampour (2025) reported three document-level FID values, we therefore use their median in our table.
Dataset
Synthetic MNIST
Synthetic Saint Gall
Methods
Scenario
WAcc (%) ↑
WAcc (%) ↑
WER (%) ↓
CER (%) ↓
Norm ED ↓
Starynska et al. (2021)
91.90
-
-
-
-
AHTR
Overlaid
97.71 ( ± 0.12)
79.63 ( ± 1.48)
20.37 ( ± 1.48)
5.75 ( ± 0.43)
0.0612 ( ± 0.0054)
Pix2pix
96.83 ( ± 0.12)
62.25 ( ± 1.87)
37.75 ( ± 1.87)
12.40 ( ± 0.68)
0.1430 ( ± 0.0075)
cDDM2
97.83 ( ± 0.11)
91.70 ( ± 0.73)
8.30 ( ± 0.73)
1.95 ( ± 0.25)
0.0210 ( ± 0.0030)
G-AHTR
Overlaid
97.68 ( ± 0.15)
78.30 ( ± 1.38)
21.70 ( ± 1.38)
6.18 ( ± 0.48)
0.0660 ( ± 0.0055)
Table 3: Comparison of the average performance of AHTR and G-AHTR on Synthetic MNIST and Synthetic Saint Gall over 10 runs with standard deviation ( ± ). Results are shown for direct recognition from the “overlaid” palimpsest images and for recognition after text separation using Pix2pix and cDDM2. cDDM1 is omitted due to its consistently lower separation performance compared to cDDM2.
Figure 3: Examples of text separation results from different models, shown alongside the input palimpsest image and the ground truth under-text. The examples are drawn from the Synthetic MNIST, Synthetic Saint Gall, and Synthetic Georgian datasets (top to bottom).
AHTR
G-AHTR
Table 4: Comparison of activation maps on example under-text “itaq” obtained from AHTR and G-AHTR during training, on every third epoch (i.e. epochs 1, 4, 7, 10, 13 and 16). The activation maps from G-AHTR show clear signs of convergence toward the CLIP map.
Figure 4: Under-text image from cDDM and the activation map from CLIP for “itaq”.
Original label
AHTR
G-AHTR
impossibile
impossibilitie
impossibile
solitudinis
solitudiiis
solitudinis
solatium
solattuum
solatium
construens
construeninininininininin
construeni
linea
lineama
linea
Table 5: Example predictions from Synthetic Saint Gall (in Latin) show G-AHTR is less likely to hallucinate than AHTR, highlighting the advantage of language-guided HTR.
Department of Classical Philology and Italian Studies, University of Bologna · Centre for Digital and Computational Humanities, University of Copenhagen