Handwritten text recognition (HTR), particularly when applied to historical documents, remains challenging due to the scarcity of annotated data and the substantial domain shifts caused by variations in handwriting styles, document degradation, acquisition conditions, and linguistic characteristics. While large amounts of unlabeled target-domain data are often available, effectively exploiting them requires adaptation strategies tailored to the specific requirements of HTR. We propose DRA-HTR, an unsupervised generative domain adaptation pipeline specifically designed for HTR and built upon the DRANet framework. The proposed approach combines source-domain pretraining, unsupervised adaptation to the visual characteristics of the target domain, relying on the generation of target-style source samples for task-specific training. We further investigate the integration of post-HTR correction to complement visual adaptation with language adaptation, relying on large language models, which we introduce as DRAL-HTR. The proposed framework is evaluated across seven HTR datasets spanning four languages, using a single set of hyperparameters for all source-target combinations. The recognition model achieves state-of-the-art performance on four datasets and competitive results on the remaining three. In addition, the proposed DRA-HTR establishes a new state-of-the-art for domain adaptation across all seven datasets and substantially improves overall robustness and training stability compared to the original DRANet framework: it reduces the Character Error Rate (CER) in average by 13.52 percentage points, and the average standard deviation decreases from 7.23 to 1.11 percentage points. Finally, post-HTR correction further reduces the CER by 2.16 percentage points.
Figures & tables
Figure 1: Overview of the proposed domain-adaptive HTR pipeline. Step (a): supervised training of the HTR model on labeled source-domain data. Step (b): unsupervised visual DA using source and unlabeled target data to generate samples in the target style. Step (c): training of the HTR model using the generated target-style images, using their corresponding source transcriptions. Step (d): post-HTR correction using Qwen to refine the recognized text by leveraging linguistic context, correcting recognition errors, and improving transcription consistency.
Figure 2: Overview of the proposed DRA-HTR architecture. The overall content–style adaptation principle follows Imbert et al. (2026) , while the architectures of its main components are redesigned in this work for HTR. The encoder ( E ) extracts visual features from source and target images, which are decomposed by the separator ( S ) into content and style representations. The generator ( G ) recombines source content with target style to generate target-style handwritten text lines while preserving their textual content. The generated images are constrained using adversarial, reconstruction, latent-consistency, and perceptual objectives. DA is performed only once during training to adapt the HTR model to the target domain. After adaptation, the model retains its original inference time.
Figure 3: Examples of handwritten document images from the datasets used in our experiments. The examples illustrate the diversity of handwriting styles, document layouts, and paper appearance across the different datasets. A cropped text line is shown below each document image to show the writing style.
Train
Validation
Test
Total Lines
Language
Dataset
Pages
Lines
Pages
Lines
Pages
Lines
English
IAM
747
6,482
116
976
336
2,915
10,373
Bentham
350
9,198
50
1,415
33
860
11,473
German
READ 2016
350
8,367
50
1,043
50
1,140
10,550
Norwegian
NorHand 72702
292
6,069
37
792
37
753
7,614
NorHand 331861
319
6,053
40
784
40
766
7,603
Table 1: Datasets used in the experiments. The number of pages and text lines is reported for each training, validation, and test set.
Parameter
Value
Learning rates
Generative components
2×10−4
HTR model
4×10−4
Separator
2×10−4
Loss weights
Generator loss
0.1
Table 2: Hyperparameters used for training the DA framework.
Dataset
IAM
Bentham
READ 2016
NorHand
NorHand
Riksarkivet
Riksarkivet
72702
331861
BK 1
GBG 2
SOTA CER (%)
3.36 Gu et al. (2025)
2.37 Chinthaginjala et al. (2024)
3.80 Coquenet (2026)
1.61 Pathirage et al. ()
3.09 Pathirage et al. ()
3.40 Pathirage et al. ()
2.96 Pathirage et al. ()
Our HTR model
3.83 ± 0.15
2.12 ± 0.10
3.55 ± 0.02
1.54 ± 0.07
3.65 ± 0.15
3.55 ± 0.26
2.58 ± 0.18
Table 3: Comparison of SOTA HTR results with our HTR models on each dataset. Recognition is evaluated on the test set with Character Error Rate (CER) ( ↓ )
Source \ Target
IAM
Bentham
READ 2016
NorHand
NorHand
Riksarkivet
Riksarkivet
72702
331861
BK 1
GBG 2
(a) HTR model
IAM
3.83
± 0.15
13.23
± 0.60
77.48
± 1.42
36.99
± 1.41
55.35
± 0.88
62.39
± 0.59
55.96
± 0.69
Bentham
29.23
± 0.85
2.12
± 0.10
74.09
± 0.90
33.17
± 0.33
59.48
± 0.24
61.01
± 0.56
49.88
± 0.45
READ 2016
82.95
± 1.91
69.35
± 3.35
3.55
± 0.02
61.70
± 2.84
70.65
± 0.98
60.57
± 1.30
75.58
± 2.43
NorHand 72702
64.60
± 2.71
33.54
± 0.92
71.61
± 0.36
1.54
± 0.07
36.78
± 0.61
50.00
± 0.31
44.08
± 0.33
Table 4: Comparison of CER (%) across seven HTR datasets. Rows indicate the source training dataset and columns indicate the target evaluation dataset.
Figure 4: Three examples of source-to-target style adaptation: (a) Bentham to NorHand 331861, (b) Bentham to READ 2016, and (c) Riksarkivet BK 1 to IAM. In each example, the top row shows a source-domain image, the middle row shows the generated image in the target-domain style, and the bottom row shows a target-domain image.
E
P-GAN
S&G
Delay
PC
CER (%)
(a)
R
22.52
(b)
T
16.57
(c)
L
18.64
(d)
T
✓
15.53
(e)
T
✓
20.72
(f)
T
✓
✓
16.92
Table 5: Ablation study of the DA components. E: ConvNeXtV2 encoder (T: Tiny, L: Large) or R for convolutional component of ResNet-34; P-GAN: PatchGAN discriminator; S&G: proposed separator and generator; Delay: delayed HTR model training followed by the integration of the task loss; PC: post-HTR correction. Row (a) corresponds to the prior DA method Imbert et al. (2026) , while row (h) represents the final DRAL-HTR pipeline.
Method
CER (%)
(a)
Baseline (no post-HTR correction)
13.53
± 0.40
(b)
CTC beam search
13.37
± 0.41
(c)
BERT scoring Shin et al. (2019)
12.77
± 0.39
(d)
n -gram Ramirez-Orta et al. (2022)
10.62
± 0.22
(e)
Gemini
13.02
± 0.30
(f)
Qwen Koynov et al. (2027)
11.65
± 0.52
Table 6: Comparison of post-correction methods for the HTR model adapted from READ 2016 to Bentham. For the Qwen ablation, NP denotes the new prompt, BS beam search, and NL neighboring lines.
Character classification and handwritten text recognition, particularly in specialized tasks, often suffer from limited labeled data, while a large amount of unlabeled data remains unused. To address this challenge, we propose a novel unsupervised domain adaptation-based pipeline to reduce the distribution gap between source and target domains. Our pipeline follows three main steps. In the first step, we pretrained an expert model on source data, to learn a strong general representation. In the second step, we perform unsupervised domain adaptation to leverage unlabeled target data, adapting the model to the specific characteristics of the target domain. In the third step, we trained the expert model on source data in the target style. By aligning the feature distributions between the source and target, our method enables the model to generalize effectively across diverse datasets. The unsupervised training increases the computation time only during training not in inference. We evaluate the pipeline on both character-level and historical handwritten line recognition tasks, demonstrating its flexibility. We establish a new state-of-the-art on the Safran-MNIST-DLS dataset defined in Track 2 of the DAGECC competition with 76.17% Macro F1-score and achieve competitive results on each of the specific datasets of READ 2018 without using the labels of the target data. Code is available at https://github.com/simon-corbi/DRANet-for-classification-and-htr
Florent Imbert, Simon Corbillé, Hui Han +1
Machine Learning Group, Lule˚a University of Technology, Lule˚a, Sweden.
Fine-tuning transformer-based handwritten text recognition (HTR) models on medieval manuscripts is challenging because these models are pre-trained on modern text and must adapt to a very different visual domain. This paper studies how three controllable fine-tuning choices (contrast normalization, data augmentation, and layer freezing) affect recognition accuracy when adapting TrOCR to small historical datasets. We run controlled experiments on a 13th-century Italian manuscript (I-CT 91 "Cortonese") and replicate the same experimental grid on the public READ-16 benchmark as robustness evidence. On Cortonese, our best configuration achieves 8.03% character error rate (CER). Statistical comparisons across 13 configurations show that freezing up to three encoder layers or six decoder layers does not significantly harm accuracy, while deeper freezing becomes progressively detrimental. Removing contrast normalization (CLAHE) yields 7.84% CER, comparable to a domain-specialized baseline, suggesting strong optimization can reduce reliance on image preprocessing. Cross-dataset validation on READ-16 shows that decoder freezing thresholds transfer more robustly than encoder thresholds, and combined freezing strategies require dataset-specific re-validation. Finally, we use Grad-CAM gradient attributions and decoder cross-attention maps to diagnose error patterns and failure modes revealed by the ablations. Source code is available at https://github.com/LaudareProject/TrOCR-analysis
Handwritten Text Recognition (HTR) models trained on synthetic handwriting often struggle to generalize to real text, and existing adaptation methods still require real samples from the target domain. In this work, we tackle the fully zero-shot synthetic-to-real generalization setting, where no real data from the target language is available. Our approach learns how model parameters change when moving from synthetic to real handwriting in one or more source languages and transfers this learned correction to new target languages. When using multiple sources, we rely on linguistic similarity to weigh their contrubition when combining them. Experiments across five languages and six architectures show consistent improvements over synthetic-only baselines and reveal that the transferred corrections benefit even languages unrelated to the sources.
Carlos Garrido-Munoz, Aniello Panariello, Silvia Cascianelli +4
University of Alicante, Spain · University of Modena and Reggio Emilia, Italy