Dataset distillation (DD) replaces large training datasets with smaller synthetic ones to reduce computational costs. While traditionally used for classification, we present, to our knowledge, the first DD study for line-level handwritten text recognition (HTR). Unlike classification tasks, which use discrete labels, HTR employs unaligned sequences from a virtually infinite output space as labels. To adapt classification-based DD methods to this scenario, we define text pseudo-classes by clustering transcriptions based on character n-grams. We adapt gradient matching (GM), distribution matching (DM) and trajectory matching (MTT) to a Convolutional Recurrent Neural Network (CRNN) trained with Connectionist Temporal Classification (CTC). In this process, synthetic line images are generated by optimising only the pixels of real starting lines (either randomly chosen or selected as text medoids), while keeping their original transcriptions fixed. Across four datasets (IAM, George Washington, READ,2016, LAM) and three budgets (1--10,% of the training lines), no distilled set outperforms its real initialisation lines. Random lines prove better than medoids, and further optimisation only degrades recognition. Methodologically, DM lowers its objective without benefit, while GM and MTT barely move theirs. We turn these findings into guidelines for evaluating data compression in HTR.
Figures & tables
Dataset
Language
Period
Writers
Train
Val
Test
∣Σ∣
K (1/5/10 %)
IAM [ 20 ]
English
modern
657
6,161
1,840
1,861
79
62/308/616
GW [ 10 ]
English
18th c.
2
357
68
68
70
4/18/36
READ 2016 [ 25 ]
German
1470–1805
several
8,366
1,037
1,134
90
84/418/835
LAM [ 4 ]
Italian
1691–1750
1
19,829
2,470
3,523
89
198/991/1983
Table 1: Datasets: lines per split as used ( Sec. 4.1 ; LAM train excludes one line too long for CTC to align), alphabet size ∣Σ∣ , and subset size K per budget.
Random lines
Text medoids
Dataset
Budget ( K )
Random
0-iter
DM
GM
MTT
Medoids
0-iter
DM med
GM med
IAM
1 % (62)
48.2 ± 3.6
47.5 ± 1.2
47.6 ± 6.2
48.8 ± 3.2
48.2 ± 4.4
46.5 ± 2.7
46.3 ± 1.0
47.4 ± 1.1
46.4 ± 3.2
5 % (308)
28.3 ± 1.9
28.4 ± 0.5
27.9 ± 0.2
28.2 ± 0.5
29.0 ± 2.7
30.1 ± 2.7
29.4 ± 1.5
29.5 ± 1.5
29.4 ± 0.5
10 % (616)
23.8 ± 2.0
23.7 ± 0.8
23.7 ± 0.8
23.8 ± 1.7
23.5 ± 1.7
24.2 ± 1.6
24.3 ± 0.7
24.6 ± 1.2
24.0 ± 0.6
GW
1 % (4)
77.1 ± 2.4
77.5 ± 1.3
76.9 ± 2.0
76.0 ± 3.0
76.7 ± 2.3
76.8 ± 2.7
77.1 ± 1.5
78.0 ± 2.5
77.5 ± 2.4
5 % (18)
53.5 ± 14.8
47.0 ± 5.5
49.3 ± 7.6
49.9 ± 9.1
53.8 ± 11.6
55.6 ± 4.7
54.6 ± 4.4
52.2 ± 2.6
56.8 ± 4.2
Table 2: Test CER (%, mean ± 95 % CI; n=3 , GW n=5 ; one decimal). Each block starts from the same K real lines: the first column trains on them as images (the selection baseline), 0-iter trains on the same lines as tensors without optimisation (the control of the DD columns), and the DD columns optimise their pixels. Random: a different subset per seed. Nothing is bold: tested differences are in Tab. 3 . A single random subset fixed across seeds is reported in the supplement.
Figure 1: Test CER against budget (mean, 95 % CI; points offset for legibility). Random: one subset per seed; DM, GM and MTT start from the same random lines. Dashed: full training pool.
A − B
p
Holm p
A better
A worse
Primary (paired, own init)
DM − 0-iter
0.97
1
1
0
GM − 0-iter
1
1
1
0
MTT − 0-iter
0.151
0.757
1
0
DM med − 0-iter
0.677
1
1
0
GM med − 0-iter
0.733
1
0
1
Table 3: Comparisons over the 12 cells (test): Wilcoxon p on per-cell mean differences, Holm-corrected per family, and cells whose 95 % CI excludes 0 with A better or worse. DM, GM, MTT: from the random lines; DM med , GM med : from the text medoids; 0-iter: the starting lines of the method without optimisation; Random: one subset per seed.
Figure 2: Difference in test CER between each distilled set and its starting lines trained without optimisation (paired by seed; mean and 95 % CI). Filled: DM, GM and MTT from random lines; open: DM med and GM med from the medoids. Negative: distillation helps.
Figure 3: Optimisation strength (one seed, validation): change in CER with respect to the starting lines against pixel drift (relative ℓ2 , log scale) for DM med and GM med (batch mode; U≈30,300,3000 updates per line along each line) and MTT, by pixel step. Moving the pixels away from the starting lines increases the error; step 10 collapses the recognizer.
Figure 4: An IAM line ( K=308 ) at its starting point and after DM med in batch mode with increasing optimisation strength (left), and its difference to the starting line (right). Drift and validation CER (one seed) refer to the whole buffer.
Handwritten Text Recognition (HTR) models trained on synthetic handwriting often struggle to generalize to real text, and existing adaptation methods still require real samples from the target domain. In this work, we tackle the fully zero-shot synthetic-to-real generalization setting, where no real data from the target language is available. Our approach learns how model parameters change when moving from synthetic to real handwriting in one or more source languages and transfers this learned correction to new target languages. When using multiple sources, we rely on linguistic similarity to weigh their contrubition when combining them. Experiments across five languages and six architectures show consistent improvements over synthetic-only baselines and reveal that the transferred corrections benefit even languages unrelated to the sources.
Carlos Garrido-Munoz, Aniello Panariello, Silvia Cascianelli +4
University of Alicante, Spain · University of Modena and Reggio Emilia, Italy
Handwritten Text Recognition (HTR) is computationally imbalanced in two ways: most image pixels are background, and many width-axis sequence positions are blank-dominated. This creates a mismatch for Spiking Neural Networks (SNNs): handwriting is observed as a static image, whereas spiking computation unfolds over timesteps. We propose Spike-HTR, a hybrid spiking recognizer that controls both the number of spiking steps and the number of width positions processed by the deep sequence mixer. To make a static image suitable for short-horizon spiking inference, InkCoder converts it into a coarse-to-fine input stream, where early steps cover broad stroke regions and later steps emphasize sharper stroke details. To reduce sequence computation, a CTC-guided length reducer keeps likely character or uncertain positions and compresses long blank-dominated stretches before deep mixing. With T=2, Spike-HTR trains only on target data, decodes without language models or lexicons, and reaches validation/test CERs of 3.5/5.4, 2.3/2.5, and 4.2/3.9 on IAM, LAM, and READ2016. Codes are available at https://github.com/QomolangmaH/SpikeHTR.
Xiubo Liang, Jinxing Han, Yuke Li +3
School of Software Technology, Zhejiang University, Ningbo, China
Fine-tuning transformer-based handwritten text recognition (HTR) models on medieval manuscripts is challenging because these models are pre-trained on modern text and must adapt to a very different visual domain. This paper studies how three controllable fine-tuning choices (contrast normalization, data augmentation, and layer freezing) affect recognition accuracy when adapting TrOCR to small historical datasets. We run controlled experiments on a 13th-century Italian manuscript (I-CT 91 "Cortonese") and replicate the same experimental grid on the public READ-16 benchmark as robustness evidence. On Cortonese, our best configuration achieves 8.03% character error rate (CER). Statistical comparisons across 13 configurations show that freezing up to three encoder layers or six decoder layers does not significantly harm accuracy, while deeper freezing becomes progressively detrimental. Removing contrast normalization (CLAHE) yields 7.84% CER, comparable to a domain-specialized baseline, suggesting strong optimization can reduce reliance on image preprocessing. Cross-dataset validation on READ-16 shows that decoder freezing thresholds transfer more robustly than encoder thresholds, and combined freezing strategies require dataset-specific re-validation. Finally, we use Grad-CAM gradient attributions and decoder cross-attention maps to diagnose error patterns and failure modes revealed by the ablations. Source code is available at https://github.com/LaudareProject/TrOCR-analysis