Whether an intermediate stage of modern Arabic handwriting helps or hurts historical Arabic HTR is usually decided from one implementation and one comparison, too thin a basis for a claim either way. We test stability by running the same nominal ablation four times, letting the base checkpoint, encoder-freezing strategy, epoch budget, precision, and learning-rate schedule vary as they naturally did during development, while holding the normalization, scorer, and interval estimation fixed. Each run compares intermediate training on modern handwriting (KHATT) then fine-tuning on historical manuscripts (Muharaf) against fine-tuning on Muharaf directly. Across the four runs the estimated effect swings from -17.64 to +14.52 CER points and reverses sign. The two extremes are exactly the two runs with an identifiable confound (a fivefold lower learning rate in one; a checkpoint of undisclosed provenance in the other); the two clean runs land at -0.25 and +0.94, i.e. no effect. A tight interval from one implementation says nothing about the next. We then run a compute-matched experiment with identical budgets over three seeds: KHATT warm-up is +2.42 CER points worse than a matched same-domain control (95% interval [+0.60, +4.25]); the part of that gap specific to the handwriting domain is only about 0.6 points a small negative effect under this configuration, not a universal result. We release a SaudiHeritage-OCR package with the normalizer, interval scorer, a verified KHATT decoder, experimental manifests, VLM baselines, and an edition-alignment protocol, so the result can be checked independently. The Al-Mahd inscription line is held strictly out and is not offered as a benchmark.
Figures & tables
Figure 1: The central result. Four separately configured implementations: one project, one month, the same two corpora, one normalizer, one scorer, one question. Does a KHATT warm-up before Muharaf fine-tuning help? Each row runs from the single-stage result (hollow) to the staged result (filled); Δ CER is at right. Rows are sorted by Δ CER, matching Table 9 . The effects span 32.16 points and the sign flips. Read carefully, the spread is driven by the two rows with identifiable confounds. I1 lowers its target-stage learning rate fivefold in the staged arm, and I2 builds on a checkpoint of undisclosed provenance. I3 and I4 (black) agree on “no effect.” Viewed in isolation, individual rows would support materially different conclusions.
Source
Surface
Role
Lines
KHATT [13]
modern ink
warm-up
9,497 / 1,901
Muharaf [14]
historic ink
train + bench.
22,091 / 1,069 / 1,334
Al-Mahd (ours)
inscribed stone
test-only, qual.
1
Table 1: Data sources and roles. Only public corpora are used for training; Al-Mahd is never trained on. Counts are train / val / test where applicable.
Base checkpoint
Encoder
Epochs
Batch
Precision
Target-stage LR (M1 / M2)
I1
RayR1/trocr-base-arabic-hw
trainable
5
8×2
fp16
5×10−5 / 1×10−5
I2
RayR1/trocr-base-arabic-hw
trainable
8
16
fp16
5×10−5 / 5×10−5
I3
ViT-B/16-384 + AraBERT
frozen
15
16
fp32
5×10−5 / 5×10−5
I4
ViT-B/16-384 + AraBERT
frozen
15
8
fp32
3×10−5 / 3×10−5
I5
ViT-B/16-384 + AraBERT
frozen
†
8×2
fp16
5×10−5 (matched, all arms)
Table 2: TrOCR-family implementations, all scored under the single normalization policy of Sec. 2 and the same scorer. I1 to I4 are the four pilots. They are not a controlled replication but the ordinary spread of choices four implementers made, which is precisely what this study measures. I5 is the compute-matched follow-up reader (Sec. 4.5 , 5.1 ); unlike I1 to I4 it is step-budgeted (Table 6 ) with a single learning rate shared across all arms and stages, so its epoch and M1/M2 columns do not apply ( † ).
Source form(s)
Normalized to
Alef with hamza, madda, or wasl
bare alef
Alef maqsura
ya
Ta marbuta
ha
Waw-hamza
waw (bare)
Ya-hamza
ya (bare)
Standalone hamza
(removed)
Table 3: Orthographic unification applied by the normalizer (Sec. 2 ); NFKC, tatweel removal, diacritic stripping, and whitespace collapse are applied in addition, identically to references and predictions. Arabic glyphs are shown in transliteration for pdf L a T e X compatibility; a Xe L a T e X variant with native script is available.
Figure 2: What each level of uncertainty can see, on this study’s data: the width of I1’s paired line-level interval (1.22), the seed-to-seed spread within I1’s single-stage condition (0.94, the range across its three seeds), and the range across the four implementations (32.16), all in CER points. The quantity that dominates is the one standard reporting practice does not measure.
Split
Lines
Muharaf train (raw)
22,091
Muharaf validation
1,069
Muharaf test
1,334
↪ Arm-E warm-up slice (from train, seed =7 )
9,497
↪ Target split (remaining train, all arms)
12,594
Overlap, target split vs. E slice
0
Table 4: Matrix data splits. The arm-E warm-up slice is carved from Muharaf train once, before training, disjoint from the target split.
Arm
Warm-up source
Role
A+
Same-domain Muharaf
Compute-matched ctrl.
B
KHATT (modern hw)
Primary contrast
A0
None (target-only)
Practitioner ref.
E
Disjoint real-Muharaf
In-domain-real ctrl.
Table 5: Matrix arms. A+ is the compute-matched control: it spends the warm-up budget on additional target-domain Muharaf passes before the common target stage. E uses a separately carved, disjoint real-Muharaf slice, an in-domain data-source control. A0 omits the warm-up budget, is therefore not compute-matched, and is never the primary control (Sec. 4.5 ).
Setting
Value
Learning rate
5×10−5
LR warmup ratio
0.05
Weight decay
0.01
Per-device batch size
8
Gradient accumulation
2 (eff. batch 16)
Warm-up steps Nw (A+, B, E)
1,200
Table 6: Matrix training hyperparameters, identical across every arm and seed.
CER (%)
WER (%)
Run
raw
norm.
raw
norm.
A
84.46
78.59
113.85
111.39
B
95.14
80.71
114.68
111.42
Table 7: Zero-shot Qwen2.5-VL-7B-Instruct (4-bit NF4), 200-line Muharaf test samples. Two runs differing only in generation control: A caps generation at min(2× reference length, 80 tokens ) with repetition penalty 1.3 and no-repeat-3-gram; B caps at 2× reference length with repetition penalty 1.05.
Stage
Eval set
CER (%)
WER (%)
1 (KHATT)
KHATT val
32.53±0.77
75.36±0.76
2 (Muharaf)
Muharaf test
45.00±0.55
81.76±1.12
Table 8: Qwen2.5-VL + LoRA, staged KHATT → Muharaf, three seeds. Normalized scores, mean ± SD over seeds (population SD, n=3 ). 1,000-line training subset per stage, 200-line evaluation sample.
Figure 3: VLM-LoRA on Muharaf across three seeds. The seed-to-seed spread ( ±0.55 ) is far smaller than the line-level bootstrap width ( ≈5 points). Within one implementation the experiment behaves as the protocol assumes, which is why the instability in Fig. 1 cannot be dismissed as ordinary noise. Dashed line: seed mean 45.00±0.55 ; bars: line-level bootstrap 95% CI.
M1
M2
Δ CER
Warm-up
I2
45.86
28.22
−17.64
helps (contam.?)
I3
82.88
82.63
−0.25
no effect
I4
77.51
78.45
+0.94
no effect
I1
42.81
57.33
+14.52
hurts (LR confound)
Range across impl. 32.16 points, sign flips
Table 9: Normalized CER on the Muharaf test split for the single-stage (M1) and staged KHATT → Muharaf (M2) conditions, across four implementations. Δ is M2 − M1; negative means the warm-up helped. Rows are sorted by Δ CER, matching Fig. 1 . I1 reports means over 3 and 2 seeds respectively; the others are single runs. The sign is not stable, and the two extremes are the two confounded runs.
Arm
CER (%)
SD
95% CI
A+ (control)
79.16
1.44
[75.59,82.74]
B (KHATT)
81.59
0.91
[79.32,83.86]
A0 (no warm-up)
79.76
0.97
[77.36,82.17]
E (in-domain real)
80.95
0.55
[79.58,82.33]
Table 10: Matrix per-arm normalized CER on the Muharaf test split, mean over n=3 seeds with 95% CI. Absolute CER is high because the reader is trained from generic weights (Sec. 4.5 ); the matrix measures the contrast, not the state of the art.
Contrast
Δ CER
95% CI
Role
B − A+
+2.42
[+0.60,+4.25]
Primary contrast
A0 − A+
+0.60
[−0.84,+2.04]
Secondary (incl. 0)
E − A+
+1.79
[−3.10,+6.68]
Secondary (incl. 0)
B − E
+0.64
(not tested)
Domain-isolating
Table 11: Matrix contrasts vs. the control A+, seed-level paired interval. Positive Δ means the warm-up hurt. Seed-level means differ from the difference of the per-arm means in Table 10 in the third decimal; +2.42 is the mean of the per-seed differences.
Arm
MMD 2 image [95% CI]
JS (3-gram)
B (KHATT)
0.4361 [0.4263,0.4429]
0.2530
E (Muharaf slice)
≈ 0.0000 [0.0003,0.0010]
0.0167
Table 12: Measured domain distance to the target split (Sec. 4.6 ). B’s gap is large on both axes; E’s is near zero, a sanity check on the metric. The unbiased MMD 2 estimator is not constrained to be positive, so E’s point estimate can fall below the bootstrap interval of the resampled statistic; both are reported as computed.
Figure 4: Qualitative sample predictions from I1’s single-stage (M1) Muharaf checkpoint. Each row shows the manuscript line crop, the normalized reference (REF), the model prediction (PRED), and per-sample CER / WER. The six lines are a fixed test sample (seed 7) shared across every checkpoint in the study. Note the range within one model: 10.5% CER on a clean modern-looking hand against 72.2% on an ornate historical one, and WER at 100% on lines whose CER is under 40%, where the reader recovers characters but not whole words.
Figure 5: Readers under one normalization policy, against the published state of the art. The best and worst TrOCR configurations differ by tens of CER points on the same benchmark; both are configurations a reasonable implementer produced.