Handwritten Urdu is hard to recognize because the cursive Nastaliq script has no fixed baseline and no clear character boundaries. We benchmark six systems on the Urdu Nastaliq Handwritten Dataset (UNHD): Tesseract (default configuration), a script-specific recognizer trained on printed Urdu (UTRNet-Large), two open-source vision-language models (VLMs), and two commercial VLMs. All six systems are evaluated without fine-tuning on 2,465 single-line images (Part A), and the four VLMs receive the same fixed prompt (decoding uses temperature 0 where configurable, except for Claude Sonnet 5, whose decoding cannot be configured). Both commercial VLMs reach lower error than the printed-Urdu recognizer and the open-source VLMs. We do not compare against handwriting-trained recognizers, which would likely do better in-domain; we characterize zero-shot behavior, not state of the art. We also describe three distinct failure modes among the ranked systems. In one of them, a commercial model explicitly declines to transcribe a few very hard-to-read images instead of guessing. In a second study (Part B), VLMs correct the output of a separate OCR system using both the source image and the noisy text. Once separated from each model's own reading ability, the benefit is small on Tesseract's output (0.7% to 7.4%) and larger (17.9% and 21.9%) only for the two commercial correctors given UTRNet's text. It appears to depend on how legible the supplied text already is, and a weaker corrector can make a more legible text worse.
Figures & tables
Fig. 1: Part A pipeline.
System
Category
CER
95% CI (CER)
WER
Empty / refusal
Tesseract
Traditional OCR
0.7215
[0.7145, 0.7286]
1.1793
5.1%
Qwen2.5-VL-7B
Open-source VLM
0.6464
[0.6343, 0.6592]
0.9495
0%
Qwen3-VL-8B
Open-source VLM
0.4861
[0.4737, 0.4990]
0.8380
0%
UTRNet-Large
Trained (script-specific)
0.3084
[0.3017, 0.3153]
0.7827
0%
Claude Sonnet 5
Commercial VLM
0.2656
[0.2550, 0.2769]
0.5225
0.20%
Gemini 3.5 Flash-Lite
Commercial VLM
0.1718
[0.1633, 0.1805]
0.3772
0%
TABLE I: Part A results ( N=2,465 ). CER and WER are diacritic-sensitive. CIs are 10,000-resample paired bootstrap intervals.
Failure mode
Seen in
Description
Rate
Empty output
Tesseract
Likely full-page mode on single lines (single-line mode untested)
5.1%
Repetition loop
Qwen2.5-VL, Qwen3-VL
Phrase repeated until the generation cap
1.66% / 0.81%
Refusal on hard images
Claude Sonnet 5
Explicit decline to transcribe
0.20%
None observed
Gemini 3.5 Flash-Lite
No empty or refused output
0%
TABLE II: Failure modes and their rates in Part A.
Corrector
Input
CER
Reduction vs. input (%) [95% CI]
Reduction vs. own PtA (%) [95% CI]
Qwen2.5-VL-7B
Tesseract
0.5984
17.1 [15.0, 19.0]
7.4 [5.0, 9.8]
Qwen2.5-VL-7B
UTRNet
0.3363
−9.0 [ −11.1 , −7.2 ]
48.0 [46.5, 49.5]
Qwen3-VL-8B
Tesseract
0.4829
33.1 [31.3, 34.7]
0.7 [ −2.1 , +3.3 ]
Qwen3-VL-8B
UTRNet
0.2963
3.9 [2.7, 5.2]
39.0 [37.4, 40.7]
Gemini 3.5 Flash-Lite
Tesseract
0.1613
77.6 [76.5, 78.7]
6.1 [3.9, 8.3]
Gemini 3.5 Flash-Lite
UTRNet
0.1410
54.3 [52.4, 56.1]
17.9 [15.7, 20.2]
TABLE III: Part B results. CER reduction vs. the input source and vs. the corrector’s own Part A CER (PtA), with 95% bootstrap CIs. Sonnet 5 rows exclude unusable empty outputs ( n=2,464 and 2,463 ).
Fig. 2: CER reduction vs. the input source and vs. each corrector’s own Part A CER.
Tesseract quality
n
Qwen2.5-VL
Qwen3-VL
Gemini
Claude Sonnet 5
Decent (CER ≤ 0.5)
266
+31.7 [27.0, 36.3]
+1.8 [ −9.0 , 11.3]
+14.0 [7.1, 20.4]
+7.7 [0.7, 14.3]
Bad (0.5–0.9)
1849
+5.9 [3.0, 8.6]
+0.9 [ −2.3 , 4.0]
+6.5 [4.4, 9.3]
+2.6 [ −0.1 , 5.5]
Very poor (CER > 0.9)
350
−0.4 [ −7.4 , 6.0]
−1.0 [ −8.5 , 5.8]
+1.8 [ −3.5 , 6.7]
+3.3 [ −5.5 , 14.8]
TABLE IV: Correction gain by Tesseract input legibility, as percent CER reduction relative to each corrector’s own Part A CER (95% bootstrap CIs).