GlotOCR Bench: OCR Models Still Struggle Beyond a Handful of Unicode Scripts

Amir Hossein Kargaran; Nafiseh Nikeghbal; Jana Diesner; François Yvon; Hinrich Schütze

cs.CLApr 14, 2026

GlotOCR Bench: OCR Models Still Struggle Beyond a Handful of Unicode Scripts

Authors: Amir Hossein Kargaran, Nafiseh Nikeghbal, Jana Diesner, François Yvon, Hinrich Schütze

Organizations: LMU Munich · MCML · TU Munich · Sorbonne Université & CNRS, ISIR

Abstract

Optical character recognition (OCR) has advanced rapidly with the rise of vision-language models, yet evaluation has remained concentrated on a small cluster of high- and mid-resource scripts. We introduce GlotOCR Bench, a comprehensive benchmark evaluating OCR generalization across 100+ Unicode scripts. Our benchmark comprises clean and degraded image variants rendered from real multilingual texts. Images are rendered using fonts from the Google Fonts repository, shaped with HarfBuzz and rasterized with FreeType, supporting both LTR and RTL scripts. Samples of rendered images were manually reviewed to verify correct rendering across all scripts. We evaluate a broad suite of open-weight and proprietary vision-language models and find that most perform well on fewer than ten scripts, and even the strongest frontier models fail to generalize beyond thirty scripts. Performance broadly tracks script-level pretraining coverage, suggesting that current OCR systems rely on language model pretraining as much as on visual recognition. Models confronted with unfamiliar scripts either produce random noise or hallucinate characters from similar scripts they already know. We release the benchmark and pipeline for reproducibility. Pipeline Code: https://github.com/cisnlp/glotocr-bench, Benchmark: https://hf.co/datasets/cis-lmu/glotocr-bench.

Figures & tables

Figure 1: Acc@5 (see § 4 ) by script resource tier (high, mid, low). Performance drops sharply on low-resource scripts. Gemini 3.1 Flash-Lite leads (95.3%, 82.7%) but falls to 7.7%; others score <1%.

Tier	Scripts	Total	Median	Min	Max
High	1	1907	1907	1907	1907
Mid	9	814	59	29	323
Low	148	380	1	1	29
All	158	3101	1	1	1907

Table 1: Font availability per resource tier. Counts are per font family.

Model	High (1 script)			Mid (9 scripts)			Low (148 scripts)			Mean (across tiers)
Model	CER $\downarrow$	A@0 $\uparrow$	A@5 $\uparrow$	CER $\downarrow$	A@0 $\uparrow$	A@5 $\uparrow$	CER $\downarrow$	A@0 $\uparrow$	A@5 $\uparrow$	CER $\downarrow$	A@0 $\uparrow$	A@5 $\uparrow$
Gemini 3.1 Flash-Lite	$\mathbf{0.9}$	$\mathbf{86.0}$	$\mathbf{95.3}$	$\mathbf{3.0}$	$\mathbf{66.1}$	$\mathbf{82.7}$	$\mathbf{79.0}$	$5.0$	$\mathbf{7.7}$	$\mathbf{27.7}$	$\mathbf{52.4}$	$\mathbf{61.9}$
dots.mocr	$\underline{1.5}$	$\underline{82.5}$	$\underline{93.1}$	$6.0$	$\underline{57.0}$	$78.1$	$84.1$	$\underline{5.1}$	$\mathbf{7.7}$	$30.5$	$\underline{48.2}$	$\underline{59.6}$
dots.ocr	$1.6$	$80.4$	$91.8$	$\underline{5.0}$	$55.4$	$\underline{78.3}$	$\underline{82.6}$	$\mathbf{5.2}$	$\mathbf{7.7}$	$\underline{29.7}$	$47.0$	$59.3$
HunyuanOCR	$3.4$	$56.0$	$85.3$	$6.3$	$52.5$	$73.9$	$87.3$	$1.7$	$\underline{3.4}$	$32.3$	$36.8$	$54.2$
Qwen3-VL-8B	$2.0$	$72.8$	$89.5$	$7.2$	$47.6$	$67.1$	$89.8$	$0.3$	$0.8$	$33.0$	$40.3$	$52.4$

Table 2: GlotOCR Bench benchmark results by resource tier. Each tier result is the macro average over its scripts. A@0 and A@5 denote the fraction of predictions with CER

\leq 0

and

\leq 0.05

, respectively. Bold = best; underline = 2nd best

Figure 2: Acc@5 distributions for four scripts (Latin, Devanagari, Arabic, Cyrillic). Boxes correspond to models; each point is the score for one language within the script.

Panel of Figure 3 — Figure 3: Script Recognition and Hint-Guided OCR Analysis.

Figure 4: Acc@5 on clean vs. degraded images for the six best-performing models across high-, mid-, and low-resource tiers, with corresponding performance drops.

Model	Correct $\uparrow$	Hall. $\downarrow$	Silent	Artifact
dots.ocr	15.8	38.3	42.1	0 3.8
dots-mocr	16.1	50.2	16.3	17.4
FireRed-OCR	0 8.3	62.4	12.7	16.6
DeepSeek-OCR2	0 6.5	63.7	0 3.6	26.2
Hunyuan-OCR	11.5	68.2	0 0.0	20.3
PaddleOCR-VL-1.5	0 7.6	69.7	0 0.0	22.7

Table 3: Script-level error rates (%) per model, macro-averaged over scripts, ranked by cross-script hallucination (Hall.) rate. The four categories are mutually exclusive and sum to 100%.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 5 — Figure 5: Example images from GlotOCR Bench for Greek (Grek, Mid tier resource) and Aghwan (Aghb, Low tier resource), each shown in clean and degraded variants.

Script	$n$	Top-2 Out.Script	Script	$n$	Top-2 Out.Script	Script	$n$	Top-2 Out.Script
Lepc	100	Latn, Arab	Limb	100	Mtei, Tibt	Mand	100	Arab, Syrc
Modi	100	Deva, Thai	Mong	100	Hani, Shaw	Newa	100	Deva, Beng
Nkoo	100	Arab, Latn	Ogam	100	Latn, Hani	Olck	100	Latn, Sinh
Hung	100	Latn, Cyrl	Ital	100	Latn, Cyrl	Ougr	100	Arab, Mong
Phli	100	Hebr, Latn	Prti	100	Hebr, Tibt	Sarb	100	Latn, Tibt
Shrd	100	Deva, Guru	Dsrt	100	Shaw, Latn	Glag	100	Latn, Beng

Appendix

Table 4: Scripts for which all models obtain zero in both Acc@5 and ScriptAcc. These scripts are not identifiable by any model. Top-2 Out.Script shows which scripts appear most frequently in model outputs for these samples (avg. over all models).

Script	$n$	Top-2 Out.Script	Paddle-1.5 A@5/SA $\uparrow$	olmOCR2 A@5/SA $\uparrow$	Gemini-FL A@5/SA $\uparrow$	dots.mocr A@5/SA $\uparrow$	dots.ocr A@5/SA $\uparrow$	Hunyuan A@5/SA $\uparrow$	GPT-4.1 A@5/SA $\uparrow$
Latn	4000	Latn	79.8/99.2	90.5/99.6	95.3/99.7	93.1/99.8	91.8/99.5	85.3/99.5	83.2/99.7
Cyrl	400	Cyrl, Latn	78.0/97.8	71.2/98.2	88.8/99.2	83.5/99.2	86.0/99.2	86.8/98.5	69.8/98.8
Hani	400	Hani	60.8/99.2	78.2/100.0	85.5/99.8	86.5/100.0	81.5/99.2	83.0/99.5	55.0/99.5
Deva	400	Deva	1.0/99.8	71.5/100.0	93.5/100.0	86.0/98.5	90.2/100.0	79.8/100.0	65.8/100.0
Arab	400	Arab, Latn	21.0/99.8	31.0/99.8	58.8/99.8	62.0/97.5	63.2/98.2	38.2/97.5	39.5/99.8
Jpan	100	Jpan	96.0/100.0	98.0/100.0	98.0/99.0	92.0/100.0	88.0/100.0	99.0/100.0	98.0/100.0

Appendix

Table 5: A@5 (Acc@5)/SA (ScriptAcc) (%) per script. In order: PaddleOCR-VL-1.5; olmOCR-2; Gemini 3.1 Flash-Lite; dots.mocr; dots.ocr; HunyuanOCR; GPT-4.1.

Script	$n$	Top-2 Out.Script	Qwen3-8B A@5/SA $\uparrow$	GLM A@5/SA $\uparrow$	Rolm A@5/SA $\uparrow$	LightOn-2 A@5/SA $\uparrow$	DeepSeek-2 A@5/SA $\uparrow$	FireRed A@5/SA $\uparrow$	Nanonets-2 A@5/SA $\uparrow$
Latn	4000	Latn	89.5/99.7	89.5/99.6	89.6/99.8	89.8/99.8	76.2/98.2	83.6/99.5	88.6/99.8
Cyrl	400	Cyrl, Latn	79.0/99.2	26.2/98.2	75.2/98.8	68.8/95.5	57.8/94.2	59.2/95.0	63.7/98.2
Hani	400	Hani, Latn	78.0/100.0	82.0/100.0	68.8/99.8	26.2/98.8	48.5/100.0	66.8/99.5	46.5/100.0
Deva	400	Deva, Guru	70.8/100.0	0.2/100.0	70.2/100.0	65.2/96.0	28.5/93.8	46.5/99.0	54.2/99.8
Arab	400	Arab, Latn	36.8/99.8	0.2/99.8	32.8/99.8	30.0/99.2	28.5/99.0	13.0/89.8	29.2/99.8
Jpan	100	Jpan	95.0/100.0	92.0/100.0	93.0/100.0	83.0/100.0	80.0/100.0	95.0/100.0	92.0/100.0

Appendix

Table 6: A@5 (Acc@5)/SA (ScriptAcc) (%) per script. In order: Qwen3-VL-8B; GLM-OCR; RolmOCR; LightOnOCR-2; DeepSeek-OCR-2; FireRed-OCR; Nanonets-OCR2.

Explore similar work

OmniOCR: Generalist OCR for Ethnic Minority Languages
Feb 24, 2026Bonan Liu, Zeyu Zhang, Bingbing Meng +5Optical Character Recognition Low-Resource Language Processing
Southwest Minzu University · AI Geeks
KazakhOCR: A Synthetic Benchmark for Evaluating Multimodal Models in Low-Resource Kazakh Script OCR
Feb 17, 2026Henry Gagnier, Sophie Gagnier, Ashwin KirubakaranOptical Character Recognition Low-Resource Language Processing
Pittsford Sutherland High School · Edison Academy Magnet School
Can OCR-VLMs Read Devanagari? A Stress-Test Benchmark and Post-Correction Study
Jun 28, 2026Aditya Pratap SinghVLM Evaluation VLM Robustness

Feb 24, 2026J/K move · Enter open · S save

OmniOCR: Generalist OCR for Ethnic Minority Languages

Bonan Liu, Zeyu Zhang, Bingbing Meng, Han Wang+4

Southwest Minzu University · AI Geeks

Optical Character Recognition Low-Resource Language Processing Parameter-Efficient Fine-Tuning Low-Rank Adaptation

Open paper AskPDF arXiv

Optical character recognition (OCR) has advanced rapidly with deep learning and multimodal models, yet most methods focus on well-resourced scripts such as Latin and Chinese. Ethnic minority languages remain underexplored due to complex writing systems, scarce annotations, and diverse historical and modern forms, making generalization in low-resource or zero-shot settings challenging. To address these challenges, we present OmniOCR, a universal framework for ethnic minority scripts. OmniOCR introduces Dynamic Low-Rank Adaptation (Dynamic LoRA) to allocate model capacity across layers and scripts, enabling effective adaptation while preserving knowledge.A sparsity regularization prunes redundant updates, ensuring compact and efficient adaptation without extra inference cost. Evaluations on TibetanMNIST, Shui, ancient Yi, and Dongba show that OmniOCR outperforms zero-shot foundation models and standard post training, achieving state-of-the-art accuracy with superior parameter efficiency, and compared with the state-of-the-art baseline models, it improves accuracy by 39%-66% on these four datasets. Code: https://github.com/AIGeeksGroup/OmniOCR.

Abstract

Explore similar work

OmniOCR: Generalist OCR for Ethnic Minority Languages

KazakhOCR: A Synthetic Benchmark for Evaluating Multimodal Models in Low-Resource Kazakh Script OCR

Can OCR-VLMs Read Devanagari? A Stress-Test Benchmark and Post-Correction Study