Optical character recognition (OCR) has advanced rapidly with deep learning and multimodal models, yet most methods focus on well-resourced scripts such as Latin and Chinese. Ethnic minority languages remain underexplored due to complex writing systems, scarce annotations, and diverse historical and modern forms, making generalization in low-resource or zero-shot settings challenging. To address these challenges, we present OmniOCR, a universal framework for ethnic minority scripts. OmniOCR introduces Dynamic Low-Rank Adaptation (Dynamic LoRA) to allocate model capacity across layers and scripts, enabling effective adaptation while preserving knowledge.A sparsity regularization prunes redundant updates, ensuring compact and efficient adaptation without extra inference cost. Evaluations on TibetanMNIST, Shui, ancient Yi, and Dongba show that OmniOCR outperforms zero-shot foundation models and standard post training, achieving state-of-the-art accuracy with superior parameter efficiency, and compared with the state-of-the-art baseline models, it improves accuracy by 39%-66% on these four datasets. Code: https://github.com/AIGeeksGroup/OmniOCR.
Figures & tables
Fig. 1: Accuracy comparison on Tibetan dataset across all models.
Dataset
Classes
Samples
Script Type
Image Size
TibetanMNIST [ 28 ]
10
17,768
Digits
111M
Shui Dataset [ 26 ]
12
5,280
Pictographic
21.6M
Ancient Yi Script [ 15 ]
30
10,840
Logographic
62.2M
Dongba Script [ 18 ]
30
14,906
Pictographic
20.2M
TABLE I: STATISTICS OF THE FOUR ETHNIC MINORITY LANGUAGE DATASETS USED FOR TRAINING AND EVALUATION OF OMNIOCR.
Fig. 2: OmniOCR. A represents the processing procedure of the model’s Vision Encoder; B represents the processing procedure of the model’s Text Encoder; C respresents two distinct parameter-efficient fine-tuning methods: Dynamic-Rank Training and Fixed-Rank Training.
Fig. 3: Demonstration of Recognition Performance for Tibetan Handwritten Digits via OmniOCR. The results show the accuracy and visual recognition effect of OmniOCR on the Tibetan handwritten digit dataset.
ΔWt,m=∑i=1rwit,mBit,mAit,m
Algorithm 1 OmniOCR with Dynamic LoRA Adaptation
Fig. 4: The performance of OmniOCR evaluated on four representative datasets—TibetanMNIST, Shui, Ancient Yi, and Dongba—highlighting its ability to generalize across heterogeneous scripts and writing systems.
Models / Methods
Tibetan [ 28 ]
Ancient Yi [ 15 ]
Shui Script [ 26 ]
Dongba Script [ 18 ]
Acc
Recall
F1
Acc
Recall
F1
Acc
Recall
F1
Acc
Recall
F1
Zero-Shot
Kimi-VL [ 24 ]
29.45
28.73
28.96
19.32
20.14
19.92
49.68
48.91
49.15
32.33
33.07
32.83
Moonshot-v1 [ 2 ]
28.84
31.12
29.95
17.14
18.75
17.92
47.27
49.35
48.29
30.31
32.01
31.14
Pixtral Large [ 1 ]
30.56
25.89
27.98
20.13
25.78
22.73
51.42
48.35
49.81
34.27
30.15
32.08
Doubao-1.5-Vision-Pro [ 6 ]
32.55
31.42
31.76
22.34
21.23
21.51
54.49
55.17
54.88
30.72
29.88
30.14
TABLE II: PERFORMANCE (%) ON FOUR ETHNIC MINORITY LANGUAGE DATASETS. ZERO-SHOT MODELS AND FINE-TUNING METHODS ON ROLMOCR ARE COMPARED.
Learning Rate
Batch Size
Tibetan [ 28 ]
Ancient Yi [ 15 ]
Shui Script [ 26 ]
Dongba Script [ 18 ]
Acc (%)
Recall (%)
F1 (%)
Acc (%)
Recall (%)
F1 (%)
Acc (%)
Recall (%)
F1 (%)
Acc (%)
Recall (%)
F1 (%)
OmniOCR (Ours)
2
90.37
91.12
90.48
89.62
90.14
89.60
95.95
96.31
95.86
95.32
94.76
95.29
1e -5
2
89.85
90.02
89.71
88.20
88.54
88.13
93.41
93.77
93.51
92.83
93.12
92.75
1e -6
2
88.41
88.77
88.23
87.07
87.35
87.03
92.22
92.56
92.11
91.67
91.98
91.59
5e -6
1
87.68
88.03
87.57
86.32
86.69
86.24
91.76
92.09
91.64
90.82
91.21
90.73
5e -6
4
89.11
89.42
88.98
88.02
88.34
87.92
93.13
93.48
93.06
92.41
92.78
92.31
TABLE III: ABLATION STUDY ON LEARNING RATE AND BATCH SIZE ACROSS FOUR DATASETS.
Configuration
Tibetan [ 28 ]
Ancient Yi [ 15 ]
Shui Script [ 26 ]
Dongba Script [ 18 ]
Acc (%)
Recall (%)
F1 (%)
Acc (%)
Recall (%)
F1 (%)
Acc (%)
Recall (%)
F1 (%)
Acc (%)
Recall (%)
F1 (%)
Full Dynamic LoRA (Ours)
90.37
91.12
90.48
89.62
90.14
89.60
95.95
96.31
95.86
95.32
94.76
95.29
✗ Dynamic Rank
83.86
83.25
83.60
82.51
81.79
82.15
86.98
86.20
86.55
85.70
84.91
85.32
✗ MLP Modules
82.56
81.92
82.30
81.17
80.49
80.88
85.10
84.39
84.78
84.03
83.24
83.65
✗ Attention Modules
88.74
89.18
88.89
87.99
88.46
88.12
91.87
92.41
92.03
90.77
91.32
90.99
✗ Regularization Sparsity
85.55
84.99
85.27
83.99
83.31
83.68
88.80
88.12
88.47
87.42
86.72
87.08
TABLE IV: ABLATION STUDY ON DYNAMIC LORA COMPONENTS ACROSS FOUR DATASETS.
Despite the rapid advancement of Vision-Language Models (VLMs), their linguistic reach remains largely confined to high-resource languages, leaving the majority of the world's 7,000+ living languages on the wrong side of a growing digital divide. This disparity is especially pronounced in Optical Character Recognition (OCR), where low-resource scripts lack the massive datasets required for traditional scaling laws. We investigate OCR adaptation in extreme data-scarce regimes (<10K real and <250K synthetic images), demonstrating that conventional fine-tuning strategies often reach a performance ceiling. Our key finding reveals a structural inefficiency in language-specific adaptation: while higher layers of specialized models diverge to capture unique script nuances, the lower layers learn redundant, highly similar features. Motivated by this observation, we propose PSMC (Pre-train, Specialize, Merge, and Co-train), a data-efficient framework that capitalizes on a cross-script "transfer effect". Our approach first derives language-specific experts from a high-resource base model, then employs task arithmetic to fuse these experts into a unified, high-performance multilingual back- bone. Extensive evaluation across 10 Indian scripts (supporting 20+ languages) shows that PSMC achieves a ~2% average improvement in Word Recognition Rate (WRR) over individual specialist models without increasing parameter count. Our results indicate that joint training in the merged latent space facilitates a constructive knowledge transfer that benefits all constituent scripts, providing a scalable pathway for inclusive VLM development. Source code and datasets will be released post publication.
Optical character recognition (OCR) has advanced rapidly with the rise of vision-language models, yet evaluation has remained concentrated on a small cluster of high- and mid-resource scripts. We introduce GlotOCR Bench, a comprehensive benchmark evaluating OCR generalization across 100+ Unicode scripts. Our benchmark comprises clean and degraded image variants rendered from real multilingual texts. Images are rendered using fonts from the Google Fonts repository, shaped with HarfBuzz and rasterized with FreeType, supporting both LTR and RTL scripts. Samples of rendered images were manually reviewed to verify correct rendering across all scripts. We evaluate a broad suite of open-weight and proprietary vision-language models and find that most perform well on fewer than ten scripts, and even the strongest frontier models fail to generalize beyond thirty scripts. Performance broadly tracks script-level pretraining coverage, suggesting that current OCR systems rely on language model pretraining as much as on visual recognition. Models confronted with unfamiliar scripts either produce random noise or hallucinate characters from similar scripts they already know. We release the benchmark and pipeline for reproducibility. Pipeline Code: https://github.com/cisnlp/glotocr-bench, Benchmark: https://hf.co/datasets/cis-lmu/glotocr-bench.
Amir Hossein Kargaran, Nafiseh Nikeghbal, Jana Diesner +2
Kazakh is a Turkic language using the Arabic, Cyrillic, and Latin scripts, making it unique in terms of optical character recognition (OCR). Work on OCR for low-resource Kazakh scripts is very scarce, and no OCR benchmarks or images exist for the Arabic and Latin scripts. We construct a synthetic OCR dataset of 7,219 images for all three scripts with font, color, and noise variations to imitate real OCR tasks. We evaluated three multimodal large language models (MLLMs) on a subset of the benchmark for OCR and language identification: Gemma-3-12B-it, Qwen2.5-VL-7B-Instruct, and Llama-3.2-11B-Vision-Instruct. All models are unsuccessful with Latin and Arabic script OCR, and fail to recognize the Arabic script as Kazakh text, misclassifying it as Arabic, Farsi, and Kurdish. We further compare MLLMs with a classical OCR baseline and find that while traditional OCR has lower character error rates, MLLMs fail to match this performance. These findings show significant gaps in current MLLM capabilities to process low-resource Abjad-based scripts and demonstrate the need for inclusive models and benchmarks supporting low-resource scripts and languages.
Henry Gagnier, Sophie Gagnier, Ashwin Kirubakaran
Pittsford Sutherland High School · Edison Academy Magnet School