OmniOCR: Generalist OCR for Ethnic Minority Languages

Bonan Liu; Zeyu Zhang; Bingbing Meng; Han Wang; Hanshuo Zhang; Chengping Wang; Daji Ergu; Ying Cai

cs.CVFeb 24, 2026

OmniOCR: Generalist OCR for Ethnic Minority Languages

Authors: Bonan Liu, Zeyu Zhang, Bingbing Meng, Han Wang, Hanshuo Zhang, Chengping Wang, Daji Ergu, Ying Cai

Organizations: Southwest Minzu University · AI Geeks

Abstract

Optical character recognition (OCR) has advanced rapidly with deep learning and multimodal models, yet most methods focus on well-resourced scripts such as Latin and Chinese. Ethnic minority languages remain underexplored due to complex writing systems, scarce annotations, and diverse historical and modern forms, making generalization in low-resource or zero-shot settings challenging. To address these challenges, we present OmniOCR, a universal framework for ethnic minority scripts. OmniOCR introduces Dynamic Low-Rank Adaptation (Dynamic LoRA) to allocate model capacity across layers and scripts, enabling effective adaptation while preserving knowledge.A sparsity regularization prunes redundant updates, ensuring compact and efficient adaptation without extra inference cost. Evaluations on TibetanMNIST, Shui, ancient Yi, and Dongba show that OmniOCR outperforms zero-shot foundation models and standard post training, achieving state-of-the-art accuracy with superior parameter efficiency, and compared with the state-of-the-art baseline models, it improves accuracy by 39%-66% on these four datasets. Code: https://github.com/AIGeeksGroup/OmniOCR.

Figures & tables

Panel of Figure 1 — Fig. 1: Accuracy comparison on Tibetan dataset across all models.

Dataset	Classes	Samples	Script Type	Image Size
TibetanMNIST [ 28 ]	10	17,768	Digits	111M
Shui Dataset [ 26 ]	12	5,280	Pictographic	21.6M
Ancient Yi Script [ 15 ]	30	10,840	Logographic	62.2M
Dongba Script [ 18 ]	30	14,906	Pictographic	20.2M

TABLE I: STATISTICS OF THE FOUR ETHNIC MINORITY LANGUAGE DATASETS USED FOR TRAINING AND EVALUATION OF OMNIOCR.

Fig. 2: OmniOCR. A represents the processing procedure of the model’s Vision Encoder; B represents the processing procedure of the model’s Text Encoder; C respresents two distinct parameter-efficient fine-tuning methods: Dynamic-Rank Training and Fixed-Rank Training.

Fig. 3: Demonstration of Recognition Performance for Tibetan Handwritten Digits via OmniOCR. The results show the accuracy and visual recognition effect of OmniOCR on the Tibetan handwritten digit dataset.

\Delta W^{t,m}=\sum_{i=1}^{r}w_{i}^{t,m}B_{i}^{t,m}A_{i}^{t,m}

Algorithm 1 OmniOCR with Dynamic LoRA Adaptation

Fig. 4: The performance of OmniOCR evaluated on four representative datasets—TibetanMNIST, Shui, Ancient Yi, and Dongba—highlighting its ability to generalize across heterogeneous scripts and writing systems.

Zero-Shot
Models / Methods	Tibetan [ 28 ]			Ancient Yi [ 15 ]			Shui Script [ 26 ]			Dongba Script [ 18 ]
Models / Methods	Acc	Recall	F1	Acc	Recall	F1	Acc	Recall	F1	Acc	Recall	F1
Kimi-VL [ 24 ]	29.45	28.73	28.96	19.32	20.14	19.92	49.68	48.91	49.15	32.33	33.07	32.83
Moonshot-v1 [ 2 ]	28.84	31.12	29.95	17.14	18.75	17.92	47.27	49.35	48.29	30.31	32.01	31.14
Pixtral Large [ 1 ]	30.56	25.89	27.98	20.13	25.78	22.73	51.42	48.35	49.81	34.27	30.15	32.08
Doubao-1.5-Vision-Pro [ 6 ]	32.55	31.42	31.76	22.34	21.23	21.51	54.49	55.17	54.88	30.72	29.88	30.14

TABLE II: PERFORMANCE (%) ON FOUR ETHNIC MINORITY LANGUAGE DATASETS. ZERO-SHOT MODELS AND FINE-TUNING METHODS ON ROLMOCR ARE COMPARED.

Learning Rate	Batch Size	Tibetan [ 28 ]			Ancient Yi [ 15 ]			Shui Script [ 26 ]			Dongba Script [ 18 ]
Learning Rate	Batch Size	Acc (%)	Recall (%)	F1 (%)	Acc (%)	Recall (%)	F1 (%)	Acc (%)	Recall (%)	F1 (%)	Acc (%)	Recall (%)	F1 (%)
OmniOCR (Ours)	2	90.37	91.12	90.48	89.62	90.14	89.60	95.95	96.31	95.86	95.32	94.76	95.29
$1e$ -5	2	89.85	90.02	89.71	88.20	88.54	88.13	93.41	93.77	93.51	92.83	93.12	92.75
$1e$ -6	2	88.41	88.77	88.23	87.07	87.35	87.03	92.22	92.56	92.11	91.67	91.98	91.59
$5e$ -6	1	87.68	88.03	87.57	86.32	86.69	86.24	91.76	92.09	91.64	90.82	91.21	90.73
$5e$ -6	4	89.11	89.42	88.98	88.02	88.34	87.92	93.13	93.48	93.06	92.41	92.78	92.31

TABLE III: ABLATION STUDY ON LEARNING RATE AND BATCH SIZE ACROSS FOUR DATASETS.

Configuration	Tibetan [ 28 ]			Ancient Yi [ 15 ]			Shui Script [ 26 ]			Dongba Script [ 18 ]
Configuration	Acc (%)	Recall (%)	F1 (%)	Acc (%)	Recall (%)	F1 (%)	Acc (%)	Recall (%)	F1 (%)	Acc (%)	Recall (%)	F1 (%)
Full Dynamic LoRA (Ours)	90.37	91.12	90.48	89.62	90.14	89.60	95.95	96.31	95.86	95.32	94.76	95.29
✗ Dynamic Rank	83.86	83.25	83.60	82.51	81.79	82.15	86.98	86.20	86.55	85.70	84.91	85.32
✗ MLP Modules	82.56	81.92	82.30	81.17	80.49	80.88	85.10	84.39	84.78	84.03	83.24	83.65
✗ Attention Modules	88.74	89.18	88.89	87.99	88.46	88.12	91.87	92.41	92.03	90.77	91.32	90.99
✗ Regularization Sparsity	85.55	84.99	85.27	83.99	83.31	83.68	88.80	88.12	88.47	87.42	86.72	87.08

TABLE IV: ABLATION STUDY ON DYNAMIC LORA COMPONENTS ACROSS FOUR DATASETS.

Explore similar work

What Can Low Resource Languages Learn From Each Other?
Aug 27, 2026Achyuth P, Kahaan Shah, Chetan AroraOptical Character Recognition VLM Adaptation
IIT Delhi, New Delhi, India
GlotOCR Bench: OCR Models Still Struggle Beyond a Handful of Unicode Scripts
Apr 14, 2026Amir Hossein Kargaran, Nafiseh Nikeghbal, Jana Diesner +2Optical Character Recognition Multilingual VLM Evaluation
LMU Munich · MCML · TU Munich +1
KazakhOCR: A Synthetic Benchmark for Evaluating Multimodal Models in Low-Resource Kazakh Script OCR
Feb 17, 2026Henry Gagnier, Sophie Gagnier, Ashwin KirubakaranOptical Character Recognition Low-Resource Language Processing
Pittsford Sutherland High School · Edison Academy Magnet School

Aug 27, 2026J/K move · Enter open · S save

What Can Low Resource Languages Learn From Each Other?

Achyuth P, Kahaan Shah, Chetan Arora

IIT Delhi, New Delhi, India

Optical Character Recognition VLM Adaptation Cross-Lingual Transfer Model Merging Multilingual VLM Evaluation

Open paper AskPDF arXiv

Despite the rapid advancement of Vision-Language Models (VLMs), their linguistic reach remains largely confined to high-resource languages, leaving the majority of the world's 7,000+ living languages on the wrong side of a growing digital divide. This disparity is especially pronounced in Optical Character Recognition (OCR), where low-resource scripts lack the massive datasets required for traditional scaling laws. We investigate OCR adaptation in extreme data-scarce regimes (<10K real and <250K synthetic images), demonstrating that conventional fine-tuning strategies often reach a performance ceiling. Our key finding reveals a structural inefficiency in language-specific adaptation: while higher layers of specialized models diverge to capture unique script nuances, the lower layers learn redundant, highly similar features. Motivated by this observation, we propose PSMC (Pre-train, Specialize, Merge, and Co-train), a data-efficient framework that capitalizes on a cross-script "transfer effect". Our approach first derives language-specific experts from a high-resource base model, then employs task arithmetic to fuse these experts into a unified, high-performance multilingual back- bone. Extensive evaluation across 10 Indian scripts (supporting 20+ languages) shows that PSMC achieves a ~2% average improvement in Word Recognition Rate (WRR) over individual specialist models without increasing parameter count. Our results indicate that joint training in the merged latent space facilitates a constructive knowledge transfer that benefits all constituent scripts, providing a scalable pathway for inclusive VLM development. Source code and datasets will be released post publication.

Abstract

Explore similar work

What Can Low Resource Languages Learn From Each Other?

GlotOCR Bench: OCR Models Still Struggle Beyond a Handful of Unicode Scripts

KazakhOCR: A Synthetic Benchmark for Evaluating Multimodal Models in Low-Resource Kazakh Script OCR