Optical character recognition (OCR) for handwritten Indic manuscripts is essential for large-scale digitization and computational access to manuscript heritage. However, existing approaches are typically developed for one script at a time and require substantial script-specific customization. This limits scalability and practical deployment across diverse collections. We present UniLipi, a unified multi-script OCR model for handwritten Indic manuscripts trained jointly across 13 Indic scripts within a single framework. UniLipi directly handles realistic manuscript conditions, including extreme variation in line geometry, large variation in line length, and partial interruptions caused by non-textual manuscript entities such as holes, stains, or pictorial illustrations. To operate effectively under ultra low-resource conditions, the model leverages script-aware synthetic manuscript data generation, substantially reducing reliance on large volumes of real annotated data. Beyond historical manuscripts, we show that UniLipi serves as an effective foundational pretrained model. Specifically, its learned representations enable good OCR performance for contemporary Indic handwriting and extend to several non-Indic scripts, including Tibetan, Italian, Latin, and Chinese scripts. In addition to transcription, UniLipi predicts script identity and per-line native character counts, supporting practical manuscript cataloging workflows.
Figures & tables
Figure 1: Diversity of manuscript styles across Indic script regions. Examples of manuscript lines from multiple historical collections grouped by geographic and script traditions: North/Central Indian , Southern Indian , Himalayan/Buddhist , Western Indian , and Eastern Indian . The examples illustrate substantial diversity in writing substrates (palm leaf, paper), layout geometry, stroke thickness, ink characteristics, and orthographic structure across scripts and regions. This diversity motivates UniLipi model and reference-conditioned synthesis pipeline, which extracts appearance and geometric priors from real manuscripts to generate realistic synthetic training data.
Figure 2: UniLipi architecture overview. Given a padded line image 1 , a CNN backbone extracts a sequence of image tokens . A learnable script token and a count token are appended to image token sequence. The combined token sequence 2 is encoded with a Transformer encoder 3 . Finally, the image-token representations are decoded by a CTC-based text head to produce a Roman WX transcription. The latent task token outputs are fed to respective heads for script identification and glyph count prediction 4 . The model is trained with a joint objective combining ocr, script, and count losses 5 .
Figure 3: Reference-conditioned synthetic line generation pipeline. 1 A text line is sampled in the unified Roman WX representation and transliterated into the target script to define the visual content. 2 Using a reference manuscript line R , we extract the writing mask, estimate font size, font color , and render glyphs sequentially within valid regions using script specific fonts, while glyphs exceeding are truncated. 3 The rendered text is composited onto the clean background to preserve realistic manuscript characteristics. 4 The final synthetic line is produced along with its Roman WX transcription target, script identity label, and glyph count supervision.
Region
Script
CER (%)
Mono CER (%)
Count-MAE
North and Central India
Devanagari
8.4
12.3
1.2
Sharada
5.2
6.9
0.7
Modi
18.6
17.4
2.3
Western India
Gurmukhi
5.5
6.5
1.2
Jaini
6.1
7.9
1.4
Eastern India
Bengali
18.3
17.1
2.2
Table 1: Script-wise performance of UniLipi on 13 scripts, grouped by geographic region. We report CER (%) for OCR transcription, Mono CER (%) for a script-specific model trained only on that script (monolingual setting), and Count-MAE for glyph count prediction.
Model
Params (M)
CER
Script-L
CER-L
Script-H
CER-H
Kraken [ 31 ]
4.1
28.5
Sharada
15.2
Bengali
42.8
PyLaia [ 50 ]
5.3
24.3
Newar
13.4
Modi
38.7
VLT [ 32 ]
7.0
19.6
Malayalam
10.8
Bengali
31.5
Tesseract [ 58 ]
8.0
32.4
Malayalam
18.6
Modi
47.3
HTR VT [ 35 ]
53.5
14.8
Newar
7.9
Modi
26.4
HTR-ConvText [ 61 ]
65.9
12.9
Sharada
6.8
Siddham
23.7
Table 2: Baseline comparison on UniLipi-13. We evaluate UniLipi (ours) against OCR baselines using overall CER (%) . To summarize script robustness, we additionally report the lowest CER (Script-L/CER-L) and highest CER (Script-H/CER-H) scripts.
Table 3: Transfer evaluation of UniLipi-13. (a) OCR across diverse writing systems; we report CER (%) and WER (%) after fine-tuning on each target benchmark. (b) UniLipi-10 is pretrained on 10 scripts (excluding Sharada, Jaini, and Grantha) and later fine-tuned on these unseen manuscript scripts, whereas UniLipi-13 is pretrained on all 13 scripts.
Figure 4: Qualitative results of UniLipi on script for each region, red means deletion , blue means substitution or addition .
Modified Component
Default
CER
Count-MAE
IAST as vocab
Roman WX
11.0
1.7
Dedicated vocab per script
Roman WX
14.1
1.9
No script token
Auxiliary tokens
7.1
1.3
No count token
Auxiliary tokens
9.3
-
No script and count token
Auxiliary tokens
9.7
-
Encoder with 4 Layers
6 Layers
9.3
1.5
Table 4: Ablation study of UniLipi. Scores of final configuration is shown in the last row. Each ablation modifies a single component while keeping all others identical.
Optical character recognition (OCR) has advanced rapidly with deep learning and multimodal models, yet most methods focus on well-resourced scripts such as Latin and Chinese. Ethnic minority languages remain underexplored due to complex writing systems, scarce annotations, and diverse historical and modern forms, making generalization in low-resource or zero-shot settings challenging. To address these challenges, we present OmniOCR, a universal framework for ethnic minority scripts. OmniOCR introduces Dynamic Low-Rank Adaptation (Dynamic LoRA) to allocate model capacity across layers and scripts, enabling effective adaptation while preserving knowledge.A sparsity regularization prunes redundant updates, ensuring compact and efficient adaptation without extra inference cost. Evaluations on TibetanMNIST, Shui, ancient Yi, and Dongba show that OmniOCR outperforms zero-shot foundation models and standard post training, achieving state-of-the-art accuracy with superior parameter efficiency, and compared with the state-of-the-art baseline models, it improves accuracy by 39%-66% on these four datasets. Code: https://github.com/AIGeeksGroup/OmniOCR.
Optical character recognition (OCR) has advanced rapidly with the rise of vision-language models, yet evaluation has remained concentrated on a small cluster of high- and mid-resource scripts. We introduce GlotOCR Bench, a comprehensive benchmark evaluating OCR generalization across 100+ Unicode scripts. Our benchmark comprises clean and degraded image variants rendered from real multilingual texts. Images are rendered using fonts from the Google Fonts repository, shaped with HarfBuzz and rasterized with FreeType, supporting both LTR and RTL scripts. Samples of rendered images were manually reviewed to verify correct rendering across all scripts. We evaluate a broad suite of open-weight and proprietary vision-language models and find that most perform well on fewer than ten scripts, and even the strongest frontier models fail to generalize beyond thirty scripts. Performance broadly tracks script-level pretraining coverage, suggesting that current OCR systems rely on language model pretraining as much as on visual recognition. Models confronted with unfamiliar scripts either produce random noise or hallucinate characters from similar scripts they already know. We release the benchmark and pipeline for reproducibility. Pipeline Code: https://github.com/cisnlp/glotocr-bench, Benchmark: https://hf.co/datasets/cis-lmu/glotocr-bench.
Amir Hossein Kargaran, Nafiseh Nikeghbal, Jana Diesner +2
Designing Optical Character Recognition (OCR) systems for India requires balancing linguistic diversity, document heterogeneity, and deployment constraints. In this paper, we study two training strategies for building multilingual OCR systems with Vision-Language Models through the Chitrapathak series. We first follow a popular multimodal approach, pairing a generic vision encoder with a strong multilingual language model and training the system end-to-end for OCR. Alternatively, we explore fine-tuning an existing OCR model, despite not being trained for the target languages. Through extensive evaluation on multilingual Indic OCR benchmarks and deployment-oriented metrics, we find that the second strategy consistently achieves better accuracy-latency trade-offs. Chitrapathak-2 achieves 3-6x speedup over its predecessor with being state-of-the-art (SOTA) in Telugu (6.69 char ANLS) and second best in the rest. In addition, we present Parichay, an independent OCR model series designed specifically for 9 Indian government documents to extract structured key fields, achieving 89.8% Exact Match score with a faster inference. Together, these systems achieve SOTA performance and provide practical guidance for building production-scale OCR pipelines in the Indian context.