UniLipi: A Unified Multi-Script OCR for Historical Indic Manuscripts

Tathagata Ghosh; Sai Madhusudan Gunda; Simran Singh Sandral; Ravi Kiran Sarvadevabhatla

cs.CVAug 28, 2026

UniLipi: A Unified Multi-Script OCR for Historical Indic Manuscripts

Authors: Tathagata Ghosh, Sai Madhusudan Gunda, Simran Singh Sandral, Ravi Kiran Sarvadevabhatla

Organizations: International Institute of Information Technology Hyderabad, INDIA · GITAM

Abstract

Optical character recognition (OCR) for handwritten Indic manuscripts is essential for large-scale digitization and computational access to manuscript heritage. However, existing approaches are typically developed for one script at a time and require substantial script-specific customization. This limits scalability and practical deployment across diverse collections. We present UniLipi, a unified multi-script OCR model for handwritten Indic manuscripts trained jointly across 13 Indic scripts within a single framework. UniLipi directly handles realistic manuscript conditions, including extreme variation in line geometry, large variation in line length, and partial interruptions caused by non-textual manuscript entities such as holes, stains, or pictorial illustrations. To operate effectively under ultra low-resource conditions, the model leverages script-aware synthetic manuscript data generation, substantially reducing reliance on large volumes of real annotated data. Beyond historical manuscripts, we show that UniLipi serves as an effective foundational pretrained model. Specifically, its learned representations enable good OCR performance for contemporary Indic handwriting and extend to several non-Indic scripts, including Tibetan, Italian, Latin, and Chinese scripts. In addition to transcription, UniLipi predicts script identity and per-line native character counts, supporting practical manuscript cataloging workflows.

Figures & tables

Figure 1: Diversity of manuscript styles across Indic script regions. Examples of manuscript lines from multiple historical collections grouped by geographic and script traditions: North/Central Indian , Southern Indian , Himalayan/Buddhist , Western Indian , and Eastern Indian . The examples illustrate substantial diversity in writing substrates (palm leaf, paper), layout geometry, stroke thickness, ink characteristics, and orthographic structure across scripts and regions. This diversity motivates UniLipi model and reference-conditioned synthesis pipeline, which extracts appearance and geometric priors from real manuscripts to generate realistic synthetic training data.

Figure 2: UniLipi architecture overview. Given a padded line image 1 , a CNN backbone extracts a sequence of image tokens . A learnable script token and a count token are appended to image token sequence. The combined token sequence 2 is encoded with a Transformer encoder 3 . Finally, the image-token representations are decoded by a CTC-based text head to produce a Roman WX transcription. The latent task token outputs are fed to respective heads for script identification and glyph count prediction 4 . The model is trained with a joint objective combining ocr, script, and count losses 5 .

Figure 3: Reference-conditioned synthetic line generation pipeline. 1 A text line is sampled in the unified Roman WX representation and transliterated into the target script to define the visual content. 2 Using a reference manuscript line R , we extract the writing mask, estimate font size, font color , and render glyphs sequentially within valid regions using script specific fonts, while glyphs exceeding are truncated. 3 The rendered text is composited onto the clean background to preserve realistic manuscript characteristics. 4 The final synthetic line is produced along with its Roman WX transcription target, script identity label, and glyph count supervision.

Region	Script	CER (%)	Mono CER (%)	Count-MAE
North and Central India	Devanagari	8.4	12.3	1.2
	Sharada	5.2	6.9	0.7
	Modi	18.6	17.4	2.3
Western India	Gurmukhi	5.5	6.5	1.2
Western India	Jaini	6.1	7.9	1.4
Eastern India	Bengali	18.3	17.1	2.2

Table 1: Script-wise performance of UniLipi on 13 scripts, grouped by geographic region. We report CER (%) for OCR transcription, Mono CER (%) for a script-specific model trained only on that script (monolingual setting), and Count-MAE for glyph count prediction.

Model	Params (M)	CER	Script-L	CER-L	Script-H	CER-H
Kraken [ 31 ]	4.1	28.5	Sharada	15.2	Bengali	42.8
PyLaia [ 50 ]	5.3	24.3	Newar	13.4	Modi	38.7
VLT [ 32 ]	7.0	19.6	Malayalam	10.8	Bengali	31.5
Tesseract [ 58 ]	8.0	32.4	Malayalam	18.6	Modi	47.3
HTR VT [ 35 ]	53.5	14.8	Newar	7.9	Modi	26.4
HTR-ConvText [ 61 ]	65.9	12.9	Sharada	6.8	Siddham	23.7

Table 2: Baseline comparison on UniLipi-13. We evaluate UniLipi (ours) against OCR baselines using overall CER (%) . To summarize script robustness, we additionally report the lowest CER (Script-L/CER-L) and highest CER (Script-H/CER-H) scripts.

Table 3: Transfer evaluation of UniLipi-13. (a) OCR across diverse writing systems; we report CER (%) and WER (%) after fine-tuning on each target benchmark. (b) UniLipi-10 is pretrained on 10 scripts (excluding Sharada, Jaini, and Grantha) and later fine-tuned on these unseen manuscript scripts, whereas UniLipi-13 is pretrained on all 13 scripts.

Figure 4: Qualitative results of UniLipi on script for each region, red means deletion , blue means substitution or addition .

Modified Component	Default	CER	Count-MAE
IAST as vocab	Roman WX	11.0	1.7
Dedicated vocab per script	Roman WX	14.1	1.9
No script token	Auxiliary tokens	7.1	1.3
No count token	Auxiliary tokens	9.3	-
No script and count token	Auxiliary tokens	9.7	-
Encoder with 4 Layers	6 Layers	9.3	1.5

Table 4: Ablation study of UniLipi. Scores of final configuration is shown in the last row. Each ablation modifies a single component while keeping all others identical.

Figure 5: Count token embedding heat map

Optical Character Recognition

Optical character recognition converts text in images into machine-readable text, from printed pages and handwriting to signs and symbols.

See all 200 papers

OmniOCR: Generalist OCR for Ethnic Minority Languages
Feb 24, 2026Bonan Liu, Zeyu Zhang, Bingbing Meng +5Optical Character Recognition Low-Resource Language Processing
Southwest Minzu University · AI Geeks
GlotOCR Bench: OCR Models Still Struggle Beyond a Handful of Unicode Scripts
Apr 14, 2026Amir Hossein Kargaran, Nafiseh Nikeghbal, Jana Diesner +2Multilingual OCR Optical Character Recognition
LMU Munich · MCML · TU Munich +1
Designing Production-Scale OCR for India: Multilingual and Domain-Specific Systems
Feb 18, 2026Ali Faraz, Raja Kolla, Ashish Kulkarni +1Multilingual OCR Optical Character Recognition
Krutrim AI, Bangalore, India

Feb 24, 2026J/K move · Enter open · S save

OmniOCR: Generalist OCR for Ethnic Minority Languages

Bonan Liu, Zeyu Zhang, Bingbing Meng, Han Wang+4

Southwest Minzu University · AI Geeks

Optical Character Recognition Low-Resource Language Processing Parameter-Efficient Fine-Tuning Low-Rank Adaptation

Open paper AskPDF arXiv

Optical character recognition (OCR) has advanced rapidly with deep learning and multimodal models, yet most methods focus on well-resourced scripts such as Latin and Chinese. Ethnic minority languages remain underexplored due to complex writing systems, scarce annotations, and diverse historical and modern forms, making generalization in low-resource or zero-shot settings challenging. To address these challenges, we present OmniOCR, a universal framework for ethnic minority scripts. OmniOCR introduces Dynamic Low-Rank Adaptation (Dynamic LoRA) to allocate model capacity across layers and scripts, enabling effective adaptation while preserving knowledge.A sparsity regularization prunes redundant updates, ensuring compact and efficient adaptation without extra inference cost. Evaluations on TibetanMNIST, Shui, ancient Yi, and Dongba show that OmniOCR outperforms zero-shot foundation models and standard post training, achieving state-of-the-art accuracy with superior parameter efficiency, and compared with the state-of-the-art baseline models, it improves accuracy by 39%-66% on these four datasets. Code: https://github.com/AIGeeksGroup/OmniOCR.

Abstract

Optical Character Recognition

Explore similar work

OmniOCR: Generalist OCR for Ethnic Minority Languages

GlotOCR Bench: OCR Models Still Struggle Beyond a Handful of Unicode Scripts

Designing Production-Scale OCR for India: Multilingual and Domain-Specific Systems