Designing Production-Scale OCR for India: Multilingual and Domain-Specific Systems

Ali Faraz; Raja Kolla; Ashish Kulkarni; Shubham Agarwal

cs.CVFeb 18, 2026

Designing Production-Scale OCR for India: Multilingual and Domain-Specific Systems

Authors: Ali Faraz, Raja Kolla, Ashish Kulkarni, Shubham Agarwal

Organizations: Krutrim AI, Bangalore, India

Abstract

Designing Optical Character Recognition (OCR) systems for India requires balancing linguistic diversity, document heterogeneity, and deployment constraints. In this paper, we study two training strategies for building multilingual OCR systems with Vision-Language Models through the Chitrapathak series. We first follow a popular multimodal approach, pairing a generic vision encoder with a strong multilingual language model and training the system end-to-end for OCR. Alternatively, we explore fine-tuning an existing OCR model, despite not being trained for the target languages. Through extensive evaluation on multilingual Indic OCR benchmarks and deployment-oriented metrics, we find that the second strategy consistently achieves better accuracy-latency trade-offs. Chitrapathak-2 achieves 3-6x speedup over its predecessor with being state-of-the-art (SOTA) in Telugu (6.69 char ANLS) and second best in the rest. In addition, we present Parichay, an independent OCR model series designed specifically for 9 Indian government documents to extract structured key fields, achieving 89.8% Exact Match score with a faster inference. Together, these systems achieve SOTA performance and provide practical guidance for building production-scale OCR pipelines in the Indian context.

Figures & tables

Figure 1: Overview of the two complementary strategies explored in our work in Section 3 . We find strategy 2 of finetuning existing model to be data efficient and performing better for multilingual and domain adaptation.

Language	Hindi	Sanskrit	Bengali	Marathi	Tamil	Telugu	Kannada	Malayalam	Punjabi	Odia
Volume	19L	11.5L	9.16L	8.5L	10.1L	8L	3.4L	2.3L	4.8L	48K

Table 1: Language-wise training data volumes used for multilingual OCR Chitrapathak-1 (L = lakh, K = thousand).

Figure 2: OCR outputs for Hindi (left) and Sanskrit (right) languages from Chitrapathak-2. More examples in Appendix.

Model	Bengali		Hindi		Kannada		Malayalam		Marathi		Odia		Punjabi		Tamil		Telugu
	Word $\downarrow$	Char $\downarrow$	Word $\downarrow$	Char $\downarrow$	Word $\downarrow$	Char $\downarrow$	Word $\downarrow$	Char $\downarrow$	Word $\downarrow$	Char $\downarrow$	Word $\downarrow$	Char $\downarrow$	Word $\downarrow$	Char $\downarrow$	Word $\downarrow$	Char $\downarrow$	Word $\downarrow$	Char $\downarrow$
LLaMA-4	31.52	13.21	25.73	11.91	36.90	11.17	75.50	45.75	20.94	8.05	97.51	86.78	29.77	12.68	31.36	10.79	57.07	18.72
Gemma-3 27B	42.15	24.41	46.47	29.50	84.22	54.24	92.06	72.64	50.40	31.06	92.67	70.72	70.88	42.65	39.52	16.51	86.76	54.14
GPT-4o	55.51	32.68	54.62	35.54	94.33	69.79	94.67	78.47	63.44	37.93	94.61	73.46	68.88	40.71	74.35	43.39	95.97	70.08
Nanonets-OCR2-3B	28.56	12.42	32.26	16.78	99.38	93.07	97.24	89.81	40.97	15.92	99.82	97.11	98.70	82.84	95.25	78.83	99.42	89.39
Surya OCR	28.76	12.61	20.11	8.38	24.4	6.37	73.46	36.37	13.41	4.33	52.7	25.58	18	7.31	25.75	7.71	51.72	16.85

Table 2: IndicVisionBench-OCR performance (ANLS; lower is better) across multiple baselines. “Word”/“Char” denote word-/character-level ANLS.

Model	English	Hindi	Others
Chitrapathak-1	14.38	36.42	32.87
Chitrapathak-2	3.10	6.59	14.26
GPT-4o	10.40	18.90	23.77

Table 3: Average end-to-end OCR latency (seconds) on our internal evaluation set (18 English, 40 Hindi and 63 images across 9 other Indian languages).

Model	Exact Match (EM %)
Base Phi-3.5 Vision Instruct	23.26
Parichay-1 (Full Fine-Tuning)	82.13
Parichay-1 + Rotation Module	88.70
Parichay-2 + Rotation Module	89.80
Gemini-2.5-flash	86.00

Table 4: Exact Match (EM) comparison across Parichay variants and a proprietary VLM. Parichay-2 with rotation achieves the highest extraction accuracy.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: Example document image from the SROIE dataset.

Panel of Figure 4 — Figure 4: OCR outputs for Odia (left) and Malayalam (right) languages from Chitrapathak-2

Figure 5: Examples of limitations of Chitrapathak-2. The left image shows an index-page in Hindi while the right image shows a page in English with a rare/old way of writing the letter ‘s’ which the model consistently reads as ‘f’.

Model	Synthdog		SROIE
	Word $\downarrow$	Char $\downarrow$	%Match $\uparrow$
Gemma-3 27B	61.56	30.29	68.37
LLaMA-4 maverick	29.37	14.09	70.32
GPT-4o	82.22	73.65	36.09
Nanonets-OCR2-3B	23.90	10.80	72.33
Chitrapathak-2	24.90	20.20	68.95

Appendix

Table 5: English OCR performance on Synthdog (ANLS; lower is better) and SROIE (%Match; higher is better).

Model	Word $\downarrow$	Char $\downarrow$
Nanonets-OCR2-3B	4.33	2.96
Chitrapathak-2	4.49	1.89
Gemini-2.5	4.36	1.86

Appendix

Table 6: English OCR performance on the Old Books OCR dataset (ANLS; lower is better).

Metric	bn	hi	kn	ml	mr	or	pa	ta	te	en
Tokens/word	5.9	4.8	11.2	12.6	6.4	11.7	6.9	9.4	13.2	1.4
Tokens (200 words)	1174.8	951.4	2242.2	2514.0	1292.4	2334.2	1387.2	1873.6	2646.6	280.0
Latency (200 words, s)	4.9	4.0	9.2	10.3	5.3	9.5	5.7	7.7	10.8	1.3

Appendix

Table 7: Token efficiency and projected decoding latency for Chitrapathak-2 across languages, assuming a

\sim

1024

\times

1024 input image. All the metrics are rounded to one decimal place.

Config	Rank ( $r$ )	Alpha ( $\alpha$ )	Mean Score (%)
Base	–	–	40.45
LoRA	128	128	70.13
LoRA	128	256	70.54
LoRA	128	512	72.34
LoRA	16	256	71.50
LoRA	256	256	71.87

Appendix

Table 8: Mean field-level extraction scores for Parichay-1 across LoRA configurations and full fine-tuning. LoRA is applied to the attention projection matrices (

W_{q}

W_{k}

W_{v}

W_{o}

). Mean Score is computed as the average of Exact Match (EM) and Percentage Match (PM).

Model	Inference Engine	Latency (sec)
Parichay-1	HF Transformers (generate)	4.10
Parichay-2	vLLM	1.03

Appendix

Table 9: Average per-document inference latency on H100 GPUs. Parichay-2 achieves nearly 4

\times

lower latency compared to Parichay-1 while improving extraction accuracy.

Document Type	Train Samples	Test Samples
Aadhaar	5166	1630
Cancelled Cheque	4104	1074
Car Fitness	495	148
Car Permit	984	45
Driving License	811	241
Insurance	5010	586

Appendix

Table 10: Distribution of training and test samples across document types used for Parichay model development and evaluation.

Doc Type	Parichay-1 +	Parichay-1	Parichay-1+	Gemini	Phi4 +	Llama-4	Azure +	Nanonets	Gemma-3	DocTR +	Tesseract +	Mini
	Rot		+ Rot (vLLM)	2.5	Rot	17B	Mistral7B	OCR-s	27B-IT	Mistral7B	Mistral7B	CPM
Aadhaar	100.00	90.00	90.00	100.00	93.33	98.00	80.00	88.00	84.00	52.00	34.00	46.00
Cancelled Cheque	60.00	60.00	15.00	80.00	65.00	70.00	75.00	40.00	25.00	35.00	10.00	0.00
Car Fitness	90.00	65.00	75.00	85.00	90.00	55.00	80.00	55.00	65.00	40.00	25.00	0.00
Car Permit	77.50	77.50	40.00	62.50	55.83	50.83	58.33	45.83	43.33	50.83	35.00	15.00
Driving Licence	85.00	70.00	90.00	90.00	83.33	85.00	60.00	85.00	75.00	35.00	15.00	0.00

Appendix

Table 11: Document-wise Exact Match (EM) scores across Parichay-1 variants and baseline systems. Each document category contains 10 evaluation instances (90 samples in total). Parichay-1 with rotation achieves the highest overall EM. The last column reports the number of samples per document type.

Explore similar work

Can OCR-VLMs Read Devanagari? A Stress-Test Benchmark and Post-Correction Study
Jun 28, 2026Aditya Pratap SinghVLM Evaluation VLM Robustness
PP-OCRv6: From 1.5M to 34.5M Parameters, Surpassing Billion-Scale VLMs on OCR Tasks
Jun 11, 2026Yubo Zhang, Xueqing Wang, Manhui Lin +13Optical Character Recognition
PaddlePaddle Team, Baidu Inc.
LightOnOCR: A 1B End-to-End Multilingual Vision-Language Model for State-of-the-Art OCR
Jan 20, 2026Said Taghadouini, Adrien Cavaillès, Baptiste AubertinVision-Language Models Optical Character Recognition
LightOn

Jun 28, 2026J/K move · Enter open · S save

Can OCR-VLMs Read Devanagari? A Stress-Test Benchmark and Post-Correction Study

Aditya Pratap Singh

VLM Evaluation VLM Robustness Optical Character Recognition

Open paper AskPDF arXiv

OCR systems, ranging from classical engines to specialised OCR vision-language models (OCR-VLMs) and frontier multimodal LLMs, report strong results on English and Chinese document benchmarks, yet their behaviour on Indic scripts is largely uncharacterised. We benchmark ten systems on Devanagari (Hindi): classical EasyOCR; open VLMs (Qwen2.5-VL-3B, Qwen3-VL-8B, olmOCR-7B); specialised OCR-VLMs (DeepSeek-OCR, Unlimited-OCR); and frontier closed models (Gemini 2.5 Flash, Claude Opus 4.7, GPT-5.5, Mistral OCR), across four synthetic degradation conditions and 300 real printed scans. We report four findings. First, on clean rendered text all ten cluster within chrF++ 91 to 98, so synthetic text does not separate them. Second, under degradation the specialised OCR-VLMs are the most fragile: DeepSeek-OCR suffers rare but catastrophic repetition failures (outputs up to 71 the reference length) that wreck its corpus mean even though its median is the best of any system, which is why we report median and catastrophic-rate instead of the mean. Third, on real scans nine of the ten systems collapse (EasyOCR falls from chrF++ 93.6 to 58.3) and the field spreads across a 76-point range, so synthetic renders badly overstate Devanagari quality. Fourth, strong English OCR does not predict Indic OCR: GPT-5.5 drops to chrF++ 58.5 (tying classical EasyOCR) and olmOCR-7B, the model behind olmOCR-Bench, falls to 40.5, while the open Qwen3-VL-8B (75.2, runnable on a single 24 GB GPU) beats GPT-5.5 and approaches Mistral; Gemini and Claude lead at 86.3 and 82.2. An error taxonomy separates surface errors (numerals, punctuation) from structural ones (conjuncts, matras, nukta), and a byte-level (ByT5) post-corrector improves a cheap engine on its own error distribution (chrF++ +1.2 to +1.5) but does not transfer across engines. We release the benchmark, code, and models.

Abstract

Explore similar work

Can OCR-VLMs Read Devanagari? A Stress-Test Benchmark and Post-Correction Study

PP-OCRv6: From 1.5M to 34.5M Parameters, Surpassing Billion-Scale VLMs on OCR Tasks

LightOnOCR: A 1B End-to-End Multilingual Vision-Language Model for State-of-the-Art OCR