Building and expanding access to digital archives supports language reclamation and creates lasting resources for community revitalization. Designing effective tools to support these efforts requires taking into account the community-, language-, and document-specific context. We investigate how small (≤3B) vision-language models (VLMs) can be adapted as optical character recognition (OCR) tools for Lakota, a North American Indigenous language. In addition to benchmarking modeling approaches, we inspect four practical facets of low-resource archival OCR that arise in previous work and conversations with community members: constrained training budgets, segmentation noise, scan degradation, and orthographic variance. We demonstrate that fine-tuning VLMs for OCR post-correction on 35 pages of transcribed data decreases character error rate (CER) from 17.1 to 3.9% on a bilingual corpus, and find that training only with nine pages increases CER by only 1.1% over this. However, evaluating under realistic automatic segmentation nearly doubles the CER of the best-performing models, stressing the importance of taking realistic use cases into account. We hope this work encourages future research on developing language technologies grounded in the workflows and constraints of ongoing revitalization efforts.
Figures & tables
Figure 1: Overview of some of the challenges in Lakota document transcription (e.g., orthographic variation, document degradation, manual vs. automatic segmentation noise, and constrained training budgets) and where they may arise along an OCR training and inference pipeline.
Sub-corpus
Pages
Lines
Chars
Lakota (%)
Deloria
Lakota
14
299
20,655
96.5%
Mixed
5
51
3,365
39.9%
English
11
136
6,812
30.1%
CULP
Lakota
1
24
142
100.0%
Table 1: Corpus overview across orthographic and language subsets. Lakota(%) is calculated using langdetect.
Model
CER (%)
WER (%)
GLM-OCR (0.9B)
17.09
50.33
Qwen3-VL (2B)
18.06
51.76
Dots.ocr (1.7B)
18.06
46.99
Nanonets OCR2 (3B)
20.32
67.72
Tesseract v5 (lat)
32.58
68.65
Transkribus (Text Titan II)
15.73
47.65
Table 2: Overall zero-shot OCR performance on all transcribed pages.
Figure 2: CER of different OCR approaches across small VLMs with direct finetuning and post-correction.
Figure 3: Solid curves show the average Character Error Rate (CER) across 10 random training-set selections under 10-fold cross-validation (100 trained models per point). Shaded regions show the 95% confidence intervals across the different random training subsets.
Approach
Gold
Kraken
Full Page
GLM-OCRZS
17.09
19.82 (+2.73)
20.06 (+2.97)
TranskribusZS †
15.73
—
18.20 (+2.47)
GLMZS+LSTMPOST
15.46
19.40 (+3.94)
—
GLMZS+LLMPOST
14.06
16.76 (+2.70)
—
GLMZS+ByT5POST
10.59
13.24 (+2.65)
—
GLMSFT
4.39
7.00 (+2.61)
11.97 (+7.58)
Table 3: CER and degradation ( Δ ) of OCR performance relative to gold bounding boxes. SFT and post-correction methods use GLM-OCR. Omitted entries (—) are not applicable to the respective pipeline or were omitted due to commercial constraints.
Orthography
Lakota
Corpus
Deloria
GLMSFT (Deloria Expert)
5.39
6.85
GLMZS+GLMPOST (Deloria Expert)
5.14
6.25
GLMSFT (Unified)
5.44
4.27
GLMZS+GLMPOST (Unified)
5.66
3.81
CULP
Table 4: CER of models trained on orthography-specific subsets versus unified corpus training. The first column indicates the performance on Lakota text only within the specified orthography, and the Corpus column indicates the performance of all text in the orthography.
Configuration
Lak
Overall
High Contrast ( Cw>0.61 )
GLMZS
30.88
28.12
GLMZS+ByT5POST
16.87
15.18
GLMSFT
5.20
4.97
GLMZS+GLMPOST
5.52
5.24
Medium Contrast ( 0.58<Cw≤0.61 )
Table 5: CER (%) across Weber optical contrast tiers on historical Deloria manuscripts.
Table 6: Parallel sentence illustrating orthographic differences between historical Deloria manuscripts and the Suggested Lakota Orthography (SLO).
Figure 4: An example of a Lakota page from the Deloria corpus. This page highlights the handwritten diacritics as well as document degradation.
Figure 5: An example of a bilingual page from the CULP learning materials. This page highlights the clean document quality in comparison to the Deloria materials as well as the more mixed language nature of the text. Where Deloria is easily separable at the page level by language, the Lakota and English are interleaved in the learning materials. Furthermore, the document highlights structural challenges like double columns and hole punches that can confuse automatic segmentation models.
Figure 6: An example of a Lakota sentence, its word-for-word translation, and free translation in the Deloria corpus.
Figure 7: An example of a low and high contrast line from the Deloria documents.
Figure 8: Examples of line crops and the corresponding transcriptions from various OCR and post-correction paradigms with GLM-OCR. We additionally include Tesseract to illustrate higher baseline error rates.
Model
ZS
Own
Best
Ensemble
GLM-OCR(0.9B)
17.09
3.87
3.87
4.05
Qwen3-VL(2.0B)
18.06
4.69
5.20
4.52
Appendix
Table 7: CER across draft conditioning settings. Own refers to post-correcting the zero-shot draft from the same model. Best refers to correcting the zero-shot draft of the VLM with the lowest zero-shot CER (GLM-OCR). Finally, ensemble refers to training the model on each zero-shot draft and corrected version as a separate data point. While we hypothesized that increasing the data size and diversity of errors may improve performance, the ensemble method actually makes performance worse for GLM and slightly improves Qwen3-VL. Because we use GLM-OCR for the majority of experiments, we omit ensembling. This also provides a simpler training setup.
Model
ZS
Diacritic
Base
Transkribus (Text Titan II)
15.73
61.20
5.79
GLM-OCR (0.9B)
17.09
85.28
2.75
Dots.ocr (1.7B)
18.06
74.32
5.28
Qwen3-VL (2B)
18.06
85.06
4.47
Nanonets OCR2 (3B)
20.32
68.37
9.42
Tesseract v5 (Lat)
32.58
84.93
28.14
Appendix
Table 9: Zero-shot character error rates (CER) across OCR and vision-language models, broken down by overall CER, diacritic-only CER, and base character CER (%).
Figure 9: Results of VLMZS+VLMcorr separated by language. The majority of gains is concentrated in Lakota whereas English and mixed language CERs improve little in comparison.
Figure 10: Results of VLMZS+VLMcorr evaluated on gold line-crops versus line-crops produced by Kraken, the automatic segmentation model. Error deltas remain relatively constant across training budgets, indicating that training on gold bounding boxes does not provide any improvements on automatic ones.
Figure 11: Edit operation counts for each of the training-based methods. For Seq2Seq models, deletions make up the largest proportion of edits. However, for LLM and VLM based methods, substitutions are more common, especially at tiny training budgets. For directly finetuning VLMs for OCR, we noticeably see a jump in insertions at 1 page of training data, likely indicating a spike in hallucinated text.
Optical character recognition (OCR) has advanced rapidly with deep learning and multimodal models, yet most methods focus on well-resourced scripts such as Latin and Chinese. Ethnic minority languages remain underexplored due to complex writing systems, scarce annotations, and diverse historical and modern forms, making generalization in low-resource or zero-shot settings challenging. To address these challenges, we present OmniOCR, a universal framework for ethnic minority scripts. OmniOCR introduces Dynamic Low-Rank Adaptation (Dynamic LoRA) to allocate model capacity across layers and scripts, enabling effective adaptation while preserving knowledge.A sparsity regularization prunes redundant updates, ensuring compact and efficient adaptation without extra inference cost. Evaluations on TibetanMNIST, Shui, ancient Yi, and Dongba show that OmniOCR outperforms zero-shot foundation models and standard post training, achieving state-of-the-art accuracy with superior parameter efficiency, and compared with the state-of-the-art baseline models, it improves accuracy by 39%-66% on these four datasets. Code: https://github.com/AIGeeksGroup/OmniOCR.
Despite the rapid advancement of Vision-Language Models (VLMs), their linguistic reach remains largely confined to high-resource languages, leaving the majority of the world's 7,000+ living languages on the wrong side of a growing digital divide. This disparity is especially pronounced in Optical Character Recognition (OCR), where low-resource scripts lack the massive datasets required for traditional scaling laws. We investigate OCR adaptation in extreme data-scarce regimes (<10K real and <250K synthetic images), demonstrating that conventional fine-tuning strategies often reach a performance ceiling. Our key finding reveals a structural inefficiency in language-specific adaptation: while higher layers of specialized models diverge to capture unique script nuances, the lower layers learn redundant, highly similar features. Motivated by this observation, we propose PSMC (Pre-train, Specialize, Merge, and Co-train), a data-efficient framework that capitalizes on a cross-script "transfer effect". Our approach first derives language-specific experts from a high-resource base model, then employs task arithmetic to fuse these experts into a unified, high-performance multilingual back- bone. Extensive evaluation across 10 Indian scripts (supporting 20+ languages) shows that PSMC achieves a ~2% average improvement in Word Recognition Rate (WRR) over individual specialist models without increasing parameter count. Our results indicate that joint training in the merged latent space facilitates a constructive knowledge transfer that benefits all constituent scripts, providing a scalable pathway for inclusive VLM development. Source code and datasets will be released post publication.
Recent work has shown that Vision-Language Models (VLMs) used for optical character recognition (OCR) can generate plausible but visually unsupported text, suggesting reliance on language priors. Comparing open-weight VLMs with traditional OCR baselines on low-resource Ancient Greek critical editions, we show that VLM errors often remain fluent even when wrong, producing plausible Greek substitutions where traditional engines produce local recognition noise. To analyze visual evidence during decoding, we introduce controlled image perturbations and token-level grounding measures based on conditional versus image-free decoding distributions. Under character-level perturbations, VLMs diverge sharply from the perturbed ground truth while traditional OCR remains comparatively faithful; however, token-level analysis shows that prior reliance is model-specific: in an OCR-specialist model, fluent lexical errors are produced with little reliance on the image, whereas general-purpose VLMs remain conditioned on the visual input even when wrong. Decode-time interventions fail to reliably restore grounding, while post-OCR language-model correction improves several systems only by repairing text after generation. Our results extend prior evidence of OCR language-prior reliance to low-resource historical documents and a broader set of models, showing that fluent output is not necessarily visually grounded and motivating interpretability-driven evaluation beyond aggregate accuracy.
Antonia Karamolegkou, Nicolas Angleraud, Benoît Sagot +1