DharmaOCR: Specialized Small Language Models for Structured OCR that outperform Open-Source and Commercial Baselines

Gabriel Pimenta de Freitas Cardoso; Caio Lucas da Silva Chacon; Jonas Felipe da Fonseca Oliveira; Paulo Henrique de Medeiros Araujo

cs.CVApr 15, 2026

DharmaOCR: Specialized Small Language Models for Structured OCR that outperform Open-Source and Commercial Baselines

Authors: Gabriel Pimenta de Freitas Cardoso, Caio Lucas da Silva Chacon, Jonas Felipe da Fonseca Oliveira, Paulo Henrique de Medeiros Araujo

Organizations: Dharma-AI

Abstract

This manuscript introduces DharmaOCR Full and Lite, a pair of specialized small language models (SSLMs) for structured OCR that jointly optimize transcription quality, generation stability, and inference cost. It also presents DharmaOCR-Benchmark, a benchmark that covers printed, handwritten, and legal/administrative documents, and proposes a unified evaluation protocol that measures fidelity and structure while explicitly tracking text degeneration as a first-class benchmark metric (alongside unit cost). Beyond reporting degeneration rates, the manuscript empirically shows degeneration is not merely a quality failure, since it materially worsens production performance by increasing response time, reducing throughput, and inflating computational cost due to abnormally long generations. To the best of the author's knowledge, as a methodological contribution, this is the first application of Direct Preference Optimization (DPO) for OCR, explicitly using degenerate generations as rejected examples to penalize looping behavior. Combined with Supervised Fine-Tuning (SFT) for enforcing a strict JSON schema (header, margin, footer, and text), DPO consistently reduces degeneration rate across model families (up to 87.6% relative) while preserving or improving extraction quality. The resulting models, namely, DharmaOCR Full (7B) and DharmaOCR Lite (3B), set a new state-of-the-art on DharmaOCR-Benchmark, outperforming each open-source and commercial baseline model evaluated regarding extraction quality, reaching 0.925 and 0.911 scores with 0.40% and 0.20% degeneration rates. AWQ quantization reduced up to 22% per-page cost with negligible quality loss, enabling a strong quality-cost trade-off in comparison to proprietary OCR APIs and open-source alternatives.

Figures & tables

Figure 1: Synthesis of the proposed approach, key contributions, and results, illustrating the progression from vanilla models to improvements enabled by specialization (SFT+DPO) and text degeneration mitigation, outperforming other models on DharmaOCR-Benchmark.

Figure 2: Pictorial example of token- and sequence-level text degeneration, in which a single token (or token sequence) dominates the conditional distribution, producing repetitions indefinitely.

Model / Technique	Score	Degeneration rate (%)	Time per page (s)	Relative cost (%)
Fine-tuned models
olmOCR-2-7B (SFT)	0.928	1.01	2.717	63.09%
olmOCR-2-7B (SFT + DPO)	0.927	0.40	2.730	63.40%
olmOCR-2-7B (LoRA SFT)	0.926	0.60	2.997	69.60%
olmOCR-2-7B (SFT + DPO + Quant)	0.925	0.40	2.132	49.50%
olmOCR-2-7B (Vanilla)	0.823	1.41	4.306	100.00%

Table 1: Results of the models evaluated on DharmaOCR-Benchmark. Parentheses in the first column indicate the specialization techniques used. When a model is not indicated as LoRA, it means that full fine-tuning has been performed. Entries marked with “Quant” indicate AWQ-quantized variant with best performance among the quantized configurations, as detailed in Appendix A.6 .

Panel of Figure 3 — Figure 3: Text degeneration rate (%) across alignment stages. SFT reduces degeneration relative to vanilla models in most cases, whereas DPO further reduces it, even compared to the SFT-tuned model.

Panel of Figure 4 — Figure 4: Quality–cost comparison among DharmaOCR models developed in this research, other open-source OCR models, and representative commercial APIs. Since the underlying model costs for commercial APIs are not available, their public prices were used as the comparison baseline.

Panel of Figure 5 — Figure 5: LLM-as-a-judge results from a comparison between DharmaOCR Full and Google Document AI. Bars show the percentage of pages in which the judge gave preference to each model or declared a tie for each evaluation criterion.

Panel of Figure 6 — Figure 6: LLM-as-a-judge results from a comparison between DharmaOCR Full and olmOCR-2-7B. Bars show the percentage of pages in which the judge gave preference to each model or declared a tie for each evaluation criterion.

Panel of Figure 7 — Figure 7: LLM-as-a-judge results from a comparison between DharmaOCR Full and DharmaOCR Lite. Bars show the percentage of pages in which the judge gave preference to each model or declared a tie for each evaluation criterion.

Figure 8: Progressive specialization strategy and comparison of two training paths. Three specialization levels are shown—vanilla generalist (Level 1), general-domain OCR specialist (Level 2), and domain-specific OCR specialist (Level 3)—plus a projected Level $N$ for future sub-domain specialization.

Model rate (%)	SFT degeneration rate (%)	SFT+DPO degeneration rate (%)	Relative reduction
olmOCR-2-7B	1.01	0.60	40.6%
Nanonets-OCR2-3B	1.61	0.20	87.6%
Qwen2.5-VL-7B	1.61	1.01	37.3%
Qwen2.5-VL-3B	3.23	1.41	56.4%
gemma-3-4b-it	16.13	4.03	75.0%

Table 2: Comparison of text degeneration rate between models tuned only with SFT and those tuned with SFT followed by DPO.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Parameter	Value
gpu_memory_utilization	0.90
tensor_parallel_size	1
max_num_batched_tokens	32000
max_model_len	65536
temperature	0
max_tokens	8192

Appendix

Table 3: vLLM configuration used for the inference runs in the text degeneration analysis.

Dataset	Observed time	No-degeneration time	$\Delta$ time	Relative reduction
1	21.7 min	19.5 min	2.2 min	10.14 %
2	11.2 min	10.8 min	0.4 min	3.57 %
3	7.3 min	4.2 min	3.1 min	42.47 %

Appendix

Table 4: Comparison between observed inference time for each dataset and the time obtained by replacing degenerate requests with requests of average (non-degenerate) duration.

Figure 9: Example of document used to illustrate the structured output format.

Figure 10: Example document used to illustrate structured output format for handwritten document.

Criterion	Prompt
Completeness	Is the extraction complete?, i.e., did the model detect and extract all text present in the image without omitting characters, words, lines, or text blocks? Does the extraction cover all textual elements visible in the original image?
Precision	Is the extraction precise?, i.e., does the extracted text exactly match the original text with no typos, character substitutions, letter inversions, or hallucinated words that are not present in the image?
Formatting	Does the extraction preserve the appropriate formatting?, i.e., did the model maintain the original text’s visual structure, respecting line breaks, paragraphs, spacing, alignment, and the overall layout of the content as it appears in the image?
Structure Adherence	Does the extraction adhere to the expected json structure? The expected structure is {"text": "extracted_text", "header": "header_contet", "margin": "margin_contet", "footer": "footer_contet"} . If json format is respected, does the extraction correctly assign values to the corresponding fields, assigning null values to fields absent from the image?

Appendix

Table 5: Prompts for the evaluation criteria used during labeling.

Condition	Requirement	Motivation	Action
$s_{w}<800$ and $s_{l}<800$	—	Unreliable signal	Discard
$\max(s_{w},s_{l})\geq 800$ and $\max(s_{w},s_{l})<900$ )	$s_{w}-s_{l}\geq 400$	Avoid small margins in mid-quality regime	Include
$\max(s_{w},s_{l})\geq 900$	$s_{w}-s_{l}\geq 200$	Informative pair with a guaranteed minimum margin	Include

Appendix

Table 6: Filtering and selection criteria for preference pairs, where

s_{w}

and

s_{l}

denote aggregated scores of chosen and rejected responses in the pair, respectively.

Model	Technique	Benchmark Score	Degeneration Rate (%)	Time per page (s)
olmOCR-2-7B (SFT + DPO)	AWQ W8A8	0.926	0.60	2.122
olmOCR-2-7B (SFT + DPO)	AWQ FP8	0.925	1.21	2.187
olmOCR-2-7B (SFT + DPO)	AWQ W4A16	0.903	1.61	2.004
Nanonets-OCR2-3B (SFT + DPO)	AWQ FP8	0.912	0.20	1.463
Nanonets-OCR2-3B (SFT + DPO)	AWQ W8A8	0.903	1.81	1.500
Nanonets-OCR2-3B (SFT + DPO)	AWQ W4A16	0.831	11.09	1.821

Appendix

Table 7: Performance Analysis: AWQ-Quantized Models

ID	Criterion	Guiding Question
C1	Comprehensibility	Which text enables information to be extracted more easily?
C2	Precision	Which text exhibits the lowest rate of extraction errors?
C3	Completeness	Which model extracted the largest amount of relevant information?
C4	Fidelity	Which text is more faithful, avoiding hallucinated words that are not present?
C5	General Adequacy	Which model is more robust for deployment in production systems?
C6	Formatting	Which model better preserves the structure?

Appendix

Table 8: Technical Evaluation Criteria (GPT-4o Judge)

Explore similar work

Feb 18, 2026

DODO: Discrete OCR Diffusion Models

Optical Character Recognition (OCR) is a fundamental task for digitizing information, serving as a critical bridge between visual data and textual understanding. While modern Vision-Language Models (VLM) have achieved high accuracy in this domain, they predominantly rely on autoregressive decoding, which becomes computationally expensive and slow for long documents as it requires a sequential forward pass for every generated token. We identify a key opportunity to overcome this bottleneck: unlike open-ended generation, OCR is a highly deterministic task where the visual input strictly dictates a unique output sequence, theoretically enabling efficient, parallel decoding via diffusion models. However, we show that existing masked diffusion models fail to harness this potential; those introduce structural instabilities that are benign in flexible tasks, like captioning, but catastrophic for the rigid, exact-match requirements of OCR. To bridge this gap, we introduce DODO, the first VLM to utilize block discrete diffusion and unlock its speedup potential for OCR. By decomposing generation into blocks, DODO mitigates the synchronization errors of global diffusion. Empirically, our method achieves near state-of-the-art accuracy while enabling up to 5x faster inference compared to autoregressive baselines.

Sean Man, Gilad Deutch, Roy Ganz +4

Technion - Israel Institute of Technology, Haifa, Israel. · Amazon Web Services.

Optical Character Recognition Diffusion Language Model Decoding Vision-Language Models Blockwise Diffusion Language Models

Ask PDF

Sep 22, 2026

Diffusion Drafts, AR Verifies: Accelerating Document OCR with Self-Speculative Decoding

Autoregressive OCR vision-language models accurately convert document images into text and structured markup, but require one sequential decoding step per output token, limiting inference speed. Unlike open-ended text generation, OCR outputs are strongly grounded in the input image, making diffusion-based parallel generation promising. However, when several tokens are predicted in one diffusion step, each is predicted before the others are known. Committing them directly can therefore introduce errors. We therefore introduce GravityOCR, a parameter-shared AR-block-diffusion model jointly trained for parallel drafting and causal AR verification. Verifying drafts before commitment lets the model commit multiple output tokens per round without a separate drafting network. The causal AR path also enables GRPO with sequence- and structure-level OCR rewards, avoiding diffusion-trajectory likelihood estimation while updating the shared drafter parameters. On OmniDocBench v1.6, AR-path GRPO improves the Overall score from 94.92 to 95.16 without reducing diffusion drafting efficiency, while the final model remains close to the original GLM-OCR score of 95.48. In an SGLang serving deployment, GravityOCR commits an average of 9.7 output tokens per forward pass and achieves a

3.94\times

decode-only speedup on region crops and a

1.32\times

end-to-end page-processing speedup over AR decoding.

Dohyun Kim, Sungjun Han, Hyungguk Kim +4

Trillion Labs · Korea University · Seoul National University

Autoregressive Diffusion Speculative Decoding Diffusion Language Model Inference Optical Character Recognition Diffusion Model Inference Acceleration

Ask PDF

Jul 6, 2026

HunyuanOCR-1.5: Making Lightweight OCR VLMs Faster and Better

We present HunyuanOCR-1.5, a lightweight end-to-end OCR-specialized vision-language model. HunyuanOCR unifies document parsing, text spotting, information extraction, text-image translation, and multi-image document understanding within a single end-to-end VLM. Building upon the lightweight architecture of HunyuanOCR-1.0, HunyuanOCR-1.5 does not redesign the backbone, but systematically improves both efficiency and capability. For efficiency, we adapt DFlash to OCR decoding, significantly reducing the latency of long structured outputs such as dense documents, tables, and formulas while preserving output distribution. Powered by DFlash, HunyuanOCR-1.5 achieves a 6.37x Transformer inference speedup and a 2.14x speedup under vLLM, delivering the fastest inference among lightweight OCR VLMs. For capability, we propose Agentic Data Flow, an agent-driven data construction system that transforms model weaknesses into executable data requirements and autonomously performs material search, quality verification, and pipeline development. It substantially improves long-tail capabilities in ancient-script OCR, fine-grained chart and table parsing, multi-image text-centric QA, low-resource multilingual parsing, and document hallucination evaluation. HunyuanOCR-1.5 ranks among the top-tier end-to-end OCR solutions on OmniDocBench v1.6 while achieving new performance milestones across these long-tail tasks. Combined with an upgraded pretraining and post-training recipe, HunyuanOCR-1.5 further extends its capability in high-resolution, long-context, and multi-task scenarios. Experiments demonstrate faster inference, broader OCR capability coverage, and the deployment advantages of a lightweight end-to-end model. We will release the model weights and training code to support future research and real-world OCR applications.

Gengluo Li, Xingyu Wan, Shangpin Peng +20

Institute of Information Engineering, Chinese Academy of Sciences · Large Language Model Department, Tencent · Nankai University

Vision-Language Models Efficient VLM Inference Optical Character Recognition Document Image Understanding

Ask PDF