GLM-OCR Technical Report

Shuaiqi Duan; Yadong Xue; Weihan Wang; Zhe Su; Huan Liu; Sheng Yang; Guobing Gan; Guo Wang

cs.CLMar 11, 2026

GLM-OCR Technical Report

Authors: Shuaiqi Duan, Yadong Xue, Weihan Wang, Zhe Su, Huan Liu, Sheng Yang, Guobing Gan, Guo Wang, +15 more

Organizations: Zhipu AI · Tsinghua University

Abstract

GLM-OCR is an efficient 0.9B-parameter compact multimodal model designed for real-world document understanding. It combines a 0.4B-parameter CogViT visual encoder with a 0.5B-parameter GLM language decoder, achieving a strong balance between computational efficiency and recognition performance. To address the inefficiency of standard autoregressive decoding in deterministic OCR tasks, GLM-OCR introduces a Multi-Token Prediction (MTP) mechanism that predicts multiple tokens per step, significantly improving decoding throughput while keeping memory overhead low through shared parameters. At the system level, a two-stage pipeline is adopted: PP-DocLayout-V3 first performs layout analysis, followed by parallel region-level recognition. Extensive evaluations on public benchmarks and industrial scenarios show that GLM-OCR achieves competitive or state-of-the-art performance in document parsing, text and formula transcription, table structure recovery, and key information extraction. Its compact architecture and structured generation make it suitable for both resource-constrained edge deployment and large-scale production systems.

Figures & tables

Figure 1: Performance of GLM-OCR on OmniDocBench v1.5.

Figure 2: Overall architecture and workflow of the GLM-OCR framework. This system supports two primary tasks: Document Parsing (Task 1), which combines layout detection and region cropping to produce structured Markdown and JSON outputs; and Key Information Extraction (Task 2), which directly extracts structured JSON data based on input visual prompts.

Stage	Phase	Data Types
Stage 1	Vision Encoder Training	Image-text pairs, Grounding / Retrieval data
Stage 2.1	Pretrain	Image-text pairs, Document parsing, Grounding, VQA
Stage 2.2	Pretrain with MTP	Document parsing, Grounding, VQA
Stage 3	SFT with MTP	Text / Formula / Table recognition, KIE
Stage 4	RL	Text / Formula / Table recognition, KIE

Table 1: Training Recipe for GLM-OCR across different stages.

Task	Primary Accuracy Reward	Additional Constraints
Text Recognition	Normalized Edit Distance	Repetition penalty
Formula Recognition	CDM score	Structural validity check
Table Recognition	TEDS score	Tag closure verification, structural parsing
Key Information Extraction (KIE)	Field-level F1 score	JSON parse validation, missing/duplicate field penalty
Global Regularization: Repetition ratio penalty, malformed structure penalty

Table 2: Reward function design in Stage 4 RL training.

Model Type	Methods	Params	Overall $\uparrow$	Text $\downarrow$ Edit	Formula $\uparrow$ CDM	Table $\uparrow$ TEDS	Table $\uparrow$ TEDS-S	Reading Order $\downarrow$ Edit
Pipeline Tools	Marker-1.8.2 [ 25 ]	-	71.30	0.206	76.66	57.88	71.17	0.250
	Mineru2-pipeline [ 23 ]	-	75.51	0.209	76.55	70.90	79.11	0.225
	PP-StructureV3 [ 7 ]	-	86.73	0.073	85.79	81.68	89.48	0.073
General VLMs	GPT-4o [ 1 ]	-	75.02	0.217	79.70	67.07	76.09	0.148
	InternVL3-76B [ 37 ]	76B	80.33	0.131	83.42	70.64	77.74	0.113
	InternVL3.5-241B [ 33 ]	241B	82.67	0.142	87.23	75.00	81.28	0.125

Table 4: Performance comparison with pipeline tools, general VLMs and specialized VLMs on OmniDocBench v1.5.

Task	GLM-OCR	PaddleOCR -VL-1.5	Deepseek -OCR2	MinerU 2.5	dots.ocr	Gemini-3 -Pro	GPT-5.2 -2025-12-11
Code Document	84.7	75.8	82.1	82.9	80.8	86.9	84.4
Real-world Table	91.5	86.1	-	70.8	81.8	90.6	86.7
Handwritten Text	87.0	87.4	73.8	54.2	71.7	90.0	78.0
Multilingual Text	69.3	54.8	56.1	27.8	65.1	86.2	70.1
Seal Recognition	90.5	42.2	40.4	-	63.0	91.3	58.8
Receipt KIE	94.5	-	-	-	-	97.3	83.5

Table 5: Performance comparison on custom real-world scenarios. The results of Gemini-3-Pro and GPT-5.2-2025-12-11 are in gray as they are provided for reference only and excluded from the best-score ranking. The best results among the evaluated models are highlighted in bold.

Model	Image Input	PDF Input
Model	(pages / s)	(pages / s)
GLM-OCR	0.67	1.86
PaddleOCR-VL-1.5	0.39	1.22
Deepseek-OCR2	0.32	$-$
MinerU2.5	0.18	0.48
dots.ocr	0.10	$-$

Table 6: Throughput comparison of different OCR models for Image and PDF inputs.

Figure 3: Example of GLM-OCR SDK for complex document parsing.

Figure 4: Example of GLM-OCR Text Recognition.

Figure 5: Example of GLM-OCR Table Recognition.

Figure 6: Example of GLM-OCR Formula Recognition.

Figure 7: Example of GLM-OCR Key Information Extraction.

Explore similar work

Multimodal OCR: Parse Anything from Documents
Mar 13, 2026Handong Zheng, Yumeng Li, Kaile Zhang +22Document Parsing Document Image Understanding
Huazhong University of Science and Technology · hi lab, Xiaohongshu Inc
PP-OCRv6: From 1.5M to 34.5M Parameters, Surpassing Billion-Scale VLMs on OCR Tasks
Jun 11, 2026Yubo Zhang, Xueqing Wang, Manhui Lin +13Optical Character Recognition
PaddlePaddle Team, Baidu Inc.
PP-OCRv5: A Specialized 5M-Parameter Model Rivaling Billion-Parameter Vision-Language Models on OCR Tasks
Mar 25, 2026Cheng Cui, Yubo Zhang, Ting Sun +11Optical Character Recognition Training Data Curation
PaddlePaddle Team, Baidu Inc.

Mar 13, 2026J/K move · Enter open · S save

Multimodal OCR: Parse Anything from Documents

Handong Zheng, Yumeng Li, Kaile Zhang, Liang Xin+21

Huazhong University of Science and Technology · hi lab, Xiaohongshu Inc

Document Parsing Document Image Understanding Optical Character Recognition

Open paper AskPDF arXiv

We present Multimodal OCR (MOCR), a document parsing paradigm that jointly parses text and graphics into unified textual representations. Unlike conventional OCR systems that focus on text recognition and leave graphical regions as cropped pixels, our method, termed dots.mocr, treats visual elements such as charts, diagrams, tables, and icons as first-class parsing targets, enabling systems to parse documents while preserving semantic relationships across elements. It offers several advantages: (1) it reconstructs both text and graphics as structured outputs, enabling more faithful document reconstruction; (2) it supports end-to-end training over heterogeneous document elements, allowing models to exploit semantic relations between textual and visual components; and (3) it converts previously discarded graphics into reusable code-level supervision, unlocking multimodal supervision embedded in existing documents. To make this paradigm practical at scale, we build a comprehensive data engine from PDFs, rendered webpages, and native SVG assets, and train a compact 3B-parameter model through staged pretraining and supervised fine-tuning. We evaluate dots.mocr from two perspectives: document parsing and structured graphics parsing. On document parsing benchmarks, it ranks second only to Gemini 3 Pro on our OCR Arena Elo leaderboard, surpasses existing open-source document parsing systems, and sets a new state of the art of 83.9 on olmOCR Bench. On structured graphics parsing, our model achieves higher reconstruction quality than Gemini 3 Pro across image-to-SVG benchmarks, demonstrating strong performance on charts, UI layouts, scientific figures, and chemical diagrams. These results show a scalable path toward building large-scale image-to-code corpora for multimodal pretraining. Code and models are publicly available at https://github.com/rednote-hilab/dots.mocr.

Abstract

Explore similar work

Multimodal OCR: Parse Anything from Documents

PP-OCRv6: From 1.5M to 34.5M Parameters, Surpassing Billion-Scale VLMs on OCR Tasks

PP-OCRv5: A Specialized 5M-Parameter Model Rivaling Billion-Parameter Vision-Language Models on OCR Tasks