Organizations: State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China. · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China. · Beijing Key Laboratory of Super Intelligent Security of Multi-Modal Information, Beijing, China. · Northwestern Polytechnical University, Xi’an 710072, China. · Baidu Inc., Beijing, China. · School of Software Engineering, Huazhong University of Science and Technology, Wuhan, China.
Optical Character Recognition (OCR) is among the oldest and most widely deployed applications of artificial intelligence. This survey reviews its development, focusing on the Foundation Model Era. We define OCR through faithful transcription: within a specified visual scope, OCR should completely reconstruct the text, notation, and document structure observable in the source image, and every textual, notational, or structural element in the output must be verifiable from visual evidence in that image. We trace the development from specialist OCR models toward unified models, examining joint text detection and recognition, structured sequence generation, and the use of pretrained vision-language models (VLMs). We organize contemporary OCR systems into five coexisting families: modular toolkits and task-specific models, specialized document VLMs, hybrid workflows, general-purpose VLMs, and commercial products and managed services. We compare their architectures, data, and training, examining hallucination and omission, insufficient spatial grounding, and inference cost in generative OCR. We review evaluation from character and scene text recognition to parsing individual structural elements and full-page transcription, highlighting gaps in consistent output definitions, attribution of performance to individual components, and coverage of real capture conditions and visible content and structure. Finally, we summarize the challenges facing OCR and discuss directions for future research. We continuously track related work at https://github.com/zhihengli-casia/Awesome-OCR-in-the-Foundation-Model-Era.
Figures & tables
Fig. 1: Scope of this survey. Within a specified visual scope, OCR aims to completely recover visible text, notation, and document structure, with every output element verifiable against the source image. Main coverage spans printed, handwritten, and scene text; formulas as notation; and tables, layout, and reading order as structure. Key information extraction, question answering, translation, and retrieval are downstream tasks.
Fig. 2: OCR and related document modeling across five historical tracks and three eras. Columns show representative pipelines; braces mark three eras: I. Classical OCR Era , II. Deep Learning Era , and III. Foundation Model Era . The dashed Document Pretraining box provides related background: these models combine upstream OCR text, layout, and visual features for document understanding. Generative parsing reconstructs text, formulas, and tables within a unified structured document, using formats such as Markdown, HTML, L a T e X , or JSON. Dates indicate approximate emergence or historical prominence; the tracks overlap in time and several remain active in contemporary systems.
Fig. 3: Review framework and evidence map. The figure connects OCR’s historical evolution and five contemporary system families with benchmark evidence, current limitations, and the three expectations of the outlook. Citations identify representative references.
Fig. 4: Representative milestones in OCR and related document modeling, following the five tracks in Fig. 2 . Years mark publications, public announcements, or system releases; horizontal spacing is schematic. Asterisks identify precursors: the optophone as a reading aid, and Donut and Pix2Struct in document understanding and image-to-sequence pretraining. Section II covers classical and deep learning methods; Section III examines unified models and generative parsing.
Fig. 5: Five system families in the OCR ecosystem of the Foundation Model Era, showing how each converts visual input into structured output. Each flow represents a family: modular toolkits and task-specific models combine task-specific prediction with modular processing, specialized document VLMs perform full-page generation with a model trained for documents, hybrid workflows scaffold generation with explicit layout analysis and VLM recognition of individual regions, general-purpose VLMs expose transcription through open prompting, and commercial products and managed services expose closed stacks through product interfaces or managed endpoints. Table I lists representative systems.
Model / System
Date
Typical output
Params
Access
F1 : Modular toolkits and task-specific models
Modular toolkits
PP-OCRv6 [ 54 ]
2026.06
Text, Geometry
1.5–34.5M
PP-StructureV3 [ 145 ]
2025.05
Markdown, JSON
–
Docling [ 55 ]
2024.07
Markdown, JSON
–
docTR [ 131 ]
2021.03
Text, Geometry
–
TABLE I: Representative systems in the OCR ecosystem across the five families ( F1 – F5 ). Date gives the first documented public appearance of each work, version, or product family (YYYY.MM; YYYY.– indicates an unavailable month). Outputs list representative content types or formats: Text includes answers and captions for downstream tasks, and Geometry covers boxes, polygons, and points. Params gives official nominal counts for the named model or family (the core VLM in F3 ); “–” marks undisclosed or non-unique values. † marks compact encoder–decoder precursors: Donut for document understanding and Nougat for transcription. Access icons link to official code, model hubs, papers, or services. Full inventory at https://github.com/zhihengli-casia/Awesome-OCR-in-the-Foundation-Model-Era .
Fig. 6: Data sources (left) and training pipelines (right) for the first four system families. Modular toolkits and task-specific models use task-specific data and supervision; specialized document VLMs use pages and region crops for pretraining, supervised fine-tuning, and optional reinforcement learning with verifiable rewards; hybrid workflows train separate layout and recognition models or a shared VLM for both tasks; and general-purpose VLMs use broad multimodal corpora with alignment, joint pretraining, instruction tuning, and optional post-training.
Fig. 7: Evolution of evaluation relevant to OCR across five stages, combining representative input–output contracts with a chronological timeline of benchmark datasets. ❶ Text recognition : controlled characters, words, and lines (MNIST, IAM, CASIA-HWDB). ❷ Scene text : detection and recognition in natural images (ICDAR, SVT, Total-Text, CTW1500). ❸ Document structure components : tables and formulas as structured markup, plus layouts as labeled spatial regions (PubTabNet, PubLayNet, DocLayNet, HME100K). ❹ Downstream document understanding : question answering about documents (DocVQA, ChartQA, DUDE). ❺ Foundation Model OCR Evaluation , split into two tracks: VLM OCR ability diagnostics (OCRBench and OCRBench v2) and benchmarks for parsing full pages or multipage documents (OmniDocBench, Real5-OmniDocBench, MDPBench, MPDocBench-Parse, and PureDocBench). These evaluation directions expand and coexist; VLM diagnostic suites may combine transcription and understanding tasks. Stars mark selected landmark benchmarks; Table II lists dataset details and metrics.
Benchmark
Year
Input
Output
Main metrics
Scale
Access
❶ Controlled text recognition
Character classification
EMNIST [ 182 ]
2017
Crop
Labels
Accuracy
814K images
SVHN [ 183 ]
2011
Crop
Labels
Accuracy
600K+ images
MNIST [ 33 ]
1998
Crop
Labels
Accuracy
70K images
Handwriting character, line, and word recognition
TABLE II: Representative benchmarks across five stages of evaluation relevant to OCR (newest first within each subgroup). Year gives the first public paper or dataset release, or the named version release. Input uses Page for a single page, Multipage for a document, and Mixed for heterogeneous inputs. Geometry denotes boxes or polygons. Main metrics follow official protocols; ED denotes edit distance, CDM evaluates formulas, and TEDS evaluates tables. Scale refers to the named release or split; K/M counts are rounded and test splits are marked. Access icons link to official repositories, dataset pages, or papers. Full list: project page .
Method
Date
IC15
Total
CTW
PSENet [ 116 ]
2018.06
85.7
80.9
82.2
TextSnake [ 203 ]
2018.07
82.6
78.4
75.6
PAN [ 204 ]
2019.08
82.9
85.0
83.7
DBNet [ 41 ]
2019.11
87.3
84.7
84.9
TextFuseNet [ 205 ]
2020.07
92.1
87.1
86.6
FCENet [ 206 ]
2021.04
86.2
85.8
85.5
TABLE III: Scene text detection Hmean/F1 (%).
Model
Date
Params
IIIT5K
SVT
IC13
IC15
SVTP
CUTE
CRNN [ 23 ]
2015.07
8.3 M
82.9
81.6
91.1
69.4
70.0
65.5
ASTER [ 42 ]
2018.06
27.2 M
93.3
90.0
90.8
74.7
80.2
80.9
ABINet [ 119 ]
2021.03
36.7 M
96.2
93.5
97.4
86.0
89.3
89.2
SVTR-B [ 120 ]
2022.05
24.6 M
96.0
91.5
97.1
85.2
89.9
91.7
PARSeq [ 105 ]
2022.07
23.8 M
97.0
93.6
97.0
86.5
88.9
92.2
SVTRv2-B [ 56 ]
2024.11
19.8 M
97.7
94.0
97.3
88.1
91.2
95.8
TABLE IV: Scene text recognition accuracy (%) under MJ+ST, following the comparison compiled in SVTRv2 [ 56 ] . IC13/IC15 use the 857/1811-image subsets.
IC15
Total-Text
CTW1500
Method
Date
None
None
Full
None
Full
FOTS [ 44 ]
2018
–
–
–
21.1
39.7
Mask TextSpotter v2 [ 209 ]
2019
–
65.3
77.4
–
–
TextDragon [ 210 ]
2019
–
48.8
74.8
39.7
72.4
ABCNet [ 121 ]
2020
–
64.2
75.7
45.2
74.1
ABCNet v2 [ 211 ]
2021
–
70.4
78.1
57.5
77.2
TABLE V: End-to-end scene text spotting F1 (%). Dates refer to the first public papers. “None” is lexicon-free; “Full” uses all words in the test set.
System
Date
Params
Overall
Text
Table
Fml.
RO
Specialized document VLMs
OvisOCR2 [ 58 ]
2026.07
0.8 B
96.47
97.35
94.58
97.49
0.112
Unlimited-OCR [ 135 ]
2026.06
3 B
94.00
96.06
90.21
95.72
0.128
Qianfan-OCR [ 59 ]
2026.03
4 B
93.90
96.00
90.53
95.08
0.130
FireRed-OCR [ 149 ]
2026.02
2 B
93.26
96.30
88.04
95.44
0.131
Hybrid workflows
TABLE VI: Selected OmniDocBench v1.6_full scores [ 155 ] as of October 2026. Overall/Text/Table/Fml. are % ( ↑ ); Text is (1−Edit)×100 , Table is TEDS, Fml. is CDM, and RO is reading order Edit ( ↓ ).
System
All
S
W
SP
I
SK
Worst
Modular toolkits and task-specific models
PP-StructureV3 [ 145 ]
64.45
84.68
59.34
66.89
73.38
37.98
37.98
Specialized document VLMs
OvisOCR2 [ 58 ]
92.29
93.77
91.40
93.09
92.88
90.33
90.33
Hybrid workflows
PaddleOCR-VL-1.6 [ 62 ]
93.19
94.74
92.48
92.78
93.28
92.66
92.48
TABLE VII: Selected Real5-OmniDocBench leaderboard scores [ 216 ] (%, ↑ ), as of October 2026, using the pinned OmniDocBench v1.5 protocol. S/W/SP/I/SK = scan/warp/screen/illum./skew; All averages the five condition scores; Worst is their minimum.
Does a general vision--language model acquire specialized OCR ability by developing a new reading circuit or by reusing an existing mechanism? We address this question in the setting of full-sequence OCR, rather than local-answer retrieval. Using an evidence-grounded protocol with held-out causal interventions, we identify sparse and stable OCR-head sets in GLM-OCR, MinerU2.5, and PaddleOCR-VL-1.6. We then investigate the mechanistic origin of these OCR heads by comparing them with independently identified textual retrieval/copy heads in general VLMs. Across two general VLMs, visual OCR heads strongly overlap independently identified textual retrieval/copy heads, yielding untuned top-20 intersections of 73.3% and all-head Spearman correlations of 0.677-0.886. The overlap and causal interventions suggest that full-sequence OCR operates as dense sequential multimodal copy-and-paste, repeatedly retrieving visual evidence and routing it to the current output position. Finally, we examine how this shared circuit changes as a general VLM becomes an OCR specialist. Matched base-to-specialized comparisons show that OCR specialization largely preserves head identity, retaining 17-20 of the top 20 heads per task with all-head rank correlations of 0.874-0.942, while redistributing their functional and causal strengths.
Optical Character Recognition (OCR) is evolving from plain-text transcription toward general visual intelligence, requiring models to recognize, localize, and reason over textual information in complex visual environments. However, existing OCR systems often excel at only some tasks and struggle to balance recognition, parsing, and reasoning across scenarios. In this report, we present PolyOCR, a family of unified OCR foundation models of varying scales. PolyOCR combines a shared instruction-following framework with a large-scale data engine that converts heterogeneous visual resources into quality-verified OCR supervision. We introduce Competence-Guided Policy Optimization, which combines verifier-based Group Relative Policy Optimization with on-policy distillation through sample-wise routing based on teacher reliability and the teacher--student competence gap. We also introduce OCRBench v2.1, our revision of OCRBench v2 with manually verified annotation corrections and task-aligned scoring metrics. Extensive experiments across OCRBench v2.1, CC-OCR, in-house KIE Benchmark, OmniDocBench v1.6 and MDPBench demonstrate that PolyOCR achieves state-of-the-art or highly competitive performance.
The advent of "OCR 2.0" and large-scale vision-language models (VLMs) has set new benchmarks in text recognition. However, these unified architectures often come with significant computational demands, challenges in precise text localization within complex layouts, and a propensity for textual hallucinations. Revisiting the prevailing notion that model scale is the sole path to high accuracy, this paper introduces PP-OCRv5, a meticulously optimized, lightweight OCR system with merely 5 million parameters. We demonstrate that PP-OCRv5 achieves performance competitive with many billion-parameter VLMs on standard OCR benchmarks, while offering superior localization precision and reduced hallucinations. The cornerstone of our success lies not in architectural expansion but in a data-centric investigation. We systematically dissect the role of training data by quantifying three critical dimensions: data difficulty, data accuracy, and data diversity. Our extensive experiments reveal that with a sufficient volume of high-quality, accurately labeled, and diverse data, the performance ceiling for traditional, efficient two-stage OCR pipelines is far higher than commonly assumed. This work provides compelling evidence for the viability of lightweight, specialized models in the large-model era and offers practical insights into data curation for OCR. The source code and models are publicly available at https://github.com/PaddlePaddle/PaddleOCR.