Organizations: State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China. · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China. · Beijing Key Laboratory of Super Intelligent Security of Multi-Modal Information, Beijing, China. · Northwestern Polytechnical University, Xi’an 710072, China. · Baidu Inc., Beijing, China. · School of Software Engineering, Huazhong University of Science and Technology, Wuhan, China.
Optical Character Recognition (OCR) is among the oldest and most widely deployed applications of artificial intelligence. This survey reviews its development, focusing on the Foundation Model Era. We define OCR through faithful transcription: within a specified visual scope, OCR should completely reconstruct the text, notation, and document structure observable in the source image, and every textual, notational, or structural element in the output must be verifiable from visual evidence in that image. We trace the development from specialist OCR models toward unified models, examining joint text detection and recognition, structured sequence generation, and the use of pretrained vision-language models (VLMs). We organize contemporary OCR systems into five coexisting families: modular toolkits and task-specific models, specialized document VLMs, hybrid workflows, general-purpose VLMs, and commercial products and managed services. We compare their architectures, data, and training, examining hallucination and omission, insufficient spatial grounding, and inference cost in generative OCR. We review evaluation from character and scene text recognition to parsing individual structural elements and full-page transcription, highlighting gaps in consistent output definitions, attribution of performance to individual components, and coverage of real capture conditions and visible content and structure. Finally, we summarize the challenges facing OCR and discuss directions for future research. We continuously track related work at https://github.com/zhihengli-casia/Awesome-OCR-in-the-Foundation-Model-Era.
Figures & tables
Fig. 1: Scope of this survey. Within a specified visual scope, OCR aims to completely recover visible text, notation, and document structure, with every output element verifiable against the source image. Main coverage spans printed, handwritten, and scene text; formulas as notation; and tables, layout, and reading order as structure. Key information extraction, question answering, translation, and retrieval are downstream tasks.
Fig. 2: OCR and related document modeling across five historical tracks and three eras. Columns show representative pipelines; braces mark three eras: I. Classical OCR Era , II. Deep Learning Era , and III. Foundation Model Era . The dashed Document Pretraining box provides related background: these models combine upstream OCR text, layout, and visual features for document understanding. Generative parsing reconstructs text, formulas, and tables within a unified structured document, using formats such as Markdown, HTML, L a T e X , or JSON. Dates indicate approximate emergence or historical prominence; the tracks overlap in time and several remain active in contemporary systems.
Fig. 3: Review framework and evidence map. The figure connects OCR’s historical evolution and five contemporary system families with benchmark evidence, current limitations, and the three expectations of the outlook. Citations identify representative references.
Fig. 4: Representative milestones in OCR and related document modeling, following the five tracks in Fig. 2 . Years mark publications, public announcements, or system releases; horizontal spacing is schematic. Asterisks identify precursors: the optophone as a reading aid, and Donut and Pix2Struct in document understanding and image-to-sequence pretraining. Section II covers classical and deep learning methods; Section III examines unified models and generative parsing.
Fig. 5: Five system families in the OCR ecosystem of the Foundation Model Era, showing how each converts visual input into structured output. Each flow represents a family: modular toolkits and task-specific models combine task-specific prediction with modular processing, specialized document VLMs perform full-page generation with a model trained for documents, hybrid workflows scaffold generation with explicit layout analysis and VLM recognition of individual regions, general-purpose VLMs expose transcription through open prompting, and commercial products and managed services expose closed stacks through product interfaces or managed endpoints. Table I lists representative systems.
Model / System
Date
Typical output
Params
Access
F1 : Modular toolkits and task-specific models
Modular toolkits
PP-OCRv6 [ 54 ]
2026.06
Text, Geometry
1.5–34.5M
PP-StructureV3 [ 145 ]
2025.05
Markdown, JSON
–
Docling [ 55 ]
2024.07
Markdown, JSON
–
docTR [ 131 ]
2021.03
Text, Geometry
–
TABLE I: Representative systems in the OCR ecosystem across the five families ( F1 – F5 ). Date gives the first documented public appearance of each work, version, or product family (YYYY.MM; YYYY.– indicates an unavailable month). Outputs list representative content types or formats: Text includes answers and captions for downstream tasks, and Geometry covers boxes, polygons, and points. Params gives official nominal counts for the named model or family (the core VLM in F3 ); “–” marks undisclosed or non-unique values. † marks compact encoder–decoder precursors: Donut for document understanding and Nougat for transcription. Access icons link to official code, model hubs, papers, or services. Full inventory at https://github.com/zhihengli-casia/Awesome-OCR-in-the-Foundation-Model-Era .
Fig. 6: Data sources (left) and training pipelines (right) for the first four system families. Modular toolkits and task-specific models use task-specific data and supervision; specialized document VLMs use pages and region crops for pretraining, supervised fine-tuning, and optional reinforcement learning with verifiable rewards; hybrid workflows train separate layout and recognition models or a shared VLM for both tasks; and general-purpose VLMs use broad multimodal corpora with alignment, joint pretraining, instruction tuning, and optional post-training.
Fig. 7: Evolution of evaluation relevant to OCR across five stages, combining representative input–output contracts with a chronological timeline of benchmark datasets. ❶ Text recognition : controlled characters, words, and lines (MNIST, IAM, CASIA-HWDB). ❷ Scene text : detection and recognition in natural images (ICDAR, SVT, Total-Text, CTW1500). ❸ Document structure components : tables and formulas as structured markup, plus layouts as labeled spatial regions (PubTabNet, PubLayNet, DocLayNet, HME100K). ❹ Downstream document understanding : question answering about documents (DocVQA, ChartQA, DUDE). ❺ Foundation Model OCR Evaluation , split into two tracks: VLM OCR ability diagnostics (OCRBench and OCRBench v2) and benchmarks for parsing full pages or multipage documents (OmniDocBench, Real5-OmniDocBench, MDPBench, MPDocBench-Parse, and PureDocBench). These evaluation directions expand and coexist; VLM diagnostic suites may combine transcription and understanding tasks. Stars mark selected landmark benchmarks; Table II lists dataset details and metrics.
Benchmark
Year
Input
Output
Main metrics
Scale
Access
❶ Controlled text recognition
Character classification
EMNIST [ 182 ]
2017
Crop
Labels
Accuracy
814K images
SVHN [ 183 ]
2011
Crop
Labels
Accuracy
600K+ images
MNIST [ 33 ]
1998
Crop
Labels
Accuracy
70K images
Handwriting character, line, and word recognition
TABLE II: Representative benchmarks across five stages of evaluation relevant to OCR (newest first within each subgroup). Year gives the first public paper or dataset release, or the named version release. Input uses Page for a single page, Multipage for a document, and Mixed for heterogeneous inputs. Geometry denotes boxes or polygons. Main metrics follow official protocols; ED denotes edit distance, CDM evaluates formulas, and TEDS evaluates tables. Scale refers to the named release or split; K/M counts are rounded and test splits are marked. Access icons link to official repositories, dataset pages, or papers. Full list: project page .
Method
Date
IC15
Total
CTW
PSENet [ 116 ]
2018.06
85.7
80.9
82.2
TextSnake [ 203 ]
2018.07
82.6
78.4
75.6
PAN [ 204 ]
2019.08
82.9
85.0
83.7
DBNet [ 41 ]
2019.11
87.3
84.7
84.9
TextFuseNet [ 205 ]
2020.07
92.1
87.1
86.6
FCENet [ 206 ]
2021.04
86.2
85.8
85.5
TABLE III: Scene text detection Hmean/F1 (%).
Model
Date
Params
IIIT5K
SVT
IC13
IC15
SVTP
CUTE
CRNN [ 23 ]
2015.07
8.3 M
82.9
81.6
91.1
69.4
70.0
65.5
ASTER [ 42 ]
2018.06
27.2 M
93.3
90.0
90.8
74.7
80.2
80.9
ABINet [ 119 ]
2021.03
36.7 M
96.2
93.5
97.4
86.0
89.3
89.2
SVTR-B [ 120 ]
2022.05
24.6 M
96.0
91.5
97.1
85.2
89.9
91.7
PARSeq [ 105 ]
2022.07
23.8 M
97.0
93.6
97.0
86.5
88.9
92.2
SVTRv2-B [ 56 ]
2024.11
19.8 M
97.7
94.0
97.3
88.1
91.2
95.8
TABLE IV: Scene text recognition accuracy (%) under MJ+ST, following the comparison compiled in SVTRv2 [ 56 ] . IC13/IC15 use the 857/1811-image subsets.
IC15
Total-Text
CTW1500
Method
Date
None
None
Full
None
Full
FOTS [ 44 ]
2018
–
–
–
21.1
39.7
Mask TextSpotter v2 [ 209 ]
2019
–
65.3
77.4
–
–
TextDragon [ 210 ]
2019
–
48.8
74.8
39.7
72.4
ABCNet [ 121 ]
2020
–
64.2
75.7
45.2
74.1
ABCNet v2 [ 211 ]
2021
–
70.4
78.1
57.5
77.2
TABLE V: End-to-end scene text spotting F1 (%). Dates refer to the first public papers. “None” is lexicon-free; “Full” uses all words in the test set.
System
Date
Params
Overall
Text
Table
Fml.
RO
Specialized document VLMs
OvisOCR2 [ 58 ]
2026.07
0.8 B
96.47
97.35
94.58
97.49
0.112
Unlimited-OCR [ 135 ]
2026.06
3 B
94.00
96.06
90.21
95.72
0.128
Qianfan-OCR [ 59 ]
2026.03
4 B
93.90
96.00
90.53
95.08
0.130
FireRed-OCR [ 149 ]
2026.02
2 B
93.26
96.30
88.04
95.44
0.131
Hybrid workflows
TABLE VI: Selected OmniDocBench v1.6_full scores [ 155 ] as of October 2026. Overall/Text/Table/Fml. are % ( ↑ ); Text is (1−Edit)×100 , Table is TEDS, Fml. is CDM, and RO is reading order Edit ( ↓ ).
System
All
S
W
SP
I
SK
Worst
Modular toolkits and task-specific models
PP-StructureV3 [ 145 ]
64.45
84.68
59.34
66.89
73.38
37.98
37.98
Specialized document VLMs
OvisOCR2 [ 58 ]
92.29
93.77
91.40
93.09
92.88
90.33
90.33
Hybrid workflows
PaddleOCR-VL-1.6 [ 62 ]
93.19
94.74
92.48
92.78
93.28
92.66
92.48
TABLE VII: Selected Real5-OmniDocBench leaderboard scores [ 216 ] (%, ↑ ), as of October 2026, using the pinned OmniDocBench v1.5 protocol. S/W/SP/I/SK = scan/warp/screen/illum./skew; All averages the five condition scores; Worst is their minimum.