Unlimited OCR Works 2

Shu Wei; Jingjing Wu; Qunyi Xie; Lingshu Zhang; Xiangcheng Du; Yannian Fu; Mengjie Wang; Zuyi Wang

cs.CLOct 29, 2026

Unlimited OCR Works 2

Authors: Shu Wei, Jingjing Wu, Qunyi Xie, Lingshu Zhang, Xiangcheng Du, Yannian Fu, Mengjie Wang, Zuyi Wang,

Abstract

Unlimited OCR ushers the era of long-horizon parsing, substantially mitigating the growth of working memory as document lengthens. However, as document length scales to hundreds of pages, the visual context grows substantially, sharply increasing the computational cost of each decoding step and limiting the scalability of ultra-long document parsing. Inspired by human book transcription via localized focus without re-scanning preceding pages, we introduce Unlimited OCR 2, a document parsing model that looks only at what it transcribes. Building on Unlimited OCR, we upgrade all the decoder attention modules to Sparse Reference Sliding Window Attention (SR-SWA), bounding active reference tokens to a fixed capacity and thus sustaining constant computational costs across arbitrarily long documents. By combining SR-SWA with Unlimited-Horizon Decoding Paradigm, Unlimited OCR 2 enables document parsing over virtually unlimited lengths without prohibitive memory overhead. At the hundred-page scale, Unlimited OCR 2 achieves 7.0 $\times$ and 5.6 $\times$ per-step speedups over Full Attention and R-SWA, respectively. Furthermore, it delivers state-of-the-art end-to-end performance on both MPDocBench and MarathonDocBench, our newly proposed benchmark targeting hundred-page document parsing. Ultimately, the capacity for uninterrupted, one-shot document parsing signifies a fundamental paradigm shift into the OCR 3.0 era.

Figures & tables

Figure 1: R-SWA vs SR-SWA. R-SWA computes attention over all visual tokens across pages, inducing extreme visual redundancy; SR-SWA sparsely attends to relevant visual regions, aligning machine decoding with human reading intuition.

Figure 2: MarathonDocBench versus MPDocBench. MarathonDocBench extends document length from tens to hundreds of pages (a), with substantially richer cross-page structures as shown in (b) and (c).

Figure 3: Inspired by the process of humans copying books, we propose Unlimited OCR 2. SR-SWA combines Page Gate and Token Sieve to construct a fixed-length set of working reference visual tokens, substantially reducing computational and memory overhead while keeping both independent of document length throughout generation.

Figure 4: Comparison of (a) standard decoding and (b) our Unlimited-Horizon Decoding paradigm. Standard decoding retains visual KV caches for all pages, causing GPU memory usage to grow linearly with document length and eventually overflow. Our paradigm divides long documents into manageable chunks and processes them sequentially, reusing the KV cache of shared boundary pages and releasing inactive caches to keep GPU memory usage bounded while preserving cross-chunk continuity.

Figure 5: Performance comparison on multi-page benchmarks. Unlimited OCR 2 (Ours) consistently achieves state-of-the-art performance across all evaluation categories.

Method	Overall $\uparrow$	Text ${}^{\mathrm{Edit}}\downarrow$	T-Text ${}^{\mathrm{Edit}}\downarrow$	Formula ${}^{\mathrm{CDM}}\uparrow$	Table ${}^{\mathrm{TEDS}}\uparrow$	T-Table ${}^{\mathrm{TEDS}}\uparrow$	Figure ${}^{\mathrm{F1}}\uparrow$	ReadOrder ${}^{\mathrm{Edit}}\downarrow$	Heading ${}^{\mathrm{TEDS}}\uparrow$
General VLMs
Gemini-3.1-Pro Preview [ 9 ]	71.79	0.070	0.232	88.37	81.68	61.26	59.08	0.127	26.94
Qwen3.8-Max [ 25 ]	78.30	0.057	0.196	89.24	85.89	67.18	70.19	0.097	48.80
Kimi-K3 [ 24 ]	60.77	0.162	0.551	41.16	75.45	81.57	40.47	0.206	39.30
Qwen3.6-plus [ 25 ]	72.14	0.096	0.245	88.77	82.99	60.50	64.80	0.179	32.06
Qwen3-VL-235B [ 1 ]	73.98	0.087	0.187	84.71	81.35	61.25	63.25	0.136	42.34

Table 1: Performance comparison on MPDocBench. All models in the table are end-to-end VLM-based architectures.

Method	Overall $\uparrow$	Text ${}^{\mathrm{Edit}}\downarrow$	T-Text ${}^{\mathrm{Edit}}\downarrow$	Formula ${}^{\mathrm{CDM}}\uparrow$	Table ${}^{\mathrm{TEDS}}\uparrow$	T-Table ${}^{\mathrm{TEDS}}\uparrow$	Figure ${}^{\mathrm{F1}}\uparrow$	ReadOrder ${}^{\mathrm{Edit}}\downarrow$	Heading ${}^{\mathrm{TEDS}}\uparrow$
General VLMs
Gemini-3.1-Pro Preview [ 9 ]	26.97	0.516	0.764	30.44	28.48	21.71	18.94	0.654	9.68
Qwen3.8-Max [ 25 ]	22.93	0.600	0.804	18.82	15.20	13.71	33.39	0.733	16.09
Kimi-K3 [ 24 ]	33.74	0.475	0.664	40.50	27.80	26.30	21.38	0.611	28.90
Minimax-M3 [ 16 ]	17.97	0.659	0.818	14.20	17.79	14.84	3.67	0.763	17.16
Gemini-3.8-Flash [ 10 ]	28.61	0.519	0.658	26.35	23.40	19.56	18.62	0.687	27.34

Table 2: Performance comparison on MarathonDocBench. Unlimited OCR is not reported on MarathonDocBench due to its 40-page input limitation.

Model	Size	Overall $\uparrow$	Text ${}^{\text{Edit}}$ $\downarrow$	Formula ${}^{\text{CDM}}$ $\uparrow$	Table ${}^{\text{TEDs}}$ $\uparrow$	Table ${}^{\text{TEDS-S}}$ $\uparrow$	ReadOrder ${}^{\text{Edit}}$ $\downarrow$
HunyuanOCR [ 23 ]	1B	89.95	0.088	87.68	91.01	92.23	0.171
DeepSeek-OCR 2 [ 29 ]	3B-A0.5B	90.25	0.050	91.84	83.89	87.75	0.144
dots.ocr [ 19 ]	3B	90.77	0.048	89.95	87.18	90.58	0.138
FireRed-OCR [ 31 ]	2B	93.26	0.037	95.44	88.04	91.06	0.131
ABot-OCR [ 14 ]	2B	93.30	0.037	94.86	88.69	91.87	0.137
Logics-Parsing-v2 [ 4 ]	4B	93.33	0.041	95.65	88.42	91.98	0.137

Table 3: Comparison on OmniDocBench v1.6. All models in the table are end-to-end VLM-based architectures.

Method	Peak KV Memory ( $B=1$ )	TPS ( $B=16$ )
Standard Decoding	47.00 GB	360
Unlimited-Horizon Decoding	1.77 GB	1315

Table 4: Feasibility evaluation of the Unlimited-Horizon Decoding Paradigm. Sequential chunk processing reduces peak KV cache memory at

B=1

and improves end-to-end throughput at

B=16

, enabling hundred-page parsing under constrained GPU memory.

Figure 6: Performance and efficiency comparison across varying document lengths. Unlimited-OCR 2 demonstrates (a) robust parsing accuracy and (b) near-constant decoding latency (ms/step) as document length scaling.

Optical Character Recognition

Optical character recognition converts text in images into machine-readable text, from printed pages and handwriting to signs and symbols.

See all 201 papers

Unlimited OCR Works
Jun 22, 2026Youyang Yin, Huanhuan Liu, YY +14Efficient Transformer Inference Long-Context Language Model Inference
Baidu Inc.
OvisOCR2 Technical Report
Jul 15, 2026Shiyin Lu, Yinglun Li, Yu Xia +10Document Parsing Optical Character Recognition
ATH-MaaS, Alibaba Group
HSD: Training-Free Acceleration for Document Parsing Vision-Language Models with Hierarchical Speculative Decoding
Feb 13, 2026Wenhui Liao, Hongliang Li, Pengyu Xie +15Efficient VLM Inference Speculative Decoding
South China University of Technology, Guangzhou, China · Shanghai Artificial Intelligence Laboratory, Shanghai, China · Nanjing University, Nanjing, China +1

Jun 22, 2026J/K move · Enter open · S save

Unlimited OCR Works

Youyang Yin, Huanhuan Liu, YY, Qunyi Xie+13

Baidu Inc.

Efficient Transformer Inference Long-Context Language Model Inference Sliding-Window Attention Optical Character Recognition

Open paper AskPDF arXiv

Recently, end-to-end OCR models, exemplified by DeepSeek OCR, have once again thrust OCR into the spotlight. A widely held view is that employing a large language model (LLM) as the decoder allows the model to leverage the prior distribution of language, leading to improved OCR performance. However, the downside is equally evident: as the output sequence lengthens, the accumulated KV cache drives up memory consumption and progressively slows down generation. This stands in stark contrast to humans, who exhibit no such decline in efficiency during long-horizon copying tasks. In this technical report, we propose Unlimited OCR, a model designed to emulate human parsing working memory. Taking DeepSeek OCR as the baseline, we replace all attention layers in the decoder with our proposed Reference Sliding Window Attention (R-SWA), which reduces attention computation costs while maintaining a constant KV cache throughout the entire decoding process. By combining the high compression rate of DeepSeek OCR's encoder with our constant KV cache design, Unlimited OCR can transcribe dozens of pages of documents in a single forward pass under a standard maximum length of 32K. More importantly, R-SWA is a general-purpose parsing attention mechanism - beyond OCR, it is equally applicable to tasks such as ASR, translation, etc. Codes and model weights are publicly available at http://github.com/baidu/Unlimited-OCR.

Abstract

Optical Character Recognition

Explore similar work

Unlimited OCR Works

OvisOCR2 Technical Report

HSD: Training-Free Acceleration for Document Parsing Vision-Language Models with Hierarchical Speculative Decoding