Unlimited OCR ushers the era of long-horizon parsing, substantially mitigating the growth of working memory as document lengthens. However, as document length scales to hundreds of pages, the visual context grows substantially, sharply increasing the computational cost of each decoding step and limiting the scalability of ultra-long document parsing. Inspired by human book transcription via localized focus without re-scanning preceding pages, we introduce Unlimited OCR 2, a document parsing model that looks only at what it transcribes. Building on Unlimited OCR, we upgrade all the decoder attention modules to Sparse Reference Sliding Window Attention (SR-SWA), bounding active reference tokens to a fixed capacity and thus sustaining constant computational costs across arbitrarily long documents. By combining SR-SWA with Unlimited-Horizon Decoding Paradigm, Unlimited OCR 2 enables document parsing over virtually unlimited lengths without prohibitive memory overhead. At the hundred-page scale, Unlimited OCR 2 achieves 7.0× and 5.6× per-step speedups over Full Attention and R-SWA, respectively. Furthermore, it delivers state-of-the-art end-to-end performance on both MPDocBench and MarathonDocBench, our newly proposed benchmark targeting hundred-page document parsing. Ultimately, the capacity for uninterrupted, one-shot document parsing signifies a fundamental paradigm shift into the OCR 3.0 era.
Figures & tables
Figure 1: R-SWA vs SR-SWA. R-SWA computes attention over all visual tokens across pages, inducing extreme visual redundancy; SR-SWA sparsely attends to relevant visual regions, aligning machine decoding with human reading intuition.
Figure 2: MarathonDocBench versus MPDocBench. MarathonDocBench extends document length from tens to hundreds of pages (a), with substantially richer cross-page structures as shown in (b) and (c).
Figure 3: Inspired by the process of humans copying books, we propose Unlimited OCR 2. SR-SWA combines Page Gate and Token Sieve to construct a fixed-length set of working reference visual tokens, substantially reducing computational and memory overhead while keeping both independent of document length throughout generation.
Figure 4: Comparison of (a) standard decoding and (b) our Unlimited-Horizon Decoding paradigm. Standard decoding retains visual KV caches for all pages, causing GPU memory usage to grow linearly with document length and eventually overflow. Our paradigm divides long documents into manageable chunks and processes them sequentially, reusing the KV cache of shared boundary pages and releasing inactive caches to keep GPU memory usage bounded while preserving cross-chunk continuity.
Figure 5: Performance comparison on multi-page benchmarks. Unlimited OCR 2 (Ours) consistently achieves state-of-the-art performance across all evaluation categories.
Method
Overall ↑
Text Edit↓
T-Text Edit↓
Formula CDM↑
Table TEDS↑
T-Table TEDS↑
Figure F1↑
ReadOrder Edit↓
Heading TEDS↑
General VLMs
Gemini-3.1-Pro Preview [ 9 ]
71.79
0.070
0.232
88.37
81.68
61.26
59.08
0.127
26.94
Qwen3.8-Max [ 25 ]
78.30
0.057
0.196
89.24
85.89
67.18
70.19
0.097
48.80
Kimi-K3 [ 24 ]
60.77
0.162
0.551
41.16
75.45
81.57
40.47
0.206
39.30
Qwen3.6-plus [ 25 ]
72.14
0.096
0.245
88.77
82.99
60.50
64.80
0.179
32.06
Qwen3-VL-235B [ 1 ]
73.98
0.087
0.187
84.71
81.35
61.25
63.25
0.136
42.34
Table 1: Performance comparison on MPDocBench. All models in the table are end-to-end VLM-based architectures.
Method
Overall ↑
Text Edit↓
T-Text Edit↓
Formula CDM↑
Table TEDS↑
T-Table TEDS↑
Figure F1↑
ReadOrder Edit↓
Heading TEDS↑
General VLMs
Gemini-3.1-Pro Preview [ 9 ]
26.97
0.516
0.764
30.44
28.48
21.71
18.94
0.654
9.68
Qwen3.8-Max [ 25 ]
22.93
0.600
0.804
18.82
15.20
13.71
33.39
0.733
16.09
Kimi-K3 [ 24 ]
33.74
0.475
0.664
40.50
27.80
26.30
21.38
0.611
28.90
Minimax-M3 [ 16 ]
17.97
0.659
0.818
14.20
17.79
14.84
3.67
0.763
17.16
Gemini-3.8-Flash [ 10 ]
28.61
0.519
0.658
26.35
23.40
19.56
18.62
0.687
27.34
Table 2: Performance comparison on MarathonDocBench. Unlimited OCR is not reported on MarathonDocBench due to its 40-page input limitation.
Model
Size
Overall ↑
Text Edit↓
Formula CDM↑
Table TEDs↑
Table TEDS-S↑
ReadOrder Edit↓
HunyuanOCR [ 23 ]
1B
89.95
0.088
87.68
91.01
92.23
0.171
DeepSeek-OCR 2 [ 29 ]
3B-A0.5B
90.25
0.050
91.84
83.89
87.75
0.144
dots.ocr [ 19 ]
3B
90.77
0.048
89.95
87.18
90.58
0.138
FireRed-OCR [ 31 ]
2B
93.26
0.037
95.44
88.04
91.06
0.131
ABot-OCR [ 14 ]
2B
93.30
0.037
94.86
88.69
91.87
0.137
Logics-Parsing-v2 [ 4 ]
4B
93.33
0.041
95.65
88.42
91.98
0.137
Table 3: Comparison on OmniDocBench v1.6. All models in the table are end-to-end VLM-based architectures.
Method
Peak KV Memory ( B=1 )
TPS ( B=16 )
Standard Decoding
47.00 GB
360
Unlimited-Horizon Decoding
1.77 GB
1315
Table 4: Feasibility evaluation of the Unlimited-Horizon Decoding Paradigm. Sequential chunk processing reduces peak KV cache memory at B=1 and improves end-to-end throughput at B=16 , enabling hundred-page parsing under constrained GPU memory.
Figure 6: Performance and efficiency comparison across varying document lengths. Unlimited-OCR 2 demonstrates (a) robust parsing accuracy and (b) near-constant decoding latency (ms/step) as document length scaling.
Recently, end-to-end OCR models, exemplified by DeepSeek OCR, have once again thrust OCR into the spotlight. A widely held view is that employing a large language model (LLM) as the decoder allows the model to leverage the prior distribution of language, leading to improved OCR performance. However, the downside is equally evident: as the output sequence lengthens, the accumulated KV cache drives up memory consumption and progressively slows down generation. This stands in stark contrast to humans, who exhibit no such decline in efficiency during long-horizon copying tasks. In this technical report, we propose Unlimited OCR, a model designed to emulate human parsing working memory. Taking DeepSeek OCR as the baseline, we replace all attention layers in the decoder with our proposed Reference Sliding Window Attention (R-SWA), which reduces attention computation costs while maintaining a constant KV cache throughout the entire decoding process. By combining the high compression rate of DeepSeek OCR's encoder with our constant KV cache design, Unlimited OCR can transcribe dozens of pages of documents in a single forward pass under a standard maximum length of 32K. More importantly, R-SWA is a general-purpose parsing attention mechanism - beyond OCR, it is equally applicable to tasks such as ASR, translation, etc. Codes and model weights are publicly available at http://github.com/baidu/Unlimited-OCR.
We introduce OvisOCR2, a 0.8B document parsing model. OvisOCR2 is designed as an end-to-end parser: given a document page image, it generates a Markdown representation in natural reading order, covering text, formulas, tables, and visual regions. We build a data engine that combines filtered real-document annotations with synthetic pages whose rendered images and Markdown targets are derived from the same HTML source. The training recipe includes supervised fine-tuning, reinforcement learning on a 4B branch with a multi-component reward design, on-policy distillation into the 0.8B model, and model fusion. On OmniDocBench v1.6, OvisOCR2 achieves a state-of-the-art overall score of 96.58, placing an end-to-end model at the top of this leaderboard previously dominated by pipeline methods and highlighting the potential of end-to-end document parsing. On PureDocBench, OvisOCR2 also achieves the highest Avg3 score of 75.06. Beyond these two public benchmarks, we evaluate OvisOCR2 on an in-house benchmark designed to cover a broader set of long-tail and challenging scenarios. OvisOCR2 obtains the best overall performance among the compared methods, providing further evidence of its generalization and robustness. OvisOCR2 is available at https://huggingface.co/ATH-MaaS/OvisOCR2.
Document parsing is a fundamental task in multimodal understanding, supporting a wide range of downstream applications such as information extraction and intelligent document analysis. Benefiting from strong semantic modeling and robust generalization, VLM-based end-to-end approaches have emerged as the mainstream paradigm in recent years. However, these models often suffer from substantial inference latency, as they must autoregressively generate long, full-page sequences when processing long-form documents. While recent hybrid methods mitigate this issue via region-level parallel decoding with VLMs, independent region decoding loses full-page context and might weaken global coherence. To address this issue, we propose Hierarchical Speculative Decoding (HSD), a two-stage local-to-global framework for document parsing. HSD first employs a lightweight pipeline drafter to predict region partitions and generate coarse drafts for each region. The first stage verifies the generated region-level drafts in parallel for efficiency, while the second stage further performs page-level verification on these refined outputs to preserve full-page coherence. Experimental results show that HSD achieves a near-lossless 2.7x speedup with HunyuanOCR on OmniDocBench v1.5 and up to 7.04x speedup on long-document parsing tasks, demonstrating the effectiveness of the proposed method. The code is available at https://github.com/whlscut/HSD.
Wenhui Liao, Hongliang Li, Pengyu Xie +15
South China University of Technology, Guangzhou, China · Shanghai Artificial Intelligence Laboratory, Shanghai, China · Nanjing University, Nanjing, China +1