Document parsing, as a fundamental yet crucial vision task, is being revolutionized by vision-language models (VLMs). However, the autoregressive (AR) decoding inherent to VLMs creates a significant bottleneck, severely limiting parsing speed. In this paper, we propose Parallel-Token Prediction (PTP), a plugable, model-agnostic and simple-yet-effective method that enables VLMs to generate multiple future tokens in parallel with improved sample efficiency. Specifically, we insert some learnable tokens into the input sequence and design corresponding training objectives to equip the model with parallel decoding capabilities for document parsing. Furthermore, to support effective training, we develop a comprehensive data generation pipeline that efficiently produces large-scale, high-quality document parsing training data for VLMs. Extensive experiments on OmniDocBench and olmOCR-bench demonstrate that our method not only significantly improves decoding speed (1.6x-2.2x) but also reduces model hallucinations and exhibits strong generalization abilities.
Figures & tables
Figure 1: An illustration of the data creation pipeline.
Figure 2: Overview of our method. The upper illustrates the training architecture of our parallel-token prediction method, along with a visualization of the attention mask used during training. The lower depicts the inference process, where accelerated inference can be achieved by incorporating register tokens in each decoding step.
Model Type
Models
Slides
Academic Papers
Book
Textbook
Exam Papers
Magazine
News
Notes
Financial Report
Overall( ↓ )
Pipeline Tools
Marker-1.8.2 [ 28 ]
0.1796
0.0412
0.1010
0.2908
0.2958
0.1111
0.2717
0.4656
0.0341
0.1990
MinerU2-pipeline [ 41 ]
0.4244
0.0230
0.2628
0.1224
0.0822
0.3950
0.0736
0.2603
0.0411
0.1872
PP-StructureV3 [ 8 ]
0.0794
0.0236
0.0415
0.1107
0.0945
0.0722
0.0617
0.1236
0.0181
0.0695
General VLMs
GPT-4o [ 1 ]
0.1019
0.1203
0.1288
0.1599
0.1939
0.1420
0.6254
0.2611
0.3343
0.2297
Gemini-2.5 Pro [ 7 ]
0.0326
0.0182
0.0694
0.1618
0.0937
0.0161
0.1347
0.1169
0.0169
0.0734
InternVL3-76B [ 50 ]
0.0349
0.1052
0.0629
0.0827
0.1007
0.0406
0.5826
0.0924
0.0665
0.1298
Table 1: Document Parsing Performance in Text Edit Distance on OmniDocBench: evaluation using edit distance across 9 PDF page types.
Model
Overall( ↑ )
AR
OSM
TA
OS
HF
MC
LTT
Base
MinerU2-pipeline [ 41 ]
55.6
61.8
13.5
60.9
17.3
96.6
59.0
39.1
96.6
GPT-4o [ 1 ]
63.2
44.1
37.6
69.1
40.9
94.2
68.9
54.1
96.7
Qwen2.5-VL-72B [ 3 ]
64.8
72.2
51.1
67.3
38.6
73.6
68.3
49.1
98.3
MonkeyOCR-pro-3B [ 19 ]
68.8
67.7
28.4
74.6
36.1
91.2
76.6
80.1
95.3
olmOCR [ 29 ]
71.8
63.9
41.0
72.9
43.9
95.1
77.3
81.2
98.9
dots.ocr [ 32 ]
73.6
66.3
35.8
88.3
40.9
94.1
82.4
81.2
99.5
Table 2: Evaluation results on olmOCR-bench grouped by document types, including arXiv Math(AR), Old Scans Math (OSM), Tables (TA), Old Scans (OS), Headers Footers (HF), Multi Column (MC) and Long Tiny Text (LTT). Some results are sourced from the official reports of olmOCR-bench [ 29 ] and dots.ocr [ 32 ] . The Overall Score (Overall) represents the average across all document types.
Model
CDM( ↑ )
BLUE( ↑ )
Norm Edit( ↓ )
Mathpix [ 25 ]
86.60
66.56
0.322
Pix2Tex
73.90
46.00
0.337
UniMERNet-B [ 39 ]
85.00
60.84
0.238
GPT4o [ 1 ]
86.80
45.17
0.282
Qwen2.5-VL-3B [ 3 ]
24.11
53.59
0.331
Qwen2.5-VL-3B-NTP
71.65
63.05
0.226
Table 3: Formula recognition results on OmniDocBench. We report results in terms of both CDM [ 39 ] , BLEU and Edit Distance.
Figure 3: Performance comparison between NTP and PTP, including average TPOT, ITL, and latency under different QPS levels, as well as decoding speed and speedup ratio in synchronous mode, which are measured on OmniDocBench 16,886 images using an H20 GPU.
Figure 4: Left : Training trajectories of NTP, PTP, and MTP; Second : Performance of different methods on normal vs. hallucination-prone data; Third : PTP performance across different model architectures; Right : Register token extrapolation results.
Model
Accuracy
Accept Rate (%)
Qwen2.5-VL-3B-NTP
92.21
N/A
Qwen2.5-VL-3B-PTP-1
91.72
100
w/ speculative decoding
92.21
82
Table 4: Results on ScienceQA dataset.
Model
Formula
Text Edit (↓)
CDM(↑)
BLEU(↑)
Edit(↓)
Qwen2.5-VL-3B
24.11
53.59
0.331
0.208
Qwen2.5-VL-3B-PTP-1
89.63
62.32
0.236
0.043
w/ distinct reg.
89.24
61.78
0.247
0.052
w/ interleaved reg.
88.46
61.45
0.244
0.128
w/o KV Cache
39.43
35.96
0.476
0.070
Table 5: Ablation study on text and formula recognition in OmniDocBench.
Figure 5: Speculative Decoding
Dataset
Task
NTP
MTP
PTP-1
OmnidocBench
OCR
0.0431
83%
95%
ScienceQA
VLU
92.2
76%
82%
GSM8K
Math
73.3
70%
88%
Table 6: The performance of NTP, and acceptance ratios of MTP / PTP on various tasks.
Figure 6: Sampled rendered images and ground truth texts from normal and hallucinated datasets.
Figure 7: Analysis of the umber of register tokens.
Figure 8: The rendered markdown output for various types of documents of olmOCR-bench.
South China University of Technology, Guangzhou, China · Shanghai Artificial Intelligence Laboratory, Shanghai, China · Nanjing University, Nanjing, China +1