Organizations: Institute of Information Engineering, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Tencent · Nanjing University of Science and Technology · Nankai University
Document parsing has recently advanced with multimodal large language models (MLLMs) that directly map document images to structured outputs. Traditional cascaded pipelines depend on precise layout analysis and often fail under casually captured or non-standard conditions. Although end-to-end approaches mitigate this dependency, they still exhibit repetitive, hallucinated, and structurally inconsistent predictions - primarily due to the scarcity of large-scale, high-quality full-page (document-level) end-to-end parsing data and the lack of structure-aware training strategies. To address these challenges, we propose a data-training co-design framework for robust end-to-end document parsing. A Realistic Scene Synthesis strategy constructs large-scale, structurally diverse full-page end-to-end supervision by composing layout templates with rich document elements, while a Document-Aware Training Recipe introduces progressive learning and structure-token optimization to enhance structural fidelity and decoding stability. We further build Wild-OmniDocBench, a benchmark derived from real-world captured documents for robustness evaluation. Integrated into a 1B-parameter MLLM, our method achieves superior accuracy and robustness across both scanned/digital and real-world captured scenarios. All models, data synthesis pipelines, and benchmarks will be publicly released to advance future research in document understanding.
Figures & tables
Figure 1: Overall Performance and Degradation from OmniDocBench to Wild-OmniDocBench. Underlined method names correspond to modular cascaded pipelines.
Figure 2: Scanned/Digital and Real-World Capture. On scanned/digital pages, both modular and E2E parsers decode correctly. Under real-world capture, modular cascades accumulate layout-analysis errors that propagate to element parsing (extra/missing regions), while generic end-to-end models exhibit repetitive outputs.
Figure 3: Overview of Realistic Scene Synthesis. Left: repositories of atomic elements and layout templates with reading order. Right: a synthesis pipeline that composes sampled elements into templates under spatial/structural constraints to produce page-level annotations, followed by capture-aware augmentation to simulate real-world images.
Figure 4: Wild-OmniDocBench Construction. We convert scanned pages into real-world–captured images by (i) printing, deforming, and photographing under varied lighting, and (ii) displaying on screens and re-shooting to induce moiré and reflections.
Model Type
Method
Size
Metrics
Overall ↑
Text Edit ↓
Formula CDM ↑
Table TEDS ↑
Reading Order Edit ↓
Pipeline Tools
Marker-1.8.2 [ 9 ]
-
71.30
0.206
76.66
57.88
0.250
Mineru2-pipeline [ 45 ]
-
75.51
0.209
76.55
70.90
0.225
PP-StructureV3 [ 8 ]
-
86.73
0.073
85.79
81.68
0.073
General MLLMs
GPT-4o [ 1 ]
-
75.02
0.217
79.70
67.07
0.148
InternVL3 [ 61 ]
78B
80.33
0.131
83.42
70.64
0.113
Table 1: Comparison of various OCR and VLM systems on document understanding benchmarks. Higher ↑ indicates better performance, lower ↓ indicates smaller error.
Method
de
it
ja
es
pt
fr
Mathpix [ 30 ]
83.90
79.00
83.95
81.23
76.8
73.29
GOT-OCR [ 48 ]
83.45
47.84
50.81
64.60
69.12
39.09
MistralOCR
78.06
63.21
63.61
60.09
58.89
59.83
Dots.ocr
53.94
70.15
70.73
66.87
60.75
59.91
GPT-4o
78.94
76.83
83.83
80.37
77.25
76.08
Qwen3-VL-235B
74.45
78.36
85.30
76.36
76.73
69.73
Table 2: Performance comparison on the XFUND.
Model Type
Model
Overall
Text 1-Edit↑
Formula CDM ↑
Table TEDS ↑
Origin
Wild
Origin
Wild
Origin
Wild
Origin
Wild
General
Qwen3-VL-235B
89.15
79.69 (-9.46)
93.1
90.1 (-3.0)
88.14
80.67 (-7.47)
86.21
68.31 (-17.90)
Modular
MonkeyOCR-pro-3B
88.85
70.00 (-18.85)
92.5
78.9 (-13.6)
87.25
63.27 (-23.98)
86.78
67.83 (-18.95)
MinerU2.5
90.67
70.91 (-19.76)
95.3
78.2 (-17.1)
88.46
64.37 (-24.09)
88.22
70.15 (-18.07)
PPOCR-VL
91.93
72.19 (-19.74)
96.1
76.8 (-19.3)
88.67
65.54 (-23.13)
91.01
74.24 (-16.77)
End2End
DeepSeek-OCR
87.01
74.23 (-12.78)
92.7
82.2 (-10.5)
83.37
70.07 (-13.30)
84.97
70.41 (-14.56)
Table 3: Performance comparison on the Wild-OmniDocBench.
#
RSS
DATR
OmniDocBench
Wild-OmniDocBench
ST
PTP
Overall ↑
Repeat ↓
Overall ↑
Repeat ↓
1
×
✓
✓
89.96
4.7
78.82
8.6
2
✓
×
✓
88.74
4.6
84.90
5.4
3
✓
✓
×
91.24
4.2
85.39
4.9
4
✓
✓
✓
93.75
2.1
87.03
4.3
Table 4: Extended ablation study on Realistic Scene Synthesis. and Document-Aware Training Recipe.
#
Data setting
OmniDocBench
Wild-OmniDocBench
Overall ↑
Repeat ↓
Overall ↑
Repeat ↓
1
Manual-100k
89.26
5.4
80.20
7.8
2
DocMix-1M
85.41
7.9
76.66
8.9
3
DocMix-2M
88.14
5.6
80.08
6.2
4
DocMix-3M
89.96
3.8
83.21
4.8
5
DocMix-4M
89.31
3.7
83.52
4.9
Table 5: Extended ablation study on Realistic Scene Synthesis. and Document-Aware Training Recipe.