MDPBench: A Benchmark for Multilingual Document Parsing in Real-World Scenarios

Zhang Li; Zhibo Lin; Qiang Liu; Ziyang Zhang; Shuo Zhang; Zidun Guo; Jiajun Song; Jiarui Zhang

cs.CVMar 30, 2026

MDPBench: A Benchmark for Multilingual Document Parsing in Real-World Scenarios

Authors: Zhang Li, Zhibo Lin, Qiang Liu, Ziyang Zhang, Shuo Zhang, Zidun Guo, Jiajun Song, Jiarui Zhang,

Organizations: Huazhong University of Science and Technology · Kingsoft Office

Abstract

We introduce Multilingual Document Parsing Benchmark, the first benchmark for multilingual digital and photographed document parsing. Document parsing has made remarkable strides, yet almost exclusively on clean, digital, well-formatted pages in a handful of dominant languages. No systematic benchmark exists to evaluate how models perform on digital and photographed documents across diverse scripts and low-resource languages. MDPBench comprises 3,400 document images spanning 17 languages, diverse scripts, and varied photographic conditions, with high-quality annotations produced through a rigorous pipeline of expert model labeling, manual correction, and human verification. To ensure fair comparison and prevent data leakage, we maintain separate public and private evaluation splits. Our comprehensive evaluation of both open-source and closed-source models uncovers a striking finding: while closed-source models (notably Gemini3-Pro) prove relatively robust, open-source alternatives suffer dramatic performance collapse, particularly on non-Latin scripts and real-world photographed documents, with an average drop of 17.8% on photographed documents and 14.0% on non-Latin scripts. These results reveal significant performance imbalances across languages and conditions, and point to concrete directions for building more inclusive, deployment-ready parsing systems. Source available at https://github.com/Yuliang-Liu/MultimodalOCR.

Figures & tables

Benchmark

Languages

Type

Image Count

Photograph Conditions

Num

Others

FoxPage [ 24 ]

✓

—

212

olmOCR-Bench [ 34 ]

✓

—

1402

OmniDocBench-v1.5 [ 31 ]

✓

—

1355

DocPTBench [ 11 ]

✓

—

DB / Photo.

2362

✓

Real5-OmniDocBench [ 55 ]

✓

—

DB / Photo.

6775

✓

Table 1: Comparison of document parsing benchmarks. DB : Digital-Born; Photo. : Photographed; SR : Screen Re-photograph; PD : Physical Deformation; ID : Image Degradation; CO : Camera Orientations; BV : Background Variation.

Figure 2: Visualization of digital-born and photographed document images.

Figure 3: Overall pipeline of the data annotation process.

Model	Overall			Latin										Non-Latin									Private
Model	All	Digit.	Photo.	Avg.	DE	EN	ES	FR	ID	IT	NL	PT	VI	Avg.	AR	HI	JP	KO	RU	TH	ZH	ZH-T	All
General VLMs
Gemini-3-pro-preview [ 14 ]	86.4	90.4	85.1	88.4	91.2	90.6	83.4	82.7	91.5	91.6	87.7	91.4	85.9	84.1	89.4	90.4	74.8	85.5	84.9	80.6	85.1	82.1	89.8
kimi-K2.5 [ 38 ]	77.5	85.0	75.0	81.6	85.9	86.2	72.7	71.0	80.6	86.6	77.4	87.6	86.2	72.9	75.8	74.5	72.5	70.9	61.8	67.0	81.7	78.6	81.2
Doubao-2.0-pro [ 4 ]	74.2	78.9	72.8	75.7	82.8	74.4	69.0	70.0	73.3	82.0	69.9	83.4	76.5	72.5	81.3	75.7	65.8	74.7	63.3	71.9	71.9	75.2	79.5
Claude-Sonnet-4.6 [ 1 ]	73.1	85.0	69.3	79.2	79.8	80.6	72.8	66.5	82.3	83.3	76.7	88.0	83.1	66.2	67.8	71.7	63.4	64.3	70.8	65.2	61.3	65.1	77.6

Table 2: Performance of general VLMs, specialized VLMs, and pipeline tools on MDPBench.

Figure 4: Visualization of document parsing results of Gemini-3-Pro on photographed documents.

Figure 5: Typical language-specific errors.

Figure 6: Existing document parsing models often exhibit hallucinations.

Figure 7: Existing document parsing models often fail to correctly handle right-to-left reading order in Arabic documents.

Model	Text ${}^{\mathrm{Edit}}\downarrow$																	Formula ${}^{\mathrm{CDM}}\uparrow$		Table ${}^{\mathrm{TEDS}}\uparrow$
Model	DE	EN	ES	FR	ID	IT	NL	PT	VI	AR	HI	JP	KO	RU	TH	ZH	ZH-T	Digit.	Photo.	Digit.	Photo.
Gemini-3-pro-preview [ 14 ]	0.026	0.041	0.082	0.026	0.025	0.021	0.016	0.061	0.050	0.040	0.017	0.234	0.043	0.031	0.042	0.104	0.418	93.4	90.5	75.9	69.2
Qwen3-VL-Instruct-8B [ 2 ]	0.126	0.135	0.124	0.023	0.066	0.018	0.016	0.043	0.046	0.412	0.065	0.124	0.059	0.039	0.226	0.104	0.342	92.9	89.3	65.0	56.2
dots.mocr [ 52 ]	0.003	0.022	0.107	0.027	0.021	0.011	0.005	0.024	0.039	0.059	0.024	0.071	0.031	0.030	0.598	0.084	0.804	89.8	78.7	59.6	55.9
PaddleOCR-VL-1.5 [ 8 ]	0.003	0.019	0.172	0.040	0.018	0.012	0.004	0.019	0.031	0.262	0.224	0.063	0.031	0.033	0.330	0.065	0.197	88.4	85.1	76.0	65.0
MonkeyOCR-pro-3B [ 21 ]	0.016	0.027	0.169	0.035	0.461	0.016	0.018	0.040	0.913	0.978	0.980	0.139	0.113	0.458	0.989	0.126	0.660	90.7	88.3	68.3	60.7

Table 3: Component-level evaluation of text, formula, and table recognition on MDPBench subsets.

Model	Layout Detection ${}^{\mathrm{PageIoU}}\uparrow$
Model	DE	EN	ES	FR	ID	IT	NL	PT	VI	AR	HI	JP	KO	RU	TH	ZH	ZH-T
dots.mocr [ 52 ]	93.1	86.6	88.3	85.9	94.4	92.3	93.6	92.2	92.6	95.4	92.7	91.6	91.6	92.9	90.2	85.7	89.9
PaddleOCR-VL [ 7 ]	93.9	86.2	88.8	94.4	85.9	88.1	86.9	84.2	81.9	84.6	87.2	85.8	86.5	88.6	80.2	86.0	84.7
PaddleOCR-VL-1.5 [ 8 ]	92.4	84.5	88.8	93.5	84.3	88.0	86.2	81.3	80.6	87.5	86.6	86.5	86.8	87.2	81.0	84.9	84.6
MinerU-2.5-VLM [ 29 ]	90.6	84.8	76.1	91.8	84.4	86.1	85.4	87.8	85.0	89.2	87.2	82.7	91.5	88.8	81.9	83.9	87.1
PP-StructureV3 [ 9 ]	65.5	63.7	58.8	70.9	61.0	62.1	62.0	59.0	56.0	59.3	66.5	59.7	62.5	65.8	55.7	59.8	58.7

Table 4: Component-level layout detection evaluation on the MDPBench layout subset.

Document Parsing Benchmarks

See all 15 papers

MORE: A Multilingual Document Parsing Benchmark and Evaluation
Jul 3, 2026Long Xu, Binghong Wu, Tinghao Yu +6Document Parsing Benchmarks Document Parsing
Tencent, Shenzhen, China
How Far Is Document Parsing from Solved? PureDocBench: A Source-Traceable Benchmark across Clean, Degraded, and Real-World Settings
May 8, 2026Zhiheng Li, Zongyang Ma, Jiaxian Chen +12Document Parsing Benchmarks Document Parsing
CASIA · UCAS · Beijing Key Laboratory of Super Intelligent Security of Multi-Modal Information +6
MPDocBench-Parse: Benchmarking Practical Multi-page Document Parsing
May 21, 2026Bangbang Zhou, Hangdi Xing, Yifan Chen +8Document Parsing Benchmarks Document Parsing
University of Science and Technology of China · Tongyi Lab, Alibaba Group

Jul 3, 2026J/K move · Enter open · S save

MORE: A Multilingual Document Parsing Benchmark and Evaluation

Long Xu, Binghong Wu, Tinghao Yu, Hao Feng+5

Tencent, Shenzhen, China

Document Parsing Benchmarks Document Parsing Multilingual VLM Evaluation

Open paper AskPDF arXiv

Multilingual documents encapsulate rich regional cultures, scientific discoveries, and historical records. Parsing this content into structured, machine-readable formats is critical for unlocking global knowledge. However, existing benchmarks predominantly focus on high-resource languages like English and Chinese, creating an evaluation blind spot concerning model performance on other languages. While recent Vision-Language Models (VLMs) claim support for hundreds of languages, the lack of ground truth makes it impossible to empirically verify these capabilities. To bridge this gap, we introduce MORE, a large-scale benchmark designed for multilingual document parsing evaluation. MORE distinguishes itself through three key dimensions: (1) Unprecedented Scale: It covers 149 languages, making it the most linguistically diverse benchmark to date; (2) Structural Complexity: Unlike previous works, it extends evaluation beyond plain text to include structural elements such as code blocks, tables, and catalogs; and (3) Data Authenticity: All samples are curated from real-world documents via a model-assisted, human-refined annotation pipeline. We evaluate state-of-the-art models using MORE, establishing new performance baselines for long-tail languages and validating the benchmark's effectiveness in diagnosing model capabilities in realistic, diverse scenarios. The MORE dataset will be available at https://github.com/zimoqingfeng/MORE.

Abstract

Document Parsing Benchmarks

Explore similar work

MORE: A Multilingual Document Parsing Benchmark and Evaluation

How Far Is Document Parsing from Solved? PureDocBench: A Source-Traceable Benchmark across Clean, Degraded, and Real-World Settings

MPDocBench-Parse: Benchmarking Practical Multi-page Document Parsing