SEA-Vision: A Multilingual Benchmark for Comprehensive Document and Scene Text Understanding in Southeast Asia

Pengfei Yue; Xingran Zhao; Juntao Chen; Peng Hou; Wang Longchao; Jianghang Lin; Shengchuan Zhang; Anxiang Zeng

cs.CLMar 16, 2026

SEA-Vision: A Multilingual Benchmark for Comprehensive Document and Scene Text Understanding in Southeast Asia

Authors: Pengfei Yue, Xingran Zhao, Juntao Chen, Peng Hou, Wang Longchao, Jianghang Lin, Shengchuan Zhang, Anxiang Zeng, +1 more

Organizations: Xiamen University, China · Shopee, China · Tongji University, China

Abstract

Multilingual document and scene text understanding plays an important role in applications such as search, finance, and public services. However, most existing benchmarks focus on high-resource languages and fail to evaluate models in realistic multilingual environments. In Southeast Asia, the diversity of languages, complex writing systems, and highly varied document types make this challenge even greater. We introduce SEA-Vision, a benchmark that jointly evaluates Document Parsing and Text-Centric Visual Question Answering (TEC-VQA) across 11 Southeast Asian languages. SEA-Vision contains 15,234 document parsing pages from nine representative document types, annotated with hierarchical page-, block-, and line-level labels. It also provides 7,496 TEC-VQA question-answer pairs that probe text recognition, numerical calculation, comparative analysis, logical reasoning, and spatial understanding. To make such multilingual, multi-task annotation feasible, we design a hybrid pipeline for Document Parsing and TEC-VQA. It combines automated filtering and scoring with MLLM-assisted labeling and lightweight native-speaker verification, greatly reducing manual labeling while maintaining high quality. We evaluate several leading multimodal models and observe pronounced performance degradation on low-resource Southeast Asian languages, highlighting substantial remaining gaps in multilingual document and scene text understanding. We believe SEA-Vision will help drive global progress in document and scene text understanding.

Figures & tables

Figure 1: Performance of representative models on SEA-Vision. (a) End-to-end text recognition performance for document parsing across 11 languages. (b) TEC-VQA accuracy by language and model, along with overall averages (Avg.).

Figure 2: SEA-Vision benchmark overview. Geographical language coverage and dataset scale (left), Document Parsing types and sample page attributes (middle), and TEC-VQA examples across consumer places and public spaces (right).

Benchmark	Langs	Low-Resource	DP Images	TEC-VQA QA Pairs	Unified Eval (DP+VQA)
ICDAR2013/2015 [ 22 , 21 ]	1	$\times$	$\times$	$\times$	$\times$
DocVQA [ 34 ]	1	$\times$	$\times$	50,000	$\times$
TextVQA [ 42 ]	1	$\times$	$\times$	45,336	$\times$
MLT-2019 [ 35 ]	10	$\checkmark$ (2)	$\times$	$\times$	$\times$
M3Exam [ 60 ]	9	$\checkmark$ (6)	$\times$	12,317	$\times$
MTVQA [ 44 ]	9	$\checkmark$ (2)	$\times$	6,778	$\times$

Table 1: Comparison of existing text-related benchmarks. “Low-Resource” denotes the number of low-resource languages included. “Unified Eval” indicates whether the benchmark jointly evaluates Document Parsing (DP, Document Parsing) and Text-Centric Visual Question Answering (TEC-VQA).

Figure 3: Overview of the data annotation pipelines. (a) Document Parsing Annotation Pipeline : Internet-sourced document pages are first collected using domain-specific keywords and filtered for quality. Metadata annotation includes layout detection and MLLM–based analysis for language and page type identification. Candidate pages are ranked by a rule-based scoring function considering block count, type diversity, text area ratio, and presence of figures or tables. Selected samples undergo region-level correction via specialized models for text, formulas, and tables, followed by final human verification. (b) TEC-VQA Annotation Pipeline : Scene images from diverse environments (e.g., public spaces, consumer places, documents) are gathered and filtered. Layout and text are detected and re-rendered with multilingual content. An MLLM first generates English QA pairs; the English questions are then translated into Chinese to obtain Chinese QA, which is aligned with the English QA for consistency. The resulting bilingual QA pairs are translated into the image language and manually verified.

Method Type	Methods	EN	FIL	ID	KM	LO	MS	MY	PT	TH	VI	ZH	Avg.
Pipeline Models	MinerU2.5 [ 36 ]	0.187	0.106	0.194	0.693	0.730	0.213	0.520	0.209	0.592	0.278	0.218	0.358
	Dolphin-1.5 [ 11 ]	0.156	0.089	0.166	0.689	0.736	0.165	0.479	0.150	0.549	0.167	0.204	0.323
	MonkeyOCR-pro-1.2B [ 26 ]	0.128	0.094	0.212	0.668	0.697	0.209	0.468	0.207	0.574	0.457	0.162	0.352
	MonkeyOCR-pro-3B [ 26 ]	0.135	0.092	0.171	0.659	0.688	0.159	0.488	0.157	0.578	0.370	0.151	0.331
	PaddleOCR-VL [ 8 ]	0.108	0.065	0.099	0.634	0.648	0.087	0.456	0.107	0.133	0.148	0.131	0.238
Expert Models	POINTS-Reader [ 29 ]	0.121	0.046	0.092	0.845	0.815	0.075	0.755	0.115	0.709	0.184	0.170	0.357

Table 2: End-to-end document parsing performance on SEA-Vision, reported as Normalized Edit Distance (NED↓) across 11 languages. Best scores per language are bolded .

Method Type	Methods	Academic Paper	Book	Exam Paper	Magazine	Newspaper	Note	Research Report	Slide	Textbook	Avg.
Pipeline Models	MinerU2.5 [ 36 ]	0.301	0.348	0.369	0.301	0.446	0.289	0.335	0.303	0.360	0.339
	Dolphin-1.5 [ 11 ]	0.235	0.304	0.377	0.221	0.158	0.319	0.281	0.297	0.285	0.275
	MonkeyOCR-pro-1.2B [ 26 ]	0.272	0.348	0.391	0.340	0.291	0.353	0.340	0.328	0.349	0.335
	MonkeyOCR-pro-3B [ 26 ]	0.245	0.325	0.371	0.478	0.232	0.361	0.314	0.315	0.326	0.330
	PaddleOCR-VL [ 8 ]	0.156	0.205	0.265	0.148	0.119	0.255	0.214	0.204	0.217	0.198
Expert Models	POINTS-Reader [ 29 ]	0.240	0.329	0.328	0.208	0.318	0.374	0.318	0.289	0.317	0.302

Table 3: End-to-end document parsing performance on SEA-Vision, reported as Normalized Edit Distance (NED↓) across 9 page types. Best scores per page type are bolded .

	EN	ZH	ID	PT	MS	VI	TH	FIL	LO	KM	MY	Avg.
Closed-Source
GPT4o [ 19 ]	60.97	42.80	49.02	31.43	30.09	25.76	28.57	21.84	19.68	16.92	14.25	35.49
Gemini2.5-Pro [ 7 ]	69.14	56.40	47.98	33.95	29.44	26.46	29.89	20.10	27.49	28.92	25.70	39.49
Open-source
Qwen2.5-VL-72B [ 4 ]	71.09	57.80	49.28	35.29	30.41	26.23	24.06	20.35	18.06	12.00	8.66	37.43
Qwen3-VL-32B [ 45 ]	71.69	64.40	49.67	37.82	32.58	28.81	28.48	24.07	22.64	15.69	11.73	40.14

Table 4: Performance of the leading closed- and open-source MLLMs on the TEC-VQA. The best results of each language are bolded .

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Language	#Pages	Avg. blocks/page	Avg. text-area ratio	Pages with tables (%)	Pages with formulas (%)
EN	1585	25.71	63.08	15.65	6.56
ZH	636	28.11	62.59	23.27	16.50
VI	1678	19.69	60.13	11.50	5.78
TH	1506	10.53	54.38	6.57	1.26
FIL	1265	12.35	49.40	8.06	0.70
MS	1580	14.12	49.50	9.68	0.89

Appendix

Table A1 : Per-language statistics for the Document Parsing subset of SEA-Vision. We report the number of pages, average number of blocks per page, average text-area ratio, and the proportions of pages with tables and formulas.

Page Type	EN	ZH	VI	TH	FIL	MS	ID	LO	KM	MY	PT
Academic Literature	164	178	185	179	161	168	140	61	192	199	176
Book	187	64	186	193	187	187	187	200	199	171	193
Textbook	183	47	184	179	177	187	177	135	188	192	182
Exam Paper	133	9	184	157	141	147	146	129	148	166	180
Magazine	189	56	200	181	65	196	194	4	20	92	196
Newspaper	175	124	185	49	55	187	186	3	6	188	171

Appendix

Table A2 : Page-type statistics for the Document Parsing subset. Each entry denotes the number of pages for a given page type and language.

Language	#Images	#QA pairs	Avg Q length	Avg A length
EN	267	1335	72.74	31.00
ZH	100	500	21.72	12.81
VI	225	854	68.35	47.19
TH	272	1064	57.01	35.31
FIL	115	403	82.61	86.71
MS	242	924	73.54	38.29

Appendix

Table A3 : Per-language statistics for the TEC-VQA subset, including the number of images, the number of QA pairs, and the average question and answer lengths.

Capability category	#QA pairs	Proportion (%)
Text recognition	5573	74.35
Numerical calculation	5065	67.57
Comparative analysis	4306	57.44
Logical reasoning	459	6.12
Spatial understanding	209	2.79

Appendix

Table A4 : Distribution of TEC-VQA capability categories in SEA-Vision, reported as the number and proportion of QA pairs attributed to each capability.

Figure A1 : Heatmap of the TEC-VQA capability category co-occurrence matrix. Color intensity indicates the frequency of QA pairs annotated with each capability combination.

Method Type	Methods	EN	FIL	ID	KM	LO	MS	MY	PT	TH	VI	ZH	Avg.
Pipeline Models	MinerU2.5	0.234	0.186	0.248	0.969	0.975	0.223	0.858	0.255	0.972	0.469	0.307	0.503
	Dolphin-1.5	0.061	0.065	0.063	0.962	0.932	0.068	0.688	0.039	0.701	0.108	0.097	0.321
	MonkeyOCR-pro-1.2B	0.059	0.07	0.241	0.985	0.985	0.248	0.806	0.24	0.987	0.878	0.091	0.51
	MonkeyOCR-pro-3B	0.054	0.057	0.121	0.971	0.974	0.097	0.756	0.059	0.972	0.721	0.087	0.435
	PaddleOCR-VL	0.044	0.046	0.036	0.966	0.964	0.041	0.787	0.039	0.081	0.172	0.083	0.268
Expert Models	POINTS-Reader	0.05	0.022	0.04	0.975	0.981	0.032	0.909	0.051	0.857	0.202	0.117	0.363

Appendix

Table A5 : Pure-text region performance on SEA-Vision, reported as Normalized Edit Distance (NED↓) across 11 languages.

Method Type	Methods	EN		FIL		ID		KM		LO		MS		MY		PT		TH		VI		ZH		Avg.
Method Type	Methods	NED	TEDS	NED	TEDS	NED	TEDS	NED	TEDS	NED	TEDS	NED	TEDS	NED	TEDS	NED	TEDS	NED	TEDS	NED	TEDS	NED	TEDS	NED	TEDS
Pipeline Models	MinerU2.5	0.071	0.913	0.061	0.918	0.051	0.932	0.54	0.394	0.648	0.366	0.079	0.908	0.308	0.666	0.059	0.922	0.416	0.515	0.109	0.872	0.055	0.929	0.128	0.851
	Dolphin-1.5	0.13	0.816	0.139	0.798	0.141	0.803	0.767	0.196	0.866	0.12	0.156	0.796	0.476	0.467	0.127	0.82	0.656	0.305	0.143	0.821	0.241	0.71	0.23	0.72
	MonkeyOCR-pro-1.2B	0.083	0.896	0.119	0.863	0.106	0.874	0.71	0.254	0.764	0.281	0.14	0.836	0.468	0.524	0.13	0.856	0.511	0.52	0.188	0.808	0.128	0.835	0.196	0.789
	MonkeyOCR-pro-3B	0.095	0.895	0.132	0.855	0.094	0.893	0.676	0.272	0.763	0.237	0.133	0.842	0.431	0.553	0.121	0.851	0.541	0.432	0.18	0.806	0.093	0.886	0.189	0.792
	PaddleOCR-VL	0.059	0.924	0.076	0.911	0.056	0.922	0.69	0.232	0.671	0.321	0.083	0.899	0.413	0.504	0.064	0.914	0.132	0.83	0.059	0.927	0.059	0.924	0.116	0.86

Appendix

Table A6 : Table-region performance (TEDS↑ / NED↓) across 11 languages.

Method Type	Methods	NED	BLEU
Pipeline Models	MinerU2.5	0.314	0.551
	Dolphin-1.5	0.332	0.491
	MonkeyOCR-pro-1.2B	0.276	0.538
	MonkeyOCR-pro-3B	0.277	0.539
	PaddleOCR-VL	0.230	0.586
Expert Models	POINTS-Reader	0.285	0.560

Appendix

Table A7 : Formula-region performance (NED↓ / BLEU↑) across 11 languages.

Method Type	Methods	EN	FIL	ID	KM	LO	MS	MY	PT	TH	VI	ZH	Avg.
Pipeline Models	MinerU2.5	0.146	0.072	0.136	0.571	0.566	0.122	0.32	0.133	0.638	0.197	0.208	0.27
	Dolphin-1.5	0.124	0.063	0.106	0.337	0.409	0.105	0.29	0.096	0.409	0.097	0.119	0.188
	MonkeyOCR-pro-1.2B	0.161	0.092	0.15	0.309	0.342	0.139	0.28	0.151	0.447	0.461	0.151	0.247
	MonkeyOCR-pro-3B	0.161	0.087	0.123	0.331	0.327	0.11	0.255	0.135	0.494	0.271	0.15	0.221
	PaddleOCR-VL	0.118	0.074	0.087	0.245	0.309	0.085	0.213	0.097	0.071	0.114	0.102	0.13
Expert Models	POINTS-Reader	0.118	0.044	0.092	0.65	0.601	0.088	0.503	0.111	0.552	0.164	0.14	0.265

Appendix

Table A8 : Reading-order NED by language.

Method Type	Methods	Academic paper	Book	Exam paper	Magazine	Newspaper	Note	Research report	Slide	Textbook	Avg.
Pipeline Models	MinerU2.5	0.290	0.306	0.268	0.235	0.233	0.218	0.344	0.205	0.307	0.267
	Dolphin-1.5	0.145	0.179	0.231	0.187	0.199	0.158	0.203	0.175	0.221	0.189
	MonkeyOCR-pro-1.2B	0.208	0.233	0.273	0.277	0.336	0.198	0.271	0.182	0.282	0.251
	MonkeyOCR-pro-3B	0.173	0.204	0.235	0.248	0.303	0.176	0.248	0.171	0.265	0.225
	PaddleOCR-VL	0.076	0.118	0.149	0.157	0.187	0.096	0.145	0.105	0.167	0.133
Expert Models	POINTS-Reader	0.233	0.304	0.237	0.233	0.345	0.225	0.346	0.163	0.302	0.265

Appendix

Table A9 : Reading-order NED by page type.

Language group	Avg. accuracy (%)	Languages
High-resource	45.78	EN, ZH, ID, MS
Low-resource	17.45	LO, KM, MY, FIL
Low-resource	17.45	PT, TH, VI

Appendix

Table A10 : TEC-VQA accuracy for high-resource vs. low-resource language groups.

Stage	Question	Answer	Issues / actions
Original generation	What is the total?	12.0	Question is underspecified (no currency or context); answer format does not match the receipt (“12.80 SGD” printed on the image).
Intermediate check	What is the total price on the receipt (in dollars)?	12.50	Question is clearer, but the amount is misread (“12.80” on the image). Fails the strict alignment check; needs correction.
Final version	What is the total amount on the receipt (in SGD)?	12.80	Question specifies the context and currency; answer exactly matches the visible text. Capability labels: text recognition + numerical calculation.

Appendix

Table A11 : Illustrative TEC-VQA example showing one question–answer pair at three stages: original MLLM generation, intermediate human checking, and the final version that satisfies all checklist items. The example is based on a receipt-style image; text here is illustrative.

Figure A3 : Representative document page samples for English (EN), Filipino (FIL), and Indonesian (ID) in SEA-Vision.

Figure A4 : Representative document page samples for Khmer (KM), Lao (LO), and Malay (MS) in SEA-Vision.

Figure A5 : Representative document page samples for Burmese (MY), Portuguese (PT), and Thai (TH) in SEA-Vision.

Figure A6 : Representative document page samples for Vietnamese (VI) and Chinese (ZH) in SEA-Vision.

Figure A7 : Representative TEC-VQA examples across multiple Southeast Asian languages.

Explore similar work

MORE: A Multilingual Document Parsing Benchmark and Evaluation
Jul 3, 2026Long Xu, Binghong Wu, Tinghao Yu +6Document Parsing Multilingual VLM Evaluation
Tencent, Shenzhen, China
JaWildText: A Benchmark for Vision-Language Models on Japanese Scene Text Understanding
Mar 30, 2026Koki Maeda, Naoaki OkazakiVLM Evaluation Scene Text Recognition
Institute of Science Tokyo, Tokyo, Japan · Research and Development Center for Large Language Models, National Institute of Informatics, Tokyo, Japan
SEA-CLIP-Tiny: Efficient Multilingual Text-Vision Embedding for Southeast Asian Languages
Sep 25, 2026Puja Ahmad Habibi, Faiz Assabil Firdaus, Ashvanth S +3Vision-Language Models Cross-Modal Retrieval
SEACrowd · University of Indonesia · Cohere Labs Community +4

Jul 3, 2026J/K move · Enter open · S save

MORE: A Multilingual Document Parsing Benchmark and Evaluation

Long Xu, Binghong Wu, Tinghao Yu, Hao Feng+5

Tencent, Shenzhen, China

Document Parsing Multilingual VLM Evaluation

Open paper AskPDF arXiv

Multilingual documents encapsulate rich regional cultures, scientific discoveries, and historical records. Parsing this content into structured, machine-readable formats is critical for unlocking global knowledge. However, existing benchmarks predominantly focus on high-resource languages like English and Chinese, creating an evaluation blind spot concerning model performance on other languages. While recent Vision-Language Models (VLMs) claim support for hundreds of languages, the lack of ground truth makes it impossible to empirically verify these capabilities. To bridge this gap, we introduce MORE, a large-scale benchmark designed for multilingual document parsing evaluation. MORE distinguishes itself through three key dimensions: (1) Unprecedented Scale: It covers 149 languages, making it the most linguistically diverse benchmark to date; (2) Structural Complexity: Unlike previous works, it extends evaluation beyond plain text to include structural elements such as code blocks, tables, and catalogs; and (3) Data Authenticity: All samples are curated from real-world documents via a model-assisted, human-refined annotation pipeline. We evaluate state-of-the-art models using MORE, establishing new performance baselines for long-tail languages and validating the benchmark's effectiveness in diagnosing model capabilities in realistic, diverse scenarios. The MORE dataset will be available at https://github.com/zimoqingfeng/MORE.

Abstract

Explore similar work

MORE: A Multilingual Document Parsing Benchmark and Evaluation

JaWildText: A Benchmark for Vision-Language Models on Japanese Scene Text Understanding

SEA-CLIP-Tiny: Efficient Multilingual Text-Vision Embedding for Southeast Asian Languages