INDOTABVQA: A Benchmark for Cross-Lingual Table Understanding in Bahasa Indonesia Documents

Somraj Gautam; Anathapindika Dravichi; Gaurav Harit

cs.CVApr 13, 2026

INDOTABVQA: A Benchmark for Cross-Lingual Table Understanding in Bahasa Indonesia Documents

Authors: Somraj Gautam, Anathapindika Dravichi, Gaurav Harit

Organizations: IIT Jodhpur · Punjabi University

Abstract

We introduce INDOTABVQA, a benchmark for evaluating cross-lingual Table Visual Question Answering (VQA) on real-world document images in Bahasa Indonesia. The dataset comprises 1,593 document images across three visual styles (bordered, borderless, and colorful) with one or more than one tables, and 1,593 question-answer sets in four languages: Bahasa Indonesia, English, Hindi, and Arabic. This enables evaluation of Vision-Language Models (VLMs) in both monolingual (Bahasa documents with Bahasa questions) and cross-lingual settings (Bahasa documents with questions in other languages). We benchmark leading open-source VLMs (Qwen2.5-VL, Gemma-3, LLaMA-3.2) and GPT-4o and reveal substantial performance gaps, particularly on structurally complex tables and in low-resource languages. Fine-tuning a compact 3B and LoRA-finetuned 7B model on our dataset yields 11.6% and 17.8% improvements in accuracy. Providing explicit table region coordinates as additional input further improves performance by 4-7%, demonstrating the value of Spatial priors for table-based reasoning. Our findings underscore the importance of language-diverse, domain-specific datasets and demonstrate that targeted fine-tuning can significantly enhance VLM performance on specialized document understanding tasks. INDOTABVQA provides a valuable resource for advancing research in cross-lingual, structure-aware document understanding, especially in underrepresented regions of the world. Full dataset can be accessed in huggingface at: https://huggingface.co/datasets/NusaBharat/INDOTABVQA}

Figures & tables

Figure 1: IndoTabVQA presents document images in Bahasa Indonesia, and semantically aligned QA pairs in four languages, enabling cross-lingual evaluation of VLMs.

Figure 2: Architecture comparison with left-to-right pipeline flow across three evaluation settings. Each row represents a complete evaluation pipeline from input to output. Setting 1: Uses a pretrained VLM without fine-tuning (zero-shot). Setting 2: Fine-tunes the model on table QA data but without spatial information. Setting 3: Introduces spatial priors through table detection, enabling the model to use table locations during reasoning.

Property	IndoTabVQA
# Document Images	1,593
# Total Tables	1,910
Avg. Tables per Image	1.20
# QA Pairs	6,372 (Bahasa+English+Hindi+Arabic)
QA per Language	1,593 per language
Languages	Bahasa Indonesia, English, Hindi, Arabic

Table 1: IndoTabVQA dataset properties covering multilingual QA, layout styles, and domain diversity.

	In-Match accuracy(%)					STS Accuracy(%)
Model [#params]	ID	EN	HI	AR	$\Delta$	ID	EN	HI	AR	$\Delta$
Open-source
Donut	10.5	5.48	4.74	4.39	6.20	15.52	9.10	5.17	6.03	8.96
Qwen2.5VL [3B]	37.8	28.7	4.1	16.4	21.9	29.0	44.9	4.4	27.5	26.5
Gemma3 [12B]	40.9	27.4	19.5	17.4	26.1	41.4	31.0	27.3	26.5	31.6
Qwen2.5VL [7B]	54.8	36.2	17.3	23.0	32.9	36.5	58.1	16.1	34.3	36.3

Table 2: Evaluation of various VLMs on In-Match and STS Accuracy across four languages here ID is IndoTabVQA-id , EN is IndoTabVQA-en , HI is IndoTabVQA-hi , AR is IndoTabVQA-ar , and SP is Spatial Prior,

\Delta

is average accuracy.

	In-Match accuracy(%)			STS Accuracy(%)
Model [#params]	Bordered	Borderless	Colorful	Bordered	Borderless	Colorful
Bahasa Indonesia
Donut	11.71	10.23	9.40	21.02	17.87	8.95
Qwen2.5VL-3B	32.73	44.25	36.36	24.62	34.27	27.9
Qwen2.5VL-7B	52.55	57.29	54.55	40.54	32.74	36.36
Gemma3-12B	48.05	34.78	39.81	48.35	32.74	43.26

Table 3: Results of various VLMs on In-Match and STS Accuracy based on table types across four languages.

Model [#params]	Agg.	Comp.	Look.	Str.
Indonesia
Donut	13.13	0	4.89	12.33
Qwen2.5VL-3B	50.59	41.18	19.14	30.14
Qwen2.5VL-7B	64.58	76.47	27.56	57.53
Gemma3-12B	40.7	47.06	38.77	47.95
Llama-3.2-11B	66.23	100	43.25	53.42

Table 4: Performance (In-match) across question types in four languages, Agg is Aggregation, Comp. is Comparison, Look. is Lookup, and Str. is Table structure related questions.

Figure 3: Global language coverage map for the IndoTabVQA benchmark. The shading intensity indicates the number of supported languages (1–3) spoken in each country. For example, Canada supports both English and Hindi. This visualization highlights the geographical and cultural reach of our cross-lingual benchmark.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Question Type	Percentage (%)
Aggregation	58.1
Lookup	33.2
Comparison	1.6
Table Structure	6.9

Appendix

Table 5: Distribution of Question Types

Figure 4: Comparative distribution of prediction error types across the four languages in the IndoTabVQA test set. The analysis categorizes failures into five types, revealing that ‘Incorrect’ and ‘Partial Correct’ are the dominant error modes. ‘Translation Error’ is a significant factor unique to the cross-lingual settings (English, Hindi, and Arabic), while ’Hallucination’ and ’Typo’ represent smaller but consistent sources of failure.

Benchmark	Cross lingual	QA Language	Visual Language	Table Focus
Tabular VQA:
DocVQA	✗	English	English	Partial
TableVQA-B	✗	English	English	✓
TabComp	✗	English	English	✓
ComTQA	✗	English	English	✓
XT-VQA	✓	(EN/FR/CH)	English	Partial

Appendix

Table 6: Comparison of VQA Benchmarks: Most existing benchmarks support the English language for visual content and QA. In contrast, our benchmark focuses on underrepresented low-resource languages, such as Bahasa Indonesia in vision, and includes a variety of languages in QA (including the reading order in case of Arabic). TableVQA-B is TableVQA-Bench Kim et al. (2024)

Aspect	Protocol Summary
Question types	Lookup, aggregation, comparison, structure
Avg. tokens per question	7–10 (across languages)
Translation workflow	MT → human validation → consistency check
Discarded items	3.1% (ambiguity, mistranslation)
Annotators	3 (Bahasa), 4 validators (EN/HI/AR)

Appendix

Table 7: Annotation protocol summary

Question	Answer	Predicted answer	E
Which column has 4 points as its contents?	form of learning	day (hour) (j), weight of value	H
What is the most common type of disability in Indonesia?	mentally disabled	physical handicap, multiple..	I
What is the unit of school that receives the l…	school	sekolah	T
What description is given from the falcon and eagle?	(all types of the family)	all species from the family	Ty
Which province has the highest number of persons with disabilities?	yogyakarta	west java, yogyakarta	P

Appendix

Table 8: Examples of errors, H=Hallucination, I=Incorrect, T=Translation, Ty=Typo, P=Partial Correct

Figure 5: Example of the IndoTabVQA correct predictions on mono-lingual and cross-lingual question answering across three table formats. Bordered (left), Borderless (middle), and Colorful (right). The examples include questions in Bahasa Indonesia, English, Hindi, and Arabic.

Table QA

Table question answering studies how AI systems retrieve, interpret, and reason over structured data to answer natural-language questions.

See all 66 papers

Jun 8, 2026

TABVERSE: Benchmarking Cross-Format Table Understanding in LLMs and VLMs

Large Language Models (LLMs) and Vision-Language Models (VLMs) are increasingly evaluated on table reasoning tasks, but the role of table representation remains under-explored. In practice, the same table content may appear in different structural formats, such as HTML, Markdown, and LaTeX, or as rendered images. However, existing evaluations often let content, format, layout, and modality vary together, making it difficult to isolate representation effects. We introduce TABVERSE, a controlled multimodal table benchmark that aligns the same table content across multiple structural formats and rendered images, with question category and difficulty tags. This design enables systematic evaluation of representation effects while holding table content fixed. We evaluate LLMs and VLMs across three tasks: Question Answering (QA), Structural Understanding Capability (SUC), and Structure Reconstruction (SR). Our results show that representation choice substantially affects table understanding. Models generally perform better with structured text than with rendered images, but the size of this gap depends on the task, model, and format. HTML is often the most robust text format, while row-sensitive structural tasks and syntactically usable LaTeX reconstruction remain challenging. These findings show that table representation is a key factor in reliable table evaluation.

Momina Ahsan, Sarfraz Ahmad, Ming Shan Hee +2

1Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) · 2Singapore University of Technology and Design (SUTD)

Document Understanding Table QA LLM Evaluation Table Structure Recognition Multimodal Model Evaluation Multimodal Understanding

Ask PDF

Sep 15, 2026

Tables Decoded: DELTA for Structure, TARQA for Understanding

Table understanding is a core task in document intelligence, encompassing two key subtasks: table reconstruction and table visual question answering (TabVQA). While recent approaches predominantly rely on vision- language models (VLMs) operating on table images, we propose a more scalable and effective alternative based on structured textual representations. These representations are easier to process, align more naturally with LLMs, and eliminate the need for language-specific visual encoders, making them particularly suitable for multilingual documents. We present DELTA, which separates physical structure recognition, logical structure recognition, and OCR to extract both layout and content accurately. DELTA outputs tables in Optimised Table Structure Language (OTSL), a compact and unified format that encodes cell arrangements and textual content. On table structure recognition (TSR), DELTA achieves TEDS- Structure scores comparable with state-of-the-art methods across FinTabNet, PubTabNet, and PubTables-1M. We further establish its robustness on non-English tables through our curated Hindi benchmark, TORQUE. Building on this, we introduce TARQA, an LLM fine-tuned on OTSL sequences. Our approach yields gains of 9.3 p.p. on WTQ (TabQA) and 9.2 p.p. on FinTabNetQA (TabVQA), respectively. On TORQUE, our method ranks second among all VLMs and DELTA + LLM variants. We release our code, models, and benchmark at: https://github.com/Tihiitborg/Tables-Decoded

Jahanvi Rajput, Dhruv Kudale, Saikiran Kasturi +2

Indian Institute of Technology Bombay · IIT Bombay · BharatGen

Table QA Table Structure Recognition

Ask PDF

May 31, 2026

HakushoBench: A Japanese Chart and Table VQA Benchmark from Governmental White Papers

Understanding chart and table images is essential for applying vision-language models (VLMs) to real-world document understanding. While English benchmarks have advanced rapidly, non-English counterparts remain scarce, leaving it unclear whether this progress generalizes across languages. A key obstacle is the difficulty of collecting realistic and diverse non-English chart and table images at scale. To address this, we leverage governmental white papers as a scalable source for benchmark construction beyond English, as they contain naturally occurring charts and tables across diverse formats and domains and are freely accessible in many countries. As a first instantiation, we introduce HakushoBench, a challenging Japanese chart and table VQA benchmark built from 33 governmental white papers. HakushoBench contains 2,053 images spanning over 10 image types, with manually annotated QA pairs, designed to assess deep and holistic understanding of charts and tables, rather than local visual cues alone. Experiments across a broad range of VLMs demonstrate that HakushoBench remains challenging for open-weight models: the best open-weight model achieves only 58.6% accuracy, and a 34.9-point gap between open-weight and proprietary models highlights substantial room for improvement in complex chart and table understanding. We release our dataset and code.

Issa Sugiura, Shuhei Kurita, Yusuke Oda +1

Institute of Science Tokyo · NII LLMC · NII

VLM Evaluation Data Visualization Table QA Chart QA Document Image Understanding

Ask PDF

Abstract

Table QA

Explore similar work

TABVERSE: Benchmarking Cross-Format Table Understanding in LLMs and VLMs

Tables Decoded: DELTA for Structure, TARQA for Understanding

HakushoBench: A Japanese Chart and Table VQA Benchmark from Governmental White Papers