PulseBench-Tab: A Multilingual Benchmark for Table Extraction with Graph-Based Evaluation

Ritvik Pandey; Sid Manchkanti; Mohammed Wazir Adain; Mohammed Hadi; Dushyanth Sekhar

cs.IRApr 21, 2026

PulseBench-Tab: A Multilingual Benchmark for Table Extraction with Graph-Based Evaluation

Authors: Ritvik Pandey, Sid Manchkanti, Mohammed Wazir Adain, Mohammed Hadi, Dushyanth Sekhar

Organizations: 1Pulse AI · 2Georgia Institute of Technology · 3S&P Global, Enterprise Data Organization

Abstract

We introduce PulseBench-Tab, an open multilingual benchmark for evaluating table extraction from document images. The benchmark comprises 1,820 human-annotated tables spanning 9 languages and 4 scripts (Latin, CJK, Arabic, Cyrillic), drawn from 380 real-world source documents including financial filings, government reports, and regulatory disclosures. Tables range from 2 to 1,183 cells, with 48.1% containing merged or spanning cells. Alongside the dataset, we propose T-LAG (Table Logical Adjacency Graph), a novel evaluation metric that models tables as directed graphs over cell adjacencies and computes structural and content fidelity in a single score via optimal bipartite matching. We evaluate 9 commercial and open-source table extraction systems across the benchmark and report per-language breakdowns. The full dataset, scoring code, and all provider outputs are publicly available.

Explore similar work

TABVERSE: Benchmarking Cross-Format Table Understanding in LLMs and VLMs
Jun 8, 2026Momina Ahsan, Sarfraz Ahmad, Ming Shan Hee +2Tabular Data Representation Quality
HakushoBench: A Japanese Chart and Table VQA Benchmark from Governmental White Papers
May 31, 2026Issa Sugiura, Shuhei Kurita, Yusuke Oda +1Chart Tabular Data
Tables Decoded: DELTA for Structure, TARQA for Understanding
Sep 15, 2026Jahanvi Rajput, Dhruv Kudale, Saikiran Kasturi +2Tabular Data Multilingual

Jun 8, 2026 · cs.AIJ/K move · Enter open · S save

TABVERSE: Benchmarking Cross-Format Table Understanding in LLMs and VLMs

Momina Ahsan, Sarfraz Ahmad, Ming Shan Hee, Roy Ka-Wei Lee+1

1Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) · 2Singapore University of Technology and Design (SUTD)

Large Language Models (LLMs) and Vision-Language Models (VLMs) are increasingly evaluated on table reasoning tasks, but the role of table representation remains under-explored. In practice, the same table content may appear in different structural formats, such as HTML, Markdown, and LaTeX, or as rendered images. However, existing evaluations often let content, format, layout, and modality vary together, making it difficult to isolate representation effects. We introduce TABVERSE, a controlled multimodal table benchmark that aligns the same table content across multiple structural formats and rendered images, with question category and difficulty tags. This design enables systematic evaluation of representation effects while holding table content fixed. We evaluate LLMs and VLMs across three tasks: Question Answering (QA), Structural Understanding Capability (SUC), and Structure Reconstruction (SR). Our results show that representation choice substantially affects table understanding. Models generally perform better with structured text than with rendered images, but the size of this gap depends on the task, model, and format. HTML is often the most robust text format, while row-sensitive structural tasks and syntactically usable LaTeX reconstruction remain challenging. These findings show that table representation is a key factor in reliable table evaluation.

Tabular Data Representation Quality

Open paper Ask about it PDF

Abstract

Explore similar work

TABVERSE: Benchmarking Cross-Format Table Understanding in LLMs and VLMs

HakushoBench: A Japanese Chart and Table VQA Benchmark from Governmental White Papers

Tables Decoded: DELTA for Structure, TARQA for Understanding