TableNet A Large-Scale Table Dataset with LLM-Powered Autonomous

Ruilin Zhang; Kai Yang

cs.DBFeb 27, 2026

TableNet A Large-Scale Table Dataset with LLM-Powered Autonomous

Authors: Ruilin Zhang, Kai Yang

Organizations: School of Computer Science and Technology, Tongji University, 4800 Cao’an Road, Shanghai, 201804, China

Abstract

Table Structure Recognition (TSR) requires the logical reasoning ability of large language models (LLMs) to handle complex table layouts, but current datasets are limited in scale and quality, hindering effective use of this reasoning capacity. We thus present TableNet dataset, a new table structure recognition dataset collected and generated through multiple sources. Central to our approach is the first LLM-powered autonomous table generation and recognition multi-agent system that we developed. The generation part of our system integrates controllable visual, structural, and semantic parameters into the synthesis of table images. It facilitates the creation of a wide array of semantically coherent tables, adaptable to user-defined configurations along with annotations, thereby supporting large-scale and detailed dataset construction. This capability enables a comprehensive and nuanced table image annotation taxonomy, potentially advancing research in table-related domains. In contrast to traditional data collection methods, This approach facilitates the theoretically infinite, domain-agnostic, and style-flexible generation of table images, ensuring both efficiency and precision. The recognition part of our system is a diversity-based active learning paradigm that utilizes tables from multiple sources and selectively samples most informative data to finetune a model, achieving a competitive performance on TableNet test set while reducing training samples by a large margin compared with baselines, and a much higher performance on web-crawled real-world tables compared with models trained on predominant table datasets. To the best of our knowledge, this is the first work which employs active learning into the structure recognition of tables which is diverse in numbers of rows or columns, merged cells, cell contents, etc, which fits better for diversity-based active learning.

Figures & tables

Datasets	Diversity	Annotation	Domain	Collection methodology	Tables
TableNet (Ours)	True	S, C, H, V	Telecom (configurable)	See Figure 2	445K
PubTabNet	False	S, C, H	Medicine	rule-based collection	510K
FinTabNet	False	T, S, C, H	Finance	augmentation	113K
TableBank	True	T, H	General	rule-based collection	145K
SynthTabNet	True	T, S, C, H	General	augmentation	600K
ICDAR 2019	False	T, X	General	institutional contribution	3K

Table 1: Comparison of TSR datasets. For TableBank and SynthTabNet, we randomly sampled 1,000 images to assess color diversity and found that fewer than 10 were colored, indicating weak diversity. For TabRecSet and WTW, their diversity came from being captured by cameras in real world. Annotation legend: T: table bounding box (supports table detection); S: structural annotation (e.g., HTML tags, spanning); C: cell-level annotation (bounding box and content); H/X: final HTML/XML output or re-constructable from S and C; V: visual labels (e.g., simplicity, color, border style) provided in our dataset. Definition of diversity: A table image dataset is deemed as diverse if it supports at least two distinct and meaningful classification standards by which its images can be systematically categorized. These standards may include structural, visual attributes, allowing for multifaceted analysis and evaluation. For example, one classification standard could be based on structural complexity, such as whether a table contains spanning cells; another could be based on visual style, including attributes like border presence, background color, or watermark interference. A dataset that enables classification along both dimensions provides more comprehensive coverage for training and evaluating table structure recognition models.

Figure 1: Workflow of our multi-agent system.

Figure 2: Data collection pipeline. From top to bottom: agent generating, web crawling and open source augmenting. Annotation explanation: is simple denotes whether the table contains cells spanning. is colored denotes whether the table includes any background color, colored borders, or non-black font colors. is lined specifies whether the cell borders are fully present; tables with only horizontal or vertical lines are considered not lined.

Panel of Figure 3 — Figure 3: Invalid tables

Panel of Figure 4 — Figure 4: examples in TableNet

Complexity	Dim.	Avg Iters	Spear.	Pear.	Ken.
Simple	Struct.	1.03	0.8203	0.8111	0.7928
	Topic	1.07	0.8004	0.8085	0.6990
	Sem.	1.03	0.9253	0.8914	0.8522
Complex	Struct.	1.11	0.8826	0.8751	0.8832
	Topic	1.08	0.7352	0.7408	0.6457
	Sem.	1.17	0.8753	0.8411	0.8019

Table 2: Correlation between human and filling checker ranks. Struct. and Sem. represent structure and semantics.

dataset	TEDS
TableNet(Ours)	0.7403
PubTabNet	0.5041
FinTabNet	0.4495
SynthTabNet	0.5242
TableBank	0.5401

Table 3: TSR performance on unseen real-world data of models trained on different existing datasets.

		is simple		is colored		is lined
Models	All	Simple	Complex	Colored	Colorless	Lined	Lineless
Qwen
Qwen2-VL-2B(FT)	0.877	0.860	0.912	0.874	0.892	0.891	0.861
Qwen2-VL-72B	0.721	0.822	0.604	0.726	0.713	0.711	0.729
Qwen2-VL-72B(1-shot)	0.696	0.834	0.587	0.706	0.685	0.699	0.695
Qwen2-VL-72B(5-shot)	0.688	0.791	0.576	0.695	0.682	0.689	0.687

Table 4: TEDS of HTML on TableNet and subsets.

	Simple		Complex
Methods	with	without	with	without
Qwen2.5	0.954	0.978	0.782	0.835
DeepSeek-R1	0.938	0.974	0.940	0.901
GPT-4o	0.980	0.978	0.931	0.928
Agent Tool	1 -0.004	1 -0.003	1 -0.012	1 -0.009

Table 5: Table Fidelity using Structure-only TEDS of HTML generated by LLM through directly prompting. Small texts after ones means TEDS structure infidelity introduced by LLM infilling, which is lowest.

Figure 6: Compared with existing CoSyn pipeline, which is an LLM end-to-end method. In the middle subfigure, H represents horizontally lined and V represents vertically lined.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Sector	Industry	Spear.	Pear.	Ken.
Energy	Equipment & Service	0.8021 ± 0.015	0.7954 ± 0.007	0.7311 ± 0.007
	Equipment & Service	0.8204 ± 0.016	0.8080 ± 0.015	0.7551 ± 0.010
	Consumable Fuel	0.8344 ± 0.021	0.8047 ± 0.010	0.7612 ± 0.005
	Consumable Fuel	0.8121 ± 0.002	0.7853 ± 0.013	0.7224 ± 0.018
Material	Chemicals	0.8006 ± 0.017	0.8105 ± 0.015	0.7290 ± 0.022
	Chemicals	0.8788 ± 0.027	0.8446 ± 0.017	0.8054 ± 0.020

Appendix

Table 6: Average semantic correlation adding or subtracting half standard deviation of multiple Global Industry Classification Standard (GICS) industries’ tables. For each industry, the upper results line represents Chinese tables and the other line represents English tables.

Company Abbreviation
China Unicom	China Telecom	China Mobile
China Broadnet
Telecom-Related Keywords
5G	optical fiber	telecommunications
operator	base station	backbone network
gigabit	data center	FTTR

Appendix

Table 7: Keywords

Explore similar work

Apr 3, 2026

InstructTable: Improving Table Structure Recognition Through Instructions

Table structure recognition (TSR) holds widespread practical importance by parsing tabular images into structured representations, yet encounters significant challenges when processing complex layouts involving merged or empty cells. Traditional visual-centric models rely exclusively on visual information while lacking crucial semantic support, thereby impeding accurate structural recognition in complex scenarios. Vision-language models leverage contextual semantics to enhance comprehension; however, these approaches underemphasize the modeling of visual structural information. To address these limitations, this paper introduces InstructTable, an instruction-guided multi-stage training TSR framework. Meticulously designed table instruction pre-training directs attention toward fine-grained structural patterns, enhancing comprehension of complex tables. Complementary TSR fine-tuning preserves robust visual information modeling, maintaining high-precision table parsing across diverse scenarios. Furthermore, we introduce Table Mix Expand (TME), an innovative template-free method for synthesizing large-scale authentic tabular data. Leveraging TME, we construct the Balanced Complex Dense Synthetic Tables (BCDSTab) benchmark, comprising 900 complex table images synthesized through our method to serve as a rigorous benchmark. Extensive experiments on multiple public datasets (FinTabNet, PubTabNet, MUSTARD) and BCDSTab demonstrate that InstructTable achieves state-of-the-art performance in TSR tasks. Ablation studies further confirm the positive impact of the proposed tabular-data-specific instructions and synthetic data.

Boming Chen, Zining Wang, Zhentao Guo +5

Meituan · Beijing Institute of Technology

Table Structure Recognition Multimodal Instruction Tuning

Ask PDF

Apr 17, 2026

TableSeq: Unified Generation of Structure, Content, and Layout

We present TableSeq, an image-only, end-to-end framework for joint table structure recognition, content recognition, and cell localization. The model formulates these tasks as a single sequence-generation problem: one decoder produces an interleaved stream of HTML tags, cell text, and discretized coordinate tokens, thereby aligning logical structure, textual content, and cell geometry within a unified autoregressive sequence. This design avoids external OCR, auxiliary decoders, and complex multi-stage post-processing. TableSeq combines a lightweight high-resolution FCN-H16 encoder with a minimal structure-prior head and a single-layer transformer encoder, yielding a compact architecture that remains effective on challenging layouts. Across standard benchmarks, TableSeq achieves competitive or state-of-the-art results while preserving architectural simplicity. It reaches 95.23 TEDS / 96.83 S-TEDS on PubTabNet, 97.45 TEDS / 98.69 S-TEDS on FinTabNet, and 99.79 / 99.54 / 99.66 precision / recall / F1 on SciTSR under the CAR protocol, while remaining competitive on PubTables-1M under GriTS. Beyond TSR/TCR, the same sequence interface generalizes to index-based table querying without task-specific heads, achieving the best IRDR score and competitive ICDR/ICR performance. We also study multi-token prediction for faster blockwise decoding and show that it reduces inference latency with only limited accuracy degradation. Overall, TableSeq provides a practical and reproducible single-stream baseline for unified table recognition, and the source code will be made publicly available at https://github.com/hamdilaziz/TableSeq.

Laziz Hamdi, Amine Tamasna, Pascal Boisson +1

LITIS, Rouen, Normandy, France · Malakoff Humanis, Paris, France

Table Structure Recognition Multi-Token Prediction Document Image Understanding

Ask PDF

Sep 8, 2026

Enhancing Table Structure Recognition via Bounding Box Guidance

Table Structure Recognition (TSR) aims to extract the bounding boxes of cells and table structure (e.g., HTML) from table images. Although current approaches have made significant progress, the latest image-to-sequence methods overlook the explicit utilization of the bounding box information when predicting HTML sequences, leading to error predictions in complex scenes. In this paper, we introduce a novel framework BGTR (Bounding Box-Guided Table Recognizer). To more effectively utilize bounding box information, we first predict the bounding boxes of cells and then use this information to guide the generation of HTML sequences. While utilizing bounding box information can enhance the accuracy of HTML sequences, for natural scene tables, the data volume is too small to allow for sufficient training of bbox-guided HTML generation. In response, we adopt a progressive training method for natural scene tables and introduce SNSTab, a synthetically generated natural scene table dataset. Our experiments on five benchmark datasets demonstrate SOTA performance.

Lei Hu, Shuangping Huang

South China University of Technology, Guangzhou, China

Table Structure Recognition

Ask PDF