TDATR: Improving End-to-End Table Recognition via Table Detail-Aware Learning and Cell-Level Visual Alignment

Chunxia Qin; Chenyu Liu; Pengcheng Xia; Jun Du; Baocai Yin; Bing Yin; Cong Liu

cs.CVMar 24, 2026

TDATR: Improving End-to-End Table Recognition via Table Detail-Aware Learning and Cell-Level Visual Alignment

Authors: Chunxia Qin, Chenyu Liu, Pengcheng Xia, Jun Du, Baocai Yin, Bing Yin, Cong Liu

Organizations: University of Science and Technology of China · iFLYTEK Research

Abstract

Tables are pervasive in diverse documents, making table recognition (TR) a fundamental task in document analysis. Existing modular TR pipelines separately model table structure and content, leading to suboptimal integration and complex workflows. End-to-end approaches rely heavily on large-scale TR data and struggle in data-constrained scenarios. To address these issues, we propose TDATR (Table Detail-Aware Table Recognition) improves end-to-end TR through table detail-aware learning and cell-level visual alignment. TDATR adopts a ``perceive-then-fuse'' strategy. The model first performs table detail-aware learning to jointly perceive table structure and content through multiple structure understanding and content recognition tasks designed under a language modeling paradigm. These tasks can naturally leverage document data from diverse scenarios to enhance model robustness. The model then integrates implicit table details to generate structured HTML outputs, enabling more efficient TR modeling when trained with limited data. Furthermore, we design a structure-guided cell localization module integrated into the end-to-end TR framework, which efficiently locates cell and strengthens vision-language alignment. It enhances the interpretability and accuracy of TR. We achieve state-of-the-art or highly competitive performance on seven benchmarks without dataset-specific fine-tuning.

Figures & tables

Figure 1: Comparison of different table recognition paradigms. (a) Modular TR pipelines suffer from complex workflows and sub-optimization. (b) End-to-end TR models underperform in data-scarce scenarios due to weak detail perception. (c) Our “perceive-then-fuse” framework enhances structure and content awareness and unifies TR and cell localization for robust end-to-end TR.

Figure 2: (a) The architecture of the model. The model consists of a d vision encoder, a language decoder, and a structure-guided cell localization module, which aggregates cell representations based on TR priors refines cell boxes using multi-resolution visual features. (b) The perceive-then-fuse training strategy for end-to-end table recognition. In the table detail-aware learning phase, we design table structure understanding and content recognition tasks under a language modeling paradigm to enhance fine-grained perception. In the fusion phase, we fine-tune the model for table HTML parsing by aggregating the learned implicitly table details, while jointly training the cell localization module to strengthen cell-level visual alignment.

Data Source	Number	Samping Rate	Type
Webpage	ZH 2.1M, EN 12.3M	0.2	R,D
Paper	ZH 71M, EN 55.6 M	0.4	D
WuKong [ 12 ]	ZH 42.2M	0.1	R
README	1.1M	0.1	R,D
In-house	12M	0.2	R,D

Table 1: The data are used for table content recognition tasks. “ZH” and “EN” denote Chinese and English datasets, respectively. “R” indicates real-world data. “D” represents digitally-born data.

Real-World Tables		Digitally-Born Tables
Data Source	Number	Data Source	Number
iFLYTAB [ 64 ]	ZH 12k	PubTables-1M [ 45 ]	EN 721k
iFLYTAB-Aug	ZH 82K	PubTabNet [ 68 ]	EN 489k
WTW [ 26 ]	10K	Table generation	ZH 924k
TabRecSet [ 57 ]	30.5K	Re-render table	ZH 184k

Table 2: Table data statistics used in the table structure understanding tasks and table HTML parsing task. The amount of real-world table data is limited.

TabRecSet
Type	Method	TEDS-S↑	TEDS↑
TSR	TableMaster [ 60 ]	93.13	-
	$\text{LORE}^{\ast}$ [ 55 ]	96.82	-
	BGTR (PT) [ 14 ]	97.21	-
E2E-TR	EDD [ 68 ]	90.68	70.70
E2E-TR	$\text{TDATR}^{{\dagger}}$	97.27	92.70

Table 3: Comparison with state-of-the-art methods on TabRecSet and iFLYTAB-full. “*” represents our reproduced results, which are obtained by training from scratch using open-source code and configurations. Bold denotes the first performances. “+” indicates that TR results are obtained by post-processing. “

{\dagger}

” denotes results from a unified model without dataset-specific fine-tuning.

PubTables-1M
Type	Method	TEDS-S	TEDS	TEDS-D↓
TSR	UniTabNet [ 65 ]	98.73	-	-
	$\text{TabPedia}^{{\dagger}}$ [ 66 ]	95.66
	$\mathop{\text{DETR}}\limits_{\text{+PDF}}$	97.65	-	-
OCR-VLM	$\text{GOT}^{{\dagger}}$ [ 53 ]	-	36.84	-
OCR-VLM	$\text{Dolphin}^{{\dagger}}$ [ 10 ]	96.82	95.48	1.34

Table 4: The comparison result on Pubtables-1M and PubTabNet. “*” represents our reproduced results, which are obtained by inference using the released weights and official code. Underline denotes the second-best performance. “-ft” denotes further fine-tuning of our unified model on the PubTabNet. “PDF” and “GT” denote table content extracted from the PDF source file and the ground-truth annotations, respectively.

Method	OmniDocBench1.5		CC-OCR		OCRBenchv2
	TEDS-S	TEDS	TEDS-S	TEDS	TEDS-S	TEDS
MiniCPM-V 4.5 [ 59 ]	-	-	68.49	77.55	85.65	80.28
InternVL3.5-241B [ 52 ]	-	-	62.87	69.52	85.81	79.50
Qwen2.5-VL-72B [ 46 ]	-	-	86.48	81.22	86.58	81.33
dots.ocr [ 42 ]	84.42	81.94	81.65	75.42	86.27	82.04
MinerU2-VLM [ 49 ]	93.69	90.02	71.80	64.61	78.24	73.22

Table 5: The comparison of our method with various MLLMs and expert OCR VLM for table recognition.

Figure 3: The visualization of cell localization on challenging tables, including borderless (b,h), complex-structured (e,d), long (b), and low-quality images (a,g,h).

Model	Cell Loc Method	iFLYTAB-full	PubTabNet
Model	Cell Loc Method	AP50	AP50
SEMv3*	Split-and-Merge	92.92	85.12
LORE*	CornerNet [ 17 ]	91.87	-
UniTabNet*	Loc Token Parallel Clf	88.43	89.67
ED Loc Gen	Loc Token Sequential Clf	93.52	90.26
TDATR	Structure-guided Cell Loc	94.37	91.80

Table 6: Comparison of table cell localization results with different localization methods on iFLYTAB-full and PubTabNet. “ED Loc Gen” denotes the using TDATR’s encoder and decoder to autoregressively generate interleaved sequences of table recognition HTML and discrete cell coordinates.

	TDAL		TDF	iFLYTAB-full		PubTabNet
	Content	Structure	TDF	TEDS-S	TEDS	TEDS-S	TEDS
T0			✓	89.29	82.63	90.75	89.19
T1		✓	✓	94.82	90.44	94.30	92.45
T2	✓		✓	95.02	91.57	94.79	93.39
T3	✓	✓	✓	96.11	92.50	95.58	94.38

Table 7: A Ablation study about table detail-aware learning (TDAL) on iFLYTAB-full and PubTabNet. ”Content” denotes to table content tasks. And ”Structure” refers to table structure understanding tasks. Table detail fusion (TDF) refers to the HTML parsing task conducted during the fine-tuning phase.

	SGCL				iFLYTAB-full		PubTabNet
	Init-Reg	Enh	Struct-M	Ref	TEDS	$\text{AP}_{50}$	TEDS	$\text{AP}_{50}$
T3					91.88	-	94.38	-
C1	✓				93.42	87.44	94.78	89.63
C2	✓	✓			93.51	89.21	94.83	90.02
C3	✓	✓	✓		93.41	89.22	94.87	90.26
C4	✓	✓	✓	✓	93.52	94.37	94.80	91.81

Table 8: Ablation study on the structure-guided cell localization (SGCL) using iFLYTAB-full and PubTabNet. “Init-Reg” refers to coordinate regression using cell representation features. “Enh” stands for the bidirectional attention cell enhancement. “Ref” indicates the cell coordinate refinement design.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: The pipeline of unified multi-source table data processing. The pipeline normalizes heterogeneous table annotations from various sources into a unified representation for model training.

Figure 5: Illustration of table content recognition tasks. These tasks leverage diverse document data to enable text recognition, text localization, and reading-order understanding.

Figure 6: Illustration of table structure understanding tasks. These tasks equip the model with structure-awareness from both the cell level and the row/column level.

Method	Paradigm	Table training data	Extra data	Dataset specific	Notes
TableMaster	TSR	PubTabNet	–	Yes	–
LORE	TSR	PubTabNet, TabRecSet, and iFLYTAB	–	Yes	20k images were randomly sampled from PubTabNet for training. TabRecSet and iFLYTAB were reproduced by us based on the released code.
BGTR (PT)	TSR	TabRecSet, iFLYTAB, PubTabNet, FinTabNet and SynthTabNet	–	Yes	–
UniTabNet	TSR	iFLYTAB, PubTables-1M, and PubTabNet.	Pre-training: a synthetic dataset comprising 1.4 million Chinese and English samples from SynthDog, and PubTables-1M	Yes	–
EDD	E2E-TR	PubTabNet	–	Yes	–
SEMv3 + PPOCR	M-TR	PubTabNet and iFLYTAB	PPOCR relies on general text recognition data	Yes	“+PPOCR” indicates that the cell content is obtained from the PPOCR model.

Appendix

Table 9: Summary of the training data configurations of the baseline methods. For each method, we report the paradigm, table training data, auxiliary data, whether table-specific fine-tuning is applied, and additional notes.

Figure 7: The architecture of the structure-guided cell localization module, illustrated with the row-based cell enhancement example.

Figure 8: Visualization of table recognition results on real-world tables. In each subfigure, the left shows the input original table image, and the right presents the HTML-rendered visualization of the corresponding recognition result.

Figure 9: Visualization of table recognition results on long tables. In each subfigure, the left shows the input original table image, and the right presents the HTML-rendered visualization of the corresponding recognition result.

Figure 10: Visualization of table recognition results complex-structure tables. In each subfigure, the left shows the input original table image, and the right presents the HTML-rendered visualization of the corresponding recognition result.

Figure 11: Qualitatively comparison of the cell localization.

Table Structure Recognition

Table structure recognition recovers rows, columns, cells, spans, and hierarchy from table images and documents.

See all 32 papers

TableSeq: Unified Generation of Structure, Content, and Layout
Apr 17, 2026Laziz Hamdi, Amine Tamasna, Pascal Boisson +1Table Structure Recognition Multi-Token Prediction
LITIS, Rouen, Normandy, France · Malakoff Humanis, Paris, France
FastTab: A Fast Table Recognizer with a Tiny Recursive Module and 1D Transformers
May 21, 2026Laziz Hamdi, Amine Tamasna, Pascal Boisson +1Table Structure Recognition Document Image Understanding
LITIS, Rouen Normandy, France · Malakoff Humanis, Paris, France
InstructTable: Improving Table Structure Recognition Through Instructions
Apr 3, 2026Boming Chen, Zining Wang, Zhentao Guo +5Table Structure Recognition Multimodal Instruction Tuning
Meituan · Beijing Institute of Technology

Apr 17, 2026J/K move · Enter open · S save

TableSeq: Unified Generation of Structure, Content, and Layout

Laziz Hamdi, Amine Tamasna, Pascal Boisson, Thierry Paquet

LITIS, Rouen, Normandy, France · Malakoff Humanis, Paris, France

Table Structure Recognition Multi-Token Prediction Document Image Understanding

Open paper AskPDF arXiv

We present TableSeq, an image-only, end-to-end framework for joint table structure recognition, content recognition, and cell localization. The model formulates these tasks as a single sequence-generation problem: one decoder produces an interleaved stream of HTML tags, cell text, and discretized coordinate tokens, thereby aligning logical structure, textual content, and cell geometry within a unified autoregressive sequence. This design avoids external OCR, auxiliary decoders, and complex multi-stage post-processing. TableSeq combines a lightweight high-resolution FCN-H16 encoder with a minimal structure-prior head and a single-layer transformer encoder, yielding a compact architecture that remains effective on challenging layouts. Across standard benchmarks, TableSeq achieves competitive or state-of-the-art results while preserving architectural simplicity. It reaches 95.23 TEDS / 96.83 S-TEDS on PubTabNet, 97.45 TEDS / 98.69 S-TEDS on FinTabNet, and 99.79 / 99.54 / 99.66 precision / recall / F1 on SciTSR under the CAR protocol, while remaining competitive on PubTables-1M under GriTS. Beyond TSR/TCR, the same sequence interface generalizes to index-based table querying without task-specific heads, achieving the best IRDR score and competitive ICDR/ICR performance. We also study multi-token prediction for faster blockwise decoding and show that it reduces inference latency with only limited accuracy degradation. Overall, TableSeq provides a practical and reproducible single-stream baseline for unified table recognition, and the source code will be made publicly available at https://github.com/hamdilaziz/TableSeq.

Abstract

Table Structure Recognition

Explore similar work

TableSeq: Unified Generation of Structure, Content, and Layout

FastTab: A Fast Table Recognizer with a Tiny Recursive Module and 1D Transformers

InstructTable: Improving Table Structure Recognition Through Instructions