LongDocBench: Benchmarking TOC Hierarchy and Contextual Relationship Recovery in Long Documents

Yuefeng Zou; Yichen Lu; Jingxiao Yang; Bingtao Fu; Gaoyang Zhang; Xiongfei Bai; Tian Chen; Xiang Qi

cs.AIAug 15, 2026

LongDocBench: Benchmarking TOC Hierarchy and Contextual Relationship Recovery in Long Documents

Authors: Yuefeng Zou, Yichen Lu, Jingxiao Yang, Bingtao Fu, Gaoyang Zhang, Xiongfei Bai, Tian Chen, Xiang Qi

Organizations: Ant Group · Zhejiang University

Abstract

Parsing visual documents into machine-readable representations is fundamental to document intelligence. Existing benchmarks focus on page-level element recognition, reading order, formula recognition, and table structure. Long documents, however, also require document-level structure recovery. This includes reconstructing cross-page table-of-contents (TOC) hierarchies and identifying typed links from tables and figures to their captions, notes, and sources, often in one-to-many form. Because these structures are covered only partially or subsumed within broader parsing protocols, existing benchmarks cannot directly evaluate two key document-level tasks: Table-of-Contents Hierarchy Recovery and Contextual Relationship Recovery. To benchmark these two tasks, we introduce LongDocBench, comprising 85 real-world financial reports, textbooks, and academic papers spanning 2,582 pages, with up to 105 pages per document. It provides human-verified annotations for 3,937 heading nodes (mean node depth 3.55; maximum depth 9) and 3,258 contextual relationships annotated across 2,680 table and figure objects. We further evaluate both the downstream utility and recoverability of these structures. Long-document question-answering experiments show that human-verified TOC hierarchies and contextual relationships improve reasoning, with their combination providing complementary benefits. Meanwhile, representative document parsers remain limited on both recovery tasks despite strong page-level performance. To support further progress, we publicly release LongDocBench and its evaluation protocol and reproducible testbed for advancing document-level structure recovery in long documents.

Figures & tables

Figure 1: Representative challenges in document-level structure recovery. (A) A parser preserves heading text but collapses the parent–child structure of the TOC hierarchy. (B) A source relationship is not recovered despite the presence of the figure and nearby contextual text.

Figure 2: Overview of LongDocBench . (A) The benchmark contains complete long-document inputs from the finance, textbook, and paper domains. (B) Table-of-Contents Hierarchy Recovery reconstructs complete, ordered TOC hierarchies across pages. (C) Contextual Relationship Recovery identifies typed, potentially one-to-many links from tables and figures to their captions, notes, and sources.

Figure 3: Construction and annotation pipeline of LongDocBench . TextIn generates candidate TOC hierarchies, typed contextual relationships, and auxiliary cross-page continuations. These candidates undergo full-document manual correction, automated validation, and two-stage expert review, yielding human-verified annotations that are visually grounded and traceable to TextIn element IDs.

Figure 4: Dataset statistics of LongDocBench . (a) Document-length distributions; circles mark domain medians. (b) Per-document TOC complexity. (c) Number of contextual elements linked to each table or figure object.

Domain	Docs	Pages	Avg. pages/doc.	Max pages
Finance	42	1,331	31.69	105
Textbook	18	884	49.11	76
Paper	25	367	14.68	35
All	85	2,582	30.38	105

Table 1: Document statistics by domain.

Domain	Heading nodes	Mean node depth	Max depth
Finance	2,582	3.81	9
Textbook	910	3.31	6
Paper	445	2.49	5
All	3,937	3.55	9

Table 2: TOC hierarchy statistics by domain.

Category	Table	Figure	Total
Objects	1,169	1,511	2,680
Caption relationships	713	990	1,703
Note relationships	587	374	961
Source relationships	206	388	594
All relationships	1,506	1,752	3,258

Table 3: Object and contextual-relationship statistics.

Model	Weighted TEDS		Macro TEDS
	w/ ign.	w/o ign.	w/ ign.	w/o ign.
TextIn	0.49	0.47	0.55	0.53
MinerU2.5-Pro	0.44	0.42	0.52	0.50
GLM-OCR	0.45	0.43	0.52	0.48
PaddleOCR-VL-1.5	0.45	0.42	0.52	0.49
PaddleOCR-VL-1.6	0.44	0.42	0.51	0.48

Table 4: TOC hierarchy recovery on all 85 documents. We report page-count-weighted and document-macro TEDS under the with- and without-ignorable settings; higher is better.

Figure 5: TOC hierarchy recovery by domain. Panels report (a) Weighted TEDS and (b) Macro TEDS. For each model and domain, the with-ignorable score is plotted upward and the without-ignorable score is mirrored downward; downward bars therefore encode nonnegative magnitudes rather than negative TEDS. Higher magnitudes indicate better recovery.

Recovery model	Caption	Note	Source	Macro
GPT-5.6-Sol (xhigh)	0.65	0.42	0.81	0.63
MiniMax-M2.5 (think)	0.53	0.17	0.61	0.44
GLM-5.2 (think)	0.64	0.30	0.74	0.56
Kimi K2.6 (w/o think)	0.62	0.22	0.82	0.55
Qwen3.5-397B (w/o think)	0.61	0.30	0.85	0.59
Qwen3.5-397B (think)	0.64	0.33	0.82	0.60

Table 5: Contextual relationship recovery using common TextIn outputs and benchmark-localized objects. Columns score individual relationship types; Macro is their unweighted mean. Higher is better.

Model	Text $\downarrow$	Formula (CDM) $\uparrow$	Table (TEDS) $\uparrow$	Table (TEDS-S) $\uparrow$	Reading Order $\downarrow$	Overall $\uparrow$
LingDT-VL-OCR-4B	0.02	95.37	91.75	93.90	0.10	95.07
MinerU2.5-Pro	0.01	95.78	93.33	94.75	0.11	95.96
PaddleOCR-VL-1.5	0.04	95.36	89.15	91.45	0.13	93.54
PaddleOCR-VL-1.6	0.04	95.50	89.67	92.12	0.13	93.67
GLM-OCR	0.03	95.71	88.76	91.25	0.14	93.86

Table 6: Page-level document parsing on all 2,582 pages under the OmniDocBench v1.6 protocol. Down arrows denote error metrics (lower is better), whereas up arrows denote similarity or accuracy metrics (higher is better).

Condition	Answer Acc.
Fixed + BM25	32.98
TextIn TOC	30.83
Verified TOC	45.95
Qwen3.5 Relations	31.90
Verified Relations	47.62
TextIn TOC + Qwen3.5 Relations	37.86

Table 7: Test-set Combined Answer Accuracy (%). Fixed + BM25 is the structure-agnostic chunk baseline. TextIn TOC and Qwen3.5 Relations use recovered structures, whereas Verified conditions use human-verified structures. All conditions share the same parsed content and answer model. The best result is shown in bold.

Document Parsing Benchmarks

See all 15 papers

XL-DocBench: Benchmarking Evidence-Grounded Extra-Long Document Understanding
Jul 21, 2026Hongchen Wei, Yuanzhe Wang, Bei Liu +9Long-Context Retrieval LLM Evaluation
1Wuhan University · 2Microsoft
Dr. DocBench: A Comprehensive Benchmark for Expert-Level and Difficult Document Parsing
May 31, 2026Minglai Yang, Xinyan Velocity Yu, Pengyuan Li +22VLM Evaluation Document Parsing Benchmarks
2077AI · University of Arizona · University of Southern California +8
DocScope: Benchmarking Verifiable Reasoning for Trustworthy Long-Document Understanding
May 9, 2026Xiang Feng, Jiawei Zhou, Zhangfeng Huang +6Multimodal Understanding LLM Answer Verification
School of Computer Science, National Engineering Research Center for Multimedia Software and Hubei Key Laboratory of Multimedia and Network Communication Engineering, Wuhan University, China · Alibaba Group, Hangzhou, China · Independent Researcher +1

Jul 21, 2026J/K move · Enter open · S save

XL-DocBench: Benchmarking Evidence-Grounded Extra-Long Document Understanding

Hongchen Wei, Yuanzhe Wang, Bei Liu, Yifan Yang+8

1Wuhan University · 2Microsoft

Long-Context Retrieval LLM Evaluation Long-Context QA Document QA Knowledge-Intensive QA

Open paper AskPDF arXiv

Real-world document tasks often ask professionals to answer questions from annual reports, regulations, clinical guidelines, and technical manuals that span hundreds or thousands of pages. Some questions also require comparing related reports. Reliable long-document understanding is therefore a prerequisite for using LLMs in compliance, clinical, financial, and engineering workflows, where decisions must be traceable to specific evidence pages and the cost of an unsupported answer is high -- yet most existing benchmarks still measure short-context or single-page QA. We introduce XL-DocBench, a fully human-verified benchmark for extra-long document understanding, with 1,519 retained questions from six professional domains and contexts up to 2,303 pages. XL-DocBench goes beyond page-level lookup. 1,103 examples (72.6%) use multiple evidence pages. The final set also includes 556 questions (36.6%) that use tables, charts, or figures, and 165 questions (10.9%) that require evidence from multiple documents. Each question has one of twelve reasoning labels, expert-annotated evidence pages, a typed verification rule, and an answer format, including 218 None-answer cases. We build the benchmark with a tree-guided synthesis pipeline followed by artifact filters and full verification by 194 human experts. By coupling extra-long professional contexts with page-level evidence and typed rules, XL-DocBench fills a gap left by prior single-page, short multi-page, or text-only long-context benchmarks, and lets future work attribute system failures to retrieval, evidence use, or rule following rather than to a single leaderboard score. The results show that current systems still struggle with long contexts, multi-page evidence, and structured reasoning over professional documents.

Abstract

Document Parsing Benchmarks

Explore similar work

XL-DocBench: Benchmarking Evidence-Grounded Extra-Long Document Understanding

Dr. DocBench: A Comprehensive Benchmark for Expert-Level and Difficult Document Parsing

DocScope: Benchmarking Verifiable Reasoning for Trustworthy Long-Document Understanding