DocSeeker: Structured Visual Reasoning with Evidence Grounding for Long Document Understanding

Hao Yan; Yuliang Liu; Xingchen Liu; Yuyi Zhang; Minghui Liao; Jihao Wu; Wei Chen; Xiang Bai

cs.AIApr 14, 2026

DocSeeker: Structured Visual Reasoning with Evidence Grounding for Long Document Understanding

Authors: Hao Yan, Yuliang Liu, Xingchen Liu, Yuyi Zhang, Minghui Liao, Jihao Wu, Wei Chen, Xiang Bai

Organizations: Huazhong University of Science and Technology · Huawei Inc.

Abstract

Existing Multimodal Large Language Models (MLLMs) suffer from significant performance degradation on the long document understanding task as document length increases. This stems from two fundamental challenges: 1) a low Signal-to-Noise Ratio (SNR), with crucial evidence buried in irrelevant pages; and 2) supervision scarcity, as datasets offering only final short answers provide a weak learning signal. In this paper, we address these challenges by proposing a paradigm that requires the model to execute a structured Analysis, Localization and Reasoning workflow. To instill this capability, we design a two-stage training framework: we first perform Supervised Fine-Tuning on high-quality data generated via an efficient knowledge distillation strategy. Subsequently, we employ an Evidence-aware Group Relative Policy Optimization which jointly optimizes for both evidence localization and answer accuracy. Additionally, we introduce a Evidence-Guided Resolution Allocation strategy to mitigate memory constraints of training on multi-pages documents. Extensive experiments demonstrate that DocSeeker achieves superior performance on both in-domain and out-of-domain tasks. We show it robustly generalizes from short-page training to ultra-long documents and is naturally synergistic with visual Retrieval-Augmented Generation systems, serving as a solid foundation for their implementation.

Figures & tables

Figure 1: Overview of Principal Experimental Results in Long Document Understanding (Left) and the ALR Reasoning Paradigm of DocSeeker (Right).

Figure 2: Overview of our training framework. (Left) Stage I SFT, where the model is supervised fine-tuned on high-quality ALR CoT data; Stage II RL, where the Policy model is optimized via reinforcement learning, guided by a multi-dimensional reward function that includes format, localization, and accuracy. (Right) Our proposed Evidence-Guided Resolution Allocation (EGRA) strategy.

Method	Venue	Param.	Evi.	RAG	DUDE	MPDocVQA	MMLong.	LongDoc.	SlideVQA
Open Source
HiVT5 [ 1 ]	PR	0.3B	T+P	×	23.1	62.0	-	-	-
CREAM [ 45 ]	ACM MM 2024	7B	T+P	✓	52.5	74.3	-	-	-
mPLUG-DocOwl2 [ 10 ]	ACL 2025	8B	P	×	46.8	69.4	13.4	5.3	-
M3DocRAG [ 46 ]	Arxiv 2024	9B	P	$\checkmark$	-	84.4	21.0	35.1	55.7
Vis-RAG [ 17 ]	ICLR 2025	4B	P	$\checkmark$	-	70.9	18.8	41.9	50.7

Table 1: Comparison of different methods on five document understanding benchmarks. The results are reported on DUDE (ANLS), MPDocVQA (ANLS), MMLongBench-doc (Acc), LongDocURL (Acc), and SlideVQA (F1). “Param.” denotes parameter scale; Baseline-SFT (short-answer) refers to the baseline model fine-tuned on raw short-answer data; “Evi.” indicates evidence type (T = text, P = pages, I = images extracted from pages); “RAG” indicates whether retrieval is used; and “MMLong.“ and “LongDoc.“ are abbreviations for MMLongBench-doc and LongDocURL. The best and second-best results are highlighted in bold and underlined, respectively.

Model	Input	Avg.	MMLong.
Model	Input	Avg.	Acc.	F1
Baseline	Evi-only	1.5	41.1	37.6
Baseline	Full-doc	43.4	25.4 (-15.7)	20.8 (-16.8)
DocSeeker	Full-doc	43.4	40.1 (-1.0)	38.4 (+0.8)

Table 2: Performance comparison of DocSeeker and the Baseline under two input conditions: Full-document input (Full-doc) and Evidence-only input (Evi-only). Avg. : Avg. input pages.

Figure 3: Performance comparison on different document length

Figure 4: Performance comparison on MMLongBench-doc following integration with a retrievar Coilpail. Acc measures answer correctness, while F1 quantifies evidence page retrieval. The k=Inf designation indicates the result for full-document input.

Data Config	Size	MMLong.
Data Config	Size	Acc.	F1
Data type ablations
Baseline	–	25.4	20.8
Raw short-answer data	6.3k	27.4	27.6
Vanilla CoT	6.3k	31.3	32.4
ALR CoT data	6.3k	33.8	33.9

Table 3: Ablation study on data types and size. The first group investigates different ALR CoT data configurations, while the second explores the effect of varying data size on performance.

Method	Strategy	Tokens	MMLong.
Method	Strategy	Tokens	Acc.	F1
Baseline	-	-	25.4	20.8
Fixed-Res	Full Low-Res.	576	34.2	33.5
Fixed-Res	Truncated	1024	36.6	35.8
EGRA	Full	1024 or 256	38.6	36.9
	-w/o Non-Evi. Hi-Res	1024 or 256	35.5	33.8

Table 4: Ablation study on training resolution strategies. “Fixed-Res” denotes fixed resolution for all pages, while EGRA adjusts page resolution during training based on evidence relevance. Tokens indicate the number of visual tokens used to encode an image.

Method	Weight	MMLong.
Method	$(\lambda_{1},\,\lambda_{2},\,\lambda_{3})$	Acc.	F1
SFT	–	38.6	36.9
Vanilla GRPO	$(0.1,\ 0.0,\ 0.9)$	38.7	36.3
	$(0.1,\ 0.1,\ 0.8)$	39.4	37.8
EviGRPO	$(0.1,\ 0.3,\ 0.6)$	40.1	38.4
	$(0.1,\ 0.5,\ 0.4)$	38.9	37.0

Table 5: Ablation on GRPO variants and reward weights. Vanilla GRPO optimizes answer correctness only; EviGRPO further includes localization-related rewards.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Data Config	Size	MMLongbench.		DUDE
Data Config	Size	Acc.	F1	ANLS
Baseline	-	25.4	20.8	35.2
Raw short-answer data	6.3k	27.4	27.6	48.8
Vanilla CoT data	6.3k	31.3	32.4	48.7
ALR CoT data	6.3k	33.8	33.9	48.9
-w/o Page id	6.3k	30.4	31.1	47.5

Appendix

Table 6: Ablation study on data types and training data size. The first group investigates different CoT data configurations, while the second explores the effect of varying data size on performance.

Panel of Figure 6 — Figure 6: ALR CoT Data Distillation Prompt Templete.

Panel of Figure 7 — Figure 7: Vanilla CoT Data Distillation Prompt Templete.

	Token	Latency	Acc
Baseline	202	19s	25.4
DocSeeker	401	25s	40.1

Appendix

Table 7: Analysis of inference latency and accuracy trade-off on MMLongBench-Doc

	Acc $\geq 0.5$	Acc $=0$	Total
Recall $=1$	300	288	588
Recall $<1$	121	373	494
Total	421	661	1082

Appendix

Table 8: Stage-wise error breakdown on MMLongBench-Doc.

Panel of Figure 8 — Figure 8: LLM-based Judge Prompt Templete in Secondary Verification.

Figure 9: Failure cases in ALR CoT Data Distillation.

Method	Model	OCR	Retrieval Config		Backbone	Trained on Dataset
Method	Model	OCR	Retriever	Top-k	Backbone	DUDE	MPDocVQA	SlideVQA
RAG	CREAM [ 45 ]	$\checkmark$	bge-large [ 51 ]	3	Pix2Struct/LLaMa2	$\checkmark$	$\checkmark$	$\times$
	M3DocRAG [ 46 ]	$\times$	Colpali [ 18 ]	4	Qwen2-VL	$\times$	$\times$	$\times$
	Vis-RAG [ 17 ]	$\times$	VisRAG-Ret [ 17 ]	3	MiniCPM-V 2.6	$\times$	$\times$	$\times$
	SV-RAG [ 34 ]	$\times$	SV-RAG-InternVL2 [ 34 ]	5	SV-RAG-InternVL2	$\times$	$\times$	$\checkmark$
	VDocRAG [ 35 ]	$\times$	VDocRetriever [ 35 ]	3	VDocGenerator	$\checkmark$	$\times$	$\times$

Appendix

Table 9: Detailed Configurations of Compared Methods on Long Document Understanding.

Method	OCR	Retriever	Backbone	Training- free	Page-level Reasoning	Evidence Localization
OCR-based
Longformer [ 31 ]	$\checkmark$	$\times$	RoBERTa	$\checkmark$	$\times$	$\times$
DocLLM [ 52 ]	$\checkmark$	$\times$	Falcon/LLaMa2	$\checkmark$	$\times$	$\times$
LayTokenLLM [ 32 ]	$\checkmark$	$\times$	LLaMa3/LLaMa2/Qwen1.5	$\times$	$\times$	$\times$
End-to-End
HiVT5 [ 1 ]	$\checkmark$	$\times$	DiT/T5	$\times$	$\times$	$\times$

Appendix

Table 10: Comparison of method architectures and attributes, where "OCR" and "Retriever" denote dependency on external modules, "Training-free" indicates the absence of additional fine-tuning on multi-page documents, "Page-level Reasoning" refers to processing at page granularity, and "Evidence Localization" marks the capability for explicit source grounding.

Figure 10: Qualitative Comparison of Reasoning Paradigms.

Figure 11: Performance Comparison across Different Training Stages.

Document Understanding

Document understanding enables machines to interpret text, layout, tables, and images to extract information and answer questions about documents.

See all 91 papers

DocPilot: Interactive Agent Orchestration and Training for Long Visual Document Understanding
Oct 15, 2026Zhuowen Liang, Xiaotian Lin, Yuyu Luo +2Document QA Multimodal Agents
The Hong Kong University of Science and Technology (Guangzhou) · EvenUp, USA
DocScope: Benchmarking Verifiable Reasoning for Trustworthy Long-Document Understanding
May 9, 2026Xiang Feng, Jiawei Zhou, Zhangfeng Huang +6Multimodal Understanding LLM Answer Verification
School of Computer Science, National Engineering Research Center for Multimedia Software and Hubei Key Laboratory of Multimedia and Network Communication Engineering, Wuhan University, China · Alibaba Group, Hangzhou, China · Independent Researcher +1
DocMemo: Dynamic Evidence Discovery via Probabilistic Memory-Guided Retrieval for Multi-Modal Document Understanding
Aug 7, 2026Hanshu Yao, Janfeng Zhong, Niu Lian +1Multimodal Memory Document QA
1Harbin Institute of Technology, Shenzhen · 2Tsinghua Shenzhen International Graduate School, Tsinghua University

Oct 15, 2026J/K move · Enter open · S save

DocPilot: Interactive Agent Orchestration and Training for Long Visual Document Understanding

Zhuowen Liang, Xiaotian Lin, Yuyu Luo, Haixun Wang+1

The Hong Kong University of Science and Technology (Guangzhou) · EvenUp, USA

Document QA Multimodal Agents Multimodal QA LLM Agent Training Long-Context VLMs

Open paper AskPDF arXiv

Multimodal long-document question answering requires models to seek sparse evidence across pages and modalities, understand heterogeneous document content, and reason over the collected evidence. Even advanced vision-language models (VLMs) can overlook sparse evidence and struggle with cross-modal reasoning in a single pass. Existing document agents further introduce multi-step interaction, but some still rely on predefined workflows or coarse-grained actions, limiting the flexibility and granularity of document reasoning. We introduce DocPilot, an agentic document framework that supports expressive orchestration and makes complex document solving learnable by small language models (SLMs). DocPilot orchestrates evidence seeking through page-level navigation and fine-grained evidence localization, and integrates local visual inspection, code-based numerical reasoning, and cross-modal reasoning for multimodal understanding. To transfer these capabilities to small models, we propose curriculum-based agentic training that progresses from single-ability learning to adaptive orchestration through supervision from data-grounded reasoning traces and executable interaction trajectories. Across MMLongBench-Doc, DocBench, FinRAGBench-V, and MADQA, DocPilot achieves strong gains with frontier VLM backbones, while DocPilot-4B delivers competitive performance against substantially larger models, demonstrating strong effectiveness and generalization across diverse long-document reasoning settings. The code is available at https://github.com/HKUSTDial/DocPilot.

Abstract

Document Understanding

Explore similar work

DocPilot: Interactive Agent Orchestration and Training for Long Visual Document Understanding

DocScope: Benchmarking Verifiable Reasoning for Trustworthy Long-Document Understanding

DocMemo: Dynamic Evidence Discovery via Probabilistic Memory-Guided Retrieval for Multi-Modal Document Understanding