Multimodal long-document question answering requires models to seek sparse evidence across pages and modalities, understand heterogeneous document content, and reason over the collected evidence. Even advanced vision-language models (VLMs) can overlook sparse evidence and struggle with cross-modal reasoning in a single pass. Existing document agents further introduce multi-step interaction, but some still rely on predefined workflows or coarse-grained actions, limiting the flexibility and granularity of document reasoning. We introduce DocPilot, an agentic document framework that supports expressive orchestration and makes complex document solving learnable by small language models (SLMs). DocPilot orchestrates evidence seeking through page-level navigation and fine-grained evidence localization, and integrates local visual inspection, code-based numerical reasoning, and cross-modal reasoning for multimodal understanding. To transfer these capabilities to small models, we propose curriculum-based agentic training that progresses from single-ability learning to adaptive orchestration through supervision from data-grounded reasoning traces and executable interaction trajectories. Across MMLongBench-Doc, DocBench, FinRAGBench-V, and MADQA, DocPilot achieves strong gains with frontier VLM backbones, while DocPilot-4B delivers competitive performance against substantially larger models, demonstrating strong effectiveness and generalization across diverse long-document reasoning settings. The code is available at https://github.com/HKUSTDial/DocPilot.
Figures & tables
Figure 1: Illustration of DocPilot ’s capabilities in evidence seeking and multimodal understanding.
Figure 2: Performance of our proposed DocPilot on MMLongBench-Doc: (a) competitive accuracy with only 4B parameters; (b) performance gains with Gemini-2.5-Pro as the backbone.
Figure 3: Overview of DocPilot .
Algorithm 1 Inference of DocPilot
Figure 4: Curriculum-based training.
Model
MMLongBench-Doc
FinRAGBench-V
DocBench
TXT
LAY
CHA
TAB
FIG
UNA
Acc
F1
TXT
TAB
CHA
LasJ
LasJ
Vanilla VLMs
Open-source Models
Qwen3.5-4B
54.6
57.7
50.8
57.1
47.2
54.7
54.9
53.5
42.4
50.4
65.8
52.7
69.3
Qwen3.5-9B
55.6
56.3
47.5
52.5
47.9
60.1
55.6
55.2
43.1
55.5
58.1
52.9
69.8
Qwen3.5-397B-A17B
65.1
73.9
66.1
72.3
60.8
36.8
62.0
63.3
52.1
64.7
69.0
62.3
80.6
Table 1: Performance comparison on multimodal long-document QA benchmarks. Results are reported by evidence source: text (TXT), layout (LAY), chart (CHA), table (TAB), figure (FIG), and unanswerable (UNA), with overall accuracy and F1. † denotes the reused result from prior papers. Blue highlights the best, while Red indicates the second-best.
Figure 5: Ablation of core tools in DocPilot on MMLongBench-Doc.
Methods
MMLong
FinRAG
ANS
UNA
ALL
DocPilot -4B
56.3
70.0
59.1
60.2
– Only Multi-ability SFT
52.3
74.4
56.8
58.4
Table 2: Ablation study of different training strategies.
Figure 9Figure 10
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Benchmarks
Document
Question
Evi. Loc.
Doc. Input
Domain
#Pages
#Num
Expert
Page
Quote
Single
Multi
MMLongBench-Doc
Multiple
47.5
1,082
✓
✓
✗
✓
✗
DocBench
Multiple
66.0
1,102
✓
✗
✗
✓
✗
FinRAGBench-V
Financial
50.7
1,394
✓
✓
✗
✓
✗
MADQA
Multiple
23.3
2,250
✓
✓
✗
✓
✓
Appendix
Table 5: Statistics and characteristics of the evaluation benchmarks.
Type
Source Datasets
Primary Capabilities
Single- ability
<Navigate>
DocSeeker ( Yan et al., 2026b )
Page-level navigation
<Extract>
HotpotQA ( Yang et al., 2018 )
Evidence localization
<Solve>
File-CSV ( Zhang et al., 2025b )
Code Generation
<Answer>
FinQA ( Chen et al., 2021 )
Table reasoning
<Answer>
ChartQAPro ( Masry et al., 2025 ) , InfographicVQA ( Mathew et al., 2022 )
Chart and visual reasoning
Multi-ability
LongDocURL ( Deng et al., 2025 ) , DUDE ( Van Landeghem et al., 2023 )
Long-document, multimodal, and
Appendix
Table 6: Datasets used for data-grounded trajectory synthesis and their capability coverage.
Figure 13
Figure 9: Curriculum-based training.
Training Methods
Navigate
Extract
Focus
Solve
Analyze
Answer
Total
Curriculum-based Agentic Training
2.87
2.07
1.46
0.85
8.20
0.95
16.41
-Only Multi-ability Fine-tuning
3.75
2.03
0.74
0.71
8.21
0.98
16.43
Appendix
Table 8: Average number of tool calls under different training strategies on MMLongBench-Doc.
Model
Document Type & Domain
Overall
Research
Acad. Paper
Guidebook
Tutorial
Finance
Brochure
Industry
Qwen3.5-4B
54.6
52.0
54.6
62.7
51.3
52.2
59.3
54.9
Qwen3.5-9B
55.7
49.7
59.7
61.6
44.4
60.0
62.4
55.6
Qwen3.5-397B-A13B
61.9
58.8
56.3
65.3
68.8
62.0
65.4
62.0
GPT-5.4
58.3
53.0
56.1
64.5
71.8
62.0
64.2
60.1
Qwen3.7-Plus
60.5
58.8
60.5
65.9
72.6
64.9
61.1
62.6
Appendix
Table 9: Performance comparison on MMLongBench-Doc by document type. Bold indicates the best result, and underlined indicates the second best. Parenthesized values for DocPilot -4B denote percentage-point changes over vanilla Qwen3.5-4B; green indicates gains and red indicates declines.
Figure 10: A numerical computation case study showing DocPilot -4B’s orchestration.
Figure 11: An evidence extraction case study showing DocPilot -4B’s orchestration.
Existing Multimodal Large Language Models (MLLMs) suffer from significant performance degradation on the long document understanding task as document length increases. This stems from two fundamental challenges: 1) a low Signal-to-Noise Ratio (SNR), with crucial evidence buried in irrelevant pages; and 2) supervision scarcity, as datasets offering only final short answers provide a weak learning signal. In this paper, we address these challenges by proposing a paradigm that requires the model to execute a structured Analysis, Localization and Reasoning workflow. To instill this capability, we design a two-stage training framework: we first perform Supervised Fine-Tuning on high-quality data generated via an efficient knowledge distillation strategy. Subsequently, we employ an Evidence-aware Group Relative Policy Optimization which jointly optimizes for both evidence localization and answer accuracy. Additionally, we introduce a Evidence-Guided Resolution Allocation strategy to mitigate memory constraints of training on multi-pages documents. Extensive experiments demonstrate that DocSeeker achieves superior performance on both in-domain and out-of-domain tasks. We show it robustly generalizes from short-page training to ultra-long documents and is naturally synergistic with visual Retrieval-Augmented Generation systems, serving as a solid foundation for their implementation.
Hao Yan, Yuliang Liu, Xingchen Liu +5
Huazhong University of Science and Technology · Huawei Inc.
Long-document understanding often requires reasoning over many visually rich pages, making inference costly and prone to context rot. In this work, we propose InSight-doc, an agentic visual perception framework that treats visual resolution as an adaptive reasoning-time resource. InSight-doc starts from low resolution and selectively zooms into high-resolution regions for finer evidence, without relying on any external retriever. To train such an agent, we construct an active-perception corpus of 17.9K high-quality SFT examples with region-level zoom-in trajectories, accompanied by 19.2K hard RL examples. Through SFT+RL, InSight-doc-8B improves the baseline by 4.3--16.4 accuracy points over document VQA benchmarks. On long documents, it reduces hallucination by more than 40% and inference latency by 41%--68% while maintaining an accuracy lead. Our code, datasets, and model are released at https://github.com/m-Just/InSight-doc .
Kaican Li, Weiyan Xie, Lewei Yao +4
1The Hong Kong University of Science and Technology · 2Huawei
Multi-modal retrieval-augmented generation (RAG) is a key technique for visually rich long document understanding. Existing multi-modal RAG methods are progressively advancing toward multi-agent systems: they first retrieve relevant pages based on a query, and then iteratively understand information within those pages. However, these methods typically rely on fixed workflows and lack the ability to dynamically scale computation at test time, often leading to insufficient evidence. To address this, we propose D2-ScaleAgent, an agentic framework that introduces a dual-dimensional scaling paradigm for retrieval and reasoning. The core of D2-ScaleAgent is a Verifier agent-driven dynamic routing loop based on the intrinsic difficulty of the query, centered around a continuously updated evidence bank that serves as the agent's dynamic working memory: when retrieval needs to be expanded, the agent routes outward (retrieval scaling), decomposing the query into attributes and performing parallel page retrieval, followed by adaptive pruning to ensure comprehensive evidence coverage. When fine-grained reasoning is required, the agent routes inward (reasoning scaling), dynamically selecting sub-agents with varying granularity and count to extract evidence from pages. Finally, D2-ScaleAgent achieves logical closure over the evidence chain. Extensive experiments demonstrate that D2-ScaleAgent is effective on long and visually rich document benchmarks like MMLongBench-Doc, LongDocURL, etc.
Hao Zhang, Longrong Yang, Lunhao Duan +3
Zhejiang University · Alibaba Group · University of Science and Technology of China