D2-ScaleAgent: Dual-Dimensional Scaling for Long Document Understanding

Hao Zhang; Longrong Yang; Lunhao Duan; Ziyang Wang; Qing-Guo Chen; Shanshan Zhao

cs.CLAug 17, 2026

D2-ScaleAgent: Dual-Dimensional Scaling for Long Document Understanding

Authors: Hao Zhang, Longrong Yang, Lunhao Duan, Ziyang Wang, Qing-Guo Chen, Shanshan Zhao

Organizations: Zhejiang University · Alibaba Group · University of Science and Technology of China

Abstract

Multi-modal retrieval-augmented generation (RAG) is a key technique for visually rich long document understanding. Existing multi-modal RAG methods are progressively advancing toward multi-agent systems: they first retrieve relevant pages based on a query, and then iteratively understand information within those pages. However, these methods typically rely on fixed workflows and lack the ability to dynamically scale computation at test time, often leading to insufficient evidence. To address this, we propose D2-ScaleAgent, an agentic framework that introduces a dual-dimensional scaling paradigm for retrieval and reasoning. The core of D2-ScaleAgent is a Verifier agent-driven dynamic routing loop based on the intrinsic difficulty of the query, centered around a continuously updated evidence bank that serves as the agent's dynamic working memory: when retrieval needs to be expanded, the agent routes outward (retrieval scaling), decomposing the query into attributes and performing parallel page retrieval, followed by adaptive pruning to ensure comprehensive evidence coverage. When fine-grained reasoning is required, the agent routes inward (reasoning scaling), dynamically selecting sub-agents with varying granularity and count to extract evidence from pages. Finally, D2-ScaleAgent achieves logical closure over the evidence chain. Extensive experiments demonstrate that D2-ScaleAgent is effective on long and visually rich document benchmarks like MMLongBench-Doc, LongDocURL, etc.

Figures & tables

Figure 1: Architecture and performance of our proposed method, D2-ScaleAgent. (a) The framework achieves dual-dimensional scaling through a Verifier agent-driven dynamic routing loop, addressing evidence insufficiency by dynamically scaling outward for retrieval and inward for fine-grained understanding. (b) It shows improvements in comprehensive end-to-end QA accuracy across multiple long-document benchmarks, consistently outperforming existing baselines.

Figure 2: Overview of D2-ScaleAgent. (1) The system first retrieves a candidate page pool and initializes the Evidence Bank. (2) A Verifier agent-driven loop then routes computation outward for retrieval scaling or inward for reasoning scaling based on the current evidence gap. (3) Multi-granularity evidence is continuously accumulated into the Evidence Bank. (4) Once logical closure is achieved, the system generates the final answer from the saturated evidence state.

Method	MMLongBench-Doc	LongDocURL	PaperTab	FetaTab	ViDoSeek	UniDoc-Bench	Avg
GPT-4o
VQA	43.7	51.1	31.3	75.7	78.8	56.1	56.1
ViDoRAG Wang et al. (2025)	44.4	50.8	30.6	66.1	70.7	58.0	53.4
MoLoRAG Wu et al. (2025)	40.7	55.5	29.3	64.7	74.1	57.1	53.6
MDocAgent Han et al. (2025)	34.7	50.9	45.5	79.8	77.4	61.2	58.3
D2-ScaleAgent	52.0	56.0	47.5	82.8	81.8	62.2	63.7

Table 1: Performance comparison of different methods across multiple benchmarks. VQA takes the entire document as image input.

Figure 3: Retrieval performance comparison on MMLongBench-Doc and LongDocURL datasets.

Retrieval Scaling	Reasoning Scaling			Verifier- Driven Loop	MMLongBench-Doc
Retrieval Scaling	Surveyor	Locator	Extractor	Verifier- Driven Loop	MMLongBench-Doc
Evidence	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	54.9
$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	52.0
$\boldsymbol{\times}$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	46.8
$\checkmark$	$\boldsymbol{\times}$	$\checkmark$	$\checkmark$	$\checkmark$	46.5
$\checkmark$	$\checkmark$	$\boldsymbol{\times}$	$\checkmark$	$\checkmark$	47.1

Table 2: Ablation study of different components on the MMLongBench-Doc dataset.

Figure 4: Case study of D2-ScaleAgent. Given a cross-page compositional question comparing population sizes, both baseline methods fail due to either breadth or depth insufficiency. In contrast, D2-ScaleAgent, guided by the Verifier agent-driven dynamic routing loop, identifies that the initial evidence chain is incomplete, then triggers Attribute-Guided Retrieval Scaling to locate the missing page and Gap-Aware Reasoning Scaling to extract the required fine-grained numerical evidence.

Metric	MMLongBench-Doc	ViDoSeek
Accuracy	52.0	81.8
Retrieval rounds	1.33	1.12
Reasoning calls (S/L/E)	1.51 / 1.12 / 0.63	1.75 / 0.34 / 0.47
Verifier calls	1.76	1.14
Routing-agent calls	5.02	3.70
Tokens	21.4K	15.9K

Table 3: Compute-cost analysis and profiling of D2-ScaleAgent on MMLongBench-Doc and ViDoSeek using GPT-4o.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: A case study illustrating the dual-dimensional scaling behavior of D2-ScaleAgent. Starting from an incomplete initial retrieval result, the agent identifies missing evidence through Verifier feedback and dynamically scales outward to retrieve additional pages, and inward to extract fine-grained attributes from them. The execution trace shows how the agent progressively gathers cross-page evidence from the evidence bank and finally derives the correct answer through compositional reasoning.

Retrieval Scaling	Reasoning Scaling			Verifier- Driven Loop	PaperTab	UniDoc-Bench
Retrieval Scaling	Surveyor	Locator	Extractor	Verifier- Driven Loop	PaperTab	UniDoc-Bench
Evidence	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	X	65.1
$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	47.5	62.2
$\boldsymbol{\times}$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	42.4	59.2
$\checkmark$	$\boldsymbol{\times}$	$\checkmark$	$\checkmark$	$\checkmark$	43.3	59.4
$\checkmark$	$\checkmark$	$\boldsymbol{\times}$	$\checkmark$	$\checkmark$	44.3	60.2

Appendix

Table 4: Extended ablation study of different components on PaperTab and UniDoc-Bench datasets.

Hyperparameter	Default Value
Pruning threshold ( $\alpha$ )	0.7
Stability threshold ( $\tau$ )	0.8
Closure threshold ( $\delta$ )	8
Maximum decomposed queries	6
Maximum retrieval rounds	5
Maximum reasoning rounds	20

Appendix

Table 5: Default values of hyperparameters used in D2-ScaleAgent.

Document Understanding

Document understanding enables machines to interpret text, layout, tables, and images to extract information and answer questions about documents.

See all 91 papers

Hierarchical Evidence-Driven Reasoning for Long Document Understanding
Jul 6, 2026Junyu Xiong, Yonghui Wang, Rongjian Gu +4Multi-Hop QA Document Understanding
University of Science and Technology of China · iFLYTEK Research
MAGE-RAG: Multigranular Adaptive Graph Evidence for Agentic Multimodal RAG in Long-Document QA
Jun 14, 2026Yilong Zuo, Xunkai Li, Jing Yuan +3Multimodal RAG Graph Retrieval-Augmented Generation
Beijing Institute of Technology
SPD-RAG: Sub-Agent Per Document Retrieval-Augmented Generation
Mar 9, 2026Yagiz Can Akay, Muhammed Yusuf Kartal, Esra Alparslan +2Multi-Document QA Retrieval-Augmented Generation
TOBB University of Economics and Technology · OSTIM Technical University

Jul 6, 2026J/K move · Enter open · S save

Hierarchical Evidence-Driven Reasoning for Long Document Understanding

Junyu Xiong, Yonghui Wang, Rongjian Gu, Chenyu Liu+3

University of Science and Technology of China · iFLYTEK Research

Multi-Hop QA Document Understanding Multimodal RAG Question Decomposition Question Answering Visual Document Retrieval+1

Open paper AskPDF arXiv

Retrieval-Augmented Generation (RAG) streamlines long-document understanding by leveraging retrieval mechanisms to restrict input images to a highly curated subset. However, existing multimodal RAG pipelines primarily face two critical challenges: first, standard semantic similarity retrievers frequently fetch topically overlapping yet answer-void distractor pages that mislead downstream generation; second, rigid single-pass pipelines heavily depend on initial retrieval success, where any omission of core evidence inevitably causes cascading errors. To address these challenges, we introduce HIEVI-RAG, a hierarchical, evidence-driven multimodal RAG framework for closed-domain document understanding. HIEVI-RAG systematically factorizes complex queries into a cooperative four-stage pipeline: (1) hierarchical question decomposition to break multi-hop root queries into atomic child questions; (2) coarse visual page retrieval leveraging a multimodal retriever to fetch candidate pages based on semantic similarity; (3) fine-grained page verification via EVIAGENT, a specialized multi-page verifier trained with GRPO to execute cross-page reasoning over multi-image blocks; and (4) memory-guided iterative generation that leverages accumulated sub-question context to execute multi-round, dynamic reasoning over the prioritized sequence. Extensive evaluations across four benchmarks demonstrate the robust efficacy and synergy of our framework, which significantly outperforms existing open-source baselines and exceeds the strongest reported baseline by an average of 8.05% in accuracy.

Abstract

Document Understanding

Explore similar work

Hierarchical Evidence-Driven Reasoning for Long Document Understanding

MAGE-RAG: Multigranular Adaptive Graph Evidence for Agentic Multimodal RAG in Long-Document QA

SPD-RAG: Sub-Agent Per Document Retrieval-Augmented Generation