DocSkill: Skill-Integrated Agentic Reasoning for Multilingual Multi-Hop Document Question Answering
Authors: Jing Huang, Zhiya Tan, Changtao Miao, Shutao Gong, Yang Wang, Joey Tianyi Zhou, Jianshu Li, Ying Yan
Organizations: Ant Digital Technologies, Ant Group · Nanyang Technological University, Singapore · Singapore Management University, Singapore · Institute of Advanced Intelligence and Computing (IAIC), Agency for Science, Technology and Research (A*STAR), Singapore
Multilingual multi-hop document question answering (MMHop-DocQA) requires locating and integrating evidence across complex layouts, document pages, and diverse languages. Existing approaches include single-pass processing and adaptive tool use, but learning to coordinate evidence acquisition, cross-page integration, and answer verification remains challenging. To address this challenge, we present DocSkill, an agentic framework with a learned skill-orchestration policy for adaptive evidence acquisition and reasoning. The policy selects, revisits, or skips skills according to the evolving evidence state. To construct supervision for skill orchestration, bounded Monte Carlo Tree Search (MCTS) explores alternative skill trajectories offline. These trajectories provide demonstrations for supervised fine-tuning (SFT) and preference pairs for Direct Preference Optimization (DPO). We further introduce MMHop-DocBench, a multilingual multi-hop document QA dataset with training and test splits, containing 60K QA pairs across 10 languages and eight document domains. Its HopGen pipeline builds hierarchical page-element graphs to derive traceable cross-page evidence paths and generate grounded QA pairs with page- and element-level evidence annotations. Experiments show that DocSkill with Qwen3-VL-8B achieves 41.0 accuracy and 49.2 F1 on MMHop-DocBench, exceeding the strongest evaluated baseline by 5.5 and 6.3 points, respectively, and achieving competitive performance on MMLongBench-Doc and DocBench.
Figures & tables
Figure 1. Overview of DocSkill and its comparison with VLM-based and tool-augmented methods. DocSkill adaptively orchestrates four skills, Locate , Inspect , Integrate , and Verify , for cross-page document reasoning and answering. Diagram comparing VLM-based methods, tool-augmented methods, and DocSkill. DocSkill adaptively selects four document skills to retrieve relevant pages, inspect fine-grained content, integrate cross-page evidence, and verify candidate answers.
Figure 2. Overview of the DocSkill framework. During offline training, bounded MCTS explores alternative skill trajectories to construct training data for SFT and preference pairs for DPO. During inference, the learned policy adaptively orchestrates four skills, Locate , Inspect , Integrate , and Verify , each combining model reasoning with optional tool use. Framework diagram showing offline trajectory construction and inference-time skill orchestration. Bounded MCTS generates supervision for SFT and DPO. At inference, the learned policy selects document skills based on the evolving interaction state. Each skill performs model-native reasoning, optionally invokes tools, and updates the state for subsequent decisions.
Benchmark
#Langs.
M-Hop
#QA
Avg. Pages
Source
ChartQA ( Masry et al., 2022 )
1
✗
32.7K
1.0
Charts
EST-VQA ( Wang et al., 2020 )
2
✗
28.2K
1.0
Scenes
MTVQA ( Tang et al., 2025 )
9
✗
28.6K
1.0
Multi
DocVQA ( Mathew et al., 2021 )
1
✗
50.0K
1.0
Multi
MP-DocVQA ( Tito et al., 2023 )
1
✗
46.2K
8.2
Multi
SlideVQA ( Tanaka et al., 2023 )
1
✓
14.5K
20.0
Slides
Table 1. Comparison of document QA datasets by language count, multi-hop support, number of QA pairs, average document length, and source.
Figure 3. Overview of HopGen, which constructs hierarchical page–element graphs to generate and verify multilingual multi-hop QA pairs with explicit evidence references. HopGen pipeline for multilingual multi-hop QA curation. Page profiles and parsed elements form a hierarchical graph whose cross-page evidence paths guide question–answer generation. Candidates are evaluated, filtered, and deduplicated, with retained pairs linked to supporting pages and elements.
Method
EN
ZH
YUE
ID
MS
TL
TA
TH
VI
MY
Acc.
F1
Vanilla VLM Methods
Naive VLM w/ Qwen3-VL-4B ( Bai et al., 2025 )
18.0/23.2
15.2/20.3
8.3/12.0
16.9/23.6
14.9/19.6
9.0/12.9
2.0/3.3
5.4/8.0
10.3/14.0
2.9/4.2
10.3
14.1
Naive VLM w/ Qwen3-VL-8B ( Bai et al., 2025 )
24.0/28.9
21.0/25.9
12.0/15.7
20.6/28.0
19.8/23.9
13.5/16.6
3.6/5.1
8.2/10.5
14.3/18.2
4.3/5.7
14.1
17.9
Naive VLM w/ Qwen3-VL-32B ( Bai et al., 2025 )
24.2/30.0
21.2/26.1
15.2/19.9
25.5/32.7
22.5/27.8
15.8/20.8
5.2/7.6
12.6/16.1
18.2/23.1
5.6/7.5
16.6
21.2
Naive VLM w/ Gemini-3.1-Pro ( Google, 2026 )
25.0/31.3
24.3 /29.8
20.8/27.5
25.4/33.7
23.8/30.5
20.4/28.6
12.9/18.6
21.8 /28.2
22.0/29.3
6.1/11.8
20.3
26.9
Naive VLM w/ Claude-Sonnet-5 ( Anthropic, 2026 )
25.6/32.0
23.8/29.5
22.1/29.2
26.0/34.5
24.8/31.4
22.5/31.8
13.5/19.8
20.8/28.1
22.4/29.8
6.4/12.5
20.8
27.9
Table 2. Performance on MMHop-DocBench across ten languages, reported as per-language and macro-averaged Acc/ F1 . Bold values indicate the best results within each method category, while underlined bold values denote the best model results overall.
Method
MMLongBench-Doc
DocBench
Acc.
F1
LASJ
LASJ
DocAgent ( Sun et al., 2025 ) w/ GPT-4o ( OpenAI, 2024 )
51.8
49.1
–
79.9
DocAgent ( Sun et al., 2025 ) w/ Claude-3.5-Sonnet ( Anthropic, 2024 )
Table 3. Performance comparison on MMLongBench-Doc and DocBench. Underlined bold values indicate the best model results. Unreported results are marked with “–”.
Figure 4. Qualitative comparison of DocAgent and DocSkill on an Indonesian document. In this example, DocAgent returns an incorrect answer with incorrect evidence citations, whereas DocSkill produces the correct answer with supporting evidence through adaptive skill selection. Intermediate skill invocations and the terminal answer action are omitted for brevity. Diagram showing a document and reference QA above two agent trajectories. DocAgent returns incorrect strategy names and evidence citations. DocSkill displays Locate, Inspect, and Verify steps, with intermediate calls omitted, and returns the five reference strategies with supporting evidence from pages 18, 31, and 38.
Variant
Acc.
F1
Rpage
Relem
Full DocSkill
41.0
49.2
82.4
68.5
Direct Tool Selection
29.8
38.2
68.5
49.3
Fixed Skill Order
33.5
41.8
70.4
56.5
DocSkill w/o Locate
23.4
30.2
41.5
33.2
DocSkill w/o Inspect
28.2
36.1
72.8
43.5
DocSkill w/o Integrate
31.8
39.5
76.2
59.8
Table 4. Ablation of DocSkill on MMHop-DocBench using Qwen3-VL-8B (SFT + DPO). Scores are percentages. Rpage and Relem denote supporting-page and supporting-element recall.
Strategy
Succ. (%)
Seq. Div.
Pairs/Q
Acc.
F1
Best-of- N
34.5±2.4
0.42±0.03
1.2±0.1
37.8
46.5
Beam Search
47.2±1.5
0.15±0.02
0.6±0.1
38.6
47.1
MCTS (Ours)
65.8±1.8
0.7±0.02
3.0±0.2
41.0
49.2
Table 5. Equal-budget trajectory construction comparison. Succ. means the questions with a correct trajectory (%). Search metrics are mean ± SD over three runs; Acc./ F1 are MMHop-DocBench scores for DocSkill w/ Qwen3-VL-8B (SFT + DPO).
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5. Instructions for generating MCTS rollouts. Prompt instructing the MCTS generator to propose skill invocations or a terminal answer based on the current interaction state.
Figure 6. Instructions for evaluating MCTS rollouts. Prompt instructing the MCTS evaluator to assess terminal outcomes, language consistency, logical soundness, and skill-use effectiveness.
Figure 7. Instructions for HopGen QA generation. Prompt instructing the HopGen generator to produce grounded question-answer pairs in the target language from cross-page evidence, with supporting page and element IDs.
Figure 8. Representative multilingual samples from MMHop-DocBench. Each example shows selected document pages, a target-language QA pair, its English translation, and supporting page- and element-level annotations. The examples illustrate cross-page evidence composition over text, tables, and images. Multilingual document QA examples showing selected pages, target-language questions and answers, English translations, and supporting page and element annotations. The examples combine evidence across pages from text, tables, and images.
Figure 9. Instructions for HopGen QA evaluation. Prompt instructing the HopGen evaluator to assess answer correctness, evidence sufficiency, cross-page necessity, and language consistency, with a score and justification for each criterion.
Figure 10. Document and QA-pair counts by language in the training and test splits of MMHop-DocBench. Document and QA-pair counts across ten languages, shown separately for the training and test splits.
Multi-hop question answering studies how systems combine evidence across documents, facts, or modalities to answer questions requiring linked reasoning.
Multimodal long-document question answering requires models to seek sparse evidence across pages and modalities, understand heterogeneous document content, and reason over the collected evidence. Even advanced vision-language models (VLMs) can overlook sparse evidence and struggle with cross-modal reasoning in a single pass. Existing document agents further introduce multi-step interaction, but some still rely on predefined workflows or coarse-grained actions, limiting the flexibility and granularity of document reasoning. We introduce DocPilot, an agentic document framework that supports expressive orchestration and makes complex document solving learnable by small language models (SLMs). DocPilot orchestrates evidence seeking through page-level navigation and fine-grained evidence localization, and integrates local visual inspection, code-based numerical reasoning, and cross-modal reasoning for multimodal understanding. To transfer these capabilities to small models, we propose curriculum-based agentic training that progresses from single-ability learning to adaptive orchestration through supervision from data-grounded reasoning traces and executable interaction trajectories. Across MMLongBench-Doc, DocBench, FinRAGBench-V, and MADQA, DocPilot achieves strong gains with frontier VLM backbones, while DocPilot-4B delivers competitive performance against substantially larger models, demonstrating strong effectiveness and generalization across diverse long-document reasoning settings. The code is available at https://github.com/HKUSTDial/DocPilot.
Zhuowen Liang, Xiaotian Lin, Yuyu Luo +2
The Hong Kong University of Science and Technology (Guangzhou) · EvenUp, USA
Multimodal Large Language Models (MLLMs) have achieved strong performance on structured visual understanding tasks such as chart and document question answering. However, existing benchmarks typically evaluate these domains in isolation, leaving underexplored a key capability: whether models can use textual context to determine how chart evidence should be selected, interpreted, and aggregated. We introduce DocHop, a benchmark for integrated chart--context reasoning in document-style images. In DocHop, the document narrative specifies multi-step compositional constraints, while charts provide the corresponding data values. Questions are grounded on a semantic reference label defined in the narrative, requiring models to resolve target entities from context before aggregating evidence across multiple charts. To enable systematic evaluation, we construct DocHop via a stochastic logic-first generation pipeline with controllable reasoning depth and visual density, covering 2,074 examples across six task categories. Experiments on a wide range of proprietary and open-source MLLMs show a substantial gap to human performance: annotators achieve over 90% accuracy, while the best model reaches only 62.83%. Reasoning-enhanced models consistently show improved results, but performance degrades as reasoning complexity increases. Overall, DocHop provides a controlled testbed for challenging multi-hop document reasoning.
Zhuoran Yu, Le Thien Phuc Nguyen, Jaden Park +5
University of Wisconsin-Madison · Massachusetts Institute of Technology · Stanford University +2
Retrieval-augmented generation (RAG) systems have made significant progress in solving complex multi-hop question answering (QA) tasks in the English scenario. However, RAG systems inevitably face the application scenario of retrieving across multilingual corpora and queries, leaving several open challenges. The first one involves the absence of benchmarks that assess RAG systems' capabilities under the multilingual multi-hop (MM-hop) QA setting. The second centers on the overreliance on LLMs' strong semantic understanding in English, which diminishes effectiveness in multilingual scenarios. To address these challenges, we first construct multilingual multi-hop QA benchmarks by translating English-only benchmarks into five languages, and then we propose DaPT, a novel multilingual RAG framework. DaPT generates sub-question graphs in parallel for both the source-language query and its English translation counterpart, then merges them before employing a bilingual retrieval-and-answer strategy to sequentially solve sub-questions. Our experimental results demonstrate that advanced RAG systems suffer from a significant performance imbalance in multilingual scenarios. Furthermore, our proposed method consistently yields more accurate and concise answers compared to the baselines, significantly enhancing RAG performance on this task. For instance, on the most challenging MuSiQue benchmark, DaPT achieves a relative improvement of 18.3% in average EM score over the strongest baseline.
Yilin Wang, Yuchun Fan, Jiaoyang Li +5
School of Computer Science and Engineering, Northeastern University, Shenyang, China · NiuTrans Research, Shenyang, China