DuQUAD: A Dual-View Framework with Quality-Aware Evidence Pruning for Multi-Document Question Answering
Authors: Yeonjun Choi, Seyeon Jeong, Jong Wook Kim, Beakcheol Jang
Organizations: Graduate School of Information, Yonsei University, Seoul, South Korea · Department of Computer Science, Sangmyung University, Seoul, South Korea
Large language models with retrieval-augmented generation systematically underutilize evidence in the middle of multi-document contexts, a phenomenon known as Lost in the Middle. Existing mitigation strategies, such as document reordering and attention steering, can reduce boundary bias but remain brittle in noisy settings, where inaccurate relevance estimation and spurious documents still obscure useful evidence in central positions. We propose DuQUAD, a multi-document QA framework that mitigates positional bias through dual-view reasoning and quality-aware evidence pruning. DuQUAD combines a Local Agent guided by a Structural Fusion Score with a Global Agent that reasons over the full document set, providing both targeted evidence recovery and broad contextual understanding. The framework further filters candidate evidence using explicit quality scoring and refines it at the sentence level to reduce noise. DuQUAD consistently outperforms strong Lost-in-the-Middle mitigation baselines, including hybrid retrieval and reranking, multi-agent reasoning, and context optimization methods, improving accuracy by 3.0% on average over the strongest competing baseline and gold evidence recall by 9.4%, while substantially reducing computational cost compared to existing multi-agent frameworks.
Figures & tables
Figure 1: Attention z-score distribution across document positions on the Natural Questions (NQ) benchmark (NQ-1 to NQ-20, where NQ- n denotes instances in which the gold document appears at the n -th position).
Figure 2: Overview of a dual-view multi-agent reasoning framework based on quality-aware evidence pruning. Given a question and a set of retrieved documents, the Structural Fusion Score (SFS) prioritizes a subset for the Local Agent, while the Global Agent operates on the full context. Selected documents are filtered via LLM-based DocScore, and key sentences are extracted. Finally, the Judge Agent aggregates reasoning histories and extracted evidence to produce the final answer.
Model
Method
NQ-1
NQ-5
NQ-10
NQ-15
NQ-20
MuSiQue
HotpotQA
Acc.
Acc.
Acc.
Acc.
Acc.
Acc.
EM
F1
Acc.
EM
F1
Llama-3.1-8B -Instruct
Vanilla LLM
73.5
70.5
65.6
67.0
64.0
33.1
19.9
27.3
73.8
52.4
68.4
SBERT
70.5
70.3
70.4
70.1
70.3
24.9
18.7
27.0
76.8
52.2
68.0
BM25
70.4
69.9
69.8
69.2
69.7
33.6
17.7
26.5
75.3
50.8
67.8
Jina-Reranker
69.0
68.4
69.2
69.4
68.6
29.3
17.6
27.0
77.3
51.6
67.4
Qwen3-Emb.
69.3
69.3
69.3
69.3
69.3
34.7
18.0
27.0
76.8
50.3
67.2
Table 1: Main experimental results comparing Llama-3.1-8B-Instruct and Qwen2.5-7B-Instruct. The highest score in each column is shown in bold , and the second-highest score is underlined .
Figure 3: Impact of Score Fusion on Robustness. Sim, PPL, and Cont denote the three SFS signals: Semantic Relevance, Structural Inference, and Attention Contrast, respectively. (a) Recall of the gold document by gold position under the Top-6 setting. (b) Overall recall aggregated across all gold positions in (a). Computed on 100 NQ examples per gold position.
Figure 4: Impact of representative local document selection scores within DuQUAD on NQ-15. We compare sparse and dense retrieval-based alternatives against the proposed Structural Fusion Score (SFS) in terms of answer accuracy.
Method
NQ-15
MuSiQue
HotpotQA
w/o Local Agent
69.8
38.1
79.1
w/o Global Agent
71.1
36.1
80.0
w/o Quality Pruning
71.3
34.6
79.8
DuQUAD
76.1
41.1
80.2
Table 2: Effect of the Local Agent, Global Agent, and Quality-Aware Pruning. Ablation analysis across three datasets evaluating the impact on accuracy when omitting each component compared to the complete DuQUAD method.
Figure 5: Accuracy–latency trade-off of representative baselines on NQ-15. The inset magnifies the low-latency region for clearer comparison among retrieval, reranking, and context optimization methods.
Experiment
Dataset
Samples
Reference
W-shape Visualization
NQ
100×5
Figure 1
Main Experiment
NQ
1,000×5
Table 1
MuSiQue
1,000
HotpotQA
1,000
30-Document Robustness
NQ-15
1,000
Table 8
Gold Recall Analysis
NQ
100×5
Figure 3
Table 3: Summary of experimental settings and datasets. For NQ experiments evaluated over multiple gold positions, the sample count is shown as (examples per position) × (number of positions); the five positions are n∈{1,5,10,15,20} .
Dataset
Samples
Docs
Gold Docs
Question Type
Input Tokens
Mean
Std.
Max
NQ
1,000
20
1
Single-hop factoid
2,781
140
3,396
MuSiQue
1,000
20
2–4
2–4 hop
2,446
495
4,751
HotpotQA
1,000
10
2
2-hop
1,351
349
3,220
NQ (30 docs)
1,000
30
1
Single-hop factoid
4,239
205
5,474
Table 4: Dataset characteristics and input context statistics. Input lengths include all documents and the question and are measured using the Llama-3.1-8B-Instruct tokenizer. For NQ, each positional split contains 1,000 examples; the 30-document setting is an additional robustness evaluation with the gold document at position 15.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Threshold ( τ )
Recall (%)
0.2
83.57
0.3
80.70
0.4
80.70
0.5
79.38
0.6
79.02
0.7
75.06
Appendix
Table 5: Sensitivity of answer-document recall to the DocScore threshold τ (operating range). We report τ∈[0.2,0.8] as extremely low thresholds behave similarly (near-pass-through), while overly high thresholds enter a degenerate pruning regime.
Figure 6: Impact of the number of documents provided to the Local Agent on Accuracy and Recall@k.
Φsim
Φppl
Φcont
Avg. Recall (%)
0.5
1.0
0.5
91.40
0.3
0.5
0.3
91.20
0.3
0.7
0.3
91.20
0.3
0.7
1.0
91.20
0.3
0.5
0.7
91.00
0.3
0.5
1.0
91.00
Appendix
Table 6: Grid search results for Structural Fusion Score (SFS) weights. Average gold evidence recall is reported across all evaluated answer positions. The selected configuration (Φsim,Φppl,Φcont)=(0.5,1.0,0.5) achieves the highest overall recall.
Dataset (DuQUAD Acc.)
Baseline
Acc. (%)
Diff. (95% CI)
p -value
NQ-15 (76.1)
Vanilla LLM
67.0
+9.1 (6.5 – 11.9)
< 0.0001***
SBERT
70.1
+6.0 (3.3 – 8.8)
< 0.0001***
BM25
69.2
+6.9 (4.1 – 9.7)
< 0.0001***
Jina-Reranker
69.4
+6.7 (4.1 – 9.4)
< 0.0001***
Qwen3-Embedding
69.3
+6.8 (4.0 – 9.6)
< 0.0001***
OpenAI
69.9
+6.2 (3.4 – 9.0)
< 0.0001***
Appendix
Table 7: Performance comparison of Accuracy (%). Values in parentheses represent the accuracy of DuQUAD. Mean differences and 95% confidence intervals (CI) are derived from paired bootstrap resampling ( N=10,000 ). Asterisks indicate statistical significance: * p<0.05 , ** p<0.01 , *** p<0.001 .
Method
20 Docs
30 Docs
Diff.
Vanilla LLM
67.0
60.8
−6.2
OpenAI
69.9
66.1
−3.8
Hybrid (RRF)
72.2
64.8
−7.4
Hybrid (RRF) + Rerank
71.4
67.8
−3.6
FiM
64.7
62.7
−2.0
DuQUAD
76.1
75.6
−0.5
Appendix
Table 8: Accuracy on NQ-15 under the 20- and 30-document settings. Diff. denotes the accuracy change when increasing the number of documents from 20 to 30.
Method
LLM Calls
Tokens
E2E Latency (s)
Value
Ratio
Value
Ratio
Value
Ratio
Vanilla LLM
1.00
1.00 ×
2863.50
1.00 ×
2.12
1.00 ×
OpenAI
1.00
1.00 ×
2900.31
1.01 ×
4.38
2.07 ×
Qwen3-Embedding
1.00
1.00 ×
2940.39
1.03 ×
3.70
1.75 ×
Jina-Reranker
1.00
1.00 ×
2940.53
1.03 ×
3.98
1.88 ×
Hybrid (RRF)
1.00
1.00 ×
2939.80
1.03 ×
3.67
1.73 ×
Appendix
Table 9: Cost comparison normalized to Vanilla LLM in terms of LLM calls, tokens, and end-to-end latency.
Dataset
Both Correct
Global Only
Local Only
Both Wrong
Disagreement
Judge Acc.
Rate
on Disagreement
NQ
579
131
112
178
24.3%
74.5% (181/243)
HotpotQA
746
72
63
119
13.5%
58.5% (79/135)
MuSiQue
291
103
83
523
18.6%
67.2% (125/186)
Appendix
Table 10: Local–Global agent agreement and disagreement statistics. Each dataset contains 1,000 evaluation examples. Disagreement denotes cases in which exactly one agent is correct, and Judge accuracy is measured only over these disagreement cases.
Model
Document Type
DS Mean
ID Avg.
Llama
Gold Doc
0.647
85.7
Non-Gold Doc
0.324
14.3
Qwen
Gold Doc
0.722
83.0
Non-Gold Doc
0.348
17.0
Appendix
Table 11: DocScore (DS) and Important Document (ID) statistics on Natural Questions (NQ-1/5/10/15/20). DS Mean denotes the average DocScore assigned to documents of each type, while ID Avg. indicates the probability that a document of the corresponding type is selected as an Important Document.
Metric
Value
Gold evidence stability
90.4%
Distractor variability
90.1%
Appendix
Table 12: Cross-model evidence selection stability on Natural Questions (NQ-1/5/10/15/20).
Figure 7: Prompt template for the Local and Global Agents.
Retrieval-augmented generation (RAG) has rapidly advanced the language model field, particularly in question-answering (QA) systems. By integrating external documents during the response generation phase, RAG significantly enhances the accuracy and reliability of language models. This method elevates the quality of responses and reduces the frequency of hallucinations, where the model generates incorrect or misleading information. However, these methods exhibit limited retrieval accuracy when faced with numerous indistinguishable documents, presenting notable challenges in their practical application. In response to these emerging challenges, we present HiQA, an advanced multi-document question-answering (MDQA) framework that integrates cascading metadata into content and a multi-route retrieval mechanism. We also release a benchmark called MasQA to evaluate and research in MDQA. Finally, HiQA demonstrates the state-of-the-art performance in multi-document environments.
Xinyue Chen, Pengyu Gao, Jiangjiang Song +1
1Nanjing University of Aeronautics and Astronautics. · 2Southeast University, Nanjing, Jiangsu, China.
Iterative retrieval-reasoning agents have recently shown promise for multimodal long-document question answering. However, most existing systems maintain a single growing context that mixes retrieval traces, observations, and intermediate reasoning. As interactions accumulate, key evidence becomes scattered and diluted, making multi-hop reasoning noisy. We propose MARDoc, a Memory-Aware Refinement Agent framework that decouples long-document QA into three specialized agents: an Explorer for multi-granularity multimodal retrieval, a Refiner for distilling interaction traces into structured evidence and reasoning memories, and a Reflector for checking evidence sufficiency and providing targeted feedback. Across iterations, the agents rely on a dynamically updated structured memory rather than a full accumulated interaction history. This design reduces context noise while preserving answer-critical facts and their logical dependencies. Experiments on MMLongBench-Doc and DocBench show that MARDoc achieves strong results, outperforming same-backbone baselines and demonstrating the effectiveness of structured memory for agentic document QA.
Kaifeng Chen, Hongtao Liu, Qiyao Peng +4
Tianjin University, Tianjin, China · Qifu Technology, Beijing, China · Beihang University, Beijing, China +1
Document Question Answering (DQA) involves generating answers from a document based on a user's query, representing a key task in document understanding. This task requires interpreting visual layouts, which has prompted recent studies to adopt multimodal Retrieval-Augmented Generation (RAG) that processes page images for answer generation. However, in multimodal RAG, visual DQA struggles to utilize a large number of images effectively, as the retrieval stage often retains only a few candidate pages (e.g., Top-4), causing informative but less visually salient content to be overlooked in favor of common yet low-information pages. To address this issue, we propose a Multi-Armed Bandit-based DQA framework (MAB-DQA) to explicitly model the varying importance of multiple implicit aspects in a query. Specifically, MAB-DQA decomposes a query into aspect-aware subqueries and retrieves an aspect-specific candidate set for each. It treats each subquery as an arm and uses preliminary reasoning results from a small number of representative pages as reward signals to estimate aspect utility. Guided by an exploration-exploitation policy, MAB-DQA dynamically reallocates retrieval budgets toward high-value aspects. With the most informative pages and their correlations, MAB-DQA generates the expected results. On four benchmarks, MAB-DQA shows an average improvement of 5%-18% over the state-of-the-art method, consistently enhancing document understanding. Codes are available at https://github.com/ElephantOH/MAB-DQA.
Yixin Xiang, Yunshan Ma, Xiaoyu Du +3
Nanjing University of Science and Technology · Singapore Management University · Nanjing Pami Intelligent Technology Co., Ltd. +2