DuQUAD: A Dual-View Framework with Quality-Aware Evidence Pruning for Multi-Document Question Answering

Yeonjun Choi; Seyeon Jeong; Jong Wook Kim; Beakcheol Jang

cs.IROct 25, 2026

DuQUAD: A Dual-View Framework with Quality-Aware Evidence Pruning for Multi-Document Question Answering

Authors: Yeonjun Choi, Seyeon Jeong, Jong Wook Kim, Beakcheol Jang

Organizations: Graduate School of Information, Yonsei University, Seoul, South Korea · Department of Computer Science, Sangmyung University, Seoul, South Korea

Abstract

Large language models with retrieval-augmented generation systematically underutilize evidence in the middle of multi-document contexts, a phenomenon known as Lost in the Middle. Existing mitigation strategies, such as document reordering and attention steering, can reduce boundary bias but remain brittle in noisy settings, where inaccurate relevance estimation and spurious documents still obscure useful evidence in central positions. We propose DuQUAD, a multi-document QA framework that mitigates positional bias through dual-view reasoning and quality-aware evidence pruning. DuQUAD combines a Local Agent guided by a Structural Fusion Score with a Global Agent that reasons over the full document set, providing both targeted evidence recovery and broad contextual understanding. The framework further filters candidate evidence using explicit quality scoring and refines it at the sentence level to reduce noise. DuQUAD consistently outperforms strong Lost-in-the-Middle mitigation baselines, including hybrid retrieval and reranking, multi-agent reasoning, and context optimization methods, improving accuracy by 3.0% on average over the strongest competing baseline and gold evidence recall by 9.4%, while substantially reducing computational cost compared to existing multi-agent frameworks.

Figures & tables

Figure 1: Attention z-score distribution across document positions on the Natural Questions (NQ) benchmark (NQ-1 to NQ-20, where NQ-

n

denotes instances in which the gold document appears at the

n

-th position).

Figure 2: Overview of a dual-view multi-agent reasoning framework based on quality-aware evidence pruning. Given a question and a set of retrieved documents, the Structural Fusion Score (SFS) prioritizes a subset for the Local Agent, while the Global Agent operates on the full context. Selected documents are filtered via LLM-based DocScore, and key sentences are extracted. Finally, the Judge Agent aggregates reasoning histories and extracted evidence to produce the final answer.

Model	Method	NQ-1	NQ-5	NQ-10	NQ-15	NQ-20	MuSiQue			HotpotQA
Model	Method	Acc.	Acc.	Acc.	Acc.	Acc.	Acc.	EM	F1	Acc.	EM	F1
Llama-3.1-8B -Instruct	Vanilla LLM	73.5	70.5	65.6	67.0	64.0	33.1	19.9	27.3	73.8	52.4	68.4
	SBERT	70.5	70.3	70.4	70.1	70.3	24.9	18.7	27.0	76.8	52.2	68.0
	BM25	70.4	69.9	69.8	69.2	69.7	33.6	17.7	26.5	75.3	50.8	67.8
	Jina-Reranker	69.0	68.4	69.2	69.4	68.6	29.3	17.6	27.0	77.3	51.6	67.4
	Qwen3-Emb.	69.3	69.3	69.3	69.3	69.3	34.7	18.0	27.0	76.8	50.3	67.2

Table 1: Main experimental results comparing Llama-3.1-8B-Instruct and Qwen2.5-7B-Instruct. The highest score in each column is shown in bold , and the second-highest score is underlined .

Figure 3: Impact of Score Fusion on Robustness. Sim, PPL, and Cont denote the three SFS signals: Semantic Relevance, Structural Inference, and Attention Contrast, respectively. (a) Recall of the gold document by gold position under the Top-6 setting. (b) Overall recall aggregated across all gold positions in (a). Computed on 100 NQ examples per gold position.

Figure 4: Impact of representative local document selection scores within DuQUAD on NQ-15. We compare sparse and dense retrieval-based alternatives against the proposed Structural Fusion Score (SFS) in terms of answer accuracy.

Method	NQ-15	MuSiQue	HotpotQA
w/o Local Agent	69.8	38.1	79.1
w/o Global Agent	71.1	36.1	80.0
w/o Quality Pruning	71.3	34.6	79.8
DuQUAD	76.1	41.1	80.2

Table 2: Effect of the Local Agent, Global Agent, and Quality-Aware Pruning. Ablation analysis across three datasets evaluating the impact on accuracy when omitting each component compared to the complete DuQUAD method.

Figure 5: Accuracy–latency trade-off of representative baselines on NQ-15. The inset magnifies the low-latency region for clearer comparison among retrieval, reranking, and context optimization methods.

Experiment	Dataset	Samples	Reference
W-shape Visualization	NQ	$100\times 5$	Figure 1
Main Experiment	NQ	$1{,}000\times 5$	Table 1
	MuSiQue	1,000
	HotpotQA	1,000
30-Document Robustness	NQ-15	1,000	Table 8
Gold Recall Analysis	NQ	$100\times 5$	Figure 3

Table 3: Summary of experimental settings and datasets. For NQ experiments evaluated over multiple gold positions, the sample count is shown as (examples per position)

\times

(number of positions); the five positions are

n\in\{1,5,10,15,20\}

Dataset	Samples	Docs	Gold Docs	Question Type	Input Tokens
Dataset	Samples	Docs	Gold Docs	Question Type	Mean	Std.	Max
NQ	1,000	20	1	Single-hop factoid	2,781	140	3,396
MuSiQue	1,000	20	2–4	2–4 hop	2,446	495	4,751
HotpotQA	1,000	10	2	2-hop	1,351	349	3,220
NQ (30 docs)	1,000	30	1	Single-hop factoid	4,239	205	5,474

Table 4: Dataset characteristics and input context statistics. Input lengths include all documents and the question and are measured using the Llama-3.1-8B-Instruct tokenizer. For NQ, each positional split contains 1,000 examples; the 30-document setting is an additional robustness evaluation with the gold document at position 15.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Threshold ( $\tau$ )	Recall (%)
0.2	83.57
0.3	80.70
0.4	80.70
0.5	79.38
0.6	79.02
0.7	75.06

Appendix

Table 5: Sensitivity of answer-document recall to the DocScore threshold

\tau

(operating range). We report

\tau\in[0.2,0.8]

as extremely low thresholds behave similarly (near-pass-through), while overly high thresholds enter a degenerate pruning regime.

Figure 6: Impact of the number of documents provided to the Local Agent on Accuracy and Recall@k.

$\Phi_{\mathrm{sim}}$	$\Phi_{\mathrm{ppl}}$	$\Phi_{\mathrm{cont}}$	Avg. Recall (%)
0.5	1.0	0.5	91.40
0.3	0.5	0.3	91.20
0.3	0.7	0.3	91.20
0.3	0.7	1.0	91.20
0.3	0.5	0.7	91.00
0.3	0.5	1.0	91.00

Appendix

Table 6: Grid search results for Structural Fusion Score (SFS) weights. Average gold evidence recall is reported across all evaluated answer positions. The selected configuration

(\Phi_{\mathrm{sim}},\Phi_{\mathrm{ppl}},\Phi_{\mathrm{cont}})=(0.5,1.0,0.5)

achieves the highest overall recall.

Dataset (DuQUAD Acc.)	Baseline	Acc. (%)	Diff. (95% CI)	$p$ -value
NQ-15 (76.1)	Vanilla LLM	67.0	+9.1 (6.5 – 11.9)	$<$ 0.0001***
	SBERT	70.1	+6.0 (3.3 – 8.8)	$<$ 0.0001***
	BM25	69.2	+6.9 (4.1 – 9.7)	$<$ 0.0001***
	Jina-Reranker	69.4	+6.7 (4.1 – 9.4)	$<$ 0.0001***
	Qwen3-Embedding	69.3	+6.8 (4.0 – 9.6)	$<$ 0.0001***
	OpenAI	69.9	+6.2 (3.4 – 9.0)	$<$ 0.0001***

Appendix

Table 7: Performance comparison of Accuracy (%). Values in parentheses represent the accuracy of DuQUAD. Mean differences and 95% confidence intervals (CI) are derived from paired bootstrap resampling (

N=10,000

). Asterisks indicate statistical significance: *

p<0.05

, **

p<0.01

, ***

p<0.001

Method	20 Docs	30 Docs	Diff.
Vanilla LLM	67.0	60.8	$-6.2$
OpenAI	69.9	66.1	$-3.8$
Hybrid (RRF)	72.2	64.8	$-7.4$
Hybrid (RRF) + Rerank	71.4	67.8	$-3.6$
FiM	64.7	62.7	$-2.0$
DuQUAD	76.1	75.6	$-0.5$

Appendix

Table 8: Accuracy on NQ-15 under the 20- and 30-document settings. Diff. denotes the accuracy change when increasing the number of documents from 20 to 30.

Method	LLM Calls		Tokens		E2E Latency (s)
	Value	Ratio	Value	Ratio	Value	Ratio
Vanilla LLM	1.00	1.00 $\times$	2863.50	1.00 $\times$	2.12	1.00 $\times$
OpenAI	1.00	1.00 $\times$	2900.31	1.01 $\times$	4.38	2.07 $\times$
Qwen3-Embedding	1.00	1.00 $\times$	2940.39	1.03 $\times$	3.70	1.75 $\times$
Jina-Reranker	1.00	1.00 $\times$	2940.53	1.03 $\times$	3.98	1.88 $\times$
Hybrid (RRF)	1.00	1.00 $\times$	2939.80	1.03 $\times$	3.67	1.73 $\times$

Appendix

Table 9: Cost comparison normalized to Vanilla LLM in terms of LLM calls, tokens, and end-to-end latency.

Dataset	Both Correct	Global Only	Local Only	Both Wrong	Disagreement	Judge Acc.
Dataset	Both Correct	Global Only	Local Only	Both Wrong	Rate	on Disagreement
NQ	579	131	112	178	24.3%	74.5% (181/243)
HotpotQA	746	72	63	119	13.5%	58.5% (79/135)
MuSiQue	291	103	83	523	18.6%	67.2% (125/186)

Appendix

Table 10: Local–Global agent agreement and disagreement statistics. Each dataset contains 1,000 evaluation examples. Disagreement denotes cases in which exactly one agent is correct, and Judge accuracy is measured only over these disagreement cases.

Model	Document Type	DS Mean	ID Avg.
Llama	Gold Doc	0.647	85.7
Llama	Non-Gold Doc	0.324	14.3
Qwen	Gold Doc	0.722	83.0
Qwen	Non-Gold Doc	0.348	17.0

Appendix

Table 11: DocScore (DS) and Important Document (ID) statistics on Natural Questions (NQ-1/5/10/15/20). DS Mean denotes the average DocScore assigned to documents of each type, while ID Avg. indicates the probability that a document of the corresponding type is selected as an Important Document.

Metric	Value
Gold evidence stability	90.4%
Distractor variability	90.1%

Appendix

Table 12: Cross-model evidence selection stability on Natural Questions (NQ-1/5/10/15/20).

Figure 7: Prompt template for the Local and Global Agents.

Figure 8: Prompt template for the Judge Agent.

Figure 9: Prompt template for the Score Agent.

Multi-Document QA

See all 22 papers

HiQA: A Hierarchical Contextual Augmentation RAG for Multi-Documents QA
Feb 1, 2024Xinyue Chen, Pengyu Gao, Jiangjiang Song +1Retrieval-Augmented Generation Multi-Document QA
1Nanjing University of Aeronautics and Astronautics. · 2Southeast University, Nanjing, Jiangsu, China.
MARDoc: A Memory-Aware Refinement Agent Framework for Multimodal Long Document QA
Jun 4, 2026Kaifeng Chen, Hongtao Liu, Qiyao Peng +4Multimodal RAG Multimodal QA
Tianjin University, Tianjin, China · Qifu Technology, Beijing, China · Beihang University, Beijing, China +1
MAB-DQA: Addressing Query Aspect Importance in Document Question Answering with Multi-Armed Bandits
Apr 10, 2026Yixin Xiang, Yunshan Ma, Xiaoyu Du +3Document QA Multimodal RAG
Nanjing University of Science and Technology · Singapore Management University · Nanjing Pami Intelligent Technology Co., Ltd. +2

Feb 1, 2024J/K move · Enter open · S save

HiQA: A Hierarchical Contextual Augmentation RAG for Multi-Documents QA

Xinyue Chen, Pengyu Gao, Jiangjiang Song, Xiaoyang Tan

1Nanjing University of Aeronautics and Astronautics. · 2Southeast University, Nanjing, Jiangsu, China.

Retrieval-Augmented Generation Multi-Document QA Document QA Information Retrieval RAG Evaluation

Open paper AskPDF arXiv

Retrieval-augmented generation (RAG) has rapidly advanced the language model field, particularly in question-answering (QA) systems. By integrating external documents during the response generation phase, RAG significantly enhances the accuracy and reliability of language models. This method elevates the quality of responses and reduces the frequency of hallucinations, where the model generates incorrect or misleading information. However, these methods exhibit limited retrieval accuracy when faced with numerous indistinguishable documents, presenting notable challenges in their practical application. In response to these emerging challenges, we present HiQA, an advanced multi-document question-answering (MDQA) framework that integrates cascading metadata into content and a multi-route retrieval mechanism. We also release a benchmark called MasQA to evaluate and research in MDQA. Finally, HiQA demonstrates the state-of-the-art performance in multi-document environments.

Abstract

Multi-Document QA

Explore similar work

HiQA: A Hierarchical Contextual Augmentation RAG for Multi-Documents QA

MARDoc: A Memory-Aware Refinement Agent Framework for Multimodal Long Document QA

MAB-DQA: Addressing Query Aspect Importance in Document Question Answering with Multi-Armed Bandits