DeepWeaver: Bridging the Evidence Synthesis Gap in Open-Ended Question Answering

Xujia Wang; Yizhe Zhang; Bin Xu; Lei Hou; Juanzi Li

cs.CLAug 19, 2026

DeepWeaver: Bridging the Evidence Synthesis Gap in Open-Ended Question Answering

Authors: Xujia Wang, Yizhe Zhang, Bin Xu, Lei Hou, Juanzi Li

Organizations: Department of Computer Science and Technology, Tsinghua University

Abstract

Retrieve-then-generate pipelines are commonly used to produce deep-research answers for open-ended questions, but retrieval alone is insufficient: LLMs must organize noisy and fragmented evidence into comprehensive, well-cited answers. We refer to this process as evidence synthesis. However, direct generation often underuses evidence, misaligns citations, and collapses diverse information into shallow summaries, exposing an evidence synthesis gap between retrieval and generation. Thus, we propose DeepWeaver, a novel framework that weaves noisy retrieved evidence into comprehensive answers by maintaining Thought Block Chains (TBCs), a structured representation that groups claims, salient information, keywords, and supporting evidence. DeepWeaver uses subordinate TBCs to inspect residual evidence, commit TBC revisions, and discover new claims before final generation. We evaluate DeepWeaver on open-ended QA over both knowledge bases and the web, and introduce LoQA, a high-density benchmark for evidence synthesis. Across multiple LLMs, DeepWeaver improves content sufficiency, citation grounding, and detail preservation on LoQA, while achieving deeper insights and higher citation quality on DeepResearch Bench. These results show that evidence weaving is an effective mechanism for bridging retrieval and generation in open-ended QA. Our code is available at https://github.com/KlozeWang/DeepWeaver.

Figures & tables

Figure 1: DeepWeaver bridges the evidence synthesis gap between retrieval and generation by weaving noisy evidence into structured Thought Block Chains (TBCs). Unlike naïve retrieval-augmented generation, DeepWeaver synthesizes noisy evidence fragments into answers with well-supported woven claims, fine-grained details, and grounded citations.

Figure 2: Overview of DeepWeaver. (A) We apply DeepWeaver to two typical retrieve-then-generate scenarios. (B) DeepWeaver maintains and refines the TBC through three stages: Draft, Subordinate, and Commit. (C) A concrete example of commitment shows how DeepWeaver synthesizes new claims from residual evidence by TBC weaving.

Figure 3: Effect of refinement rounds on final performance. Increasing the number of refinement rounds from

n=1

n=2

improves answer comprehensiveness, while a third round brings no clear additional gain.

n=2

provides the best cost–performance trade-off.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Statistic	Value
# Questions	100
Avg. evidence tokens / question	206,550.41
Avg. relevant chunks / question	91.79
Avg. scoring points / question	20.45
Avg. cloze blanks / question	47.79
Avg. reference-answer tokens / question	9,858.64

Appendix

Table 6: Statistics of LoQA. Each question is paired with a large, noisy evidence pool of approximately 200K tokens, enabling fine-grained evaluation of evidence synthesis quality.

Figure 4: Overview of LoQA construction and evaluation. LoQA is built from a water-environment expert knowledge base through data preparation, professional question generation and filtering, and evidence-fragment collection. For evaluation, the evidence pool is split into relevant and irrelevant fragments, multiple reference answers are generated from different evidence partitions, and atomic arguments and cloze blanks are constructed to measure citation grounding, content sufficiency, and detail preservation.

Benchmark	Task Focus	Evidence Setting	Main Difference from LoQA
ALCE ( Gao et al., 2023 )	Citation-grounded text generation	Retrieval corpora built from ASQA, QAMPARI, and ELI5	Focuses on fluency, correctness, and citation quality. Its tasks primarily involve producing short answers or entity lists from retrieved passages, whereas LoQA targets comprehensive answers grounded in knowledge-intensive evidence pools of approximately 200K tokens.
FreshWiki ( Shao et al., 2024 )	Wikipedia-like article generation from scratch	Web sources collected during the pre-writing stage	Evaluates whether systems can research a topic, construct an outline, and generate Wikipedia-like articles. In contrast, LoQA uses a noisy evidence pool and emphasizes comprehensive evidence synthesis in question answering, rather than outline construction or survey writing.
LongBench ( Bai et al., 2024 )	Long-context understanding	Single-document QA, multi-document QA, summarization, few-shot learning, synthetic and code tasks	Assesses general long-context capability, but most tasks require short-form outputs or automatically verifiable answers rather than open-ended long-form answers.
L-Eval ( An et al., 2024 )	Long-context model evaluation	Long documents across diverse domains and task types	Covers long-context QA and generation, but many tasks focus on short factual verification, extractive QA, or summarization rather than comprehensive open-ended answering.
InfiniteBench ( Zhang et al., 2024a )	Extremely long-context processing	100K+ token contexts across retrieval, code, math, novels, and dialogue	Stresses context length and reasoning capacity, but its targets are mainly verifiable short answers instead of long-form citation-grounded answers.
Loong ( Wang et al., 2024b )	Extended multi-document QA	Multiple relevant documents where missing any document may hurt the answer	Emphasizes reasoning over all relevant documents, while LoQA further introduces noisy irrelevant evidence and evaluates citation grounding, content sufficiency, and detail preservation.

Appendix

Table 7: Comparison with prior benchmarks. Unlike benchmarks that emphasize short-form answers, survey-style article generation, general long-context understanding, or end-to-end web research, LoQA isolates evidence synthesis quality for open-ended QA by pairing each question with a fixed, extensive, and noisy evidence pool and evaluating citation grounding, content sufficiency, and detail preservation.

Figure 5: Overview of DeepWeaver’s Extension to Web-Based Deep-Research Scenarios

Statistic	Value
Avg. first-level outline node $\|O_{1}\|$ / question	4.41
Avg. routed outline node / question	2.35
Avg. routed-item ratio / question	59.39%
Avg. evidence fragments / item	88.92
Avg. evidence fragments / routed item	98.00
Avg. raw tokens of evidence / routed item	92,752.79

Appendix

Table 8: Routing statistics of DeepWeaver on DeepResearch Bench. Each question contains 4.41 first-level outline nodes on average, among which 2.35 nodes are routed to DeepWeaver, accounting for 59.39% of the items. Routed items are evidence-heavy, containing 98 chunks and 92.75K raw tokens on average. In LoQA, the evidence length is much larger, with about 200K tokens per question, indicating that DeepWeaver is sufficient to handle evidence-context lengths in realistic applications, such as multi-webpage reading.

Figure 6: Token usage of DeepWeaver (excluding the searcher) across stages. We report token consumption on DeepResearch Bench. POLISH, INTRO, and OUTRO denote polishing, introduction generation, and conclusion generation, respectively (§ A.2 ). Although the Subordinate stage is the main additional cost, its output tokens are only slightly higher than the Summary output, and its total tokens are comparable to the Draft stage , showing that TBC-based evidence weaving is lightweight beyond evidence input and answer generation.

Evidence-Grounded Generation

Evidence-grounded generation produces answers and documents whose claims are supported by traceable sources.

See all 148 papers

ProvenAI: Provenance-Native Traces of Evidence in Generated Answers
Jun 24, 2026Mohammad Faizan, Dalal AlharthiMulti-Hop QA Data Provenance
College of Information Science University of Arizona
EnSI-RAG: Entity-Structure-Indexed Retrieval-Augmented Generation for Long-Document Question Answering
Aug 21, 2026Xuanyu Meng, Jiashuo Sun, Jash Rajesh Parekh +1Document QA Retrieval-Augmented Generation
University of Illinois Urbana-Champaign Urbana, Illinois, USA
CAGE: Cognitive Attribution Graphs for Faithful Inline Citation Generation in Long-Form Question Answering
Jul 27, 2026Zhichao Yan, Shizhao Li, Jiapu Wang +5Retrieval-Augmented Generation Question Answering
Shanxi University, Taiyuan, China · Nanjing University of Science and Technology, Nanjing, China · Nanyang Technological University, Singapore +3

Jun 24, 2026J/K move · Enter open · S save

ProvenAI: Provenance-Native Traces of Evidence in Generated Answers

Mohammad Faizan, Dalal Alharthi

College of Information Science University of Arizona

Multi-Hop QA Data Provenance Citation Verification RAG Evaluation RAG Citation Faithfulness Evidence-Grounded QA

Open paper AskPDF arXiv

Retrieval-augmented systems routinely present citations alongside generated answers, yet a citation does not confirm that the corresponding source meaningfully shaped the output. This paper introduces ProvenAI, a framework that decomposes transparency in multi-hop question answering into three independently measurable layers: answer correctness, citation fidelity against benchmark supporting evidence, and per-document influence under leave-one-resource-out intervention. Targeting the HotpotQA distractor benchmark through a seven-stage pipeline covering data normalisation, retrieval indexing, citation-aware answer generation, attribution auditing, ablation-based influence estimation, batch evaluation, and interactive inspection, ProvenAI evaluates 7,405 validation examples drawn from a canonical corpus of 509,300 passages. The system achieves 53.53% answer accuracy alongside a mean citation-fidelity score of 71.55%, and a worked example surfaces what we call the citation-influence gap: a clean citation audit co-occurring with a profile in which one cited source registers only weak influence while seven uncited sources demonstrably shift the output. We formalise the relationship between the implemented surface proxy and a token-level KL-divergence target through a stated faithfulness condition, ground the framework in causal-mediation analysis and database-provenance theory, and discuss how the three measurement layers compose with cryptographic provenance architectures emerging in autonomous scientific discovery. ProvenAI establishes that meaningful transparency in retrieval-grounded QA requires traceable links across retrieved, cited, and behaviourally influential evidence as three distinct, independently measured layers.

Abstract

Evidence-Grounded Generation

Explore similar work

ProvenAI: Provenance-Native Traces of Evidence in Generated Answers

EnSI-RAG: Entity-Structure-Indexed Retrieval-Augmented Generation for Long-Document Question Answering

CAGE: Cognitive Attribution Graphs for Faithful Inline Citation Generation in Long-Form Question Answering