Organizations: The Hong Kong University of Science and Technology (HKUST) · Independent Researcher · School of Computer Science and Engineering, Southeast University · Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications · Hong Kong Generative AI Research and Development Center (HKGAI)
Retrieval-augment generation is a prevalent strategy to mitigate hallucinations of LLMs. The attributable RAG (RAGQ) generates quotes for its answers. The quotes indicate which input contexts support the RAG to derive the answers, enhancing the answer's verifiability and trustworthiness. However, existing RAGQs exhibit significant degradation when dealing with questions that require multi-hop reasoning and multi-modal understanding, suffering from over-citation, implicit entity identification failure, and poor generalization. In this paper, we propose a novel RAGQ framework, namely QDRAG. QDRAG breaks down the input question into atomic subquestions to identify the implicit entities. Then, the reranker prunes context distractors to eliminate the downstream over-citation. To facilitate query decomposition, we propose two zero-shot approaches: QD-C and QD-R, which guide the QD MLLM to decompose the question based on context knowledge and retrieval rewards, respectively. One interesting finding is that finetuning on the QD task shows better generalizability compared to directly finetuning on the downstream RAGQ task. Experiments on four multi-modal QA benchmarks demonstrate QDRAG's efficacy in grounding answers and generating faithful citations. The framework significantly outperforms all the baselines on both in-domain and out-of-domain tests, even surpassing Gemini-Pro.
Figures & tables
Figure 1: The massive retrieved data in RAG misleads the MLLM to cite the incorrect but similar contexts; Sparse RAG drops key contexts in the reranking step for multi-hop questions; QDRAG filters the distractors while maintaining the key contexts for accurate downstream citations.
Figure 2: QDRAG finetunes the MLLM for the QD task, leveraging its fast training and better generalizability.
Figure 3: Evaluate MLLM’s capability to cite images from a multi-modal statement on two datasets: CIRCO and CIRR.
Figure 4: Dense RAG vs Sparse RAG in terms of answer accuracy and attribution capability. (a) , (b) : Sparse RAG shows superiority on both metrics; (c) , (d) : Sparse RAG still excels on attribution performance, but has worse answer quality.
Figure 5: QD-R samples multiple MLLM outputs and uses the recall of golden contexts as reward to train the model.
Model
Model Size
In-domain
Out-of-domain
MMQA
WebQA
InfoSeek
E-VQA
EM
LLM
Cite
EM
LLM
Cite
EM
LLM
Cite
EM
LLM
Cite
Retrieval-Generation
Gemini-Pro
-
43.2
34.2
68.4
55.1
26.6
64.6
9.8
0.0
0.9
6.1
3.4
23.6
Gemini-Pro CLIP
-
-
-
60.4
-
-
57.9
-
-
0.4
-
-
27.9
Qwen2.5VL
7B
42.4
40.6
38.2
57.8
41.5
33.0
42.5
36.7
59.6
10.4
7.8
18.5
Table 1: Overall in-domain and out-of-domain test, where best and second-best results are highlighted in bold and underlined , respectively. The retriever for all models is CLIP-ViT-L/14@336px. Single model size means that QD/Reranking/QA use the same MLLM, while multiple model sizes (e.g., 13B/7B) means the reranker model size and the QD/QA model size, respectively.
MMQA
WebQA
EM
LLM
Cite
EM
LLM
Cite
RAG
67.7
64.1
51.3
70.7
64.0
60.9
MMEmbed
69.7
65.1
71.5
68.0
57.1
69.4
SparseRAG
67.9
66.1
71.9
68.5
60.9
65.3
LlamaIndex
68.0
65.7
69.9
69.6
61.2
72.1
QDRAG-C
70.7
66.4
74.7
70.7
63.7
75.1
Table 2: Comparison of single-round RAGs with 7B retriever
MMQA
WebQA
EM
LLM
Cite
EM
LLM
Cite
RaFe-C
43.3
42.2
49.9
55.6
40.3
44.4
QD-C
43.3
42.2
52.6
56.7
40.2
45.8
RaFe-R
43.6
42.0
49.3
55.7
38.7
42.3
RaFe-R DPO
43.8
42.7
50.2
57.4
41.7
48.4
QD-R
46.7
45.4
53.8
61.0
48.3
56.8
Table 3: Compare QD with rewriting models
Figure 6: Failure case analysis. IWCW ↓ : Input wrong context and cite incorrectly; IWCC ↑ : Input wrong context but cite correctly; ICCW ↓ : Input correct context but cite incorrectly; ICCC ↑ : Input correct context and cite correctly.
Pipe
FT task
InfoSeek
EVQA
LLM
Cite
LLM
Cite
RAG
-
36.7
59.6
7.8
18.5
RAGQ
32.9
55.6
6.4
18.7
QD-R
36.6
62.3
7.8
19.1
QDRAG-R
-
35.2
66.3
10.6
28.9
RAGQ
33.0
68.4
8.6
26.0
Table 4: Out-of-domain test: Front vs QDRAG
FT task
Feedback
Auto
Cost
RAGQ
LLM-judge
✓
28h
QD-R
Retrieval
✓
2h
Table 5: Cost of feedback when building training data
D
R
S
MMQA
WebQA
EM
LLM
Cite
EM
LLM
Cite
✓
✓
✓
43.3
42.2
52.6
56.7
40.2
45.8
✓
✓
43.3
41.8
51.0
56.8
40.8
45.4
✓
✓
43.8
41.4
50.9
56.9
39.4
45.2
✓
✓
44.0
42.0
52.0
56.5
40.7
45.8
43.5
42.0
48.8
55.8
39.8
41.9
Table 6: Ablation on QDRAG-C
M
W
S
MMQA
WebQA
EM
LLM
Cite
EM
LLM
Cite
✓
✓
✓
49.2
46.3
57.5
61.7
48.0
59.9
✓
✓
46.9
44.6
56.9
61.1
48.1
56.9
✓
✓
46.4
45.4
54.3
61.6
48.3
58.3
✓
✓
48.8
46.0
56.5
61.9
47.8
60.6
46.9
45.0
54.8
61.0
48.1
56.9
Table 7: Ablation on QDRAG-R Traing Data
Figure 7: Case study: QDRAG generates better query decompositions. GTR: Golden context Recall.
Figure 8: Efficient & cost analysis
Figure 9: Visualization of image contexts with the largest attention values when MLLM generates the references after “What color is Giorgio Armani’s hair? White. According to Document ␣”
Retrieval-augmented generation (RAG) lets a large language model (LLM) answer questions by retrieving relevant passages from a large corpus. However, RAG struggles with multi-hop questions, which require chaining facts spread across several documents through intermediate entities that the question never mentions. Graph-based RAG methods retrieve once from the entities a question names and so rarely recover the full evidence chain. Iterative RAG methods recover it step by step, but their accuracy is limited by the many tasks they ask the LLM to perform, such as planning the sub-questions, deciding their dependencies, rewriting them as answers arrive, and keeping track of what is known. We propose DeFT-RAG, a RAG framework built on three components. First, DeFT-RAG generates one self-contained sub-question at a time from the original question and the sub-questions answered so far with their answers (QA pairs). Each sub-question names every entity it asks about, which removes the need to plan, track dependencies or rewrite, and lets the LLM focus on a single fine-grained task at a time. Second, DeFT-RAG moves the remaining LLM work outside the LLM: rule-based updates to a QA table that keeps track of what is known, and yes/no decisions made by a fine-tuned lightweight decision model. Third, DeFT-RAG caches the QA pairs and reuses an answer when a later question asks a sub-question with the same meaning. On MuSiQue, 2WikiMultiHopQA and HotpotQA, DeFT-RAG achieves the best accuracy among thirteen baselines with both gpt-4o-mini and Qwen3.6 as the backbone, while giving the LLM fewer input tokens than any other iterative method. With a QA cache shared across 1,000 MuSiQue questions, 23.1% of sub-questions are answered from the cache, which reduces LLM calls by 12.9% without lowering accuracy.
Dingyi Kang, Bingzhe Li
University of Texas at Dallas Richardson, Texas, USA
Retrieval-Augmented Generation (RAG) over Knowledge Graphs (KGs) suffers from the fact that indexing approaches may lose important contextual nuance when text is reduced to triples, thereby degrading performance in downstream Question-Answering (QA) tasks, particularly for multi-hop QA, which requires composing answers from multiple entities, facts, or relations. We propose a domain-agnostic, KG-based QA framework that covers both the indexing and retrieval/inference phases. A new indexing approach called Map-Disambiguate-Enrich-Reduce (MDER) generates context-derived triple descriptions and subsequently integrates them with entity-level summaries, thus avoiding the need for explicit traversal of edges in the graph during the QA retrieval phase. Complementing this, we introduce Decompose-Resolve (DR), a retrieval mechanism that decomposes user queries into resolvable triples and grounds them in the KG via iterative reasoning. Together, MDER and DR form an LLM-driven QA pipeline that is robust to sparse, incomplete, and complex relational data. Experiments show that on standard and domain specific benchmarks, MDER-DR achieves substantial improvements over standard RAG baselines (up to 66%), while maintaining cross-lingual robustness. Our code is available at https://github.com/DataSciencePolimi/MDER-DR_RAG.
Multi-hop question answering remains challenging for Retrieval-Augmented Generation (RAG) because existing approaches either propagate errors across iterative retrieval rounds or over-generate reasoning steps, increasing cost without improving accuracy. We propose Grounded Delta Planning RAG (GDP-RAG), a plan-based framework that targets only the information delta based on three simple design choices: (1) preliminary retrieval to ground planning before execution, (2) a gap-conditioned planning prompt that asks only for missing information, and (3) a skeletal trajectory that pairs each subquery with a Thought capturing evidence from preliminary retrieval and carrying it through to the final answer. GDP-RAG focuses computation on unresolved gaps, yielding concise, reliable reasoning trajectories. Extensive experiments on HotpotQA, 2WikiMultiHopQA, and MuSiQue show that GDP-RAG achieves the highest accuracy (60.63%) among all compared systems while maintaining a cost-of-pass of 0.51, 22% lower than PAR-RAG (0.65) and 68% lower than KnowTrace (1.57), with no method achieving both higher accuracy and lower cost.
Wei-Chieh Chou, Xuanjun Chen, Jian-Ren Lin +3
Dept. of Computer Science and Information Engineering, National Taiwan University · Graduate Institute of Communication Engineering, National Taiwan University · Department of Economics, National Taiwan University +2