cs.CLOct 27, 2026

DeFT-RAG: Incrementally Decomposing Multi-Hop Questions into Self-Contained Sub-Questions with Reusable Answers

Authors: Dingyi Kang, Bingzhe Li

Organizations: University of Texas at Dallas, Richardson, Texas, USA

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Retrieval-augmented generation (RAG) lets a large language model (LLM) answer questions by retrieving relevant passages from a large corpus. However, RAG struggles with multi-hop questions, which require chaining facts spread across several documents through intermediate entities that the question never mentions. Graph-based RAG methods retrieve once from the entities a question names and so rarely recover the full evidence chain. Iterative RAG methods recover it step by step, but their accuracy is limited by the many tasks they ask the LLM to perform, such as planning the sub-questions, deciding their dependencies, rewriting them as answers arrive, and keeping track of what is known. We propose DeFT-RAG, a RAG framework built on three components. First, DeFT-RAG generates one self-contained sub-question at a time from the original question and the sub-questions answered so far with their answers (QA pairs). Each sub-question names every entity it asks about, which removes the need to plan, track dependencies or rewrite, and lets the LLM focus on a single fine-grained task at a time. Second, DeFT-RAG moves the remaining LLM work outside the LLM: rule-based updates to a QA table that keeps track of what is known, and yes/no decisions made by a fine-tuned lightweight decision model. Third, DeFT-RAG caches the QA pairs and reuses an answer when a later question asks a sub-question with the same meaning. On MuSiQue, 2WikiMultiHopQA and HotpotQA, DeFT-RAG achieves the best accuracy among thirteen baselines with both gpt-4o-mini and Qwen3.6 as the backbone, while giving the LLM fewer input tokens than any other iterative method. With a QA cache shared across 1,000 MuSiQue questions, 23.1% of sub-questions are answered from the cache, which reduces LLM calls by 12.9% without lowering accuracy.

Figures & tables

Figure 1. Failures of graph-based and iterative RAG on a 3-hop MuSiQue question (gpt-4o-mini runs). Top: the gold chain, which passes through two bridge entities the question never names. (a) HippoRAG 2, a graph-based method, finds the first bridge entity (Kevin Drew) but no passage about his band or Toronto. (b) LogicRAG, which plans its sub-questions in advance, searches for the city of “the performer” before finding the performer (dependency error). (c) IRCoT, which keeps all retrieved passages in its long prompt, has the whole chain there and even states that the band is based in Toronto, but then loses track of this fact (tracking error).
Figure 2. Overview of DeFT-RAG on the question of Figure 1 . The LLM performs only three fine-grained tasks (blue), and a fine-tuned DM makes every yes/no decision (orange). The QA table, updated by rules, holds the QA pairs found for the question; the QA cache holds those found for other questions. The first sub-question was answered for an earlier question, so its answer is reused from the QA cache (marked “cached”) without retrieval or an LLM call. The values next to Answer directly? and Enough facts? are the DM’s yes probabilities from our gpt-4o-mini run.
 MuSiQue2Wiki
PassagesAcc.GoldAcc.Gold
Top-50.62062%0.73387%
Top-100.63088%0.74898%
Top-5 + gold only0.66362%0.80087%
Top-10 + gold only0.79388%0.87498%
Table 1. Accuracy on questions that can be answered in one retrieval step, when the LLM is given the top- kk retrieved passages vs. only the gold passages among them (sampled questions: MuSiQue n=92n{=}92 , 2Wiki n=135n{=}135 ; gpt-4o-mini reader). Acc. = LLM-Acc; Gold = share of questions whose gold passages are all among the passages given to the LLM.
SystemMuSiQue2WikiHotpotQAAvg
EMF1AccEMF1AccEMF1AccEMF1Acc
gpt-4o-mini backbone
No Retrieval0.1220.2260.2390.2730.3510.4100.2680.3970.5270.2210.3250.392
Vanilla RAG0.2380.3470.4120.4590.5530.6590.4680.6190.7990.3880.5060.623
LightRAG0.0000.0870.3050.0000.1500.5050.0000.1900.6960.0000.1420.502
PathRAG0.1230.2330.2760.2780.3790.4510.3970.5490.6930.2660.3870.473
Table 2. Results on 1,000 questions per benchmark with two backbones: exact match (EM), token F1, and LLM-judged accuracy (Acc). gpt-4o-mini runs are judged by Qwen3.6 and Qwen3.6 runs by gpt-4o-mini, so no model grades its own answers. EM and F1 are computed from the answers as given and understate systems that answer in full sentences, such as LightRAG with gpt-4o-mini; Acc is our main metric. Avg: mean over the three benchmarks. Best in bold, second best underlined.
Method2-hop3-hop4-hopWords
Vanilla RAG46%7%1%870
Graph-based
PathRAG42%10%8%12,390
HippoRAG33%7%0%520
HippoRAG 253%8%0%500
MemGraphRAG41%7%1%500
Table 3. Evidence coverage on MuSiQue (gpt-4o-mini), for questions that need 2, 3 or 4 hops: the share of questions for which the LLM received all passages needed to answer them, as annotated in the dataset. Words: the average number of words of passages the LLM received per question.
 MuSiQue2WikiHotpotQAAvg
DeFT-RAG (top-10, selected)0.6330.8600.8800.791
No selection (top-5)0.6100.8230.8270.753
Difference+2.3+3.7+5.3+3.8
Table 4. Effect of selecting relevant passages: DeFT-RAG versus the same process without selection (the top-5 retrieved passages, all kept). gpt-4o-mini backbone, 300 test questions per benchmark (a subset of those in Table 2 ); Avg is the mean over the three benchmarks.
DecisionMuSiQue2WikiHotpotQA
Relevance0.931 / 0.9790.996 / 1.0000.990 / 0.997
Direct answer0.850 / 0.9330.938 / 0.9970.915 / 0.995
Enough facts0.903 / 0.9040.943 / 0.9590.816 / 0.845
Table 5. Effect of fine-tuning on decision quality, base DM / fine-tuned DM, on 300 test questions per benchmark. AUC measures how well the DM’s yes probabilities separate yes cases from no cases (1.0 is perfect). HotpotQA has no answer for each step, so its enough-facts label is yes when the passages used include all gold supporting passages (167 cases).
Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

SettingValue
Retrieved passages ww (retry and fallback)10 (20)
Step limit TT6
Relevance: keep if p(yes)≥p(\text{yes})\geq0.5
Passages kept when only one / no passage is judged relevant2 / 5
Passages kept at most5
Direct answer: sufficient ≥\geq0.7
Appendix
Table 6. Settings of DeFT-RAG used for all results.
 Index (per corpus)Answering (per question)
MethodLLM callsTokens (M)Time (h)Latency (s)LLM callsInput tokensOutput tokens
No Retrieval000.002.41.065153
Vanilla RAG000.092.41.01,279142
LightRAG2,10710.91.85.32.034,32485
PathRAG a2,10710.91.83.82.017,29147
HippoRAG b18,39110.70.83.42.0956150
Appendix
Table 7. Overhead of every method with gpt-4o-mini, mean over the three benchmarks (1,000 questions each). Index: LLM calls, LLM tokens (input + output) and build time, per corpus. Answering: per question, from question to answer (grading excluded). Methods with an index of passage embeddings only use no LLM to build it; their build time is that of embedding the corpus with Qwen3-Embedding-4B. a Uses LightRAG’s graph. b Uses HippoRAG 2’s graph. c Includes the extraction it shares with HippoRAG 2. d Build time not recorded; no LLM is used. e Mean latency (only totals are recorded). Latency is the median over questions, because a few API stalls of several minutes dominate the mean. f DeFT-RAG with the closed decision model Jev, accessed through an API, in place of the local DM; its time per call is set to its typical 0.16 s.
More in this topic

Multi-Hop QA

Multi-hop question answering studies how systems combine evidence across documents, facts, or modalities to answer questions requiring linked reasoning.

See all 232 papers

Explore similar work

CardsList
Aug 3, 2026

MEGRAG: Multi-Granular Evidence Graphs for Answer-Aware Multi-Hop RAG

Multi-hop question answering is a fundamental challenge in retrieval-augmented generation (RAG), because deriving an answer requires integrating dispersed evidence. Iterative RAG (iRAG) is widely used for this challenge, but existing methods have two limitations. First, most methods still support each reasoning step with single-granularity evidence, making it difficult to balance information density and contextual noise. Second, existing methods often answer the original question only after aggregating evidence retrieved across intermediate steps, so redundant evidence and intermediate retrieval errors may accumulate and degrade the final answer. To address these limitations, we propose MEGRAG, an answer-aware framework that represents multi-hop reasoning as a path-structured multi-granular evidence graph. Offline, MEGRAG links passages to their sentences and extracted triples through a cross-granularity index. Online, it retrieves passages for the current query and selects aligned evidence, starting with compact triples and adding sentence or passage context as needed. MEGRAG uses the resulting intermediate answer and prior reasoning to decide whether the Initial Query has been resolved. If not, it identifies the missing information and formulates a focused next query; otherwise, it stops retrieval and returns the answer. Extensive experiments demonstrate consistent gains over a diverse set of RAG baselines.

Weidong Bao, Yingying Sun, Jun Yang +7

Northeastern University, Shenyang, China

Multi-Hop QARetrieval-Augmented GenerationGraph Retrieval-Augmented GenerationAdaptive RAG
AskPDF
Aug 5, 2026

D2^2F-ReAG: Dynamic Decomposition and Filtering for Multi-Hop Reasoning-Augmented Generation

Large language models (LLMs) often generate inaccurate answers due to their reliance on static internal knowledge. Retrieval-augmented generation (RAG) addresses this limitation by integrating external knowledge and excelling at single-hop queries. However, it struggles with multi-hop questions that require cross-document reasoning. Existing methods, such as graph structured RAG or question decomposition, often lack dynamic decomposition and effective filtering, which leads to lower efficiency and accuracy. To overcome these limitations, we propose Dynamic Decomposition and Filtering for Multi-Hop Reasoning-Augmented Generation (D2F-ReAG), a novel paradigm that adaptively controls reasoning depth by judging the reliability of the root-level reasoning. If the root reasoning is reliable, the model directly generates the answer. Otherwise, the question is logically decomposed into sub-questions, and the verified reasoning derived from these sub-questions is used to refine the root reasoning. Experiments on three multi-hop benchmarks demonstrate the effectiveness of our method in handling complex multi-hop questions.

Jiaoyang Li, Junhao Ruan, Shengwei Tang +4

Northeastern University, Shenyang, China · Kunming University of Science and Technology, Kunming, China · NiuTrans Research, Shenyang, China

Multi-Hop QARetrieval-Augmented GenerationAdaptive RAGQuestion DecompositionQuestion AnsweringMulti-Hop Reasoning
AskPDF
May 13, 2026

Retrieval is Cheap, Show Me the Code: Executable Multi-Hop Reasoning for Retrieval-Augmented Generation

Retrieval-Augmented Generation (RAG) has become a standard approach for knowledge-intensive question answering, but existing systems remain brittle on multi-hop questions, where solving the task requires chaining multiple retrieval and reasoning steps. Key challenges are that current methods represent reasoning through free-form natural language, where intermediate states are implicit, retrieval queries can drift from intended entities, and errors are detected by the same model that produces them making self-reflection an unreliable, ungrounded signal. We observe that multi-hop question answering is a typical form of step-by-step computation, and that this structured process aligns closely with how code-specialized language models are trained to operate. Motivated by this, we introduce \pyrag, a framework that reformulates multi-hop RAG as program synthesis and execution. Instead of free-form reasoning trajectories, \pyrag represents the reasoning process as an executable Python program over retrieval and QA tools, exposing intermediate states as variables, producing deterministic feedback through execution, and yielding an inspectable trace of the entire reasoning process. This formulation further enables compiler-grounded self-repair and execution-driven adaptive retrieval without any additional training. Experiments on five QA benchmarks (PopQA, HotpotQA, 2WikiMultihopQA, MuSiQue, and Bamboogle) show that \pyrag consistently outperforms strong baselines under both training-free and RL-trained settings, with especially large gains on compositional multi-hop datasets. Our code, data and models are publicly available at https://github.com/GasolSun36/PyRAG.

Jiashuo Sun, Jimeng Shi, Yixuan Xie +10

University of Illinois Urbana-Champaign · Hong Kong University of Science and Technology · Texas A&M University +1

Multi-Hop QARetrieval-Augmented GenerationRetrieval-Augmented Reasoning
AskPDF