Recent literature highlights the potential of graph-based approaches within large language model (LLM) retrieval-augmented generation (RAG) pipelines for answering queries of varying complexity, particularly those that fall outside the LLM's prior knowledge. However, LLMs are prone to hallucination and often face technical limitations in handling contexts large enough to ground complex queries effectively. To address these challenges, we propose a novel iterative, feedback-driven GraphRAG architecture that leverages response quality assessment to iteratively refine outputs until a sound, well-grounded response is produced. Evaluating our approach with queries from the HotPotQA dataset, we demonstrate that this iterative RAG strategy yields responses with higher semantic quality and improved relevance compared to a single-shot baseline.
Figures & tables
Figure 1: System architecture
1. for each question:
2. extract entities using NER
3. embed the question for semantic search
4. initialize context using entity-based semantic search
5. repeat up to 4 times:
6. if first iteration:
7. retrieve initial evidence from graph
Figure 2: KGiRAG processing pseudocode
You are a helpful question-answering assistant. \n
Question: { question } \n
Context: { context } \n
Instructions: \n
- Use the information provided in the context to answer the question. \n
- Pay special attention to the ’Relevant text units’, which contain structured facts from the knowledge graph. \n
- Be direct and concise in your answer. \n
Figure 3: The prompt template supplied to the LLM.
Metric
KGiRAG with NER
KGiRAG without NER
Microsoft GraphRAG
RARR
Faithfulness
0.95 ± 0.01881
0.78 ± 0.03613
0.95 ± 0.01749
0.90 ± 0.02615
Completeness
0.86 ± 0.029802
0.69 ± 0.03663
0.62 ± 0.0396
0.41 ± 0.04145
Relevance
0.64 ± 0.042269
0.76 ± 0.0371
0.23 ± 0.03549
0.59 ± 0.04282
BERTScore
0.79 ± 0.001603
0.81 ± 0.001423
0.77 ± 0.001551
0.797 ± 0.0021
Cosine Similarity
0.325 ± 0.02692
0.52 ± 0.01828
0.075 ± 0.01831
0.33 ± 0.02427
Table 1: Comparison of retrieval architectures across evaluation metrics (mean and margin of error for computing the 95% confidence intervals).
Figure 4: Score density plots of the evaluation metrics for the evaluated architectures
Iteration
KGiRAG plus NER
KGiRAG
1
274
299
2
83
41
3
59
15
4
84
145
Table 2: Number of responses solved in each of the four iterations of the KGiRAG
Figure 5: Metrics evolution during iterations for KGiRAG
Large language models (LLMs) frequently generate confident yet factually incorrect content when used for language generation (a phenomenon often known as hallucination). Retrieval augmented generation (RAG) tries to reduce factual errors by identifying information in a knowledge corpus and putting it in the context window of the model. While this approach is well-established for document-structured data, it is non-trivial to adapt it for Knowledge Graphs (KGs), especially for queries that require multi-node/multi-hop reasoning on graphs. We introduce UltRAG, a training-free KG-RAG recipe that combines LLM query generation, a fully inductive neural query executor, and LLM arbitration. This off-the-shelf composition achieves state-of-the-art results on Knowledge Graph Question Answering (KGQA) tasks without retraining the LLM or executor, while enabling language models to interface with Wikidata-scale graphs (116M entities, 1.6B relations) at comparable or lower costs. Our ablation studies indicate that these gains come from the full system design rather than from any single component.
Dobrik Georgiev, Kheeran K. Naidu, Alberto Cattaneo +3
Graph-RAG systems achieve strong multi-hop question answering by indexing documents into knowledge graphs, but strong retrieval does not guarantee strong answers. Evaluating KET-RAG, a leading Graph-RAG system, on three multi-hop QA benchmarks (HotpotQA, MuSiQue, 2WikiMultiHopQA), we find that 77% to 91% of questions have the gold answer in the retrieved context, yet accuracy is only 35% to 78%, and 73% to 84% of errors are reasoning failures. We propose two augmentations: (i) SPARQL chain-of-thought prompting, which decomposes questions into triple-pattern queries aligned with the entity-relationship context, and (ii) graph-walk compression, which compresses the context by ~60% via knowledge-graph traversal with no LLM calls. SPARQL CoT improves accuracy by +2 to +14 pp; graph-walk compression adds +6 pp on average when paired with structured prompting on smaller models. Surprisingly, we show that, with question-type routing, a fully augmented budget open-weight Llama-8B model matches or exceeds the unaugmented Llama-70B baseline on all three benchmarks at ~12x lower cost. A replication on LightRAG confirms that our augmentations transfer across Graph-RAG systems.
Yasaman Zarrinkia, Venkatesh Srinivasan, Alex Thomo
Graph-based Retrieval-Augmented Generation (RAG) systems leverage interconnected knowledge structures to capture complex relationships that flat retrieval struggles with, enabling multi-hop reasoning. Yet most existing graph-based methods suffer from (i) heuristic designs lacking theoretical guarantees for subgraph quality or relevance and/or (ii) the use of static exploration strategies that ignore the query's holistic meaning, retrieving neighborhoods or communities regardless of intent. We propose Query-Aware Flow Diffusion RAG (QAFD-RAG), a training-free framework that dynamically adapts graph traversal to each query's holistic semantics. The central innovation is query-aware traversal: during graph exploration, edges are dynamically weighted by how well their endpoints align with the query's embedding, guiding flow along semantically relevant paths while avoiding structurally connected but irrelevant regions. These query-specific reasoning subgraphs enable the first statistical guarantees for query-aware graph retrieval, showing that QAFD-RAG recovers relevant subgraphs with high probability under mild signal-to-noise conditions. The algorithm converges exponentially fast, with complexity scaling with the retrieved subgraph size rather than the full graph. Experiments on question answering and text-to-SQL tasks demonstrate consistent improvements over state-of-the-art graph-based RAG methods.