GraphRAGProv: GraphRAG Meets How-Provenance
Organizations: Independent Toulouse, France · INRIA Rennes, France
Abstract
GraphRAG systems often use knowledge graphs to retrieve relevant context for an LLM that produces the final answer. That makes answers more inscrutable and therefore less appealing for domains where transparency is paramount. We present GraphRAGProv, an architecture that executes SPARQL queries over source-grounded RDF graphs to compute answers and their how-provenance polynomials. These polynomials record which statement occurrences were used and how they were combined. Because each occurrence is linked to its source text, users can follow these links to check whether the extracted statements reflect the text and whether the query combines them to answer the question. Querying thereby becomes a way to inspect and evaluate the text-to-graph transformation. Our interactive demo examines 400 cases from two multi-hop QA benchmarks, revealing extraction and query errors even when the final answer is correct.
Figures & tables
| Stage | 2Wiki | MuSiQue | Total |
|---|---|---|---|
| Cases | 200 | 200 | 400 |
| SPARQL executed | 200 | 198 | 398 |
| Answer returned | 193 | 134 | 327 |
| Provenance decoded | 193 | 118 | 311 |
| Complete text trace | 193 | 118 | 311 |
| Question and case ID | Inspection finding |
|---|---|
| Meesaya Murukku (2Wiki) Which country is its director from? ba413fca0bd911eba7f7acde48001122 | Correct answer, India , but the cited text describes the film as Indian, not its director. \begin{aligned} &q\to Q\to\mu\to P_{\mu}\to{\color[rgb]{1,0,0}id_{i}}\\[-1.0pt] &\hskip 8.19447pt\end{aligned} |
| Labyrinth (MuSiQue) When did its publisher cease operating? 2hop__170823_120171 | Correct answer, 1986, but the query omits the game–publisher relation. \begin{aligned} &q\to{\color[rgb]{1,0,0}Q}\to\mu\to P_{\mu}\to id_{i}\\[-1.0pt] &\hskip 8.19447pt\end{aligned} |
| Aldobrandino (2Wiki) When was his father born? 96a72db40bdd11eba7f7acde48001122 | Two IRIs represent the same father, preventing the join. \begin{aligned} &q\to Q\to{\color[rgb]{1,0,0}\mu=\varnothing}\to{\color[rgb]{0.5,0.5,0.5}P_{\mu}\to id_{i}}\\[-1.0pt] &\hskip 8.19447pt{\color[rgb]{0.5,0.5,0.5}}\end{aligned} |
| Caledon River (MuSiQue) Where is the source of the receiving river? 2hop__207597_63853 | One passage supports both required relations, although the benchmark designates two. |
Graph Retrieval-Augmented Generation
Graph Retrieval-Augmented Generation connects language models to knowledge graphs to retrieve relational evidence for grounded, multi-hop answers.
See all 232 papers