cs.CLApr 6, 2026

GROUNDEDKG-RAG: Grounded Knowledge Graph Index for Long-document Question Answering

Authors: Tianyi Zhang, Andreas Marfurt

Organizations: getAbstract · Lucerne University of Applied Sciences and Arts Switzerland

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Retrieval-augmented generation (RAG) systems have been widely adopted in contemporary large language models (LLMs) due to their ability to improve generation quality while reducing the required input context length. In this work, we focus on RAG systems for long-document question answering. Current approaches suffer from a heavy reliance on LLM descriptions resulting in high resource consumption and latency, repetitive content across hierarchical levels, and hallucinations due to no or limited grounding in the source text. To improve both efficiency and factual accuracy through grounding, we propose GroundedKG-RAG, a RAG system in which the knowledge graph is explicitly extracted from and grounded in the source document. Specifically, we define nodes in GroundedKG as entities and actions, and edges as temporal or semantic relations, with each node and edge grounded in the original sentences. We construct GroundedKG from semantic role labeling (SRL) and abstract meaning representation (AMR) parses and then embed it for retrieval. During querying, we apply the same transformation to the query and retrieve the most relevant sentences from the grounded source text for question answering. We evaluate GroundedKG-RAG on examples from the NarrativeQA dataset and find that it performs on par with a state-of-the art proprietary long-context model at smaller cost and outperforms a competitive baseline. Additionally, our GroundedKG is interpretable and readable by humans, facilitating auditing of results and error analysis.

Figures & tables

ModelExact matchSequence matchBertscoreRougeL F1
Peter Rabbit: 300 nodes, 700 edges
No context16164922
Full context13165932
GraphRAG1620638
GroundedKG-RAG16236234
The Phantom of the Opera: 25k nodes, 56k edges
Table 1: Main result of GroundedKGRAG compared to baselines on our three selected books.
ModelExact matchSequence matchBertscoreRougeL F1
No context16164922
Full context13165932
GroundedKG-RAG with SRL and K=5K=510165230
GroundedKG-RAG with SRL,    
K=5K=5 , and VectorSim ( τ=0.2\tau=0.2 )10135125
GroundedKG-RAG16236234
Table 2: Ablation of our GroundedKG-RAG on Peter Rabbit.
Input Text: Peter’s mother put Peter to bed and made some camomile tea; and Peter’s mother gave a dose of some camomile tea to Peter!
AMR Graph:SRL Graph:
(a / and :op1 (p / put-01 :ARG0 (p2 / person :ARG0-of (h / have-rel-role-91 :ARG1 (p3 / person :name (n / name :op1 "Peter")) :ARG2 (m / mother))) :ARG1 p3 :ARG2 (b / bed)) :op2 (m2 / make-01 :ARG0 p2 :ARG1 (t / tea :mod (c / camomile) :quant (s / some))) :op3 (d / dose-01 :ARG0 p2 :ARG1 p3 :ARG2 t))Predicate: put ARG-0: Peter’s mother ARG-1: Peter ARG-2: to bed Predicate: made ARG-0: Peter’s mother ARG-1: some camomile tea Predicate: gave ARG-0: Peter’s mother ARG-1: a dose of some camomile tea ARG-2: to Peter
GroundedKG with AMR:GroundedKG with SRL:
  
Table 3: Example comparing AMR and SRL graphs.
Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

ModelExact matchSequence matchBertscoreRougeL F1
Robinson Crusoe
No context1313258
Full context36365531
GroundedKG-RAG40434630
GroundedKG-RAG with RetCount ≥2\geq 230304323
Appendix
Table 4: Comparing AMR with sentence count of 2 or more on Robinson Crusoe.
Tyep I Error 
QuestionWhat happened to the young rabbits’ father?
Matched Nodeto the young rabbits’ father [’with four little rabbits mother’, ’The Tale Of Peter Rabbit’, ’from Peter cousin, little Benjamin Bunny’] happen [’happen_112’, ’come_100’, ’come_113’]
Grounded Texttext_0-3, text_0-6, text_0-20, text_0-21, text_0-27, text_0-37, text_0-40, text_0-42
Predicted AnswerThe text does not mention the father of the young rabbits.
Gold AnswerMrs. McGregor baked him into a pie.
SolutionImprove GroundedKG construction for node matching
Appendix
Table 5: Error analysis with an example for each type of error.
More in this topic

Question Answering

Question answering studies how AI systems interpret questions and produce accurate, evidence-grounded answers across text, dialogue, and multimodal data.

See all 260 papers

Explore similar work

CardsList
Aug 21, 2026

EnSI-RAG: Entity-Structure-Indexed Retrieval-Augmented Generation for Long-Document Question Answering

Question answering (QA) over long, connected documents remains challenging because relevant evidence may span multiple entities and their relationships. Existing retrieval-augmented generation (RAG) methods typically index documents as raw chunks and retrieve them through embedding similarity. Their performance degrades when chunk boundaries separate entities from supporting evidence or when a question requires multi-hop reasoning across the corpus. We propose EnSI-RAG (Entity-Structure-Indexed Retrieval-Augmented Generation), a framework that constructs a query-independent, entity-centered index. Each record (e, t, k, v) represents an entity e, its type t, a semantic category k in {property, relation, aspect}, and a value v, while retaining links to the original source passages. At query time, these records serve as retrieval handles, and an LLM synthesizes the retrieved passages into the final answer. This design separates evidence localization from answer synthesis while preserving traceable source evidence. Across Loong and Oolong, EnSI-RAG achieves an average accuracy of 78.24. Relative to the published baseline scores used as references, this is 6.62 points higher, suggesting its effectiveness across these settings. The code is available at https://github.com/RamonMeng/EnSI-RAG.

Xuanyu Meng, Jiashuo Sun, Jash Rajesh Parekh +1

University of Illinois Urbana-Champaign Urbana, Illinois, USA

Document QARetrieval-Augmented GenerationStructured RAGMulti-Hop QA
AskPDF
Jan 28, 2026

UltRAG: a Universal Simple Scalable Recipe for Knowledge Graph RAG

Large language models (LLMs) frequently generate confident yet factually incorrect content when used for language generation (a phenomenon often known as hallucination). Retrieval augmented generation (RAG) tries to reduce factual errors by identifying information in a knowledge corpus and putting it in the context window of the model. While this approach is well-established for document-structured data, it is non-trivial to adapt it for Knowledge Graphs (KGs), especially for queries that require multi-node/multi-hop reasoning on graphs. We introduce UltRAG, a training-free KG-RAG recipe that combines LLM query generation, a fully inductive neural query executor, and LLM arbitration. This off-the-shelf composition achieves state-of-the-art results on Knowledge Graph Question Answering (KGQA) tasks without retraining the LLM or executor, while enabling language models to interface with Wikidata-scale graphs (116M entities, 1.6B relations) at comparable or lower costs. Our ablation studies indicate that these gains come from the full system design rather than from any single component.

Dobrik Georgiev, Kheeran K. Naidu, Alberto Cattaneo +3

Graphcore Research

Multi-Hop QALLM Reasoning with GraphsGraph Retrieval-Augmented GenerationKnowledge Graph Question AnsweringKnowledge Graphs
AskPDF
Mar 8, 2026

KohakuRAG: A simple RAG framework with hierarchical document indexing

Retrieval-augmented generation (RAG) systems that answer questions from document collections face compounding difficulties when high-precision citations are required: flat chunking strategies sacrifice document structure, single-query formulations miss relevant passages through vocabulary mismatch, and single-pass inference produces stochastic answers that vary in both content and citation selection. We present KohakuRAG, a hierarchical RAG framework that preserves document structure through a four-level tree representation (document →\rightarrow section →\rightarrow paragraph →\rightarrow sentence) with bottom-up embedding aggregation, improves retrieval coverage through an LLM-powered query planner with cross-query reranking, and stabilizes answers through ensemble inference with abstention-aware voting. We evaluate on the WattBot 2025 Challenge, a benchmark requiring systems to answer technical questions from 32 documents with ±\pm0.1% numeric tolerance and exact source attribution. KohakuRAG achieves first place on both public and private leaderboards (final score 0.861), as the only team to maintain the top position across both evaluation partitions. Ablation studies reveal that prompt ordering (+80% relative), retry mechanisms (+69%), and ensemble voting with blank filtering (+1.2pp) each contribute substantially, while hierarchical dense retrieval alone matches hybrid sparse-dense approaches (BM25 adds only +3.1pp). We release KohakuRAG as open-source software at https://github.com/KohakuBlueleaf/KohakuRAG.

Shih-Ying Yeh, Yueh-Feng Ku, Ko-Wei Huang +1

National Tsing Hua University · Comfy Org Research · Kohaku-Lab

Retrieval-Augmented GenerationStructured RAGMulti-Document QAKnowledge-Intensive QA
AskPDF