KG-Reasoner: A Reinforced Model for End-to-End Multi-Hop Knowledge Graph Reasoning
Authors: Shuai Wang, Yinan Yu
Organizations: Department of Computer Science and Engineering, Chalmers University of Technology and University of Gothenburg, SE-41296 Gothenburg, Sweden
Large Language Models (LLMs) exhibit strong abilities in natural language understanding and generation, yet they struggle with knowledge-intensive reasoning. Structured Knowledge Graphs (KGs) provide an effective form of external knowledge representation and have been widely used to enhance performance in classical Knowledge Base Question Answering (KBQA) tasks. However, performing precise multi-hop reasoning over KGs for complex queries remains highly challenging. Most existing approaches decompose the reasoning process into a sequence of isolated steps executed through a fixed pipeline. While effective to some extent, such designs constrain reasoning flexibility and fragment the overall decision process, often leading to incoherence and the loss of critical intermediate information from earlier steps. In this paper, we introduce KG-Reasoner, an end-to-end framework that integrates multi-step reasoning into a unified "thinking" phase of a Reasoning LLM. Through Reinforcement Learning (RL), the LLM is trained to internalize the KG traversal process, enabling it to dynamically explore reasoning paths, and perform backtracking when necessary. Experiments on eight multi-hop and knowledge-intensive reasoning benchmarks demonstrate that KG-Reasoner achieves competitive or superior performance compared to the state-of-the-art methods. Codes are available at the repository: https://github.com/Wangshuaiia/KG-Reasoner.
Figures & tables
Figure 1: Comparison of KG-based reasoning paradigms. (a) and (b) illustrate two representative pipeline-style approaches that leverage knowledge graphs through staged retrieval and reasoning. (c) shows our proposed end-to-end holistic reasoning framework, which integrates retrieval and multi-hop reasoning within a unified thinking process.
Figure 2: The architecture of our end-to-end reasoning framework.
Figure 3: Backtrack for error correction.
Variant
CWQ
WebQSP
WebQuestion
GrailQA
Full Model
78.14
93.15
86.02
76.86
– SFT Warm-Up
74.33
89.42
83.35
74.91
– Hard-Case Sampling
75.63
90.16
84.21
75.06
– Search Reward ( Rsearch )
75.41
90.66
84.79
73.23
– Format Reward ( Rformat )
76.82
91.38
83.54
74.48
– Answer Reward ( Ranswer )
67.55
80.23
75.45
68.41
Table 3: Ablation study of reinforcement learning components in KG-Reasoner using Qwen3-30B-A3B.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
KG
Task
Train
Test
Complexity
Reasoning
Annotation
Source
CWQ
Freebase
KBQA
27,689
3,531
Complex Qs
Multi-hop
SPARQL, Ans.
Web-derived
WebQSP
Freebase
KBQA
3,098
1,639
Moderate Qs
Single-hop (most)
SPARQL, Ans.
Web-derived
WebQuestion
Freebase
KBQA
3,778
2,032
Simple–Moderate Qs
Single-hop (most)
Ans. only
Web-derived
GrailQA
Freebase
KBQA
35,138
1,000
Complex (varied)
Multi-hop
Logic forms, Ans.
Human-controlled
QALD10-en
Wikidata
KBQA
412
394
Complex Qs
Multi-hop
SPARQL, Ans.
Human experts
T-REx
Wikidata
Slot Filling
∼ 11M
5,000
Text (abstracts)
N/A (extraction)
Triples + text
Wikidata
Appendix
Table 4: Overview of datasets for KBQA and related tasks
Figure 4: The RL training process under two settings: with and without hard case sampling. The figure shows how reward, response length, and retrieval count change over training steps.
Figure 7Figure 8
Method
QALD10-en
T-REx
Zero-Shot RE
Creek
Qwen3-30B-A3B
55.93
19.68
17.89
83.35
Qwen3-30B-A3B + KG rag
71.86
65.39
80.43
86.83
Qwen3-30B-A3B + fixed pipeline
73.06
70.48
83.71
87.67
Qwen3-30B-A3B + RL
81.65
80.32
88.05
93.76
Qwen3-30B-A3B + RL + GAT
83.73
83.68
91.43
97.45
LLaMA-3.1-8B
40.25
23.04
12.54
75.80
Appendix
Table 8: Performance comparison of different reasoning and training strategies on WikiData benchmarks.
Large Language Models (LLMs) demonstrate impressive natural language capabilities but often struggle with knowledge-intensive reasoning tasks. Knowledge Base Question Answering (KBQA), which leverages structured Knowledge Graphs (KGs) exemplifies this challenge due to the need for accurate multi-hop reasoning. Existing approaches typically perform sequential reasoning steps guided by predefined pipelines, restricting flexibility and causing error cascades due to isolated reasoning at each step. To address these limitations, we propose KG-Hopper, a novel Reinforcement Learning (RL) framework that empowers compact open LLMs with the ability to perform integrated multi-hop KG reasoning within a single inference round. Rather than reasoning step-by-step, we train a Reasoning LLM that embeds the entire KG traversal and decision process into a unified ``thinking'' stage, enabling global reasoning over cross-step dependencies and dynamic path exploration with backtracking. Experimental results on eight KG reasoning benchmarks show that KG-Hopper, based on a 7B-parameter LLM, consistently outperforms larger multi-step systems (up to 70B) and achieves competitive performance with proprietary models such as GPT-3.5-Turbo and GPT-4o-mini, while remaining compact, open, and data-efficient. The code is publicly available at: https://github.com/Wangshuaiia/KG-Hopper.
Shuai Wang, Yinan Yu
Department of Computer Science and Engineering, Chalmers University of Technology and University of Gothenburg, SE-41296 Gothenburg, Sweden
Large Language Models (LLMs) often suffer from hallucination and struggle with complex reasoning tasks requiring multi-hop domain knowledge. While integrating Knowledge Graphs (KGs) provides a structured and verifiable information source, current KG-enhanced LLM paradigms usually rely on single-agent path extraction and fixed prompting, lacking adaptability and facing huge search spaces. To address these challenges, we propose RACER, a Reinforced Agent Collaboration framework for Explainable Reasoning on knowledge graphs. RACER employs a semantic-aware action pruning and teacher-guided reinforcement learning mechanism to efficiently extract high-quality reasoning pathways from large-scale KGs. Furthermore, to mitigate single-path generation pitfalls, we introduce a cross-task accumulated shared memory graph paired with an attention-driven multi-path knowledge refinement module. Finally, RACER orchestrates these components through a four-role multi-agent collaboration system (GraphAgent, TemplateAgent, AnswerAgent, and CriticAgent) to dynamically refine prompts and evaluate answers. Extensive experiments on CommonsenseQA and OpenBookQA datasets demonstrate that RACER significantly outperforms state-of-the-art KG-enhanced LLM baselines with an average improvement of 5%, offering robust and highly interpretable reasoning capabilities.
Yuwei Lou, Hao Hu, Yuzhou Jiang +5
State Key Laboratory for Novel Software Technology, Nanjing University, China · Independent Researcher · Chinaunicom Software Nanjing Branch
Large language models (LLMs) have shown remarkable capabilities in natural language processing. However, in knowledge graph question answering tasks (KGQA), there remains the issue of answering questions that require multi-hop reasoning. Existing methods rely on entity vector matching, but the purpose of the question is abstract and difficult to match with specific entities. As a result, it is difficult to establish reasoning paths to the purpose, which leads to information loss and redundancy. To address this issue, inspired by human reverse thinking, we propose Ontology-Guided Reverse Thinking (ORT), a novel framework that constructs reasoning paths from purposes back to conditions. ORT operates in three key phases: (1) using LLM to extract purpose labels and condition labels, (2) constructing label reasoning paths based on the KG ontology, and (3) using the label reasoning paths to guide knowledge retrieval. Experiments on the WebQSP and CWQ datasets show that ORT achieves state-of-the-art performance and significantly enhances the capability of LLMs for KGQA.
Runxuan Liu, Bei Luo, Jiaqi Li +5
Harbin Institute of Technology, Harbin, China · Beijing University of Posts and Telecommunications, Beijing, China · Joint Laboratory of HIT and iFLYTEK, Beijing, China +2