KG-Reasoner: A Reinforced Model for End-to-End Multi-Hop Knowledge Graph Reasoning

Shuai Wang; Yinan Yu

cs.CLApr 14, 2026

KG-Reasoner: A Reinforced Model for End-to-End Multi-Hop Knowledge Graph Reasoning

Authors: Shuai Wang, Yinan Yu

Organizations: Department of Computer Science and Engineering, Chalmers University of Technology and University of Gothenburg, SE-41296 Gothenburg, Sweden

Abstract

Large Language Models (LLMs) exhibit strong abilities in natural language understanding and generation, yet they struggle with knowledge-intensive reasoning. Structured Knowledge Graphs (KGs) provide an effective form of external knowledge representation and have been widely used to enhance performance in classical Knowledge Base Question Answering (KBQA) tasks. However, performing precise multi-hop reasoning over KGs for complex queries remains highly challenging. Most existing approaches decompose the reasoning process into a sequence of isolated steps executed through a fixed pipeline. While effective to some extent, such designs constrain reasoning flexibility and fragment the overall decision process, often leading to incoherence and the loss of critical intermediate information from earlier steps. In this paper, we introduce KG-Reasoner, an end-to-end framework that integrates multi-step reasoning into a unified "thinking" phase of a Reasoning LLM. Through Reinforcement Learning (RL), the LLM is trained to internalize the KG traversal process, enabling it to dynamically explore reasoning paths, and perform backtracking when necessary. Experiments on eight multi-hop and knowledge-intensive reasoning benchmarks demonstrate that KG-Reasoner achieves competitive or superior performance compared to the state-of-the-art methods. Codes are available at the repository: https://github.com/Wangshuaiia/KG-Reasoner.

Figures & tables

Figure 1: Comparison of KG-based reasoning paradigms. (a) and (b) illustrate two representative pipeline-style approaches that leverage knowledge graphs through staged retrieval and reasoning. (c) shows our proposed end-to-end holistic reasoning framework, which integrates retrieval and multi-hop reasoning within a unified thinking process.

Figure 2: The architecture of our end-to-end reasoning framework.

Figure 3: Backtrack for error correction.

Variant	CWQ	WebQSP	WebQuestion	GrailQA
Full Model	78.14	93.15	86.02	76.86
– SFT Warm-Up	74.33	89.42	83.35	74.91
– Hard-Case Sampling	75.63	90.16	84.21	75.06
– Search Reward ( $R_{\text{search}}$ )	75.41	90.66	84.79	73.23
– Format Reward ( $R_{\text{format}}$ )	76.82	91.38	83.54	74.48
– Answer Reward ( $R_{\text{answer}}$ )	67.55	80.23	75.45	68.41

Table 3: Ablation study of reinforcement learning components in KG-Reasoner using Qwen3-30B-A3B.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	KG	Task	Train	Test	Complexity	Reasoning	Annotation	Source
CWQ	Freebase	KBQA	27,689	3,531	Complex Qs	Multi-hop	SPARQL, Ans.	Web-derived
WebQSP	Freebase	KBQA	3,098	1,639	Moderate Qs	Single-hop (most)	SPARQL, Ans.	Web-derived
WebQuestion	Freebase	KBQA	3,778	2,032	Simple–Moderate Qs	Single-hop (most)	Ans. only	Web-derived
GrailQA	Freebase	KBQA	35,138	1,000	Complex (varied)	Multi-hop	Logic forms, Ans.	Human-controlled
QALD10-en	Wikidata	KBQA	412	394	Complex Qs	Multi-hop	SPARQL, Ans.	Human experts
T-REx	Wikidata	Slot Filling	$\sim$ 11M	5,000	Text (abstracts)	N/A (extraction)	Triples + text	Wikidata

Appendix

Table 4: Overview of datasets for KBQA and related tasks

Panel of Figure 4 — Figure 4: The RL training process under two settings: with and without hard case sampling. The figure shows how reward, response length, and retrieval count change over training steps.

Method	QALD10-en	T-REx	Zero-Shot RE	Creek
Qwen3-30B-A3B	55.93	19.68	17.89	83.35
Qwen3-30B-A3B + KG rag	71.86	65.39	80.43	86.83
Qwen3-30B-A3B + fixed pipeline	73.06	70.48	83.71	87.67
Qwen3-30B-A3B + RL	81.65	80.32	88.05	93.76
Qwen3-30B-A3B + RL + GAT	83.73	83.68	91.43	97.45
LLaMA-3.1-8B	40.25	23.04	12.54	75.80

Appendix

Table 8: Performance comparison of different reasoning and training strategies on WikiData benchmarks.

Explore similar work

KG-Hopper: Empowering Compact Open LLMs with Knowledge Graph Reasoning via Reinforcement Learning
Mar 22, 2026Shuai Wang, Yinan YuLLM Reasoning with Graphs Knowledge Graph Question Answering
Department of Computer Science and Engineering, Chalmers University of Technology and University of Gothenburg, SE-41296 Gothenburg, Sweden
RACER: Reinforced Agent Collaboration for Explainable Reasoning on Knowledge Graphs
Aug 29, 2026Yuwei Lou, Hao Hu, Yuzhou Jiang +5LLM Reasoning with Graphs Knowledge Graph Question Answering
State Key Laboratory for Novel Software Technology, Nanjing University, China · Independent Researcher · Chinaunicom Software Nanjing Branch
Ontology-Guided Reverse Thinking Makes Large Language Models Stronger on Knowledge Graph Question Answering
Feb 17, 2025Runxuan Liu, Bei Luo, Jiaqi Li +5LLM Reasoning with Graphs Graph-Based Retrieval
Harbin Institute of Technology, Harbin, China · Beijing University of Posts and Telecommunications, Beijing, China · Joint Laboratory of HIT and iFLYTEK, Beijing, China +2

Mar 22, 2026J/K move · Enter open · S save

KG-Hopper: Empowering Compact Open LLMs with Knowledge Graph Reasoning via Reinforcement Learning

Shuai Wang, Yinan Yu

Department of Computer Science and Engineering, Chalmers University of Technology and University of Gothenburg, SE-41296 Gothenburg, Sweden

LLM Reasoning with Graphs Knowledge Graph Question Answering Multi-Hop Reasoning RL for Language Model Reasoning

Open paper AskPDF arXiv

Large Language Models (LLMs) demonstrate impressive natural language capabilities but often struggle with knowledge-intensive reasoning tasks. Knowledge Base Question Answering (KBQA), which leverages structured Knowledge Graphs (KGs) exemplifies this challenge due to the need for accurate multi-hop reasoning. Existing approaches typically perform sequential reasoning steps guided by predefined pipelines, restricting flexibility and causing error cascades due to isolated reasoning at each step. To address these limitations, we propose KG-Hopper, a novel Reinforcement Learning (RL) framework that empowers compact open LLMs with the ability to perform integrated multi-hop KG reasoning within a single inference round. Rather than reasoning step-by-step, we train a Reasoning LLM that embeds the entire KG traversal and decision process into a unified ``thinking'' stage, enabling global reasoning over cross-step dependencies and dynamic path exploration with backtracking. Experimental results on eight KG reasoning benchmarks show that KG-Hopper, based on a 7B-parameter LLM, consistently outperforms larger multi-step systems (up to 70B) and achieves competitive performance with proprietary models such as GPT-3.5-Turbo and GPT-4o-mini, while remaining compact, open, and data-efficient. The code is publicly available at: https://github.com/Wangshuaiia/KG-Hopper.

Abstract

Explore similar work

KG-Hopper: Empowering Compact Open LLMs with Knowledge Graph Reasoning via Reinforcement Learning

RACER: Reinforced Agent Collaboration for Explainable Reasoning on Knowledge Graphs

Ontology-Guided Reverse Thinking Makes Large Language Models Stronger on Knowledge Graph Question Answering