Learning to Retrieve and Reason on Knowledge Graph through Active Self-Reflection

Han Zhang; Langshi Zhou; Hanfang Yang

cs.CLFeb 20, 2025

Learning to Retrieve and Reason on Knowledge Graph through Active Self-Reflection

Authors: Han Zhang, Langshi Zhou, Hanfang Yang

Organizations: Center for Applied Statistics, Renmin University of China · School of Statistics, Renmin University of China

Abstract

Extensive research has investigated the integration of large language models (LLMs) with knowledge graphs to enhance the reasoning process. However, understanding how models perform reasoning utilizing structured graph knowledge remains underexplored. Most existing approaches rely on LLMs or retrievers to make binary judgments regarding the utilization of knowledge, which is too coarse. Meanwhile, there is still a lack of feedback mechanisms for reflection and control throughout the entire reasoning path. This paper proposes an Active self-Reflection framework for knowledge Graph reasoning (ArG), introducing for the first time an end-to-end training approach to achieve iterative reasoning grounded on structured graphs. Within the framework, the model leverages special tokens to actively control the reasoning process, achieving on-demand retrieval and multi-dimensional, finer-grained knowledge evaluation. The reasoning paths generated by the model exhibit high interpretability, enabling deeper exploration of the model's understanding of structured knowledge. Ultimately, the proposed model achieves outstanding results compared to existing baselines in knowledge graph reasoning tasks.

Figures & tables

Figure 1. An example of ArG performs a more fine-grained assessment and actively retrieves knowledge compared to LLM pruning and direct retrieval. ArG not only provides reasonable evaluations, but also captures useful indirect relationships.

Figure 2. The overall framework of ArG . Given an input query, the trained generator model $\mathcal{M}$ iteratively performs knowledge retrieval over the structural graph based on the retrieval token. Subsequently, the retrieved knowledge undergoes processes of critique and reflection, where implausible information is filtered. The iterative procedure culminates in the generation of an answer. ArG exhibits strong interpretability when applied to structured graph. As demonstrated in the example, the step-by-step reasoning path is organized in the lower half.

Token Type	Category	Definitions
Retrieval	{relation, entity, no retrieval}	trigger knowledge retrieval
Relevance	{relevant, partially, unrelevant}	assess relevance with $q$
Rationality	{reasonable, partially, unreasonable}	evaluate logical coherence
Utility	{5, 4, 3, 2, 1}	$a$ is a useful response to $q$ .

Table 1. Four types of self-reflection tokens used in ArG . The exact token values are summarized in Appendix A.1 .

Panel of Figure 4 — Algorithm 1 ArG Workflow

$\clubsuit$ SP-based methods
Model	Hit@1 (%)
Model	WebQSP	CWQ
DECAF ( Yu et al., 2023 )	82.1	70.4
TIARA ( Shu et al., 2022 )	75.2	-
ArcaneQA ( Gu and Su, 2022 )	75.6	-
ChatKBQA ( Luo et al., 2024 )	86.4	86.0

Table 2. Results of different methods on WebQSP and CWQ. (We use underline to denote the second-best performance, bold to denote the best performance.

B

stands for beam-width and Exhausted means exhausted search.)

Figure 3. Ablation results of different reasoning depth and search depth on the WebQSP and CWQ.

Panel of Table 3 — Table 3. Ablation results for investigating the impact of special tokens, the check mark indicates the retention of the contribution of that particular token type to the final score.

Methods	Hit@1
Methods	WebQSP	CWQ
Ours ( $K=5$ )	90.2	72.4
w/o. hypo generator	85.1	69.3
w/o. naive retriever	87.2	70.3
ArG ( $K=3$ )	89.5	71.1

Table 4. Ablation results of different retrieval settings, where w/o. hypo-generator represents using the query directly as the retrieval input.

Methods	Hit@1 (%)
Methods	MetaQA-3hop
StructGPT ( Jiang et al., 2023a )	80.2
Retrieval & Reasoning ( Ji et al., 2024 )	76
RoG (train from scratch) ( Luo et al., 2023 )	84.81
RoG (transfer from Freebase) ( Luo et al., 2023 )	88.98
Ours (train from scratch)	87.7

Table 5. Results of transferability of ArG on MetaQA-3hop dataset based on Wiki-Movies KG.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 10 — Algorithm 2 $\mathcal{M}_{gen}$ Data Creation

LLM Reasoning with Graphs

Large language model reasoning with graphs uses structured relationships to support multi-step inference, evidence retrieval, and planning.

See all 210 papers

KG-Reasoner: A Reinforced Model for End-to-End Multi-Hop Knowledge Graph Reasoning
Apr 14, 2026Shuai Wang, Yinan YuLLM Reasoning with Graphs Graph Reasoning
Department of Computer Science and Engineering, Chalmers University of Technology and University of Gothenburg, SE-41296 Gothenburg, Sweden
GraphScout: Empowering Large Language Models with Intrinsic Exploration Ability for Agentic Graph Reasoning
Mar 2, 2026Yuchen Ying, Weiqi Jiang, Tongya Zheng +4LLM Reasoning with Graphs Agentic RAG
Zhejiang University, Hangzhou, China · Hangzhou City University, Hangzhou, China · Nanyang Technological University, Singapore, Singapore
RACER: Reinforced Agent Collaboration for Explainable Reasoning on Knowledge Graphs
Aug 29, 2026Yuwei Lou, Hao Hu, Yuzhou Jiang +5LLM Reasoning with Graphs Knowledge Graph Question Answering
State Key Laboratory for Novel Software Technology, Nanjing University, China · Independent Researcher · Chinaunicom Software Nanjing Branch

Apr 14, 2026J/K move · Enter open · S save

KG-Reasoner: A Reinforced Model for End-to-End Multi-Hop Knowledge Graph Reasoning

Shuai Wang, Yinan Yu

Department of Computer Science and Engineering, Chalmers University of Technology and University of Gothenburg, SE-41296 Gothenburg, Sweden

LLM Reasoning with Graphs Graph Reasoning Knowledge Graph Question Answering RL for Language Model Reasoning

Open paper AskPDF arXiv

Large Language Models (LLMs) exhibit strong abilities in natural language understanding and generation, yet they struggle with knowledge-intensive reasoning. Structured Knowledge Graphs (KGs) provide an effective form of external knowledge representation and have been widely used to enhance performance in classical Knowledge Base Question Answering (KBQA) tasks. However, performing precise multi-hop reasoning over KGs for complex queries remains highly challenging. Most existing approaches decompose the reasoning process into a sequence of isolated steps executed through a fixed pipeline. While effective to some extent, such designs constrain reasoning flexibility and fragment the overall decision process, often leading to incoherence and the loss of critical intermediate information from earlier steps. In this paper, we introduce KG-Reasoner, an end-to-end framework that integrates multi-step reasoning into a unified "thinking" phase of a Reasoning LLM. Through Reinforcement Learning (RL), the LLM is trained to internalize the KG traversal process, enabling it to dynamically explore reasoning paths, and perform backtracking when necessary. Experiments on eight multi-hop and knowledge-intensive reasoning benchmarks demonstrate that KG-Reasoner achieves competitive or superior performance compared to the state-of-the-art methods. Codes are available at the repository: https://github.com/Wangshuaiia/KG-Reasoner.

Abstract

LLM Reasoning with Graphs

Explore similar work

KG-Reasoner: A Reinforced Model for End-to-End Multi-Hop Knowledge Graph Reasoning

GraphScout: Empowering Large Language Models with Intrinsic Exploration Ability for Agentic Graph Reasoning

RACER: Reinforced Agent Collaboration for Explainable Reasoning on Knowledge Graphs