Extensive research has investigated the integration of large language models (LLMs) with knowledge graphs to enhance the reasoning process. However, understanding how models perform reasoning utilizing structured graph knowledge remains underexplored. Most existing approaches rely on LLMs or retrievers to make binary judgments regarding the utilization of knowledge, which is too coarse. Meanwhile, there is still a lack of feedback mechanisms for reflection and control throughout the entire reasoning path. This paper proposes an Active self-Reflection framework for knowledge Graph reasoning (ArG), introducing for the first time an end-to-end training approach to achieve iterative reasoning grounded on structured graphs. Within the framework, the model leverages special tokens to actively control the reasoning process, achieving on-demand retrieval and multi-dimensional, finer-grained knowledge evaluation. The reasoning paths generated by the model exhibit high interpretability, enabling deeper exploration of the model's understanding of structured knowledge. Ultimately, the proposed model achieves outstanding results compared to existing baselines in knowledge graph reasoning tasks.
Figures & tables
Figure 1. An example of ArG performs a more fine-grained assessment and actively retrieves knowledge compared to LLM pruning and direct retrieval. ArG not only provides reasonable evaluations, but also captures useful indirect relationships.
Figure 2. The overall framework of ArG . Given an input query, the trained generator model M iteratively performs knowledge retrieval over the structural graph based on the retrieval token. Subsequently, the retrieved knowledge undergoes processes of critique and reflection, where implausible information is filtered. The iterative procedure culminates in the generation of an answer. ArG exhibits strong interpretability when applied to structured graph. As demonstrated in the example, the step-by-step reasoning path is organized in the lower half.
Token Type
Category
Definitions
Retrieval
{relation, entity, no retrieval}
trigger knowledge retrieval
Relevance
{relevant, partially, unrelevant}
assess relevance with q
Rationality
{reasonable, partially, unreasonable}
evaluate logical coherence
Utility
{5, 4, 3, 2, 1}
a is a useful response to q .
Table 1. Four types of self-reflection tokens used in ArG . The exact token values are summarized in Appendix A.1 .
Algorithm 1 ArG Workflow
Model
Hit@1 (%)
WebQSP
CWQ
♣ SP-based methods
DECAF ( Yu et al., 2023 )
82.1
70.4
TIARA ( Shu et al., 2022 )
75.2
-
ArcaneQA ( Gu and Su, 2022 )
75.6
-
ChatKBQA ( Luo et al., 2024 )
86.4
86.0
Table 2. Results of different methods on WebQSP and CWQ. (We use underline to denote the second-best performance, bold to denote the best performance. B stands for beam-width and Exhausted means exhausted search.)
Figure 3. Ablation results of different reasoning depth and search depth on the WebQSP and CWQ.
Table 3. Ablation results for investigating the impact of special tokens, the check mark indicates the retention of the contribution of that particular token type to the final score.
Methods
Hit@1
WebQSP
CWQ
Ours ( K=5 )
90.2
72.4
w/o. hypo generator
85.1
69.3
w/o. naive retriever
87.2
70.3
ArG ( K=3 )
89.5
71.1
Table 4. Ablation results of different retrieval settings, where w/o. hypo-generator represents using the query directly as the retrieval input.
Methods
Hit@1 (%)
MetaQA-3hop
StructGPT ( Jiang et al., 2023a )
80.2
Retrieval & Reasoning ( Ji et al., 2024 )
76
RoG (train from scratch) ( Luo et al., 2023 )
84.81
RoG (transfer from Freebase) ( Luo et al., 2023 )
88.98
Ours (train from scratch)
87.7
Table 5. Results of transferability of ArG on MetaQA-3hop dataset based on Wiki-Movies KG.
Large Language Models (LLMs) exhibit strong abilities in natural language understanding and generation, yet they struggle with knowledge-intensive reasoning. Structured Knowledge Graphs (KGs) provide an effective form of external knowledge representation and have been widely used to enhance performance in classical Knowledge Base Question Answering (KBQA) tasks. However, performing precise multi-hop reasoning over KGs for complex queries remains highly challenging. Most existing approaches decompose the reasoning process into a sequence of isolated steps executed through a fixed pipeline. While effective to some extent, such designs constrain reasoning flexibility and fragment the overall decision process, often leading to incoherence and the loss of critical intermediate information from earlier steps. In this paper, we introduce KG-Reasoner, an end-to-end framework that integrates multi-step reasoning into a unified "thinking" phase of a Reasoning LLM. Through Reinforcement Learning (RL), the LLM is trained to internalize the KG traversal process, enabling it to dynamically explore reasoning paths, and perform backtracking when necessary. Experiments on eight multi-hop and knowledge-intensive reasoning benchmarks demonstrate that KG-Reasoner achieves competitive or superior performance compared to the state-of-the-art methods. Codes are available at the repository: https://github.com/Wangshuaiia/KG-Reasoner.
Shuai Wang, Yinan Yu
Department of Computer Science and Engineering, Chalmers University of Technology and University of Gothenburg, SE-41296 Gothenburg, Sweden
Knowledge graphs provide structured and reliable information for many real-world applications, motivating increasing interest in combining large language models (LLMs) with graph-based retrieval to improve factual grounding. Recent Graph-based Retrieval-Augmented Generation (GraphRAG) methods therefore introduce iterative interaction between LLMs and knowledge graphs to enhance reasoning capability. However, existing approaches typically depend on manually designed guidance and interact with knowledge graphs through a limited set of predefined tools, which substantially constrains graph exploration. To address these limitations, we propose GraphScout, a training-centric agentic graph reasoning framework equipped with more flexible graph exploration tools. GraphScout enables models to autonomously interact with knowledge graphs to synthesize structured training data which are then used to post-train LLMs, thereby internalizing agentic graph reasoning ability without laborious manual annotation or task curation. Extensive experiments across five knowledge-graph domains show that a small model (e.g., Qwen3-4B) augmented with GraphScout outperforms baseline methods built on leading LLMs (e.g., Qwen-Max) by an average of 16.7% while requiring significantly fewer inference tokens. Moreover, GraphScout exhibits robust cross-domain transfer performance. Our code will be made publicly available~\footnote{https://github.com/Ying-Yuchen/_GraphScout_}.
Yuchen Ying, Weiqi Jiang, Tongya Zheng +4
Zhejiang University, Hangzhou, China · Hangzhou City University, Hangzhou, China · Nanyang Technological University, Singapore, Singapore
Large Language Models (LLMs) often suffer from hallucination and struggle with complex reasoning tasks requiring multi-hop domain knowledge. While integrating Knowledge Graphs (KGs) provides a structured and verifiable information source, current KG-enhanced LLM paradigms usually rely on single-agent path extraction and fixed prompting, lacking adaptability and facing huge search spaces. To address these challenges, we propose RACER, a Reinforced Agent Collaboration framework for Explainable Reasoning on knowledge graphs. RACER employs a semantic-aware action pruning and teacher-guided reinforcement learning mechanism to efficiently extract high-quality reasoning pathways from large-scale KGs. Furthermore, to mitigate single-path generation pitfalls, we introduce a cross-task accumulated shared memory graph paired with an attention-driven multi-path knowledge refinement module. Finally, RACER orchestrates these components through a four-role multi-agent collaboration system (GraphAgent, TemplateAgent, AnswerAgent, and CriticAgent) to dynamically refine prompts and evaluate answers. Extensive experiments on CommonsenseQA and OpenBookQA datasets demonstrate that RACER significantly outperforms state-of-the-art KG-enhanced LLM baselines with an average improvement of 5%, offering robust and highly interpretable reasoning capabilities.
Yuwei Lou, Hao Hu, Yuzhou Jiang +5
State Key Laboratory for Novel Software Technology, Nanjing University, China · Independent Researcher · Chinaunicom Software Nanjing Branch