Think Parallax: Solving Multi-Hop Problems via Multi-View Knowledge-Graph-Based Retrieval-Augmented Generation

Jinliang Liu; Jiale Bai; Shaoning Zeng

cs.CLOct 17, 2025

Think Parallax: Solving Multi-Hop Problems via Multi-View Knowledge-Graph-Based Retrieval-Augmented Generation

Authors: Jinliang Liu, Jiale Bai, Shaoning Zeng

Organizations: Yangtze Delta Region Institute (Huzhou), University of Electronic Science and Technology of China, Huzhou, Zhejiang, China · School of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan, China

Abstract

Large language models (LLMs) still struggle with multi-hop reasoning over knowledge-graphs (KGs), and we identify a previously overlooked structural reason for this difficulty: Transformer attention heads naturally specialize in distinct semantic relations across reasoning stages, forming a hop-aligned relay pattern. This key finding suggests that multi-hop reasoning is inherently multi-view, yet existing KG-based retrieval-augmented generation (KG-RAG) systems collapse all reasoning hops into a single representation, flat embedding space, suppressing this implicit structure and causing noisy or drifted path exploration. We introduce ParallaxRAG, a symmetric multi-view framework that decouples queries and KGs into aligned, head-specific semantic spaces. By enforcing relational diversity across multiple heads while constraining weakly related paths, ParallaxRAG constructs more accurate, cleaner subgraphs and guides LLMs through grounded, hop-wise reasoning. On WebQSP and CWQ, it achieves state-of-the-art retrieval and QA performance, substantially reduces hallucination, and generalizes strongly to the biomedical BioASQ benchmark.

Figures & tables

Figure 1: Comparison of ParallaxRAG and standard KG-RAG embedding strategies. (a) In ParallaxRAG, attention heads specialize in distinct semantic relations across reasoning stages, where early heads capture shallow relational patterns while later heads handle longer-hop dependencies. (b) In standard KG-RAG, all reasoning hops are collapsed into a single embedding space, suppressing hop-specific semantic structure.

Figure 2: The architecture of ParallaxRAG, illustrating its three main stages: Preprocess, Retrieval, and Generate. The core retrieval process begins by creating specialized multi-view representations for both the graph (b) and the question (c), a process guided by directional distance encoding and regularized by pairwise similarity (a). Next, candidate triples are scored in parallel across these diverse views (d). Finally, a query-aware gating mechanism computes head importance (e) to produce a weighted aggregation of the scores, identifying the most relevant triples for the final generation stage.

Model	Shortest Path Triple Recall					GPT-4o Triple Recall					Answer Entity Recall
	WebQSP		CWQ			WebQSP		CWQ			WebQSP		CWQ
	1	2	1	2	$\geq 3$	1	2	1	2	$\geq 3$	1	2	1	2	$\geq 3$
	(65.8%)	(34.2%)	(28.0%)	(65.9%)	(6.1%)	(65.8%)	(34.2%)	(28.0%)	(65.9%)	(6.1%)	(65.8%)	(34.2%)	(28.0%)	(65.9%)	(6.1%)
cosine similarity	0.874	0.405	0.629	0.442	0.333	0.847	0.483	0.629	0.511	0.464	0.943	0.253	0.903	0.472	0.289
SR+NSM w E2E	0.565	0.324	-	-	-	0.580	0.376	-	-	-	0.916	0.301	-	-	-
Retrieve-Rewrite-Answer	0.064	0.046	-	-	-	0.062	0.061	-	-	-	0.745	0.729	-	-	-

Table 1: Main retrieval recall evaluation results for different models on WebQSP and CWQ datasets.

	WebQSP			CWQ
Model	Macro-F1	Hit	Hallu.	Macro-F1	Hit	Hallu.
(A) Neural Methods
UniKGQA	72.2	-	-	49.0	-	-
SR+NSM w E2E	64.1	-	64.44	46.3	-	-
(B) Multi-turn LLM Reasoning Methods
KD-CoT	52.5	68.6	-	-	55.7	-

Table 2: Question-answering performance on WebQSP and CWQ. The Hallucination (Hallu.) score is evaluated on a subset that excludes samples where the answer entity is not in the KG, following Li et al. (2025) . Our generaters use the top 100 retrieved triples by default; results for 200 and 500 (indicated in parentheses) are also shown. Best results are in bold . Results with (

\leftrightarrow

) evaluate retriever generalizability, where the retriever is trained on one dataset and applied to the other.

Panel of Figure 3 — Figure 3: Visualization of attention head specialization, revealing a task-adaptive division of labor. The heatmaps display three metrics (Contribution, Use Rate, Hit Rate) for the top 5 specialist heads, selected from our BGE-based model’s 16 heads according to their overall contribution. The Others row aggregates the remaining 11 heads: for Contribution, this value is the sum of their contributions, while for Use Rate and Hit Rate, it is their average, see more explanation in (Appendix C ). We define steps as the BFS expansion depth along reasoning paths, rather than the conventional hop (shortest-path length), in order to capture head behaviors at each layer of candidate exploration. (a) On WebQSP (short chains). (b) On CWQ (long chains).

	Answer Entity Recall
Condition	1-hop	2-hop	$\geq$ 3-hop	Overall
ParallaxRAG Baseline	0.959	0.926	0.763	0.883
Drop Long-Hop Heads	0.958	0.777	0.501	0.745
Shuffle Heads	0.734	0.532	0.468	0.578
All	0.734	0.446	0.421	0.534

Table 3: Answer Entity Recall under controlled disruption on CWQ, broken down by hop depth. Long-hop-associated heads are functionally non-substitutable.

Model	Yes/No Questions				Factoid Questions			List Questions
Model	Acc.	F1 Yes	F1 No	Macro F1	Strict	Lenient	MRR	Prec.	Recall	F1
Deepseek-R1-8B	0.9077	0.9312	0.8558	0.8935	0.2620	0.2733	0.2677	0.3982	0.3427	0.3517
GPT4-Turbo	0.9129	0.9357	0.8616	0.8986	0.5505	0.6515	0.6010	0.5788	0.4857	0.5051
GPT-4o	0.9140	0.9347	0.8670	0.9009	0.3462	0.3462	0.3462	0.5102	0.4025	0.4330
Qwen3-Plus	0.9012	0.9255	0.8402	0.8829	0.3219	0.6213	0.4132	0.5682	0.3989	0.4563
Single-View + Qwen3-Plus	0.9112	0.9267	0.8374	0.8821	0.3773	0.7214	0.8462	0.6833	0.4280	0.4464

Table 4: Generalization on BioASQ Task B, where the retriever is trained on CWQ and transferred without fine-tuning. We compare the performance of Qwen3-Plus, Single-View and ParallaxRAG (Multi-View), isolating the contribution of the retrieval design

		WebQSP		CWQ
	Configuration	Macro-F1	Hit	Macro-F1	Hit
	ParallaxRAG (Full Model)	71.73	86.85	48.33	58.41
Multi-Head Architecture Validation
(a)	Split Vector Baseline	69.42 ( $\downarrow$ 2.31)	85.02 ( $\downarrow$ 1.83)	42.27 ( $\downarrow$ 6.06)	51.54 ( $\downarrow$ 6.87)
(b)	Single Vector Baseline	70.60 ( $\downarrow$ 1.13)	86.11 ( $\downarrow$ 0.74)	44.82 ( $\downarrow$ 3.51)	52.63 ( $\downarrow$ 5.78)
Synergistic Mechanism Validation

Table 5: Ablation study on WebQSP and CWQ, analyzing the impact of ParallaxRAG’s core multi-head architecture and its synergistic mechanisms. Values in parentheses (

\downarrow

) indicate the performance drop compared to the full model (Llama3.1-8B as generator).

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Sensitivity analysis of the PSR strength parameter $\beta$ on the generalization task, where the retriever was trained on WebQSP and tested on CWQ-sub, using Llama-3.1-8B as the generator.

Panel of Figure 5 — Figure 5: Attention Head Specialization Analysis on CWQ Dataset with Alternative Backbone Encoders. (a) Results using the E5-large-v2 model. (b) Results using the GTE-large model.

Encoder	WebQSP		CWQ
	Macro-F1	Hit	Macro-F1	Hit
E5	69.23	85.63	44.59	53.95
GTE	69.75	85.92	47.12	57.48

Appendix

Table 6: End-to-End QA Performance with Alternative Backbone Encoders using Llama3.1-8B as KGQA generator

	WebQSP			CWQ
Model	Macro-F1	Hit	Hallu.	Macro-F1	Hit	Hallu.
ParallaxRAG	75.97 $\pm$ 0.88	92.36 $\pm$ 1.60	75.85 $\pm$ 0.63	59.15 $\pm$ 1.39	66.08 $\pm$ 1.92	57.72 $\pm$ 1.01

Appendix

Table 7: Performance stability across 5 independent runs on WebQSP and CWQ (Qwen3-30B generator). Standard deviations are all below 2.0 points.

Method	1-hop	2-hop	$\geq$ 3-hop	Retrieval	KG Access	Generation
ParallaxRAG	22.10 $\pm$ 7.80	70.47 $\pm$ 14.28	–	38.78	0.023	3.124 $\pm$ 0.232

Appendix

Table 8: Per-stage latency (seconds/query, mean

\pm

std) on WebQSP. “Retrieval (mean)” is averaged over all queries; hop-specific values reflect subgraph complexity.

Method	1-hop	2-hop	$\geq$ 3-hop	Retrieval	KG Access	Generation
ParallaxRAG	25.58 $\pm$ 8.20	87.95 $\pm$ 17.17	183.63 $\pm$ 63.01	82.35	0.026	3.784 $\pm$ 0.227

Appendix

Table 9: Per-stage latency (seconds/query, mean

\pm

std) on CWQ.

Figure 6: Comparison of retrieved subgraphs and reasoning chains for case WebQTrn-241

Figure 7: Comparison of retrieved subgraphs and reasoning chains for case WebQTest-626

Figure 8: Comparison of retrieved subgraphs and reasoning chains for case WebQTest-810

Multi-Hop QA

Multi-hop question answering studies how systems combine evidence across documents, facts, or modalities to answer questions requiring linked reasoning.

See all 228 papers

KG-Hopper: Empowering Compact Open LLMs with Knowledge Graph Reasoning via Reinforcement Learning
Mar 22, 2026Shuai Wang, Yinan YuLLM Reasoning with Graphs Knowledge Graph Question Answering
Department of Computer Science and Engineering, Chalmers University of Technology and University of Gothenburg, SE-41296 Gothenburg, Sweden
KG-Reasoner: A Reinforced Model for End-to-End Multi-Hop Knowledge Graph Reasoning
Apr 14, 2026Shuai Wang, Yinan YuLLM Reasoning with Graphs Graph Reasoning
Department of Computer Science and Engineering, Chalmers University of Technology and University of Gothenburg, SE-41296 Gothenburg, Sweden
PAR $^2$ -RAG: Planned Active Retrieval and Reasoning for Multi-Hop Question Answering
Mar 30, 2026Xingyu Li, Rongguang Wang, Yuying Wang +5Retrieval-Augmented Reasoning Multi-Document QA
Oracle AI

Mar 22, 2026J/K move · Enter open · S save

KG-Hopper: Empowering Compact Open LLMs with Knowledge Graph Reasoning via Reinforcement Learning

Shuai Wang, Yinan Yu

Department of Computer Science and Engineering, Chalmers University of Technology and University of Gothenburg, SE-41296 Gothenburg, Sweden

LLM Reasoning with Graphs Knowledge Graph Question Answering Multi-Hop Reasoning RL for Language Model Reasoning

Open paper AskPDF arXiv

Large Language Models (LLMs) demonstrate impressive natural language capabilities but often struggle with knowledge-intensive reasoning tasks. Knowledge Base Question Answering (KBQA), which leverages structured Knowledge Graphs (KGs) exemplifies this challenge due to the need for accurate multi-hop reasoning. Existing approaches typically perform sequential reasoning steps guided by predefined pipelines, restricting flexibility and causing error cascades due to isolated reasoning at each step. To address these limitations, we propose KG-Hopper, a novel Reinforcement Learning (RL) framework that empowers compact open LLMs with the ability to perform integrated multi-hop KG reasoning within a single inference round. Rather than reasoning step-by-step, we train a Reasoning LLM that embeds the entire KG traversal and decision process into a unified ``thinking'' stage, enabling global reasoning over cross-step dependencies and dynamic path exploration with backtracking. Experimental results on eight KG reasoning benchmarks show that KG-Hopper, based on a 7B-parameter LLM, consistently outperforms larger multi-step systems (up to 70B) and achieves competitive performance with proprietary models such as GPT-3.5-Turbo and GPT-4o-mini, while remaining compact, open, and data-efficient. The code is publicly available at: https://github.com/Wangshuaiia/KG-Hopper.

Abstract

Multi-Hop QA

Explore similar work

KG-Hopper: Empowering Compact Open LLMs with Knowledge Graph Reasoning via Reinforcement Learning

KG-Reasoner: A Reinforced Model for End-to-End Multi-Hop Knowledge Graph Reasoning

PAR2^22-RAG: Planned Active Retrieval and Reasoning for Multi-Hop Question Answering

PAR $^2$ -RAG: Planned Active Retrieval and Reasoning for Multi-Hop Question Answering