NeuroSymActive: Differentiable Neural-Symbolic Reasoning with Active Exploration for Knowledge Graph Question Answering

Rong Fu; Yang Li; Zeyu Zhang; Jiekai Wu; Yaohua Liu; Shuaishuai Cao; Yangchen Zeng; Yuhang Zhang

cs.CLFeb 17, 2026

NeuroSymActive: Differentiable Neural-Symbolic Reasoning with Active Exploration for Knowledge Graph Question Answering

Authors: Rong Fu, Yang Li, Zeyu Zhang, Jiekai Wu, Yaohua Liu, Shuaishuai Cao, Yangchen Zeng, Yuhang Zhang, +2 more

Organizations: Independent Researcher

Abstract

Large pretrained language models and neural reasoning systems have advanced many natural language tasks, yet they remain challenged by knowledge-intensive queries that require precise, structured multi-hop inference. Knowledge graphs provide a compact symbolic substrate for factual grounding, but integrating graph structure with neural models is nontrivial: naively embedding graph facts into prompts leads to inefficiency and fragility, while purely symbolic or search-heavy approaches can be costly in retrievals and lack gradient-based refinement. We introduce NeuroSymActive, a modular framework that combines a differentiable neural-symbolic reasoning layer with an active, value-guided exploration controller for Knowledge Graph Question Answering. The method couples soft-unification style symbolic modules with a neural path evaluator and a Monte-Carlo style exploration policy that prioritizes high-value path expansions. Empirical results on standard KGQA benchmarks show that NeuroSymActive attains strong answer accuracy while reducing the number of expensive graph lookups and model calls compared to common retrieval-augmented baselines.

Figures & tables

Figure 1: Architectural overview of the NeuroSymActive framework for knowledge graph question answering. The framework operates via a coupled dual-loop optimization process across three main stages: Stage 1: Uncertainty-Aware Active Retrieval , which utilizes a Bayesian head to model heteroscedastic uncertainty in hop prediction and a neural entropy predictor $\eta_{\theta}$ to estimate information gain ( $IG$ ). This stage selectively invokes a Human Oracle via the $\mathrm{QUERY\_HUMAN}$ action when uncertainty $\mathcal{U}(\mathbf{s}_{t})$ exceeds the threshold $\tau_{\mathrm{hop}}$ . Stage 2: Differentiable Neural-Symbolic Fusion , where a hybrid Knowledge Adapter merges continuous path embeddings ( $\mathbf{z}_{f}^{\mathrm{neural}}$ ) with soft symbolic plausibility scores ( $\mathbf{z}_{f}^{\mathrm{sym}}$ ) derived from a Differentiable Inductive Logic Layer (DILL) . Rule confidences $w_{\rho}$ are updated via gradient signals from active supervision stored in the human replay buffer $\mathcal{D}_{\mathrm{human}}$ . Stage 3: Differentiable MCTS with Active Exploration , which implements a relaxed Monte Carlo Tree Search. It employs Progressive Widening governed by predictive uncertainty and treats human intervention as explicit nodes with value $V_{\mathrm{human}}(s)$ . The search is fully differentiable via Gumbel-Softmax relaxations, allowing end-to-end joint training of the policy $\pi_{\phi}$ , value $v_{\psi}$ , and symbolic weights through a composite multi-objective loss $\mathcal{L}_{\mathrm{total}}$ .

Method	WebQSP (Hits@1)	CWQ (Hits@1)
KV-Mem [ 65 ]	46.7	18.4
EmbedKGQA [ 66 ]	66.6	45.9
NSM [ 67 ]	68.7	47.6
KGT5 [ 68 ]	56.1	36.5
GraftNet [ 53 ]	66.4	–
PullNet [ 52 ]	68.1	–

Table 1: Performance comparison between NeuroSymActive, LightPROF and representative baselines on WebQSP and CWQ. Bold indicates the best reported result.

Variant	WebQSP	CWQ
	(Hits@1)	(Hits@1)
Full NeuroSymActive	87.1	62.5
Core mechanisms
w/o Knowledge Adapter	79.5	54.2
w/o Uncertainty-aware triggering	83.4	59.8
w/o Progressive Widening (fixed $k$ =3)	84.6	60.3

Table 2: Extended ablation: disentangling architectural choices.

Encoder Type	WebQSP	CWQ
Triplet-Only (H, R, T)	84.3	59.0
Context-Aware (order-sensitive)	87.1	62.5
Relation-Path (path-centric)	84.7	60.8

Table 3: Effect of structure-encoder design on NeuroSymActive.

Dataset	Retrieval (%)	Reasoning (%)	Generation (%)
WebQSP	31.2	42.8	26.0
CWQ	48.5	32.4	19.1

Table 4: Error mode distribution across datasets. Percentages denote proportions of incorrect predictions.

Figure 2: Reduction in error rates for each failure mode as annotation budget increases. Confidence intervals obtained by bootstrapping.

Figure 3: Composition of residual errors under strong active supervision. Components include KG incompleteness, question ambiguity, and entropy-predictor bias.

Component	Observed trace / output
Question	Which films starring Bruce Willis were directed by a filmmaker born in Germany?
Ground truth	[“Die Hard”, “The Fifth Element”]
Reasoning (selected steps)	Anchor extraction found “Bruce Willis”. Hop prediction indicated three hops. During exploration, uncertainty rose for a candidate relation (director nationality) and the system invoked QueryHuman ; human feedback confirmed relevance. The differentiable logic layer pruned implausible paths and prioritized validated relation links. The adapter produced a compact soft prompt; the LLM generated the final answer based on the pruned subgraph.
Final answer	[“Die Hard”, “The Fifth Element”] (correct)

Table 5: Illustrative case: NeuroSymActive reasoning trace for a complex WebQSP query.

Backbone (Framework)	WebQSP	CWQ	Gain
	(Hits@1)	(Hits@1)	(avg)
LLaMa2-7B	61.4	31.5	+19.1
LLaMa2-7B (LightPROF)	71.2	48.5
LLaMa2-7B (NeuroSymActive)	76.3	54.8
LLaMa3-8B	66.8	48.9	+17.0
LLaMa3-8B (LightPROF)	83.8	59.3

Table 6: Performance comparison of different LLM backbones integrated with the LightPROF and NeuroSymActive frameworks.

Method	Time (H:M)	Tokens used	NPR
NeuroSymActive (LLaMa3-8B)	1:05:30	338,200	207
StructGPT (ChatGPT)	1:42:12	24,750,610	6400

Table 7: Efficiency comparison on WebQSP: NeuroSymActive vs. StructGPT. NPR = average tokens per request.

Error mode	Question	System output	Analysis
Retrieval	What language is spoken in the country where the film “Inception” was produced?	English (incorrect)	Hop-depth underestimated; a depth-query before retrieval would prevent this omission.
Reasoning	Which actor starred in both “Titanic” and “The Revenant”?	Johnny Depp (incorrect)	Correct path present but misranked by the value estimator; relation relevance verification does not fully resolve path-ranking ambiguity.
Generation	Who directed the sequel to “The Matrix”?	The Wachowskis, James Cameron (incorrect)	Correct supporting path was provided; generation introduced a spurious director due to prompt ambiguity and entity-linking noise.

Table 8: Representative failure cases by error mode. Suggested human-query opportunities are noted where applicable.

$\beta$	Hits@1 (%)	Query Rate	$\Delta$ Accuracy
0.1	87.1	2.8	baseline
0.5	86.9	1.6	$-0.2$
1.0	86.4	1.0	$-0.7$
2.0	84.2	0.6	$-2.9$
5.0	81.5	0.4	$-5.6$
10.0	78.3	0.3	$-8.8$

Table 9: Sensitivity to human-cost penalty

\beta

on WebQSP. “Query rate” denotes average human invocations per question.

Temperature $\tau$	Hits@1 (%)	Training iterations to convergence
2.0	82.4	18,500
1.0	84.7	12,200
0.5	86.2	9,800
0.1	87.1	8,400
STE (hard)	85.3	11,600

Table 10: Influence of Gumbel temperature

\tau

on WebQSP performance and training stability.

$(\lambda_{1},\lambda_{2},\lambda_{3})$	WebQSP	CWQ
$(0.3,0.5,0.2)$	87.1	62.5
$(0.0,0.5,0.2)$	84.5	58.9
$(0.3,0.0,0.2)$	83.2	57.4
$(0.3,0.5,0.0)$	85.8	60.1
$(0.5,0.3,0.2)$	86.3	61.2
$(0.2,0.6,0.2)$	86.7	61.8

Table 11: Ablation over multi-objective weights

\lambda_{1:3}

. Default is

(0.3,0.5,0.2)

Query strategy	Avg. queries per question	Hits@1 (%)	Efficiency (%↑ / query)
Random sampling	3.0	82.4	2.8
Fixed-interval	2.0	84.1	3.2
Uncertainty-only ( $\tau_{\mathrm{human}}=0.9$ )	0.8	83.6	6.5
Uncertainty-only ( $\tau_{\mathrm{human}}=0.7$ )	1.2	86.4	5.2
Uncertainty-only ( $\tau_{\mathrm{human}}=0.5$ )	2.1	87.0	3.1
Adaptive (NeuroSymActive)	1.2	87.1	5.8

Table 12: Annotation economy on WebQSP. See text for interpretation.

Figure 4: Accuracy versus annotation cost as controlled by uncertainty threshold $\tau_{\mathrm{human}}$ . The adaptive strategy is marked.

Figure 5: Marginal information gain across successive human queries within episodes. Shaded area denotes variance across queries.

Supervision mode	Hits@1 (%)	Total queries (K)	Efficiency ( $\times 10^{-3}$ )
Unsupervised (no human)	80.9	0	—
Passive (fixed schedule)	84.1	9.5	3.4
Active (random selection)	85.3	7.1	6.2
Active (uncertainty-only)	86.4	5.7	9.6
Active (information-gain)	87.1	5.2	11.9
Fully supervised	88.3	23.8	3.1

Table 13: Comparison of supervision paradigms on WebQSP. Efficiency is accuracy gain per query normalized by annotation count.

Figure 6: Accuracy under varying annotation budgets. NeuroSymActive outperforms uncertainty-agnostic baselines across budgets.

Figure 7: Normalized loss-term magnitudes across training iterations. Shaded bands show seed-wise variation.

Figure 8: Evolution of rule confidences in DILL, grouped by rule category. Solid lines denote means; shading indicates distribution density.

Loss component	Final value	CV	Gradient norm
$\mathcal{L}_{\mathrm{answer}}$	0.847	0.08	2.34
$\mathcal{L}_{\mathrm{explore}}$	0.623	0.12	1.89
$\mathcal{L}_{\mathrm{symbolic}}$	0.291	0.09	3.12
$\mathcal{L}_{\mathrm{active}}$	0.445	0.11	1.56

Table 14: Convergence statistics for loss components. CV denotes coefficient of variation over the final 1,000 iterations.

$(k,\alpha)$	Hits@1 (%)	Avg. nodes	Avg. depth	Branching factor	Rollout time (ms)
(1.5, 0.4)	85.2	24.3	2.8	4.2	12.4
(2.5, 0.5)	87.1	38.7	3.4	5.9	18.6
(4.0, 0.6)	87.4	67.2	4.2	7.8	31.2
Fixed $k=3$	84.6	45.1	3.1	6.2	22.3

Table 15: Progressive widening sensitivity on WebQSP. Tree statistics averaged over 500 evaluation queries.

Figure 9: Distribution of node expansions by tree depth and query hop count. Human query nodes marked separately.

Figure 10: Effective branching factor versus visit count, stratified by uncertainty quartiles; dashed line shows the non-adaptive baseline.

Explore similar work

Search-on-Graph-R1: Training Large Language Models to Search Knowledge Graphs with Reinforcement Learning
Jul 20, 2026Jia Ao Sun, Hao Yu, Fengran Mo +4LLM Reasoning with Graphs RL for Language Model Reasoning
Université de Montréal · Mila – Québec AI Institute · McGill University +2
Better Later Than Sooner: Neuro-Symbolic Knowledge Graph Construction via Ontology-grounded Post-extraction Correction
May 27, 2026Lorenzo Loconte, Timothy Hospedales, Cristina CornelioKG Construction Neuro-Symbolic AI
University of Edinburgh, UK · Samsung AI Center, Cambridge, UK
Code-on-Graph: Iterative Programmatic Reasoning via Large Language Models on Knowledge Graphs
Jun 2, 2026Weiwei Ding, Zixuan Li, Long Bai +7LLM Grounding LLM Reasoning with Graphs
1Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences · 2Shandong University · 3Shandong University-Weihai Research Institute of Industrial Technology

Jul 20, 2026J/K move · Enter open · S save

Search-on-Graph-R1: Training Large Language Models to Search Knowledge Graphs with Reinforcement Learning

Jia Ao Sun, Hao Yu, Fengran Mo, Zhan Su+3

Université de Montréal · Mila – Québec AI Institute · McGill University+2

LLM Reasoning with Graphs RL for Language Model Reasoning Graph-Based Retrieval LLM Tool Use Knowledge Graph Question Answering

Open paper AskPDF arXiv

Knowledge graph question answering (KGQA) requires navigating from topic entities to an answer several relations away. Recent methods prompt a frontier LLM to explore the graph through a retrieval tool, but their reliance on frontier-scale inference makes them costly to deploy. We present Search-on-Graph-R1 (\sogrone{}), which internalizes this navigation into a compact 8B model through supervised fine-tuning (SFT) followed by reinforcement learning (RL). Our central idea is to scaffold a frontier teacher with each question's gold SPARQL query, so the teacher traverses a known answer-bearing path with a live Search tool rather than having to discover the path itself. Since every call executes against a live Freebase server, the resulting trajectories are grounded in the knowledge graph by construction. On WebQSP, CWQ, and GrailQA, \sogrone{} at 8B surpasses every frozen frontier-LLM system in our comparison and posts the strongest results on CWQ of any system we compare against. It does so using no auxiliary module at inference and no LLM judge during training. Isolating each training stage shows that SFT and RL contribute complementary gains, our approach transfers across model families, and RL learns to reach answers in fewer Search calls than its SFT initialization.

Abstract

Explore similar work

Search-on-Graph-R1: Training Large Language Models to Search Knowledge Graphs with Reinforcement Learning

Better Later Than Sooner: Neuro-Symbolic Knowledge Graph Construction via Ontology-grounded Post-extraction Correction

Code-on-Graph: Iterative Programmatic Reasoning via Large Language Models on Knowledge Graphs