Retrieval-augmented methods improve factual accuracy by grounding language models in external knowledge, but retrieving over unstructured text often introduces irrelevant context and offers limited control over the retrieved information. Structured knowledge bases offer a more controllable alternative, yet they are expensive to construct and often brittle to reason over. To address these limitations, we propose KBevo: a co-evolving framework that jointly learns to construct a structured knowledge base and reason over it for knowledge-intensive question answering. By optimizing both components end-to-end with QA outcome rewards, our method enables reasoning success to directly improve the quality of the constructed knowledge base. This leads to larger, better-connected knowledge structures with higher answer reachability, while also improving compositional factual reasoning and controllability compared to standard retrieval baselines.
Figures & tables
Figure 1: Co-evolving knowledge and reasoning. Standard retrieval-based RL improves the reasoning policy while operating over fixed text chunks. In contrast, KBevo jointly optimizes reasoning and structured knowledge base construction, allowing the model and its structured knowledge base to improve together during training.
Method
Construction
Schema
Quality Control
Downstream-Coupled
Persistence
EDC
Extract–Define–Canonicalize
Induced
Canonicalization
✗
Persistent
Wikontic
Multi-stage pipeline
Wikidata ontology
Ontology filtering + dedup
✗
Persistent
AutoSchemaKG
Autonomous pipeline
Induced
Schema-guided filtering
✗
Persistent
Ours
Joint RL training
Emergent (task-driven)
QA reward signal
✓
Persistent + Editable
Table 1: Comparison of knowledge graph construction methods.
Figure 2: Overview of our co-evolving framework. The model first constructs a structured knowledge base from context, then answers multihop questions by retrieving and reasoning over it. Both phases are jointly optimized so that knowledge construction and reasoning improve together.
{Gb,k}b=1,k=1B,K∼πθ(⋅∣pkb(cb))∀b
Algorithm 1 Co-evolving KB Construction and QA via GRPO
Method
HotpotQA ∗
MuSiQue
2Wiki
PopQA
Avg
Qwen3-1.7B
Direct
12.1
0.7
21.7
16.0
12.6
RAG
25.4
6.3
21.8
57.8
27.8
IRCoT
23.7
12.2
44.0
60.7
35.2
Search-R1 †
44.1
14.3
45.9
65.3
42.4
KBevo -SFT
29.5
13.3
44.9
57.0
36.2
Table 2: Evaluation on multi-hop QA benchmarks and PopQA (EM). ∗ marks the in-domain benchmark. † Reimplemented on Qwen3 backbone. See Section 4 .
Figure 3: Reasoning with edited external knowledge on ConFiQA-MR (EM). KBevo maintains strong performance under counterfactual knowledge edits, showing that its structured KB can be directly updated and effectively used for reasoning without model retraining.
Figure 4: Co-evolution improves KB coverage and grounded reasoning. (a) GRPO improves answer reachability across hop depths and outperforms static KB construction methods. (b–c) GRPO also improves answer grounding and reasoning correctness over the constructed KB. The analysis is conducted on Qwen3-4B using 1K HotpotQA examples.
Method
HotpotQA ∗
MuSiQue
2Wiki
Avg
(a) Swap KB at inference (reasoning policy fixed)
KBevo -1.7B GRPO †
40.3
19.3
47.3
35.6
– EDC DB (Mistral-7B)
30.2
8.3
40.1
26.2
– AutoSchemaKG DB (Llama-3.1-8B)
29.0
16.1
28.8
24.6
– Gemini DB
38.1
19.0
54.5
37.2
(b) Train with a fixed KB (w/o Co-evolution)
Table 3: Ablating co-evolution. Our learned KB substantially outperforms KBs from static construction methods, while training with a fixed KB generally underperforms joint co-evolution. Results are EM on 1K examples per dataset due to the cost of Gemini-based KB construction. † KBevo results are evaluated on this same 1K subset and therefore differ slightly from the full-set results in Table 2 . ∗ marks the in-domain benchmark.
Figure 5: Effect of SFT warmup before RL. Starting RL directly from the base model leads to collapse in both reward and lookup usage, while even relatively early SFT checkpoints are sufficient to enter the structured reasoning regime and achieve similar improvement during RL.
Figure 6: A representative example. The model constructs a knowledge base from source documents and then reasons over it to answer a multi-hop question.
Built offline once per document, reused across queries
Amortized
Retrieval
top- k=4 MiniLM lookup, no LLM call
Per lookup
Appendix
Table 4: Cost breakdown of KBevo . The additional expense is confined to a one-time SFT warm-start and the K+K×M rollouts during GRPO. At inference the knowledge base is constructed offline once per document and reused across queries, so per-query cost is a single Phase-2 generation plus embedding lookups; no K×M sampling and no teacher model recur at inference.
Category
SFT
GRPO
Model & Data
Base model
Qwen3-1.7B / Qwen3-4B
SFT checkpoint (step 368 for 1.7B, step 735 for 4B)
Dataset
Gemini 2-phase trajectories (HotpotQA, 6k)
HotpotQA
Train size / Eval size
6k / –
7000 / 100
Use special DB lookup tokens
Yes
Yes
Optimization
Appendix
Table 5: Training configurations for KBevo on multi-hop QA. We first perform supervised fine-tuning (SFT) on two-phase KB-construction and QA trajectories, then continue joint optimization with GRPO.
Qwen3-1.7B
Qwen3-4B
Metric
SFT
GRPO
SFT
GRPO
KB Structure
∣E∣ (unique entities)
156.6
169.4
156.8
186.6
∣R∣ (unique relations)
111.3
135.9
109.7
175.3
#Triplets
166.2
225.9
165.7
299.1
Avg Degree
2.15
2.71
2.14
3.30
Appendix
Table 6: Knowledge base structure and lookup efficiency on HotpotQA † , averaged per example. Redundancy Rate is the fraction of lookups repeating an entity–relation pair already queried; Unknown Rate is the fraction matching no entry. Co-evolution yields larger, better-connected knowledge bases that the reasoning policy also queries more successfully. † These diagnostics were computed using an earlier set of model checkpoints and are retained for reference.
Triplet Quality
Graph Structure
Answer Reachability
KB Source
Prec. ↑
Rec. ↑
F1 ↑
∣R∣
#Comp. ↓
Giant % ↑
≤ 2 hop ↑
≤ 4 hop ↑
EDC (Mistral-7B)
0.864
0.477
0.600
–
–
–
63.7%
69.5%
AutoSchemaKG (Llama-3.1-8B)
0.888
0.838
0.860
58.8
18.8
51.0%
72.1%
80.8%
KBevo -1.7B GRPO
0.925
0.923
0.924
135.9
10.3
65.1%
81.3%
88.4%
KBevo -4B GRPO
0.938
0.952
0.945
175.3
7.9
73.8%
84.0%
90.4%
Appendix
Table 7: Intrinsic knowledge base quality on HotpotQA, independent of downstream QA. KBevo produces more accurate and better-connected KBs than static construction pipelines, with substantially higher multi-hop answer reachability.
Category
AutoSchemaKG
Qwen3-1.7B
Qwen3-4B
(Llama-3.1-8B)
SFT
GRPO
SFT
GRPO
Valid Triplets ↑
35.6%
92.3%
85.0%
96.0%
84.3%
Hallucinated ↓
11.0%
3.3%
7.5%
1.5%
6.2%
Soundness Issues ↓
53.4%
4.4%
7.4%
2.6%
9.5%
Appendix
Table 8: Triplet-level correctness on HotpotQA (100 samples), judged with the same LLM-as-judge protocol across all knowledge bases (Appendix B.4 ). Valid Triplets are free of hallucination and soundness issues. GRPO increases both hallucination and soundness-issue rates relative to SFT; we examine whether this explains the downstream gains in Appendix C.2 .
Post-hoc Filtering (EM)
Faithfulness by Reward Exposure
Model
No Filter
Filter Unfaithful
Used Faithful%
Used Cov.%
Unused Faithful%
Unused Cov.%
KBevo -1.7B SFT
–
–
97.6
5.3
96.8
94.7
KBevo -1.7B GRPO
36.2
38.4
94.3
5.3
92.7
94.7
KBevo -4B SFT
–
–
98.6
5.4
98.2
94.6
KBevo -4B GRPO
41.2
43.6
94.1
5.1
93.2
94.9
Appendix
Table 9: Faithfulness analysis of the constructed knowledge base on HotpotQA (1K examples). Left: removing triplets judged unfaithful slightly improves EM, indicating that hallucinated triplets act primarily as retrieval noise rather than an exploitable shortcut. Right: triplets retrieved during reasoning ( Used ) and never retrieved ( Unused ) show similar faithfulness under the same evaluation protocol (Appendix B.4 ), suggesting that sparse reward exposure does not substantially degrade the unretrieved portion of the knowledge base.
Model
Retrieval Faithfulness
Share%
EM
Fully Grounded
Partially Grounded
Ungrounded
KBevo -1.7B SFT
All faithful
84.0
33.0
49.6
10.6
30.2
KBevo -1.7B SFT
Has hallucinated
16.0
17.5
28.1
9.4
39.4
KBevo -1.7B GRPO
All faithful
60.7
40.3
59.7
11.1
27.1
KBevo -1.7B GRPO
Has hallucinated
39.3
24.2
41.6
10.2
39.5
KBevo -4B SFT
All faithful
90.7
35.0
50.1
8.8
27.2
KBevo -4B SFT
Has hallucinated
9.3
24.7
35.5
7.5
35.5
Appendix
Table 10: Faithfulness-conditioned grounding and QA performance on HotpotQA (1K examples). Reasoning chains are partitioned by whether all retrieved triplets are judged faithful or at least one is hallucinated. GRPO substantially improves both EM and grounding even when reasoning relies entirely on faithful triplets, indicating that the gains are not driven by hallucinated content. The fraction of chains retrieving at least one hallucinated triplet nevertheless increases under GRPO.
We propose a new method that allows an LLM to automatically pull in factual knowledge from a knowledge base during token generation. This means that (1)~factual knowledge in the LLM output can be updated without retraining the LLM, (2)~facts in the LLM output can be traced to the knowledge base for transparency and explainability, and (3)~smaller models can achieve the same factual accuracy as larger models. Our core idea is to train the model to produce special tokens that trigger a query to the knowledge base. Our experiments show that our method improves factual grounding in both short and long-form generation, and allows factual revisions to take effect through KB edits rather than parameter updates.
Francois Crespin, Fabian M. Suchanek, Nils Holzenberger
IP Paris, LTCI · Télécom Paris · Institut Polytechnique de Paris, France
Large language models (LLMs) have demonstrated strong capabilities in question answering, yet they still frequently suffer from hallucinations on knowledge-intensive tasks. Knowledge graphs (KGs) provide LLMs with structured, interpretable, and updatable factual grounding, making them a promising external knowledge source for reliable reasoning. However, existing LLM-guided graph reasoning methods typically rely on hop-wise greedy or beam-style pruning during evidence retrieval. Such local decision processes are inherently myopic: evidence that appears weak near the source may become crucial only after deeper graph context is explored, causing answer-critical branches to be discarded prematurely and making the reasoning chain difficult to recover. To address this limitation, we propose Foresight-over-Graph (FoG), a foresight-aware evidence retrieval framework for knowledge base question answering (KBQA). FoG iteratively constructs a question-relevant evidence subgraph and uses far-to-near feedback to guide path exploration, and maintains a compact memory subgraph to support continued exploration. Extensive experiments on widely used KBQA benchmarks demonstrate that FoG achieves state-of-the-art performance, with a particularly large improvement of 16.58% in Hit on CWQ, while also reducing LLM calls and token usage. Our code is available at https://github.com/yhong7/FoG .
Yang Hong, Yajun Yang, Xin Wang +2
Tianjin University, Tianjin, China · Beijing Jiaotong University, Beijing, China
Knowledge Base Question Answering (KBQA) challenges models to bridge the gap between natural language and strict knowledge graph schemas by generating executable logical forms. While Large Language Models (LLMs) have advanced this field, current approaches often struggle with a dichotomy of failure: they either generate hallucinated queries without verifying schema existence or exhibit rigid, template-based reasoning that mimics synthesized traces without true comprehension of the environment. To address these limitations, we present KBQA-R1, a framework that shifts the paradigm from text imitation to interaction optimization via Reinforcement Learning. Treating KBQA as a multi-turn decision process, our model learns to navigate the knowledge base using a list of actions, leveraging Group Relative Policy Optimization (GRPO) to refine its strategies based on concrete execution feedback rather than static supervision. Furthermore, we introduce Referenced Rejection Sampling (RRS), a data synthesis method that resolves cold-start challenges by strictly aligning reasoning traces with ground-truth action sequences. Extensive experiments on WebQSP, GrailQA, and GraphQuestions demonstrate that KBQA-R1 achieves state-of-the-art performance, effectively grounding LLM reasoning in verifiable execution.
Xin Sun, Zhongqi Chen, Xing Zheng +6
University of Science and Technology of China, Hefei, China · Independent · Institute of Automation, Chinese Academy of Sciences, Beijing, China.