Knowledge Graphs (KGs) provide a critical foundation for many AI systems, yet their automated construction inevitably introduces noisy triples that undermine data trustworthiness. Reliable KG auditing requires jointly examining internal structural constraints and open-world evidence, whereas existing methods often rely on a single evidence source or static, one-shot inference, limiting adaptive evidence acquisition and auditability. We propose SHARP (Schema-Hybrid Agent for Reliable Prediction), a training-free agent that reformulates triple verification as a dynamic process of strategic planning, active investigation, and evidential reasoning. SHARP uses retrieved historical verification trajectories and entity--relation schema information to construct a task-specific investigation plan. It then iteratively coordinates KG and textual tools to cross-validate structural and semantic evidence, producing a verification verdict with a transparent evidence chain. On independently sampled, balanced test sets constructed from FB15K-237 and Wikidata5M-Ind, SHARP achieves average accuracies of 87.1% and 93.6%, respectively. Controlled comparisons with standard RAG using the same evidence sources, together with ablation studies, further show that the gains arise not merely from external retrieval, but from the combined effects of schema-aware planning, memory augmentation, and iterative hybrid tool use.
Figures & tables
Figure 1: Comparisons between previous methods and our proposed SHARP. Existing graph embedding and static LLM/RAG methods may produce false positives, while SHARP integrates structural and textual evidence through dynamic reasoning to detect errors.
Figure 2: Overview of the SHARP architecture. The framework consists of three components: (1) Schema-Aware Initialization, which retrieves similar trajectories from memory to generate an initial plan; (2) an Iterative Reasoning Loop, which dynamically updates reasoning through an improved ReAct cycle; and (3) a Hybrid Knowledge Toolset, which combines internal KG and external text tools for cross-validated verification.
Method
FB15K-237
Wikidata5M-Ind
Acc.
F1
Prec.
Rec.
Acc.
F1
Prec.
Rec.
Graph Embedding-based
TransE ( Bordes et al. 2013 )
66.4
73.7
60.5
94.2
49.6
49.2
49.6
48.8
DistMult ( Yang et al. 2015 )
61.2
70.9
56.7
94.4
50.2
50.7
51.4
50.1
RotatE ( Sun et al. 2019 )
66.6
74.4
60.3
97.0
50.8
51.3
50.5
52.2
PLM-based
Table 1: Results on FB15K-237 and Wikidata5M-Ind. All methods are evaluated on the same three independently sampled test sets; SHARP values are reported as mean ± standard deviation. Cross-paradigm methods retain their native evidence access “–” denotes unavailable results. Bold indicates the best-performing model; underlining indicates the second-best model.
Figure 3: Comparison of LLM vs. LLM+RAG vs. LLM+SHARP on FB15K-237 and Wikidata5M-Ind. RAG uses the same evidence sources as SHARP but performs one non-interactive verification pass.
Method
FB15K-237
WD5M-Ind
Acc
F1
Acc
F1
SHARP
87.1
86.5
93.6
93.3
w/o Memory Augmentation
81.0
80.9
86.8
86.8
w/o Schema-Aware Planning
77.9
76.7
79.1
76.7
w/o KG Tools
78.4
77.2
83.7
81.9
w/o External Tools
75.7
70.6
80.5
76.1
Table 2: Component ablation results on benchmarks
Figure 4: Tool-use behavior analysis. The upper panel presents the tool invocation distribution on FB15K-237 and Wikidata5M-Ind, while the lower panel shows transition probabilities between consecutive tool calls.
Split
n
Relevant
Sufficient
Support
Wikidata5M-Ind
50
100%
86%
84%
FB15K-237
50
98%
84%
84%
Overall
100
99%
85%
84%
Table 3: Evidence-chain audit of 100 correctly predicted triples.
Benchmark
Seeds
Size/seed
Positive
Negative
FB15K-237
3
1,000
500
500
Wikidata5M-Ind
3
1,000
500
500
Table 4: Evaluation-set construction. Counts are per independently sampled seed.
Tool API Name
Parameters
Output Evidence & Core Usage
1. Internal Structural Tools (KG)
KG_Definition
entity or relation : str
Returns schema metadata (labels, domains, ranges) to establish basic definitions for nodes/edges.
KG_Neighbor
entity , relation : str
Returns top-20 semantically correlated target entities to analyze local subgraphs and co-occurrences.
KG_Path
entity_a , entity_b : str
Returns explicit 1 to 3-hop relational paths to verify structural connectivity and implicit links.
2. External Semantic Tools
Wiki_Evidence
(1) entity : str (2) entity_a , entity_b : str
(1) Returns Wikipedia summaries/attributes to provide encyclopedic background for a single entity. (2) Returns sentences mentioning both entities to discover explicit textual links.
Table 5: Overview of the Hybrid Knowledge Toolset API definitions.
Figure 5: F1 comparison between SHARP and zero-shot Qwen3-Max across six schema categories.
Figure 6: The prompt template for the Schema-Aware Planning phase.
Dataset
Avg.
Amort. Time
In Tokens
Out Tokens
Cost
FB15K-237
1.86
23,031.39
2,275.24
$0.011
Wikidata5M-Ind
1.26
13,099.55
1,254.77
$0.006
Table 6: Amortized wall-clock time, token use, and API cost per completed query under 50-worker batch execution.
Stage
Included or documented artifact
Supported output
Data construction
Fixed evaluation splits for seeds 2024, 2025, and 2026; entity/relation mappings; seeded construction protocol documented in the appendix
Three balanced 1,000-instance sets per benchmark (Table 4 )
Memory preparation
The 200 curated trajectories; relation/category metadata; FAISS embedding/index construction code
Top- 3 procedural demonstrations for each query
Tool execution
Entity/relation resolver; KG filtering logic; Wikipedia/web wrappers; live evidence retrieval code
Anti-leakage KG observations and refreshed text evidence under live retrieval
Agent inference
Prompt/controller logic; tool-call parser; verdict parser; Tmax=10 ; batch runners for both benchmarks
Regenerated per-instance predictions, action traces, and aggregate summaries
Baselines
Baseline protocols, evaluation settings, and comparison methodology documented in the appendix; full baseline outputs are omitted from the size-limited review supplement
Context for the main comparison, controlled RAG comparison, and ablation results
Analysis
Metric definitions, per-seed evaluation protocol, and error-category summaries documented in the appendix; large cached prediction traces and plotting artifacts are omitted from the review upload
Tables, figures, variation summaries, and audited case summaries
Table 7: Reproducibility coverage of the size-limited anonymous Code and Data Supplement.
Error Category
FB15K-237
Wikidata5M-Ind
Retrieval failure
37
10
Relation granularity
25
11
Plausibility bias
18
4
Entity grounding
12
14
Temporal/context mismatch
6
13
KG–text mismatch
11
5
Note. “Retrieval failure” denotes evidence sparsity or missed retrieval; “Relation granularity” denotes over-rejection caused by schema or relation-scope mismatch; “Plausibility bias” denotes plausibility-driven over-acceptance; “Entity grounding” includes alias and identifier errors; “KG–text mismatch” denotes conflicts between KG labels and external semantics.
Figure 7: The prompt template for the Reasoning phase
Figure 8: The prompt template for the mandatory judgment phase
Figure 9: A representative reasoning trajectory from the Memory Bank. This case demonstrates the “Think-Act-Observe” loop where the agent fuses KG structural info with Web evidence.
Knowledge graphs (KGs) are often automatically constructed from large-scale corpora, but they inevitably contain factual errors due to noisy sources and extraction failures, and verifying them reliably at industrial scale remains a critical challenge. To address this, we propose AgentKGV, the Agentic LLM-RAG framework for KG fact Verification, that integrates dynamic routing and iterative query rewriting, which handles surface-form mismatch in document-level retrieval. To make this framework more accurate and cost-efficient for industrial deployment, we further introduce a two-stage training strategy: turn-level distillation-based SFT that transfers reasoning ability from a large teacher model into a small model for stable query rewriting and reasoning, and trajectory-level GRPO that optimizes the search policy to reduce unnecessary retrieval at scale. On the long-tail-predicate split of the open-domain T-REx benchmark, our framework improves macro-F1 over single-turn RAG by 5.5 %p, and two-stage training does it further by 9.4 %p. GRPO also cuts the average number of search calls from 3.24 to 1.63 without lowering accuracy.
Knowledge Graph-based Question Answering (KGQA) plays a pivotal role in complex reasoning tasks but remains constrained by two persistent challenges: the structural heterogeneity of Knowledge Graphs(KGs) often leads to semantic mismatch during retrieval, while existing reasoning path retrieval methods lack a global structural perspective. To address these issues, we propose Structure-Tracing Evidence Mining (STEM), a novel framework that reframes multi-hop reasoning as a schema-guided graph search task. First, we design a Semantic-to-Structural Projection pipeline that leverages KG structural priors to decompose queries into atomic relational assertions and construct an adaptive query schema graph. Subsequently, we execute globally-aware node anchoring and subgraph retrieval to obtain the final evidence reasoning graph from KG. To more effectively integrate global structural information during the graph construction process, we design a Triple-Dependent GNN (Triple-GNN) to generate a Global Guidance Subgraph (Guidance Graph) that guides the construction. STEM significantly improves both the accuracy and evidence completeness of multi-hop reasoning graph retrieval, and achieves State-of-the-Art performance on multiple multi-hop benchmarks.
Peng Yu, En Xu, Bin Chen +2
AI Product Center, Kingsoft Corporation, Beijing, China
Multi-hop fact verification, which verifies claims by reasoning over multiple pieces of evidence, is critical for combating misinformation on social media yet remains highly challenging. Recent methods primarily rely on multi-agent collaboration to decompose fact verification into specialized subtasks. However, these methods face two critical limitations: (1) agents may perform individual subtasks without sufficient awareness of the global verification objective, causing their reasoning to deviate from the intended direction; and (2) conflicts between parametric knowledge and the provided evidence may undermine evidence-grounded reasoning and lead to incorrect verdicts. To address these challenges, we propose ReflectFact, a novel self-reflective agent framework for multi-hop fact verification. ReflectFact introduces three key tasks. Explicit Reasoning Path Planning builds an evidence-grounded reasoning path by resolving implicit entities, decomposing the claim into sub-questions, and integrating the verified facts into a verdict. Evidence-Drift Verification makes the agent re-answer by quoting the supporting evidence when a grounded answer merely echoes its parametric prior, thereby calibrating evidence deviation to ensure grounded comprehension. Reasoning Reflection Verification re-examines each reasoning step and regenerates it once an inconsistency is detected, correcting reasoning flaws such as location bias and replacement bias through a global task perspective. Subsequently, the agent aggregates validated reasoning chains to yield reliable verdicts. Extensive experiments on HOVER and EX-FEVER demonstrate that ReflectFact effectively remedies the comprehension and reasoning defects of existing methods, achieving state-of-the-art performance and respectively outperforming the strongest baseline by 3.32% and 2.78% on the two datasets.
Runze Zhao, Zixin Tang, Xiaoshuai Hao +4
Institute of Information Engineering, Chinese Academy of Sciences · 1Zhongguancun Laboratory · 3Xiaomi EV