Knowledge Graphs (KGs) provide a critical foundation for many AI systems, yet their automated construction inevitably introduces noisy triples that undermine data trustworthiness. Reliable KG auditing requires jointly examining internal structural constraints and open-world evidence, whereas existing methods often rely on a single evidence source or static, one-shot inference, limiting adaptive evidence acquisition and auditability. We propose SHARP (Schema-Hybrid Agent for Reliable Prediction), a training-free agent that reformulates triple verification as a dynamic process of strategic planning, active investigation, and evidential reasoning. SHARP uses retrieved historical verification trajectories and entity--relation schema information to construct a task-specific investigation plan. It then iteratively coordinates KG and textual tools to cross-validate structural and semantic evidence, producing a verification verdict with a transparent evidence chain. On independently sampled, balanced test sets constructed from FB15K-237 and Wikidata5M-Ind, SHARP achieves average accuracies of 87.1% and 93.6%, respectively. Controlled comparisons with standard RAG using the same evidence sources, together with ablation studies, further show that the gains arise not merely from external retrieval, but from the combined effects of schema-aware planning, memory augmentation, and iterative hybrid tool use.
Figures & tables
Figure 1: Comparisons between previous methods and our proposed SHARP. Existing graph embedding and static LLM/RAG methods may produce false positives, while SHARP integrates structural and textual evidence through dynamic reasoning to detect errors.
Figure 2: Overview of the SHARP architecture. The framework consists of three components: (1) Schema-Aware Initialization, which retrieves similar trajectories from memory to generate an initial plan; (2) an Iterative Reasoning Loop, which dynamically updates reasoning through an improved ReAct cycle; and (3) a Hybrid Knowledge Toolset, which combines internal KG and external text tools for cross-validated verification.
Method
FB15K-237
Wikidata5M-Ind
Acc.
F1
Prec.
Rec.
Acc.
F1
Prec.
Rec.
Graph Embedding-based
TransE ( Bordes et al. 2013 )
66.4
73.7
60.5
94.2
49.6
49.2
49.6
48.8
DistMult ( Yang et al. 2015 )
61.2
70.9
56.7
94.4
50.2
50.7
51.4
50.1
RotatE ( Sun et al. 2019 )
66.6
74.4
60.3
97.0
50.8
51.3
50.5
52.2
PLM-based
Table 1: Results on FB15K-237 and Wikidata5M-Ind. All methods are evaluated on the same three independently sampled test sets; SHARP values are reported as mean ± standard deviation. Cross-paradigm methods retain their native evidence access “–” denotes unavailable results. Bold indicates the best-performing model; underlining indicates the second-best model.
Figure 3: Comparison of LLM vs. LLM+RAG vs. LLM+SHARP on FB15K-237 and Wikidata5M-Ind. RAG uses the same evidence sources as SHARP but performs one non-interactive verification pass.
Method
FB15K-237
WD5M-Ind
Acc
F1
Acc
F1
SHARP
87.1
86.5
93.6
93.3
w/o Memory Augmentation
81.0
80.9
86.8
86.8
w/o Schema-Aware Planning
77.9
76.7
79.1
76.7
w/o KG Tools
78.4
77.2
83.7
81.9
w/o External Tools
75.7
70.6
80.5
76.1
Table 2: Component ablation results on benchmarks
Figure 4: Tool-use behavior analysis. The upper panel presents the tool invocation distribution on FB15K-237 and Wikidata5M-Ind, while the lower panel shows transition probabilities between consecutive tool calls.
Split
n
Relevant
Sufficient
Support
Wikidata5M-Ind
50
100%
86%
84%
FB15K-237
50
98%
84%
84%
Overall
100
99%
85%
84%
Table 3: Evidence-chain audit of 100 correctly predicted triples.
Benchmark
Seeds
Size/seed
Positive
Negative
FB15K-237
3
1,000
500
500
Wikidata5M-Ind
3
1,000
500
500
Table 4: Evaluation-set construction. Counts are per independently sampled seed.
Tool API Name
Parameters
Output Evidence & Core Usage
1. Internal Structural Tools (KG)
KG_Definition
entity or relation : str
Returns schema metadata (labels, domains, ranges) to establish basic definitions for nodes/edges.
KG_Neighbor
entity , relation : str
Returns top-20 semantically correlated target entities to analyze local subgraphs and co-occurrences.
KG_Path
entity_a , entity_b : str
Returns explicit 1 to 3-hop relational paths to verify structural connectivity and implicit links.
2. External Semantic Tools
Wiki_Evidence
(1) entity : str (2) entity_a , entity_b : str
(1) Returns Wikipedia summaries/attributes to provide encyclopedic background for a single entity. (2) Returns sentences mentioning both entities to discover explicit textual links.
Table 5: Overview of the Hybrid Knowledge Toolset API definitions.
Figure 5: F1 comparison between SHARP and zero-shot Qwen3-Max across six schema categories.
Figure 6: The prompt template for the Schema-Aware Planning phase.
Dataset
Avg.
Amort. Time
In Tokens
Out Tokens
Cost
FB15K-237
1.86
23,031.39
2,275.24
$0.011
Wikidata5M-Ind
1.26
13,099.55
1,254.77
$0.006
Table 6: Amortized wall-clock time, token use, and API cost per completed query under 50-worker batch execution.
Stage
Included or documented artifact
Supported output
Data construction
Fixed evaluation splits for seeds 2024, 2025, and 2026; entity/relation mappings; seeded construction protocol documented in the appendix
Three balanced 1,000-instance sets per benchmark (Table 4 )
Memory preparation
The 200 curated trajectories; relation/category metadata; FAISS embedding/index construction code
Top- 3 procedural demonstrations for each query
Tool execution
Entity/relation resolver; KG filtering logic; Wikipedia/web wrappers; live evidence retrieval code
Anti-leakage KG observations and refreshed text evidence under live retrieval
Agent inference
Prompt/controller logic; tool-call parser; verdict parser; Tmax=10 ; batch runners for both benchmarks
Regenerated per-instance predictions, action traces, and aggregate summaries
Baselines
Baseline protocols, evaluation settings, and comparison methodology documented in the appendix; full baseline outputs are omitted from the size-limited review supplement
Context for the main comparison, controlled RAG comparison, and ablation results
Analysis
Metric definitions, per-seed evaluation protocol, and error-category summaries documented in the appendix; large cached prediction traces and plotting artifacts are omitted from the review upload
Tables, figures, variation summaries, and audited case summaries
Table 7: Reproducibility coverage of the size-limited anonymous Code and Data Supplement.
Error Category
FB15K-237
Wikidata5M-Ind
Retrieval failure
37
10
Relation granularity
25
11
Plausibility bias
18
4
Entity grounding
12
14
Temporal/context mismatch
6
13
KG–text mismatch
11
5
Note. “Retrieval failure” denotes evidence sparsity or missed retrieval; “Relation granularity” denotes over-rejection caused by schema or relation-scope mismatch; “Plausibility bias” denotes plausibility-driven over-acceptance; “Entity grounding” includes alias and identifier errors; “KG–text mismatch” denotes conflicts between KG labels and external semantics.
Figure 7: The prompt template for the Reasoning phase
Figure 8: The prompt template for the mandatory judgment phase
Figure 9: A representative reasoning trajectory from the Memory Bank. This case demonstrates the “Think-Act-Observe” loop where the agent fuses KG structural info with Web evidence.