AutoSchema: Live Schema Grounding for Agentic Text-to-Sparql over Heterogeneous Knowledge Graphs
Authors: Yiming Zhang, Koji Tsuda
Organizations: The University of Tokyo Kashiwa, Chiba, Japan · Center for Basic Research on Materials, National Institute for Materials Science Tsukuba, Ibaraki, Japan · RIKEN Center for Advanced Intelligence Project Tokyo, Japan
Life science knowledge graphs make large collections of structured data available through SPARQL, but each resource uses its own schema, identifiers, and links. TogoMCP helps language model agents query these resources by providing curated Metadata Interoperability Exchange files. Creating and maintaining these files still requires language model assisted drafting, validation, and manual review. We study live schema grounding, where an agent obtains the schema evidence needed for a question directly from the current endpoints. We present autoschema, a general framework for live schema grounding that requires no training. It inspects live schemas, maps entity names in a question to graph identifiers, explores relation paths, and finds possible connections between resources during iterative query construction. We use TogoMCP as our main comparison framework. We evaluate autoschema on Resource Focused Biomedical KGQA, Multi Resource Biomedical KGQA, Longitudinal Biomedical Semantic QA over BioASQ Task B, and Chemistry Knowledge Graph Transfer to a previously undocumented RDF graph. autoschema improves mean factoid accuracy over TogoMCP in the biomedical KGQA tasks and gives consistent gains in the longitudinal BioASQ evaluation. It also reduces iteration budget exhaustion and uses fewer tool calls on average in the core evaluation. The transfer study gives preliminary evidence that live schema grounding can support irregular and previously unseen graphs without first creating a curated schema file.
Figures & tables
Figure 1. AutoSchema in an agentic text-to-SPARQL loop over heterogeneous knowledge graphs. Given a natural-language question, the agent alternates planning, live grounding, query generation, execution, and reflection. AutoSchema supplies four question-time capabilities: source summarization, entity resolution, relation and path discovery, and cross-database bridging. Each capability returns current endpoint evidence rather than a complete schema dump. The bottom panel contrasts AutoSchema with TogoMCP and a no-schema lower bound. The left column shows a natural-language question and an LLM agent iterating through planning, AutoSchema tool calls, SPARQL generation, endpoint execution, and reflection before producing an answer. The center column details four live grounding capabilities with examples: source summarization, entity resolution, relation and path discovery, and cross-database bridging. The right column shows heterogeneous gene, disease, drug, and pathway graphs connected by possible bridges. A bottom comparison summarizes the no-schema lower bound, TogoMCP, and AutoSchema.
Task
Data and evaluation focus
Resource Focused Biomedical KGQA
RDF Portal. Factoid and list questions within one resource.
Multi Resource Biomedical KGQA
RDF Portal. Factoid and list questions across resources.
Longitudinal Biomedical Semantic QA
Six BioASQ Task B sets. Factoid, list, and yes or no questions collected over time.
Chemistry Knowledge Graph Transfer
tmQM-RDF. Factoid and list questions on a graph outside biomedicine.
Table 1. Evaluation tasks. Graph-supported means that each gold answer is verified by executing a gold SPARQL query.
Model
Framework
Factoid acc.
List F1
Limit rate
gpt-oss-120b
No schema
0.164±.019
0.122±.006
53.3±1.8%
TogoMCP
0.198±.009
0.134±.001
44.0±1.0%
AutoSchema
0.260±.005
0.199±.012
33.5±1.5%
gemma4-31b
No schema
0.239±.022
0.117±.009
17.6±1.0%
TogoMCP
0.258±.008
0.134±.003
20.2±0.8%
AutoSchema
0.281±.010
0.145±.006
17.1±2.3%
Table 2. Resource Focused Biomedical KGQA with 559 questions. Results are mean ± standard deviation over three runs.
Model
Framework
Factoid acc.
List F1
Limit rate
gpt-oss-120b
No schema
0.178±.004
0.138±.001
45.7±1.4%
TogoMCP
0.167±.024
0.165±.005
49.4±0.3%
AutoSchema
0.203±.020
0.200±.019
40.0±1.1%
gemma4-31b
No schema
0.110±.027
0.105±.010
25.1±2.0%
TogoMCP
0.140±.008
0.125±.019
32.9±1.6%
AutoSchema
0.216±.041
0.134±.003
26.1±1.6%
Table 3. Multi Resource Biomedical KGQA with 226 questions. Results are mean ± standard deviation over three runs.
Model
Framework
Fact.
List F1
Yes/no
8B (2020)
gpt-oss-120b
No schema
0.468±.008
0.404±.031
0.829±.026
TogoMCP
0.510±.024
0.360±.027
0.838±.015
AutoSchema
0.581±.040
0.396±.022
0.860±.014
gemma4-31b
No schema
0.614±.027
0.494±.013
0.875±.013
TogoMCP
0.565±.030
0.466±.008
0.877±.004
Table 4. Longitudinal Biomedical Semantic QA on two BioASQ Task B test sets. Results are mean ± standard deviation over three runs.
Figure 2. Factoid accuracy gain of autoschema over TogoMCP. Bars show differences between three-run means. Error bars show the root sum square of the reported standard deviations and are descriptive rather than confidence intervals. Two bar charts show the change in factoid accuracy from TogoMCP to AutoSchema. The first compares two models across Resource Focused Biomedical KGQA, Multi Resource Biomedical KGQA, BioASQ Task 8B, and BioASQ Task 13B. The second shows gpt-oss-120b results across six yearly BioASQ Task B test sets. Every mean difference is positive.
Figure 3. Paired trace for a Multi Resource Biomedical KGQA question. TogoMCP continues from an incorrect static relation and reaches the interaction limit. Autoschema uses live entity resolution and local path discovery to return the correct answer. The example explains a mechanism and does not estimate how often it occurs. A question and gold answer appear above two agent traces. The TogoMCP trace plans a direct collection-number relation, issues queries that return no matches, and reaches the iteration limit. The AutoSchema trace resolves the collection record, discovers its path to a strain and phylum, and returns Actinobacteria in six calls.
Model
Framework
Latency (s)
Tool calls
gpt-oss-120b
No schema
71.4±5.6
13.22±.04
TogoMCP
67.1±5.9
12.91±.05
AutoSchema
59.5±11.9
11.13±.12
gemma4-31b
No schema
80.5±1.3
9.31±.08
TogoMCP
74.2±1.6
9.22±.05
AutoSchema
76.4±3.2
8.27±.23
Table 5. Mean per-question interaction cost across the two biomedical KGQA tasks and two selected BioASQ sets. Values are mean ± standard deviation over three runs.
Framework
Fact.
List F1
Limit
Time (s)
Calls
TogoMCP
0.273
0.167
35.9%
1060.3
15.1
AutoSchema
0.394
0.167
23.1%
1306.5
16.7
Table 6. Chemistry Knowledge Graph Transfer on 39 tmQM-RDF questions with gemma4-31b. Results are from one run.
Task
Access
Fact.
List F1
Limit
Time (s)
gpt-oss-120b
R
Iterative
0.260±.005
0.199±.012
33.5±1.5%
69.6±17.6
R
Single shot
0.246±.013
0.185±.011
35.5±1.0%
66.3±1.9
M
Iterative
0.203±.020
0.200±.018
40.0±1.1%
77.2±19.4
M
Single shot
0.185±.017
0.155±.010
41.9±2.9%
68.4±1.6
gemma4-31b
Table 7. Iterative and single shot grounding. Values are mean ± standard deviation over three runs. R and M denote the Resource Focused and Multi Resource Biomedical KGQA tasks.
Query scope, frequent classes with counts and example IRIs, observed properties with range kinds and values, and example SPARQL queries. The optional class name narrows the property view.
Candidate entity IRIs and labels found through common label and identifier properties. For recognized biomedical accessions, the response may provide a directly constructed IRI instead of a label match.
expand_entity database , entity_iri
Direct properties and paths that pass through as many as three nested blank nodes. Each nested path includes a SPARQL pattern that can be run again to obtain a filterable or joinable result.
get_cross_db_bridges database_a , database_b
Candidate links based on an optional identifier conversion service, direct IRI reuse, or property names observed in both sources. The response includes a join skeleton when a shared property is found.
Appendix
Table 8. Core MCP tools exposed by autoschema . Defaults are shown in the signatures.
Real-world data spans tables, documents, and semi-structured files with implicit semantics. Querying this data requires integrating evidence across inconsistent schemas and formats, yet existing approaches either demand costly manual engineering or bypass structure entirely. We present a system that automatically discovers an executable schema from raw multi-source data and uses it as a shared contract for knowledge graph construction and query-time retrieval. A closed-world field catalog constrains LLM-based schema discovery to attested fields; deterministic structural analysis infers identity keys, foreign keys, and source hierarchy; and the resulting schema drives extraction, deduplication, and cross-source linking into a provenance-aware knowledge graph. At query time the schema -- optionally extended via a monotonic protocol -- conditions a multi-tool agent routing retrieval across structured lookup, graph traversal, and vector search, returning grounded answers with traceable citations. In controlled zero-shot comparisons using the same LLM, data, and evaluation harness, the system improves over retrieval-only and decomposition-based baselines across four QA benchmarks, with ablations showing that schema-conditioned routing, structural intelligence, and schema-guided construction each contribute to the gains.
Complex knowledge base question answering (KBQA) is commonly approached through either information retrieval over a question-specific subgraph or semantic parsing into an executable logical form. We study the latter paradigm. Recent large language model agents make semantic parsing interactive: they alternate between reasoning, querying the knowledge base, and extending a partial SPARQL query. This interleaving reduces reliance on one-shot generation, but makes the quality of KB grounding depend on what the interaction tools expose. Existing agents retrieve or prune candidate properties mainly through lexical relevance and instance-level observations, without systematically conditioning on entity types, property domains and ranges, or the expected answer type. We call this failure mode type-blind grounding. It enlarges the grounding search space and often produces plausible-looking but semantically incompatible triple patterns that execute to empty results. We propose SAGA (Schema-Aware Grounding for Agentic Text-to-SPARQL Generation), a training-free framework that turns property exploration into a schema-constrained grounding operation. SAGA maintains a persistent bidirectional type state, filters known-incompatible property candidates at construction time, presents the remaining graph patterns in a compact schema-annotated format, and handles missing schema information permissively through empirical and trace-local evidence. Across nine benchmark settings over Wikidata and Freebase, SAGA achieves the highest F1 on all nine settings and the highest exact-match accuracy on eight, while reducing empty-result queries across all reported Wikidata settings.
Yiming Zhang, Koji Tsuda
1The University of Tokyo, Japan · Center for Basic Research on Materials, National Institute for Materials Science, Japan · 3RIKEN Center for Advanced Intelligence Project, Japan
We present EXYGEN (EXplore Your Graphs ENgine), a framework for knowledge graph (KG) understanding that enables conversational access to KGs at scale. We address two questions in sequence. First, how effectively can LLMs perform text-to-SPARQL generation given only automatically derived structured metadata and small graph samples, rather than task-specific fine-tuning? We integrate VoID descriptions and ShEx schemas into a retrieval-augmented generation (RAG) pipeline and ablate KG-derived context on the SciQA benchmark. Our best configuration -- combining ShEx schemas, retrieved triples, and example question-query pairs -- reaches an exact match of 0.419 on execution results without any LLM fine-tuning. We further find that lexical metrics such as F1 poorly predict query correctness, and that larger general-purpose LLMs can outperform smaller code-specialized ones once given sufficient context. Second, we ask how to generate the structured metadata that this method relies on from very large KGs, where KG metadata generation becomes computationally intractable. We introduce a predicate-coverage-aware parallel graph sampling strategy that preserves structural diversity while remaining computationally tractable. On OpenCitations Meta and GESIS, it retains high predicate coverage with minimal triple loss and reduces runtime by over 80x; on ORKG, sampling is not just faster but the only tractable path to obtain complete metadata. Together, these results show that structured schema context and lightweight prompting can substantially reduce reliance on fine-tuning for scalable conversational access to KGs, though closing the remaining gap to fully fine-tuned approaches will likely require reducing dependence on curated question-query exemplars -- whether through synthetic generation or an execution-feedback-driven approach -- and validating these findings beyond a single benchmark.