Schema-Free Text-to-SQL: Learning Enterprise Database Structure from Query Logs

Ömer Faruk Akgül; Nikhil Kapahi; Prajit Reddy Muppidi

cs.CLOct 24, 2026

Schema-Free Text-to-SQL: Learning Enterprise Database Structure from Query Logs

Authors: Ömer Faruk Akgül, Nikhil Kapahi, Prajit Reddy Muppidi

Organizations: University of Southern California · Amazon

Abstract

Text-to-SQL systems achieve remarkable accuracy on academic benchmarks, yet performance drops dramatically in enterprise deployments. This gap stems from a fundamental assumption: existing approaches require complete schema documentation, but enterprise databases typically lack comprehensive documentation, featuring cryptic naming and implicit relationships. We present the first text-to-SQL framework that operates without schema documentation, instead learning database structure from historical query logs. Query logs encode implicit structural knowledge: which tables are queried together, how they join, and what columns map to business concepts. We construct a knowledge graph from parsed query patterns and introduce entity-driven GraphRAG retrieval that grounds user questions in domain terminology rather than relying solely on embedding similarity. Validated on both an industrial deployment with expert evaluation and the Beaver benchmark, our approach substantially outperforms similarity-based retrieval methods, the dominant industry approach, in both executability and answer quality. On Beaver, we achieve 95.7% execution validity compared to 60.8% for embedding-based retrieval, with expert data engineers preferring our system 70% to 10% on internal deployment. Our work demonstrates that text-to-SQL can operate effectively in schema-ambiguous enterprise environments, enabling natural language database interfaces "in the wild".

Figures & tables

Figure 1: State-of-the-art text-to-SQL systems require complete schema documentation unavailable in enterprise settings. Operating with only historical query logs, our approach achieves 95.7% execution validity and 70% expert preference over embedding-based retrieval.

Figure 2: System architecture. Offline: query logs are processed to construct a knowledge graph encoding table metadata, domain entities, and join relationships. Online: given a question, we extract entities, traverse the graph to retrieve relevant context, and generate SQL.

Metric	Ours	RAG
Execution Validity	200/209 (95.7%)	127/209 (60.8%)
LLM Judge Wins	95 (45.5%)	71 (34.0%)
Rubric Judge Wins	100 (47.8%)	64 (30.6%)

Table 1: BEAVER benchmark results (209 questions). Ties: 21 both adequate (LLM), 45 both (Rubric).

Dimension	Ours	RAG	Both
Executability	74	1	126
Schema Validity	60	17	132
Filter Correctness	52	28	120
Join Relevance	44	55	62
Result Fidelity	85	69	38

Table 2: Rubric dimension wins on BEAVER test set.

Evaluator	Ours	RAG
Expert Consensus	70%	10%
LLM Judge	80%	16%
LLM-Human Agreement	83%

Table 3: Amazon evaluation. Remaining cases are ties.

	Deterministic	Open-Ended
	(n=114)	(n=95)
Gold Avg Score	8.50	8.85
Ours Avg Score	7.55	9.00
Gold Wins	64.9%	61.1%
Ours Wins	26.3%	28.4%
Both Adequate	3.5%	7.4%

Table 4: Generated vs Gold SQL by question type. On open-ended questions, generated SQL matches or exceeds gold quality.

Configuration	Score	$\Delta$	vs Full
Full Pipeline	7.76	–	–
$-$ Entity Matching	7.34	$-$ 5.4%	43% / 21%
$-$ KG Traversal	7.60	$-$ 2.1%	41% / 28%
$-$ PPR Ranking	7.62	$-$ 1.7%	21% / 18%

Table 5: Ablation on BEAVER. Score: open-ended evaluation (0–10). vs Full: pairwise wins (Full / Ablated).

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Statistic	Value
Total tables	463
Explicit FK relationships	127
Implicit joins discovered	1,113
Total KG edges	1,310
Unique entities extracted	1,844

Appendix

Table 6: BEAVER knowledge graph construction statistics.

	Amazon	BEAVER
Total Tables	240	463
Query Logs	$\sim$ 500K	–
KG Join Edges	1,847	1,310
Test Questions	35	209
Deterministic	–	114
Open-Ended	–	95

Appendix

Table 7: Dataset statistics for both evaluation settings.

Metric	Ours	RAG
Avg tables selected	12.61	12.00
Median	13	12
Min / Max	5 / 16	12 / 12
% of total (463)	2.72%	2.59%

Appendix

Table 8: Table selection statistics on BEAVER. Our system adaptively selects 5–16 tables based on query complexity; RAG uses fixed top-

k

=12.

Ablation	Exec	Schema	Filter	Join	Result
$-$ Entity	16/11	21/11	28/19	37/22	75/42
$-$ KG Trav	16/10	28/13	33/21	51/36	78/54
$-$ PPR	13/12	14/10	15/14	17/12	41/34

Appendix

Table 9: Ablation dimension breakdown (Full wins / Ablated wins). Entity matching has largest impact on result fidelity.

	Deterministic		Open-Ended
	Gold	Ours	Gold	Ours
Avg Score	8.50	7.55	8.85	9.00
Blind Judge Wins	74	30	58	27
Both Adequate	4		7
Rubric Judge Wins	65	39	51	33
Both Adequate	10		11

Appendix

Table 10: Generated vs Gold comparison by question type.

	Deterministic		Open-Ended
	Ours	RAG	Ours	RAG
Avg Score	7.54	6.76	8.99	8.65
Exec Valid	95.6%	58.8%	96.8%	63.2%

Appendix

Table 11: Ours vs RAG by question type.

Database	n	Ours Exec	RAG Exec	$\Delta$
dw	121	94.2%	47.1%	+47.1
csail_nova	43	100%	72.1%	+27.9
csail_neutron	30	100%	93.3%	+6.7
keystone	8	100%	100%	0
Other	7	100%	100%	0

Appendix

Table 12: Execution validity by database.

Failure Pattern	Count	%
Missing entity coverage	12	40%
Disconnected graph regions	10	33%
Ambiguous terminology	5	17%
Schema name confusion	3	10%

Appendix

Table 13: Failure pattern analysis on low-scoring BEAVER queries (n=30).

Text-to-SQL

Text-to-SQL translates natural-language questions into executable database queries, linking user intent to schemas, values, and SQL logic.

See all 173 papers

EntSQL: A Benchmark for Grounding Text-to-SQL in Long-Context Enterprise Knowledge
Jun 2, 2026Chengxi Liao, Tao Xu, Zulong Chen +7LLM Grounding Text-to-SQL
1HKUST (GZ) · 2Alibaba Group · 4Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ) +1
ProSPy: A Profiling-Driven SQL-Python Agentic Framework for Enterprise Text-to-SQL
Jun 4, 2026Zhaorui Yang, Huawei Zheng, Sen Yang +14Text-to-SQL Relational Databases
1State Key Lab of CAD&CG, Zhejiang University · School of Software Technology, Zhejiang University · 3Tencent TEG +2
BIRD-History: A Benchmark for History-Driven Text-to-SQL with Fine-Grained Knowledge Annotations
Aug 29, 2026Yunfan Zhou, Qiming Shi, Yizhou Yang +2Text-to-SQL Text-to-SQL Evaluation
State Key Lab of CAD&CG, Zhejiang University · School of Software Technology, Zhejiang University

Jun 2, 2026J/K move · Enter open · S save

EntSQL: A Benchmark for Grounding Text-to-SQL in Long-Context Enterprise Knowledge

Chengxi Liao, Tao Xu, Zulong Chen, Chuanfei Xu+6

1HKUST (GZ) · 2Alibaba Group · 4Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)+1

LLM Grounding Text-to-SQL Text-to-SQL Evaluation Database Systems

Open paper AskPDF arXiv

Text-to-SQL enables natural language access to databases, and recent LLMs have substantially advanced its capabilities. Existing benchmarks such as Spider, BIRD, and Spider~2.0 evaluate schema generalization, large-scale databases, and realistic workflows, but largely overlook enterprise scenarios where SQL generation depends on private business knowledge, such as internal metrics, reporting conventions, and organizational rules. We introduce EntSQL, an enterprise-oriented Text-to-SQL benchmark for evaluating long-context grounding over proprietary business documents. EntSQL contains 1,066 aligned Chinese-English semantic examples across five business domains, with most examples requiring domain knowledge beyond the question and schema and involving complex SQL structures. On English inputs, the best evaluated system reaches only 15.9% when long-form documents are provided, highlighting the difficulty of grounding SQL generation in enterprise knowledge.

Abstract

Text-to-SQL

Explore similar work

EntSQL: A Benchmark for Grounding Text-to-SQL in Long-Context Enterprise Knowledge

ProSPy: A Profiling-Driven SQL-Python Agentic Framework for Enterprise Text-to-SQL

BIRD-History: A Benchmark for History-Driven Text-to-SQL with Fine-Grained Knowledge Annotations