Evidence-Guided Schema Normalization for Temporal Tabular Reasoning

Ashish Thanga; Vibhu Dixit; Abhilash Shankarampeta; Vivek Gupta

cs.CLNov 29, 2025

Evidence-Guided Schema Normalization for Temporal Tabular Reasoning

Authors: Ashish Thanga, Vibhu Dixit, Abhilash Shankarampeta, Vivek Gupta

Organizations: Arizona State University · UC San Diego

Abstract

Temporal reasoning over evolving semi-structured tables poses a challenge to current QA systems. We propose an approach that recasts the task as automated knowledge base construction: (1) prompting an LLM to synthesize a 3NF-compliant relational schema from Wikipedia infobox timelines, (2) populating the schema to obtain a queryable database, and (3) generating and executing SQL queries against it, with QA accuracy serving as an extrinsic evaluation of the constructed knowledge base. In a controlled grid of three schema generators crossed with six query models, the schema source accounts for 79.5% of the exact match (EM) variance against 1.6% for the query model: replacing the schema, and the prompt scaffolding derived from it, shifts EM by 14.7 to 20.0 points, whereas replacing the query model under a fixed schema shifts it by 4.4 to 12.1. From this evidence, we distill three candidate schema-design principles: balanced normalization, semantic naming, and consistent temporal anchoring, framed as correlational hypotheses. Our best configuration (Gemini 2.5 Flash schemas + Gemini-2.0-Flash queries) reaches 80.39 EM, 11.5 points above the strongest reported baseline (68.89 EM); an open-weights configuration reaches 79.52.

Figures & tables

Figure 1: Our three-stage pipeline, read left to right. Stage 1, Schema Generation : an LLM turns infobox timelines into a 3NF schema. Stage 2, Schema Population : scripts clean and insert each JSON snapshot, one per snapshot_id . Stage 3, SQL Generation & Execution : a schema-guided prompt turns a question into SQL, executed against the database. Stages 1–2 run once per domain, offline. Worked example: Appendix A.1 .

	Baseline		Schema Generation LLMs
	IRE+CoT		Gemini 2.5 Flash		Llama-3.3-70B-Instruct		Llama-3.1-8B-Instruct
SQL/CoT LLMs	EM	F1	EM	F1	EM	F1	EM	F1
Gemini-2.0-Flash	48.98	55.93	80.39	82.11	72.70	73.33	60.37	60.32
Qwen-2.5-7B-Instruct	30.40	29.22	77.84	78.86	75.30	75.07	60.91	60.32
Llama-3.1-8B-Instruct	29.83	37.95	78.08	79.49	78.70	77.85	61.78	60.66
Llama-3.3-70B-Instruct	41.08	54.91	69.86	75.54	79.52	79.91	64.79	63.66

Table 1: Exact Match (EM) and F1 scores on TransientTables across schema generators (columns) and SQL query generators (rows).

Source	df	Sum Sq.	% of total	F
Schema generator	2	737.9	79.5	21.08 ∗
Query model	5	15.2	1.6	0.17
Residual (interaction)	10	175.0	18.9
Total	17	928.2	100.0

Table 2: Two-way analysis of variance without replication over the 18 EM scores in Table 1 , with schema generator and query model as the two factors. df : degrees of freedom. Sum Sq. : sum of squared deviations attributable to that source.

F

: the source’s mean square (Sum Sq. divided by df) over the residual mean square, so

F\gg 1

marks a source explaining far more variation than noise. With one observation per cell, the schema

\times

query-model interaction cannot be separated from error and serves as the residual.

{}^{\ast}p<0.001

; the query-model effect is not significant (

p=0.97

	Avg. outer-join depth per query				Tables per domain
Domain	Flash	Pro	70B	8B	Flash	Pro	70B	8B
country	2.12	2.00	2.67	2.40	5	5	7	7
cricket_team	2.31	2.67	2.14	2.82	6	7	8	8
cricketer	0.00	0.00	0.00	0.00	3	4	3	7
economy	0.48	0.67	0.67	0.51	2	2	4	18
table_tennis_player	0.05	1.75	0.00	1.40	4	5	12	5

Table 3: Structural properties of the generated schemas (Flash = Gemini-2.5-Flash, Pro = Gemini-2.5-Pro, 70B = Llama-3.3-70B-Instruct, 8B = Llama-3.1-8B-Instruct). Left: average outer-join depth per executed query, from the outermost SELECT only. Right: relational tables generated per domain.

Issue (#Samples)	Error Category	Count
Data Quality (35)	Wrong Calculations	15
	Empty Results	12
	Wrong Entity Mapping	5
	Precision/Format Issues	3
SQL Generation (12)	Aggregate Function Misuse	6
	Syntax Errors	3

Table 4: Error distribution from 50 failure cases. Data quality dominates over SQL generation and schema issues.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Domain	Gemini-2.0-Flash				Llama-3.3-70B-Instruct				GPT-4o-mini
	EM	F1	R-1	R-L	EM	F1	R-1	R-L	EM	F1	R-1	R-L
country	80.00	84.30	70.84	70.26	66.50	78.44	48.67	48.13	81.50	63.78	67.59	66.38
cricket_team	82.70	80.00	67.99	66.58	75.61	84.50	65.00	60.27	79.00	55.72	61.80	60.66
cricketer	78.70	82.60	85.79	85.79	64.00	68.00	58.30	58.91	72.00	87.80	87.80	87.80
economy	84.30	81.40	46.87	46.87	62.57	73.12	54.12	54.12	82.10	81.50	81.50	81.50
table_tennis_player	76.70	78.30	49.60	48.30	80.21	78.10	67.23	68.91	65.00	60.67	61.00	61.00

Appendix

Table 5: Query Generation Models: Performance Comparison of Query Generation Models (Gemini-2.0-Flash, Llama-3.3-70B-Instruct, GPT-4o-Mini) across various domains. The models’ query generation quality is evaluated using Exact Match (EM), F1-score (F1), R-1 (Rouge-1), and R-L (Rouge-L) metrics. The schema used for the queries was generated using Gemini 2.5 Flash.

Domain	Gemini-2.5-Pro				Qwen-2.5-7B-Instruct				Llama-3.1-8B-Instruct
	EM	F1	R-1	R-L	EM	F1	R-1	R-L	EM	F1	R-1	R-L
country	66.00	78.43	78.43	75.23	78.14	81.24	65.41	65.18	76.24	80.11	66.54	64.78
cricket_team	87.00	78.34	78.34	75.30	79.92	77.00	62.13	61.35	78.31	77.10	61.98	61.23
cricketer	60.50	60.50	60.50	60.50	76.50	79.50	80.95	80.89	78.20	80.54	78.41	77.69
economy	80.50	60.00	60.00	60.00	81.00	77.93	42.17	42.08	80.15	78.46	50.35	52.71
table_tennis_player	50.00	48.83	48.83	48.83	74.50	75.50	44.42	43.46	76.40	77.23	47.56	46.79

Appendix

Table 6: Performance Comparison of three Query Generation Models (Gemini-2.5-Pro, Qwen-2.5-7B-Instruct, and Llama-3.1-8B-Instruct) across diverse domains. The models’ query generation quality is evaluated using Exact Match (EM), F1-score (F1), R-1 (Rouge-1), and R-L (Rouge-L) metrics. Llama-3.1-8B-Instruct demonstrated the highest average performance with an F1 score of 79.49 and an EM score of 78.08. The schema used for the queries was generated using Gemini 2.5 Flash.

Domain	Llama-3.1-8B-Instruct				Qwen 2.5 7B Instruct				Llama-3.3-70B-Instruct
	EM	F1	R-1	R-L	EM	F1	R-1	R-L	EM	F1	R-1	R-L
country	82.25	79.22	65.71	68.24	78.34	76.22	65.93	67.12	81.17	81.80	65.18	62.86
cricket_team	81.97	80.03	66.29	67.12	76.99	74.56	63.88	65.45	77.52	79.28	62.70	62.52
gov_agencies	80.89	81.10	63.98	66.52	73.11	74.55	61.99	62.99	80.05	81.79	65.32	66.95
economy	81.99	78.89	64.91	67.77	76.45	74.22	63.22	65.11	78.80	77.57	66.37	64.28
table_tennis_player	80.45	76.99	62.88	65.55	74.28	75.10	62.37	63.49	78.80	78.09	63.29	65.97

Appendix

Table 7: Performance Evaluation of three Query Generation Models (Llama-3.1-8B-Instruct, Qwen-2.5-7B-Instruct, and Llama-3.3-70B-Instruct) across diverse domains. The models’ query generation quality is assessed using Exact Match (EM), F1-score (F1), R-1 (Rouge-1), and R-L (Rouge-L) metrics. Llama-3.3-70B-Instruct achieved the highest overall average scores, leading with an average F1 of 79.91 and an average EM of 79.52. The schema used for these queries was generated by Llama-3.3-70B-Instruct.

Domain	GPT-4o-mini				Gemini-2.5-pro				Gemini-2.0-Flash
	EM	F1	R-1	R-L	EM	F1	R-1	R-L	EM	F1	R-1	R-L
country	79.44	80.95	64.69	64.10	79.88	77.00	64.37	60.81	71.91	74.15	59.57	60.33
cricket_team	80.07	79.28	64.37	64.72	76.59	78.86	62.22	62.90	73.33	72.33	57.53	57.65
cricketer	80.06	77.26	65.90	60.58	80.04	77.96	66.06	61.75	75.55	75.05	58.61	60.31
economy	77.53	76.60	65.35	61.71	81.87	79.79	63.38	66.90	72.14	70.26	61.23	59.77
table_tennis_player	77.06	79.42	61.36	64.01	78.37	77.74	63.64	64.29	71.83	72.39	61.27	58.92

Appendix

Table 8: Query Generation Model Performance for GPT-4o-mini, Gemini-2.5-Pro, and Gemini-2.0-Flash evaluated across multiple domains. Both GPT-4o-mini and Gemini-2.5-Pro demonstrated strong, competitive results, with less than 1 point separating their average Exact Match (EM) and F1 scores. Gemini-2.0-Flash showed a lower average performance across all four metrics (EM, F1, R-1, R-L). The schema for generating these queries was derived from Llama-3.3-70B-Instruct.

Domain	Llama-3.1-8B-Instruct				Qwen-2.5-7B-Instruct				Gemini-2.5-pro
	EM	F1	R-1	R-L	EM	F1	R-1	R-L	EM	F1	R-1	R-L
country	64.82	62.70	52.92	50.01	62.85	62.05	57.08	54.78	61.27	64.93	52.81	50.45
cricket_team	59.15	64.89	44.34	47.91	58.82	64.47	49.19	52.42	65.84	60.11	49.33	54.19
gov_agencies	62.43	57.01	46.87	43.12	59.80	64.44	49.54	48.32	60.55	63.89	53.94	52.37
economy	57.98	61.21	43.55	45.68	56.01	60.80	46.70	49.83	62.41	61.56	54.91	51.58
table_tennis_player	64.50	58.12	47.10	46.05	62.57	57.10	50.78	50.18	66.50	59.13	51.02	49.88

Appendix

Table 9: Performance comparison of Llama-3.1-8B-Instruct, Qwen-2.5-7B-Instruct, and Gemini-2.5-Pro using a schema generated by Llama-3.1-8B-Instruct. The models’ average scores across all metrics (EM, F1, R-1, R-L) are significantly lower than in previous tables, suggesting a more challenging evaluation set. Gemini-2.5-Pro was the top performer, achieving the highest average F1 score of 63.07.

Domain	Llama-3.3-70B-Instruct				GPT-4o-mini				Gemini-2.0-Flash
	EM	F1	R-1	R-L	EM	F1	R-1	R-L	EM	F1	R-1	R-L
country	67.82	65.70	59.92	57.01	60.03	64.07	53.53	52.66	60.59	58.12	49.16	52.18
cricket_team	62.15	67.89	51.34	54.91	63.50	59.45	49.72	54.88	62.81	61.03	50.57	51.54
cricketer	65.43	60.01	53.87	50.12	66.82	65.59	51.34	50.93	57.34	60.05	53.54	50.25
economy	60.98	64.21	50.55	52.68	62.80	61.08	54.09	53.31	59.22	62.17	51.05	52.78
table_tennis_player	67.50	61.12	54.10	53.05	59.76	63.03	52.50	49.07	61.47	57.88	50.14	48.29

Appendix

Table 10: Query Generation Model Performance for Llama-3.3-70B-Instruct, GPT-4o-Mini, and Gemini-2.0-Flash. The evaluation uses the challenging schema generated by the Small Language Model (Llama-3.1-8B-Instruct). Despite the resultant drop in average scores, the larger Llama-3.3-70B-Instruct model successfully crossed previous performance baselines, leading the group with an average EM of 64.78 and F1 score of 63.66.

Domain	GPT-4o-mini				Llama-3.3-70B-Instruct				Qwen-2.5-7B-Instruct
	EM	F1	R-1	R-L	EM	F1	R-1	R-L	EM	F1	R-1	R-L
country	68.50	79.03	80.38	76.85	64.50	79.00	79.88	76.35	61.50	73.66	77.88	74.60
cricket_team	52.50	60.22	60.25	56.76	65.00	73.06	73.25	69.76	53.00	60.22	60.25	57.01
cricketer	74.50	74.50	74.50	74.50	75.00	75.00	75.00	75.00	73.50	73.83	74.00	74.00
economy	47.00	51.50	51.50	49.46	55.00	60.50	62.00	58.50	44.50	49.00	49.00	47.00
table_tennis_player	51.50	54.21	57.50	57.50	53.00	55.81	57.00	57.00	43.00	46.06	47.50	47.50

Appendix

Table 11: Out-of-family schema check: performance of three query generation models (GPT-4o-mini, Llama-3.3-70B-Instruct, Qwen-2.5-7B-Instruct) across the ten domains, evaluated with Exact Match (EM), F1, ROUGE-1 (R-1), and ROUGE-L (R-L). GPT-5.4-mini is used only to synthesize the database schema that these queries are written against; it generates no queries itself. This is a partial sweep: only three of the six query models in Table 1 were run against this schema. Average EM nonetheless lands within 2.7 points of the Llama-3.1-8B schemas for every query model tested, despite the schema generator being a substantially more recent model.

Schema source	In	Out	Total	Best EM
Gemini 2.5 Flash	4040	116	4156	80.39
Llama-3.3-70B	4119	111	4230	79.52
Llama-3.1-8B	3598	154	3752	64.79

Appendix

Table 12: Average input and output tokens per SQL-generation call by schema source, averaged over six query models and ten domains, alongside the best EM achieved with that schema. A 13% spread in token cost accompanies a 15.6-point spread in accuracy.

Domain	Gemini-2.0-Flash		Llama-3.3-70B-Instruct		GPT-4o-mini		Gemini-2.5-Pro		Qwen-2.5-7B-Instruct		Llama-3.1-8B-Instruct
	In	Out	In	Out	In	Out	In	Out	In	Out	In	Out
country	3155	100	3571	113	3214	102	3603	114	3286	104	3518	111
cricket_team	4450	119	4991	133	4435	119	5057	135	4388	117	4582	122
cricketer	4220	146	4774	165	4279	148	4736	163	4369	151	4782	165
economy	3327	79	3020	98	3427	101	3061	102	3010	101	3389	98
table_tennis_player	2857	59	2537	99	2546	103	2804	103	2863	102	2474	99

Appendix

Table 13: Average input and output token counts per SQL-generation call across six query models and ten domains, using Gemini 2.5 Flash as the schema generator.

Domain	Gemini-2.0-Flash		Llama-3.3-70B-Instruct		GPT-4o-mini		Gemini-2.5-Pro		Qwen-2.5-7B-Instruct		Llama-3.1-8B-Instruct
	In	Out	In	Out	In	Out	In	Out	In	Out	In	Out
country	3475	110	3603	112	3576	103	3710	107	3517	103	3239	108
cricket_team	4864	130	4802	125	4693	126	4966	124	5175	139	4966	129
cricketer	4636	160	4751	171	4618	168	4501	168	4565	171	4704	149
economy	3286	78	3253	82	3181	73	3515	75	3104	83	3103	79
table_tennis_player	2737	57	2559	57	2890	60	2679	53	2610	60	2605	53

Appendix

Table 14: Average input and output token counts per SQL-generation call across six query models and ten domains, using Llama-3.3-70B-Instruct as the schema generator.

Domain	Gemini-2.0-Flash		Llama-3.3-70B-Instruct		GPT-4o-mini		Gemini-2.5-Pro		Qwen-2.5-7B-Instruct		Llama-3.1-8B-Instruct
	In	Out	In	Out	In	Out	In	Out	In	Out	In	Out
country	3422	139	3187	140	3376	133	3198	136	3344	131	3486	146
cricket_team	4679	120	4452	112	4633	117	4971	126	4711	123	4652	121
cricketer	3370	142	3219	133	3186	148	3530	134	3299	133	3514	137
economy	3180	150	3025	154	3260	153	3272	156	3081	157	3079	153
table_tennis_player	3716	131	3835	128	3915	129	3877	136	3503	139	3743	132

Appendix

Table 15: Average input and output token counts per SQL-generation call across six query models and ten domains, using Llama-3.1-8B-Instruct as the schema generator. Output length is uniformly higher here than under the other two schema sources, tracking this generator’s heavier table fragmentation (Table 3 ).

Domain	Fragmentation Metrics				Remarks
	C	R	RC	D
Country	1.000	1.000	1.000	1.000	All countries participate fully across snapshot and leadership fragments.
Cricket Team	0.992	1.000	1.000	1.000	A few snapshots lack captain assignments; captaincy is modeled as an optional extension.
Cricketer	0.929	1.000	1.000	1.000	Some players do not appear in statistical snapshots due to partial format participation.
Cyclist	0.444	1.000	1.000	1.000	Optional annotation fragments (nicknames, medals) cover only a subset of riders.
Economy	1.000	1.000	1.000	1.000	All countries are fully represented across economic snapshots.

Appendix

Table 16: Fragment metrics for schema made using Gemini-2.5. Values below

1.0

arise from domain semantics such as optional annotation fragments, realistic sparsity in achievements or leadership roles, and intentional one-to-many relationships, rather than schema or reconstruction errors.

Domain	Fragmentation Metrics				Remarks
	C	R	RC	D
Country	0.863	1.000	1.000	1.000	HDI and Gini indicators are unavailable for a subset of countries, reflecting missing socio-economic coverage rather than key loss.
Cricket Team	0.974	1.000	1.000	1.000	One team lacks an associated coach entry; coaching data is optional and incomplete for a small subset.
Cricketer	1.000	1.000	1.000	1.000	All cricketers, years, formats, and statistics are fully covered across fragments.
Cyclist	1.000	1.000	1.000	1.000	All rider, ride, nickname, and snapshot identifiers participate completely in fragments.
Economy	0.000	1.000	1.000	1.000	The EconomyValues fragment contains no temporal records as it is empty; however, other economic fragments jointly reconstruct all time snapshots.

Appendix

Table 17: Fragment metrics for schema made using llama-3.1-8B. Across domains, deviations from perfect completeness arise primarily from optional or sparsely populated fragments (e.g., awards, indicators, leadership roles) and empty indicator tables, while losslessness is generally preserved through fragment unions. All schemas maintain strict referential consistency and disjointness, indicating correct key preservation and absence of duplicate fact units despite domain-specific sparsity.

Domain	Fragmentation Metrics				Remarks
	C	R	RC	D
Country	1.00	1.00	1.00	1.00	All snapshots contain GDP, HDI, and Gini fragments; joins fully reconstruct original records.
Cricket Team	1.00	1.00	1.00	1.00	Rank and record fragments fully cover all team snapshots without duplication or FK violations.
Cricketer	1.00	1.00	1.00	1.00	Snapshot and career statistics form a perfectly complete and lossless temporal decomposition.
Cyclist	0.11	1.00	1.00	1.00	Amateur-career data is sparse by design; however, all rider identifiers remain recoverable via fragment union.
Economy	1.00	1.00	1.00	1.00	Each economic snapshot is associated with indicator values, preserving temporal coverage and reconstruction.

Appendix

Table 18: Fragmentation Metrics Evaluation (LLaMA 3.3 70B Instruct Schema). Most domains achieve perfect referential consistency and lossless reconstruction. Deviations from full completeness arise primarily from optional or sparsely populated fragments, while reduced disjointness reflects intentional one-to-many relationships such as officials, rankings, or equipment histories rather than schema errors.

Explore similar work

Executable Schema Contracts: From Automatic Ingestion to Multi-Source Retrieval
Jun 3, 2026Padmaja Jonnalagedda, Yuguang Yao, Xiang Gao +2Schema Knowledge Graphs
Intuit AI Research
MDB-Link: Hierarchical Schema Linking for Multi-Database Text-to-SQL
Aug 10, 2026Beiyu Xu, Zhenyu Wu, Jiaoyan Chen +1Text-To-Sql Schema
University of Manchester Faculty of Science and Engineering Department of Computer Science
Can LLMs Normalize Databases? A Benchmark and Multi-Agent Framework for Schema Normalization
Sep 11, 2026Dong-Jae Koh, Huisu Kim, SeongHwan Yoon +4Schema Normalization
School of Computer Science and Engineering, Kyungpook National University, South Korea

Jun 3, 2026, cs.CLJ/K move · Enter open · S save

Executable Schema Contracts: From Automatic Ingestion to Multi-Source Retrieval

Padmaja Jonnalagedda, Yuguang Yao, Xiang Gao, Hilaf Hasson+1

Intuit AI Research

Schema Knowledge Graphs Discovery

Open paper AskPDF arXiv

Real-world data spans tables, documents, and semi-structured files with implicit semantics. Querying this data requires integrating evidence across inconsistent schemas and formats, yet existing approaches either demand costly manual engineering or bypass structure entirely. We present a system that automatically discovers an executable schema from raw multi-source data and uses it as a shared contract for knowledge graph construction and query-time retrieval. A closed-world field catalog constrains LLM-based schema discovery to attested fields; deterministic structural analysis infers identity keys, foreign keys, and source hierarchy; and the resulting schema drives extraction, deduplication, and cross-source linking into a provenance-aware knowledge graph. At query time the schema -- optionally extended via a monotonic protocol -- conditions a multi-tool agent routing retrieval across structured lookup, graph traversal, and vector search, returning grounded answers with traceable citations. In controlled zero-shot comparisons using the same LLM, data, and evaluation harness, the system improves over retrieval-only and decomposition-based baselines across four QA benchmarks, with ablations showing that schema-conditioned routing, structural intelligence, and schema-guided construction each contribute to the gains.

Abstract

Explore similar work

Executable Schema Contracts: From Automatic Ingestion to Multi-Source Retrieval

MDB-Link: Hierarchical Schema Linking for Multi-Database Text-to-SQL

Can LLMs Normalize Databases? A Benchmark and Multi-Agent Framework for Schema Normalization