SQLoom: Set-Contrastive Learning of Syntactic, Semantic, and Logical Dependencies for Text-to-SQL

Hao Fu; Degang Sun; Wan Wei; Jinxia Wei; Chun Long; Peng Gao

cs.DBNov 2, 2026

SQLoom: Set-Contrastive Learning of Syntactic, Semantic, and Logical Dependencies for Text-to-SQL

Authors: Hao Fu, Degang Sun, Wan Wei, Jinxia Wei, Chun Long, Peng Gao

Organizations: Virginia Tech · Computer Network Information Center, Chinese Academy of Sciences

Abstract

Text-to-SQL translates natural language questions into SQL queries, enabling database access without writing SQL. Fine-tuning open-weight large language models (LLMs) is cost-effective, but existing methods treat SQL as ordinary text and optimize token-level likelihood without explicitly modeling its syntactic, semantic, and logical requirements. Thus, a single incorrect alias or comparison operator can invalidate a query while contributing little to training loss. We propose SQLoom, a constraint-aware fine-tuning framework with supervision at three levels: syntactic validity under SQL grammar and database schema, semantic alignment between query decisions and the question, and logical consistency across dependent clause decisions. SQLoom extends supervised fine-tuning with a unified set-contrastive objective favoring sets of correct outputs over minimally different incorrect ones at each level. LLM agents propose this supervision, which a SQL parser and execution on counterexample databases verify. We also release Spider-Refined, a training dataset correcting 378 flawed Spider annotations and enriching database schemas. Experiments on Spider, BIRD, and three perturbed Spider variants show that SQLoom improves execution accuracy for Qwen3-1.7B from 62.9% zero-shot to 88.7% on Spider development, reaching 89.2% on Spider test and 73.4% on BIRD development. It outperforms fine-tuning, reinforcement learning, and agent-based baselines with up to 70 billion parameters. On question perturbations, SQLoom loses at most 0.6 accuracy points versus up to 12.1 for agent-based methods. It generates SQL in a single pass at 0.42s per query, 23 $\times$ faster than a GPT-4-based agent method. These results demonstrate the effectiveness of set-contrastive supervision over token-level training.

Figures & tables

Figure 1. Three challenges of SQL generation on the running example. Each row shows a query with one type of error, and highlighted spans mark the wrong decisions. A question with its reference SQL query, followed by three queries that violate syntactic validity, semantic alignment, and logical consistency.

Method	Training signal	Syn.	Sem.	Log.
CodeS ( Li et al., 2024b ) , OmniSQL ( Li et al., 2025 )	token likelihood	✗	✗	✗
ROUTE ( Qin et al., 2025 ) , JOLT-SQL ( Song et al., 2025 )	+ schema linking	✗	✗	✗
ExCoT ( Zhai et al., 2025 )	query-level preference	✗	✗	✗
SQL-R1 ( Ma et al., 2025 ) , Arctic ( Yao et al., 2026 )	query-level reward	✗	✗	✗
Ma et al. ( Ma et al., 2026 )	AST-matching reward	✓	✗	✗
EXPO-SQL ( Lee et al., 2026 )	clause-level reward	✓	✗	✗

Table 1. Supervision signals of representative fine-tuning methods. Syn., Sem., and Log. denote explicit supervision of syntactic validity, semantic alignment beyond tables and columns, and cross-clause consistency.

Panel of Figure 2 — Figure 2. Model errors of LoRA fine-tuning on Spider Dev (a) per difficulty level, with question counts, and (b) by type. Left, horizontal stacked bars of error counts by type for four difficulty levels. Right, horizontal bars of the share of each error type.

Figure 3. Overview of SQLoom. The constraint synthesizer builds positive and negative sets, a set-contrastive loss favors positives (colored) over minimally different negatives (red) at three levels, and training proceeds in three stages. A question and a database schema feed an LLM, whose outputs are contrasted against positive and negative sets at the syntactic, semantic, and logical levels.

	Syntactic	Semantic	Logical
Unit $u$	decision position $t$	AST node $d_{i}$ aligned to element $e_{i}$	dependent decision pair (edge of $G$ )
$\mathcal{P}_{c}$	valid next tokens	reference decision	propagated $y^{\prime}$
$\mathcal{N}_{c}$	invalid next tokens	verified confusions	partially propagated $\tilde{y}^{\prime}$
$M_{c}$	position $t$	tokens of $d_{i}$	tokens of affected clauses
Oracle	parser	execution on $\mathcal{D},\mathcal{D}^{\prime}$	execution on $\mathcal{D},\mathcal{D}^{\prime}$
Negatives	invalid	valid, misaligned	valid, aligned, inconsistent

Table 2. Instantiations of the set-contrastive constraint (Eq. 2 ) at the three levels. The last row lists each level’s negatives.

Figure 4. Component dependency graph $G$ on an example query. Each labeled edge marks a component that must be checked when the source component changes. Graph of SQL components connected by labeled dependency edges.

Figure 5. Agent-based constraint synthesizer. Agents or a parser propose candidates in three lanes, LLM-free oracles accept or reject them (dashed), and counterexample databases $\mathcal{D}^{\prime}$ serve the semantic and logical lanes. Three lanes, each with an agent, candidate constraints, and an oracle, feed accepted syntactic, semantic, and logical constraints. A counterexample generator on the left supplies databases to the semantic and logical lanes.

Panel of Figure 6 — Figure 6. Robustness and transfer. (a) EX on the Spider variants. (b) EX on Spider Dev (hollow) and Spider-DK (filled); a shorter segment means higher robustness. (c) Cross-dataset EX (S = Spider, B = BIRD). (d) EX when training on Spider vs. Spider-Refined. Four bar charts of accuracy on Spider variants, accuracy drop, cross-dataset transfer, and training data comparison.

	Spider		BIRD
Method	Dev	Test	Dev
Fine-tuning
T5-3B + PICARD	79.3	75.1	–
RESDSQL-3B + NatSQL	84.1	79.9	–
SQL-PaLM	82.8	–	–
CodeS-7B	85.4	83.5	57.2

Table 3. EX (%) on Spider and BIRD. The best results are in bold red and the second best are underlined, and † denotes self-consistency. SQLoom uses no BIRD evidence and reports mean

\pm

std over three seeds.

Panel of Figure 7 — Figure 7. Ablation and cost. (a) EX drop of ablated stages (above the dashed line) and constraint levels (below). (b) Spider Test EX by difficulty after Stages 1 to 3 (S1 to S3). (c) Training time. (d) Per-query latency (log scale). A horizontal bar chart of ablation drops, a line chart of accuracy by difficulty across stages, and two bar charts of training time and inference latency.

Panel of Figure 8 — Figure 8. Zero-shot EX of general-purpose LLMs on Spider Dev, compared with Qwen3-1.7B fine-tuned by SQLoom (top bar). Horizontal bar chart of zero-shot accuracy of open-weight, proprietary, and code-specialized LLMs, with the fine-tuned model on top.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 9. Semantic units of the running example. Each question element is mapped to its SQL decision and contrasted with a confusion (red) that changes the result on $\mathcal{D}$ (bottom row) and is pushed down by the semantic constraint. Four columns for target entity, quantifier, distinctness, and join, each showing a question phrase, its SQL decision, a confusion, and the results of the correct and confused queries.

Edit $\delta$ on	Dependent	Required check
join path (1:N added)	aggregation, SELECT	duplicate counting; DISTINCT or key
target entity	GROUP BY , SELECT , join	grouping granularity
quantifier	HAVING , sub-query	“at least” / “all” / “none”
boundary / comparison	WHERE , HAVING	inclusive vs. exclusive
superlative / top- $k$	ORDER BY , LIMIT , aggregation	ties; MAX vs. sort
filter scope	WHERE vs. HAVING	pre- vs. post-aggregation

Appendix

Table 4. Dependency edges of

G

, the components each edit affects, and the check the propagated query must pass.

Name	Value
Base LLM	Qwen3-1.7B
Agent LLM	GPT-4
Fine-tuning method	LoRA, rank 16, $\alpha$ 32, dropout 0.05
LoRA target modules	q, k, v, o, gate, up, down projections
Learning rate / batch size	5e-5 / 1
Epochs (Stage 1 / 2 / 3)	1 / 2 / 2

Appendix

Table 5. Hyperparameters of SQLoom, selected on a held-out 10% of the training set.

Panel of Figure 10 — Figure 10. Sensitivity of Spider Dev EX to the temperature $\beta$ and the logical weight $\lambda_{\mathrm{log}}$ . Two line charts of execution accuracy against beta and lambda_log.

Edge	Edited question $q^{\prime}$	Propagated $y^{\prime}$	Partially propagated $\tilde{y}^{\prime}$
join path	Courses with a lab section per student?	…JOIN section …COUNT(DISTINCT T.course_id)	same join, keeps COUNT(*)
target entity	Students per department ?	…COUNT(DISTINCT T.id) …GROUP BY D.dept_id	keeps GROUP BY T.id
quantifier	Students who passed all courses.	…NOT EXISTS (…T.grade = ’F’)	keeps WHERE T.grade <> ’F’ (any passed)
boundary	Courses with at least 4 credits.	WHERE credits >= 4	keeps WHERE credits > 4
top- $k$	Top student in each course.	…grade = (SELECT MAX(grade) …)	keeps ORDER BY grade DESC LIMIT 1
filter scope	Departments with over 100 enrollments in total .	…GROUP BY dept_id HAVING COUNT(*) > 100	keeps the condition in WHERE

Appendix

Table 6. Counterfactual pairs per edge type.

q^{\prime}

is the edited question,

y^{\prime}

the propagated query, and

\tilde{y}^{\prime}

the partially propagated query that updates only the edited clause.

Figure 11. Legal and illegal next tokens derived by the grammar oracle at two decision positions of the running example. A SQL query split into tokens, with boxes listing valid and invalid tokens at two positions.

Panel of Figure 12 — Figure 12. A counterexample database for the join-path edit. The inserted lab section (highlighted) makes $\tilde{y}^{\prime}$ wrongly return s1 (✗), while $y^{\prime}$ stays correct (✓). Two small tables, takes and section, with one inserted row highlighted, and the results of two queries.

Panel of Figure 13 — Figure 13. EX versus model size. Agent-based methods are omitted because the size of GPT-4 is undisclosed. Two scatter plots of execution accuracy against the number of parameters on a log scale.

Panel of Figure 14 — Figure 14. Annotation issues in Spider flagged by the error-mining agent and confirmed manually. Horizontal bar chart of the number of cases for nine issue types.

Type	Definition	Example
Wrong table	wrong or misunderstood table	takes instead of student
Wrong column	mismatched, wrong-key, or non-existent column	course_id when the title is asked
Semantic error	misread question, wrong alias or aggregate, or missing condition	AVG for “total”
Wrong ordering or LIMIT	wrong LIMIT or sort direction	ASC for “highest”
Other model error	e.g., hard-coded outputs	returns a constant

Appendix

Table 7. Error categories used in Figure 2 .

Element	Reference decision	Confusion
entity	FROM student	FROM takes
attribute	course.title	course.course_id
value	dept_name = ’Physics’	dept_name = ’physics’
aggregation	COUNT(DISTINCT course_id)	COUNT(*)
comparison	credits >= 4	credits > 4
quantifier	NOT EXISTS (…)	NOT IN with nullable key

Appendix

Table 8. Confusion types of the semantic constraint. Each confusion replaces one decision of the reference query.

Text-to-SQL

Text-to-SQL translates natural-language questions into executable database queries, linking user intent to schemas, values, and SQL logic.

See all 185 papers

VCoT-SQL: Enhancing Text-to-SQL Generation through Dual Chain-of-Thought Validation Training
Oct 31, 2026Rui Zhang, Yujia Su, Ruixuan Li +2Text-to-SQL Execution-Guided Text-to-SQL
School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, China
SQL-o1: Automatic Text-to-SQL Program Synthesis via Self-Reward Heuristic Search
Feb 17, 2025Shuai Lyu, Haoran Luo, Ripeng Li +3Text-to-SQL LLM-Based Program Synthesis
School of Computer Science, Beijing University of Posts and Telecommunications, China. · College of Computing and Data Science, Nanyang Technological University, Singapore. · State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, China.
LIMIT: Less Is More for Instruction Tuning in Text-to-SQL
Sep 21, 2026Haoyuan Ma, Hengwei Liu, Linjuan Wu +2Text-to-SQL Instruction-Tuning Data Selection
Zhejiang University

Oct 31, 2026J/K move · Enter open · S save

VCoT-SQL: Enhancing Text-to-SQL Generation through Dual Chain-of-Thought Validation Training

Rui Zhang, Yujia Su, Ruixuan Li, Yuhua Li+1

School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, China

Text-to-SQL Execution-Guided Text-to-SQL Structured Reasoning in Language Models Synthetic Data Generation for Language Models

Open paper AskPDF arXiv

Open-source LLMs provide a flexible alternative to closed-source models for Text-to-SQL but still struggle with complex queries. Finetuning with chain-of-thought (CoT) data improves reasoning, yet standard datasets lack CoT annotations, requiring synthetic generation. Existing synthesis methods only verify the final answer, allowing flawed reasoning steps to corrupt the training data. We propose VCoT-SQL, a framework that improves Text-to-SQL generation through intermediate-step validated CoT supervision and a dedicated SQL validator, which enables small open-source models to outperform frontier closed-source models. VCoT-SQL trains two complementary models on curated reasoning traces: an SQL generator using clause-level CoT templates where reasoning steps are verified via execution and schema alignment checks, and an SQL validator using diagnostic CoT templates to detect syntactic, semantic, and type level errors. We further design a hierarchical schema filtering pipeline that integrates and refines complementary retrieval strategies to distill relevant database elements. VCoT-SQL achieves 66.82% execution accuracy on challenging dataset BIRD dev and 86.8% on Spider test, improving the base 7B model by 7% on BIRD. Notably, our CoT synthesis relies on 7B models yet outperforms methods that use 72B-scale or even proprietary closed source teachers for data generation, demonstrating that rigorous step-level validation can be more effective than scaling up teacher model size. Our dataset and code are available at https://github.com/INTREBID/VCoT-SQL.

Abstract

Text-to-SQL

Explore similar work

VCoT-SQL: Enhancing Text-to-SQL Generation through Dual Chain-of-Thought Validation Training

SQL-o1: Automatic Text-to-SQL Program Synthesis via Self-Reward Heuristic Search

LIMIT: Less Is More for Instruction Tuning in Text-to-SQL