Text-to-SQL translates natural language questions into SQL queries, enabling database access without writing SQL. Fine-tuning open-weight large language models (LLMs) is cost-effective, but existing methods treat SQL as ordinary text and optimize token-level likelihood without explicitly modeling its syntactic, semantic, and logical requirements. Thus, a single incorrect alias or comparison operator can invalidate a query while contributing little to training loss. We propose SQLoom, a constraint-aware fine-tuning framework with supervision at three levels: syntactic validity under SQL grammar and database schema, semantic alignment between query decisions and the question, and logical consistency across dependent clause decisions. SQLoom extends supervised fine-tuning with a unified set-contrastive objective favoring sets of correct outputs over minimally different incorrect ones at each level. LLM agents propose this supervision, which a SQL parser and execution on counterexample databases verify. We also release Spider-Refined, a training dataset correcting 378 flawed Spider annotations and enriching database schemas. Experiments on Spider, BIRD, and three perturbed Spider variants show that SQLoom improves execution accuracy for Qwen3-1.7B from 62.9% zero-shot to 88.7% on Spider development, reaching 89.2% on Spider test and 73.4% on BIRD development. It outperforms fine-tuning, reinforcement learning, and agent-based baselines with up to 70 billion parameters. On question perturbations, SQLoom loses at most 0.6 accuracy points versus up to 12.1 for agent-based methods. It generates SQL in a single pass at 0.42s per query, 23× faster than a GPT-4-based agent method. These results demonstrate the effectiveness of set-contrastive supervision over token-level training.
Figures & tables
Figure 1. Three challenges of SQL generation on the running example. Each row shows a query with one type of error, and highlighted spans mark the wrong decisions. A question with its reference SQL query, followed by three queries that violate syntactic validity, semantic alignment, and logical consistency.
Method
Training signal
Syn.
Sem.
Log.
CodeS ( Li et al., 2024b ) , OmniSQL ( Li et al., 2025 )
token likelihood
✗
✗
✗
ROUTE ( Qin et al., 2025 ) , JOLT-SQL ( Song et al., 2025 )
+ schema linking
✗
✗
✗
ExCoT ( Zhai et al., 2025 )
query-level preference
✗
✗
✗
SQL-R1 ( Ma et al., 2025 ) , Arctic ( Yao et al., 2026 )
query-level reward
✗
✗
✗
Ma et al. ( Ma et al., 2026 )
AST-matching reward
✓
✗
✗
EXPO-SQL ( Lee et al., 2026 )
clause-level reward
✓
✗
✗
Table 1. Supervision signals of representative fine-tuning methods. Syn., Sem., and Log. denote explicit supervision of syntactic validity, semantic alignment beyond tables and columns, and cross-clause consistency.
Figure 2. Model errors of LoRA fine-tuning on Spider Dev (a) per difficulty level, with question counts, and (b) by type. Left, horizontal stacked bars of error counts by type for four difficulty levels. Right, horizontal bars of the share of each error type.
Figure 3. Overview of SQLoom. The constraint synthesizer builds positive and negative sets, a set-contrastive loss favors positives (colored) over minimally different negatives (red) at three levels, and training proceeds in three stages. A question and a database schema feed an LLM, whose outputs are contrasted against positive and negative sets at the syntactic, semantic, and logical levels.
Syntactic
Semantic
Logical
Unit u
decision position t
AST node di aligned to element ei
dependent decision pair (edge of G )
Pc
valid next tokens
reference decision
propagated y′
Nc
invalid next tokens
verified confusions
partially propagated y~′
Mc
position t
tokens of di
tokens of affected clauses
Oracle
parser
execution on D,D′
execution on D,D′
Negatives
invalid
valid, misaligned
valid, aligned, inconsistent
Table 2. Instantiations of the set-contrastive constraint (Eq. 2 ) at the three levels. The last row lists each level’s negatives.
Figure 4. Component dependency graph G on an example query. Each labeled edge marks a component that must be checked when the source component changes. Graph of SQL components connected by labeled dependency edges.
Figure 5. Agent-based constraint synthesizer. Agents or a parser propose candidates in three lanes, LLM-free oracles accept or reject them (dashed), and counterexample databases D′ serve the semantic and logical lanes. Three lanes, each with an agent, candidate constraints, and an oracle, feed accepted syntactic, semantic, and logical constraints. A counterexample generator on the left supplies databases to the semantic and logical lanes.
Figure 6. Robustness and transfer. (a) EX on the Spider variants. (b) EX on Spider Dev (hollow) and Spider-DK (filled); a shorter segment means higher robustness. (c) Cross-dataset EX (S = Spider, B = BIRD). (d) EX when training on Spider vs. Spider-Refined. Four bar charts of accuracy on Spider variants, accuracy drop, cross-dataset transfer, and training data comparison.
Spider
BIRD
Method
Dev
Test
Dev
Fine-tuning
T5-3B + PICARD
79.3
75.1
–
RESDSQL-3B + NatSQL
84.1
79.9
–
SQL-PaLM
82.8
–
–
CodeS-7B
85.4
83.5
57.2
Table 3. EX (%) on Spider and BIRD. The best results are in bold red and the second best are underlined, and † denotes self-consistency. SQLoom uses no BIRD evidence and reports mean ± std over three seeds.
Figure 7. Ablation and cost. (a) EX drop of ablated stages (above the dashed line) and constraint levels (below). (b) Spider Test EX by difficulty after Stages 1 to 3 (S1 to S3). (c) Training time. (d) Per-query latency (log scale). A horizontal bar chart of ablation drops, a line chart of accuracy by difficulty across stages, and two bar charts of training time and inference latency.
Figure 8. Zero-shot EX of general-purpose LLMs on Spider Dev, compared with Qwen3-1.7B fine-tuned by SQLoom (top bar). Horizontal bar chart of zero-shot accuracy of open-weight, proprietary, and code-specialized LLMs, with the fine-tuned model on top.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9. Semantic units of the running example. Each question element is mapped to its SQL decision and contrasted with a confusion (red) that changes the result on D (bottom row) and is pushed down by the semantic constraint. Four columns for target entity, quantifier, distinctness, and join, each showing a question phrase, its SQL decision, a confusion, and the results of the correct and confused queries.
Edit δ on
Dependent
Required check
join path (1:N added)
aggregation, SELECT
duplicate counting; DISTINCT or key
target entity
GROUP BY , SELECT , join
grouping granularity
quantifier
HAVING , sub-query
“at least” / “all” / “none”
boundary / comparison
WHERE , HAVING
inclusive vs. exclusive
superlative / top- k
ORDER BY , LIMIT , aggregation
ties; MAX vs. sort
filter scope
WHERE vs. HAVING
pre- vs. post-aggregation
Appendix
Table 4. Dependency edges of G , the components each edit affects, and the check the propagated query must pass.
Name
Value
Base LLM
Qwen3-1.7B
Agent LLM
GPT-4
Fine-tuning method
LoRA, rank 16, α 32, dropout 0.05
LoRA target modules
q, k, v, o, gate, up, down projections
Learning rate / batch size
5e-5 / 1
Epochs (Stage 1 / 2 / 3)
1 / 2 / 2
Appendix
Table 5. Hyperparameters of SQLoom, selected on a held-out 10% of the training set.
Figure 10. Sensitivity of Spider Dev EX to the temperature β and the logical weight λlog . Two line charts of execution accuracy against beta and lambda_log.
Edge
Edited question q′
Propagated y′
Partially propagated y~′
join path
Courses with a lab section per student?
…JOIN section …COUNT(DISTINCT T.course_id)
same join, keeps COUNT(*)
target entity
Students per department ?
…COUNT(DISTINCT T.id) …GROUP BY D.dept_id
keeps GROUP BY T.id
quantifier
Students who passed all courses.
…NOT EXISTS (…T.grade = ’F’)
keeps WHERE T.grade <> ’F’ (any passed)
boundary
Courses with at least 4 credits.
WHERE credits >= 4
keeps WHERE credits > 4
top- k
Top student in each course.
…grade = (SELECT MAX(grade) …)
keeps ORDER BY grade DESC LIMIT 1
filter scope
Departments with over 100 enrollments in total .
…GROUP BY dept_id HAVING COUNT(*) > 100
keeps the condition in WHERE
Appendix
Table 6. Counterfactual pairs per edge type. q′ is the edited question, y′ the propagated query, and y~′ the partially propagated query that updates only the edited clause.
Figure 11. Legal and illegal next tokens derived by the grammar oracle at two decision positions of the running example. A SQL query split into tokens, with boxes listing valid and invalid tokens at two positions.
Figure 12. A counterexample database for the join-path edit. The inserted lab section (highlighted) makes y~′ wrongly return s1 (✗), while y′ stays correct (✓). Two small tables, takes and section, with one inserted row highlighted, and the results of two queries.
Figure 13. EX versus model size. Agent-based methods are omitted because the size of GPT-4 is undisclosed. Two scatter plots of execution accuracy against the number of parameters on a log scale.
Figure 14. Annotation issues in Spider flagged by the error-mining agent and confirmed manually. Horizontal bar chart of the number of cases for nine issue types.
Type
Definition
Example
Wrong table
wrong or misunderstood table
takes instead of student
Wrong column
mismatched, wrong-key, or non-existent column
course_id when the title is asked
Semantic error
misread question, wrong alias or aggregate, or missing condition
AVG for “total”
Wrong ordering or LIMIT
wrong LIMIT or sort direction
ASC for “highest”
Other model error
e.g., hard-coded outputs
returns a constant
Appendix
Table 7. Error categories used in Figure 2 .
Element
Reference decision
Confusion
entity
FROM student
FROM takes
attribute
course.title
course.course_id
value
dept_name = ’Physics’
dept_name = ’physics’
aggregation
COUNT(DISTINCT course_id)
COUNT(*)
comparison
credits >= 4
credits > 4
quantifier
NOT EXISTS (…)
NOT IN with nullable key
Appendix
Table 8. Confusion types of the semantic constraint. Each confusion replaces one decision of the reference query.
Open-source LLMs provide a flexible alternative to closed-source models for Text-to-SQL but still struggle with complex queries. Finetuning with chain-of-thought (CoT) data improves reasoning, yet standard datasets lack CoT annotations, requiring synthetic generation. Existing synthesis methods only verify the final answer, allowing flawed reasoning steps to corrupt the training data. We propose VCoT-SQL, a framework that improves Text-to-SQL generation through intermediate-step validated CoT supervision and a dedicated SQL validator, which enables small open-source models to outperform frontier closed-source models. VCoT-SQL trains two complementary models on curated reasoning traces: an SQL generator using clause-level CoT templates where reasoning steps are verified via execution and schema alignment checks, and an SQL validator using diagnostic CoT templates to detect syntactic, semantic, and type level errors. We further design a hierarchical schema filtering pipeline that integrates and refines complementary retrieval strategies to distill relevant database elements. VCoT-SQL achieves 66.82% execution accuracy on challenging dataset BIRD dev and 86.8% on Spider test, improving the base 7B model by 7% on BIRD. Notably, our CoT synthesis relies on 7B models yet outperforms methods that use 72B-scale or even proprietary closed source teachers for data generation, demonstrating that rigorous step-level validation can be more effective than scaling up teacher model size. Our dataset and code are available at https://github.com/INTREBID/VCoT-SQL.
Rui Zhang, Yujia Su, Ruixuan Li +2
School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, China
Efficient human-database interaction is a core problem in software engineering. Text-to-SQL (Text2SQL) tackles this by mapping natural language questions to executable SQL queries, enabling automatic SQL synthesis and reliable interaction. Although large language models (LLMs) have advanced significantly, current approaches struggle with poor transferability to open-source LLMs, limited robustness against logic and function errors in complex queries, and inefficiencies in structured search. We introduce SQL-o1, a self-reward-driven heuristic search framework built on an agent-based architecture to enhance model reasoning capabilities. SQL-o1 leverages Monte Carlo Tree Search (MCTS) for structured, multi-step exploration and incorporates a dynamic pruning strategy to accelerate inference without sacrificing accuracy. On the Spider and BIRD benchmarks, SQL-o1 achieves a 10.8-point execution accuracy improvement on the complex BIRD dataset, surpassing even GPT-4-based models. It exhibits strong few-shot generalization and robust cross-model transferability across open-source LLMs. Our code is available at https://github.com/ShuaiLyu0110/SQL-o1.
Shuai Lyu, Haoran Luo, Ripeng Li +3
School of Computer Science, Beijing University of Posts and Telecommunications, China. · College of Computing and Data Science, Nanyang Technological University, Singapore. · State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, China.
Large language models have achieved remarkable progress on Text-to-SQL through reasoning-enhanced fine-tuning, yet existing approaches predominantly rely on massive instruction corpora under the assumption that scale drives performance. We challenge this paradigm by investigating a fundamental question: what is the minimal data requirement for effective Text-to-SQL instruction tuning? We propose LIMIT(Less Is More for Instruction Tuning in Text-to-SQL), a data-centric framework that demonstrates strong database reasoning can emerge from an extremely compact training set when examples are strategically selected. LIMIT operates through four stages: difficulty-aware filtering that identifies samples within the model's learning frontier, chain-of-thought synthesis with consistency-based selection, multi-dimensional quality scoring via LLM-as-judge, and genetic algorithm optimization that jointly maximizes schema coverage and sample quality. On the BIRD and Spider benchmark, LIMIT selects only 796 and 863 samples while achieving 100% table coverage, enabling Qwen3-8B to reach 69.1% and 88.9% execution accuracy.This result surpasses methods trained on 20 times more data and establishes a new state-of-the-art among open-source approaches. Our findings suggest that careful data curation, rather than scale, is the key to efficient Text-to-SQL learning.