Open-source LLMs provide a flexible alternative to closed-source models for Text-to-SQL but still struggle with complex queries. Finetuning with chain-of-thought (CoT) data improves reasoning, yet standard datasets lack CoT annotations, requiring synthetic generation. Existing synthesis methods only verify the final answer, allowing flawed reasoning steps to corrupt the training data. We propose VCoT-SQL, a framework that improves Text-to-SQL generation through intermediate-step validated CoT supervision and a dedicated SQL validator, which enables small open-source models to outperform frontier closed-source models. VCoT-SQL trains two complementary models on curated reasoning traces: an SQL generator using clause-level CoT templates where reasoning steps are verified via execution and schema alignment checks, and an SQL validator using diagnostic CoT templates to detect syntactic, semantic, and type level errors. We further design a hierarchical schema filtering pipeline that integrates and refines complementary retrieval strategies to distill relevant database elements. VCoT-SQL achieves 66.82% execution accuracy on challenging dataset BIRD dev and 86.8% on Spider test, improving the base 7B model by 7% on BIRD. Notably, our CoT synthesis relies on 7B models yet outperforms methods that use 72B-scale or even proprietary closed source teachers for data generation, demonstrating that rigorous step-level validation can be more effective than scaling up teacher model size. Our dataset and code are available at https://github.com/INTREBID/VCoT-SQL.
Figures & tables
Figure 1: Overview of the proposed VCoT-SQL framework. (1) Model Training : two pipelines collect CoT supervision data to train the CoT-enhanced SQL Generator and Validator. (2) Schema Filtering : the question and full database are processed through Pre-SQL, Column, and Value Retrieval to obtain a filtered schema (see Figure 2 ). (3) Inference : the Generator produces an initial SQL from the filtered schema; the SQL is executed, and any syntax errors along with the initial SQL are passed to the Validator to yield the final SQL.
Method
Model
Training Data Size
Spider Dev
Spider Test
BIRD Dev
Spider- DK
Spider- Syn
Spider- Realistic
Prompting with Closed-Source LLMs
GPT-4o OpenAI (2024c)
-
-
81.9
83.0
61.93
70.8
74.2
82.9
GPT-4o-mini OpenAI (2024b)
-
-
82.9
82.8
60.43
73.6
74.2
81.7
GPT-4-turbo OpenAI (2024c)
-
-
83.8
85.2
62.06
71.8
77.5
83.2
Prompting with Open-Source LLMs
Llama3 Touvron et al. (2023)
Llama3-8B
-
69.3
69.1
39.05
57.2
62.6
72.8
Table 1: Performance of Text-to-SQL Models and Methods. To ensure fairness in comparison, we used the results from the greedy decoding released officially by OmniSQL. SQL Generator denotes single-pass generation; VCoT-SQL further applies our SQL Validator. For Training Data Size, prompting-based methods require no Text-to-SQL data (marked as "-"). Among fine-tuning methods, 18K refers to the combined Spider and BIRD training sets, 2.5M to OmniSQL’s SynSQL-2.5M, and our 9K/10K to CoT data constructed from the Spider and BIRD training splits.
Base Model
Spider Dev
Spider Test
BIRD Dev
Spider- DK
Spider- Syn
Spider- Realistic
EX
Δ
EX
Δ
EX
Δ
EX
Δ
EX
Δ
EX
Δ
Only SQL Generator
DeepSeek-Coder-6.7B-Instruct
84.9
-
86.9
-
60.63
-
74.6
-
77.5
-
80.1
-
Qwen2.5-Coder-7B-Instruct
85.3
-
86.7
-
64.86
-
76.8
-
77.6
-
82.7
-
XiYanSQL-QwenCoder-7B
87.2
-
86.4
-
65.38
-
76.6
-
80.0
-
84.3
-
w/ Base Model Validation
Table 2: Ablation study on SQL validation strategies across three backbones. EX denotes execution accuracy (%) and Δ indicates absolute improvement over the generation-only baseline.
Method
Spider Dev
Spider Test
BIRD Dev
Spider- DK
Spider- Syn
Spider- Realistic
Base Model (Qwen2.5-Coder-7B-Instruct)
82.3
81.0
58.87
70.9
71.3
80.9
FT w/ original Spider and BIRD
82.6
83.1
60.04
73.6
74.5
79.5
FT w/ unvalidated CoT-enhanced (Syntax-Order) Spider and BIRD
83.8
84.1
60.69
73.3
74.9
80.0
FT w/ unvalidated CoT-enhanced (Logic-Order) Spider and BIRD
84.4
84.9
61.60
73.5
75.0
79.9
FT w/ validated CoT-enhanced (Syntax-Order) Spider and BIRD
85.0
86.8
62.58
76.3
77.2
82.3
FT w/ validated CoT-enhanced (Logic-Order) Spider and BIRD
85.3
86.7
64.86
76.8
77.6
82.7
Table 3: Ablation Results for Validated SQL Generation on different data construction settings (EX).
Method
Spider Dev
Spider Test
BIRD Dev
Precision
Recall
EX
Precision
Recall
EX
Precision
Recall
EX
Full schema
-
-
83.6
-
-
86.6
-
-
59.71
CodeS schema filter
18.02
99.83
85.4
20.22
99.87
85.1
16.44
97.76
62.63
VCoT-SQL schema filter
30.85
99.79
85.3
34.51
99.89
86.7
31.36
98.09
64.86
w/o Pre-SQL retriever
30.77
98.94
83.8
33.92
97.74
84.4
31.39
94.91
61.15
w/o column retriever
67.35
97.02
83.6
70.02
96.26
84.5
65.48
92.30
61.08
Table 4: Comparison and ablation results of the three-path schema retrieval framework. Precision and Recall assess the discrepancy between the retrieved schema and the gold schema (only the tables and columns appearing in the gold SQL). EX evaluates the downstream generation accuracy of our Generator (Qwen2.5-Coder-7B-Instruct).
Qwen2.5-Coder -7B-Instruct
Full Schema
Schema Filter
Δ Schema Filter
Base Model
53.92
58.87
+4.95
Dual-Validation
61.34
66.49
+5.15
Δ Dual-Validation
+7.42
+7.62
—
Table 5: 2×2 ablation of Dual-Validation and Schema Filter on BIRD Dev (EX).
Method
Data Synthesis Model
Spider
BIRD
Dev
Test
Dev
EXCoT
GPT-4o
-
81.2
59.65
OmniSQL
Qwen2.5-Coder- 32B-Instruct Qwen2.5- 72B-Instruct
80.3
86.6
59.60
VCoT-SQL
Qwen2.5-Coder- 7B-Instruct Qwen2.5- 7B-Instruct
85.3
86.7
64.86
VCoT-SQL
Qwen2.5-Coder- 32B-Instruct Qwen2.5- 72B-Instruct
86.2
87.3
65.06
Table 6: Data synthesis setting and downstream performance (EX). ExCoT and OmniSQL report results trained on their synthesized CoT data from the Spider and BIRD training sets, VCoT-SQL follows the same setting. For the last row, the downstream Generator is fixed to Qwen2.5-Coder-7B-Instruct; only the data synthesis models are scaled.
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 2: An example of the serialized schema representation used in our framework
Method
Spider Dev
Spider Test
BIRD Dev
Spider- DK
Spider- Syn
Spider- Realistic
only SQL Generator
85.3
86.7
64.86
76.8
77.6
82.7
Validator FT w/o multi-stage verification
85.6
86.7
65.78
77.0
77.9
83.1
Validator FT w/ multi-stage verification
85.7
86.9
66.49
77.2
78.5
83.3
Appendix
Table 7: Effect of multi-stage verification on CoT training data quality.
After Validation
Correct
Incorrect
Total
Before
Correct
64.67
0.20
64.86
Incorrect
1.83
33.31
35.14
Validation
Total
66.49
33.51
100
Appendix
Table 8: Transition matrix of execution correctness (%) before and after applying the validator on BIRD Dev.
Unval.
Val.
Val. Gain
Syntax-order
76.13
78.36
+2.23
Logic-order
76.55
78.99
+2.44
Logic-order Gain
+0.42
+0.63
+0.21
Appendix
Table 9: 2 × 2 factorial analysis of template ordering and CoT validation.
Method
Spider Dev
Spider Test
BIRD Dev
Spider- DK
Spider- Syn
Spider- Realistic
Generator only
85.3
86.7
64.86
76.8
77.6
82.7
w/ fine-tuned 7B Validator
85.7
86.9
66.49
77.2
78.5
83.3
w/ GPT-5.4 Validator
86.3
87.4
67.99
80.9
79.3
82.7
Appendix
Table 10: Upper-bound comparison of validators using outputs from the Qwen2.5-Coder-7B-Instruct Generator.
Backbone
DB perturbation
NLQ perturbation
SQL perturbation
Average
DeepSeek-Coder-6.7B-Instruct
86.2–69.6
85.8–74.2
84.3–82.3
85.4–75.8
Qwen2.5-Coder-7B-Instruct
85.0–72.5
84.8–74.9
85.7–81.7
85.1–76.5
XiYanSQL-QwenCoder-7B
87.0–73.5
87.3–76.6
86.6–84.2
87.1–78.3
Appendix
Table 11: VCoT-SQL robustness on Dr.Spider (pre-/post-perturbation EX).
Figure 3: The prompt template for synthesizing clause-level CoT rationales in SQL generation.
Figure 4: The example 1 for synthesizing clause-level CoT rationales in SQL generation.
Figure 5: The example 2 for synthesizing clause-level CoT rationales in SQL generation.
Figure 6: The example 3 for synthesizing clause-level CoT rationales in SQL generation.
Figure 7: The prompt template for synthesizing multi-type CoT rationales in SQL validation.
Figure 8: The example 1 for synthesizing multi-type CoT rationales in SQL validation.
Figure 9: The example 2 for synthesizing multi-type CoT rationales in SQL validation.
Figure 10: The example 3 for synthesizing multi-type CoT rationales in SQL validation.
Figure 11: Inference prompt template for the SQL generation stage.
Figure 12: Inference prompt template for the SQL validation stage.
Text-to-SQL aims to translate natural language questions into executable SQL queries over structured databases, enabling non-expert users to access data intuitively. While recent advances in large language models (LLMs) have shown promise in this task, existing LLM-based approaches often struggle to strike a balance between strong reasoning capabilities and robust generalization. To address these limitations, we propose CoTE-SQL to enhance the LLM-based text-to-SQL generation with three key innovations: (i) self-enhanced reasoning traces distilled from LLMs without human annotation, (ii) structured chain-of-thought (CoT) prompting with modular decomposition and examples retrieval, and (iii) error-aware revision based on SQL execution feedback. Extensive experiments on the Spider and Bird benchmarks demonstrate that CoTE-SQL achieves new state-of-the-art performance among methods built on open-source LLMs with comparable model sizes on Bird (53.39% EX / 59.02 VES) and strong results on Spider (79.60% EX / 77.19 VES), with especially significant gains on complex queries. Results highlight the effectiveness of combining self-enhancement, structured reasoning, and execution-time feedback within an LLM-based framework for text-to-SQL design.
Feng Lyu, Jinfeng Cen, Sijing Duan +4
Central South University · Tsinghua University · Nanjing University +1
Large language models have achieved remarkable progress on Text-to-SQL through reasoning-enhanced fine-tuning, yet existing approaches predominantly rely on massive instruction corpora under the assumption that scale drives performance. We challenge this paradigm by investigating a fundamental question: what is the minimal data requirement for effective Text-to-SQL instruction tuning? We propose LIMIT(Less Is More for Instruction Tuning in Text-to-SQL), a data-centric framework that demonstrates strong database reasoning can emerge from an extremely compact training set when examples are strategically selected. LIMIT operates through four stages: difficulty-aware filtering that identifies samples within the model's learning frontier, chain-of-thought synthesis with consistency-based selection, multi-dimensional quality scoring via LLM-as-judge, and genetic algorithm optimization that jointly maximizes schema coverage and sample quality. On the BIRD and Spider benchmark, LIMIT selects only 796 and 863 samples while achieving 100% table coverage, enabling Qwen3-8B to reach 69.1% and 88.9% execution accuracy.This result surpasses methods trained on 20 times more data and establishes a new state-of-the-art among open-source approaches. Our findings suggest that careful data curation, rather than scale, is the key to efficient Text-to-SQL learning.
Text-to-SQL maps natural language questions to executable SQL queries. Modern databases often contain large and complex schemas, making schema linking a critical step for accurate SQL generation. Existing methods either rely on full-schema generation, which leaves schema linking implicit within a large search space, or use a separate retriever trained with static gold-column supervision, whose targets may be suboptimal for the current generator policy. To address this issue, we propose Adaptive Co-optimization via Empirical Credit Assignment for Text-to-SQL (ACE-SQL), a reinforcement learning (RL) framework that jointly optimizes schema retrieval and SQL generation under execution feedback. ACE-SQL constructs an online column-set pool from generator rollouts and derives adaptive on-policy retrieval targets from the column set most frequently associated with execution-correct rollouts. This induces bidirectional adaptation, where the retriever adapts toward column sets that the generator can execute correctly, while the generator adapts to the retriever's evolving schema selections under execution feedback. With approximately 3k synthetic Text-to-SQL question-database pairs for RL training, ACE-SQL achieves 65.3% greedy execution accuracy on BIRD Dev while using 0.93k output tokens per query. The repository is available at https://github.com/xbchen1/ACE-SQL.
Xiaobing Chen, Ai Jian, Eryu Guo +1
Harbin Engineering University, Harbin, China · Harbin Institute of Technology, Harbin, China · Beijing University of Posts and Telecommunications, Beijing, China