Recent work has shown that reinforcement learning from execution feedback can substantially improve text-to-SQL performance, often enabling smaller models to match or exceed much larger systems. However, most existing approaches treat SQL generation as a single-turn task, limiting the model's ability to recover from errors through iterative refinement. We present ReToolSQL, a two-stage training framework for text-to-SQL that combines (i) a supervised warm-start on rejection-sampled reasoning traces with (ii) agentic reinforcement fine-tuning (RFT) over multi-turn tool-use trajectories. The key insight is that the two stages act on complementary axes, the supervised fine-tuning (SFT) on verified privileged-teacher traces expands the set of solvable questions (raising pass@k coverage on the hardest cases), while RFT converts that expanded capability into higher single-pass accuracy by teaching the model when to verify, what evidence to retrieve, and how to repair faulty SQL from execution feedback. Applied to Gemma 4 instruction-tuned (31B), RFT alone achieves 73.66% execution accuracy (EX) on the BIRD-SQL development benchmark (74.12% EX with self-consistency). Initializing RFT from the SFT checkpoint (SFT→RFT) yields our strongest model at 74.32% EX single-pass and 74.77% EX with self-consistency. At the time of writing, this ranked first on the BIRD single-model development-set leaderboard. The approach uses composite rewards anchored on execution correctness, requires no human annotation beyond the benchmark itself, and operates within a single dense 31B model, showing that a properly designed SFT→RFT pipeline over tool-use trajectories is a practical path toward robust enterprise-grade text-to-SQL.
Figures & tables
Figure 1: High-level overview of ReToolSQL . A natural-language question and enriched schema are processed by the Gemma 4 31B policy, which interacts with database tools across multiple turns to produce verified SQL. In the training loop, B1,B2,… denote the prompt groups sampled per training step (each containing G=16 multi-turn trajectories for a single question), and R1,R2,… the trajectory-level composite rewards R=∑kwkRk (Section 3.8 ) that yield the group-relative advantages driving the GRPO update. Groups with zero reward variance ( std=0 ) are discarded by dynamic sampling (Section 3.5 ), as they contribute no gradient signal.
Figure 2: M-Schema encoding example (abbreviated). Each column carries its type, semantics, nullability, summary statistics, and representative values, enabling evidence-based schema linking.
Component
Type
Signal
wk
Weighted range ( wkRk )
Rformat
binary
Correct output template compliance
0.2
{0,0.2}
Rsyntax
binary
Predicted SQL parses and executes
0.5
{0,0.5}
Rexec
binary
BIRD EX: row-set equality with gold
2.0
{0,2.0}
Rtables
binary
Predicted table set ⊇ gold table set
0.5
{0,0.5}
Rcolumns
continuous
Jaccard(pred columns, gold columns)
0.5
[0,0.5]
Rnonempty
binary
Executes and returns ≥1 non-null cell
0.1
{0,0.1}
Table 1: Composite reward components and their weights wk . The execution-result term Rexec dominates, while linking terms supply dense schema-grounding signal. The final column lists the already-weighted contribution wkRk of each term (not the raw signal Rk ); summing the positive upper bounds gives a maximum attainable reward of 3.8 .
Configuration
SC
Δ vs. Base
EX (%)
pass@16 (%)
Gemma 4 31B Instruct (base)
✗
—
71.19
76.9
+ SFT
✗
+1.50
72.69
81.29
+ RFT (from base)
✗
+2.47
73.66
77.2
+ RFT + SC
✓
+2.93
74.12
—
+ SFT → RFT
✗
+3.13
74.32
81.94
+ SFT → RFT + SC
✓
+3.58
74.77
—
Table 2: Training-stage ablation on BIRD-SQL dev (1,534 questions) under a common evaluation protocol (parallel execution, 30s timeout, set-based comparison). All rows use the tool suite. SFT = supervised fine-tuning on rejection-sampled privileged-teacher and self-traces (Section 3.10 ); RFT = agentic reinforcement fine-tuning via GRPO from the base model (Section 3.4 ); SC = execution-based self-consistency (majority vote over 16 samples). Δ is temp-0 (greedy) EX relative to the base model.
Rank
System
Params
SC
EX (%)
1
ReToolSQL (SFT → RFT) + SC ‡
31B
Many
74.77
2
ReToolSQL (SFT → RFT)
31B
—
74.32
3
Gemini-SQL2
UNK
Many
74.12
3
ReToolSQL (RFT) + SC ‡
31B
Many
74.12
4
SiriusAI-Text2SQL (Tencent)
27B
Many
73.70
5
ReToolSQL (RFT)
31B
—
73.66
Table 3: BIRD-SQL dev leaderboard (single-model track, 1,534 questions, Aug 26 2026). SC indicates self-consistency. Shaded rows show our experiments and contributions. ‡ pass@16 majority vote.
Inference-Time Tools
Training
No Tools
+ Tools
Tool Δ
No RL (Gemma 4 31B Instruct)
69.69%
71.71%
+2.02
Agentic RL ( ReToolSQL )
70.14%
73.66%
+3.52
RL Δ
+0.45
+1.95
+ Self-Consistency (SC@16)
—
74.12%
+4.43 vs. base
Table 4: Component isolation: RL training × tool access on BIRD-SQL dev (1,534 questions). The interaction effect (+1.5% beyond additive components) confirms that agentic RL specifically teaches the model to leverage tools effectively.
Table 6: Accuracy by difficulty level for key configurations. All rows use the same difficulty annotation (860/443/231 split).
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 3: A representative ReToolSQL trajectory. The policy drafts SQL, detects an empty result via execution, retrieves the correct literal with BM25 search, profiles the denominator column to discover nulls, and emits a guarded final query.
Figure 4: Entropy collapse in the Gemma 4 E4B no-KL run ( β=0 ): execution accuracy, self-consistency (SC@16), and policy entropy versus training checkpoint (shared scale). Entropy collapses within ∼40 steps, after which accuracy plateaus while pass@ k headroom shrinks.
Figure 5: Gemma 4 E4B trained with the decaying beta schedule ( β=0.005→0.001→0 at steps 0/40/80): execution accuracy, self-consistency (SC@16), and policy entropy versus training checkpoint (shared scale). Retaining a small KL penalty early preserves entropy, avoiding the collapse of Figure 4 and sustaining learning.
Reinforcement learning has recently shown promise in improving large language models for Text-to-SQL generation, yet existing methods typically optimize one-shot rewards defined over a single SQL state. Such rewards provide limited guidance for iterative SQL correction and are insufficient to capture the improvement of multi-turn SQL refinement. In this paper, we propose Progress-SQL, a multi-turn reinforcement learning framework with progressive rewards for Text-to-SQL. Our approach introduces an Oracle-guided Diagnostic Tree (ODT), which abstracts SQL queries into clause-level structural profiles and produces diagnostic feedback for next-turn refinement. To provide dense and robust reward signals, we combine ODT-based structural alignment with lexical alignment and define a progressive reward that measures the improvement from the initial SQL to the final SQL. We further incorporate a progression latency reward that favors earlier correctness and an execution status reward that encourages recovery from the invalid SQL. Experiments on BIRD, Spider, and Spider robustness variants demonstrate that our method consistently improves Text-to-SQL performance across both primary and robustness evaluations.
Large language models have driven major advances in Text-to-SQL generation. However, they suffer from high computational cost, long latency, and data privacy concerns, which make them impractical for many real-world applications. A natural alternative is to use small language models (SLMs), which enable efficient and private on-premise deployment. Yet, SLMs often struggle with weak reasoning and poor instruction following. Conventional reinforcement learning methods based on sparse binary rewards (0/1) provide little learning signal when the generated SQLs are incorrect, leading to unstable or collapsed training. To overcome these issues, we propose FINER-SQL, a scalable and reusable reinforcement learning framework that enhances SLMs through fine-grained execution feedback. Built on group relative policy optimization, FINER-SQL replaces sparse supervision with dense and interpretable rewards that offer continuous feedback even for incorrect SQLs. It introduces two key reward functions: a memory reward, which aligns reasoning with verified traces for semantic stability, and an atomic reward, which measures operation-level overlap to grant partial credit for structurally correct but incomplete SQLs. This approach transforms discrete correctness into continuous learning, enabling stable, critic-free optimization. Experiments on the BIRD and Spider benchmarks show that FINER-SQL achieves up to 67.73% and 85% execution accuracy with a 3B model -- matching much larger LLMs while reducing inference latency to 5.57~s/sample. These results highlight a cost-efficient and privacy-preserving path toward high-performance Text-to-SQL generation. Our code is available at https://github.com/thanhdath/finer-sql.
Thanh Dat Hoang, Thanh Trung Huynh, Matthias Weidlich +4
Griffith University (Australia) · VinUniversity (Vietnam) · Humboldt-Universitat zu Berlin (Germany) +1
Recent Text-to-SQL systems increasingly rely on multi-turn interaction, execution feedback, and reinforcement learning. However, most existing methods use execution correctness only as a trajectory-level reward, which provides limited guidance for identifying the SQL decisions responsible for success or failure. We propose SERL-SQL, a selective execution-grounded reinforcement learning framework for multi-turn Text-to-SQL agents. SERL-SQL samples on-policy SQL interaction trajectories and uses a training-only teacher to re-score student actions with execution feedback. The resulting teacher--student likelihood gap is converted into bounded, masked weights that reweight GRPO advantages only on SQL and tool-action tokens. In this way, task rewards preserve the optimization direction, while execution hindsight provides localized credit assignment. Experiments on BIRD, Spider, and cross-domain benchmarks show that SERL-SQL achieves competitive performance, reaching 76.56% execution accuracy on BIRD-Dev and 89.92% on Spider-Test. Moreover, our reward-based selection strategy closely approaches the oracle Best-of-N upper bound and consistently outperforms consistency-based selection, showing that SERL-SQL produces high-quality candidates that can be reliably identified by lightweight execution-grounded rewards. Our code will be released at https://github.com/Ffunkytao/SERL-SQL.
Tao Liu, Tao Feng, Xiangheng Li +9
Zhejiang University, College of Artificial Intelligence · Zhengzhou University, School of Computer Science of Artificial Intelligence · Byte Dance