Human-Level Text-to-SQL via Reinforcement Learning on Verified Data, Without Pipeline Engineering

Yuxuan Zhu; Tengjun Jin; Yoojin Choi; Daniel Kang

cs.DBMar 20, 2026

Human-Level Text-to-SQL via Reinforcement Learning on Verified Data, Without Pipeline Engineering

Authors: Yuxuan Zhu, Tengjun Jin, Yoojin Choi, Daniel Kang

Organizations: University of Illinois · Bridgewater AIA Labs

Abstract

Translating natural language questions to SQL queries (Text-to-SQL) is a long-standing problem in database research. Recent efforts have focused on improving accuracy by building increasingly complex multi-stage large LLM pipelines, layering task decomposition, schema linking, and model-based query selection on top of an LLM. Despite this growing complexity, a substantial gap (>10%) between such systems and human experts persists on benchmarks, suggesting that pipeline engineering alone has hit a ceiling. We show that human-level Text-to-SQL performance is achievable by fine-tuning an LLM using RLVR on clean data, without pipeline components. In this paper, we identified the dominant bottleneck for RLVR on Text-to-SQL: existing training data contains pervasive annotation errors that mislead optimization. To address this, we developed a multi-round, expert-driven verification pipeline and used it to curate BIRD-Platinum, a dataset of 2.5k verified instances sampled from BIRD Train, with errors corrected in 61% of instances. We show that fine-tuning Qwen3-235B on BIRD-Platinum yields consistent improvements (11-16%) over BIRD Train on Arcwise-Plat (an expert-verified version of BIRD) and Spider2, outperforming SOTA open-source systems by 0.6-16%. Furthermore, we diagnosed two failure modes of standard RLVR on Text-to-SQL. We find that (1) result-based rewards have non-trivial false positive rates, and (2) models systematically ignore the external knowledge in BIRD-style problems. To address them, we propose ReViSQL-BIRD, a specialized reward shaping method that combines result-based rewards with SQL equivalence verification and leverages process rewards for incentivizing external-knowledge use. We fine-tuned Kimi-K2.6 with ReViSQL-BIRD. On Arcwise-Plat, ReViSQL-BIRD-K2.6 is the first method to achieve human-level accuracy (92.96%), outperforming top five open-source systems by 10-22%.

Figures & tables

Figure 1. On Arcwise-Plat-SQL, an expert-verified BIRD Mini-Dev set constructed by prior work ( Jin et al., 2026 ; Arcwise, 2025 ) , ReViSQL-BIRD achieves human-level performance ( Team, 2024 ) using RLVR (Kimi-K2.6) with verified data and reward shaping. ReViSQL-BIRD obviates the need for the complex pipelines ( Xie et al., 2025 ; Sheng and Xu, 2025a ; Agrawal and Nguyen, 2025 ; Dönder et al., 2025 ; Qu et al., 2025 ) with 10–22% higher accuracy.

Method	Dynamic pipeline	Dynamic prompts	Schema linking	Iterative revision	Selection
JoyDataAgent ( at JDCHO, 2025 )	$\bullet$	$\bullet$	$\bullet$	$\bullet$	$\circ$
SiriusAI ( Jiang et al., 2024 )	$\bullet$	$\circ$	$\bullet$	$\bullet$	$\circ$
Agentar-Scale ( Wang et al., 2025 )	$\circ$	$\bullet$	$\circ$	$\bullet$	$\bullet$
SHARE ( Qu et al., 2025 )	$\circ$	$\bullet$	$\bullet$	$\bullet$	$\circ$
OpenSearch ( Xie et al., 2025 )	$\circ$	$\bullet$	$\bullet$	$\bullet$	$\circ$
Distillery ( Maamari et al., )	$\circ$	$\bullet$	$\bullet$	$\bullet$	$\circ$

Table 1. Comparison of the pipeline complexity among top-30 BIRD Leaderboard methods that have disclosed technical details. Each filled dot marks a component the method requires at inference in addition to SQL query generation. ReViSQL-BIRD exceeds these methods on accuracy while requiring none of them.

Correction	Arcwise-Plat-Full	Arcwise-Plat-SQL
SQL query	132 (26.5%)	80 (16.1%)
Question	88 (17.7%)	0
External knowledge	97 (19.4%)	0

Table 2. Correction statistics of two expert-verified datasets based on BIRD mini-Dev.

Figure 2. Our hybrid LLM–human-expert pipeline for correcting text-to-SQL annotation errors in BIRD.

Per-query stats	# tables	# joins	# functions	# aggregations	# set operations	# subqueries	# CTEs	# window func.
BIRD Train	2.08	1.02	1.67	0.62	0.0024	0.088	0.0	0.0012
BIRD-Platinum	2.36	1.06	2.10	0.80	0.024	0.26	0.11	0.0012
Difference	+13.5%	+3.9%	+25.7%	+29.0%	+900%	+195.5%	— $*$	+2.0%

Table 3. Structural complexity comparison between the original BIRD Train and BIRD-Platinum. Values for the datasets denote the average count of each structural component per query; the difference is the relative change from BIRD Train to BIRD-Platinum. We find that gold SQL queries exhibit a significant increase in complexity across every structural metric.

Panel of Figure 3 — Figure 3. Quantitative analysis of the BIRD-Platinum expert curation process. Relying solely on automated LLMs for data correction is insufficient due to poor recall (Fig. 3(a) ). Consequently, BIRD-Platinum uses an expert verification pipeline that requires up to four iterative rounds to fully resolve errors (Fig. 3(b) ). We identified and corrected errors across 52.1% of SQL queries, 26.2% of natural language questions, and 18.2% of external knowledge contexts (Fig. 3(c) ).

Panel of Figure 6 — Figure 6. ReViSQL-BIRD -K2.6 achieves human-level accuracy on BIRD while dominating prior methods. (a, b) Accuracy vs. cost on Arcwise-Plat-SQL and Arcwise-Plat-Full, comparing ReViSQL-BIRD -K2.6 against the top 5 open-source methods on BIRD Leaderboard. (c, d) Accuracy vs. candidate count, comparing ReViSQL-BIRD -K2.6 against single-model baselines.

Method	Arcwise-Plat-SQL (%)	Arcwise-Plat-Full (%)	Spider 2-SQLite (%)	Per-query cost (USD)
SHARE (GPT-5.2)	$71.29$	$75.70$	$21.48$	$5.6\times 10^{-3}$
CSC-SQL (XiYan-32B)	$71.08$	$75.70$	$10.37$	$2.9\times 10^{-3}$
Contextual-SQL (XiYan-32B)	$75.10$	$79.12$	$11.11$	$3.8\times 10^{-1}$
GenaSQL (GPT-5.2)	$81.73$	$84.94$	$34.81$	$6.3\times 10^{-2}$
OpenSearch (GPT-5.2)	$82.93$	$88.15$	$34.07$ $*$	$5.6\times 10^{-2}$
ReViSQL -235B (greedy)	87.34	88.75	37.04	$7.7\times 10^{-3}$

Table 4. ReViSQL -235B (RLVR on BIRD-Platinum) outperforms the strongest prior open-source pipelines across two BIRD-derived benchmarks (Arcwise-Plat-SQL, Arcwise-Plat-Full) and the out-of-distribution Spider 2-SQLite benchmark, at lower per-query cost, demonstrating the generalizability of BIRD-Platinum.

Figure 7. Ablation of ReViSQL-BIRD -K2.6 on Arcwise-Plat-SQL and -Full under greedy decoding. Training on BIRD-Platinum rather than the original BIRD Train improves accuracy by 13.9–17.1%. Reward shaping then further improves accuracy by 2.8–3.8%, reducing error rates by 24.6–38.0%.

Figure 8. Training Qwen3-235B-A22B with RLVR on BIRD-Platinum ( ReViSQL ) yields consistent accuracy improvements (11.0–16.1%) over training on the original BIRD Train, across Arcwise-Plat and Spider2 benchmarks.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 9. A representative instance of two expert-verified variants of BIRD Mini-Dev. (a) The original instance contains an incorrect phrasing in the question (“grades 1 through 12” as opposed to “K-12”) and a gold SQL query that omits a NOT NULL filter, causing it to include data about districts and schools. (b) Arcwise-Plat-Full corrects all three error sources: rephrasing the question to “K-12,” removing the redundant external-knowledge entry, and augmenting gold SQL. (c) Arcwise-Plat-SQL corrects only the gold SQL, preserving the original noisy question and external knowledge.

Attribution	Mechanism	# cases
Question ambiguity	Entity / column / value mapping	9
	Output specification	20
	Aggregation / filter scope	15
	Subtotal	44 (69%)
Model defect	Schema / value grounding	12
	Ignored external knowledge	3

Appendix

Table 5. Classification of the 64 residual ReViSQL-BIRD -K2.6 failures (the most frequent incorrect self-consistency query on each problem where the model disagrees with the gold, across both Arcwise-Plat benchmarks). Most failures trace to ambiguity in the question rather than to defects in the model’s SQL generation.

	System prompt	Verification-aware reward ( $\beta$ )	Process penalties ( $\lambda_{k}$ )
Base model	Evidence	not trained
ReViSQL	Base	$\circ$	$\circ$
ReViSQL-BIRD	Evidence	$\bullet$	$\bullet$

Appendix

Table 6. Prompt and reward configuration of each method. The pre-RLVR base model receives the same evidence-structured prompt as ReViSQL-BIRD . ReViSQL , our general recipe, uses the base prompt and a purely result-based reward. ReViSQL-BIRD couples the evidence-structured prompt with the penalties that enforce it.

Text-to-SQL

Text-to-SQL translates natural-language questions into executable database queries, linking user intent to schemas, values, and SQL logic.

See all 169 papers

ReToolSQL: Agentic Reinforcement Learning for Robust Text-to-SQL
Aug 28, 2026Pratik Kakkar, Chandra Dhir, Ravi Shankar +2Text-to-SQL Execution-Guided Text-to-SQL
JPMorganChase
DualSQL: Text-to-SQL with Multi-Agent Reinforcement Learning
Sep 16, 2026Shijie Chen, Yu Gan, Yeounoh Chung +6Multi-Agent LLM Systems Text-to-SQL
The Ohio State University · Google LLC
FINER-SQL: Boosting Small Language Models for Text-to-SQL
May 5, 2026Thanh Dat Hoang, Thanh Trung Huynh, Matthias Weidlich +4Reward Modeling Small Language Models
Griffith University (Australia) · VinUniversity (Vietnam) · Humboldt-Universitat zu Berlin (Germany) +1

Aug 28, 2026J/K move · Enter open · S save

ReToolSQL: Agentic Reinforcement Learning for Robust Text-to-SQL

Pratik Kakkar, Chandra Dhir, Ravi Shankar, Pareekshit Reddy Gaddam+1

JPMorganChase

Text-to-SQL Execution-Guided Text-to-SQL RL Fine-Tuning RL for Tool Use

Open paper AskPDF arXiv

Recent work has shown that reinforcement learning from execution feedback can substantially improve text-to-SQL performance, often enabling smaller models to match or exceed much larger systems. However, most existing approaches treat SQL generation as a single-turn task, limiting the model's ability to recover from errors through iterative refinement. We present ReToolSQL, a two-stage training framework for text-to-SQL that combines (i) a supervised warm-start on rejection-sampled reasoning traces with (ii) agentic reinforcement fine-tuning (RFT) over multi-turn tool-use trajectories. The key insight is that the two stages act on complementary axes, the supervised fine-tuning (SFT) on verified privileged-teacher traces expands the set of solvable questions (raising pass@k coverage on the hardest cases), while RFT converts that expanded capability into higher single-pass accuracy by teaching the model when to verify, what evidence to retrieve, and how to repair faulty SQL from execution feedback. Applied to Gemma 4 instruction-tuned (31B), RFT alone achieves 73.66% execution accuracy (EX) on the BIRD-SQL development benchmark (74.12% EX with self-consistency). Initializing RFT from the SFT checkpoint (SFT

\to

RFT) yields our strongest model at 74.32% EX single-pass and 74.77% EX with self-consistency. At the time of writing, this ranked first on the BIRD single-model development-set leaderboard. The approach uses composite rewards anchored on execution correctness, requires no human annotation beyond the benchmark itself, and operates within a single dense 31B model, showing that a properly designed SFT

\to

RFT pipeline over tool-use trajectories is a practical path toward robust enterprise-grade text-to-SQL.

Abstract

Text-to-SQL

Explore similar work

ReToolSQL: Agentic Reinforcement Learning for Robust Text-to-SQL

DualSQL: Text-to-SQL with Multi-Agent Reinforcement Learning

FINER-SQL: Boosting Small Language Models for Text-to-SQL