While agentic reinforcement learning (RL) has emerged as a promising paradigm for training small language models (SLMs), existing frameworks are predominantly tool-centric and struggle with data-centric environments. Specifically, applying tool-centric frameworks to stateful tasks like SQL debugging introduces severe spatial inefficiencies since proper environment isolation typically requires a separate database container per rollout. Naively reusing databases without tracking evolving states across multi-turn interactions causes trajectory contamination and cross-trajectory interference, which severely destabilizes policy updates. To overcome these bottlenecks, we introduce BIRD-RL, a novel four-stage agentic RL curriculum that alternately enhances foundational agent reasoning and trajectory-adaptive on-policy exploration to circumvent the instability inherently associated with direct end-to-end learning. Crucially, to support this curriculum, BIRD-RL is driven by a Trajectory-scoped Persistent Execution Infrastructure, which enables safe database replica reuse while preserving trajectory isolation. This design improves spatial RL efficiency by 12.8x, successfully enabling scalable agentic RL for complex data-centric tasks without needs of Kubernettes. Empirical evaluations on the BIRD-CRITIC-SQLite benchmark demonstrate that our 7B and 14B models achieve Success Rates of 44.40% and 48.00%, respectively, showing competitive performance against strong proprietary model-based agents such as Claude-Opus-4.6 Agent (48.20%). Furthermore, BIRD-RL successfully trains unified SLMs capable of both advanced SQL debugging and generation, highlighting its broader potential for multi-objective optimization across heterogeneous, real-world data-centric tasks.
Figures & tables
Figure 1: Overview of the Bird-RL framework. (Left) The four-stage agentic RL curriculum transitioning from static reasoning imitation to interactive stateful exploration. (Right) The Trajectory-Scoped Persistent Execution Infrastructure, enabling scalable and isolated database replica reuse via a customized agent loop and worker-partitioned pool.
Type
Model
Quer.
Mana.
Pers.
Overall
General-Purpose Models
GPT-5.4-Pro
44.01
38.67
39.72
42.00
Claude-Opus-4.6
50.35
44.00
39.01
46.20
Claude-Sonnet-4.5
46.83
36.00
34.75
41.80
Gemini-3.1-Pro
53.52
44.00
41.84
48.80
GLM-4.7
48.24
40.00
33.33
42.80
Kimi-K2.5
46.48
37.33
35.46
42.00
Table 1: Success Rate (SR %) of different models on BIRD-CRITIC-SQLite , evaluated under greedy decoding and grouped by issue category. Bold numbers indicate the highest score in each column , and underlined numbers indicate the second highest. "Quer." refers to query-like issues, "Mana." refers to data-management issues, and "Pers." refers to personalized-function issues.
Model
Base
SQL-Act
Bird-Fixer
Bird-Talon
Δ (%)
Qwen-2.5-Coder-7B
26.20
29.80
33.60
44.40
+18.20
Qwen-2.5-Coder-14B
33.80
36.40
40.00
48.00
+14.20
Table 2: Detailed comparison of Bird-Talon with other strong baselines on BIRD-CRITIC-SQLite . The SQL-ACT agent employs SQLs as actions. Δ denotes the absolute improvement of Bird-Talon compared with the base model.
Strategy
Training-time I/O
Emp. I/O ( ↓ )
Cost Ratio
Per-rollout Container
O(NRSˉ)
2.94 TiB
12.8×
Dynamic Reusable Pool
O((B+q/k)RSˉ)
236 GiB
1.0×
Table 3: Comparison of database lifecycle I/O during interactive rollout execution under trajectory-level isolation. “Emp. I/O” reports the measured online database lifecycle I/O from our implementation and excludes the one-time offline pool materialization cost.
Model
Gen.
Debug.
Think2SQL-7B
56.1
24.0
SQL-R1-7B
58.9
26.6
ReEX-SQL-7B
63.4
-
Reasoning-SQL-7B
64.0
-
Arctic-T2S-R1-7B
68.9
29.8
SQL-Trail-7B
60.1
-
Table 4: Comparison of Bird-Zeno with strong RL-based generation-specialized methods. "Gen." = "Generation" and "Debug." = "Debugging".
Figure 2: Effect of data mixture ratio.
Setting
Reasoning Trace
Action Trace
w/o Stage 3
“… the problematic query joins budget and expense … To get the correct total budget per event, I need to aggregate budget amounts at the event level using MAX or MIN, since each event has one budget …”
SELECT ... SUM(expense.cost), MAX(budget.amount) AS total_budget ... GROUP BY budget.link_to ...
w/ Stage 3 (Ours)
“… there are multiple budget lines …, and each budget line has multiple expense records … The solution is to pre-aggregate expenses per budget line first , then join with budget …”
SELECT SUM(sub_exp.total_cost), SUM(budget.amount) ... FROM ( SELECT link_to_budget, SUM(cost) ... GROUP BY link_to_budget ) sub_exp ...
Table 5: Case study illustrating the effectiveness of Stage 3, with the full trajectory shown in Figure 4 .
Type
Method
Quer.
Mana.
Pers.
Overall
BIRD-Talon-7B
46.13
45.33
40.43
44.40
Training Strategy
(a) Up to Reasoning RL (Stages 1-3)
45.77
36.00
33.33
40.60 ( ↓ 3.80)
(b) Up to Tool-Use Imitation (Stages 1-2)
45.07
33.33
30.50
39.20 ( ↓ 5.20)
(c) Reasoning Imitation Only (Stage 1)
41.55
28.00
26.24
35.20 ( ↓ 9.20)
Reward Design
(d) w/ Partial Correctness
45.77
41.33
37.59
42.80 ( ↓ 1.60)
(e) w/ LLM-as-a-Judge
45.77
38.67
34.04
41.40 ( ↓ 3.00)
Table 6: Ablation study on the training curriculum and reward design of Bird-RL .
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
Statistic
SIX-GYM-SQLite
Total Issues
5,000
# of query-like issues
3,094
# of management issues
1,087
# of personalization issues
819
user query length (mean/max)
162.55/917
issue SQL length (mean/max)
85.50/898
Appendix
Table 7: Data characteristics of SIX-GYM-SQLite. Length statistics are reported as mean/max token counts using the cl100k_base tokenizer.
Category
Count
User Query
Issue SQL
Solution SQL
Test Cases
Query
3,094
183.93/917
86.76/751
96.22/1,976
143.74/1,601
Management
1,087
120.20/535
73.51/802
83.84/802
446.16/1,085
Personalization
819
138.01/421
96.63/898
121.40/1,041
216.33/1,418
Overall
5,000
162.55/917
85.50/898
97.66/1,976
221.38/1,601
Appendix
Table 8: SIX-GYM-SQLite statistics grouped by task category. Length statistics are reported as mean/max token counts.
Figure 3: Statistics of SIX-GYM-SQLite training data, covering SQL feature usage and database-domain distribution.
Model
GPU Hours
BIRD-Talon-7B
93.38
BIRD-Zeno-7B
80.36
BIRD-Talon-14B
152.75
Appendix
Table 9: GPU hours spent to train each backbone model.
Figure 4: Case study comparing debugging trajectories with and without Stage 3 adaptive reasoning reinforcement.
Figure 5: The success rate of Bird-Talon on unseen dialects shows its generalization ability.
Model
pass@1
pass@4
pass@8
pass@16
BIRD-Talon-14B
46.40
57.40
60.60
65.00
Appendix
Table 10: Test-time scaling results of Bird-Talon -14B on BIRD-CRITIC-SQLite .
Figure 6: The inference prompt used for SQL debugging task (system prompt).
Figure 7: The inference prompt used for SQL debugging task (user prompt).
Figure 8: The inference prompt used for SQL generation task (system prompt).
Figure 9: The inference prompt used for SQL generation task (user prompt).
Figure 10: The baseline prompt used for SQL debugging tasks.
Figure 11: The prompt used in Stage 1 reasoning imitation data construction to elicit reasoning thought from teacher model.
Figure 12: The prompt used in Stage 1 reasoning imitation data construction to verify whether teacher model could leverage the reasoning thought to regenerate the correct fixed SQL.
Figure 13: The prompt used to get LLM-as-judge reward (1/2).
Figure 14: The prompt used to get LLM-as-judge reward (2/2).
Figure 15: The prompt for the heuristic rubric reward: instructions used to extract a bug checklist from the task context.
Figure 16: The prompt for the heuristic rubric reward: instructions used to verify whether the reasoning of agent correctly identifies the checklist bugs.
Figure 17: The example of trajectory from Bird-Talon solving SQL debugging task.
Agentic Reinforcement Learning (RL) shows promise for complex tasks, but Text-to-SQL remains mostly restricted to single-turn paradigms. A primary bottleneck is the credit assignment problem. In traditional paradigms, rewards are determined solely by the final-turn feedback, which ignores the intermediate process and leads to ambiguous credit evaluation. To address this, we propose Agentic SQL, a framework featuring a universal two-tiered reward mechanism designed to provide effective trajectory-level evaluation and dense step-level signals. First, we introduce Aggregated Trajectory Reward (ATR) to resolve multi-turn credit assignment. Using an asymmetric transition matrix, ATR aggregates process-oriented scores to incentivize continuous improvement. Leveraging Lyapunov stability theory, we prove ATR acts as an energy dissipation operator, guaranteeing a cycle-free policy and monotonic convergence. Second, Column-Set Matching Reward (CSMR) provides immediate step-level rewards to mitigate sparsity. By executing queries at each turn, CSMR converts binary (0/1) feedback into dense [0, 1] signals based on partial correctness. Evaluations on BIRD show a 5% gain over binary-reward GRPO. Notably, our approach outperforms SOTA Arctic-Text2SQL-R1-7B on BIRD and Spider 2.0 using identical models, propelling Text-to-SQL toward a robust multi-turn agent paradigm.
Long Li, Zhijian Zhou, Jiangxuan Long +5
Griffith University, Brisbane, Australia · Fudan University, Shanghai, China · Shanghai Innovation Institute, China +4
Text-to-SQL models are commonly trained to map questions directly to static queries, whereas real-world database agents operate through stateful, multi-turn interaction with live databases -- inspecting schemas, executing probe queries, diagnosing errors, and revising hypotheses. This creates a critical train-deploy mismatch, as the execution harness that mediates this interaction is introduced only at inference time. To bridge this gap, we propose HarnessSQL, a harness-native post-training framework that preserves the full interaction structure throughout both supervised fine-tuning and reinforcement learning. HarnessSQL builds isolated, executable database environments paired with hidden execution oracles, rolls out teachers directly inside the target SQL harness, and retains only verified trajectories for full-sequence SFT, followed by execution-reward RL. Across Spider 2.0-SQLite, HarnessSQL dramatically boosts the execution accuracy of compact models, raising Qwen3-8B from 15.5% to 45.2% and Qwen3-14B from 22.2% to 54.8%, while transferring effectively to out-of-distribution interactive benchmarks such as BIRD-Interact and LiveSQLBench. Our findings demonstrate that training database agents directly within their execution harness is essential for mastering complex, long-horizon database workflows.
Haolin Yang, Jipeng Zhang, Zhiqiang Qian +4
Hong Kong University of Science and Technology · Microsoft Research · Tsinghua University +1
State-of-the-art Text-to-SQL systems are typically multi-agent pipelines centered around two fundamental tasks: schema linking and SQL generation. However, existing work trains separate models for each task, failing to leverage the synergy between these interrelated tasks. In this work, we propose DualSQL, a new Text-to-SQL system consisting of two agents powered by a single model backbone. The agents share the same model weights and agentic scaffold, enabling joint optimization through a robust multi-agent reinforcement learning (RL) framework. We design three database access tools to facilitate effective multi-step reasoning grounded to interactions with the databases. To improve training and avoid model collapse, we introduce a set of rollout guardrail mechanisms that stabilizes multi-agent RL training, supporting DualSQL to keep improving during training. We also introduce a new SQL correctness metric, robust execution match (REX), to more accurately judge SQL correctness and assign reward signals. Being trained on only 3755 examples, DualSQL-4B achieves an impressive 68.0% execution accuracy on the BIRD development set, matching previous 7B models. DualSQL-8B further improves to 71.1%, outperforming previous state-of-the-art single-model solutions with 32B parameters. These results demonstrate the strength of joint multi-agent reinforcement learning for building high performance Text-to-SQL pipelines.