Text-to-SQL systems typically assume a one-to-one mapping between natural-language queries and correct SQL translations. In practice, however, many queries are inherently ambiguous, admitting multiple valid interpretations that yield distinct outputs. We introduce Adversarial Intent Resolution (AIR), a multi-agent framework that actively probes and resolves such ambiguity without any task-specific training. AIR first constructs competing interpretations, verifies through SQL execution that they produce genuinely different results, and generates targeted clarification questions grounded in the observed data differences. Evaluating on leading benchmarks for ambiguous questions, AIR demonstrates strong performance, efficiency, and robust cross-benchmark transferability. On the AMBROSIA benchmark under schema-only context, a realistic production setting where exposing the full database is infeasible, AIR outperforms IntentRL, the state-of-the-art reinforcement learning model, across all metrics (e.g., F1 0.679 vs. 0.618). Crucially, AIR exhibits only ~7% performance degradation without the full database, compared to ~20% for IntentRL. On the AmbiQT benchmark, where ambiguity arises from schema design rather than linguistic structure, AIR achieves higher F1 under both context settings without any benchmark-specific adaptation.
Figures & tables
Figure 1: Overview of the Adversarial Intent Resolution (AIR) framework. Given an ambiguous query, the Anchor agent proposes a primary interpretation while the Adversary agent—constrained to diverge—discovers an alternative. Both are compiled to SQL and executed against the database. If the result sets differ ( Dext=∅ ), a contrastive clarification grounded in actual data differences is presented to the user for further clarification; otherwise the duplicate interpretation is pruned.
Context
Method
Recall
Precision
F1
One-Found
All-Found
Full database
Prompt Baseline (Sonnet 4.5)
0.697
0.592
0.610
0.797
0.590
SOTA (IntentRL)
0.854
0.758
0.786
0.886
0.823
AIR – Claude Sonnet 4.5 (ours)
0.818
0.707
0.729
0.880
0.758
AIR – Gemma 4 26B A4B (ours)
0.804
0.584
0.640
0.876
0.729
AIR – gpt-oss-20b (ours)
0.759
0.728
0.718
0.845
0.673
Schema only
Prompt Baseline (Sonnet 4.5)
0.610
0.539
0.548
0.698
0.520
Table 1: Performance on AMBROSIA test split. Percentages indicate degradation from full database to schema-only context. IntentRL leads under full database context, but under schema-only, the realistic production setting, AIR (Claude Sonnet 4.5) outperforms IntentRL across all metrics with roughly one-third the degradation (e.g., F1: ∼ 7% vs. ∼ 20%). AIR with smaller open-source models also exhibits less degradation than IntentRL, confirming that the robustness stems from the framework architecture.
Metric
AIR
IntentRL
Avg. output / query
1.76
5.00
Avg. unique outputs / query
1.76
2.99
Extra outputs ( > gold)
145 (33.0%)
336 (76.4%)
Table 2: Comparison of SQL execution outputs between AIR and IntentRL on AMBROSIA with the full database as context.
Context
AIR (Claude Sonnet 4.5)
IntentRL
Schema only
204 (46.4%)
102 (23.2%)
Full database
216 (49.1%)
89 (20.2%)
Table 3: Perfect match accuracy on AMBROSIA. A query is correct only if all gold outputs are perfectly matched and no extra outputs are produced. Under schema-only context, AIR retains 94.4% of the best full database performance without requiring actual data for intent discovery or SQL generation.
Figure 3: Effect of confidence thresholds on AMBROSIA with the full database as context. (a) The adversary threshold τadv controls a clear recall–precision tradeoff: higher values filter more aggressively, improving precision at the cost of recall. (b) The anchor threshold τanchor has minimal effect, with all metrics varying within ∼ 3 percentage points.
Context
Method
Recall
Precision
F1
One-Found
All-Found
Full database
IntentRL
0.695
0.590
0.594
0.866
0.525
AIR – Claude Sonnet 4.5 (ours)
0.650
0.758
0.686
0.862
0.439
Schema only
IntentRL
0.684
0.549
0.570
0.842
0.527
AIR – Claude Sonnet 4.5 (ours)
0.642
0.743
0.675
0.847
0.436
Table 4: Performance on AmbiQT validation set to test cross-benchmark generalization. AIR under schema-only achieves near-identical performance to its full database setting. AIR achieves higher F1 through substantially better precision, while IntentRL’s exhaustive generation yields higher recall and All-Found.
As text-to-SQL systems move beyond demonstrations toward real-world deployment, ambiguity in user questions becomes a primary source of errors. Such ambiguities are often subtle, domain- or data-specific, and can silently cause system outputs to deviate from the user's true intent. Ambiguity is traditionally addressed through conversational clarification, which is often inefficient, cognitively demanding, and poorly aligned with real-world user workflows. We propose structured disambiguation, a new paradigm in which ambiguity is resolved through explicit, constrained interactions rather than free-form dialogue. We construct ARCS (Ambiguity Resolution Corpus for SQL), the first text-to-SQL benchmark featuring naturally occurring, unconstrained ambiguities over real-world databases, with complete annotations of all valid ambiguity points, interpretations, and SQL queries. Experimental results show that text-to-SQL remains challenging in the presence of ambiguity: gpt-6-sol achieves only 51% end-to-end execution accuracy, and no open-source model exceeds 27%.
Natural language interfaces to databases aim to translate user questions into executable SQL, yet remain brittle in real-world settings where questions are underspecified and schemas are large and ambiguous. Ambiguity across user questions, database schemas, and model interpretations are central failure modes in NL2SQL, leading to misaligned intent, incorrect schema grounding, and erroneous SQL generation. Existing approaches rely on human clarification or treat ambiguity as a schema representation problem, but these do not scale nor resolve ambiguity autonomously. We propose SOMA-SQL to automatically resolve ambiguity via targeted synthetic query log and ambiguity-driven probing. SOMA-SQL constructs synthetic query log to ground schema interpretation and guide candidate SQL generation; it then executes targeted probing queries, driven by a structured ambiguity taxonomy and candidate disagreements, to produce disambiguation evidence for final SQL selection and repair. This active approach to ambiguity discovery and resolution generalizes across unseen schemas and query distributions without human-in-the-loop. Experiments on six public benchmarks demonstrate that SOMA-SQL improves execution accuracy by 13.0% on average over state-of-the-art baselines, with gains of up to 16.7% on ambiguous questions.
Sai Ashish Somayajula, Marianne Menglin Liu, Chuan Lei +9
Recent advances in large language models (LLMs) have established conversational text-to-SQL as a practical interface between users and databases, often involving multiple turns of clarification and revision. However, existing benchmarks primarily evaluate execution accuracy, leaving the unfolding and shifting of user intent across turns largely uncovered. To address this, we introduce TIDE-Bench, a benchmark for conversational text-to-SQL under chain ambiguity and intent drift evaluation, targeting two recurring patterns: chain ambiguity, where an underspecified question triggers layered clarification with conditional dependencies, and intent drift, where the user retracts and replaces a previously committed request element. Built on 514 anchor SQLs from BIRD, TIDE-Bench comprises 1,542 samples and introduces dedicated metrics for chain identification and drift recognition-resolution beyond execution accuracy. Evaluating 12 advanced LLMs reveals a persistent chain identification bottleneck unaffected by clarification frequency, a wide drift recognition-resolution gap, and overlap between failure modes when jointly activated. The corresponding code of TIDE-Bench is released for further research.
Yujia Liu, Jiayan Lin, Zijin Hong +6
Jinan University · The Hong Kong Polytechnic University · City University of Macau +2