Probing Ambiguity: Adversarial Intent Resolution for Robust Text-to-SQL
Organizations: Capital One
Abstract
Text-to-SQL systems typically assume a one-to-one mapping between natural-language queries and correct SQL translations. In practice, however, many queries are inherently ambiguous, admitting multiple valid interpretations that yield distinct outputs. We introduce Adversarial Intent Resolution (AIR), a multi-agent framework that actively probes and resolves such ambiguity without any task-specific training. AIR first constructs competing interpretations, verifies through SQL execution that they produce genuinely different results, and generates targeted clarification questions grounded in the observed data differences. Evaluating on leading benchmarks for ambiguous questions, AIR demonstrates strong performance, efficiency, and robust cross-benchmark transferability. On the AMBROSIA benchmark under schema-only context, a realistic production setting where exposing the full database is infeasible, AIR outperforms IntentRL, the state-of-the-art reinforcement learning model, across all metrics (e.g., F1 0.679 vs. 0.618). Crucially, AIR exhibits only ~7% performance degradation without the full database, compared to ~20% for IntentRL. On the AmbiQT benchmark, where ambiguity arises from schema design rather than linguistic structure, AIR achieves higher F1 under both context settings without any benchmark-specific adaptation.
Figures & tables
| Context | Method | Recall | Precision | F1 | One-Found | All-Found |
|---|---|---|---|---|---|---|
| Full database | Prompt Baseline (Sonnet 4.5) | 0.697 | 0.592 | 0.610 | 0.797 | 0.590 |
| SOTA (IntentRL) | 0.854 | 0.758 | 0.786 | 0.886 | 0.823 | |
| AIR – Claude Sonnet 4.5 (ours) | 0.818 | 0.707 | 0.729 | 0.880 | 0.758 | |
| AIR – Gemma 4 26B A4B (ours) | 0.804 | 0.584 | 0.640 | 0.876 | 0.729 | |
| AIR – gpt-oss-20b (ours) | 0.759 | 0.728 | 0.718 | 0.845 | 0.673 | |
| Schema only | Prompt Baseline (Sonnet 4.5) | 0.610 | 0.539 | 0.548 | 0.698 | 0.520 |
| Metric | AIR | IntentRL |
|---|---|---|
| Avg. output / query | 1.76 | 5.00 |
| Avg. unique outputs / query | 1.76 | 2.99 |
| Extra outputs ( gold) | 145 (33.0%) | 336 (76.4%) |
| Context | AIR (Claude Sonnet 4.5) | IntentRL |
|---|---|---|
| Schema only | 204 (46.4%) | 102 (23.2%) |
| Full database | 216 (49.1%) | 89 (20.2%) |
| Context | Method | Recall | Precision | F1 | One-Found | All-Found |
|---|---|---|---|---|---|---|
| Full database | IntentRL | 0.695 | 0.590 | 0.594 | 0.866 | 0.525 |
| AIR – Claude Sonnet 4.5 (ours) | 0.650 | 0.758 | 0.686 | 0.862 | 0.439 | |
| Schema only | IntentRL | 0.684 | 0.549 | 0.570 | 0.842 | 0.527 |
| AIR – Claude Sonnet 4.5 (ours) | 0.642 | 0.743 | 0.675 | 0.847 | 0.436 |
Text-to-SQL
Text-to-SQL translates natural-language questions into executable database queries, linking user intent to schemas, values, and SQL logic.
See all 169 papers