Natural language database interfaces broaden data access, yet they remain brittle under input ambiguity. Standard approaches often collapse uncertainty into a single query, offering little support for mismatches between user intent and system interpretation. We reframe this challenge through pragmatic inference: while users economize expressions, systems operate on priors over the action space that may not align with the users'. In this view, pragmatic repair -- incremental clarification through minimal interaction -- is a natural strategy for resolving underspecification. We present PleaSQLarify, which operationalizes pragmatic repair by structuring interaction around interpretable decision variables that enable efficient clarification. A visual interface complements this by surfacing the action space for exploration, requesting user disambiguation, and making belief updates traceable across turns. In a study with twelve participants, PleaSQLarify helped users recognize alternative interpretations and efficiently resolve ambiguity. Our findings highlight pragmatic repair as a design principle that fosters effective user control in natural language interfaces.
Figures & tables
Figure 1. Overview of PleaSQLarify , an interactive approach to pragmatic repair applied to text-to-SQL. (1) A user poses an underspecified natural language query. (2) The system generates an initial action space of possible SQL interpretations. (3) The user is presented with the most informative decision variable, i.e., one that splits the action space in a meaningful way. (4) The user iteratively filters the space by engaging with clarifications that contrast alternative interpretations. (5) The predicted query is continuously updated and made traceable across turns, enabling the user to explore, compare, and refine interpretations until their intent is matched. Diagram with five numbered panels connected by arrows. Panel 1 shows a user query (“What was the review of the drama film?”) leading to an SQL query and output. Panel 2 displays a mosaic-like “Action Space” of many small cells representing query variations. Panel 3 highlights decision variables in a color-coded map labeled “Decision Space.” Panel 4 shows a filtered version of this map, narrowing down options. Panel 5 displays the final “Predicted Query” box with tracked SQL components. Arrows loop between panels 3 and 4, indicating iterative refinement, while the flow moves from 1 through 5 overall.
Figure 2. Pragmatic repair in HCI, applied to the text-to-SQL task. A misalignment of priors over system actions, i.e., a user not knowing what actions are possible or considered probable by a system, amplifies ambiguity. Diagram of pragmatic repair in text-to-SQL. It shows how misalignment between user and system priors over actions and intents creates ambiguity. Examples of system actions include valid, unknown, or invalid SQL queries. Through belief updates and successive clarifications ("Not all columns", "Without the product_ids"), the user and system iteratively align their posteriors, narrowing toward the intended query.
Figure 3. Visualization of the algorithm described in Section 5 applied to text-to-SQL. First, an LLM generates diverse outputs, which are then embedded and clustered together. For each cluster, we extract the most characteristic feature based on the lift. Finally, we order decision variables based on expected information gain. Once a user has made a decision, the candidate set is filtered, and the remaining candidates are reclustered. Visualization of the PleaSQLarify algorithm. A user asks a question, and the LLM generates multiple SQL interpretations. These outputs are clustered, and characteristic features are extracted to identify decision variables such as SELECT Opinion, SELECT Title, or SELECT *. Decision variables are ranked by expected information gain, and users iteratively filter and recluster the candidate set until the intended query is isolated.
Figure 4. Iterative candidate filtering and query population using the proposed algorithm Section 5 applied to text-to-SQL. The initial candidate set is iteratively filtered through targeted user feedback on the decision variables proposed by our algorithm until the candidate set consists of a single query, i.e., until the user has identified the query that most closely matches their intent. Visualization of the PleaSQLarify clarification process. Given a user query, the system generates multiple SQL candidates. The user responds to targeted clarification prompts based on decision variables that partition the candidate space, progressively narrowing it until only one executable query remains.
Figure 5. Median per-turn gold-label entropy (top) and functional output similarity (bottom) across three ambiguity types in the AMBROSIA dataset. Clustering-based methods (solid blue) resolve uncertainty more rapidly and converge to functionally coherent hypotheses in fewer turns than baselines (dashed red/orange). Six line plots compare baselines and clustering-based methods across three ambiguity types (Attachment, Vague, Scope). The top row shows median entropy decreasing over turns, while the bottom row shows median similarity increasing. Clustering-based methods (solid blue) reduce uncertainty faster and achieve higher similarity than baselines (dashed red/orange). Shaded regions indicate 95% confidence intervals.
Figure 6. The visual interactive interface consists of the user’s input/utterance field (1), three main analysis views, i.e., the Action Space (2) for exploration, Decision Space (3) for decision making, and Predicted Query (4) for confirmation tasks, and the Predicted Output view (5) showing the predicted result of the database. The interactive interface for PleaSQLarify. It includes (1) a user input field for natural language interfaces, (2) an Action Space visualization showing diverse SQL candidates, (3) a Decision Space view where users select among decision variables, (4) a Predicted Query panel with confidence scores and Yes/No confirmations, and (5) a Predicted Output table showing the retrieved database results.
Figure 7. The Action Space is a UMAP projection of the SQL queries visualized as glyphs and surrounded by Voronoi cells. Their colors correspond to the cluster indices, whereby red and blue shades represent the queries that include the decision variables and those that exclude them. The main purpose of this view is to support data exploration. The users can filter interesting queries by either clicking on a query glyph (1), a cluster of query glyphs (2), or by selecting a group of query glyphs with a lasso (3). The Action Space visualized as a UMAP projection of SQL queries, shown as glyphs within Voronoi cells. Queries are colored red or blue depending on whether they include specific decision variables. Users can explore by selecting a single query glyph, a cluster of queries, or a group of queries with a lasso tool.
Figure 8. When the user accepts a decision variable by clicking on the “Yes” button in the Decision Space (1), the queries that contain/exclude the variables will be filtered in the Action Space (2). In addition, the Predicted Query panel will be updated with atomic features that are likely to occur in the final query (3). Interface view showing how selecting a decision variable updates the query space. When the user clicks “Yes” in the Decision Space, the Action Space is filtered to highlight queries that include the chosen variable, and the Predicted Query panel is updated with corresponding features and likelihoods for the final query.
Figure 9. Every time a decision is made by the user (1), the number of atomic features in the Predicted Query is reduced. In particular, the features are retained only if they are included in the filtered queries. We color the selected decision variables and update the probabilities for the remaining ones. In case a feature has a 100% probability to be part of the final query, it is marked as determined . Every decision made can be reversed by clicking on the Back button displayed in the Decision Space (2). Example of iterative decision-making in the interface. At each step, the user selects a decision variable, and the Predicted Query panel is updated. Atomic features not included in the filtered queries are removed, probabilities for remaining features are adjusted, and features with 100% probability are marked as determined. Every decision made can be reversed by clicking on the Back button displayed in the Decision Space.
Table 1. Task completion by participant. The table summarizes task completion by participant for five query tasks of varying ambiguity type and complexity. Each row lists a task, its ambiguity type, complexity level, and subtasks. Tasks include examples such as finding reviews of “Pulp Fiction,” identifying ratings of movies with Leonardo DiCaprio, and showing film categories offered by festivals. Participants P1 through P12 are shown across the top, with their SQL expertise level indicated as either intermediate or advanced. Cells within the table are color-coded to indicate completion outcomes: green for solved, yellow for partial success with a better query, orange for partial success without the original query, and red for not solved. For example, most participants solved the low-complexity “Pulp Fiction” and “Leonardo DiCaprio” tasks, while the high-complexity festival category task had many red cells, indicating it was not solved by most. The table also distinguishes subtasks 2a (identify interpretation clusters), 2b (understand semantic differences between clusters), and 3 (navigate to utterance interpretation). Overall, the table illustrates how task completion varied by task difficulty, subtask type, and participant expertise.
Figure 10. System utility and usability questionnaire responses. Bar charts summarizing questionnaire responses. On the left, system utility ratings show participants could discover alternative interpretations, clarify queries efficiently and understandably, and feel in control of outputs. On the right, System Usability Scale (SUS) items show mixed but generally positive responses, with most users finding the system easy to use, functions well integrated, and expressing confidence, though some noted complexity and inconsistency.
Figure 11. Observed user interaction flows during exploration and intent clarification. After exploring the initial action space, three common approaches for intent clarification were chosen. In workflow , users navigated to their target query primarily using the Decision Space , whereas in users navigated by filtering individual clauses in the Predicted Query panel. Finally, an output-based approach emerged in which users explicitly chose desired output behavior. Diagram of observed user interaction flows for intent clarification. Three workflows are shown: W1, where users segment and filter the Decision Space to reach a final query; W2, where users filter individual clauses in the Predicted Query panel; and W3, an output-based approach where users select clusters, identify equivalent queries, and validate SQL or outputs. Arrows connect the steps, illustrating alternative navigation paths.
Figure 12. Component utility questionnaire responses, grouped by component. Stacked bar charts of component utility questionnaire responses across three interface components: Action Space, Decision Space with grouped decision variables, and Predicted Query with atomic decision variables. Ratings cover utility for exploration, understanding outputs and clusters, decision variable understandability, decision transparency, and interaction effort. Action Space and grouped Decision Space are rated more positively, while atomic Predicted Query shows lower scores and higher effort.
Pragmatic reasoning in language models studies how models infer and communicate meanings shaped by context, intent, shared knowledge, and social norms.
Natural language interfaces to databases aim to translate user questions into executable SQL, yet remain brittle in real-world settings where questions are underspecified and schemas are large and ambiguous. Ambiguity across user questions, database schemas, and model interpretations are central failure modes in NL2SQL, leading to misaligned intent, incorrect schema grounding, and erroneous SQL generation. Existing approaches rely on human clarification or treat ambiguity as a schema representation problem, but these do not scale nor resolve ambiguity autonomously. We propose SOMA-SQL to automatically resolve ambiguity via targeted synthetic query log and ambiguity-driven probing. SOMA-SQL constructs synthetic query log to ground schema interpretation and guide candidate SQL generation; it then executes targeted probing queries, driven by a structured ambiguity taxonomy and candidate disagreements, to produce disambiguation evidence for final SQL selection and repair. This active approach to ambiguity discovery and resolution generalizes across unseen schemas and query distributions without human-in-the-loop. Experiments on six public benchmarks demonstrate that SOMA-SQL improves execution accuracy by 13.0% on average over state-of-the-art baselines, with gains of up to 16.7% on ambiguous questions.
Sai Ashish Somayajula, Marianne Menglin Liu, Chuan Lei +9
Text-to-SQL systems typically assume a one-to-one mapping between natural-language queries and correct SQL translations. In practice, however, many queries are inherently ambiguous, admitting multiple valid interpretations that yield distinct outputs. We introduce Adversarial Intent Resolution (AIR), a multi-agent framework that actively probes and resolves such ambiguity without any task-specific training. AIR first constructs competing interpretations, verifies through SQL execution that they produce genuinely different results, and generates targeted clarification questions grounded in the observed data differences. Evaluating on leading benchmarks for ambiguous questions, AIR demonstrates strong performance, efficiency, and robust cross-benchmark transferability. On the AMBROSIA benchmark under schema-only context, a realistic production setting where exposing the full database is infeasible, AIR outperforms IntentRL, the state-of-the-art reinforcement learning model, across all metrics (e.g., F1 0.679 vs. 0.618). Crucially, AIR exhibits only ~7% performance degradation without the full database, compared to ~20% for IntentRL. On the AmbiQT benchmark, where ambiguity arises from schema design rather than linguistic structure, AIR achieves higher F1 under both context settings without any benchmark-specific adaptation.
Agentic Text-to-SQL systems can interact with users to clarify underspecified queries before generating SQL. However, a correct execution result does not necessarily imply that the agent has adequately resolved the underlying underspecification: the agent may silently make unverified assumptions that happen to match the intended answer. We show that this behavior is driven in part by premature clarification termination. Although forcing an agent to ask more questions improves execution accuracy, ambiguities are concentrated in earlier interactions, making brute-force questioning inefficient. More importantly, even when explicitly prompted to plan its clarification process, the agent frequently abandons questions that it has already identified as relevant. To address this failure mode, we introduce PlanPool, which externalizes the clarification plan as a mutable question pool. Every planned question must be explicitly asked or dropped before submission, while newly discovered ambiguities can be added during interaction. Across three benchmarks derived from BIRD-Interact and Spider, PlanPool consistently improves ambiguity coverage and reduces silent failures over unconstrained and prompt-based alternatives, while maintaining competitive execution accuracy. Our results highlight an important distinction in agentic reasoning: identifying missing information is not sufficient, and the agent must also reliably maintain and resolve it before committing to an answer.
Wen-Zhi Li, Yue Gong, Konstantinos Kanellis +1
Department of Computer Science Cornell University · Amazon Web Services