What Is a Clarification? Estimating Model-Dependent Input Ambiguity

Jingqin Tang; Tim Tomov; Sebastian Schmidt; Dominik Fuchsgruber; Stephan Günnemann

cs.CLOct 14, 2026

What Is a Clarification? Estimating Model-Dependent Input Ambiguity

Authors: Jingqin Tang, Tim Tomov, Sebastian Schmidt, Dominik Fuchsgruber, Stephan Günnemann

Organizations: School of Computation, Information & Technology, Technical University of Munich · Munich Data Science Institute · Munich Center for Machine Learning

Abstract

Ambiguous inputs can admit multiple interpretations, each potentially yielding a different answer. Existing work largely treats this ambiguity as a property of the input, yet for a particular language model, only some alternatives may be known and answerable. We therefore distinguish world ambiguity, capturing all alternatives admitted by an input, from model ambiguity, capturing those accessible to the responding model. To characterize these alternatives, we formalize natural-language clarifications through three desiderata: uniqueness, faithfulness, and answer equivalence. We show that these desiderata induce a one-to-one correspondence between valid answers and equivalence classes of clarifications, implying that any interpretation can be represented either by its answer or by the clarifications that uniquely express it. We leverage this correspondence to introduce \circleclar{}, a black-box method that estimates the set of answers available to a model by checking consistency along an answer--clarification--answer cycle. Across open-domain QA and executable Text-to-SQL, \circleclar{} achieves strong recovery of these answer sets. In QA, the recovered support yields the strongest overall performance in model-dependent ambiguity detection and enables effective selection among answering, clarifying, and abstaining. Together, these results demonstrate the practical value of a model-dependent view of ambiguity and its characterization through an answer-clarification correspondence.

Figures & tables

Figure 1: Overview of our framework : illustrated with the ambiguous question “Who voiced the snake in The Jungle Book ?”. (a) The world answer support $\mathcal{W}_{T}(x)$ is fixed by the input, while the model-realizable support depends on the model’s knowledge. (b) Proposition 4.1 establishes a one-to-one correspondence between world-supported answers and clarification classes representing the same interpretations. (c) CircleClar estimates the model-realizable support by testing whether candidate answers are recovered through answer–clarification–answer cycles.

Panel of Figure 2 — Figure 2: Model-realizable support on AmbigQA. Mean size (top) and Spearman correlation with world support size (bottom). Error bars show 95% bootstrap CIs over questions.

Dataset	Method	Qwen3-4B	Qwen3-30B-A3B	GPT-5.4-mini	GPT-5.5
Dataset	Method	$F_{1}$ / EM	$F_{1}$ / EM	$F_{1}$ / EM	$F_{1}$ / EM
AmbigQA HQ	ICE	${\color[rgb]{0.8711,0.5625,0.0195}\mathbf{39.23}}$ $\scriptscriptstyle{\pm 46.09}$ / ${\color[rgb]{0.8711,0.5625,0.0195}\mathbf{33.00}}$	${\color[rgb]{0.8711,0.5625,0.0195}\mathbf{52.52}}$ $\scriptscriptstyle{\pm 41.97}$ / ${\color[rgb]{0.8711,0.5625,0.0195}\mathbf{33.00}}$	${\color[rgb]{0.8711,0.5625,0.0195}\mathbf{63.93}}$ $\scriptscriptstyle{\pm 34.35}$ / ${\color[rgb]{0.8711,0.5625,0.0195}\mathbf{31.00}}$	${\color[rgb]{0.0078,0.6211,0.4492}\mathbf{77.91}}$ $\scriptscriptstyle{\pm 29.61}$ / ${\color[rgb]{0.0078,0.6211,0.4492}\mathbf{50.00}}$
	Sampling-50	$33.07$ $\scriptscriptstyle{\pm 43.78}$ / $25.00$	$43.63$ $\scriptscriptstyle{\pm 40.11}$ / $22.00$	$55.51$ $\scriptscriptstyle{\pm 35.23}$ / $24.00$	$61.79$ $\scriptscriptstyle{\pm 27.13}$ / $21.00$
	1-Enumeration	$23.42$ $\scriptscriptstyle{\pm 38.45}$ / $15.00$	$41.38$ $\scriptscriptstyle{\pm 36.37}$ / $12.00$	$59.40$ $\scriptscriptstyle{\pm 34.74}$ / $28.00$	$68.48$ $\scriptscriptstyle{\pm 29.52}$ / ${\color[rgb]{0.8711,0.5625,0.0195}\mathbf{35.00}}$
	CircleClar	${\color[rgb]{0.0078,0.6211,0.4492}\mathbf{60.02}}$ $\scriptscriptstyle{\pm 46.26}$ / ${\color[rgb]{0.0078,0.6211,0.4492}\mathbf{52.00}}$	${\color[rgb]{0.0078,0.6211,0.4492}\mathbf{64.88}}$ $\scriptscriptstyle{\pm 40.76}$ / ${\color[rgb]{0.0078,0.6211,0.4492}\mathbf{49.00}}$	${\color[rgb]{0.0078,0.6211,0.4492}\mathbf{74.09}}$ $\scriptscriptstyle{\pm 34.78}$ / ${\color[rgb]{0.0078,0.6211,0.4492}\mathbf{54.00}}$	${\color[rgb]{0.8711,0.5625,0.0195}\mathbf{68.85}}$ $\scriptscriptstyle{\pm 31.59}$ / ${\color[rgb]{0.8711,0.5625,0.0195}\mathbf{35.00}}$
AmbigQA	ICE	${\color[rgb]{0.8711,0.5625,0.0195}\mathbf{33.71}}$ $\scriptscriptstyle{\pm 44.54}$ / ${\color[rgb]{0.8711,0.5625,0.0195}\mathbf{26.90}}$	${\color[rgb]{0.8711,0.5625,0.0195}\mathbf{41.04}}$ $\scriptscriptstyle{\pm 42.63}$ / ${\color[rgb]{0.8711,0.5625,0.0195}\mathbf{25.50}}$	${\color[rgb]{0.0078,0.6211,0.4492}\mathbf{55.18}}$ $\scriptscriptstyle{\pm 42.50}$ / ${\color[rgb]{0.0078,0.6211,0.4492}\mathbf{35.50}}$	${\color[rgb]{0.0078,0.6211,0.4492}\mathbf{63.93}}$ $\scriptscriptstyle{\pm 35.82}$ / ${\color[rgb]{0.0078,0.6211,0.4492}\mathbf{31.70}}$

Table 1: Model-realizable answer support recovery on QA and Text-to-SQL. We report mean per-question set

F_{1}\pm

standard deviation and exact set match (EM) ( Best and second-best ).

Panel of Figure 3 — Figure 3: Effect of consistency check on support recovery. Change in macro precision and recall from the initial candidate set $\widetilde{\mathcal{Y}}_{T}(x)$ to the final CircleClar estimate $\widehat{\mathcal{M}}_{T}(x)$ .

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Original question	Original AmbigQA rewrites	Decision
“When did the Ottoman Empire take over Palestine?”	“When did the ottoman empire take over palestine for the first time ?” “When did the ottoman empire take over palestine for the second time ?”	Included: take over can refer to either of two historical events.
“Is red bell pepper a fruit or vegetable?”	“In terms of botanicals , is a red bell pepper a fruit or vegetables?” “In culinary , is a red bell pepper a fruit or vegetable?”	Included: botanical and culinary classifications give different valid answers.
“When did the Winter Olympic Games begin and where?”	“ When did the winter olympic games begin?” “ Where did the first winter olympic games begin?”	Not ambiguous: the rewrites split one two-part request.
“What is the latest version of iPhone software?”	“What is the latest version of iphone software, released in 2017 ?” “What is the latest version of iphone software, released in 2016 ?” “What is the latest version of iphone software, released in June 2014 ?”	Not ambiguous: latest has one answer; the years change its scope.
“Who landed the first quad jump in figure skating?”	“Who was the first person to land a quad jump in figure skating?” “Who was the first woman to land a quad jump in figure skating?” “Who landed the first quad jump in figure skating ever ?”	Not ambiguous: first has one answer; “woman” changes its scope.
“Who has the highest number of likes on Instagram?”	“Who has the highest number of likes on instagram in 2017 ?” “Who has the highest number of likes on instagram in 2016 ?” “Who has the highest number of likes on instagram in 2015 ?”	Not ambiguous: highest has one answer; the years change its scope.

Appendix

Table 2: Examples of question selection.

Original AmbigQA	Curated AmbigQA-HQ
D1 (Uniqueness): Split a multi-answer rewrite
Question: “Who is the girl in ZZ Top’s ‘Legs’ video?”
Salesgirl $\rightarrow$ Wendy Frazier Eliminator girls $\rightarrow$ {Danièle Arnaud, Jeana Tomasino, Kymberly Herrin}	Shoe-store clerk $\rightarrow$ Wendy Frazier Dark-haired girl $\rightarrow$ Jeana Tomasino Girl wearing black $\rightarrow$ Danièle Arnaud Girl wearing red $\rightarrow$ Kymberly Herrin
D2 (Faithfulness): Remove an unsupported rewrite
Question: “When did ‘The Haves and the Have Nots’ start?”
TV series $\rightarrow$ May 28, 2013 Stage play $\rightarrow$ 2011 Episodes 105–107 $\rightarrow$ three dates in 2017	TV series $\rightarrow$ May 28, 2013 Stage play $\rightarrow$ 2011

Appendix

Table 3: Examples of rewrite and answer curation in AmbigQA-HQ.

Dataset	Method	Qwen3-4B	Qwen3-30B-A3B	GPT-5.4-mini	GPT-5.5
Dataset	Method	$P$ / $R$	$P$ / $R$	$P$ / $R$	$P$ / $R$
AmbigQA HQ	ICE	${\color[rgb]{0.8711,0.5625,0.0195}\mathbf{39.83}}$ / ${\color[rgb]{0.8711,0.5625,0.0195}\mathbf{83.58}}$	${\color[rgb]{0.8711,0.5625,0.0195}\mathbf{54.58}}$ / ${\color[rgb]{0.8711,0.5625,0.0195}\mathbf{74.67}}$	${\color[rgb]{0.8711,0.5625,0.0195}\mathbf{69.08}}$ / ${\color[rgb]{0.0078,0.6211,0.4492}\mathbf{75.77}}$	$78.50$ / ${\color[rgb]{0.0078,0.6211,0.4492}\mathbf{84.75}}$
	Sampling-50	$37.00$ / $81.58$	$54.37$ / $69.15$	$66.55$ / $67.40$	${\color[rgb]{0.0078,0.6211,0.4492}\mathbf{84.19}}$ / $58.02$
	1-Enumeration	$26.34$ / ${\color[rgb]{0.0078,0.6211,0.4492}\mathbf{84.58}}$	$39.92$ / ${\color[rgb]{0.0078,0.6211,0.4492}\mathbf{79.98}}$	$60.25$ / $74.57$	$79.15$ / ${\color[rgb]{0.8711,0.5625,0.0195}\mathbf{68.62}}$
	CircleClar	${\color[rgb]{0.0078,0.6211,0.4492}\mathbf{63.50}}$ / $79.58$	${\color[rgb]{0.0078,0.6211,0.4492}\mathbf{72.83}}$ / $73.72$	${\color[rgb]{0.0078,0.6211,0.4492}\mathbf{81.93}}$ / ${\color[rgb]{0.8711,0.5625,0.0195}\mathbf{75.48}}$	${\color[rgb]{0.8711,0.5625,0.0195}\mathbf{80.90}}$ / $67.58$
AmbigQA	ICE	${\color[rgb]{0.8711,0.5625,0.0195}\mathbf{33.39}}$ / ${\color[rgb]{0.0078,0.6211,0.4492}\mathbf{87.51}}$	${\color[rgb]{0.8711,0.5625,0.0195}\mathbf{39.86}}$ / ${\color[rgb]{0.0078,0.6211,0.4492}\mathbf{84.58}}$	${\color[rgb]{0.0078,0.6211,0.4492}\mathbf{56.78}}$ / ${\color[rgb]{0.0078,0.6211,0.4492}\mathbf{77.83}}$	${\color[rgb]{0.0078,0.6211,0.4492}\mathbf{61.58}}$ / ${\color[rgb]{0.0078,0.6211,0.4492}\mathbf{83.11}}$

Appendix

Table 4: Macro precision and recall (%) corresponding to the set-

F_{1}

results in Table 1 . Each cell reports

P/R

; recall is one when the reference support is empty. Green and orange mark the best and second-best method, respectively.

Panel of Figure 6 — Figure 6: Sensitivity to the recovery threshold $\tau$ . The dashed line marks the default value $\tau=0.6$ .

Dataset	Method	Qwen3-4B	Qwen3-30B-A3B	GPT-5.4-mini	GPT-5.5
Dataset	Method	$F_{1}$ / EM	$F_{1}$ / EM	$F_{1}$ / EM	$F_{1}$ / EM
AmbigQA Single-answer	ICE	$43.90$ / $37.00$	$61.32$ / $54.00$	$71.20$ / $59.00$	${\color[rgb]{0.8711,0.5625,0.0195}\mathbf{88.50}}$ / ${\color[rgb]{0.0078,0.6211,0.4492}\mathbf{86.00}}$
	Sampling-50	${\color[rgb]{0.8711,0.5625,0.0195}\mathbf{54.22}}$ / ${\color[rgb]{0.8711,0.5625,0.0195}\mathbf{53.00}}$	${\color[rgb]{0.8711,0.5625,0.0195}\mathbf{69.17}}$ / ${\color[rgb]{0.0078,0.6211,0.4492}\mathbf{68.00}}$	${\color[rgb]{0.8711,0.5625,0.0195}\mathbf{78.83}}$ / ${\color[rgb]{0.8711,0.5625,0.0195}\mathbf{77.00}}$	${\color[rgb]{0.0078,0.6211,0.4492}\mathbf{88.67}}$ / ${\color[rgb]{0.0078,0.6211,0.4492}\mathbf{86.00}}$
	1-Enumeration	$48.67$ / $48.00$	$57.40$ / $40.00$	$71.83$ / $59.00$	$87.67$ / ${\color[rgb]{0.8711,0.5625,0.0195}\mathbf{84.00}}$
	CircleClar	${\color[rgb]{0.0078,0.6211,0.4492}\mathbf{68.67}}$ / ${\color[rgb]{0.0078,0.6211,0.4492}\mathbf{68.00}}$	${\color[rgb]{0.0078,0.6211,0.4492}\mathbf{70.07}}$ / ${\color[rgb]{0.8711,0.5625,0.0195}\mathbf{67.00}}$	${\color[rgb]{0.0078,0.6211,0.4492}\mathbf{82.00}}$ / ${\color[rgb]{0.0078,0.6211,0.4492}\mathbf{80.00}}$	$87.17$ / ${\color[rgb]{0.8711,0.5625,0.0195}\mathbf{84.00}}$

Appendix

Table 5: Macro set

F_{1}

and exact set match (EM) (%) for recovering

\mathcal{M}_{T}(x)

on 100 unambiguous AmbigQA questions. Green and orange mark the best and second-best method, respectively.

Panel of Figure 7 — Figure 7: Content-aware response-policy evaluation of the four methods. Each panel reports the unweighted mean of the success rates in the Abstain , Answer , and Clarify reference-cardinality regimes. Higher is better.

Method	Generation	Set construction
Reference construction	10 answers per dataset-provided rewrite, $T=0.3$ .	Accept if $m\geq 0.9$ , and the top answer matches the candidate.
CircleClar	6 answer-set enumerations, $T=0.5$ ; one clarification per candidate, $T=0$ ; 10 re-answers per clarification, $T=0.3$ .	Apply the same consistency check as in reference construction to each candidate, with $m\geq 0.6$ .
ICE	2 clarification samples, $T=1.0$ , and 8 answers per clarification, $T=0.5$ , using the released AmbigQA prompts.	Threshold answer counts separately within each clarification, union the groups, and sweep $t=1,\ldots,8$ .
Sampling-50	50 single-answer samples from the unresolved question, $T=0.5$ .	Discard UNKNOWN outputs, use the remaining sampled answers directly.
1-Enumeration	One answer-set enumeration, $T=0.5$ , using the CircleClar prompt.	Use the enumerated answers directly.

Appendix

Table 6: QA generation budgets and set-construction rules.

Method	Generation	Set construction
Reference construction	5 SQL samples per dataset-provided resolved interpretation, $T=0.2$ .	Accept if $m\geq 0.8$ and the top answer matches the candidate.
CircleClar	5 answer-set enumerations, $T=1.0$ ; one clarification per grouped candidate, $T=0$ ; 5 recovery queries per clarification, $T=0.2$ .	Execute and semantically group the candidates; apply the same consistency check with $m\geq 0.6$ .
ICE	2 clarification samples, $T=1.0$ , using up to 8 distinct rewrites (or the original request if none parse); 8 SQL samples per rewrite, $T=0.5$ .	Threshold answer counts separately within each clarification, union the groups, and sweep $t=1,\ldots,8$ .
Sampling-50	50 single-query samples from the unresolved request, $T=0.5$ .	Use the sampled answers directly.
1-Enumeration	One answer-set enumeration, $T=1.0$ , using the CircleClar prompt.	Use the enumerated answers directly.

Appendix

Table 7: Text-to-SQL generation budgets and set-construction rules.

LLM Uncertainty Estimation

Large language model uncertainty estimation measures when outputs may be wrong, ambiguous, or unreliable to guide safer decisions.

See all 403 papers

Knowing but Not Showing: LLMs Recognize Ambiguity but Rarely Ask Clarifying Questions
May 24, 2026Jinyan Su, Claire CardieLLM Evaluation Clarification Question Generation
1Cornell University
Detect Before You Disambiguate: Perturbation-Based Input Ambiguity Detection in LLMs
Oct 26, 2026Tiejin Chen, Xiaoou Liu, Michael Clifford +5LLM Uncertainty Estimation
Arizona State University · Toyota InfoTech Labs
From Answers to Interpretations: Rethinking Ambiguity-Induced Aleatoric Uncertainty Estimation in LLMs
Sep 3, 2026Omer Nahum, Niv Nayman, Jonathan Fhima +4LLM Uncertainty Estimation Aleatoric Uncertainty
Data and Decision Sciences, Technion, Israel · Amazon Web Services · Computer Vision Group, University of Bern, Switzerland

May 24, 2026J/K move · Enter open · S save

Knowing but Not Showing: LLMs Recognize Ambiguity but Rarely Ask Clarifying Questions

Jinyan Su, Claire Cardie

1Cornell University

LLM Evaluation Clarification Question Generation LLM Decision-Making

Open paper AskPDF arXiv

User queries are often underspecified and may admit multiple valid interpretations. Rather than silently making assumptions about the user's intent, a helpful assistant should surface such ambiguity by asking a clarifying question. Doing so requires two abilities: recognizing that a query is ambiguous, and acting on that recognition by seeking clarification instead of answering directly. To study these abilities, we evaluate models on ambiguous, unambiguous, and disambiguated questions in three settings: standard question answering, explicit ambiguity judgment, and behavioral analysis, where a judge model classifies responses as direct answers, refusals, or clarifying questions. We find a clear gap between recognition and behavior: models often identify ambiguity when explicitly asked to judge it, yet in the QA setting they overwhelmingly default to direct answers. Retrieved context further widens this gap by improving answerability while making models even less likely to ask clarifying questions.

Abstract

LLM Uncertainty Estimation

Explore similar work

Knowing but Not Showing: LLMs Recognize Ambiguity but Rarely Ask Clarifying Questions

Detect Before You Disambiguate: Perturbation-Based Input Ambiguity Detection in LLMs

From Answers to Interpretations: Rethinking Ambiguity-Induced Aleatoric Uncertainty Estimation in LLMs