When to Ask, When to Answer: Strategic Clarification for Ambiguous Reference Resolution in Dialogue

Ibrahim Khebour; Nikhil Krishnaswamy

cs.CLNov 3, 2026

When to Ask, When to Answer: Strategic Clarification for Ambiguous Reference Resolution in Dialogue

Authors: Ibrahim Khebour, Nikhil Krishnaswamy

Organizations: Colorado State University

Abstract

For humans, referential disambiguation is a collaborative process involving clarification questions asked when they will meaningfully change the outcome. However, large language models (LLMs) have been shown to be biased toward premature answering, preferring definite single-turn responses over more clarification exchanges. In this paper, we study the problem of "strategic clarification" -- asking clarification questions in a reference game when their answers will narrow the set of candidate referents. We construct a controlled test environment from the SIMMC 2.1 dataset that records how many questions each ambiguous reference needs to be successfully resolved, and examine how baseline LLM behavior and common post-training methods ask clarifying questions vs. attempt to answer the referent. We also propose BACE (Belief-Aware Critic with Epistemic tracking), a lightweight action chooser method on top of a frozen LM backbone, and show that BACE more consistently chooses the optimal number of clarifying questions than methods such as GRPO and ArCHer.

Figures & tables

Figure 1: Strategic Clarification. At each turn the policy either asks about one attribute, at a cost of 0.1, or answers with an object ID, and the scripted user replies with the gold referent’s value.

Depth	0	1	2	3	4–5	Unres.	Total
Train	7,405	2,613	840	162	15	47	11,082
Dev	617	247	74	18	3	1	960
Devtest	1,701	550	184	33	3	12	2,483
Synth.	–	4,792	6,565	7,243	–	–	18,600

Table 1: Episodes per gold depth, the number of questions the oracle needs (Unres.: unresolvable). Synth.: synthetic episodes added to the real training split.

Figure 2: BACE at one turn. The frozen SFT model writes a candidate question and a candidate answer. The chooser scores both from the backbone’s hidden state and features of a belief $B$ over the candidates, built from the questions and replies so far, and picks the one with the higher value. The environment’s reward trains only the chooser.

Panel of Figure 3 — Figure 3: Questions the oracle needs (rows) against questions the policy asked (columns) on devtest , at the points of Table 3 , with the seeds of a method pooled. Each cell gives its share of the row and the accuracy of the episodes in it. The diagonal is a hit; cells left of it answer early and cells right of it answer late.

Panel of Figure 4 — Figure 4: Hit (top) and hit-and-correct (bottom) on devtest against training episodes, by the number of questions the oracle needs. A hit answers after exactly as many questions as the oracle; a hit-and-correct also ends on the correct referent. Ask-3 includes episodes the oracle can’t resolve within the budget.

Appendix figures & tables27 assets

Supplementary material from the paper’s appendix.

Appendix

Metric	What it measures
Accuracy and cost
Reward	Mean episode return (Eq. 1 ), weighing correct answers against the questions spent.
Accuracy	Fraction of episodes that end on the correct referent.
Questions	Average number of questions asked per episode.
When to ask
Ask F1	Balances precision (share of episodes with a question that were ambiguous) and recall (share of ambiguous episodes that received a question).

Appendix

Table 2: Evaluation metrics, as fractions of all episodes unless stated. Hit and hit-and-correct are also reported within each ask-

k

group. Questions refused past the budget are not counted.

	Natural	SFT	GRPO	ArCHer	BACE
Devtest, 2,483 episodes
Ask F1	.218	1.000	1.000 (.000)	1.000 (.000)	1.000 (.001)
Over	.519	.000	.000 (.000)	.000 (.000)	.000 (.000)
Under	.739	.000	.000 (.000)	.000 (.000)	.001 (.001)
Halluc.	.520	.000	.000 (.000)	.000 (.000)	.000 (.000)
Hit	.369	.981	.964 (.027)	.958 (.013)	.981 (.000)

Appendix

Table 3: End of one pass over the train split (BACE: 150 warm-start episodes, then 552 RL iterations of 20). Natural: Natural Behavior. SFT is the checkpoint every RL run starts from. Metrics as in Table 2 . RL methods: mean over five seeds, SD in parentheses; their training episodes and optimizer steps are in Table 13 .

Ask-1, 550 episodes
	Natural	SFT	GRPO	ArCHer	BACE
Hit	.155	.978	.992 (.005)	.985 (.011)	.978 (.000)
H+C	.044	.976	.990 (.003)	.983 (.010)	.978 (.000)
Ask-2, 184 episodes
Hit	.049	.891	.702 (.342)	.647 (.144)	.853 (.000)
H+C	.016	.859	.660 (.324)	.623 (.148)	.853 (.000)

Appendix

Table 4: Hit and hit-and-correct (H+C) on devtest by the number of questions the oracle needs, at the points of Table 3 . Ask-3 includes episodes the oracle can’t resolve within the budget. Columns as in Table 3 . Mean over seeds, SD in parentheses.

	Natural	SFT	GRPO	ArCHer	BACE
Dev, 960 episodes
Ask F1	.245	1.000	.800 (.447)	1.000 (.000)	.999 (.001)
Over	.455	.000	.000 (.000)	.000 (.000)	.000 (.000)
Under	.746	.000	.200 (.447)	.000 (.000)	.002 (.002)
Halluc.	.507	.000	.148 (.330)	.000 (.000)	.001 (.001)
Hit	.390	.979	.904 (.146)	.954 (.015)	.980 (.000)

Appendix

Table 5: Table 3 on dev , at the same points. Columns and metrics as in Table 3 . RL methods: mean over five seeds, SD in parentheses.

Ask-1, 247 episodes
	Natural	SFT	GRPO	ArCHer	BACE
Hit	.150	.972	.786 (.440)	.969 (.009)	.964 (.002)
H+C	.057	.972	.783 (.438)	.967 (.009)	.964 (.000)
Ask-2, 74 episodes
Hit	.014	.946	.668 (.376)	.649 (.147)	.905 (.000)
H+C	.000	.851	.608 (.341)	.611 (.138)	.905 (.000)

Appendix

Table 6: Table 4 on dev , at the points of Table 5 . Columns as in Table 3 . Mean over seeds, SD in parentheses.

Panel of Figure 5 — Figure 5: Figure 3 on dev.

Panel of Figure 6 — Figure 6: Hit on dev against training episodes, by the number of questions the oracle needs: the share of episodes where the policy answers after exactly as many questions as the oracle. Ask-3 includes episodes the oracle can’t resolve within the budget. Each line is a method’s five-seed mean, starting at SFT (dashed).

Panel of Figure 7 — Figure 7: Hit-and-correct on dev against training episodes, by the number of questions the oracle needs: a hit that also ends on the correct referent. Plots and lines as in Figure 6 .

Dev, 960 episodes
Policy	Acc.	Qs	Reward	Ask F1	Over	Under	Ask-1	Ask-2	Ask-3
Direct Answer	.761	.00	.523	.000	.000	1.000	.397	.176	.136
Natural Behavior	.766	.69	.463	.245	.455	.746	.393	.270	.045
Ask Once Then Answer	.881	1.00	.662	.526	1.000	.000	.785	.378	.318
Ask Until Resolved	.865	1.37	.593	.526	1.000	.000	.721	.392	.273
Uncertainty-gated	.823	.55	.591	.762	.000	.385	.555	.365	.409

Appendix

Table 7: The untrained policies. Ask-1, Ask-2, Ask-3: accuracy on the episodes that need one, two, and three or more questions. Other metrics as in Table 2 .

	Pooled	Depth 1	Depth 2	Depth 3
Real episodes	3,677	2,613	841	223
Real vs. real	0.52	0.50	0.54	0.57
Real vs. synthetic	0.70	0.59	0.60	0.63

Appendix

Table 8: Held-out AUC of a bag-of-n-grams classifier separating two sets of training episodes (a value of 0.5 means the two are indistinguishable). Each comparison uses as many synthetic episodes as real ones. Real vs. real splits the real episodes at random.

Training pool	real + synthetic, 29,635 episodes
Examples	54,405 of 74,110 (within 1,280 tokens)
LoRA rank / $\alpha$	16 / 32
Learning rate	$10^{-4}$
Batch	4, gradient accumulation 4
Epochs	1 (3,400 optimizer steps)
Question text	one template per attribute

Appendix

Table 9: SFT.

LoRA rank / $\alpha$	16 / 32
Rollouts per episode ( $K$ )	4
Sampling temperature	0.8
Episodes per batch	8 (gradient accumulation 4)
PPO clip $\epsilon$ / passes per batch	0.2 / 2
KL coefficient to SFT ( $\beta$ )	0.04
Learning rate	$5\times 10^{-5}$

Appendix

Table 10: GRPO (trajectory level).

Actor LoRA rank / $\alpha$	16 / 32
Actor learning rate	$5\times 10^{-5}$
Actor KL coefficient to SFT	0.04
Actor schedule	10 critic-only iterations, then 3 steps
	of 4 transitions per iteration (54)
Actor sampling temperature	0.8
Question attribute	sampled from the actor (argmax at evaluation)

Appendix

Table 11: ArCHer.

Generator	SFT checkpoint, frozen
Prompt	ArCHer’s, plus the candidates in $B$
Samples per branch	2, temperature 0.8
Finalist	random in training, highest length-
	normalized log-probability at evaluation
Critic LoRA rank / $\alpha$	8 / 16
Critic heads	2 Q/V pairs; hidden state and features

Appendix

Table 12: BACE.

Method	Learns from	Episodes	Env. steps	Opt. steps	Params	GPU h
SFT	oracle labels	29,635	–	3,400	40.4M	3.3
GRPO	reward	11,035	58.1k–58.7k	38–66	40.4M	3.4–3.5
ArCHer	reward	11,032	18.3k–18.5k	5,654	67.9M	5.2
BACE	reward	11,190	22.3k–22.5k	28,164–28,308	27.7M	18.2–19.0

Appendix

Table 13: Training cost, as a range over seeds where it varies. Opt. steps: optimizer steps on every trained parameter, BACE’s 564 to 708 warm-start steps included. Params: trained parameters (ArCHer: actor 40.4M, critic adapter 20.2M, heads 7.4M; BACE: critic adapter 20.2M, heads 7.5M). The RL methods start from the SFT checkpoint, whose cost comes on top of theirs. BACE’s environment steps are counted on seeds 0, 1 and 7.

Dev
Pool	Episodes	Ask F1	Acc.	Qs	Reward	Multi
Pilot	150	.978	.857	.54	.661	.354
Real	11,035	1.000	.965	.47	.883	.708
Real + synth.	29,635	1.000	.986	.48	.925	.865
Devtest
Pilot	150	.973	.867	.46	.688	.330

Appendix

Table 14: SFT trained on each pool. Multi: accuracy on a fixed sample of up to 100 episodes that need two or more questions (all 96 on dev, a sample of 100 on devtest).

	Reward		Devtest
	Dev	Devtest	Hit	Acc.	Ask-2 H+C	Ask-3 H+C
SFT (real + synth.)	.925	.938	.981	.990	.859	.479
BACE, with the line	.930 (.004)	.935 (.009)	.977 (.007)	.989 (.005)	.832 (.038)	.514 (.084)
BACE, without the line	.917 (.005)	.922 (.005)	.974 (.007)	.982 (.003)	.804 (.028)	.361 (.241)

Appendix

Table 15: BACE with and without the Still possible line in the prompt that the generator and the chooser share, after 100 RL iterations (2,000 episodes); mean over seeds 0, 1 and 7, SD in parentheses. SFT (real + synth.) is the generator of both rows and of the main comparison.

Change	All	$b=1$	Gap
	Flipped
No line	0	0	.0003
Line lists every candidate	0	0	.0002
$\varphi,\psi$ from all candidates	220	218	.2988

Appendix

Table 16: The trained chooser of seed 0’s full pass on 300 dev episodes, at the 690 states where it may ask (272 with

b=1

), with one route of the belief changed and the sampled candidates fixed. Flipped: decisions that change. Gap: mean absolute change in the ask-minus-answer value gap.

	Reward		Devtest
	Dev	Devtest	Hit	Acc.	Ask-2 H+C	Ask-3 H+C
SFT (real + synth.)	.925	.938	.981	.990	.859	.479
BACE, generator SFT (real + synth.)	.930 (.004)	.935 (.009)	.977 (.007)	.989 (.005)	.832 (.038)	.514 (.084)
BACE, generator SFT (real)	.936 (.000)	.945 (.000)	.954 (.000)	.995 (.000)	.739 (.000)	.604 (.000)

Appendix

Table 17: BACE with each SFT checkpoint as its generator, after 100 RL iterations (2,000 episodes); mean over seeds 0, 1 and 7, SD in parentheses. SFT (real): trained on real episodes only, scored on its own in Table 14 . The first BACE row is the run with the line in Table 15 .

	8-iteration runs					Full pass
	Warm	It. 5		End (it. 8)		Warm		End
Seed	dev	dev	devtest	dev	devtest	dev	devtest	dev	devtest
0	.923	.905	.900	.836	.827	.923	.932	.932	.940
1	.932	.932	.939	.909	.918	.775	.772	.932	.940
7	.931	.931	.940	.930	.939	.910	.912	.932	.940
17	.932	.922	.931	.927	.935	.886	.876	.930	.940

Appendix

Table 18: BACE’s reward per seed. Warm: the chooser after its warm start, before RL; the 8-iteration runs fit their own warm start.

It.	Amb.	Res.	Asks	Reward	Amb.	Res.	Asks	Reward
	Seed 0 (rerun)				Seed 7
Warm	$+1.71$	$-0.09$	.95	.923	$+1.16$	$-0.05$	.98	.931
1	$+1.86$	$-0.06$	.96	.931	$+1.59$	$+0.03$	.98	.791
2	$+1.12$	$-0.37$	.95	.926	$+1.54$	$-0.05$	.98	.932
3	$-0.08$	$-0.15$	.00	.500	$+1.57$	$-0.14$	.98	.932
4	$-0.01$	$-0.14$	.21	.591	$+1.68$	$-0.12$	.98	.932

Appendix

Table 19: The chooser on dev after each RL iteration. Amb., Res.: median ask-minus-answer value gap on states with more than one candidate left and with one. Asks: share of the ambiguous states where it asks. Reward: dev reward.

Seed 0
Draw	Ask rate	Sign agr.	Check	Reward	Under
a	.449	.847	pass	.905	.023
b	.390	.856	pass	.712	.531
c	.415	.883	pass	.932	.000
d	.458	.838	pass	.881	.009
Seed 7

Appendix

Table 20: Four draws of the 8-iteration recipe on each of two seeds. Ask rate and sign agreement of the warm-started chooser, and dev reward and under-asking at the end of training.

Draw	Not exact	$B\neq S$	Episodes
0	4	83	8
1	4	91	12
2	8	86	10
3	5	95	13
4	9	83	9

Appendix

Table 21: Belief against the survivor set

S

over 348,600 replayed states per draw. Not exact: states where the belief no longer claims to equal

S

B\neq S

: states where it does claim to, and differs. Episodes: the episodes those states come from.

Dev
Method	Qs asked	Spatial	Bits	Best attr.	Repeat	No split
Natural Behavior	758	.029	0.18	.127	.496	.865
SFT	460	.265	1.65	.858	.015	.007
GRPO	443	.234	1.68	.843	.024	.001
ArCHer	452	.237	1.62	.799	.016	.032
BACE	473	.309	1.57	.815	.032	.000

Appendix

Table 22: Questions per method, mean over seeds (BACE: seeds 0, 1 and 7). Qs asked: questions on the split. Spatial: share about a half of the scene or a neighbor. Bits: mean entropy removed from the survivor set. Best attr.: share whose attribute has the highest information gain. Repeat: share that repeats an attribute already asked or pinned by the history. No split: share that rules out no survivor.

Figure 10: A devtest episode that needs two questions: “the grey one” names a color the policy can’t see, so it asks for the brand and then for the side of the scene before answering.

Gold referent of the ambiguous episodes
	Train	Dev	Devtest
Labelled turns	11,753	1,050	2,657
Dropped	671	90	174
Unambiguous	7,400	617	1,701
Recoverable from history	5	0	0
Needs a question	3,677	343	782

Appendix

Table 23: From labelled user turns to episodes, and how the gold referent of each ambiguous episode was recovered.

Devtest, 380 and 402 episodes
	Single gold			Fallback gold
Method	Hit	Acc.	Reward	Hit	Acc.	Reward
Natural Behavior	.176	.329	$-$ .402	.075	.289	$-$ .451
SFT (start of RL)	.953	.976	.825	.925	.963	.783
GRPO	.907 (.060)	.961 (.033)	.802 (.058)	.863 (.108)	.930 (.057)	.730 (.101)
ArCHer	.901 (.026)	.956 (.012)	.788 (.022)	.833 (.057)	.913 (.028)	.694 (.051)

Appendix

Table 24: The ambiguous episodes split by how the gold referent was recovered (Table 23 ). Single gold: a later turn names one object. Fallback gold: a later turn narrows the mention to two objects, or a seeded random choice. Points and metrics as in Table 3 . Mean over seeds, SD in parentheses.

Clarification Question Generation

Clarification question generation teaches AI systems to identify missing intent and ask targeted follow-ups before answering or acting.

See all 44 papers

When and How Should an Agent Clarify? CIGAsk: Teaching LLMs to Clarify via Counterfactual Information Gain
Sep 21, 2026Yunxiang Li, Xixin Wu, Helen MengClarification Question Generation LLM Agent Training
The Chinese University of Hong Kong, Hong Kong SAR, China
Knowing but Not Showing: LLMs Recognize Ambiguity but Rarely Ask Clarifying Questions
May 24, 2026Jinyan Su, Claire CardieLLM Evaluation Clarification Question Generation
1Cornell University
Clarify, Abstain or Answer? Strategising in Conversation with Belief-Augmented Generation
May 25, 2026Joris Baan, Wilker Aziz, Barbara Plank +1Clarification Question Generation Question Answering
University of Amsterdam · LMU Munich · European Commission, Joint Research Centre (JRC), Brussels

Sep 21, 2026J/K move · Enter open · S save

When and How Should an Agent Clarify? CIGAsk: Teaching LLMs to Clarify via Counterfactual Information Gain

Yunxiang Li, Xixin Wu, Helen Meng

The Chinese University of Hong Kong, Hong Kong SAR, China

Clarification Question Generation LLM Agent Training RL for Language Models

Open paper AskPDF arXiv

Instruction-tuned LLMs faced with underspecified queries often commit to a single interpretation rather than ask for clarification, producing confidently wrong answers. In our experiments, prompting alone is insufficient: models either ask for clarification on every query or ask vague questions that fail to recover the missing information. Addressing this failure requires learning two coupled skills: when to ask rather than answer and how to ask a question that recovers the disambiguating information. Existing recipes either address only one of these skills or require a separately trained critic. We propose CIGAsk, an RL recipe that teaches both skills through two complementary reward signals within a multi-turn GRPO loop. Counterfactual Information Gain (CIG) compares the gold-answer log-likelihood under a frozen reference model with and without the user response, providing per-turn credit that guides how to ask. The Asymmetric Ambiguity Bonus assigns a signed reward at the terminal token based on the gold ambiguity label, guiding when to ask. Across three clarification benchmarks spanning table, passage, and open-domain QA, CIGAsk-7B outperforms the strongest external baseline despite using a smaller backbone. It also transfers across datasets without per-dataset tuning while preserving single-turn QA performance on out-of-distribution benchmarks.

Abstract

Clarification Question Generation

Explore similar work

When and How Should an Agent Clarify? CIGAsk: Teaching LLMs to Clarify via Counterfactual Information Gain

Knowing but Not Showing: LLMs Recognize Ambiguity but Rarely Ask Clarifying Questions

Clarify, Abstain or Answer? Strategising in Conversation with Belief-Augmented Generation