When and How Should an Agent Clarify? CIGAsk: Teaching LLMs to Clarify via Counterfactual Information Gain

Yunxiang Li; Xixin Wu; Helen Meng

cs.AISep 21, 2026

When and How Should an Agent Clarify? CIGAsk: Teaching LLMs to Clarify via Counterfactual Information Gain

Authors: Yunxiang Li, Xixin Wu, Helen Meng

Organizations: The Chinese University of Hong Kong, Hong Kong SAR, China

Abstract

Instruction-tuned LLMs faced with underspecified queries often commit to a single interpretation rather than ask for clarification, producing confidently wrong answers. In our experiments, prompting alone is insufficient: models either ask for clarification on every query or ask vague questions that fail to recover the missing information. Addressing this failure requires learning two coupled skills: when to ask rather than answer and how to ask a question that recovers the disambiguating information. Existing recipes either address only one of these skills or require a separately trained critic. We propose CIGAsk, an RL recipe that teaches both skills through two complementary reward signals within a multi-turn GRPO loop. Counterfactual Information Gain (CIG) compares the gold-answer log-likelihood under a frozen reference model with and without the user response, providing per-turn credit that guides how to ask. The Asymmetric Ambiguity Bonus assigns a signed reward at the terminal token based on the gold ambiguity label, guiding when to ask. Across three clarification benchmarks spanning table, passage, and open-domain QA, CIGAsk-7B outperforms the strongest external baseline despite using a smaller backbone. It also transfers across datasets without per-dataset tuning while preserving single-turn QA performance on out-of-distribution benchmarks.

Figures & tables

Figure 1: Clarification resolves a missing slot. The underspecified query admits two plausible year-specific answers. A standard LLM answers immediately and combines them incorrectly, whereas CIGAsk learns when to clarify through the asymmetric reward and how to ask through CIG, recovering the intended year and the correct answer in one turn.

Figure 2: CIGAsk overview. (a) A trainable policy

\pi_{\theta}

interacts with a fixed user simulator with oracle access to the latent intent

q^{*}

, optionally requests clarification, and then produces a terminal answer. (b) CIG compares the gold-answer log-likelihood under the real history

h_{\leq t}

and the response-ablated history

h_{\leq t}\setminus u_{t}

using the same frozen reference policy

\pi_{\mathrm{ref}}

, providing local credit for how to clarify. The outcome reward and Asymmetric Ambiguity Bonus provide terminal credit for answer quality and when to clarify. After separate normalization, the intrinsic and outcome streams are combined into

\hat{A}^{\mathrm{CIG}}_{i,t}

for the GRPO update.

Dataset	Train	Val/Test	%Ambig
PACIFIC	15,087	1,952	16%
AbgCoQA	7,269	1,184	22%
AmbigNQ	19,244	4,377	79%

Table 1: Training datasets. Three clarification benchmarks spanning table-, passage-, and open-domain QA.

Prompting-based (no training)
Method	Backbone	PACIFIC fullval (1,952)				AbgCoQA	AmbigNQ
Method	Backbone	F1	F1 ${}_{\text{post}}$	Clr-amb	Clr-clr	F1	EM
Direct	Qwen2.5-7B	0.312	—	0.000	0.000	0.320	0.058
FATA	Qwen2.5-7B	0.177	0.177	1.000	1.000	0.228	0.076
ReAct ( Yao et al., 2023 )	Qwen2.5-7B	0.481	0.418	0.329	0.410	0.554	0.076
SFT only (no RL)

Table 2: Main results. Bold = our best model (CIGAsk 7B). PACIFIC fullval (

n{=}1{,}952

, GPT-4o sim); AbgCoQA / AmbigNQ

n{=}500

seed-42 (GPT-4o sim). — = inapplicable or not reported.

\ddagger

SGP / SGP-Oracle: Gemma-2-9b on 50/50 balanced dev; we report ambig/unambig F1 mean and place AmbigQA F1 in the AmbigNQ column.

Figure 3: CIG grows during training (CIGAsk 7B). Its rise coincides with the emergence of the recall–FPR gap.

Variant	CIG	Asym	F1	F1 post	Clr-amb	Clr-clr
CIGAsk 7B	✓	✓	.795	.768	.873	.260
$-$ asym	✓	—	.736	.733	.171	.036
$-$ CIG	—	✓	.744	.700	.763	.167
$-$ both	—	—	.683	.657	.087	.035

Table 3: Component ablation on PACIFIC fullval.

CIG reference	F1	F1 post	Clr-amb	Sel.	Reward
Frozen (ours)	.795	.768	.873	+.613	.62/.78
Current actor	.607	.359	.233	+.135	.47/.53

Table 4: Frozen-reference ablation on PACIFIC fullval. Reward reports mean training reward over steps 1–100 / 200–300.

Bench	CIGAsk F1	Base F1	Clarify-rate
TriviaQA	56.2	54.0	5.6%
NQ Open	25.9	25.1	9.4%

Table 6: OOD retention on TriviaQA

+

NQ Open closed-book (

n{=}500

, seed

42

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Group	Parameter	Value
Reward	$\alpha$ (CIG weight)	0.3
	CIG clip range (nats)	$[-0.5,2.0]$
	$\rho$ (asym bonus)	0.30
	malformed-clarify penalty	$-0.30$
GRPO	framework	verl ( Sheng et al., 2025 )
	rollouts per prompt	5

Appendix

Table 7: CIGAsk training hyperparameters. Identical across PACIFIC and AbgCoQA.

Dataset	Description	Used size & protocol	Source	License
Training and in-domain evaluation
PACIFIC	Proactive conversational QA over financial tables and text; each example carries a gold ambiguity label and clarification reference.	Train $15{,}087$ ; fullval $1{,}952$ ( $n_{\text{amb}}{=}316$ , $n_{\text{cl}}{=}1{,}636$ ). Multi-turn with GPT-4o simulator; up to $3$ clarify turns.	Deng et al. (2022)	MIT
AbgCoQA	Conversational QA derived from CoQA with annotated ambiguous turns and clarification references.	Train $7{,}269$ ; fullval $1{,}184$ ; eval subset $n{=}500$ seed $42$ with GPT-4o simulator.	Guo et al. (2021)	MIT
AmbigNQ	Ambiguous open-domain questions derived from Natural Questions, with multiple disambiguated targets per query.	Train $19{,}244$ ; test $4{,}377$ ; eval subset $n{=}500$ seed $42$ with GPT-4o simulator.	Min et al. (2020)	CC BY-SA 3.0
Out-of-distribution probes (never trained on)
TriviaQA (closed-book)	Trivia QA over short factoid questions; closed-book setting used as a single-turn retention check.	$n{=}500$ seed $42$ ; forced-direct and multi-action prompts (§ 5.5 ).	Joshi et al. (2017)	Apache 2.0

Appendix

Table 8: Dataset details. Source, used size, prompting regime, and license for every dataset in the paper.

Setting	F1	Clr-amb	Clr-clr	Sel.
$\rho{=}0.1$	.702	.681	.175	+.506
$\mathbf{\rho{=}0.3}$ (main)	.795	.873	.260	+.613
$\rho{=}0.5$	.703	.789	.188	+.601

Appendix

Table 9: Sensitivity to the ambiguity-bonus weight on PACIFIC fullval.

Simulator	F1	F1 post	Clr-amb	Sel.
GPT-4o (main)	.795	.768	.873	+.613
Qwen2.5-7B-Instruct	.783	.752	.856	+.636
Qwen2.5-3B-Instruct	.677	.536	.462	+.332

Appendix

Table 10: Sensitivity to the user simulator on PACIFIC fullval. GPT-4o is the main setting.

Clarification	Latent intent	Qwen2.5-7B	Qwen2.5-3B
Which period?	Q1 to Q2	From Q1 to Q2 (✓)	Fiscal Year 2019 ( $\times$ )
Which expense?	SG&A	SG&A expenses (✓)	Research and development ( $\times$ )
Which period?	2017 to 2018	From 2017 to 2018 (✓)	2018; 2019 ( $\times$ )

Appendix

Table 11: Representative simulator responses on ambiguous PACIFIC cases. Three examples from the eight-case manual audit are shown.

Gemma-2-9B backbone
Method	F1	F1 ${}_{\text{post}}$	Clr-amb	Clr-clr
SGP (reported)	0.726	—	0.429	0.206
SGP-Oracle $\ddagger$	0.787	—	0.435	0.156
CIGAsk (ours)	0.774	0.749	0.813	0.031
Zephyr-7B backbone
ACT (reported)	0.681	0.620	—	—

Appendix

Table 12: Matched-backbone comparisons on PACIFIC. CIGAsk uses the same recipe across backbones. Reported baseline results follow their original evaluation protocols. Matching the backbone controls for model architecture, but not evaluation protocol.

\ddagger

uses gold ambiguity labels. — denotes not reported.

Clarification Question Generation

Clarification question generation teaches AI systems to identify missing intent and ask targeted follow-ups before answering or acting.

See all 34 papers

Knowing but Not Showing: LLMs Recognize Ambiguity but Rarely Ask Clarifying Questions
May 24, 2026Jinyan Su, Claire CardieLLM Evaluation Clarification Question Generation
1Cornell University
Uncertainty-Aware Clarification in LLM Agents with Information Gain
Jun 2, 2026Mengyi Deng, Zhiwei Li, Xin Li +4Clarification Question Generation Tool-Using Agents
Information Hub, HKUST(GZ) · Jilin University · et al.
Clarify, Abstain or Answer? Strategising in Conversation with Belief-Augmented Generation
May 25, 2026Joris Baan, Wilker Aziz, Barbara Plank +1Clarification Question Generation Question Answering
University of Amsterdam · LMU Munich · European Commission, Joint Research Centre (JRC), Brussels

May 24, 2026J/K move · Enter open · S save

Knowing but Not Showing: LLMs Recognize Ambiguity but Rarely Ask Clarifying Questions

Jinyan Su, Claire Cardie

1Cornell University

LLM Evaluation Clarification Question Generation LLM Decision-Making

Open paper AskPDF arXiv

User queries are often underspecified and may admit multiple valid interpretations. Rather than silently making assumptions about the user's intent, a helpful assistant should surface such ambiguity by asking a clarifying question. Doing so requires two abilities: recognizing that a query is ambiguous, and acting on that recognition by seeking clarification instead of answering directly. To study these abilities, we evaluate models on ambiguous, unambiguous, and disambiguated questions in three settings: standard question answering, explicit ambiguity judgment, and behavioral analysis, where a judge model classifies responses as direct answers, refusals, or clarifying questions. We find a clear gap between recognition and behavior: models often identify ambiguity when explicitly asked to judge it, yet in the QA setting they overwhelmingly default to direct answers. Retrieved context further widens this gap by improving answerability while making models even less likely to ask clarifying questions.

Abstract

Clarification Question Generation

Explore similar work

Knowing but Not Showing: LLMs Recognize Ambiguity but Rarely Ask Clarifying Questions

Uncertainty-Aware Clarification in LLM Agents with Information Gain

Clarify, Abstain or Answer? Strategising in Conversation with Belief-Augmented Generation