Instruction-tuned LLMs faced with underspecified queries often commit to a single interpretation rather than ask for clarification, producing confidently wrong answers. In our experiments, prompting alone is insufficient: models either ask for clarification on every query or ask vague questions that fail to recover the missing information. Addressing this failure requires learning two coupled skills: when to ask rather than answer and how to ask a question that recovers the disambiguating information. Existing recipes either address only one of these skills or require a separately trained critic. We propose CIGAsk, an RL recipe that teaches both skills through two complementary reward signals within a multi-turn GRPO loop. Counterfactual Information Gain (CIG) compares the gold-answer log-likelihood under a frozen reference model with and without the user response, providing per-turn credit that guides how to ask. The Asymmetric Ambiguity Bonus assigns a signed reward at the terminal token based on the gold ambiguity label, guiding when to ask. Across three clarification benchmarks spanning table, passage, and open-domain QA, CIGAsk-7B outperforms the strongest external baseline despite using a smaller backbone. It also transfers across datasets without per-dataset tuning while preserving single-turn QA performance on out-of-distribution benchmarks.
Figures & tables
Figure 1: Clarification resolves a missing slot. The underspecified query admits two plausible year-specific answers. A standard LLM answers immediately and combines them incorrectly, whereas CIGAsk learns when to clarify through the asymmetric reward and how to ask through CIG, recovering the intended year and the correct answer in one turn.
Figure 2: CIGAsk overview. (a) A trainable policy πθ interacts with a fixed user simulator with oracle access to the latent intent q∗ , optionally requests clarification, and then produces a terminal answer. (b) CIG compares the gold-answer log-likelihood under the real history h≤t and the response-ablated history h≤t∖ut using the same frozen reference policy πref , providing local credit for how to clarify. The outcome reward and Asymmetric Ambiguity Bonus provide terminal credit for answer quality and when to clarify. After separate normalization, the intrinsic and outcome streams are combined into A^i,tCIG for the GRPO update.
Dataset
Train
Val/Test
%Ambig
PACIFIC
15,087
1,952
16%
AbgCoQA
7,269
1,184
22%
AmbigNQ
19,244
4,377
79%
Table 1: Training datasets. Three clarification benchmarks spanning table-, passage-, and open-domain QA.
Method
Backbone
PACIFIC fullval (1,952)
AbgCoQA
AmbigNQ
F1
F1 post
Clr-amb
Clr-clr
F1
EM
Prompting-based (no training)
Direct
Qwen2.5-7B
0.312
—
0.000
0.000
0.320
0.058
FATA
Qwen2.5-7B
0.177
0.177
1.000
1.000
0.228
0.076
ReAct ( Yao et al., 2023 )
Qwen2.5-7B
0.481
0.418
0.329
0.410
0.554
0.076
SFT only (no RL)
Table 2: Main results. Bold = our best model (CIGAsk 7B). PACIFIC fullval ( n=1,952 , GPT-4o sim); AbgCoQA / AmbigNQ n=500 seed-42 (GPT-4o sim). — = inapplicable or not reported. ‡ SGP / SGP-Oracle: Gemma-2-9b on 50/50 balanced dev; we report ambig/unambig F1 mean and place AmbigQA F1 in the AmbigNQ column.
Figure 3: CIG grows during training (CIGAsk 7B). Its rise coincides with the emergence of the recall–FPR gap.
Variant
CIG
Asym
F1
F1 post
Clr-amb
Clr-clr
CIGAsk 7B
✓
✓
.795
.768
.873
.260
− asym
✓
—
.736
.733
.171
.036
− CIG
—
✓
.744
.700
.763
.167
− both
—
—
.683
.657
.087
.035
Table 3: Component ablation on PACIFIC fullval.
CIG reference
F1
F1 post
Clr-amb
Sel.
Reward
Frozen (ours)
.795
.768
.873
+.613
.62/.78
Current actor
.607
.359
.233
+.135
.47/.53
Table 4: Frozen-reference ablation on PACIFIC fullval. Reward reports mean training reward over steps 1–100 / 200–300.
Bench
CIGAsk F1
Base F1
Clarify-rate
TriviaQA
56.2
54.0
5.6%
NQ Open
25.9
25.1
9.4%
Table 6: OOD retention on TriviaQA + NQ Open closed-book ( n=500 , seed 42 ).
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Group
Parameter
Value
Reward
α (CIG weight)
0.3
CIG clip range (nats)
[−0.5,2.0]
ρ (asym bonus)
0.30
malformed-clarify penalty
−0.30
GRPO
framework
verl ( Sheng et al., 2025 )
rollouts per prompt
5
Appendix
Table 7: CIGAsk training hyperparameters. Identical across PACIFIC and AbgCoQA.
Dataset
Description
Used size & protocol
Source
License
Training and in-domain evaluation
PACIFIC
Proactive conversational QA over financial tables and text; each example carries a gold ambiguity label and clarification reference.
Train 15,087 ; fullval 1,952 ( namb=316 , ncl=1,636 ). Multi-turn with GPT-4o simulator; up to 3 clarify turns.
Deng et al. (2022)
MIT
AbgCoQA
Conversational QA derived from CoQA with annotated ambiguous turns and clarification references.
Table 8: Dataset details. Source, used size, prompting regime, and license for every dataset in the paper.
Setting
F1
Clr-amb
Clr-clr
Sel.
ρ=0.1
.702
.681
.175
+.506
ρ=0.3 (main)
.795
.873
.260
+.613
ρ=0.5
.703
.789
.188
+.601
Appendix
Table 9: Sensitivity to the ambiguity-bonus weight on PACIFIC fullval.
Simulator
F1
F1 post
Clr-amb
Sel.
GPT-4o (main)
.795
.768
.873
+.613
Qwen2.5-7B-Instruct
.783
.752
.856
+.636
Qwen2.5-3B-Instruct
.677
.536
.462
+.332
Appendix
Table 10: Sensitivity to the user simulator on PACIFIC fullval. GPT-4o is the main setting.
Clarification
Latent intent
Qwen2.5-7B
Qwen2.5-3B
Which period?
Q1 to Q2
From Q1 to Q2 (✓)
Fiscal Year 2019 ( × )
Which expense?
SG&A
SG&A expenses (✓)
Research and development ( × )
Which period?
2017 to 2018
From 2017 to 2018 (✓)
2018; 2019 ( × )
Appendix
Table 11: Representative simulator responses on ambiguous PACIFIC cases. Three examples from the eight-case manual audit are shown.
Method
F1
F1 post
Clr-amb
Clr-clr
Gemma-2-9B backbone
SGP (reported)
0.726
—
0.429
0.206
SGP-Oracle ‡
0.787
—
0.435
0.156
CIGAsk (ours)
0.774
0.749
0.813
0.031
Zephyr-7B backbone
ACT (reported)
0.681
0.620
—
—
Appendix
Table 12: Matched-backbone comparisons on PACIFIC. CIGAsk uses the same recipe across backbones. Reported baseline results follow their original evaluation protocols. Matching the backbone controls for model architecture, but not evaluation protocol. ‡ uses gold ambiguity labels. — denotes not reported.
User queries are often underspecified and may admit multiple valid interpretations. Rather than silently making assumptions about the user's intent, a helpful assistant should surface such ambiguity by asking a clarifying question. Doing so requires two abilities: recognizing that a query is ambiguous, and acting on that recognition by seeking clarification instead of answering directly. To study these abilities, we evaluate models on ambiguous, unambiguous, and disambiguated questions in three settings: standard question answering, explicit ambiguity judgment, and behavioral analysis, where a judge model classifies responses as direct answers, refusals, or clarifying questions. We find a clear gap between recognition and behavior: models often identify ambiguity when explicitly asked to judge it, yet in the QA setting they overwhelmingly default to direct answers. Retrieved context further widens this gap by improving answerability while making models even less likely to ask clarifying questions.
Large Language Model (LLM) agents often operate under underspecified user instructions, where latent uncertainty over user intent leads to erroneous tool actions. To address this challenge, we propose a goal-oriented clarification framework that aligns clarification behavior with ambiguity resolution. Central to our approach is the Information Gain Reward, a metric that quantifies the utility of clarification questions by measuring the Bayesian belief update towards the ground-truth goal induced by the clarification exchange. We train the clarifier (LLM) using this reward to optimize for high information gain, ensuring that clarifications effectively reduce uncertainty and improve task completion within the agent-tool-user environment. We validate our framework within a clarification-enhanced τ-Bench environment, conducting cross-agent evaluations across five heterogeneous backbones. Empirical results demonstrate that our method consistently improves the success rate by 3.7% over the no-clarification baseline, while adding only 0.3 total interaction steps on average.
Mengyi Deng, Zhiwei Li, Xin Li +4
Information Hub, HKUST(GZ) · Jilin University · et al.
Large language models (LLMs) define a distribution over text, which can be viewed as a probabilistic representation of uncertainty: sampling K responses yields a belief state - responses a model deems plausible. Existing work exploits this representation for narrow tasks like either decoding or selective prediction, and often requires manual interventions, not controlling generation directly. We propose Belief-Augmented Generation (BAG): grounding LLMs in their own belief state via the prompt and letting them reason over these K samples to decide on and execute a conversational strategy: clarify, abstain, or answer. In a multi-turn ambiguous question answering (QA) setting, we find that LLMs by default rarely clarify or abstain, ignoring uncertainty about the input (aleatoric) or facts (epistemic). BAG improves QA accuracy across six models and yields strategy decisions more faithful to their belief state than prompt-only baselines. Disentangling when to clarify from when to abstain, however, remains challenging.
Joris Baan, Wilker Aziz, Barbara Plank +1
University of Amsterdam · LMU Munich · European Commission, Joint Research Centre (JRC), Brussels